A new adaptive bidirectional diffusion process that denoises and enhances passive millimeter wave images is designed and analyzed, which can diffuse adaptively according to the local characteristics of the image. In the flat areas of the image, it executes an isotropic diffusion to denoise. On the edges of the image, it executes a forward diffusion along the tangent direction to the isophote lines to denoise, and a backward diffusion along the normal directions to sharpen edges. Experimental results, completed with simulated image as well as actual passive millimeter-wave image at 91.5 GHz, demonstrated that it can substantially denoise and enhance passive millimeter wave images.