A new readout circuit structure for infrared focal plane array with pixel-level shared source follower and double column buses is proposed in this paper. The voltage signal of the pixel is transferred through double column buses instead of single column bus, and therefore the non-uniformity and non-linearity caused by the parasitic resistor of the column bus is eliminated. By sharing the source follower within four adjacent pixels in the same column, the aspect ratio (W/L) and area of the source follower are increased to suppress the thermal noise, flicker noise and non-uniformity caused by process. A 640×512 readout circuit using this structure is designed and fabricated in 0.35μm 2P3M CMOS process and the pixel pitch is 15μm. The test results indicate the readout circuit receives a high dynamic range (DR) of 81dB high-quality infrared images with a low power consumption of 30mW. The nonlinearity is 0.11%, and the non-uniformity is less than 1%. Medium-wave infrared detector assembly is fabricated and tested. The assembly’s non-uniformity is less than 5% and the NETD is 18mK. High-quality infrared images are obtained.