Abstract Infrared astronomy is currently benefiting from three main technologies providing high-performance hybrid active pixel sensors. In the near infrared from 1 to 5 μm two technologies, both aiming for buttable 2K×2K mosaics, are competing, namely InSb and HgCdTe grown by LPE or MBE on Al 2 O 3 , Si or CdZnTe substrates. Blocked impurity band Si:As arrays cover the mid infrared spectral range from 8 to 28 μm. Adaptive optics combined with multiple integral field units feeding high-resolution spectrographs drive the requirements for the array format of infrared sensors used at ground-based infrared observatories. The pixel performance is now approaching fundamental limits. In view of this development, a detection limit for the photon flux of the ideal detector will be derived, depending only on the temperature and the impedance of the detector. It will be shown that this limit is approximated by state of the art infrared arrays for long on-chip integrations. Different detector materials are compared and strategies to populate large focal planes are discussed. The need for the development of small-format low noise sensors for adaptive optics and interferometry will be pointed out.