Abstract
In this paper, mid-wave infrared (MWIR) sensor optimization is provided as a function of the parameter ${F}\lambda /{d}$, where ${F}$ is the ${f}$-number, $\lambda $ is the effective wavelength, and $d $ is the detector pitch. For diffraction limited systems, acquisition range is related to the instantaneous field of view (detector limited operation) when ${F}\lambda /{d} \lt {1}$, and to the optical properties (optics limited operation) when ${F}\lambda /{d} \gt {2}.{0}$. Range performance is a combination of detector and optics resolution limits when ${F}\lambda /{d}$ is in between. When the system is not strictly diffraction or sampling limited, the optimal ${F}\lambda /{d}$ depends on other system component characteristics and conditions. Optical system aberrations affect system resolution and decrease range performance. As background shot noise, dark current shot noise, and read noise increase, range decreases. In the infrared spectral region, atmospheric absorption leads to reemission of thermal energy. The detected reemission creates additional shot noise. Atmospheric attenuation greatly affects MWIR sensor range performance. Next-generation MWIR sensors will have smaller detectors, larger arrays, and better sensitivity to enable ${F}\lambda /{d}$-based optimization. Previous studies ($\Delta T = {4}\;{\rm K}$ for tracked vehicles) suggest that an initial design point is ${F}\lambda /{d} \approx {2}.{0}$. When detecting low contrast targets ($\Delta T \sim {0}.{1}\;{\rm K}$), sensor gain is used to increase the signal for a desired displayed contrast. This gain increases displayed noise and reduces acquisition range. This is typically not an issue for long-wave infrared sensors due to the excess number of photons in the 8–12 µm band but poses a problem for MWIR sensors, which are photon starved. Under such scenarios, the optimum ${F}\lambda /{d}$ appears to be about 1.5 for MWIR sensors. The results here provide reasonable strategies for MWIR system optimization and a direction associated with future MWIR focal plane development.
© 2020 Optical Society of America under the terms of the Optical Society of America Open Access Publishing Agreement
1. INTRODUCTION
Numerous infrared system performance analyses have focused on long-wave infrared (LWIR) detector pitch optimization [1,2]. Although the work is similar, those analyses do not address the mid-wave infrared (MWIR) band. The MWIR band is more difficult to understand because it tends to be photon starved for many terrestrial background applications (the LWIR band has plenty of photons). Detailed analysis allows a knowledgeable selection of detector sizes and system implementations.
LWIR has long been a historical band that works well for many applications, and in particular, ground combat applications [1,2]. The U.S. Army still uses LWIR in primary targeting sights due to the abundance of photons under different conditions including cold weather, high turbulence, degraded atmosphere, smoke, and dirty battlefield conditions [3–5]. For an aperture-limited system, a MWIR sensor provides a greater acquisition range than a comparable LWIR sensor under good target and atmospheric conditions (i.e., high altitude). As a result, MWIR has become common for U.S. Air Force and U.S. Navy applications since their fighting environments are not as severe as ground combat. Considering the benefits of MWIR, the U.S. Army is pursuing dual-band (MWIR + LWIR) systems, where the primary benefit of the MWIR channel is that of long-range target identification under good atmospheric conditions.
Small pitch (5–10 µm) LWIR sensors with high pixel count arrays allow imaging of two fields of view without switching optics [6–10]. The benefit of such systems is the speed in detecting and identifying a target. This approach provides all of the LWIR benefits associated with classical switched optical systems but still does not compete with MWIR targeting range under good conditions. The MWIR large format small pitch approach has largely been ignored by the armed services for tactical applications, but has been studied in the applications of missile launch detection, infrared search and track, infrared countermeasures, and unmanned aerial vehicle detection.
Large format MWIR detectors have been under development for quite a few years for the applications listed above. The “standard” MWIR detector in production and/or under development has 12 µm pitch (Fig. 1). Many of the government laboratories are pushing to an 8 µm pitch, and it is likely that this will be the new standard. Changing the pitch will have significant implications on system design and optimization. Next-generation MWIR sensors will have even smaller detectors and better sensitivity (easily detect $\Delta T = {0}.{1}\;{\rm K}$ targets).
There have been significant technical advances in MWIR detectors in the past 7–10 years. The advances have been primarily motivated by the US-funded Tri-Service Vital Infrared Sensor Technology Acceleration (VISTA) programs. The first program emphasized the development of antimony (Sb)-based focal plane technology, and the second program focused on high-operating temperature (HOT) detectors and multiband detectors [11]. The second program also established an industrial base for Sb-based MWIR HOT focal plane manufacturing with large format and small pitch. The process improvement effort of the program provided for lower-cost focal planes with higher yield and operability. The program resulted in two material substrate foundries, two epitaxial material growers, and five infrared focal plane producers. The results of the VISTA program as well as the manufacturing technology follow-on programs have provided the MWIR focal plane development emphasis that will allow extremely high performance MWIR sensors in the near future. The applications that are impacted by VISTA are targeting, infrared search and track, threat warning, and intelligence–surveillance–reconnaissance (ISR).
The impact of the VISTA work is highly likely to replace InSb MWIR detectors with quantum structures such as type II superlattice (T2SL) and nBn detectors. Both of these newer detector technologies provide InSb-like performance, but with much higher operating temperatures. HgCdTe is also a candidate for the displacement of InSb, as it also has a higher operating temperature (130–150 K). There is a rule of thumb for detector coolers that states that an increase in operating temperature of 10 K provides for a cooler power reduction of one half. This rule emphasizes the importance of high operating temperature MWIR detectors since the cooler power impacts size, weight, power, and cost.
In this paper, night vision integrated performance model (NVIPM) predictions provide a path for system optimization. While NVIPM is valid for sensors sensitive anywhere in the spectral region from 0.4–14 µm, this paper focuses on next-generation MWIR sensor design. LWIR sensors have been addressed in previous work [6–10]. Important system parameters and conditions modeled in NVIPM are used as inputs in this study. Each input (background temperature, optical aberrations, noise, atmospheric transmission, notch filter, viewing distance, and well capacity) is individually analyzed to determine those that are most deleterious to system range performance. Previous studies [11] with large target $\Delta T$ (NVIPM recommends $\Delta T = {4}\;{\rm K}$ for tracked vehicles) suggested that a starting design point is ${F}\lambda /{d} \approx {2}.{0}$, where ${F}$ is the ${f}$-number, $\lambda $ is the effective wavelength averaged over the spectral photon irradiance and sensor spectral responsivity, and $d$ is the detector pitch.
With low $\Delta T$s, noise affects acquisition range, and the optimum ${F}\lambda /{d}$ for MWIR systems appears to be about 1.5. The acquisition ranges presented here are unique to the values selected (e.g., spectral quantum efficiency, atmospheric spectral transmission) but the shapes of the curves and conclusions are representative of all MWIR sensors. Any size detector with the appropriate f-number optics can achieve a desired ${F}\lambda /{d}$ [12,13] and corresponding range performance. The limitations are how small detectors can be manufactured, physical space constraints that limit the focal length and aperture diameter, and minimum practical ${f}$-number (sometimes taken as 1.2). Fiete [14] labeled the detector pitch as $\rho $ and provided imagery as a function of ${F}\lambda /\rho $ (sometimes called ${Q}$). This paper assumes 100% fill factor so that detector size and pitch are the same; we will refer to this as $d$ (${ \rho } = d$) in the analysis and discussion in this paper. We note that the results for a somewhat less than 100% closely match the results reported here.
2. NVIPM THEORY
NVIPM was developed for the detection, recognition, and identification of ground targets. It consists of an eye (observer) term and a noise term. The system contrast threshold function is
For mathematical convenience, the MTFs are considered separable. The targeting task performance (TTP) is
Figures 2 and 3 illustrate the effects of MTF, noise, and low contrast targets on the TTP calculation.
Considering a simplified system, ${{\rm MTF}_{\rm SYS}} = {{\rm MTF}_{\rm DIFFRACTION}}{{\rm MTF}_{\rm DETECTOR}}{{\rm MTF}_{\rm DISPLAY}}$. Using the monochromatic diffraction limited optical MTF as an approximation to the polychromatic MTF,
For preliminary analysis, we set all noise except photoelectron shot noise to zero. This requires a transparent atmosphere ($\tau = 1/{\rm km}$ with Beer’s law) to avoid atmospheric thermal emission shot noise. With low noise, the eye term in Eq. (1) dominates, and Eq. (4) becomes
For this analysis we use a typical InSb MWIR sensor sensitive from 3.34–4.85 µm and includes a ${{\rm CO}_2}$ notch filter at 4.2 µm. To explore the eye and displayed noise term properties only in the presence of shot noise, the sensor has zero dark current, zero downstream noise, and a well capacity of ${{10}^{10}}$ electrons (considered an infinite well). The aperture diameter is 50 mm. The display element is 0.2 mm, and the viewing distance is 30 cm. The detection criterion is ${{V}_{50}} = {2}$. From Eq. (7), range is linearly proportional to target size so that selecting 1.0 m is representative of range performance. As $\Delta {T}$ decreases, the sensor gain increases to maintain a constant ${{C}_{\rm DISPLAY}}$. But the value of NF increases with gain, and the noise term in Eq. (1) can dominate for low $\Delta {T}$s. With only background limited noise and $\Delta {T} = {1}\;{\rm K}$, maximum range occurs at high ${F}\lambda /{d}$ (Fig. 4). This curve is based upon MTFs only and its shape is generic to all systems. The optimum ${F}\lambda /{d}$ decreases as displayed noise (due to gain) increases. Figure 4 has three distinct regions: detector limited, transition, and optics limited. Selecting ${F}\lambda /{d} = {0}.{5}$ and 1.5 as boundaries is arbitrary.
It was previously shown [12] that the integral in Eq. (7) can be expressed as a sixth-order polynomial valid up to ${F}\lambda /{d} = {4}$:
The above equations and figures are generic to all sensors.
In the following sections, we introduce system component and condition parameters that affect system performance deviating from the detector limited or optics limited regions.
3. BACKGROUND TEMPERATURE
The differential signal between the resolved target at temperature ${{T}_T}$ and the background at temperature ${{T}_B}$ is
4. OPTICAL ABERRATION
Aberration is a generic term applied to any lens “defect” that degrades the image. It is usually characterized by an MTF. In the detector limited region, optical aberrations have minimal effect on acquisition range. In the optics limited region, aberrations reduce the optics MTF and thereby reduce range. For convenience, ${{\rm MTF}_{\rm OPTICS}}$ is approximated by
${{\rm MTF}_{\rm ABERRATION}}$ is a fictitious MTF such that the combination approximates the measured sensor optical MTF. For a rms wave front error of ${{\rm W}_{\rm RMS}}$, the fictitious aberration MTF is5. DARK CURRENT DENSITY
Dark current is the flow of thermally generated electrons. Usually the manufacturer provides the dark current density, ${{J}_{D}}$. For 100% fill factor arrays, the number of dark current electrons is
where ${q}$ has the usual meaning of electronic charge, and the dark current flows during an entire frame integration time, ${t_{\rm INT}}$. The number of generated background photoelectrons is6. DOWNSTREAM NOISE
Downstream noise is the root-sum-of-the-squares combination of read noise, analog-to-digital noise, and clock noise. Since downstream noise is entered as rms electrons, it should be less than 10% of the background shot noise:
7. ATMOSPHERIC TRANSMISSION
In the IR region, there is little scattering, and the atmosphere mostly absorbs radiation. By Kirchhoff’s law, all absorbed radiation is reemitted. When detected, the reemission contributes to photoelectron shot noise. Over long distances, reemission dominates, producing significant shot noise, which is incorporated into the noise component of Eq. (1). Figure 12 illustrates detection range for $\Delta {T} = {0}.{1}\;{\rm K}$ and 1 m target. Targets with $\Delta {T} = {1}\;{\rm K}$ provide similar results. While MODTRAN can be used for specific spectral cases, we use Beer’s law to generalize. For targets that have spectral features embedded in spectrally varying atmosphere, the results may differ, but Beer’s law provides good generalization for most ground targets of interest.
8. NOTCH FILTER
In Section 6, the atmospheric transmission was modeled by Beer’s law. While Beer’s law provides generalized results, MODTRAN provides a unique spectral transmission for each scenario. The results can be quite different for each of MODTRAN’s standard environment models (e.g., mid-latitude summer, mid-latitude winter, 1962 U.S. standard, etc.) and aerosol models (e.g., rural, maritime, etc.). The transmission at the ${{\rm CO}_2}$ absorption band (4.2 µm) is essentially zero, and only reemission exists. Therefore, it is worthwhile to place a blocking filter (aka a notch filter or twin-peak filter) at this wavelength. The advantage of the blocking filter becomes obvious when detecting target features that have a low $\Delta {T}$ (Fig. 13).
9. VIEWING DISTANCE
Reducing the visual angle defined by the display element size and observer viewing distance minimizes perceived noise. There is always an optimum [13] viewing distance (Fig. 14). Most observers will move closer to the display to discern detail but rarely move away from the display to reduce perceived noise. As the viewing distance increases, ${{\rm CTF}_{\rm NAKED\,EYE}}$ increases relative to ${{\rm MTF}_{\rm SYS}}$ and thereby reduces the perceived noise. Maximum range becomes a function of the noise level and visual angle. The noise level depends upon the sensor and atmospheric transmission, which is a function of range.
10. NEDT AND SNR
The noise equivalent differential temperature (NEDT) is a historical noise metric, and early target acquisition models were based upon the minimum discernible signal-to-noise ratio (SNR). Traditionally, SNR is expressed by
where the target $\Delta {T}$ is modified by the atmospheric transmittance. Only background shot noise (background limited performance or BLIP) was considered in the original NEDT expression. NVIPM includes all thermal noise sources as part of its predicted NEDT (labeled as ${{\rm NEDT}_{\rm NVIPM}}$, not discussed here). These include dark current shot noise, atmospheric thermal emission shot noise, optical thermal emission shot noise, and downstream noise. The incorporation of noise into Eq. (1) is complex, and its effects are illustrated in the numerous figures in this paper. As indicated in the last section, perceived noise depends upon the viewing distance.11. FINITE WELL CAPACITY
With the infinite well discussed in the previous sections, the number of photoelectrons was limited by the frame time of 33.3 ms (30 Hz operation). Generally, the desired well capacity is twice that of the background photoelectrons. This allows detection of targets that are either warmer or cooler than the ${{T}_B}$. Equation (16) becomes
For $\Delta {T} \gt {1}$, detection ranges are not significantly affected by the well capacity. On the other hand, image processing algorithms perform better when more photoelectrons are available. While Eq. (22) appears complex, an estimate of desired well capacity is
12. DISCUSSION
${F}\lambda /{d}$ links focal length, detector size, aperture diameter, and wavelength to acquisition range. As previously stated, the next-generation thermal imagers will have smaller detectors, larger arrays, and better sensitivity. Previous studies suggested an initial design point for LWIR sensors is ${F}\lambda /{d} \approx {2}.{0}$. For MWIR sensors that are photon starved, when detecting low contrast targets ($\Delta {T} \sim {0}.{1}\;{\rm K}$), sensor gain is used to achieve a desired displayed contrast. This also increases the perceived noise, and that reduces range performance. When considering noise aberrations and atmospheric transmission, the optimum ${F}\lambda /{d}$ for the photon-starved MWIR band appears to be on average, about 1.5. Any combination of f-number and detector size that provides 1.5 is acceptable (Fig. 18).
The background temperatures (240–350 K) in this paper are consistent with ground-to-ground scenarios. The acquisition ranges presented here are unique to the values selected (e.g., spectral quantum efficiency, atmospheric spectral transmission) but the shapes of the curves and conclusions are general.
All figures are functions of ${F}\lambda /{d}$ with the aperture diameter fixed at 50 mm and the focal length varied. Changing the aperture size, D, to, say, 150 mm changes ${F}\lambda /{d}$, and the range increases linearly [Eq. (10)]. Normalizing the range (dividing by D) provides identical curves. Simply put, the curves presented here are generic to all MWIR sensors. Likewise, according to Eq. (4), as the target size increases, the range increases linearly, suggesting that selection of a 1 m target is representative. The idealized sensor presented here considers monochromatic diffraction limited optics, 100% fill factor array (detector dimension = pitch), no crosstalk [15], and no other MTF degrading issues.
The HVS affords tremendous temporal and spatial integration. This integration is included in NVIPM. As a contrast model, neither SNR nor NEDT is used to predict acquisition range. Because of the HVS spatial frequency dependence, perceived noise can be reduced by increasing the viewing distance. However, it is rare that an observer will change his/her viewing distance.
When ${F}\lambda /{d} \gt {1}.{5}$, the sensor is operating in the optics limited region. Here, range is sensitive to any optical degrading MTF. Even small changes such as $\lambda /{4}$ optical wave front error affect acquisition range. This puts additional burden on the optical designer. Spatial noise (also called three-dimensional or 3D noise) was not considered here and was assumed to be small compared to temporal noise.
Considering all the figures and analyses in this paper, it appears that ${F}\lambda /{d}\;\sim\;{1}.{5}$ is optimum for a MWIR sensor when viewed by a human observer. This is a general observation that applies to MWIR systems in a wide variety of conditions. To minimize size, weight, and cost (SWAP), a detector pitch of 3.6 µm is appropriate for an ${F}/{1}.{2}$ system ($\lambda = {4}.{46}\;\unicode{x00B5} {\rm m}$), and a detector pitch of 4.2 µm is appropriate for an ${F}/{1}.{4}$ system. Larger pitches (future 8 µm or current 12 µm) require larger chip size resulting in more weight and power consumption.
With a small detector pitch, it is possible to create arrays with ${5}\;{\rm K} \times {5}\;{\rm K}$ detectors. This poses a challenge due to the extremely high data rates required. One solution is a smart readout integrated circuit (ROIC) design where the full frame is not read out but regions of interest can be read out at conventional rates. Detector outputs can be combined for search and then addressed individually for target identification. There are clever ROIC approaches that essentially trade the ROIC functionality for optical switching.
Optical switching is when lenses are moved into the optical system to create a different magnification. An example of this is when an afocal system is moved into the optical train to convert a wide-field-of-view sensor to a narrow field of view. An afocal of ${5\times }$ can narrow the field of view by a factor of five. Optical switching is common when a smaller format detector array (e.g., ${640} \times {480}$) is used in a wide field of view, and then an optical switch causes the detector array to image a narrower field of view. The wide field of view is used for searching, and the narrow field of view is used for target identification and object interrogation. Optical switching is time consuming, where the switching can take a large fraction of a second to execute, so searching and target identification are a long series of wide-field searching and narrow-field interrogation. Once a potential target is found in the wide field, the imager has to be centered on the target (which also takes time) before the optical switch is invoked to provide the narrow-field interrogation. Small formats require optical switches that are expensive and time consuming. Large format, small pitch focal planes have the potential to replace optical switches with electronic magnification.
There are a number of ROIC programs currently underway that are looking at advanced ROIC functionality that can achieve the necessary technology improvements to realize these suggested advanced MWIR systems. With the large formats, optical switching can be accomplished by using the entire array for the wide field of view and decimation/interpolation in the ROIC. For a narrow field of view, a subsection of the ROIC can be displayed anywhere on the focal plane so that optical centering is not required, and interpolation and electronic zoom can provide the desired magnification. There are many ROIC functions such as time-delay-integration, electronics stabilization, and super-resolution that can be integrated with the ROIC or off-board interactions in concert with the ROIC. These ROIC enhancements continue to be the subject of many development programs.
As a final comment, NVIPM was released in May 2013. A major improvement over previous models is its loop capability (aka batch mode operation). Depending upon on the number of variables and loops, a single push of the run button exercises NVIPM up to several thousand times. While not counted, it is estimated that NVIPM was run well over 100,000 times to create the figures in this paper.
13. CONCLUSION
The ${F}\lambda /{d}$ parameter is generic to all imaging systems. In a photon-rich environment (typical of the LWIR band), it uniquely predicts acquisition range for low noise systems; ${F}\lambda /{d} \approx {2}$ provides optimum range performance. For the MWIR band (sometimes referred to as photon starved) ${F}\lambda /{d} \approx {1}.{5}$ is appropriate over a wide variety of conditions. This conclusion supports focal plane development efforts for smaller detectors. With a minimum practical f-number of 1.2 to 1.4, MWIR detector pitch ranges from 3.6–4.2 µm, respectively. However, as suggested in this paper, current and near future detector sizes are still larger than they need to be for a high-performance, compact, low-cost system. Cooled detector selection is increasing as detector operating temperature continues to rise (aka HOT) [16–18]. Rapid advances in size reduction and in HOT technology will provide for extremely high-performance systems in the near future.
Disclosures
The authors declare no conflicts of interest.
REFERENCES
1. J. N. Vizgaitis, “Dual f/number optics for third generation FLIR systems,” Proc. SPIE 5783, 875–886 (2005). [CrossRef]
2. J. Vizgaitis, “Third generation infrared optics,” Proc. SPIE 6940, 69400S (2008). [CrossRef]
3. D. A. Reago, S. B. Horn, J. Campbell Jr., and R. H. Vollmerhausen, “Third-generation imaging sensor system concepts,” Proc. SPIE 3701, 108–117 (1999). [CrossRef]
4. P. Richardson and R. G. Driggers, “Atmospheric turbulence effects on 3rd generation FLIR performance,” Proc. SPIE 6207, 620706 (2006). [CrossRef]
5. A. Hodgkin, B. Kowalewski, D. Tomkinson, B. P. Teaney, T. Corbin, and R. Driggers, “Modeling of IR sensor performance in cold weather,” Proc. SPIE 6207, 620708 (2006). [CrossRef]
6. R. Vollmerhausen and R. Driggers, “Extending the range performance of diffraction limited imagers,” Proc. SPIE 10178, 101780F (2017). [CrossRef]
7. G. C. Holst and R. G. Driggers, “Small detectors in infrared system design,” Opt. Eng. 51, 096401 (2012). [CrossRef]
8. R. E. Short, D. Littlejohn, M. J. Scholten, C. J. Rivera-Ortiz, R. H. Vollmerhausen, and R. G. Driggers, “Holistic approach to high-performance long-wave infrared system design,” Opt. Eng. 58, 023113 (2019). [CrossRef]
9. R. Short, D. Littlejohn, and R. Driggers, “An update on PWP enhancement for LWIR target acquisition sensors,” Proc. SPIE 11001, 110010M (2019). [CrossRef]
10. R. G. Driggers, R. H. Vollmerhausen, J. P. Reynolds, J. D. Fanning, and G. C. Holst, “Infrared detector size: how low should you go?” Opt. Eng. 51, 063202 (2012). [CrossRef]
11. “High Operating Temperature and Multi-Band Focal Plane Arrays Manufacturing Technology,” https://www.armymantech.com/SucessStories/Hot_MBFPA.pdf.
12. G. C. Holst, “Imaging system performance based upon Fλ/d,” Opt. Eng. 46, 103204 (2007). [CrossRef]
13. G. C. Holst, “Optimum viewing distance for target acquisition,” Proc. SPIE 9452, 94520K (2015). [CrossRef]
14. R. D. Fiete, “Image quality and λFN/ρ for remote sensing systems,” Opt. Eng. 38, 1229–1240 (1999). [CrossRef]
15. T. M. Goss and C. J. Willers, “Small pixel cross-talk MTF and its impact on MWIR sensor performance,” Proc. SPIE 10178, 101780L (2017). [CrossRef]
16. A. Williams and M. Tidrow, “III-V infrared focal plane array development in US (Conference Presentation),” Proc. SPIE 10624, 106240P (2018). [CrossRef]
17. M. Z. Tidrow and D. A. Reago Jr., “VISTA video and overview (Conference Presentation),” Proc. SPIE 10177, 101770M (2017). [CrossRef]
18. E. H. Steenbergen, C. P. Morath, D. Maestas, G. D. Jenkins, and J. V. Logan, “Comparing II-VI and III-V infrared detectors for space applications,” Proc. SPIE 11002, 110021B (2019). [CrossRef]