Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Column coded scanning aperture hyperspectral imaging system

Open Access Open Access

Abstract

The line scanning hyperspectral imaging system (LS-HIS), which relies on a mechanical slit or spatial light modulation device for single channel spatial scanning, is widely used in various fields such as biomedical imaging and remote sensing. However, in scenes that require low light illumination, a decrease in luminous flux will increase exposure time, leading to a significant decrease in scanning efficiency and signal-to-noise ratio (SNR). To address this issue, we present a flexible column coded scanning aperture hyperspectral imaging system (CCSA-HIS) using a spatial light modulator digital micromirror device (DMD). By introducing the concept of multiplex and constructing a multiplexing encoding matrix, we form a one-dimensional multi-column coded scanning aperture, which greatly improves scanning efficiency. Experimental comparisons demonstrate that this approach achieves higher SNR and equivalent spatial and spectral resolution in significantly less sampling time compared to LS-HIS. In short, our scheme provides a new imaging technology for the field of hyperspectral imaging with good theoretical value and engineering significance.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Hyperspectral imaging (HSI) technology is a powerful tool that seamlessly integrates spectrometers with cameras to capture not only two-dimensional spatial information but also one-dimensional spectral data. This fusion creates a comprehensive three-dimensional data cube of the target under examination. This innovative approach has found diverse applications across various fields, including remote sensing imaging [1,2], microscopic imaging [3,4], biomedical imaging [5,6], and so on. At present, there are three main types of typical hyperspectral imaging methods: line scanning hyperspectral imaging systems (LS-HIS) using slit and dispersion elements (prisms or gratings) [7], staring hyperspectral imaging systems (S-HIS) using tunable filters such as LCTF or AOTF [8], and snapshot hyperspectral imaging systems (SS-HIS) achieved through once exposure and subsequent computational processing [9].

In recent years, digital micromirror device (DMD) has revolutionized the development of traditional spectral imaging technology. Typically, LS-HIS employs a structure in which either the device or the object moves relative to each other, requiring an unwieldy moving slit, making the system’s structure more complex and accompanied by mechanical vibration. Over the past few years, the application of DMD in LS-HIS has gained momentum [10,11]. In our previous research, a novel hyperspectral imaging system that combines the DMD with traditional line scanning techniques was proposed, resulting in tunable spatial and spectral resolutions [1214]. Compared to conventional slit-scanning approaches, DMD-based LS-HIS significantly reduces energy consumption due to its compact size, low driving voltage, and absence of mechanical moving parts. At the same time, significant progress has been made in the application of DMD as a programmable mask in the field of coded aperture snapshot spectral imager (CASSI) [1517]. Numerous researchers have leveraging DMD for compressed sensing spectral imaging [1820]. The flexible modulation mode endows DMD with unique advantages in the field of spectral imaging, allowing for flexible setting of the imaging field of view, thereby replacing the mechanical slits and masks in traditional line scanning and snapshot spectral imaging systems.

Due to the huge amount of data in hyperspectral imaging, there has always been a trade-off between acquisition time and spatial/spectral resolution. For example, typical LS-HIS can achieve high spatial/spectral resolution, but the acquisition time is long. Especially under poor lighting conditions, the exposure time of each slit needs to be further extended, and the overall collection time will be significantly prolonged. By comparison, the SS-HIS can quickly collect target data, but its spatial/spectral resolution is very low. Researchers have always grappled with the challenge of achieving high data collection efficiency while maintaining high spatial/spectral resolution, even with the introduction of DMD to HIS.

To surmount the trade-off, numerous methods have been proposed. In 2015, Orth et al. introduced a Gigapixel multispectral microscope that enhances the optical throughput by simultaneously processing multiple multispectral data points [21]. However, this approach lacks flexibility in adjustability. In 2021, Dong et al. proposed a multiline parallel scanning method based on the DMD [22]. Although parallel scanning improves the collection efficiency to some extent, the number of scanned columns is severely limited due to the requirement of non-overlapping spectra between different scanning columns. Consequently, the article only implemented double-line parallelized imaging, resulting in limited improvement. Moreover, the application of Fourier transform [23], Hadamard transform [24], and compressed sensing [25] technologies in the spectral imaging field has provided additional insights for addressing this problem. However, Fourier transform spectral imaging technology is usually based on the principle of dual beam interference, which inverts the target spectrum from the interference pattern obtained by the detector through Fourier transform. Hadamard transforms are commonly used to encode spectral dimensions, often necessitating two splitting and combining components [26]. The optical systems of these two methods are complex. Compressed sensing spectral imaging, which is usually used in SS-HIS, employs a single dispersive component but requires precise proportional matching between DMD pixels and detector pixels, imposing higher design requirements on the optical system. Additionally, due to the ill-posed problems and constraints of iterative algorithms, the processing time is significantly prolonged, while spatial and spectral resolutions remain low [25], often falling short of practical needs.

The primary goal of this paper is to achieve high spatial/spectral resolution while minimizing acquisition time. To achieve this goal, we propose a novel column coded scanning aperture hyperspectral imaging system (CCSA-HIS) that replaces the mechanical slit with a spatial light modulator DMD. We introduce the concept of spatial multiplexing to LS-HIS and design a multiplexing encoding matrix. This matrix treats each column of LS-HIS as an individual channel and performs one-dimensional multi-column coded combination measurements on the target channels. Compared with LS-HIS, CCSA-HIS significantly improves the SNR of images and acquisition efficiency.

2. Principle and methods

2.1 CCSA-HIS technical principle

The structure of CCSA-HIS is depicted in Fig. 1. It consists of a primary imaging optical path and a spectral dispersion optical path. The former primarily comprises an objective lens that can be customized based on the specific observation target. Designers need to consider multiple parameters when designing the objective lens, including field of view (FOV), DMD format, primary magnification (PMAG), resolution, and depth of field (DOF). Currently, there are various objective lens structures available, such as double Gauss lens, reverse telephoto, Petzval lens, and Coke triplet, each with its own advantages and disadvantages. The selection of an appropriate structure should be based on the specific requirements of the application. The DMD serves as both the image surface in the primary imaging optical path and the object surface in the spectral dispersion optical path. Acting as a reflector and a slit in the optical path, the DMD differs from a conventional mirror as it consists of millions of micromirrors. Each micromirror can rotate ± 12° around its diagonal axis to modulate the light field. In our setup, the DMD is rotated 45° along the Z-axis, which is perpendicular to the paper surface and facing outward. This arrangement aligns all optical paths on the same horizontal plane. Besides, we enable the development of a novel one-dimensional encoding mode suitable for the rotated DMD. This adjustment enhances the SNR and acquisition efficiency. In the spectral dispersion optical path, a set of collimating lens, converging lens, dispersion elements, and a final photodetector are required. The dispersion element can either be a prism or a grating.

 figure: Fig. 1.

Fig. 1. (a) A 3D model of CCSA-HIS. (b) DMD rotated 45° along the Z-axis with the multiplex column coded scanning mode. (c) Single channel scanning mode.

Download Full Size | PDF

The CCSA-HIS system offers a wave band range of 400-750 nm, providing a comprehensive coverage of the visible spectrum. With a spatial resolution of approximately 300 µm and a spectral resolution of about 1 nm, the system enables detailed and precise imaging and spectral analysis. To achieve these capabilities, we employ an adjustable focus telecentric objective lens. This lens offers a wide FOV of around 30°, allowing clear imaging of objects within a distance range of 30-100 cm. The objective lens plays a crucial role in transforming the primary image of the target onto the DMD (DLP7000, 1024 × 768 resolution, pixel size of 13.68 µm, and ± 12° rotation capability). The DMD utilizes multiplexing encoding to open specific multi-channels, enabling the passage of selected light for further processing. The light selected by the DMD then passes through a collimating lens before reaching a transmission grating with a density of 300 lines per millimeter. This grating disperses the light into its constituent wavelengths, allowing for spectral analysis. Subsequently, the dispersed light is imaged by a converging lens onto a sCMOS imaging sensor (Pco. edge 4.2bi, 2048 × 2048 resolution, pixel size of 6.5 µm). The sCMOS sensor plays a critical role in capturing the dispersed light for subsequent analysis and interpretation.

2.2 Multiplexing encoding matrix

Assuming the resolution of DMD is $m \times n$, the total number of diagonal columns for a 45° rotation is $m + n - 1$. However, it is important to note that the image of the target does not completely cover all diagonal columns. Let us denote the number of columns used as N. In this case, each column corresponds to an independent channel, giving us N unknown variables ${x_1},\textrm{ }{x_2},\textrm{ }\ldots ,\textrm{ }{x_i},\textrm{ }\ldots ,\textrm{ }{x_N}$ that need to be solved for. Here, ${x_i}$ represents the spectral image corresponding to column i. By performing N combined measurements on N variables, we obtain N measurement results ${y_1},\;{y_2},\,\;\ldots ,\;{y_i},\textrm{ }\ldots ,\;{y_N}$. Consequently, we can establish

$$\left[ {\begin{array}{{c}} {{y_1}}\\ \vdots \\ {{y_i}}\\ \vdots \\ {{y_N}} \end{array}} \right] = \left[ {\begin{array}{ccccc} {{a_{11}}}& \cdots &{{a_{1i}}}& \cdots &{{a_{1N}}}\\ \vdots & \ddots & \vdots & \ddots & \vdots \\ {{a_{i1}}}& \cdots &{{a_{ii}}}& \cdots &{{a_{iN}}}\\ \vdots & \ddots & \vdots & \ddots & \vdots \\ {{a_{N1}}}& \cdots &{{a_{Ni}}}& \cdots &{{a_{NN}}} \end{array}} \right]\left[ {\begin{array}{{c}} {{x_1}}\\ \vdots \\ {{x_i}}\\ \vdots \\ {{x_N}} \end{array}} \right] + \left[ {\begin{array}{{c}} {{e_1}}\\ \vdots \\ {{e_i}}\\ \vdots \\ {{e_N}} \end{array}} \right]$$

Equation (1) can be further abbreviated as

$$y = Ax + e$$

In Eq. (2), A represents the multiplexing matrix, while e represents the measurement error. Equation (2) should satisfy the following conditions:

  • • The variables ${x_i}$ are uncorrelated.
  • • The error ${e_i}$ are uncorrelated and it is assumed that the error term follows a random variable with a mean of zero and a variance of ${\sigma ^2}$.
  • • The error ${e_i}$ are independent of the ${x_i}$, which mathematically represents the linear behavior of the detector.
  • • Matrix A is nonsingular, ensuring that Eq. (2) is solvable.

Hence, we can derive the linear and unbiased estimate for x as shown in Eq. (3).

$$\left\{ {\begin{array}{{c}} {\hat{x} = {A^{ - 1}}y = x + {A^{ - 1}}e = x + Be}\\ {B = {A^{ - 1}} = {{({{b_{ij}}} )}_{N \times N}}} \end{array}} \right.$$

At the same time, we can obtain the error and variance of each variable as

$$\left\{ {\begin{array}{{c}} {{e_i} = {{\hat{x}}_i} - {x_i} = \sum\limits_{j = 1}^N {{b_{ij}}e} }\\ {\sigma_i^2 = E{{({{{\hat{x}}_i} - {x_i}} )}^2} = \sum\limits_{j = 1}^N {b_{ij}^2{\sigma^2}} } \end{array}} \right.$$

Based on weighing designs, matrix A can take two different forms: matrices with entries -1, 0, 1 and matrices with entries 0, 1, which correspond to balance weighing and spring weighing, respectively. However, matrices composed of 0 and 1 are more commonly used in engineering applications. Therefore, in the context that follows, the term matrix A specifically refers to the multiplexing matrix consisting of 0 and 1. The Hadamard matrix (H-matrix) is considered the optimal balance weighing matrix, and the Hadamard 2-design matrix (S-matrix) is the optimal spring weighing matrix, which can be easily derived by converting from the H-matrix. However, it should be noted that not all H-matrix of any order can be easily found. Currently, the n-order H-matrix can only be generated with recursive algorithms when n, n/12, or n/20 is the power of 2. By converting the H-matrix of order n, we can obtain the S-matrix of order n-1. In this converted matrix, the number of multiplexing channels, represented by the variable p, in each row is limited to n/2. Consequently, the H-matrix lacks flexible modularity because the detector must possess a large dynamic range to accommodate both single channel measurements and measurements of n/2 channel. The dynamic range of a detector refers to the ratio of the maximum permissible input signal to the detector's noise. In most cases, using a n/2 multiplexing channel is not necessary and will increase the cost of the detector.

In practical applications, the selection of an appropriate value for N should be based on the size of the target's image on the DMD surface. On the other hand, the suitable number of multiplexing channels p should be chosen based on the detector's dynamic range. It is crucial to ensure that both single-channel and p-channel measurements fall within the detector's dynamic range. If the number of channels N equals n-1 and the number of channels per row p equals n/2, matrix A can be easily generated from transformation of H-matrix. However, if n and p take any integer values, matrix A cannot be obtained using this approach.

Here, we propose a method for generating the matrix A that can be applied to any integers N and p, as depicted in Fig. 2. If the input values of N and p satisfy the conditions for generating the H-matrix, we can obtain the multiplexing matrix A by transforming the H-matrix into a S-matrix. However, if N and p are not suitable for generating a H-matrix, we will employ an alternative method to generate the multiplexing matrix. In this alternative method, we start by generating a vector of length N, where the first element is set to 1. The remaining N-1 elements are randomly filled with p-1 ones. This process yields a vector of length N with p ones. The second vector is obtained by cyclically shifting the first vector one position to the right. By following this pattern, we can generate N vectors, which together form a matrix. We then check whether this matrix is full rank. If it is not, we restart the process by generating a new first vector. If the matrix is found to be full rank, the multiplexing matrix A is obtained. In matrix A, if ${a_{11}},{a_{22}},\ldots ,{a_{nn}} = 1$ and all other values are 0, it is the mathematical model of line scanning method. The matrix A obtained by shifting the first vector is actually a circulant matrix. Circulant matrix is a special form of Toeplitz matrix, where each element of its row vector is the result of each element of the previous row vector being sequentially shifted one position to the right. A circulant matrix can be quickly solved by using discrete Fourier transform. As a consequence, multiplexing matrix A in the form of circulant matrices actually provides us with a fast solution method, which is very helpful to quickly obtain spectral imaging data. Once we have obtained the multiplexing matrix A, the next step is to generate the control pattern for the DMD.

 figure: Fig. 2.

Fig. 2. The algorithm flowchart for generating the matrix A.

Download Full Size | PDF

2.3 Acquisition and reconstruction of spectral image

The method of controlling the DMD through a multiplexing matrix is depicted in Fig. 3. Initially, it is necessary to determine the active area on the DMD by defining the starting and ending columns as ${c_1}$ and ${c_N}$, respectively. The total number of columns is denoted by N, including the starting and ending columns. Subsequently, each of the N columns of the DMD is associated with the corresponding N columns of matrix A, generating control patterns. Specifically, for the f-th frame control pattern, the c-th column of the DMD is controlled by the f-th column in the c-th row of matrix A to implement corresponding deflection. In total, N control patterns are generated.

 figure: Fig. 3.

Fig. 3. Principle of multiplexing pattern generation. (a) Multiplexing matrix A. (b) Multiplexing patterns.

Download Full Size | PDF

To acquire spectral images, N patterns are sequentially loaded onto the DMD, and corresponding images of these patterns are captured using the external trigger mode of the camera. This process yields N multiplexed measurement values, denoted as ${y_1},\;{y_2},\,\;\ldots ,\;{y_i},\textrm{ }\ldots ,\;{y_N}$. By applying Eq. (3), we can solve for N individual measurement results. These results ${x_1},\textrm{ }{x_2},\textrm{ }\ldots ,\textrm{ }{x_i},\textrm{ }\ldots ,\textrm{ }{x_N}$ are stored in the BIL (band interleaved by line) format, a widely adopted format for push-broom hyperspectral imaging. The spectral information of the target can be easily obtained from the BIL format data, while the BSQ (band sequential) data format is utilized to represent the two-dimensional spatial data of the target, as depicted in Fig. 4. Prior to accurately extracting spectral information from the BIL format data or extracting spectral information from the BIL format data and then converting it to the BSQ format, spectral calibration must be performed on each column of the DMD. The spectral calibration enables the determination of the spatial distribution function of the spectrum.

 figure: Fig. 4.

Fig. 4. (a)Target in column c. (b)The BIL data format for the target in column c. (c)BSQ data format with the target at the blue wavelength.

Download Full Size | PDF

Assuming that the spectral calibration process has been successfully conducted, we have established the relationship between the spatial distribution $f({w,h} )$ and the wavelength $\lambda $, specific column number c. Equation (5) describes this relationship, considering the pixel positions as $({w,h} )$ and the image size as $W \times H$.

$$f({w,h} )= g({\lambda ,c} )$$

If we neglect the bending of spectral lines and assume they are perfectly vertical, we can simplify the relationship to Eq. (6).

$$f(w )= g({\lambda ,c} )$$

In some cases, it is reasonable to assume that is a linear spatial distribution relationship caused by grating dispersion and different columns, resulting in Eq. (7), where ${k_1}$, ${k_2}$ , and ${k_3}$ are constants and ${k_1}$ represents the reciprocal of the spectral sampling rate.

$$f(w )= {k_1}\lambda + {k_2}c + {k_3}$$

However, in practice, the correlation between $\lambda $ and $f(w )$ is not strictly linear for a given c of the DMD. The coefficient between $\lambda $ and $f(w )$ often varies with the column c. Therefore, Eq. (6) needs to be represented by more complex formulas. For instance, if there is a linear correlation between ${k_1}$ and c, Eq. (7) can be modified to Eq. (8).

$$f(w )= {r_1}\lambda c + {r_2}\lambda + {r_3}c + {r_4}$$

After completing the spectral calibration, we can extract and concatenate images captured at different spatial positions but at the same wavelength in the BIL data format. This process enables us to obtain data in the BSQ format for a specific wavelength. However, if our focus is solely on spectral data at a particular location, the BIL data format is sufficient.

3. Results and discussions

3.1 Multiplexing encoding matrix

To address the multiplexing requirements, we employ N = 639 diagonal columns in the central region of the DMD. This choice is motivated by the fact that 639 columns effectively cover the imaging area, and a multiplexing matrix of order 639 can be obtained through Hadamard matrix transformation. The resulting multiplexing matrix consists of 320 multiplexing channels, denoted as p = 320. Consequently, the camera's dynamic range must accommodate 320 multiplexing channels and ensure that a single channel excites the detector to generate an appropriate response, thereby avoiding issues such as underexposure or overexposure. In cases where the camera's dynamic range is relatively limited, the value of p can be reduced appropriately to generate a multiplexing matrix using random methods. However, it is crucial to note that this adjustment will lead to increased errors and variance.

In the following content, we present the results and analysis of the multiplexing matrix and encoding, specifically focusing on the multiplexing matrix generation and its properties. Figure 5(a) illustrates the error and variance of the multiplexing matrix generated using two different methods: random generation and Hadamard generated matrix, with N = 639 and p = 320. Although the two methods yield similar errors when the number of times varies, the random method exhibits higher variance compared to the Hadamard transform method. Notably, the variance of the former method displays significant volatility.

 figure: Fig. 5.

Fig. 5. (a) Comparison of errors and variances of two different generated matrices when N = 639 and p = 320. (b) Influence on the error and variance of randomly generated matrix when N = 639 and p changes.

Download Full Size | PDF

To further investigate the influence of p on the multiplexing matrix generated by the random method, Fig. 5(b) presents the error and variance when N = 639 and p varies. As the number of multiplexing channels p increases, both the error and variance gradually decrease, indicating an inverse proportional trend. It is worth mentioning that the error and variance corresponding to each p value are the outcomes of multiple randomly generated mean values. However, it is important to note that excessively large p values necessitate a higher dynamic range, which does not contribute significantly to further reducing errors and variance.

3.2 System spectral calibration

In the present study, we performed spectral calibration of the system using a pen-shaped mercury lamp (Gp3Hg-2, Zolix Instruments Co., Ltd). Mercury lamps are known for emitting light at five distinct wavelengths: 404.66nm, 435.84nm, 546.07nm, 576.96nm, and 579.07nm. To establish calibration equations for the 639 columns, we obtained a total of 27 dispersion spectral images of the mercury lamp, specifically for columns ${c_1},\,{c_{20}},\;{c_{45}},\;{c_{70}},\;\ldots ,\;{c_{570}},\;{c_{595}},\;{c_{620}}$, and ${c_{639}}$. To minimize errors, we collected 100 dispersion spectral images for each column and averaged them.

In Fig. 6(a), the spectral calibration image of column ${c_{320}}$ is displayed. This image clearly reveals the five wavelengths and their corresponding pixel positions of the mercury lamp. However, the 404.66 nm wavelength may not be readily distinguishable unless the contrast is adjusted. It is reasonable to assume a linear relationship between the variable w and the wavelength $\lambda $, thereby determining the spectral distribution relationship $f(w )= g(\lambda )$. Subsequently, we derived the spectral calibration equations using two methods: Eq. (7) and Eq. (8), leading to the expressions in Eq. (9) and Eq. (10), respectively. By substituting the five wavelengths of the mercury lamp back into Eq. (9) and Eq. (10), we calculated the spectral calibration errors for both methods.

$$f(w )={-} 1.6478\lambda + 0.5440c + 1876.4008$$
$$f(w )= 1.1424 \times {10^{ - 4}}\lambda c - 1.6835\lambda + 0.5440c + 1876.4008$$

 figure: Fig. 6.

Fig. 6. (a) Spectral calibration image of column. (b) Spectral calibration errors for both methods.

Download Full Size | PDF

Figure 6(b) demonstrates the spectral calibration errors for both methods. It is evident that the spectral calibration relationship described by Eq. (10) exhibits higher accuracy compared to Eq. (9). This improvement is attributed to the correction of linear errors present in Eq. (9).

3.3 Spectral imaging verification

To validate the spectral imaging functionality of the CCSA-HIS system, we performed a verification experiment using a color checker chart (Fig. 7(a)) and employed multiplexing encoding matrix. The color checker chart is directly displayed on an iPad (iPad 2020) and the iPad is set to a constant brightness (480 nit when white is displayed).

 figure: Fig. 7.

Fig. 7. (a) The color checker chart. (b) The spectral curves of 4 different colors (red, yellow, green, blue) measured by the CCSA-HIS system. (c) The spectral curves of 4 different colors (red, yellow, green, blue) measured by the Oceanview spectrometer. (d) Reconstructed full spectrum spectral images.

Download Full Size | PDF

To capture the spectral information, we utilized 639 diagonal columns of the DMD, obtaining a total of 639 coded images, each consisting of 128 multiplexed channels. The exposure time for each image was set to 38 ms, resulting in a total acquisition time of approximately 24 seconds. The spectral curves for four different colors (red, yellow, green, blue) measured by the CCSA-HIS system are presented in Fig. 7(b). For comparison, we also measured the spectral curves of the same four colors using the Oceanview commercial spectrometer, as shown in Fig. 7(c). This difference in shape can be easily eliminated through radiation calibration, but it is not within the scope of this paper's discussion.

To evaluate the system's spectral imaging capabilities across the full spectral range, we obtained pseudo-color spectral images with 10 nm intervals ranging from 400 nm to 750 nm through experiments. As illustrated in Fig. 7(d), the spectral image reflects the iPad's three distinct wavelengths located near 450 nm, 550 nm, and 610 nm, appearing relatively bright. Additionally, changes in brightness are observed in the spectral images corresponding to variations in color across different regions of the color checker chart. When the intensity is extremely weak at certain wavelengths, the spectral image appears black. These observations provide further confirmation of the CCSA-HIS system's spectral imaging capability, albeit with a measure of limitations due to the state without radiation correction.

Furthermore, to demonstrate the improved SNR and acquisition efficiency achieved through CCSA-HIS system, we collected experimental data with p = 32, 64, 128, 256, 320. In addition, we also collected data in line scanning mode (p = 1) at the same exposure time. As shown in Fig. 8, the SNR of the image can been improved by around four times at the same exposure time compared to the line scanning. Moreover, the more multiplexing channels p there are, the higher the SNR of the image. When p = 1, 32, 64, 128, 256, 320 and the image reaches the same grayscale value, the exposure times are 1000 ms, 119 ms, 69 ms, 38 ms, 19 ms, and 10 ms, respectively. Therefore, when using CCSA-HIS and multiplexing technology, the exposure time can be significantly reduced, which can greatly improve scanning efficiency.

 figure: Fig. 8.

Fig. 8. Comparison of SNR between multiplexed and single column measurements at the same exposure time.

Download Full Size | PDF

4. Conclusion

In this study, we propose a CCSA-HIS that improves the SNR and acquisition efficiency compared with LS-HIS. The optical system architecture of CCSA-HIS is similar to the novel DMD-based LS-HIS, and the main function of DMD is to replace the mechanical slits in traditional LS-HIS. We introduce the concept of multiplexing to DMD-based LS-HIS, treating a single column as a spatial channel. Then, a method to generate a multiplexing encoding matrix with any order N and any multiplexing channel p is presented. Notably, this is the first multiplexing method where the order N and the number of multiplexing channels p can be set as desired. We applied the multiplexing encoding matrix to different columns of DMD to form a one-dimensional multi-column encoding scanning aperture. By employing this, we significantly improve the SNR and acquisition efficiency compared with LS-HIS. Specifically, when N = 639 and p = 128, the SNR and acquisition efficiency of the CCSA-HIS system are 4 and 26 times higher than that of the LS-HIS system, respectively. Additionally, we propose a more accurate spectral calibration method to improve the precision of wavelength calibration CCSA-HIS. Future research will focus on three aspects: 1) studying the mathematical construction of the optimal multiplexing matrix A; 2) Improve the processing speed of the system, especially involving converting BIL data to BSQ data; 3) Improve the calibration accuracy and efficiency of the system. Overall, our CCSA-HIS system offers a promising solution for hyperspectral imaging, providing significant improvements in performance and addressing key challenges faced by traditional LS-HIS systems.

Funding

National Natural Science Foundation of China (51975483, 52205603); Natural Science Foundation of Ningbo (202003N4033); Chinese Aeronautical Establishment (2019ZD053005).

Acknowledgments

We gratefully acknowledge the useful discussions and encouragement from Miss Wenjie Li.

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. G. Vivone, “Multispectral and hyperspectral image fusion in remote sensing: A survey,” Inf. Fusion 89, 405–417 (2023). [CrossRef]  

2. M. David, U. Sergio, G. Hans, et al., “Hyperspectral camera as a compact payload architecture for remote sensing applications,” Appl. Opt. 62(8), C88–C98 (2023). [CrossRef]  

3. H. Zhang, H. Zhou, W. Pan, et al., “Accumulation of nanoplastics in human cells as visualized and quantified by hyperspectral imaging with enhanced dark-field microscopy,” Environ. Int. 179, 108134 (2023). [CrossRef]  

4. T. Yang, Z. Xu, W. Ren, et al., “Compressive hyperspectral microscopic imaging using spectral-coded illumination,” Opt. Laser Technol. 166, 109631 (2023). [CrossRef]  

5. K. Shahid, Q. Akeel, F. Umar, et al., “Hyperspectral imaging: a review and trends towards medical imaging,” Curr. Med. Imaging 19(5), 417–427 (2022). [CrossRef]  

6. M. Tran and B. Fei, “Compact and ultracompact spectral imagers: technology and applications in biomedical imaging,” J. Biomed. Opt. 28(04), 040901 (2023). [CrossRef]  

7. B. Wu, C. Liu, R. Xu, et al., “A Target-Based Non-Uniformity Self-Correction Method for Infrared Push-Broom Hyperspectral Sensors,” Remote Sens. 15(5), 1186 (2023). [CrossRef]  

8. L. Zhang, Y. Yang, S. Zhao, et al., “Study on Hyperspectral full polarization imaging system based on spatial modulation,” Appl. Opt. 62(5), 1428–1435 (2023). [CrossRef]  

9. H. Xie, Z. Zhao, J. Han, et al., “Dual camera snapshot hyperspectral imaging system via physics-informed learning,” Opt. Laser Eng. 154, 107023 (2022). [CrossRef]  

10. R. Arablouei, E. Goan, S. Gensemer, et al., “Fast and robust pushbroom hyperspectral imaging via DMD-based scanning,” in Novel Optical Systems Design and Optimization XIX (SPIE) 9948, 7–17 (2016). [CrossRef]  

11. T. Yoon, C. Kim, K. Kim, et al., “Emerging applications of digital micromirror devices in biophotonic fields,” Opt. Laser Technol. 104, 17–25 (2018). [CrossRef]  

12. X. Dong, X. Xiao, Y. Pan, et al., “DMD-based hyperspectral imaging system with tunable spatial and spectral resolution,” Opt. Express 27(12), 16995–17006 (2019). [CrossRef]  

13. X. Dong, Y. Shi, X. Xiao, et al., “Non-paraxial diffraction analysis for developing DMD-based optical systems,” Opt. Lett. 47(18), 4758–4761 (2022). [CrossRef]  

14. X. Xiao, X. Dong, and Y. Yu, “MEMS-based linear micromirror array with a high filling factor for spatial light modulation,” Opt. Express 29(21), 33785–33794 (2021). [CrossRef]  

15. K. J. Kearney and Z. Ninkov, “Characterization of a digital micromirror device for use as an optical mask in imaging and spectroscopy,” in Spatial Light Modulators (SPIE) 3292, 81–92 (1998). [CrossRef]  

16. R. DeVerse, R. Hammaker, and W. Fateley, “Hadamard transform Raman imagery with a digital micro-mirror array,” Vib. Spectrosc. 19(2), 177–186 (1999). [CrossRef]  

17. K. Kearney, M. Corio, and Z. Ninkov, “Imaging spectroscopy with digital micromirrors,” in Sensors and Camera Systems for Scientific Industrial, and Digital Photography Applications (SPIE) 3965, 11–20 (2000). [CrossRef]  

18. J. Zhou, Y. Yang, L. Li, et al., “Developing, integrating and validating a compressive hyperspectral video imager,” in Signal Processing,” Sensor/Information Fusion, and Target Recognition XXIX (SPIE) 11423, 31–202 (2020). [CrossRef]  

19. P. Gattinger, I. Zorin, C. Rankl, et al., “Spectral-coding-based compressive single-pixel nir spectroscopy in the sub-millisecond regime,” Sensors 21(16), 5563 (2021). [CrossRef]  

20. W. Chen, M. Sun, W. Deng, et al., “Hyperspectral imaging via a multiplexing digital micromirror device,” Opt. Lasers Eng. 151, 106889 (2022). [CrossRef]  

21. A. Orth, M. J. Tomaszewski, R. N. Ghosh, et al., “Gigapixel multispectral microscopy,” Optica 2(7), 654–662 (2015). [CrossRef]  

22. X. Dong, G. Tong, X. Song, et al., “DMD-based hyperspectral microscopy with flexible multiline parallel scanning,” Microsystems Nanoeng. 7(1), 68 (2021). [CrossRef]  

23. H. Xie, J. Lu, J. Han, et al., “Fourier coded aperture transform hyperspectral imaging system,” Opt. Lasers Eng. 163, 107443 (2023). [CrossRef]  

24. Z. Xia, C. Xu, H. Cai, et al., “Radiative transfer modeling of Hadamard coded infrared imaging spectrometer based on DMD,” in Seventh Asia Pacific Conference on Optics Manufacture and 2021 International Forum of Young Scientists on Advanced Optical Manufacturing (APCOM and YSAOM 2021, SPIE), 12166, 560–565 (2022).

25. C. Xu, T. Xu, G. Yan, et al., “Super-resolution compressive spectral imaging via two-tone adaptive coding,” Photonics Res. 8(3), 395–411 (2020). [CrossRef]  

26. X. Sun, B. Hu, L. Li, et al., “An engineering prototype of Hadamard transform spectral imager based on Digital Micro-mirror Device,” Opt. Laser Technol. 44(1), 210–217 (2012). [CrossRef]  

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. (a) A 3D model of CCSA-HIS. (b) DMD rotated 45° along the Z-axis with the multiplex column coded scanning mode. (c) Single channel scanning mode.
Fig. 2.
Fig. 2. The algorithm flowchart for generating the matrix A.
Fig. 3.
Fig. 3. Principle of multiplexing pattern generation. (a) Multiplexing matrix A. (b) Multiplexing patterns.
Fig. 4.
Fig. 4. (a)Target in column c. (b)The BIL data format for the target in column c. (c)BSQ data format with the target at the blue wavelength.
Fig. 5.
Fig. 5. (a) Comparison of errors and variances of two different generated matrices when N = 639 and p = 320. (b) Influence on the error and variance of randomly generated matrix when N = 639 and p changes.
Fig. 6.
Fig. 6. (a) Spectral calibration image of column. (b) Spectral calibration errors for both methods.
Fig. 7.
Fig. 7. (a) The color checker chart. (b) The spectral curves of 4 different colors (red, yellow, green, blue) measured by the CCSA-HIS system. (c) The spectral curves of 4 different colors (red, yellow, green, blue) measured by the Oceanview spectrometer. (d) Reconstructed full spectrum spectral images.
Fig. 8.
Fig. 8. Comparison of SNR between multiplexed and single column measurements at the same exposure time.

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

[ y 1 y i y N ] = [ a 11 a 1 i a 1 N a i 1 a i i a i N a N 1 a N i a N N ] [ x 1 x i x N ] + [ e 1 e i e N ]
y = A x + e
{ x ^ = A 1 y = x + A 1 e = x + B e B = A 1 = ( b i j ) N × N
{ e i = x ^ i x i = j = 1 N b i j e σ i 2 = E ( x ^ i x i ) 2 = j = 1 N b i j 2 σ 2
f ( w , h ) = g ( λ , c )
f ( w ) = g ( λ , c )
f ( w ) = k 1 λ + k 2 c + k 3
f ( w ) = r 1 λ c + r 2 λ + r 3 c + r 4
f ( w ) = 1.6478 λ + 0.5440 c + 1876.4008
f ( w ) = 1.1424 × 10 4 λ c 1.6835 λ + 0.5440 c + 1876.4008
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.