Open Access
Add to BibSonomy
Abstract
This research presents a practical approach for wavefront reconstruction and correction adaptable to variable targets, with the aim of constructing a high-precision, general extended target adaptive optical system. Firstly, we delve into the detailed design of a crucial component, the distorted grating, simplifying the optical system implementation while circumventing potential issues in traditional phase difference-based collection methods. Subsequently, normalized fine features (NFFs) and structure focus features (SFFs) which both are independent of the imaging target but corresponded precisely to the wavefront aberration are proposed. The two features provide a more accurate and robust characterization of the wavefront aberrations. Then, a Noise-to-Denoised Generative Adversarial Network (N2D-GAN) is employed for denoising real images. And a lightweight network, Attention Mechanism-based Efficient Network (AM-EffNet), is applied to achieve efficient and high-precision mapping between features and wavefronts. A prototype of object-independent adaptive optics system is demonstrated by experimental setup, and the effectiveness of this method in wavefront reconstruction for different imaging targets has been verified. This research holds significant relevance for engineering applications of adaptive optics, providing robust support for addressing challenges within practical systems.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Image-based wavefront sensing is a technology to measure wavefront error, which utilizes the far-field spot quality as an objective function, employing continuous iterative optimization to retrieve the wavefront phase [1]. Such a methodology dispenses with specialized wavefront sensors, offering advantages such as system simplicity, high optical efficiency, and consistency between control performance and image quality evaluation [2], is currently used in a wide range of applications such as inertial confinement fusion [3], microscopy [4], imaging of the human eye [5], optical tracking [6], and free-space laser communications [7]. In recent years, there has been rapid development in deep learning, which has caught the attention of scholars due to its inherent advantage of not requiring iteration or optimization [8–13].
Researchers have demonstrated that state-of-the-art convolutional neural networks can directly estimate wavefronts represented by Zernike coefficients not only from point source intensity images but also from intensity images of specified extended objects [10]. This has indeed been shown to be feasible. In 2019, Nishizaki et al. used the MNIST dataset as the imaging targets and estimated the 2-32nd order Zernike coefficients [10]. In 2022, Lu et al. also utilized neural networks to achieve wavefront reconstruction for handwritten digit datasets using the MNIST dataset as an extended object dataset [14]. In the same year, Li et al. implemented wavefront reconstruction for satellite images using EfficientNet-B0 [15].
However, phase retrieval methods based on a single image encounter the issue of multiple solutions and can easily stall during the iterative process [16]. Besides, these methods are tailored to specific types of extended objects or single extended objects, making them less readily transferable to achieve high-precision wavefront reconstruction in different extended object scenarios. Such deep learning-based methods are somewhat limited in their applicability and lack broad generalization, especially for variable imaging targets. Therefore, The development of object-independent wavefront reconstruction technology would be very interesting and practical.
To date, some progress has been made in the domain of wavefront reconstruction for variable extended objects. Some scholars have started collecting two or more images to solve the multiple solutions problem, and then in order to mitigate the algorithm’s dependence on the imaging targets, preprocessing steps before feeding the acquired target images into neural networks have been introduced to remove the algorithm’s correlation with the imaged targets [17–19]. In our previous works [20,21], it has been verified through simulation analysis that object-independent wavefront sensing can be achieved by the combination of phase difference (PD) method and feature extraction method. These new approaches have broadened the applications of deep learning in extended objects wavefront reconstruction and correction. However, most of the existing literature addressing object-independent wavefront detection techniques primarily remain at the simulation stage, with limited experimental work on wavefront reconstruction and correction, and there is little in-depth research on practical issues related to image acquisition methods, noise impact, wavefront correction, and other real-world concerns. All of these aspects are of significant importance for the practical engineering applications of such methodologies.
In this research, we propose a practical method for achieving wavefront reconstruction and correction for variable extended objects, providing a reference for the development of real-time, widely applicable extended object adaptive optical systems. Our main contributions can be summarized as follows. (1) A detailed process and methodology for the design of a distorted grating to enhance system real-time performance and data accuracy are presented; (2) We delve into image registration algorithms and noise models to minimize interference with subsequent feature extraction method; (3) We propose two new features, NFFs and SFFs, which have refinement and robustness characteristics, respectively; (4) A lightweight network AM-EffNet is introduced to make the model more efficient and capable of nonlinear fitting between features and wavefront aberrations with high precision. The effectiveness of our method for wavefront reconstruction concerning different imaging targets have been experimentally validated.
The rest of the paper is organized as follows: Section 2 introduces the proposed approach and details related to feature extraction, distorted design, image preprocessing and proposed network. In Section 3, the optical system and experimental results are presented in detail. Some discussions about this method are listed in Section 4, and the conclusion is finally given in Section 5.
2. Methodology
In this section, we begin with the definition and proposal of feature extraction methods. Subsequently, we provided a detailed description of the design process and details of the distorted grating. Then we analyzed and demonstrated the image preprocessing work. Finally, we provided an explanatory description of the proposed wavefront reconstruction network.
2.1 Feature extraction of imaging target
In our previous work [20], inspired by the concept of object-independent function proposed by Kendric [22], a series of simulations and analyses in the domain of different features have been conducted. Now, considering the noise present during the actual image acquisition process, a more comprehensive and detailed discussion on the selection and definition of features is provided. The power matrix $M_{P}$ and sharpness matrix $M_{S}$ can be calculated from the spectrum of the positive and negative defocus images [22]:
As depicted in Fig. 1(a-c), we observed that by employing a multiplication operation, it is possible to simultaneously capture the sharpness features listed in Fig. 1(a) as well as the power features listed in Fig. 1(b), meaning that the information contained in Fig. 1(c) encompasses both sets of sharpness and power features. Therefore, our approach involved the use of multiplication operations to enhance the effective information contained within the features.
Fig. 1. (a-c): Multiplication operation increases effective information; (d-f): Square root operation enhances fuzzy information.
Furthermore, we observed that using the square root operation could further enhance the details of blurriness, as illustrated in Fig. 1(d-f). By examining the circled regions in Fig. 1(d) and (e), we can discern that the feature image, after being subjected to the fourth root operation, contains more details and can to some extent mitigate the impact of local blurriness. However, using an insufficient or excessive number of square root operations can result in an uneven distribution of feature values, with little difference between feature points, as shown in Fig. 1(f). In this case, a large portion of data may become concentrated within a small range, potentially causing the neural network to overlook some crucial features and thereby resulting in information loss. We define the new feature as follows:
to further select an appropriate value for "n", we conducted a statistical analysis of the average variance values over 1000 sets of different "n" values. The results indicated that the variance of the features obtained after the fourth root operation is the highest, suggesting that the feature distribution is most uniform at this point. Consequently, we selected "n=4" as the optimal choice. Ultimately, we define the feature as follows: from the above Eqs. (1),(2) and (4), it is evident that this feature extraction method eliminates the information related to the imaged target while retaining the information pertaining to wavefront aberrations.However, In cases where signal and noise coexist, there is usually some overlap in their frequency domain representations. Within these overlapping frequency bands, noise can mask or affect the signal, making it more difficult to accurately distinguish or recognize portions of the signal in these bands. In the time domain, however, the signal tends to be concentrated in the major structures of the image, while the noise is more dispersed. When the frequency domain features are inverse transformed to the time domain, the image structure and the main part of the signal are usually more concentrated, while the noise is relatively dispersed, so the effect of the noise on the time domain features will be relatively small. Convert the frequency domain feature of Eq. (4) to the intensity feature of the time domain:
where $\mathcal {F}^{-1}$ denotes inverse Fourier transformation. Fig. 2 shows the results of the proposed frequency domain features and the corresponding time domain features in the presence of noise, and we can see that they are more sensitive to the noise in the frequency domain, while they are relatively less sensitive to the noise in the time domain.Fig. 2. Comparison of frequency and time domain features under noisy condition. Structure Similarity Index Measure(SSIM) is used to judge the similarity between the noisy and noise-free features.
Therefore, we consider introducing the inverse transform to the time domain based on the frequency domain features and combining the two features to enhance the robustness of the method to noise. The proposed frequency domain features with data distributed between 0 and 1 incorporate both sharpness and power features and enrich the feature detail information, so the frequency domain features are defined as normalized fine features (NFFs), and the time domain features focus on the overall structure and main characteristics of the images, and dilute the influence of noise, so are defined as structural focus features (SFFs). The two features are both capable of eliminating the imaging target during calculations. As shown in Fig. 3, under the same atmospheric turbulence, the feature extraction results before and after changing the imaging target are completely consistent.
Fig. 3. Feature extraction results before and after changing the imaging target. The computational results demonstrate that both the NFF and SFF features of the different imaging targets are identical (SSIM= 1).
2.2 Design of distorted grating
2.2.1 Parameter design
Traditional PD methods, in order to collect in-focus and defocused images, typically require the simultaneous operation of two cameras with the same frequency or the movement of a single camera to capture the images twice. These image acquisition methods entail relatively complex structures [23,24], demand high precision in installation, and introduce a certain time delay during data collection, thereby impacting real-time data processing. As a result, they have certain limitations in applications where dynamic wavefront reconstruction is crucial [25]. As shown in Fig. 4, the distorted grating is actually an off-axis Fresnel zone plate, which possesses a symmetrically distributed $\pm$ 1st-order diffraction optical axis and a pair of conjugate focal lengths exists at the $\pm$ 1st-order. It is used in close proximity to a short-focal-length lens, where it serves the purpose of splitting the light, while the short-focal-length lens is responsible for focusing. This section will provide a detailed parameter calculation and design scheme for distorted gratings.
Fig. 4. The schematic of the imaging process with the distorted grating. (a) represents a schematic of the distorted grating itself. (b) shows the far-field image of a point source in the aberration-free case when imaged through the distorted grating.
By definition, the defocus amount $W_{20}$ refers to the grating’s defocusing capability, which is equal to the additional optical path difference introduced by the $\pm$1st-order diffracted wavefront at the edge of the pupil. $f$ is the focal length of the off-axis Fresnel zone plate, $R$ is the radius of the distorted grating, $x_0$ is the off-axis displacement of the center of the distorted grating pupil from the center of the Fresnel zone plate, $W_{L}$ is the central wavelength of the incident light, and $d_0$ is the grating period at the center of the pupil.
Algorithm 1. Parameterization Design of Distorted Grating
We provide the pseudo-code for parameterization design of distorted grating in Algorithm 1. With the $W_{L}$, $W_{20}$, $R$, $f_{L}$, and $\triangle y$, one can intuitively and comprehensively establish the design parameters of the distorted grating, sequentially computing the focal length $f_g$ and off-axis amount $x_0$ of the distorted grating.
Furthermore, it is necessary to determine the number of phase steps, step width, and step height for the distorted grating. In this research, considering the fabrication complexity, we set the step width to be divided into two equal parts, and in order to achieve equal and higher efficiency for $\pm$ 1st-order diffraction, a binary phase-type distorted grating is adopt, whose step height is set to 0.96$\pi$.
2.2.2 Impact of lateral distance on imaging performance
The lateral distance is defined as the distance between the $\pm$ 1st-order diffraction spots on the focal plane departing from the 0-order spot. Combining the equations from Steps I–IV in Algorithm 1 yields the formula for the lateral distance $\triangle y$ as:
When this distance is moderate, signals from individual diffraction far-field spots do not interfere with each other, facilitating spot localization and segmentation. As shown in Fig. 5, if the distance is too small, positive and negative defocused far-field spots may mutually influence each other, leading to far-field spots overlap and distortion. On the other hand, excessive distance may also pose problems. A too-large lateral distance may cause the diffraction far-field spots to spill over the camera target surface, resulting in information loss and compromising the integrity of data acquisition.
Fig. 5. Far-field imaging scenarios corresponding to different lateral distances. Figures (a-c) show the situations where the lateral distance is too close, moderate and too large in sequence.
Therefore, the selection of lateral distance is crucial in the design of the distorted grating. In practical applications, the appropriate lateral distance should be set based on actual optical system parameters and the size of the imaging target. Through rational design and control, issues such as far-field spots crosstalk and overflow can be avoided, achieving complete far-field spots extraction, and ensuring the accuracy of subsequent feature extraction results.
2.2.3 Influence of spectral splitting ratio on the features
Currently, the technology for manufacturing binary optical elements is the fabrication technology of ultra-large-scale integrated circuits, including photolithography, masking, etching, or coating steps [26]. In this paper, the fabrication of defocus gratings employed photolithography. The process involves creating a mask according to the design pattern, transferring the pattern onto a photoresist applied to the substrate’s surface using photolithography techniques, and subsequently transferring the pattern from the photoresist surface to the substrate using etching. This sequence is repeated as needed to complete the production of multi-step phase gratings. Inevitably, the manufacturing process introduces processing errors into the device, impacting the precision of grating step height, flatness, and the steepness of both sidewalls, all of which influence diffraction efficiency. We define the diffraction efficiency of the $\pm$1st and 0th-order diffraction orders as the splitting ratio.
Fig. 6 simulates the influence of the splitting ratio on the NFFs under identical turbulent conditions. Fig. 6(a1) illustrates the effect of feature extraction on different expanded targets with a splitting ratio set to 1:1:1 under the same turbulent conditions, while Fig. 6 displays the situation when the splitting ratio is set to 3:1:1. It is evident that variations in the splitting ratio lead to changes in the feature maps but do not compromise the object-independent feature extraction capability of the method proposed in this paper. Under the same turbulent conditions and by replacing different expanded targets, the feature maps remain unchanged. Different distorted gratings may have different spectral ratios, but the derived feature maps with a fixed spectral ratio will always have a stable and unique mapping relationship with the wavefronts.
Fig. 6. Feature extraction results with identical wavefront aberration but different splitting ratios.
Once produced, the grating possesses a predetermined shape and step height. Therefore, even if processing errors exist in the production of defocus gratings during the manufacturing process, leading to certain differences between the actual and designed splitting ratio, training using collected experimental data remains unaffected by the differences in the spectral splitting ratio.
2.3 Image registration
Following the acquisition of far-field images resulting from distorted gratings, it is essential to accurately segment the spot images of positive and negative defocused images. Once there is some deviation in positioning, it will inevitably introduce some tilt errors in the dataset, causing discrepancies between the feature images and labels, thereby affecting the wavefront reconstruction. So, how to locate the center positions of the $\pm$1st-order diffraction far-field spots? In the presence of aberrations in the optical path, positive and negative defocused images exhibit random drift, intensity fluctuations, and light intensity spreading, making it challenging to achieve precise spot center localization. Since the feature extraction method proposed in this paper is independent of the imaging target, we employ point sources to create a training dataset for training. When there is no aberration in the optical path, the positive and negative defocused images are nearly identical, making the far-field spot center positioning simpler and more accurate. Based on this idea, we propose the sliding window traversal method for the registration of positive and negative defocused images. The specific algorithm workflow is illustrated in Fig. 7.
Firstly, in the case of imaging a point target without introducing additional aberrations, we define an N*N (in this research, N=340) sized sliding window. This window exactly matches the pixel size and resolution of the actual camera. A circular mask matrix is defined where the region inside the circle is set to 1, and outside is set to 0. The radius of the circular mask can be determined based on the number of pixels in simulation or actual imaging. The window is then traversed separately over the left and right halves of the far-field image. A summation operation is applied to the spots within the window, and the results are sequentially recorded and compared. After traversing both halves, the position of the maximum sum corresponds to the location of the positive and negative defocused spots that need to be localized. Recording the coordinates of the two sliding window enables image registration for other imaging targets and under different wavefront aberration scenario.
2.4 Image denoising
In practical applications, feature extraction is obtained through the operation of real images, and noise has a significant impact on the feature map. In order to establish an accurate and stable mapping relationship between the feature map and wavefront, accurate simulation and effective filtering of noise are the key to image preprocessing. Extracting unknown noise from noisy images is a challenging task, and this field faces many difficulties due to the lack of effective real training data. This paper employs a sim2real research approach to train a denoising network with strong generalization capabilities. Initially, a dataset comprising both point and extended targets is constructed, consisting of 100,000 pairs of images. Synthetic noisy images are generated by simulating various noise types and magnitudes found in real-world scenarios. These synthetic noises primarily include Gaussian noise and Poisson noise. Using these synthetic noisy images and their corresponding pristine images, a Noise-to-Denoised Generative Adversarial Network (N2D-GAN) is trained. Specifically, the N2D-GAN comprises a generator and a discriminator, and the network architecture is illustrated in Fig. 8.
The generator’s role is to transform the noisy input image into a denoised output image. We employ a combined loss function that encompasses adversarial loss and content loss. The formula for calculating the content loss is as follows:
The calculation formula for the adversarial loss is as follows [28]:
$D_{\theta _{G}}(G_{\theta _{G}}(I^{LR}))$ represents the probability of the reconstructed image $G_{\theta _{G}}(I^{LR})$ being the real high-resolution image. For better gradient performance, we choose to minimize $-logD_{\theta _{G}}(G_{\theta _{G}}(I^{LR}))$ instead of $log[1-D_{\theta _{G}}(G_{\theta _{G}}(I^{LR}))]$ [29].
Combining the two aforementioned losses results in the perceptual loss function:
The discriminator’s task is to determine whether the input image is real or generated by the generator. The BCELoss (binary cross entropy loss) is selected as the loss function for the discriminator. Through the adversarial interplay between the generator and discriminator, the generator gradually learns to produce clearer images to deceive the discriminator, making it challenging to distinguish between real and generated images. When the discriminator can hardly differentiate between real and generated images, we consider the network training to be sufficiently thorough, and the generator has successfully learned noise removal.
It is worth emphasizing that the key to this approach lies in the large-scale dataset and the GAN architecture. The large dataset provides the network with a sufficient number of training samples, allowing it to learn various types of noise and their corresponding denoising strategies. The GAN structure, through adversarial learning, compels the generator to continuously enhance the quality of generated images. This strategy has been widely applied in the field of image denoising and has demonstrated outstanding performance in experiments [30,31].
2.5 Wavefront reconstruction network
As shown in Fig. 9, the AM-EffNet is selected as the backbone network for the wavefront reconstruction task, endowing the network with both efficiency and high precision characteristics. EffNet is a lightweight convolutional neural network structure designed to maintain model performance while reducing computational and parameter overhead, ensuring a balance between efficiency and high precision [32]. EffNet block uses deep separable convolutions instead of traditional convolutional operations. It decomposes standard convolutions into two steps: first, depthwise convolution, and then pointwise convolution, thus reducing computational demands and parameter count.
Fig. 9. Algorithm flowchart for wavefront reconstruction based on AM-EffNet, including the network architecture of AM-EffNet.
Furthermore, AM-EffNet incorporates the convolutional block attention module (CBAM), which is crucial for wavefront reconstruction tasks. Wavefront reconstruction requires the network to accurately capture and reconstruct the phase information present in subtle features and textures of images. The CBAM module contains two key components: the channel attention module and the spatial attention module. They learn the importance of features in the channel and spatial dimensions, respectively, and combine the two to generate the final attention weights [33]. Channel attention enables the network to automatically learn which feature channels are most useful for the task, thereby enhancing the expressive power of features, which can be summarized as:
Spatial attention, on the other hand, assists in capturing the feature relationships in different regions of the image, whose progress can be summarized as:
The overall attention process can be summarized as:
where $F$ represents the input feature maps, $MLP$ denotes a multilayer perception, $\sigma$ denotes the sigmoid function, $F^{7\times 7}$ expresses a convolution operation with the filter of $7\times 7$, $\otimes$ denotes element-wise multiplication, $F^{'}$ expresses the result of the final refined output. The combination of these two attention mechanisms allows the model to better understand the information in the wavefront data and perform more accurate reconstruction.3. Results
3.1 Wavefront reconstruction and correction experimental system construction
The experimental procedure is illustrated in Fig. 10. Due to the proposed method’s ability to achieve object-independent feature extraction, and compared to extended targets, point targets can be more easily controlled and generated. Therefore, during the training phase, to reduce the complexity and time cost of dataset creation, we fixed the imaging target as a point target (green laser with a wavelength of 520nm) for dataset generation and network training. The specific algorithmic approach is as follows: Randomly generate 20,000 sets of Zernike coefficients to simulate random aberrations present in the system. Use a spatial light modulator (SLM) to introduce known aberrations. The far-field images obtained through distorted grating imaging undergo image registration and denoising to obtain NFFs and SFFs. The size of the feature images is 340*340. Then establish the mapping relationship between the feature images and aberrations using AM-EffNet, enabling wavefront reconstruction and correction tasks for point targets or other extended targets.
Fig. 10. Wavefront reconstruction and correction experimental workflow for variable extended Targets.
In the testing phase, in order to verify the method’s applicability to other imaging targets, both point targets and different extended targets are used for creating a test dataset. Feature images of 3000 sets of test data under three different degrees of atmospheric turbulence are computed and the trained AM-EffNet is used to solve the aberration coefficients, which ultimately achieves the reconstruction and correction of wavefront aberrations.
The optical experiment setup for the testing phase is depicted in the Fig. 11 below. The design parameters of distorted gratings are: $W_{L}$ = 520 nm; $W_{20}$ = 3 * $W_{L}$; $f_{L}$ = 0.2 m; $R$ = 0.006 m; $\triangle y$ = 0.003 m. We utilized a projector to provide varying imaging targets. Within the optical path, a filter was placed to control the passage of green light with a central wavelength of 520nm. SLM1 was responsible for introducing wavefront aberrations, while SLM2 facilitated the correction of these aberrations. The well-trained network, with a input of the computed NFFs and SFFs images, can output the corresponding aberration coefficients. Subsequently, the control terminal sequentially calculated the conjugate phase. Finally, the corresponding image was loaded onto SLM2, achieving the wavefront distortion correction.
3.2 Pre and post denoising far-field images and features
In the process of acquiring images, there is often noise pollution. When dealing with extended targets for wavefront detection, whether it is iterative algorithms or current deep learning, it is easy to be affected by noise and has not enough robustness, resulting in inaccurate calculations. Therefore, we chose the N2D-GAN shown in Fig. 8 to achieve blind denoising of real images. The denoising and feature extraction comparison results between N2D-GAN and other denoising methods are provided in Supplement 1.
As illustrated in Fig. 12. N2D-GAN achieves clearer target contours and a cleaner background, resulting in almost no presence of noise or mixed features in the characteristics solved. Using incomplete denoising features for training will lead to non-convergence issues. Therefore, the denoising preprocessing step is crucial as it directly influences the subsequent feature extraction results, thereby affecting the accuracy of the final wavefront reconstruction.
3.3 Object-independent validation of proposed features
To validate the object-independent nature of the features proposed in Eqs. (4) and (5), different extended targets were imaged under the same wavefront aberration conditions. Subsequently, Eqs. (4) and (5) were used to calculate the NFFs and SFFs. Two sets of exemplary results are shown in Fig. 13.
Fig. 13. Feature results extracted for different extended targets under the same aberration conditions. A1 and A2 represent two different sets of aberration conditions.
From Fig. 13, we observe that when aberrations in the system change, the NFFs and SFFs of imaging targets also change accordingly. However, under the same aberration conditions, the features extracted for different extended targets are highly similar, thus validating the effectiveness of the object-independent feature extraction method proposed in this study.
3.4 Wavefront reconstruction and correction effect
The preprocessed NFFs and SFFs along with their corresponding Zernike coefficients are fed into the AM-EffNet for training. The network employs the Adam optimizer with an initial learning rate set to 0.0001, and the training duration is about 2 hours with an inference time about 2.9 ms. Testing is conducted for various imaging targets such as symbols, numbers, and point targets. The spatial expansion degree of imaging targets is categorized into levels based on the number of grid spaces occupied by the target. Level I encapsulates targets occupying a small portion of the surface (constituting an area ratio less than 1/3). Level II denotes targets occupying a moderate proportion of the surface area (ranging between 1/3 and 2/3), while Level III pertains to targets encompassing a significant or complete area of the target surface (constituting an area ratio exceeding 2/3). The test set consists of 3000 samples of four types of imaging targets, as shown in Table 1, averaged into ten groups of 300 imaging targets each based on their expansion level. For the above data, tests were performed at three different aberration levels, and the average RMS of the original incident wavefronts at these three different aberration levels were 0.2$\lambda$, 0.5$\lambda$ and 0.8$\lambda$, respectively.
Table 1. Test results for different types of imaging targets (original average RMS=0.2$\lambda$).
Tables 1–3 outline the results of wavefront reconstruction tests corresponding to different levels of target expansion under different degrees of aberrations. For the case where the original average RMS is 0.2$\lambda$, 0.5$\lambda$, and 0.8$\lambda$, the average residual wavefronts after reconstruction are 0.0224$\lambda$, 0.0773$\lambda$, and 0.129$\lambda$, respectively. The experimental results demonstrate that when the expansion level is less than or equal to level II (including point target), the recovered residual wavefront average RMSE can reach a level of less than 0.07$\lambda$ for a test set with an original incident wavefront average RMS of 0.5$\lambda$. Fig. 14 illustrates example results of wavefront reconstruction for two sets of test data.
Table 2. Test results for different types of imaging targets (original average RMS=0.5$\lambda$).
Table 3. Test results for different types of imaging targets (original average RMS=0.8$\lambda$).
Based on the wavefront reconstruction results, the conjugate phases can be sequentially solved. Finally, by loading the corresponding maps onto the calibrated SLM2, the corrections of wavefront distortions are achieved. Fig. 15 displays partial correction results from the test set, where NIQE (natural image quality evaluator) is an indicator used to evaluate image quality, based on the statistical characteristics of natural images and the perceptual features of the human visual system [34]. Lower values of NIQE mean better image quality and more natural images. From the images before and after correction in Fig. 15, it is evident that the corrected images show a effective improvement in resolution, demonstrating the effectiveness and accuracy of the proposed method in the experimental setup. Furthermore, various extended targets in the experiment have achieved favorable results, the average NIQE before and after correction was reduced from 19.02 to 15.25 for test set, confirming the applicability of the proposed method for aberration computation and correction in scenes with variable extended targets.
Fig. 15. Example illustrations of correction effects for different extended targets. Group (A) represents the imaging effect of the distorted grating before correction, while Group (B) represents the imaging effect after correction
4. Discussion
This research proposes a wavefront reconstruction and correction method tailored for variable extended targets, aiming to establish an efficient and widely applicable adaptive optical system. However, through testing and validation, several limitations in this approach have been identified under specific circumstances. Specifically, while this method has made progress in handling target-agnostic scenarios, its performance is observed to be less satisfactory for a small number of targets with complex contours and large scalability. More specific cases can be found in Supplement 1, which covers more wavefront senssing and correction cases for imaging targets of different taget scales. We speculate that the main reason may be that complex targets may introduce additional uncontrollable errors during the imaging process, such as surface texture changes or irregular shapes that may bring more complex noise. This makes it more difficult to decouple noise from complex target signals cleanly, while residual noise interferes with the wavefront reconstruction process and affects the final accuracy. In addition, issues related to the processing precision and error of distorted gratings may also lead to observed limitations. Additionally, the current methodology primarily focuses on single wavelength investigations, while wavefront reconstruction for wide-band imaging targets has not been fully considered. This oversight potentially curtails the method’s adaptability and effectiveness across diverse wavelength spectra. Moreover, in outdoor environments, the presence of atmospheric turbulence may introduce severe fluctuations in the imaging beam, leading to non-uniform distribution of light intensity within the system. This non-uniform distribution might impede the camera’s acquisition of far-field images under ideal conditions, thereby exerting a certain impact on the effectiveness of the proposed feature extraction method in this study.
Despite achieving some progress and encouraging results, we acknowledge areas for improvement concerning the aforementioned issues. Future research avenues could encompass the exploration of more efficient feature extraction methods for complex targets, wavefront reconstruction and correction schemes across broad band spectral sources, and the investigation of more robust approaches to mitigate external environmental disturbances.
5. Conclusion
A prototype of deep learning-based target independent adaptive optic system is demonstrated in this research, which comprehensively addresses issues concerning optical system construction, challenges in optical path setup, noise influences and aberrations correction. To enhance system real-time performance and data accuracy, a distorted grating is designed, mitigating issues related to same-frequency triggering and acquisition delay. In the preprocessing of far-field images, advanced image registration algorithms and denoising models are employed to reduce interference with feature extraction methods. Regarding feature extraction, we introduced the NFFs that simultaneously considers sharpness and power information, and the SFFs to enhance the noise immunity of our method. Additionally, an efficient lightweight network is adopted to achieve a nonlinear fitting of the proposed features and wavefront aberrations.
Through simulation and experimental validation, our method demonstrates its effectiveness in achieving wavefront reconstruction under different imaging targets. This research marks a new stride for adaptive optical systems in practical engineering applications, offering a reference for the development of more intelligent optical systems.
Funding
National Natural Science Foundation of China (62005285; 62305343).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. R. W. Gerchberg, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–250 (1972).
2. Linhai Huang and Rao Changhui, “Wavefront sensorless adaptive optics: a general model-based approach,” Opt. Express 19(1), 371–379 (2011). [CrossRef]
3. W. H. Jiang, C. L. Guan, X. J. Zhang, et al., “New progress on adaptive optics in inertial confinement fusion facility,” Chin. J. Laser 36(7), 1625–1634 (2009). [CrossRef]
4. O. Albert, L. Sherman, G. Mourou, et al., “Smart microscope: an adaptive optics learning system for aberration correction in multiphoton confocal microscopy,” Opt. Lett. 25(1), 52 (2000). [CrossRef]
5. N. Ling, Y. Zhang, X. Rao, et al., “Small adaptive optical imaging system for cells of living human retina,” Acta Opt. Sin. (2004).
6. K. D. Wulff, D. G. Cole, R. L. Clark, et al., “Holographic optical tweezers aberration correction using adaptive optics without a wavefront sensor,” Proc. SPIE 6236, 63262Y (2006). [CrossRef]
7. Z. Li, J. Cao, W. Liu, et al., “Comparison of swarm intelligence algorithms in atmospheric compensation for free space optical communication,” in Selected Papers from Conferences of the Photoelectronic Technology Committee of the Chinese Society of Astronautics, (2015).
8. Z. Li and X. Zhao, “Bp artificial neural network based wave front correction for sensor-less free space optics communication,” Opt. Commun. 385, 219–228 (2017). [CrossRef]
9. S. Lohani and R. T. Glasser, “Turbulence correction with artificial neural networks,” Opt. Lett. 43(11), 2611–2614 (2018). [CrossRef]
10. Y. Nishizaki, M. Valdivia, R. Horisaki, et al., “Deep learning wavefront sensing,” Opt. Express 27(1), 240 (2019). [CrossRef]
11. M. Wang, X. H. Yuan, and W. Guo, “Single-shot wavefront sensing with deep neural networks for free-space optical communications,” Opt. Express 29(3), 3465–3478 (2021). [CrossRef]
12. Y. Guo, L. Zhong, L. Min, et al., “Adaptive optics based on machine learning: a review,” Opto-Electron. Adv. 5(7), 200082 (2022). [CrossRef]
13. L. Ziqiang, L. Xinyang, G. Zeyu, et al., “Review of wavefront sensing technology in adaptive optics based on deep learning,” High Power Laser and Particle Beams 33, 081001 (2021). [CrossRef]
14. C. Lu, Q. Tian, L. Zhu, et al., “Mitigating the ambiguity problem in the cnn-based wavefront correction,” Opt. Lett. 47(13), 3251–3254 (2022). [CrossRef]
15. Y. Li, D. Yue, and Y. He, “Prediction of wavefront distortion for wavefront sensorless adaptive optics based on deep learning,” Appl. Opt. 61(14), 4168–4176 (2022). [CrossRef]
16. J. R. Fienup and C. C. Wackerman, “Phase-retrieval stagnation problems and solutions,” J. Opt. Soc. Am. A 3(11), 1897–1907 (1986). [CrossRef]
17. D. Li, S. Xu, D. Wang, et al., “Large-scale piston error detection technology for segmented optical mirrors via convolutional neural networks,” Opt. Lett. 44(5), 1170–1173 (2019). [CrossRef]
18. Q. Xin, G. Ju, C. Zhang, et al., “Object-independent image-based wavefront sensing approach using phase diversity images and deep learning,” Opt. Express 27(18), 26102–26119 (2019). [CrossRef]
19. X. Ma, Z. Xie, H. Ma, et al., “Piston sensing for sparse aperture systems with broadband extended objects via a single convolutional neural network,” Opt. Lasers Eng. 128, 106005 (2020). [CrossRef]
20. X. Ge, L. Zhu, Z. Gao, et al., “Target-independent dynamic wavefront sensing method based on distorted grating and deep learning,” Chin. Opt. Lett. 21(6), 060101 (2023). [CrossRef]
21. X. Ge, L. Zhu, Z. Gao, et al., “Object-independent wavefront sensing method based on an unsupervised learning model for overcoming aberrations in optical systems,” Opt. Lett. 48(17), 4476–4479 (2023). [CrossRef]
22. R. L. Kendrick, D. S. Acton, and A. Duncan, “Phase-diversity wave-front sensor for imaging systems,” Appl. Opt. 33(27), 6533–6546 (1994). [CrossRef]
23. R. A. Gonsalves, “Phase retrieval and diversity in adaptive optics,” Opt. Eng. 21(5), 829–832 (1982). [CrossRef]
24. R. G. Paxman, T. J. Schulz, R. Fienup, et al., “Joint estimation of object and aberrations by using phase diversity,” J. Opt. Soc. Am. A 9(7), 1072–1085 (1992). [CrossRef]
25. M. G. Löfdahl and G. B. Scharmer, “Wavefront sensing and image restoration from focused defocused solar images,” Astronomy and Astrophysics Supplement 107, 243–264 (1994).
26. X. Dai and H. Xie, “New methods of fabricating gratings for deformation measurements: A review,” Opt. Lasers Eng. 92, 48–56 (2017). [CrossRef]
27. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” Computer Science (2014).
28. C. Ledig, L. Theis, F. Huszar, et al., “Photo-realistic single image super-resolution using a generative adversarial network,” IEEE Computer Society (2016).
29. I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., “Generative adversarial nets,” in Neural Information Processing Systems, (2014).
30. H. J. Kim and D. Lee, “Image denoising with conditional generative adversarial networks (cgan) in low dose chest images,” Nucl. Instrum. Methods Phys. Res., Sect. A 954, 161914 (2019). [CrossRef]
31. Z. Yue, Q. Zhao, L. Zhang, et al., “Dual adversarial network: Toward real-world noise removal and noise generation,” in Computer Vision – ECCV 2020, (Springer International Publishing, Cham, 2020), pp. 41–58.
32. I. Freeman, L. Roese-Koerner, and A. Kummert, “Effnet: An efficient structure for convolutional neural networks,” IEEE (2018).
33. S. Woo, J. Park, J.-Y. Lee, et al., “Cbam: Convolutional block attention module,” in Computer Vision – ECCV 2018, V. Ferrari, M. Hebert, C. Sminchisescu, et al., eds. (Springer International Publishing, Cham, 2018), pp. 3–19.
34. A. Mittal, R. Soundarajan, A. C. Bovik, et al., Fellow, IEEE, “Making a ’completely blind’ image quality analyzer,” IEEE Signal Process. Lett. 20(3), 209–212 (2013). [CrossRef]
