Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Object-independent tilt detection for optical sparse aperture system with large-scale piston error via deep convolution neural network

Open Access Open Access

Abstract

The optical sparse aperture technique can improve the imaging resolution significantly under the ideal co-phase condition. However, the position deviation between different sub-apertures leads to notorious co-phase errors, seriously impacting the image quality. While the position deviation arises in practical applications, it is difficult to detect the errors in real-time for traditional iterative algorithms because of their narrow detection range and long-time iteration process. The deep neural network has shown its potential in optical information process, and it has some attempts in the detection of piston error. However, all existing deep learning-based methods just focus on the detection of piston error with the weak or corrected tilt error, which is not in line with reality. Here we implement the deep convolutional neural network to detect tilt error with large-scale piston error, and compare the detection performance of two kinds of network, one takes the point spread function as input while the other takes the phase diversity features as the input. The detection ability and generalization capability of network are verified under single wavelength, broadband light and turbulence aberration in simulation. The object-independent of tilt error detection ability is offered because the phase diversity features and point spread function are all unrelated to the object. In addition, the cyclic correction strategy is carried out to improve the generalization performance facing the larger errors. As a result, the deep learning-based method can detect the tilt error accurately with fast calculation, and the trained network is hopeful for the real-time correction with cyclic correction strategy.

© 2021 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

The optical synthetic aperture technique, which has developed rapidly for large aperture telescope manufacturing in recent years, can improve the resolution significantly under ideal co-phase condition. However, the position deviation between different sub-apertures leads to the exist of co-phase errors, impacting the quality of imaging seriously. Considerable effort has been taken to solve the misalignment of sub-apertures in optical method, such as Hartmann wavefront sensor [1], pyramid sensor [2], curvature sensor [3], Mach–Zehnder interferometer sensor [4] and Zernike phase contrast sensor [5]. The added special sensors can analyze the wavefront state in real time, but are expensive and inconvenient to detect the co-phase errors in practical applications. Several phase retrieval ways are also suitable for wavefront sensing, but the iterative process is really time consuming [68]. The phase diversity (PD) is an image-based wavefront sensing technique without additional sensors [910]. It utilizes the focused and defocused images with a known defocused phase to estimate the system aberrations. Combined with the PD theory, the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm is used for detecting the piston and tilt errors in the segmented mirror system [1112]. However, even the modified BFGS algorithm needs 500 steps iteration, and can only detect the [0, 0.5λ] range co-phase errors [12].

The rise of deep learning technology has shown its potential in hologram reconstruction [1314], scattering medium imaging [1516], phase information processing [1719] and image restoration [2021], etc. The trained deep neural network is suitable for the fast detection of co-phase errors. Guerra-Ramos et al. take the junction of three mirror segments as the feature to train the network [22]. Although the performance is excellent with [0, 11λ] detection range, the method doesn’t consider the exists of tilt error, and is not suitable for the sparse aperture system. Li et al. and Hui et al. both combine the PD theory with the convolutional neural networks for segmented mirrors, but Li just corrects the piston error from 10λ to λ under [−0.5, 0.5λ] tilt error, and Hui use the same method from λ to 0.06 λ supposing that the tilt error is corrected previously [2324]. Ma et al. propose two deep learning-based methods, which take the PD features or point spread function (PSF) as inputs, for the piston error detection of sparse aperture system [2526], but she doesn’t compare these two methods and analyze the characteristics. The above-mentioned methods all only focus on the detection of piston error, assuming that the tilt error is relatively weak or even effectively corrected. Moreover, the characteristics of PD-based and PSF-based network, and the generalization capability of error range are not discussed. In fact, the tilt error is always simultaneous with the piston error, and has a more serious impact on the image quality. Although the modified BFGS algorithm can detect piston and tilt errors meanwhile, its narrow detection range and long-time iteration, make itself difficult to apply in dynamic detection and real-time correction. Compared with the piston error, the tilt error involves more variations and needs more parameters to characterize, which may bring challenges for detection. To implement the fast co-phase errors detection of neural network, its performance and characteristics to tilt error detection need to be investigated urgently.

In this paper, we propose the deep learning-based tilt error detection methods, and compare the effect of three deep convolutional neural networks (DCNN), which use PD features and PSF as the inputs, respectively. At first, the optical sparse aperture imaging (OSAI) system is built for acquiring data. The two feature matrices by PD theory are obtained as the inputs of DCNN, as well as the PSF of system. Through simulation, the PSF-based and PD-based DCNNs both show their detection ability and generalization capability under single wavelength, broadband light and turbulence aberration. At last, all DCNNs can detect the tilt error of OSAI system with fast calculation, and are expected to the real-time correction with cyclic correction strategy in practical applications.

2. Principle

2.1 Optical imaging model

According to the linear system theory, the imaging process of synthetic aperture system is expressed as

$$g({x,y} )= f({x,y} )\ast \textrm{PSF}({x,y} ),$$
where f (x, y) is the ideal intensity distribution of objects, g (x, y) and PSF (x, y) are the observed image on the focal plane and the point spread function of system, * is the convolution operation and (x, y) is the coordinates of imaging plane. Considering the piston and tilt errors without the turbulence aberration, the pupil function of OSAI system is
$$\textrm{P}({\xi ,\eta } )= \sum\limits_{i = 1}^N {{\textrm{P}_{sub}}({\xi - {x_i},\eta - {y_i}} )\exp [{\textrm{j}{\varphi_i}({\xi ,\eta } )} ]} ,$$
$${\textrm{P}_{sub}}({\xi ,\eta } )= \textrm{circ}\left( {\frac{{\sqrt {{\xi^2} + {\eta^2}} }}{r}} \right) = \left\{ {\begin{array}{cc} {1,}&{\sqrt {{\xi^2} + {\eta^2}} \le r}\\ {0,}&{other} \end{array}} \right.,$$
$${\varphi _i}({\xi ,\eta } )= \frac{{2\mathrm{\pi }}}{\lambda }({{\varphi_{p - i}} + {\varphi_{t - i}}} ),$$
where Psub (ξ, η) is the ideal pupil function of one sub-aperture, circ (·) is the circular function, r is the radius of one sub-aperture, (ξ, η) is the coordinates of the pupil plane, (xi, yi) and φi (ξ, η) are the center coordinate and wave aberration phase of i-th sub-aperture respectively, φp-i and φt-i are the corresponding phase to the piston and tilt errors, and λ is the central wavelength. As shown in Fig. 1, the piston phase φp-i is controlled by the translation distance Δz and the tilt phase φtx-i and φty-i are introduced by tilt angles Δθx and Δθy from actual plane to reference plane, respectively. The PSF can be obtained by
$$\textrm{PSF}({x,y} )\textrm{ = }{|{\textrm{FT}\{{\textrm{P}({\xi ,\eta } )} \}} |^\textrm{2}},$$
$$\textrm{MTF}({u,v} )\textrm{ = }\left|{\frac{{\textrm{FT}\{{\textrm{PSF}({x\textrm{,}y} )} \}}}{{\int\!\!\!\int {\textrm{PSF}({x,y} )dxdy} }}} \right|,$$
where MTF (u, v) is the modulation transfer function (MTF) and FT{·} is the Fourier transform operation. Therefore, PSF and MTF are both features which can effectively reflect the imaging characteristics of system under different co-phase errors.

 figure: Fig. 1.

Fig. 1. (a) Piston error introduced by the distance Δz. (b) Tilt error introduced by the angle Δθx and Δθy.

Download Full Size | PDF

2.2 Feature matrices

As a data-driven method, the neural network can establish the underlying relationship between inputs and labels from a large number of samples through learning. It means that the dataset should satisfy both of the following characteristics: 1) There is a strict one-to-one correspondence between the inputs and labels, while the labels are different co-phase errors here; 2) The data samples are easy to obtain relatively. The PSF is an ideal feature as the input because it can reflect the imaging effect accurately. PD is an adaptive optics technology to infer the phase aberrations from the intensity images. The PD method requires two images, one in the focal plane and the other in the defocus plane. The defocused image can be captured in similar way

$${g_d}({x,y} )= f({x,y} )\ast \textrm{PS}{\textrm{F}_d}({x,y} ),$$
$$\textrm{PS}{\textrm{F}_d}({x,y} )\textrm{ = }{|{\textrm{FT}\{{{\textrm{P}_d}({\xi ,\eta } )} \}} |^\textrm{2}},$$
$${\textrm{P}_d}({\xi ,\eta } )= \sum\limits_{i = 1}^N {{\textrm{P}_{sub}}({\xi - {x_i},\eta - {y_i}} )\exp [{\textrm{j}({{\varphi_i} + \Delta {\varphi_d}} )} ]} ,$$
$$\Delta {\varphi _d} = 2\pi \Delta \omega \cdot {\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}} \over Z} _4},$$
$$\Delta \omega = \frac{{\Delta d}}{{8\lambda {{({{F_\# }} )}^2}}} = \frac{{\Delta d}}{{8\lambda {{({{f / {2R}}} )}^2}}},$$
where gd (x, y), PSFd (x, y), Pd (ξ, η) are the image, point spread function and pupil function of the defocused system, respectively, Δφd is the known defocus phase which can be represented by the number of waves of quadratic diversity Δω and the normalized defocus amount in the Zernike polynomial ${\hat{Z}_4}$, Δd is the defocused length, F# is the F number of the system, f is the effective focal length and R is the radius of the primary aperture [2728]. The sharpness matrix and the power matrix can be calculated by the focused and defocused images
$${M_{sharpness}} = \frac{{G({u,v} )\cdot G_d^ \ast ({u,v} )- {G^ \ast }({u,v} )\cdot {G_d}({u,v} )}}{{G({u,v} )\cdot {G^ \ast }({u,v} )+ {G_d}({u,v} )\cdot G_d^ \ast ({u,v} )}},$$
$${M_{power}} = \frac{{G({u,v} )\cdot {G^ \ast }({u,v} )- {G_d}({u,v} )\cdot G_d^ \ast ({u,v} )}}{{G({u,v} )\cdot {G^ \ast }({u,v} )+ {G_d}({u,v} )\cdot G_d^ \ast ({u,v} )}},$$
where Msharpness and Mpower are the sharpness matrix and the power matrix, G (u, v) and Gd (u, v) are the Fourier transform of the focused and defocused images, and G* (u, v) and Gd* (u, v) are their complex conjugates. Compared with the sharpness matrix, the power matrix is insensitive to the small registration error between two images [9]. It is worth emphasizing that the sharpness matrix and the power matrix are both unrelated to the imaging content, which is the same as the PSF. Taking these features as the inputs will empower the network with the object-independent tilt detection ability. To generate the simulation data, the “Golay-3” optical sparse aperture imaging system is chosen. The parameters are shown in Table 1, and the Δω = 1 is set for defocusing [29].

Tables Icon

Table 1. Parameters of the “Golay-3” optical sparse aperture imaging system

To calculate the relative error of the other two sub-apertures, the sub-aperture 2 is regarded as the standard aperture in Fig. 2(a). The piston and tilt errors are added into sub-aperture 1 for analyzing characteristics. From Fig. 2(b), the energy distribution of the PSF expands around and the shape of MTF also changes with the increase of piston error until π. That means the piston error causes the shift of interference position, lead to the frequency transfer response attenuation. The change reaches its maximum at π, and then decays to original shape until 2π. The focused and defocused images are almost the same in imaging process. It can be concluded that the imaging effect of piston error has the period of 2π, and it is relatively weak to imaging effect at a single wavelength. The sharpness matrix and the power matrix also keep the periodic change, which proves their consistency with the piston error. However, the period does not exist in tilt error from Fig. 3, and the energy center of PSF is not displaced. The influence to imaging effect becomes more serious with the increase of tilt angle, due to the continuous cluttered distribution of PSF and MTF. The sharpness matrix and the power matrix still keep their consistency with the tilt error. As for the combination of piston error and tilt error in Fig. 4, the influence of piston error is almost covered by that of tilt error, which indicates that the tilt error should be controlled in a small range before correcting the piston error. A common advantage of PD features and PSF is offered as the object independent, which means that two feature matrices and PSF only depend on the co-phase errors of system, rather than the content of extended objects. In conclusion, the feature matrices of PD are corresponding to the co-phase errors in OSAI system, which is the same as PSF and MTF.

 figure: Fig. 2.

Fig. 2. (a) Extended object and the “Golay-3” sparse aperture arrangement. (b) Comparison for the results with different piston errors in sub-aperture 1.

Download Full Size | PDF

 figure: Fig. 3.

Fig. 3. Comparison for the results with different tilt errors in sub-aperture 1.

Download Full Size | PDF

 figure: Fig. 4.

Fig. 4. Comparison for the results with piston and tilt errors in sub-aperture 1.

Download Full Size | PDF

We can use the PSF, the sharpness and power matrices as the features to extract the corresponding co-phase errors in “Golay-3” OSAI system. The neural network has shown its potential in extracting piston error [2326]. However, the tilt error will introduce more changes compared with the piston error, leading to the more complex features. To achieve the deep learning-based co-phase errors sensing, the tilt error detection should be taken into consideration first.

3. Method

The deep convolutional neural network is a widely-used classic backbone [30]. Figure 5 denotes that the cascaded convolution layers and pooling layers extract and compress the features from inputs, and then the features are reshaped into one-dimensional vectors with the fully connected layers (FC). The convolution layers contain the convolution operation (Conv) with 3×3 kernel, the batch normalization (BN), and the leaky rectified linear unit (LRelu) function. The pooling layers are the max pooling operation with 2×2 kernel. The number in the lower left of features is channel × size. The mean absolute error (MAE) is calculated to back propagate and update parameters of DCNN. At last, the output is a one-dimensional vector with N values. In fact, N can vary while extracting the tilt error of one sub-aperture or all sub-apertures with DCNN. In this paper, N is set as 2 because extracting tilt error of any one sub-aperture is the basis for detecting that of all sub-apertures.

 figure: Fig. 5.

Fig. 5. Structure of the deep convolutional neural network.

Download Full Size | PDF

The imaging process is simulated through the optical imaging model. To verify the independence of imaging objects, 10000 and 2000 optical remote sensing images from DIOR dataset [31] are chosen for training and testing. In Fig. 6, the PSF, the focused and defocused images are obtained by the OSAI system. The power and sharpness matrices can be calculated by the two images. Then we respectively set the power matrices Mpower, the sharpness matrices Msharpness, and the PSF as inputs, and regard the tilt angles Δθx and Δθy of one sub-aperture as labels to construct the dataset. The networks trained by Mpower, Msharpness and PSF are named as DCNN-Mp and DCNN-Ms, DCNN-P, respectively. The relative piston and tilt errors are both added into sub-aperture 1 and 3 with the limit of [-X1, X1] µm and [-X2, X2] µrad in Fig. 6, and the specific Δz, Δθx and Δθy are all generated independently by random numbers. From Fig. 6, the piston and tilt error limited [−3, 3] µm and [−3, 3] µrad are equal to [−5λ, 5λ] and [−2λ, 2λ] under 600 nm wavelength in train set. The time consumption is ∼5.5 h using 10000 images, which is divided into 4:1 for train and validation. The test process takes ∼12 s for 2000 images with the Intel Core Processor CPU (2.5 GHz) and NVIDIA GeForce RTX 2080Ti GPU.

 figure: Fig. 6.

Fig. 6. Cyclic correction process of tilt error using DCNN.

Download Full Size | PDF

4. Simulation results

4.1 Tilt error detection at a single wavelength

The detection ability for tilt error is taken into consideration first. The dataset, containing one train set and four test sets, is obtained at a single wavelength. Three DCNNs are trained with the different train sets to extract the tilt error of sub-aperture 1. The loss curves during training are shown in Fig. 7. In comparison, The DCNN-P converges faster with less oscillation and lower values than the PD-based DCNNs. The detection performance of DCNNs in four test sets is shown as data distribution in Fig. 8. The original tilt error along X and Y axis increase from Test1 to Test4 sets in Fig. 8(a). All three DCNNs can perform well in Test1 and Test2 because of the similar data distribution in Figs. 8(b-d). The Test3 and Test4 sets with a larger error range than train set are used to verify the generalization capability. The DCNN-P performances better than DCNN-Mp and DCNN-Ms in Test3 and Test4, although all DCNNs have declining effects. Figure 9 shows the corresponding percentages of phase distribution in four test sets. From Figs. 9(a-b), the tilt phase φtx and φty are almost corrected into [0, λ/10] with three DCNNs in Test1 and Test2. However, the percentages of [0, λ/10] in Test3 and Test4 are below 60% and 30%, which is higher than original distribution but lower than expected level. It is noteworthy that the percentage of [0, λ] in Test3 and Test4 also increase, which indicates the overall tilt error reduces after correction in Figs. 9(c-d). It is feasible to use cyclic correction instead of one-step correction theoretically, thanks to the high speed of trained DCNN in test process.

 figure: Fig. 7.

Fig. 7. Loss curves of three DCNNs in train and valid process.

Download Full Size | PDF

 figure: Fig. 8.

Fig. 8. Detection performance of three DCNNs at a single wavelength. (a) Original data distribution of four test sets. (b-d) Corrected data distribution of four test sets with DCNN-P, DCNN-Mp and DCNN-Ms.

Download Full Size | PDF

 figure: Fig. 9.

Fig. 9. Comparison of phase distribution with three DCNNs at a single wavelength. (a) Test1. (b) Test2. (c) Test3. (d) Test4.

Download Full Size | PDF

4.2 Tilt error detection under broadband light

The single wavelength is unsuitable for the practical applications, so the broadband light is taken into research. The PSF of system under the broadband light can be obtained as

$$\textrm{PSF}({x,y} )= \int_{{\lambda _1}}^{{\lambda _N}} {c(\lambda )\textrm{PSF}({x,y,\lambda } )} d\lambda ,$$
where, PSF (x, y, λ) represents the PSF of system at a single wavelength, and c (λ) represents the weight factor. Here we adopt the wavelengths of [500, 700] nm as the broadband light with 10 nm bandwidth interval, and other parameters are all the same as before. In imaging process, the PSF is obtained by the integration of 21 wavelengths, supposing the uniform wavelength distribution c (λ) = 1. From Fig. 10, we can see that the focused and defocused images are more blurred. The corresponding PSF, MTF, Mpower and Msharpness all have slight changes because the same error results in the different phase at different wavelengths. After constructing the broadband light dataset, three DCNNs are all retrained.

 figure: Fig. 10.

Fig. 10. Comparison of the imaging results under single wavelength and broadband light, in which Δz = 2.369 µm and Δθ = 2.89 µrad in sub-aperture 1.

Download Full Size | PDF

The similar detection performance and phase distribution are both shown in Fig. 11 and Fig. 12. Compared with those in Fig. 8, there are slight decreases in the detection effect of three DCNNs in Fig. 11, due to the broadband light. In general, all DCNNs still can correct the phase error into [0, λ/10] in Test1 and Test2, and reduce the overall errors in Test3 and Test4 in Fig. 12. However, the DCNN-P keeps the same while the DCNN-Mp and DCNN-Ms decrease in Test1 and Test2 from Figs. 12(a-b). And the percentages of [0, λ/10] in Test3 and Test4 from Figs. 12(c-d) are close to 50% and 20%, which are lower than those in Figs. 9(c-d). All in all, the detection ability of DCNNs receive impact due to the broadband light, while the decreases of DCNN-Mp and DCNN-Ms are larger. In following discussion, DCNN-P and DCNN-Mp are used for comparison without DCNN-Ms, because Mpower and Msharpness are obtained from the same two images and have similar properties and change trends.

 figure: Fig. 11.

Fig. 11. Detection performance of three DCNNs under broadband light. (a) Original data distribution of four test sets. (b-d) Corrected data distribution of four test sets with DCNN-P, DCNN-Mp and DCNN-Ms.

Download Full Size | PDF

 figure: Fig. 12.

Fig. 12. Comparison of phase distribution with three DCNNs under broadband light. (a) Test1. (b) Test2. (c) Test3. (d) Test4.

Download Full Size | PDF

4.3 Tilt error detection considering turbulence aberration

The turbulence aberration is an inevitable problem in remote target imaging. With the additional phase introduced by turbulence aberration and the position deviation of sub-apertures, the real-time correction of tilt error will become important and difficult in applications. The turbulence aberration limited to 5λ is added into the simulation process under broadband light to explore its impact. The turbulence phase is constructed by Zernike polynomial obtained from the Hartmann wavefront sensor in the turbulence generation pool. In simulation, we randomly select the turbulence phase loading from the collected data to increase the randomness, because the experimental acquisition speed is only tens of Hz. The influence of turbulence aberration is clear from the changes in Mpower and PSF in Fig. 13(a). Thanks to the fast calculation of trained DCNN, we can use cyclic correction here for the real-time correction of “Golay-3” OSAI system. Four DCNNs, including DCNN-P and DCNN-Mp for the tilt error detection of sub-aperture 1 and 3, are all retrained with the turbulence aberration dataset respectively.

 figure: Fig. 13.

Fig. 13. (a) Comparison of Mpower and PSF with and without turbulence. (b) Tilt error correction effect with DCNN-P and DCNN-Mp.

Download Full Size | PDF

Figure 13(b) shows the effect after one-time tilt error correction in Test2. The impact of tilt error in PSF, Mpower, focused and defocused images are controlled effectively, and Δθx and Δθy of sub-aperture 1 and 3 are reduced to a small value in Fig. 13(b). There is the comparison of phase distribution belong to [0, λ/10] after cyclic correction in Test2, Test3 and Test4 in Fig. 14. It is worth mentioning that the one-time correction effect in Test3 and Test4 is bad because of the larger tilt error range, which is proved in Section 4.1. The cyclic correction with N times (C-N) is taken into consideration and has good performance in Figs. 14(b-c). The correction with three and five times in Test3 and Test4 only take ∼20 ms and ∼33 ms in all. DCNN-P can both achieve over 90% accuracy in Test3 and Test4, while DCNN-Mp can only get close to 80% in Test3 and 60% accuracy in Test4. In fact, Mpower is more complex and changeable than PSF, resulting in the mistaken detection during the generalization process. Some large tilt errors are corrected to smaller gradually, while the mistakes may accumulate resulting in the correction failure of others. To solve this problem, the train set containing a larger range of tilt error may be used to let DCNN have better generalization capability, and the constraints may be added to limit the output value in cyclic correction.

 figure: Fig. 14.

Fig. 14. Comparison of phase distribution belong to [0, λ/10] after cyclic correction with DCNN-P and DCNN-Mp under turbulence aberration. (a) Test2. (b) Test3. (c) Test4.

Download Full Size | PDF

5. Analysis and discussions

The research focuses on the fast correction of tilt error in “Golay-3” OSAI system in the last section, and detection ability and generalization capability of DCNNs is verified. To further prove the performance of DCNN-P and DCNN-Mp, different arrangements and number of sub-apertures are considered. The “Annular-4”, “Annular-5”, “Annular-6”, “Goaly-6” and “Y-6” OSAI system are simulated with the same main parameters. Several DCNNs with corresponding datasets of X1=X2=3 are taken to train and test.

From Table 2, DCNN-P can keep the stable performance in detecting tilt error of one sub-aperture in all OSAI systems, while DCNN-Mp declines seriously. To analysis the reasons behind this, more data about DCNN-Mp is shown in Fig. 15. The shape of Mpower is really similar to the MTF of system in Fig. 15(a). Different number of sub-apertures and arrangements corresponding to different shapes of MTF and Mpower. Figures 15(b) and (c) show the loss curves of DCNN-Mp in different number of sub-apertures with “Annular” arrangement and in different arrangements with 6 sub-apertures, respectively. Compared with the loss curves in Fig. 7, these loss curves are much higher and even not converge. The features in Mpower are combined more complex due to the increase of sub-apertures, but the training set and network parameters keep the same capacity. In Figs. 15(b-c), DCNN-Mp of “Annular-6” gets a relatively better convergence effect than others in valid loss, while all train loss curves have similar convergence level. It shows that all DCNNs have overfitting phenomenon and can’t get a good generalization effect in test sets. From their corresponding Mpower, the shape of “Annular-6” is the most uniform while others are scattered or crowded. Some features with tilt error information may be covered or hidden, resulting in the network can’t demodulate smoothly. Deepening the network and increasing the amount of data can help solving the overfitting problem effectively, but this will also increase the time costs. As a result, the number of sub-apertures and the arrangements will lead to the shape changes of Mpower, and then affect the detection of tilt error using DCNN-Mp. By contrast, DCNN-P has strong detection ability, generalization capability and robustness.

 figure: Fig. 15.

Fig. 15. (a) Different aperture arrangements and corresponding MTF, Mpower and Mpower under error. (b) Loss curves of DCNNs in different number of sub-apertures with “Annular” arrangement. (c) Loss curves of DCNNs in different arrangements with 6 sub-apertures.

Download Full Size | PDF

Tables Icon

Table 2. Correction effect of DCNN-P and DCNN-Mp in different apertures

6. Conclusions

In conclusion, the deep learning-based method can extract the tilt error of OSAI system under single wavelength, broadband light and turbulence aberration, and once detection takes less than 10 ms, which is suitable for the real-time correction. A common advantage of PD-based and PSF-based networks is offered as the object independent, which greatly increase the generalization capability of the trained DCNN on the object diversity. As for the generalization problem with the large error, cyclic correction can be an effective solution to improve performance. The PD-based DCNN have advantages in applications because it only uses focused and defocused images to form the inputs of network, but it may be affected by the apertures. By comparison, the DCNN-P shows stronger detection ability, generalization capability and robustness than DCNN-Mp. In following research, the actual effect of DCNN needs closed-loop correction experiment to prove, and how to correct the piston and tilt errors meanwhile through network is still a problem for research.

Funding

National Natural Science Foundation of China (61905197, 61927810, 62075183); State Key Laboratory of Transient Optics and Photonics (SKLST202008).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. G. A. Chanan, Design of the Keck Observatory alignment camera, in Precision Instrument Design, vol. 1036 (International Society for Optics and Photonics, 1989), p. 59–71.

2. S. Esposito, E. Pinna, A. Puglisi, A. Tozzi, and P. Stefanini, “Pyramid sensor for segmented mirror alignment,” Opt. Lett. 30(19), 2572–2574 (2005). [CrossRef]  

3. V. G. Orlov, S Cuevas, and F. Garfias, Co-phasing of segmented mirror telescopes with curvature sensing, in Telescope Structures, Enclosures, Controls, Assembly/Integration/Validation, and Commissioning, vol. 4004 (International Society for Optics and Photonics, 2000), p. 540–551.

4. N. Yaitskova, K. Dohlen, and P. Dierickx, “Mach–Zehnder interferometer for piston and tip–tilt sensing in segmented telescopes: theory and analytical treatment,” J. Opt. Soc. Am. A 22(6), 1093–1105 (2005). [CrossRef]  

5. I. Surdej, N. Yaitskova, and F. Gonte, “On-sky performance of the Zernike phase contrast sensor for the phasing of segmented telescopes,” Appl. Opt. 49(21), 4052–4062 (2010). [CrossRef]  

6. R. E. Carlisle and D. S. Acton, “Demonstration of extended capture range for James Webb Space Telescope phase retrieval,” Appl. Opt. 54(21), 6454–6460 (2015). [CrossRef]  

7. P. Bao, F. Zhang, G. Pedrini, and W. Osten, “Phase retrieval using multiple illumination wavelengths,” Opt. Lett. 33(4), 309–311 (2008). [CrossRef]  

8. M. Guizar-Sicairos and J. R. Fienup, “Phase retrieval with transverse translation diversity: a nonlinear optimization approach,” Opt. Express 16(10), 7264–7278 (2008). [CrossRef]  

9. R. L. Kendrick, D. S. Acton, and A. L. Duncan, “Phase-diversity wave-front sensor for imaging systems,” Appl. Opt. 33(27), 6533–6546 (1994). [CrossRef]  

10. M. G. Lofdahl, R. L. Kendrick, and A. Harwit, Phase diversity experiment to measure piston misalignment on the segmented primary mirror of the Keck II telescope, in Space Telescopes and Instruments V, vol. 3356 (International Society for Optics and Photonics, 1998), p. 1190–1201.

11. D. H. Li and M. Fukushima, “On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,” SIAM J. Optim. 11(4), 1054–1064 (2001). [CrossRef]  

12. D. Yue, S. Xu, and H. Nie, “Co-phasing of the segmented mirror and image retrieval based on phase diversity using a modified algorithm,” Appl. Opt. 54(26), 7917–7924 (2015). [CrossRef]  

13. J. Wu, K. Liu, X. Sui, and L. Cao, “High-speed computer-generated holography using an autoencoder-based deep neural network,” Opt. Lett. 46(12), 2908–2911 (2021). [CrossRef]  

14. Z. Ren, Z. Xu, and E. Y. Lam, “End-to-end deep learning framework for digital holographic reconstruction,” Adv. Photonics 1(1), 016004 (2019). [CrossRef]  

15. E. Guo, S. Zhu, Y. Sun, L. Bai, C. Zuo, and J. Han, “Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect,” Opt. Express 28(2), 2433–2446 (2020). [CrossRef]  

16. M. Lyu, H. Wang, G. Li, S. Zheng, and G Situ, “Learning-based lensless imaging through optically thick scattering media,” Adv. Photonics 1(3), 036002 (2019). [CrossRef]  

17. K. Wang, Y. Li, Q. Kemao, J. Di, and J. Zhao, “One-step robust deep learning phase unwrapping,” Opt. Express 27(10), 15100–15115 (2019). [CrossRef]  

18. K. Wang, M. Zhang, J. Tang, L. Wang, L. Hu, X. Wu, W. Li, J. Di, G. Liu, and J. Zhao, “Deep learning wavefront sensing and aberration correction in atmospheric turbulence,” PhotoniX 2(1), 1–11 (2021). [CrossRef]  

19. K. Yan, Y. Yu, T. Sun, A. Asundi, and Q. Kemao, “Wrapped phase denoising using convolutional neural networks,” Opt Lasers Eng 128, 105999 (2020). [CrossRef]  

20. J. Tang, K. Wang, Z. Ren, W. Zhang, X. Wu, J. Di, G. Liu, and J. Zhao, “RestoreNet: a deep learning framework for image restoration in optical synthetic aperture imaging system,” Opt Lasers Eng 139, 106463 (2021). [CrossRef]  

21. J. Tang, J. Wu, K. Wang, Z. Ren, X. Wu, L. Hu, J. Di, G. Liu, and J. Zhao, “RestoreNet-Plus: Image restoration via deep learning in optical synthetic aperture imaging system,” Opt Lasers Eng 146, 106707 (2021). [CrossRef]  

22. D. Guerra-Ramos, L. Díaz-García, and J. Trujillo-Sevilla, “Piston alignment of segmented optical mirrors via convolutional neural networks,” Opt. Lett. 43(17), 4264–4267 (2018). [CrossRef]  

23. D. Li, S. Xu, D. Wang, and D. Yan, “Large-scale piston error detection technology for segmented optical mirrors via convolutional neural networks,” Opt. Lett. 44(5), 1170–1173 (2019). [CrossRef]  

24. M. Hui, W. Li, M. Liu, L. Dong, L. Kong, and Y. Zhao, “Object-independent piston diagnosing approach for segmented optical mirrors via deep convolutional neural network,” Appl. Opt. 59(3), 771–778 (2020). [CrossRef]  

25. X. Ma, Z. Xie, H. Ma, Y. Xu, D. He, and G. Ren, “Piston sensing for sparse aperture systems with broadband extended objects via a single convolutional neural network,” Opt Lasers Eng 128, 106005 (2020). [CrossRef]  

26. X. Ma, Z. Xie, H. Ma, Y. Xu, G. Ren, and Y. Liu, “Piston sensing of sparse aperture systems with a single broadband image via deep learning,” Opt. Express 27(11), 16058–16070 (2019). [CrossRef]  

27. R. G. Paxman and J. R. Fienup, “Optical misalignment sensing and image reconstruction using phase diversity,” J. Opt. Soc. Am. A 5(6), 914–923 (1988). [CrossRef]  

28. F. Cao, Y. Zhu, and Z. Wu, “Optical misalignment sensing for optical aperture synthesis telescope by using phase diversity,” in Adaptive Optics Systems vol. 7015 (International Society for Optics and Photonics, 2008), p. 70154Q.

29. J. R. Fienup, B. J. Thelen, R. G. Paxman, and D. A. Carrara, “Comparison of phase diversity and curvature wavefront sensing,” in Adaptive Optical System Technologies vol. 3353 (International Society for Optics and Photonics, 1998), p. 930–940.

30. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” vol. 25 (Advances in neural information processing systems, 2012), p. 1097–1105.

31. K. Li, G. Wan, G. Cheng, L. Meng, and J. Han, “Object detection in optical remote sensing images: A survey and a new benchmark,” ISPRS J. Photogramm 159, 296–307 (2020). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1.
Fig. 1. (a) Piston error introduced by the distance Δz. (b) Tilt error introduced by the angle Δθx and Δθy.
Fig. 2.
Fig. 2. (a) Extended object and the “Golay-3” sparse aperture arrangement. (b) Comparison for the results with different piston errors in sub-aperture 1.
Fig. 3.
Fig. 3. Comparison for the results with different tilt errors in sub-aperture 1.
Fig. 4.
Fig. 4. Comparison for the results with piston and tilt errors in sub-aperture 1.
Fig. 5.
Fig. 5. Structure of the deep convolutional neural network.
Fig. 6.
Fig. 6. Cyclic correction process of tilt error using DCNN.
Fig. 7.
Fig. 7. Loss curves of three DCNNs in train and valid process.
Fig. 8.
Fig. 8. Detection performance of three DCNNs at a single wavelength. (a) Original data distribution of four test sets. (b-d) Corrected data distribution of four test sets with DCNN-P, DCNN-Mp and DCNN-Ms.
Fig. 9.
Fig. 9. Comparison of phase distribution with three DCNNs at a single wavelength. (a) Test1. (b) Test2. (c) Test3. (d) Test4.
Fig. 10.
Fig. 10. Comparison of the imaging results under single wavelength and broadband light, in which Δz = 2.369 µm and Δθ = 2.89 µrad in sub-aperture 1.
Fig. 11.
Fig. 11. Detection performance of three DCNNs under broadband light. (a) Original data distribution of four test sets. (b-d) Corrected data distribution of four test sets with DCNN-P, DCNN-Mp and DCNN-Ms.
Fig. 12.
Fig. 12. Comparison of phase distribution with three DCNNs under broadband light. (a) Test1. (b) Test2. (c) Test3. (d) Test4.
Fig. 13.
Fig. 13. (a) Comparison of Mpower and PSF with and without turbulence. (b) Tilt error correction effect with DCNN-P and DCNN-Mp.
Fig. 14.
Fig. 14. Comparison of phase distribution belong to [0, λ/10] after cyclic correction with DCNN-P and DCNN-Mp under turbulence aberration. (a) Test2. (b) Test3. (c) Test4.
Fig. 15.
Fig. 15. (a) Different aperture arrangements and corresponding MTF, Mpower and Mpower under error. (b) Loss curves of DCNNs in different number of sub-apertures with “Annular” arrangement. (c) Loss curves of DCNNs in different arrangements with 6 sub-apertures.

Tables (2)

Tables Icon

Table 1. Parameters of the “Golay-3” optical sparse aperture imaging system

Tables Icon

Table 2. Correction effect of DCNN-P and DCNN-Mp in different apertures

Equations (14)

Equations on this page are rendered with MathJax. Learn more.

g ( x , y ) = f ( x , y ) PSF ( x , y ) ,
P ( ξ , η ) = i = 1 N P s u b ( ξ x i , η y i ) exp [ j φ i ( ξ , η ) ] ,
P s u b ( ξ , η ) = circ ( ξ 2 + η 2 r ) = { 1 , ξ 2 + η 2 r 0 , o t h e r ,
φ i ( ξ , η ) = 2 π λ ( φ p i + φ t i ) ,
PSF ( x , y )  =  | FT { P ( ξ , η ) } | 2 ,
MTF ( u , v )  =  | FT { PSF ( x , y ) } PSF ( x , y ) d x d y | ,
g d ( x , y ) = f ( x , y ) PS F d ( x , y ) ,
PS F d ( x , y )  =  | FT { P d ( ξ , η ) } | 2 ,
P d ( ξ , η ) = i = 1 N P s u b ( ξ x i , η y i ) exp [ j ( φ i + Δ φ d ) ] ,
Δ φ d = 2 π Δ ω Z 4 ,
Δ ω = Δ d 8 λ ( F # ) 2 = Δ d 8 λ ( f / 2 R ) 2 ,
M s h a r p n e s s = G ( u , v ) G d ( u , v ) G ( u , v ) G d ( u , v ) G ( u , v ) G ( u , v ) + G d ( u , v ) G d ( u , v ) ,
M p o w e r = G ( u , v ) G ( u , v ) G d ( u , v ) G d ( u , v ) G ( u , v ) G ( u , v ) + G d ( u , v ) G d ( u , v ) ,
PSF ( x , y ) = λ 1 λ N c ( λ ) PSF ( x , y , λ ) d λ ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.