Open Access
Add to BibSonomy
Abstract
Spectral interferometry is utilized in a wide range of biomedical and scientific applications and metrology. Retrieving the magnitude and phase of the complex electric field from the interferogram is central to all its applications. We report a spectral interferometry system that utilizes a neural network to infer the magnitude and phase of femtosecond interferograms directly from the measured single-shot interference patterns and compare its performance with the widely used Hilbert transform. Our approach does not require apriori knowledge of the shear frequency, and achieves higher accuracy under our experimental conditions. To train the network, we introduce an experimental technique that generates a large number of femtosecond interferograms with known (labeled) phase and magnitude profiles. While the profiles for these pulses are digitally generated, they obey causality by satisfying the Kramer-Kronig relation. This technique is resilient against nonlinear optical distortions, quantization noise, and the sampling rate limit of the backend digitizer – valuable properties that relax instrument complexity and cost.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Spectral interferometry is a powerful technique for the characterization of femtosecond optical pulses and the classification of materials or biological cells [1–3]. In the biomedical domain, it is the basis of ultrafast implementations of Fourier domain optical coherence tomography (OCT) [4–6] and its time stretch variant has led to phase contrast microscopes with extreme throughput for label-free classification of blood cells [7]. The same method led to the discovery of previously unobserved single-shot ultrafast phenomena such as the internal motion of soliton molecules [8]. Spectral interferometry has emerged as the workhorse of single-shot photon Doppler velocimetry for the study of single-shot extreme phenomena such as combustion or explosions [9–11]. Combined with nonlinear frequency generation spectral interferometry is the basis of a powerful method for laser pulse characterizations, known as SPIDER [12]. A method related to SPIDER uses spectral interferometry for pulse characterization without requiring nonlinear optical effects [13] albeit with compromised phase measurement sensitivity.
In spectral interferometry, the magnitude and phase information of the optical electric field is encoded onto spectral fringes produced by the interference of a pulse with a delayed copy of itself. The most successful algorithms for retrieval of magnitude and phase are the Fourier transform [14–20] and closely related Hilbert transform [7,21]. Being deterministic, as opposed to data driven, these algorithms are challenged in situations where the signal is distorted by nonideal effects in the experimental setup. Such can be the case when time stretch data acquisition is used for single shot and high throughput measurements. As will be shown, the neural network approach is better able to recognize and distinguish the true spectral phase and magnitude from distortions induced by system components including optical nonlinearities in the dispersive fiber used to implement time stretch. At the same time, the neural network approach will fail under conditions outside of those present in the training data. For example, when the input laser spectrum or the group delay dispersion change significantly, the network must be retrained with new data.
Recently, convolutional neural network based-fringe pattern analysis in the spatial fringe projection techniques has been demonstrated [22] and the spectral interferogram is composed of many fringes. Thus, combining spectral interferometry with neural network-based regression, and a novel method for generating labeled and causal femtosecond interferograms for training, we demonstrate high-performance magnitude and phase retrieval in which a neural network solves for the complex electric field. The neural network is trained and tested using 6000 experimentally measured spectral interferometry spectra each corresponding to a randomly generated but causal complex phase and magnitude profiles. The interferometer output is captured using both a traditional grating-based spectrometer and the time stretch spectrometer for single-shot operation.
Our neural network-based spectral interferometry approach outperforms the Hilbert transform under both time-averaged and single-shot conditions. We also show that this approach is more robust against quantization noise of the digitizer, a major advantage in single-shot systems where the resolution of the analog to digital converter becomes the bottleneck.
Spectral interferometry with femtosecond pulses is the basis for the time stretch phase-contrast microscopy [21] that has achieved record accuracy in the classification of cancer cells in blood [7,23]. In those papers, the neural network was performing a simple binary classification. In the present work, the neural network is performing the far more difficult task of regression into the equivalent of six thousand classes (vs. 2 classes in prior arts [7,23]). Therefore, it extracts orders of magnitude more information from the interferometry. We believe our technique can potentially become the standard approach to spectral interferometry.
Neural networks have been successful in solving a wide range of inverse problems. In the context of phase retrieval, they have been applied to ultrashort pulse reconstruction using frequency resolved optical gating (FROG) [24,25] and the analysis of modulation instability in an optical fiber [26].
2. System description
2.1 System block diagram
The experimental system shown in Fig. 1 consists of a broadband pulsed laser source, an interferometer, spectrum acquisition and digital signal processing where both the neural network and the Hilbert transform are implemented. A broadband pulsed laser is split into two arms of a Mach-Zehnder interferometer. The pulse in the upper arm is the signal and the lower arm serves as the reference which undergoes a delay that determines the interferometric shear frequency. The signal and reference meet in a coupler generating spectral interference. The magnitude and phase spectrum imposed by the spectral modulator are encoded into the fringe pattern. The interference pattern can be measured using either a spatial readout with a grating spectrometer, or by temporal readout using a time stretch spectrometer [27]. The latter maps the spectrum into a temporal signal with a time scale that is slow enough for signal acquisition with a real-time digitizer. To achieve the maximum time stretch and highest signal to noise ratio (SNR), a Raman-pumped amplified time stretch [27–29] module was used. Thus, after photodetection, the spectral interferogram can be digitized with real-time data acquisition equipment.
Fig. 1. The experimental testbed consists of a broadband pulsed laser, an interferometer, readout using either time stretch or a grating-based spectrometer, analog to digital converter (data acquisition) followed by a neural network. The experiments used a spectral modulator in the sample arm of the interferometer to emulate a sample under test, and a Raman amplified time stretch module for single-shot readout. Random profiles for magnitude and phase spectra were generated subject to the Kramer-Kronig (KK) relation.
To train and test the neural network, 6000 complex spectral profiles, representing the properties of the sample under test are randomly generated. For each complex frequency response, the magnitude and phase frequency responses are subject to the Kramers-Kronig relations, to ensure causality. These profiles are modulated onto the laser spectrum using a Waveshaper [30]. Further details about the experiments are provided in the Supplementary Materials.
2.2 Preprocessing
The data is a sequence of many frames each consisting of an interference pattern produced by one laser pulse. Before applying the retrieval algorithm, we first perform frame segmentation, which segments a continuous waveform into synchronized individual frames, using an algorithm reported in Ref. [31]. The frames are processed, to retrieve the electric field magnitude and phase, in two ways: (i) using neural network regression models, and (ii) using the standard approach of Hilbert transform.
2.3 Neural network
For the neural network approach, a single frame is the input to a five-layer fully-connected network (FC-NN), which is implemented via PyTorch. The network performs regression inferring the magnitude and phase spectra from the measured spectral interferogram. The input size (the size of a single input frame) of the network is determined by dividing the sampling rate of the real-time analog to digital converter (ADC) by the repetition of the laser. Given a single frame, the network directly outputs a vector that is the concatenated magnitude and phase spectra imposed by the spectral modulator, as shown in Fig. 1. The output size is determined by the time-stretch dispersion D, the frequency range of the profile (in wavelength $\varDelta \lambda$) and the sampling rate ${F_s}$, as shown in Eq. (1).
For the experiments, we use Labview to automatically input the 6000 profiles into the spectral modulator and to collect the output interference signal. For the fully-trained neural network, we partition this data into 5400 profiles for the training, 300 profiles for the validation, and the remaining 300 profiles for the test. All the hyperparameters including network architecture-related parameters and training-related parameters are optimized empirically. The training can take several hours, however, the inference is even faster than the Hilbert transform.2.4 Hilbert transform
Traditional algorithms for retrieving the relative magnitude and phase in spectral interferometry include the Fourier transform [14–20] and the closely related Hilbert transform-based algorithms [7,21]. Here we compare our AI techniques with the Hilbert technique but it should be noted that the Fourier and Hilbert techniques are equivalent in principle. The single-sideband signal is obtained by filtering in the former but in the latter, the single-sideband signal is implemented by constituting analytic signal via Hilbert transform after filtering out the DC components. The retrieved magnitude and phase of the Hilbert technique demonstrated in this article are time-averaged results over multiple frames, whereas for the neural network the results are single shot.
3. Results
3.1 Magnitude and phase retrieval performance comparisons
Figure 2(a) shows the retrieved spectral magnitude. For comparison, we show results for the Hilbert transform along with two neural networks, one where the network was trained with a small dataset (225 pulses) and the second trained with a large dataset (5400 pulses). Both neural networks outperform the Hilbert transform. The dispersive fiber used for time stretch induces unwanted distortions evidenced by the ripples due to self-phase modulation observed in the top trace in part (a). The Hilbert transform fails to compensate for these distortions whereas both neural networks can deal with these distortions.
Fig. 2. Experimental results for magnitude and phase retrieval from a femtosecond laser interferogram. The red curve is the ground truth (the magnitude and phase applied to the waveshaper) and the blue curve is the retrieved magnitude or phase. Hilbert transform is compared with two neural networks, one where the network was trained with a small dataset (225 pulses) and the second trained with a large dataset (5400 pulses). The magnitude retrieved by the Hilbert transform suffers from the distortions induced by optical nonlinearities (mainly self-phase modulation) generated in the dispersion media used for time stretch. Both AI techniques can compensate for these distortions whereas the Hilbert technique is not.
The corresponding retrieved phases of the three cases are shown in Fig. 2(b) and reveal the same behavior. The nonlinearity induced distortions significantly influence the retrieved phase by the Hilbert transform but much less so in the neural network. The AI models outperform the Hilbert transform in both magnitude and phase retrieval. The same behavior is observed for other profiles as shown in the Supplementary Materials.
Figures 3(a) and 3(b) show distributions of the magnitude and the phase root mean square error (RMSE) of both networks (trained with small and large experimental datasets) and the Hilbert transform, over a test set consisting of 300 pulse profiles. Experimentally trained networks produce lower mean errors (as shown in Figs. 3(c) and 3(d)) with smaller standard deviations (as shown in Figs. 3(e) and 3(f)). The Hilbert transform has a much larger mean RMSE and standard deviation. The same comparative behavior holds for the phase RMSE.
Fig. 3. Distributions of the (a) magnitude root mean square error (RMSE) and the (b) phase RMSE over an experimentally generated test set of 300 pulse profiles. The distribution curves here are fits to histograms. (c) Mean magnitude and (d) phase RMSEs over the same 300-profile test set. Standard deviations of (e) magnitude and (f) phase RMSEs over the same 300-profile test set from experiments. Both networks trained with small and large experimental datasets outperform the Hilbert transform.
3.3 Sensitivity to quantization noise and sampling rate of the backend digitizer
For single shot and fast real-time measurement of the interferometer output, the resolution and sampling rate of the real-time digitizer can become a bottleneck. We use the physical simulator to quantify the accuracy of magnitude and phase retrieval for different digitizer resolutions and sampling rates. The details of the simulator are described in the Supplementary materials. The 6000 interferometric waveforms from the simulator are quantized with 6 different resolutions as measured by the effective number of bits (ENOB) resulting in 36000 profiles. An FC-NN is trained on the 36000-profile dataset. Among 36000 profiles, 32400 profiles are used for training, 1800 for validation, and the remaining 1800 for testing.
To study the effect of the digitizer sampling rate, the interferometric waveforms from the physical simulator are downsampled to 6 different sampling rates with a fixed ENOB of 6 bits. The 6 bit was used because this is the typical ENOB of high-speed real-time oscilloscopes used in time stretch experiments. Note that the input size (i.e., the frame size) and output size of the network linearly increase as the sampling rate increases. Hence, we build different networks for different sampling rates and then train them separately. For each sampling rate, the size of the dataset is 6000, where 5400 profiles are for training, 300 profiles for validation, and 300 for testing.
The quantization error of the ADC impacts the accuracy of the magnitude and phase retrieval [32] therefore a retrieval algorithm must be robust against quantization noise. Here, we show the sensitivity of our retrieval technique to quantization noise by plotting the RMSEs vs. the ADC’s resolution as quantified by its effective number of bits (ENOB) for a 300-profile test set (Fig. 4). We see that the neural network has far better RMSE than Hilbert. For the Hilbert transform, the mean RMSE of magnitude drops when ENOB is below 6 bits. However, for AI, the mean magnitude RMSE is constant for ENOB above 4 bits (Fig. 4(a)). Similar results are observed for the retrieved phase (Fig. 4(b)). The results clearly indicate that the AI technique is more robust to quantization noise and performs requires less resolution from the digitizer.
Fig. 4. Dependence of magnitude and phase retrieval accuracy on the resolution of the digitizer. (a) Mean magnitude RMSE and (b) mean phase RMSE comparison between the Hilbert transform and the AI model under different ENOB of quantization and a fixed sampling rate of 50 GSamples/sec in the simulation. The RMSE for each pulse profile is averaged over the 300 profiles, in the test set, to obtain the mean RMSE. The circles and triangles are the mean RMSE of the Hilbert transform and the AI model. The solid curves are fitted trends. The AI model is more robust to quantization noise than the Hilbert transform, requiring only 4 bits of ENOB compares to 6 bits of ENOB before quantization noise reduces accuracy. ENOB: effective number of bits, RMSE: root-mean-squared error.
Requiring high-speed digitizers for data capture, real-time measurements are affected by the maximum available sampling rate of the analog to digital converter (ADC). Figure 5 shows the magnitude and phase RMSE versus the ADC sampling rate. The ADC resolution is 6 bits. The results are the mean square error over a 300-profile test set. The input and output sizes of the neural network linearly increase as the sampling rate increases. The AI model not only produces less error but can operate with a lower ADC sampling rate. For the Hilbert transform, the magnitude RMSE improves significantly when the sampling rate increases from 1 GSamples/sec to 20 GSamples/sec. Further increasing the sampling rate does not help improve performance because the sampling rate is enough to capture the fringe frequency of ∼1 GHz. For the AI model, both magnitude RMSE and phase RMSE improve when the sampling rate increases from 1 to 20 GSamples/sec.
Fig. 5. Dependence of magnitude and phase retrieval accuracy on the sampling rate of the digitizer (a) Magnitude RMSE and (b) phase RMSE versus the ADC sampling rate. The ADC resolution is 6 bits as measured by the effective number of bits (ENOB). The results are the mean over a 300-profile test set. The circles and triangles are the mean RMSE of the Hilbert transform and the AI model. The solid curves are fitted trends. The main observation is that the AI model not only is more accurate but also requires a lower ADC sampling rate.
4. Discussion
There are several reasons behind the observed differences in the performance of the Hilbert transform vs. AI methods. First, the Hilbert technique is incapable of compensating for optical nonlinearities (mainly self-phase modulation) induced distortions. Additional results in the Supplementary materials section support this hypothesis. The second reason is that the Hilbert method requires removing the pulse envelope and is susceptible to any error introduced in this process. On the other hand, the AI model works with raw laser pulses and does not require envelope removal. Another reason is that the AI model can learn the nonideal behavior of the spectral modulator whereas the Hilbert method cannot. Additionally, the Hilbert transform performs well for profiles whose magnitude attenuation and phase shift are in a moderate range while AI works for a wider range of profiles. At the same time, the AI method is not without its limitation. When the laser spectrum or the dispersion change significantly, the network must be retrained.
5. Conclusion
In this article, we demonstrated the direct retrieval of magnitude and phase profiles of femtosecond interferograms from raw spectral interferograms using a neural network. Utilizing experiments in the single-shot regime, we showed this technique significantly outperforms the conventional Hilbert transform in terms of the error rate. Compared to our prior work in which the neural network was performing binary classification [7,23], here the neural network is performing regression into the equivalent of six thousand classes. We showed that the AI model is also more resilient than Hilbert transform with respect to ADC quantization noise and sampling rate. The experimental system utilized Raman amplified time stretch for single-shot spectrum acquisition with a high signal to noise ratio. Our results suggest that AI methods could replace conventional phase retrieval algorithms in a wide range of applications that are based on spectral interferometry.
Funding
China Scholarship Council.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. L. Lepetit, G. Chériaux, and M. Joffre, “Linear techniques of phase measurement by femtosecond spectral interferometry for applications in spectroscopy,” J. Opt. Soc. Am. B 12(12), 2467–2474 (1995). [CrossRef]
2. C. Iaconis and Ian A. Walmsley, “Self-referencing spectral interferometry for measuring ultrashort optical pulses,” IEEE J. Quantum Electron. 35(4), 501–509 (1999). [CrossRef]
3. C. Dorrer, N. Belabas, J. P. Likforman, and M. Joffre, “Spectral resolution and sampling issues in Fourier-transform spectral interferometry,” J. Opt. Soc. Am. B 17(10), 1795–1802 (2000). [CrossRef]
4. A. F. Fercher, C. K. Hitzenberger, G. Kamp, and Y. S. El-Zaiat, “Measurement of intraocular distances by backscattering spectral interferometry,” Opt. Commun. 117(1-2), 43–48 (1995). [CrossRef]
5. R. Leitgeb, C. K. Hitzenberger, and A. F. Fercher, “Performance of fourier domain vs. time domain optical coherence tomography,” Opt. Express 11(8), 889–894 (2003). [CrossRef]
6. A. Ahmad, N. D. Shemonski, S. G. Adie, H. S. Kim, W. M. W. Hwu, P. S. Carney, and S. A. Boppart, “Real-time in vivo computed optical interferometric tomography,” Nat. Photonics 7(6), 444–448 (2013). [CrossRef]
7. C. L. Chen, A. Mahjoubfar, L. C. Tai, I. K. Blaby, A. Huang, K. R. Niazi, and B. Jalali, “Deep learning in label-free cell classification,” Sci. Rep. 6(1), 1–16 (2016). [CrossRef]
8. G. Herink, F. Kurtz, B. Jalali, D. R. Solli, and C. Ropers, “Real-time spectral interferometry probes the internal dynamics of femtosecond soliton molecules,” Science 356(6333), 50–54 (2017). [CrossRef]
9. J. G. Mance, B. M. La Lone, D. H. Dolan, S. L. Payne, D. L. Ramsey, and L. R. Veeser, “Time-stretched photonic Doppler velocimetry,” Opt. Express 27(18), 25022–25030 (2019). [CrossRef]
10. D. H. Dolan, “Extreme measurements with Photonic Doppler Velocimetry (PDV),” Rev. Sci. Instrum. 91(5), 051501 (2020). [CrossRef]
11. A. Mahjoubfar, K. Goda, A. Ayazi, A. Fard, S. H. Kim, and B. Jalali, “High-speed nanometer-resolved imaging vibrometer and velocimeter,” Appl. Phys. Lett. 98(10), 101107 (2011). [CrossRef]
12. C. Iaconis and Ian A. Walmsley, “Spectral phase interferometry for direct electric-field reconstruction of ultrashort optical pulses,” Opt. Lett. 23(10), 792–794 (1998). [CrossRef]
13. N. K. Berger, B. Levit, V. Smulakovsky, and B. Fischer, “Complete characterization of optical pulses by real-time spectral interferometry,” Appl. Opt. 44(36), 7862–7866 (2005). [CrossRef]
14. M. Takeda, H. Ina, and S. Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. 72(1), 156–160 (1982). [CrossRef]
15. A.P. Kovaecs, K. Osvay, G. Kurdi, M. Görbe, J. Klenbniczki, and Z. Bor, “Dispersion Control of a pulse stretchercompressor system with two-dimensional spectral interferometry,” Appl. Phys. B 80(2), 165–170 (2005). [CrossRef]
16. D. Meshulach, D. Yelin, and Y. Silberberg, “Real-time spatial-spectral interference measurements of ultrashort optical pulses,” J. Opt. Soc. Am. B 14(8), 2095–2099 (1997). [CrossRef]
17. A. P. Börzsönyia, M. Kovácsa, K. Görbea, and Osvayab, “Advances and limitations of phase dispersion measurement by spectrally and spatially resolved interferometry,” Opt. Commun. 281(11), 3051–3061 (2008). [CrossRef]
18. Pamela Bowlan, Pablo Gabolde, Matthew A. Coughlan, Rick Trebino, and Robert J. Levis, “Measuring the spatiotemporal electric field of ultrashort pulses with high spatial and spectral resolution,” J. Opt. Soc. Am. B 25(6), A81–A92 (2008). [CrossRef]
19. Tobias Kampfrath, Daryl M. Beggs, Thomas F. Krauss, and L. Kobus Kuipers, “Complete response characterization of ultrafast linear photonic devices,” Opt. Lett. 34(21), 3418–3420 (2009). [CrossRef]
20. Michael K. Yetzbacher, Trevor L. Courtney, William K. Peters, Katherine A. Kitney, Eric Ryan Smith, and David M. Jonas, “Spectral restoration for femtosecond spectral interferometry with attosecond accuracy,” J. Opt. Soc. Am. B 27(5), 1104–1117 (2010). [CrossRef]
21. A. Mahjoubfar, C. L. Chen, K. R. Niazi, S. Rabizadeh, and B. Jalali, “Label-free high-throughput cell screening in flow,” Biomed. Opt. Express 4(9), 1618–1625 (2013). [CrossRef]
22. Shijie Feng, Qian Chen, Guohua Gu, Tianyang Tao, Liang Zhang, Yan Hu, Wei Yin, and Chao Zuo, “Fringe pattern analysis using deep learning,” Adv. Photonics 1(2), 025001 (2019). [CrossRef]
23. Y. Li, A. Mahjoubfar, C. L. Chen, K. R. Niazi, L. Pei, and B. Jalali, “Deep cytometry: deep learning with real-time inference in cell sorting and flow cytometry,” Sci. Rep. 9(1), 1–12 (2019). [CrossRef]
24. M. A. Krumbügel, C. L. Ladera, K. W. DeLong, D. N. Fittinghoff, J. N. Sweetser, and R. Trebino, “Direct ultrashort-pulse intensity and phase retrieval by frequency-resolved optical gating and a computational neural network,” Opt. Lett. 21(2), 143–145 (1996). [CrossRef]
25. T. Zahavy, A. Dikopoltsev, D. Moss, G. I. Haham, O. Cohen, S. Mannor, and M. Segev, “Deep learning reconstruction of ultrashort pulses,” Optica 5(5), 666–673 (2018). [CrossRef]
26. M. Närhi, L. Salmela, J. Toivonen, C. Billet, J. M. Dudley, and G. Genty, “Machine learning analysis of extreme events in optical fiber modulation instability,” Nat. Commun. 9(1), 4923 (2018). [CrossRef]
27. A. Mahjoubfar, D. V. Churkin, S. Barland, N. Broderick, S. K. Turitsyn, and B. Jalali, “Time stretch and its applications,” Nat. Photonics 11(6), 341–351 (2017). [CrossRef]
28. J. Chou, O. Boyraz, D. Solli, and B. Jalali, “Femtosecond real-time single-shot digitizer,” Appl. Phys. Lett. 91(16), 161105 (2007). [CrossRef]
29. K. Goda, D. R. Solli, K. K. Tsia, and B. Jalali, “Theory of amplified dispersive Fourier transformation,” Phys. Rev. A 80(4), 043821 (2009). [CrossRef]
30. A. M. Weiner, “Femtosecond pulse shaping using spatial light modulators,” Rev. Sci. Instrum. 71(5), 1929–1960 (2000). [CrossRef]
31. Z. Bai, C. K. Lonappan, T. Jiang, A. M. Madni, F. Yan, and B. Jalali, “Tera-sample-per-second single-shot device analyzer,” Opt. Express 27(16), 23321–23335 (2019). [CrossRef]
32. J. Chan, A. Mahjoubfar, M. Asghari, and B. Jalali, “Reconstruction in time-bandwidth compression systems,” Appl. Phys. Lett. 105(22), 221105 (2014). [CrossRef]
