The Computational Optical Sensing and Imaging (COSI) conference has a long history, being held over the past 15 years. One of its original aims was to develop and promote integrated optical, electronic, and algorithmic designs of optical systems. After years of development, this field has been expanded from topics such as wavefront sensing and compressive coding to a much broader scope, including computational holography, computational microscopy, compressive imaging, non-line-of-sight (NLOS) imaging, phase imaging, and novel applications with new types of sensors. The 2021 COSI conference was held virtually 19–23 July. The virtual delivery allowed the meeting to take place in spite of ongoing travel and sanitary restrictions. Fourteen invited talks and more than 120 contributed papers were presented at the event. In addition to the annual conferences, COSI organizers have published several feature issues in the Applied Optics and Optics Express journals over the years, the most recent one in Optics Express in 2020. After a two-year break due to the COVID19 pandemic, the conference now has, for the first time, a joint feature issue with Optics Express and Applied Optics. We are excited to publish 36 papers in this feature issue, highlighting the latest works in this field.

Among the papers, there are two invited reviews devoted to deep learning on holography [1] and ptychography [2]. Another invited research paper focuses on an angle measuring system [3]. The contributed papers [4–34] cover image systems engineering, algorithms, devices, and mathematical models mainly in the following research areas: computational microscopy, NLOS imaging, compressive imaging, ghost imaging/single-pixel imaging, imaging through scattering media, phase retrieval, and 3D imaging. In multiple areas, the use of deep learning to solve challenging inverse problems has become widely adopted.

In the following, we summarize the papers in this issue. The invited reviews and papers are presented first. Then we present the contributed papers based on research topics ordered as appearing on the conference website. These papers can be found on the feature issue website of Optics Express (https://opg.optica.org/oe/virtual_issue.cfm?vid=503) and in the ninth issue of 2022 in Applied Optics.

1. INVITED REVIEWS AND PAPERS

Holographic imaging has important applications in microscopy, metrology, and other domains. Deep-learning-based reconstruction techniques have recently emerged as versatile and powerful tools for digital holography across these application areas. A review paper [1] surveys recent deep learning approaches in this area and introduces a taxonomy that makes this topic accessible to the novice while providing comprehensive and in-depth coverage of the field.

Ptychography is a well-known computational lensless imaging technique; however, its application to extreme ultraviolet (XUV) microscopy and wavefront sensing is relatively new. The invited review [2] covers the state of the art in XUV ptychography with tabletop high-harmonic generation sources. Different hardware options including illumination optics and detector concepts as well as algorithmic aspects in the analysis of multispectral ptychography data are considered. Cutting-edge technological applications of XUV ptychography such as multispectral wavefront sensing, attosecond pulse characterization, and depth-resolved imaging, and the perspectives for its development are discussed. This comprehensive review is an excellent guide for researchers working in the area as well as for graduate students.

The invited research paper in this feature issue is on a high-performance, optoelectronic angle measuring system capable of high-accuracy and high-precision angle measurements [3]. The proposed system uses a diversity of diffractive and refractive optical elements, combined with custom analog and digital electronics, to achieve six degrees of freedom localization and angular velocity estimation.

2. COMPUTATIONAL MICROSCOPY (INCLUDING PTYCHOGRAPHY), DIGITAL HOLOGRAPHIC MICROSCOPY

Authors demonstrate in simulation and experiment a multi-lens microscopic imaging system that overlaps multiple independent fields of view on a single sensor for automated specimen analysis based on CNN [4]. In [5], the authors describe a computational light-sheet microscope for hyperspectral acquisition at high spectral resolution. The method uses a structured light pattern to acquire and reconstruct the spatial dimension orthogonal to the slit of the used spectrometer. The paper demonstrates the feasibility of the method and reports the first in vivo results for hydra specimens labeled using two fluorophores.

A computational method for fast single-photon counting of directly sampled time-domain fluorescence lifetime imaging microscopy data is presented in [6]. It is capable of accurate fluorescence lifetime and intensity measurements while acquiring over 160 mega-counts-per-second with sub-nanosecond time resolution between consecutive photon counts. A method to improve the image quality of a synthetic aperture imaging interferometric microscopy system based on analysis of cross-correlation between two consecutive sub-images in the overlapping regions is proposed and experimentally verified [7].

In [8], the authors report on a simple, low-cost method for quantitative phase imaging based on in-line Gabor holography realizable in a standard bright-field microscope with coherent sensing capabilities. The authors in [9] present an approach to enhance cryo-electron microscopy (cryo-EM) postprocessed maps based on a multiscale tubular filter. The method locally determines the tubularness measure and uses this information to enhance elongated local structures and to attenuate blob-like and plate-like structures, which contributes to an improvement in the obtained reconstructions.

3. COMPRESSED SENSING

In [10], the authors develop a neural-network-based reconstruction method for spatial compressive imaging. The method incorporates sensing matrix information in the form of degraded maps as input to the network, resulting in high-quality and fast reconstructions at low compression rates. A hardware modification to the rolling shutter (RS) mechanism found in CMOS detectors by shuffling the pixels in every scanline is proposed in [11]. With the improved sampling of the space–time datacube and sophisticated reconstruction methods including a neural-network-based method, the proposed shuffled RS approach provides an alternative for snapshot temporal imaging, if ever implemented in hardware.

4. IMAGING THROUGH SCATTERING AND TURBID MEDIA

A phase plate model to analyze the morphology and statistics of speckles produced by a point-like source with a wide spectrum is presented [12]. In [13], the authors develop a wavefront shaping method based on adaptive stochastic parallel gradient descent optimization with the Hadamard basis to focus light through scattering soil samples. In [14], speckle correlation pre-processing is combined with neural-network-based reconstruction to achieve color object recovery through unknown opaque scattering layers by training with only one diffuser.

5. LENSLESS IMAGING, COHERENT DIFFRACTION IMAGING

A new reconstruction method for lensless inline holography is developed [15]. This method can produce high-resolution images of the amplitude and phase of a thin sample over a large field of view using aliased intensity measurements taken at a lower resolution.

In [35], the authors report a maximum-likelihood-estimation-based framework for holographic coherent diffraction imaging that provides improved image reconstruction results for various practical settings, e.g., missing low-frequency data due to occlusion from a beamstop apparatus, or data that are highly corrupted by Poisson shot noise. The developed mathematical framework is also applicable beyond holographic coherent diffraction imaging.

6. MACHINE LEARNING FOR COMPUTATIONAL IMAGING

The machine learning approach has been very popular in almost all imaging problems. In this feature issue, there are eight papers using neural networks for multiple problems. In addition to papers [1,10,11,14,19,23] that have been or will be introduced under more specific topics, the following are two more works:

A simple fiber-optic tip sensor is used to identify liquid via a neural network [16]. The fiber–droplet and droplet–air interfaces work together as an EFPI with a liquid cavity. As the droplet evaporates, the length of the cavity reduces. Thus, using a probing light source, the evaporation event can be captured. A CNN network is used for the classification task.

In [17], the authors propose a backpropagation neural-network-based method for improving the measurement accuracy of four-quadrant detectors with a small amount of real data.

7. NOVEL APPLICATIONS OF HOLOGRAPHY

The authors explore a new approach to overlay metrology for semiconductor chip alignment [18]. For this purpose, they develop a compact dark-field digital holographic microscope that uses only a single imaging lens. Aberrations from this optical system are calibrated and corrected using nano-sized point scatterers on a silicon substrate together with computational wavefront imaging techniques.

8. PHASE RETRIEVAL AND ITS APPLICATIONS

A paper to demonstrate the use of deep neural networks in combination with coded diffraction patterns to solve the phase retrieval inverse problem efficiently and with surprising accuracy is presented [19], while in [20], a general frequency-shifting technique to pixelwise retrieve the absolute phase in fringe projection profilometry without any phase unwrapping is developed. A proposal and demonstration of a computational method capable of performing single-shot phase retrieval, achieving pixel-level super-resolution in a compact optical system is presented in [21].

9. POLARIZATION IMAGING AND SENSING

In [36], the authors integrate a microscope objective and a lenslet array with a polarization mask, then use this optical element as a replacement of the conventional lens in a camera for polarimetric microscopy.

10. 3D IMAGING (STRUCTURED ILLUMINATION, ToF SENSING, LiDAR, LIGHT FIELDS)

Here, [22], authors propose a novel method for the center detection of laser lines in line triangulation setups. The method exploits the different spectral filters provided by the camera’s Bayer pattern to synthesize a high-dynamic-range image of the projected laser line. This leads to an improvement in line center detection, which yields an improved 3D surface reconstruction. Using neural networks to perform decomposition of LiDAR signal waveforms, resulting in significant computational efficiency gains is described. Different network versions are proposed and analyzed, and the applicability of each version depends on the SNR of the input LiDAR signals. In [24], a method capable of correcting phase and amplitude errors caused by mirror movement uncertainties in mechanically scanned structured illumination imaging is proposed.

11. UNCONVENTIONAL IMAGING MODALITIES (INTENSITY INTERFEROMETRY, GHOST IMAGING, MUTUAL INTENSITY IMAGING)

In [25], the authors use simulation to demonstrate a method for three-dimensional ghost imaging that integrates the differential-correlation-sampling technique and a modulated continuous-wave laser source and allows suppressing the effect of dynamic ambient light and electrical noise. A proposal for a forgery attacking algorithm for grayscale single-pixel intensity values in computational ghost imaging is presented in [26]. Then the generalized forgery attack scheme is also discussed for the well-known double random phase encoding system in Fourier domain and Fresnel domain.

In [27], the authors introduce arbitrary-order fractional derivative operations into single-pixel imaging that offers a good trade-off between image SNR and performance of edge enhancement. An image-free target tracking scheme based on the discrete cosine transform and single-pixel detection is described [28]. The tracking speed can reach 208 fps at a spatial resolution of ${{128}} \times {{128}}$ pixels with a tracking error of no more than one pixel.

12. COMPUTER GENERATED HOLOGRAPHY AND COMPUTATIONAL DISPLAYS FOR AR/VR

The authors in [29] present an alternative algorithm for single-shot color hologram generation, which offers more flexibility when applied to imaging with holographic displays.

13. EVENT-DRIVEN COMPUTATIONAL IMAGING USING NEUROMORPHIC SENSORS

In [30], the authors present a novel approach in the analysis of motion using speckle imaging, demonstrating that event-driven cameras can be effectively used for that purpose in a lens-free optoelectronic system.

14. NON-LINE-OF-SIGHT IMAGING

The problem of white-light-illuminated NLOS imaging is studied, where the authors incorporate a speckle-correlation-based model into a deep neural network and use a two-step strategy to learn the optimization of the scattered pattern autocorrelation and then object image reconstruction [31]. The authors in [32] propose physical and virtual NLOS image acquisition systems for capturing simulated and realistic images under various ambient light conditions that are used further to train/fine-tune a multi-task CNN to perform simultaneous background illumination correction and NLOS object localization. In [33], the authors propose a new approach to localizing 3D objects outside the LOS of a camera. Their method operates in the thermal infrared wavelength range and works in a passive manner, i.e., it does not require active light sources. The authors demonstrate experimental results of their NLOS 3D localization approach.

15. OPTICAL COMPUTING

An optical computing engine for performing fractional Fourier transform on non-stationary temporal signals is developed in [34], which can replace the computer-based FRFT computation in a swept-source optical coherence tomography system.

We hope readers enjoy these exciting works in this field. We thank OE Editor James Leger, AO Editor-in-Chief Gisele Bennett, and Chris Dainty from Optica for the opportunity to organize this feature issue. We also thank Optica staff Carmelita Washington, Nicole Williams-Jones, Kelly Cohen, and Rebecca Robinson for their assistance through the manuscript review process. The next COSI conference will take place in Vancouver, British Columbia, Canada, 11–15 July 2022. It will be a hybrid meeting to accommodate both in-person and online participation.