The quantitative three-dimensional reconstruction and tracking of micro or nano-objects spread in a volume is of great interest in many fields of science (e.g., biomedicine, chemical engineering, fluid mechanics). Digital Holography (DH) is a quantitative 3D imaging system that is time-resolved. It is consequently an appropriate tool to study fast 3D phenomena. However, increasing the accuracy of the reconstruction is a major challenge in DH, particularly when the signal-to-noise ratio (SNR) is low, the pixel is large (e.g., for high speed cameras), or the numerical aperture is low. In the last case, depth accuracy and lateral accuracy can differ by an order of magnitude [1].

Many numerical methods for DH accuracy improvement have been described in the literature in recent decades. Recently, “pixel Super-Resolution (SR)” approaches, which are generally used for conventional imaging, were used to improve the resolution of reconstructed digital holograms [2,3]. These approaches consist of four main steps: recording multiple holographic images that are shifted with respect to one another, image registration, computation of the super-resolved hologram, and holographic reconstruction. This enables a significant gain in resolution. However, as the image is processed as one block [2,4], this is very sensitive to the presence of any objects in the field whose motion differs from that of the objects of interest [5].

“Inverse Problems” (IP) approaches have also been proposed to more accurately reconstruct digital holograms. Rather than reconstructing the hologram (e.g., using numerical diffraction), they aim to find the simulation (i.e., direct problem) that is closest to the data. This approach is sometimes referred to as Bayesian approach [6] or as compressive holography [7,8]. For arbitrarily shaped objects, this approach makes it possible to reconstruct their opacity distribution sampled on a 3D grid by inverting the hologram formation model using an appropriate regularization [6,9]. For parametric objects (described by few parameters) whose diffraction patterns are given by an analytic formula, inverse reconstruction can be performed by maximum-likelihood fitting of an imaging model to the data [10,11]. This approach is sometimes called “super-resolution” (see e.g., [12]). These IP methods have been shown to be optimal in certain cases [12,13] and lead to almost unsupervised algorithms. To further improve the 3D reconstruction accuracy of parametric objects, we suggest combining the potential of such IP approach and pixel SR.

The IP approach applied to one image in the stack enables an optimal estimation, and in some cases, reaches Cramer Rao lower bounds [12,13]. In the case of a $n_{\text{img}}$ image stack, applying this IP approach to each image and averaging the estimated parameters decreases the standard deviation of the estimator by $\sqrt{n_{\text{img}}}$ [14]. However, it has been shown that joint estimation of the parameters leads to more accurate results, particularly since information intercorrelation between parameters is non-negligible [5]. In that case, the gain in accuracy can be greater than $\sqrt{n_{\text{img}}}$. In the following, without loss of generality, we consider opaque spherical objects. The imaging model is the Thompson model [15] and has four parameters: radius $r$, and 3D position $(x,y,z)$. We now describe the algorithm used to jointly estimate, in a stack of images, the object parameters, which are the transverse shifts of the object ${\{\mathrm{\Delta }{x}_i,\mathrm{\Delta }{y}_i\}}_{i=1\text{:}{n}_{\text{img}}}$, the axial position $z$, and the radius $r$. We then present the results we obtained using this approach on synthetic and experimental holograms and compare them to independent parameter estimation and classical pixel SR reconstruction.

The proposed algorithm (Fig. 1) is a modified version of the unsupervised algorithm proposed by Soulez et al. [10] adapted to the processing of shifted hologram stacks. It performs joint estimation of the shifts ${\{\mathrm{\Delta }{x}_i,\mathrm{\Delta }{y}_i\}}_{i=1\text{:}{n}_{\text{img}}}$ and object parameters $\{z,r\}$. It makes it possible to reconstruct objects outside of the classical field of view and account for irrelevant data regions in the hologram thanks to a weighted mask [16]. It consists of two main steps:

 

Fig. 1. Joint parameter estimation.

Download Full Size | PDF

This algorithm can be used iteratively when several objects of interest are in the field of view. In that case, a cleaning step, which subtracts the detected pattern from the hologram stack, allows the SNR of the remaining patterns to be increased.

To demonstrate the advantages of the proposed SR scheme, the method was validated in both numerical and bench-top experiments. Figure 2 shows the experimental configuration, which consists in a classical in-line DH arrangement. The light emitted by a $\lambda =662\mathrm{nm}$, 75-mW fiber pigtailed (FP) laser diode (Coherent OBIS 660 LX) was collimated to act as the reference beam. The sample is represented by two objects. The first is moving, while the other remains stationary. Interference between the reference and the object beams are recorded by a high-speed $1280\times 1024$-pixel CMOS sensor (NAC MEMRECAM K4), with a pixel pitch of 21.7 μm and a fill-factor (i.e., active area over total area of pixel) of 0.7 at a frame-rate of 1000 Hz. SR is performed by jointly optimizing the parameters of a stack of $n_{\text{img}}$ particle holograms.

 

Fig. 2. Experimental configuration for SR of particle holograms.

Download Full Size | PDF

Validation of the concept is proposed through numerical simulation of particle holograms. Here, $n_{\text{img}}=20$ holograms with both a moving and a fixed object are considered. The moving object consists of a 100-μm diameter particle with random sub-pixel shifts between each simulated frame, whereas the fixed object is an opaque square. Joint optimization of the simulated sequence allows both estimation of $\{z,r\}$ and of the object shifts from one frame to the next. Moreover, independent optimization over each holograms as well a standard pixel SR method are considered for comparison. This latter method, is tested using an algorithm based on a global estimation of the shift from one frame to the next in the Fourier space [17]. The estimation of the parameters is then performed by analysis of the super-resolved hologram by IP approach.

A stack of $n_{\text{img}}=20$ $1024\times 1024$-pixel low-resolution holograms are simulated according to the experimental conditions shown in Fig. 2. A Gaussian white noise has been added to the simulated hologram resulting in a SNR of 5. A close-up view over $128\times 128$ pixels is proposed Fig. 3(a).

 

Fig. 3. Illustration of the SR by IP approach with joint optimization. (a) Simulated low-resolution hologram; a Gaussian white noise has been added so that the SNR of hologram is 5. The image area represents $128\times 128$ pixels. (b) Hologram super-resolved by a factor 4 using the proposed method (given for illustration purposes).

Download Full Size | PDF

For visualization purposes, a super-resolved image is computed. The registration is performed using the estimated shifts, and the holograms are interpolated to the desired SR factor value and superimposed. The maximal SR factor ${\mathrm{SR}}_f$ achievable can be linked to the number of acquired image by ${\mathrm{SR}}_f\approx \sqrt{n_{\text{img}}}\approx 4.5$ [5]. Therefore, the SR factor is fixed to ${\mathrm{SR}}_f=4$. Results obtained are proposed Fig. 3. A close-up over $512\times 512$ pixels of the super-resolved holograms is shown in Fig. 3(b). Here, the advantages of the approach are clearly visible. On one hand, higher spatial frequencies can be visualized on the super-resolved hologram, thereby increasing the accuracy of the reconstruction. On the other hand, the super-resolved hologram has a higher SNR. Low SNR holograms can thus also benefit from this jointly optimized reconstruction method. Quantitative confirmation of this result can be found in the Table 1. Here, comparison between independent estimation over 1224 images, joint estimation over 61 stacks of 20 images, and super-resolved images based on standard pixel SR are realized. The improvement in the standard deviation of both $z$ and $r$ value is close to $\sqrt{n_{\text{img}}}$ as predicted by Robinson and Milanfar [5] in the case where optimized parameters are decorrelated. As seen in Table 1, Vandewalle’s methods do not succeed in improving the standard deviation values. As a matter of fact, this method is based on a global motion estimation over the whole image, whereas only the object is moving. Let us note that our approach can be applied on stacks of images that present nonsubpixel shifts contrary to classical pixel SR schemes [17].

Table 1. Comparison of Particle Parameter Estimation, with Simulated Hologram Sequences, for Standard Pixel SR Method, Independent and Joint Estimation

View Table | View all tables in this article

We here propose to take benefits of the jointly optimized IP approach reconstruction scheme for high-accuracy reconstruction of experimental holograms. The investigated object is a calibrated opaque chromium disk with a diameter of 100 μm deposited on a glass substrate (diameter $\pm 1\mathrm{\mu m}$, roundness $\pm 0.25\mathrm{\mu m}$ Optimask), whose position is shifted using a 2-axis mechanical stage. The fixed object is a human hair. Low-resolution holograms are acquired over the complete area of the CMOS sensor. A close-up on the recorded low-resolution holograms over $320\times 320$ pixels is presented Fig. 4(a). Part of the moving-object interference pattern is occluded by the hair. Jointly optimized reconstruction by IP approach is performed over $n_{\text{img}}=20$ images with shifts of several tens of pixels between each acquisition. For illustration purposes, the super-resolved hologram with ${\mathrm{SR}}_f=4$ is presented in Fig. 4(b). It should be noted that the disturbance of the fixed object is weaker than that of the low-resolution hologram. This is due to the image-registration step that results in averaging of the fixed object signal over the sequence of holograms. As for the simulation results in Fig. 3, the super-resolved hologram contains higher spatial frequencies that improve estimation of the object radius $r$ and object-to-sensor distance $z$.

 

Fig. 4. Illustration of the SR by IP approach with joint optimization. (a) Acquired low-resolution hologram. The original image was cropped to $320\times 320$ pixels. (b) Hologram super-resolved by a factor of 4 using the proposed method (given for illustration purposes).

Download Full Size | PDF

The super-resolved holograms shown in Figs. 3 and 4 are given for illustration purposes and qualitatively demonstrate the benefits of using the proposed scheme. For quantitative assessment, comparison between a standard IP approach reconstruction method (i.e. optimization of the model parameters hologram by hologram), and the proposed joint optimization scheme is realized. A sequence of 1224 individual images (this amount corresponds to the camera buffer size) of the moving calibrated object are recorded with a frame-rate of 100 Hz and an exposure duration of ${\tau }_e=6\mathrm{\mu s}$. From these images, stacks of $n_{\text{img}}=20$ images are built by taking one frame out of 61. According to the amount of acquired images, 61 image stacks are used to point out benefits of the approach. Values of $z$ and $r$ are optimized for each frame in the case of the classical IP approach and for each image stack for the joint IP approach. Obtained results are proposed Fig. 5. Here, estimated values of $z$ (blue dots) and $r$ (green crosses) for individual IP approach reconstruction (a), and joint IP approach reconstruction (b) are plotted versus either the frame number or the stack number.

 

Fig. 5. Evolution of the estimated $z$ (blue dots) and $r$ (green crosses) parameters according to the optimization scheme. (a) Individual IP approach over each acquired hologram. (b) Joint IP approach of the 61 image stacks. Each stack consists of 20 images.

Download Full Size | PDF

One can notice that dispersion of $z$ and $r$ values is higher considering an individual processing scheme [Fig. 5(a)] rather than a joint scheme [Fig. 5(b)]. This can be quantified by analyzing averaged values of $z$ and $r$ as well as their respective standard deviations ${\sigma }_z$ and ${\sigma }_r$. Obtained results are listed Table 2 and compared to manufacturer’s specifications. The estimations of $z$ and $r$ are improved by more than a factor $\sqrt{n_{\text{img}}}$. This is due to the existence of correlation between estimated parameters as discussed in Ref. [5]. This proves that the joint estimation provides in this case an improvement of the accuracy of the estimator. It should be noted that $z$ values are slightly changed along the image sequence (probably due to the translation stage), thus leading to an overestimate of ${\sigma }_z$ [Fig. 5(a)]. In order to be able to apply Vandewalle’s method on our experimental holograms, the estimated particle shifts have been converted to subpixel shifts by means of an integer translation step. Results obtained with a classical pixel SR scheme [17] have been calculated. They illustrate the inappropriateness of this method for our application. Indeed standard pixel SR schemes are based on a global estimation of the subpixel shifts from one frame to the next.

Table 2. Comparison of Particle Parameter Estimation, with Experimental Hologram Sequences, for Standard Pixel SR Method, Independent and Joint Estimation

View Table | View all tables in this article

We have proposed a joint optimization method, applied to particle in-line hologram reconstruction, improving $z$ and $r$ estimation accuracy. It should be noted that this approach is valid as long as the imaging model is parametric. Benefits of the proposed scheme have been demonstrated with both numerical and bench-top experiments. Improvement of the accuracy on $z$ and $r$ have been proved to be higher than $\sqrt{n_{\text{img}}}$, thus reducing the gap between axial and lateral positioning accuracy. This approach can be generalized to high-resolution 3D tracking considering joint estimation. For instance by estimating ${\{x,y,z\}}_{i=1\text{:}{n}_{\text{img}}}$ and $r$ along an image sequence.