Application of GRIN-lens-based in-line digital holographic microscopy to automatic detection and localization of particles released from a MEMS device

Ali Akbar Khorshad; Nicholas Devaney; Ruth Houlihan

doi:10.1364/AO.507315

1. INTRODUCTION

Recently, the topic of digital holographic microscopy (DHM) has extensively developed and found significant applications in the quantitative phase imaging of living cells [1]; particle size, shape, and velocity measurements [2–5], and the characterization/inspection of microelectro-mechanical systems (MEMS) [6–8]. Typically, DHM is set up in either of two main experimental configurations: in-line or off-axis. The in-line DHM system is schematically akin to the original Gabor setup, with the illuminating wave being either spherical [4] or plane parallel [2,3,5]. The former can be implemented in a high magnification regime to provide higher resolution at the cost of field-of-view (FOV). The latter achieves maximum FOV; however, its resolution is limited by the pixel pitch of the detector [9]. The in-line DHM system is simple, compact, and mechanically stable, but it suffers from twin image noise, an intrinsic property of this holographic configuration. More recently, we developed a new in-line DHM system based on a GRIN lens that operates in a high-magnification regime and compared its resolution and image quality with two popular pinhole-based in-line DHM setups [10]. The experimental results showed that the GRIN-based system provides better resolution with respect to the pinhole-based setups (1.38 µm versus 1.74 µm) having a 1 µm aperture. Moreover, the GRIN-based setup is more compact, easy to implement, and no alignment process is required.

In DHM, the holographic images can be numerically reconstructed at different planes of the object space, often referred to as numerical refocusing [1]. To this end, a quantitative focus criterion and/or an autofocus algorithm are required to determine the best reconstruction distance or focused image. In the past few decades, various focus value functions have been developed and compared [11–16]. Note that the main task of autofocus functions is the determination and maximization of the image sharpness. Three widely used methods to find the optimal focus position of holographic images include [16]: (1) weighted spectral analysis; (2) variance of gray level distribution; and (3) edge detection. In particle field holography, these general autofocus methods cannot be directly applied to detect particles, because particles are widely distributed throughout the sample volume at different depths.

In the literature, three automatic detection techniques are found that bring all particles of a sample volume into focus and estimate the best focusing depth for each particle [3–5]. The first technique introduced by Tian et al., hereafter Tian’s method, is based on the minimum edge intensity focus metric [3]. Indeed, it has been separately shown that the intensity of the reconstructed image would be minimum when a particle is in focus [2]. Tian’s method provides a 2D projection minimum intensity image and its corresponding depth map. In this method, some image processing techniques such as intensity thresholding, edge detection, and morphological operations are used to determine the size and to estimate the $z$ position of the particles. As mentioned in [5] and also experienced by the authors, Tian’s approach is superior for detecting spherical particles and finding their in-plane positions $(x,y)$, but its limitations are that it requires manual thresholding and has a large uncertainty in the measured $z$ position of particles. These disadvantages led Guildenbecher et al. to develop a hybrid detection method (hereafter Guildenbecher’s approach), which is a combination of Tian’s technique and the maximum edge sharpness or Tenengrad focus metric [5]. Indeed, they have shown that the Tenengrad focus measure is superior for the estimation of particle depth [17]. Guildenbecher’s method is computationally intensive, taking up to 30 min to process a single recorded hologram, as reported by the authors in [18]. The third detection approach has been proposed by Darakis et al. [4]. In Darakis’ method, the popular Canny edge detector (CED) [19] is first utilized to segment the reconstructed images, and then the variance of gray level distribution is used as a focus metric to estimate the particle $z$ position. Darakis’ method is relatively fast and can provide a precise depth for each particle, thanks to the variance of gray level distribution focus measure [20]. However, it requires applying the CED to all the reconstructed images that contain in-focus and out-of-focus images of particles, which increases computation time. Moreover, the CED finds edges appropriately provided it is applied to very nearly in-focus images. There is also a single camera technique, used in microfluidics, which estimates the particle position based on astigmatism. In this method, a cylindrical lens is employed in the optical setup, and the depth position of the particles is coded by optical distortions. Finally, the 3D velocity distribution of the particles could be determined by using wavefront deformation particle tracking velocimetry (refer to [21] for further details).

In this paper, we introduce a new automatic hybrid detection and localization method, which takes advantage of Tian and Darakis’ methods and is suitable for particle field holography. In brief, we first utilize the minimum intensity focus metric to bring all particles, spread over the sample volume, into focus or make a 2D projection minimum intensity image. Then, the CED is applied to segment all the already in-focus particles; finally, the variance of gray level distribution focus metric is used to accurately measure the $z$ position of each detected particle. Further, we apply our recently developed GRIN-lens-based inline DHM setup [10] in conjunction with the proposed hybrid detection method to automatically detect and localize silica microparticles with different sizes released from a custom MEMS device. A procedure for measuring the size and 3D velocity of the released microparticles is also presented. The measured sizes are in agreement with the nominal sizes stated in the particle data sheets, and the measured velocities are of the order of a few tens of cm/s. To the best of our knowledge, this is an original application of the in-line DHM technique. Indeed, what we have developed in [10] and advanced here introduces a diagnostic method for European Space Agency (ESA) activity entitled “MEMS-based nano-particle storage and release system for quantum physics payload platform (QPPF)” (MEMS4QPPF) [22,23]. The objective of the MEMS4QPPF project is to demonstrate a MEMS storage and release device capable of loading massive (100 nm) silica particles into an optical trap where they can be cooled and released for matter-wave interferometry aimed at testing the transition from quantum to classical physics [24]. With this background to the project, we are required to measure the particle velocity. Because in order to trap a particle, its kinetic energy should be similar to or less than the trap potential depth of the optical tweezers (i.e., slower particles are easier to trap) [25].

This paper is organized as follows: in Section 2, we first introduce our automated hybrid particle detection and localization method in detail and then present the general procedure for measuring the size and velocity of particles detected with our high-magnification DHM setup. A photograph of the MEMS device, a schematic of the experimental setup, and the three experiments carried out will be presented in Section 3. The typical size and velocity measurement results for particles launched from the MEMS device are also given and discussed in Section 3. Finally, suggestions for future work and a short conclusion are provided.

2. METHODOLOGY

The principles of the proposed hybrid detection and localization method are first presented, followed by guidelines on measuring particle size and velocity.

A. Automatic Hybrid Particle Detection and Localization Method

Detection and localization of particles in DHM generally consists of segmentation and localization parts. The former is required to identify the exact area occupied by each particle and to eliminate other unrelated sections of the image. This part is commonly performed either by intensity thresholding of the reconstructed image or by applying an edge detection algorithm to the reconstructed image. Localization is needed to find the best focusing depth and consequently estimate the $z$ position of each particle and therefore requires a proper focus metric.

1. Segmentation

In order to segment particles, we first reconstruct the holographic images from the recorded hologram at a sequence of reconstruction distances/depths and store them in a cell array. Then, to make a 2D projection image, we collapse the previously stored 3D reconstructed images by recording the minimum intensity value at each pixel. As an example, the above-mentioned steps are demonstrated in Figs. 1(a)–1(c). Figure 1(a) shows a typical hologram of 300 µm silica particles vertically launched from a MEMS device and recorded by our GRIN-lens-based inline DHM setup, both schematically shown in Fig. 2. The MEMS device and the imaging system will be fully described in Section 3. Figure 1(b) presents the numerically reconstructed image of the hologram at a reconstruction distance at which only particles shown by the yellow arrows are in focus, according to the variance of gray level distribution focus criterion. Figure 1(c) shows the resulting 2D projection image in which all particles, spread over the sample volume, are in focus. The black spots and dark gray rings around them indicate, respectively, potential particles and the twin images. The CED is sensitive to noise; as such, before applying the CED to the 2D projection image, the image is first smoothed using a low-pass Gaussian filter with a standard deviation slightly larger than the typical spatial period of fringes visible in the projected image. The CED with thresholds chosen to minimize visually spurious edges is then applied to the filtered 2D projection image. In summary, the CED finds edges by looking for local maxima of the image gradient. It benefits from two thresholds to detect strong and weak edges. The weak edges are included in the final output if they are connected to the strong edges. As shown in Fig. 1(d), the final result is a binary matrix with ones corresponding to pixels where edges are detected and zeros elsewhere. In the example presented here, we used seven pixels as the standard deviation for Gaussian filtering of the 2D projection image, and the low and high thresholds of the CED were chosen 0.1 and 0.25, respectively. It was found that the CED output is not sensitive to the selected parameters for standard deviation and thresholds; they can therefore be fixed while the recorded holograms of a particular size-range particle are processed. To identify particles, the dark areas in the binary image of Fig. 1(d) fully enclosed by edges are filled. Hereafter, these filled areas are called “spots.” Morphological operations are utilized to remove open-ended lines and also spots with diameters less than a cutoff value through an erosion and dilation process. In other words, all the remaining segments unrelated to particles are removed by these operations. Finally, as shown in Fig. 1(e), a set of spots corresponding to particles detected in the sample volume is obtained and labeled. From Fig. 1(e), which is the final reconstructed 3D holographic image, we notice that particles 2, 6, and 10 are clusters of two particles.

Fig. 1. Demonstration of key steps that our proposed hybrid automatic detection and localization approach is based on. (a) ${800} \times {1280}\;{\rm pixel}$ sample hologram of 300 µm silica particles vertically released from a MEMS device with the active area of 5 mm in diameter. As shown in Fig. 2, this active area is considered as the sample depth here. (b) The reconstructed image at 300 mm reconstruction distance, which shows in focus only particles denoted by the yellow arrows. (c) Minimum intensity projection that shows all released particles, being at different depths, in focus. (d) Edges extracted from the minimum intensity projection image [i.e., (c)] by applying a suitable Gaussian filter and CED to it. (e) Detected in-focus released particles colored by their real ${\rm depths}/z$ locations in $\times {10^{- 1}}\,\,\rm mm$. (f) Normalized variance of gray level distribution focus metric measures the best focusing depth/reconstruction distance of each detected particle. The letter ${P}$ in the legend indicates particle.

Download Full Size | PDF

Fig. 2. Schematic diagram of the MEMS device, used to vertically launch silica particles dry-loaded onto it, and the GRIN lens-based in-line DHM, employed to image launched particles and measure their sizes and velocities. L, F, H, G, Det, D, and ${d}$ indicate, respectively, laser, neutral density filter, black holder, GRIN lens, high-speed CMOS image sensor, GRIN lens-detector distance, and particle-detector distance or particle ${\rm depth}/{z}$ position. Black filled circles, denoted by letter ${P}$, are silica particles.

Download Full Size | PDF

2. Localization

In order to localize the particles detected in Fig. 1(e), the normalized variance of gray level distribution is first calculated using the following equation [11–14,16]:

(1)$${VAR_N} = \frac{1}{{MN{{\overline g}^2}}}\sum\limits_{i,j}^{M,N} {[{g(i,j) - {\overline g}} ]^2}$$

for each particle at each reconstruction distance/depth. In Eq. (1), $g(i,j)$, ${\overline g}$, $M$, and $N$ indicate the intensity of the reconstructed image at pixel $(i,j)$, which belongs to a selected particle, the mean of the gray level distribution within the selected particle, and the height and width of the region occupied by the selected particle. In Eq. (1), variance is normalized with the mean intensity (i.e., ${\overline g}$) in order to compensate for changes in the mean image brightness among reconstructed images. Then, as demonstrated in Fig. 1(f), a graph of the normalized variance focus function versus reconstruction distance/focusing depth shows a maximum for each detected particle (in this case, particles 1, 4, 5, 10, and 12 were chosen randomly). The reconstruction depth corresponding to the maximum point of the graph yields the best focusing depth for that detected particle. However, the actual depth or $z$ position of each particle (i.e., the real distance between the particle and the detector, as shown in Fig. 2) still needs to be measured. This is obtained using the following equation [26]:

(2)$$d = {\left[{\frac{1}{{d^{\prime}}} + \frac{1}{D}} \right]^{- 1}},$$

where $d^{\prime} $ and $D$ denote the best focusing depth for each particle extracted from Fig. 1(f) and the GRIN lens to detector distance in the setup of the DHM (see Fig. 2). Note that Eq. (2) can be derived from the holographic imaging equations [26] provided: (1) the wavelengths used for recording the hologram and reconstructing the image are similar; and (2) the hologram is recorded with a spherical reference wave; for the sake of simplicity, however, the images are numerically reconstructed using a plane parallel reference wave. It is worth mentioning that the recording and reconstruction distances (i.e., $d$ and $d^{\prime} $) would be equal if the same reference wave is used for recording the hologram and for the image reconstruction. In Fig. 1(e), a color is assigned to each detected particle and its actual depth (i.e., $d$) or $z$ position is associated with the color bar. In the example shown here, the experimental GRIN lens-detector distance and the center of the MEMS device to the detector were set at 67 and 52 mm, respectively. As shown in Fig. 2, the active area of the MEMS device is 5 mm in diameter, so the measured $z$ position of vertically launched particles with respect to the detector plane should be in the 49–55 mm range. Note that this is already verified by the color bar of Fig. 1(e).

It is worth mentioning that it took only 8 s using a personal computer (Intel Core i5-8400 CPU operating at 2.8 GHz, 16.0 GB of available RAM, MATLAB R2022b) to analyze the single hologram, as presented in Fig. 1(a), and obtain all the results shown in Figs. 1(b)–1(f). In this case, the hologram was numerically reconstructed at 50 different reconstruction distances.

To estimate the $x$- and $y $- coordinates of the center of each detected particle, assumed to be round in shape, we average over the position of its pixels weighted by their gray level values as follows [27]:

(3)$${x_p} = \frac{{\sum\nolimits_i {{x_i}{g_i}}}}{{\sum\nolimits_i {{g_i}}}},\quad {y_p} = \frac{{\sum\nolimits_i {{y_i}{g_i}}}}{{\sum\nolimits_i {{g_i}}}},$$

where the sum is over all pixels belonging to the detected particle.

B. Procedure for Particle Size and Velocity Measurement

As shown in Fig. 2, our GRIN lens-based in-line DHM operates in the high magnification regime. This is because the coherent reference wave used for recording the hologram is spherical, and the sample is positioned relatively near the source (i.e., GRIN lens). In this situation, a lateral magnification is introduced by the DHM setup, and it can be derived from the holographic imaging equations as follows [26]:

(4)$$M = \frac{{d^{\prime}}}{d}\quad {\rm or}\quad M = {\left[{1 - \frac{d}{D}} \right]^{- 1}},$$

where $d$, $d^{\prime }$, and $D$ are as defined earlier. The effective pixel size of the detector can be written as

(5)$${P_{{\rm eff}}} = \frac{P}{M},$$

where $P$ indicates the actual pixel pitch of the detector array. We can determine the radius of each detected particle using the following equation:

(6)$$R = {R_{{\rm pixel}}} \times {P_{{\rm eff}}} = \frac{{{R_{{\rm pixel}}} \times P}}{M},$$

where ${R_{{\rm pixel}}}$ is the radius of particle measured in pixels [it is directly obtained from Fig. 1(e)]. The lateral components of the average velocity can be measured as follows:

(7)$${\overline{V} _{x,y}} = \frac{{\Delta r}}{{\Delta T}} = \frac{{{n_{x,y}} \times P}}{{M \times \Delta T}},$$

where ${n_{x,y}}$ and $\Delta T$ are, respectively, the number of pixels the particle has moved in the $x$ or $y$ direction and the time passed during this particle movement. Finally, the axial velocity of particle can be measured:

(8)$${\overline{V} _z} = \frac{{\Delta d}}{{\Delta T}} = \frac{{{d_2} - {d_1}}}{{\Delta T}},$$

where ${d_1}$ and ${d_2}$ denote the initial and final particle-detector distances, respectively. According to Eqs. (4) and (6)–(8), the measured size and velocity of particles are directly proportional to the particle-detector distance (i.e., $d$), which is why the precise measurement of particle $z$ position or depth is crucial in the application of a DHM, operating in the high-magnification regime, to particle field holography.

Fig. 3. Demonstration of the MEMS device. (a) Schematic of the MEMS device composed of a flat silicon membrane mounted on a commercial annular piezoelectric driver. (b) Photograph of the fabricated ultrasonic silicon membrane. (c) Photograph of the assembled MEMS device. (d) Resonant frequencies of the MEMS device, characterized using a laser Doppler vibrometer. (e) [Visualization 1], (f) [Visualization 2] Typical surface profiles of a vibrating circular membrane at first and second resonant modes, respectively [28].

Download Full Size | PDF

3. EXPERIMENT

A. MEMS Device

As schematically shown in Fig. 3(a), the MEMS device, utilized to launch silica particles, consists of a 50 µm thick flat silicon ultrasonic membrane of 5 mm diameter [see Fig. 3(b)] strongly bonded to an off-the-shelf annular piezoelectric transducer (from Noliac ceramics, PZT Navy type I, NCE40 hard material). The flat silicon membrane was fabricated by the Tyndall National Institute using silicon-on-insulator micromachining technology. A photograph of the final MEMS device assembly is shown in Fig. 3(c). A microscanning laser Doppler vibrometer (Polytec MSA-400) has been used for resonance testing of the MEMS device; the result is shown in Fig. 3(d). According to Fig. 3(d), the MEMS device has two resonant peaks. The first peak, corresponding to the (0,1) resonant mode, takes place at 29 kHz, and the second peak, corresponding to the (0,2) mode, occurs at 109 kHz. The MEMS device can be driven at either frequency, and the piezo transducer can withstand voltage amplitudes up to ${{200V}_{\textit{PP}}}$. If a sinusoidal wave signal with proper voltage amplitude is applied across the piezo transducer, the flat silicon membrane vibrates and results in releasing silica particles previously dry-loaded onto the membrane. Figures 3(e) and 3(f) (Visualization 1 and Visualization 2, respectively) demonstrate typical surface profiles of a resonating membrane at both resonant modes. Regarding the loading procedure, a microspoon is first used to put a small amount of the selected dry powder silica particles on the flat silicon membrane and then a long-tip air blower/hand duster can be used gently to widely distribute particles over the membrane.

Fig. 4. (a) Schematic diagram of integrating the MEMS device into the GRIN-lens-based in-line DHM setup. L, F, H, G, P, V and Det denote, respectively, laser, neutral density filter, semicircular holder, GRIN lens, silica particles, and variable-height post and high-speed CMOS image sensor. Black arcs represent the light diffracted by particles that reaches the detector. (b) Photograph of the experimental setup along with (c) its zoomed-in central part, consisting of the MEMS device and the GRIN lens positioned inside a D-shape holder. (d) The best-resolved reconstructed image obtained from the DHM setup shown in (b) where the sample and detector distances from the GRIN lens were, respectively, 2 and 55 mm. In this image, the maximum resolution is estimated to be 1.3 µm.

Download Full Size | PDF

B. Experimental Setup

The GRIN lens-based in-line DHM system, utilized for the MEMS device testing with particles, is schematically shown in Fig. 4(a). A blue diode-laser (450 nm, 40 mW, ${Z}$-laser) beam passes through a set of neutral density filters and illuminates a rod GRIN lens (effective focal length = 0.92 mm, NA = 0.55, diameter = 1 mm, length = 2.36 mm, Edmund Optics), mounted in a black semicircular holder. The bottom part of the light cone from the GRIN lens covers the MEMS device surface. It is diffracted by particles released from the membrane and interferes with the undiffracted direct light on a high-speed CMOS detector array (${1280} \times {800}$ resolution, 20 µm pixel pitch, 3250 fps at full resolution, Phantom V310). Finally, a hologram is recorded by the detector and then transferred to the computer for subsequent numerical image reconstruction. The semicircular shape of the holder allows positioning of the MEMS right beneath the GRIN lens, if required to image released particles of a few micrometers in diameter. To finely adjust the MEMS device with respect to the GRIN lens in three dimensions, it is first positioned on a variable height translating post; then, the post is mounted on $x - y$ micromechanical translation stages. Photographs of the whole setup and its central part, showing the mounted MEMS device and the GRIN lens, are presented in Figs. 4(b) and 4(c), respectively.

In accordance with Eq. (4), this setup provides a variable magnification microscope using only a single microscope objective (in this case the GRIN lens), with magnification depending on the sample-detector distance. For a given GRIN lens-detector distance, a higher magnification is provided where the sample is positioned relatively close to the GRIN lens. Therefore, the MEMS device can be tested with a wide range of particle sizes, provided that the MEMS loaded with smaller particles is located closer to the GRIN lens.

To find the best setting that provides the maximum resolution, a positive USAF-1951 high-resolution test target (down to 645 lp/mm, Edmond Optics) was fixed at 2 mm distance from the GRIN lens. The setup resolution was then measured at different GRIN lens-detector distances, similar to what was performed in [10]. As the best reconstructed image given in Fig. 4(d) shows, the maximum resolution of 1.3 µm is achieved where the GRIN lens-detector distance is 55 mm (detailed results not shown here).

As demonstrated in Fig. 4(a), an electronic setup composed of a signal generator (AFG3021B, Tektronix), $50\times$ amplifier (DC-5MHz high voltage, WMA-300, Falco Systems), and oscilloscope (350 MHz-2.5GS/s, MSO 4034, Tektronix) is used to generate an amplified sine wave with appropriate frequency (either 29 or 109 kHz) for application across the piezo driver. Moreover, it is required to apply the amplified voltage across the piezo and start the image acquisition simultaneously so that the detector starts acquiring the holograms just as the MEMS device begins to launch particles. To this end, a pulse/pattern generator (80 MHz, Agilent 81104A) is employed as an external clock source in combination with the signal generator to externally trigger the detector when the signal is applied to the MEMS device.

C. Automatic Detection of Released Particles

In this research, we utilized three size ranges of silica microparticles in dry powder physical form: (1) 212–300 µm (G1277, Sigma-Aldrich); (2) $\le\! 106\,\,{\unicode{x00B5}}\rm m$ (G4649, Sigma-Aldrich); and (3) 9–13 µm (440345, Sigma-Aldrich). In the third case, the mean particle size has been reported to be in the 9–13 µm range. Hereafter, particles with these size ranges will be referred to as large-, medium-, and small-sized microparticles. In the set of experiments performed here, the detector operated in the global shutter mode, and its exposure time and sampling rate were respectively set at 100 µs and 3139 fps. Note that this sampling rate results in 318.57 µs frame interval/period. The GRIN lens-detector distance and the center of the MEMS device from the GRIN lens were fixed at 67 and 17 mm, respectively. A reference background hologram is first recorded prior to loading of silica particles onto the MEMS device. Afterward, particles are loaded, and the process of data acquisition for each test is automatically initiated, so that a predefined number of holograms is recorded and transferred to the computer for further analysis. Note that a contrast hologram is then obtained by subtraction of the reference background hologram from recorded holograms with the particles released. Finally, the holographic 2D projection images/movies of particles released by the MEMS device are numerically reconstructed from the contrast holograms using a MATLAB-based software developed for this purpose. The angular spectrum approach to image reconstruction, discussed in [10], and the detection methodology, described here in Section 2, are implemented in the software. Note that using the contrast hologram can help to remove the intensity variations in the primary laser beam profile, the light reflected from the silicon membrane and also the possible effects of contamination/aberrations present in the setup of DHM.

1. Large-Sized Particles

In this test, an amplified sine wave with frequency of 29 kHz (corresponding to the first resonant mode) and amplitude of $40\,\,{\rm V_{\textit{PP}}}$ was continuously applied to the MEMS device, and 521 holograms were captured and processed. A typical hologram of (212–300) µm silica particles launched from the MEMS device along with its reconstructed 2D projection amplitude image are presented in Figs. 5(a) and 5(b) (Visualization 3 and Visualization 4, respectively). To visualize the release event, we produced a holographic video from all the reconstructed 2D projection images (see Visualization 4). This video demonstrates that all particles are successfully released from the MEMS device with different initial velocities/accelerations, depending on particle’s position over the flat silicon membrane, and they are automatically detected by the GRIN-lens-based in-line DHM system. Note that the black area in the bottom left corner of the images shown in Figs. 5(a) and 5(b) represents the image of one of the wires soldered to the piezo transducer. The twin-image noise is present in the holographically reconstructed amplitude image of Fig. 5(b), as it is an intrinsic property of the in-line DHM method. In addition, we repeated this test with different signal amplitudes and found that (data not shown) the initial velocities/accelerations of released particles increase when a signal with higher voltage amplitude is applied to the MEMS device, and a minimum voltage amplitude of about $20\,\,{\rm V_{\textit{PP}}}$ is required to release this size range of particles. In this situation, silica particles only jump up and down over the MEMS; therefore, they remain near the surface of the vibrating membrane and do not leave the FOV.

Fig. 5. Experimental result demonstrating automatic detection of (212–300) µm particles, released from the MEMS device using the setup presented in Fig. 4. (a) [Visualization 3] A typical recorded hologram, and (b) [Visualization 4] its reconstructed 2D projection holographic image obtained using the proposed hybrid particle detection and localization method.

Download Full Size | PDF

2. Medium-Sized Particles

In this case, test parameters were similar to the large-sized particles, but the amplitude of the applied signal was changed to $45\,\,{\rm V_{\textit{PP}}}$ and 400 holograms were acquired. Figures 6(a) and 6(b) (Visualization 5) show the recorded hologram and its holographically reconstructed 2D projection amplitude image at frame number 30, respectively. As seen from Visualization 5, all silica particles of $\le\! 106\,\,{\unicode{x00B5}}\rm m$ in diameter are successfully launched from the MEMS device. Note that this visualization is a 3D movie that displays all particles, launched from the surface of the vibrating membrane in focus. This experiment was also repeated at $40\,\,{\rm V_{\textit{PP}}}$, $60\,\,{\rm V_{\textit{PP}}}$, and $100\,\,{\rm V_{\textit{PP}}}$ amplitudes, and similar results were achieved (data not shown).

Fig. 6. Experimental result demonstrating that silica particles of $\le\! 106\,\,\unicode{x00B5}\rm m$ in diameter are successfully launched from the MEMS device and automatically detected by the setup shown in Fig. 4. (a) The recorded hologram. (b) [Visualization 5] Its reconstructed 2D projection holographic image at frame number 30, which brings all released particles into focus, thanks to the minimum intensity focus metric.

Download Full Size | PDF

3. Small-Sized Particles

To test the MEMS device with mean particle diameter in the 9–13 µm range and automatically detect all released particles, we only changed the amplitude of the applied signal to $100\,\,{\rm V_{\textit{PP}}}$, and the other test parameters remained unchanged. In a typical test, 220 holograms were acquired and analyzed. Figures 7(a) and 7(b) (Visualization 6) show, respectively, the acquired hologram at frame number 24 and its reconstructed 2D projection image obtained from the hybrid detection and localization method, introduced in Section 2. The reconstructed 2D projection images of all the recorded holograms form the frames of a 3D movie, which demonstrates 10 µm particle release from the MEMS device (see Visualization 6). By carefully inspecting Visualization 6, it can be seen that some released particles are a cluster of a few single 10 µm particles, which indicates the amount of dry powder 10 µm silica particles manually loaded onto the MEMS device surface is relatively high, and the motion of released particles is affected by a turbulent air flow. This air flow directs particles in different directions, as soon as they are released from the surface of the vibrating membrane. Careful examination of the experimental setup revealed that this is an artefact of the detector. Indeed, an air flow is generated inside the CMOS detector/camera box by its cooling fans and the flow then comes out from the camera C-mount (the C-mount seems to be not fully sealed). We could set the camera fan speed to a minimum, but it was not possible to completely turn it off. Note that only small-sized particles are affected by the flow, because they experience smaller net force (the sum of gravitational and propulsion forces) compared with the large- and medium-sized particles. However, this experimental result shows that the holography is an appropriate 3D imaging technique to study flow dynamics.

Fig. 7. Typical experimental result showing that silica particles of 10 µm in diameter are released by the MEMS device and automatically detected by the setup presented in Fig. 4. (a) The recorded hologram. (b) [Visualization 6] Its digitally reconstructed 2D projection image at frame number 24.

Download Full Size | PDF

D. Size and Velocity Measurement Results

The size and velocity of some released particles were measured using the methods proposed in Section 2. To do so, three tests were performed, and three separate sets of raw data were recorded. All test parameters such as the frequency and amplitude of the applied signal to the MEMS device (i.e., 29 kHz and $100\,\,{\rm V_{\textit{PP}}}$, respectively) were identical between all tests, while each test and set of recorded data belonged to a particular size range of silica particles. Raw data were analyzed, and holographically reconstructed 2D projection images were obtained from the recorded holograms. From each analyzed data set, two released particles were randomly chosen and manually tracked frame by frame. Finally, implementation of the proposed hybrid detection and localization method in conjunction with the procedure for particle size and velocity measurement provided all the required parameters (previously introduced in Subsection 2.B.) for each tracked particle.

Fig. 8. Demonstration of tracking released particles to measure their size and velocity. The yellow arrows indicate the tracked particles. (a) and (b) Reconstructed images of large-sized particles at frame numbers 51 and 76, respectively. (d) and (e) Reconstructed images of medium-sized particles at frame numbers 9 and 29, respectively. (g) and (h) Reconstructed images of small-sized particles at frame numbers 473 and 483, respectively. (c), (f), and (i) Normalized variance focus functions that provide the best focusing depths for the tracked particles shown in panels (a), (e), and (g), respectively.

Download Full Size | PDF

As an example, one of the tracked released particles selected from each size category along with its normalized variance focus function is shown in Fig. 8, where panels (a)–(c), (d)–(f), and (g)–(i) correspond to large-, medium-, and small-sized silica particles, respectively. The images and measured parameters for the second tracked particle from each size category are not shown here; however, the summarized measured size and velocity of that particle will be given later in Table 4. The first tracked particle is specified with a yellow arrow symbol in Fig. 8. Note that, in the case of releasing larger particles (e.g., 300 and 100 µm in diameter), we observe that the motion of released particles is mainly perpendicular to the membrane surface (called $y$ direction), and there is only a negligible velocity component parallel to the membrane surface and detector plane (called “$x$ direction”). Therefore, in the case of small-sized particles, we track these particles as far as their motion trajectories are still in the $y$ direction. In this way, we can assume that there is a negligible contribution to the measured lateral velocity of small-sized particles from the air flow artifact that intends to move particles in the $x$ or $z$ direction. Here, the $z$ direction is defined as the axis parallel to the membrane surface and normal to the detector plane (i.e., optical axis). Thus, the resulting lateral velocities of released particles with different sizes can be compared correctly. It should be noted that (1) Figs. 8(a) and 8(b) are, respectively, the reconstructed images at frame numbers 51 and 76 of the analyzed data set for large-sized particles; (2) Figs. 8(d) and 8(e) are, respectively, the reconstructed images corresponding to frame numbers 9 and 29 of the analyzed data set for medium-sized particles; and (3) Figs. 8(g) and 8(h) present, respectively, the reconstructed images at frame numbers 473 and 483 of the analyzed data set for small-sized particles. The normalized variance focus value functions presented in Figs. 8(c), 8(f), and 8(i) correspond to the tracked particle shown in panels (a), (e), and (g) of Fig. 8, respectively. As discussed in Subsection 2.A., the normalized variance of gray level distribution is utilized to find the best focusing depth ($d^{\prime} $) of each tracked particle at every frame. Then, the actual depth ($d$) of the tracked particle at each frame is obtained from Eq. (2), and finally all the required quantities are measured through Eqs. (4)–(8). The measured quantities for the tracked large-, medium-, and small-sized particles shown in Fig. 8 are now summarized in Tables 1–3, respectively. The quantity $|{{\overline v}_l}|$ in Tables 1–3 denotes the magnitude of the lateral average velocity of the released particles. Regarding the error values provided in Tables 1–3, we reconstructed the holographic amplitude images in a 5 mm sequence and then calculated the normalized variance of gray level distribution for each particle at each reconstruction distance. Thus, we included a 5 mm error value in all the best image reconstruction distances ($d^{\prime}$) obtained from the graphs of normalized variance versus reconstruction distance. A 1 mm error value was also included in the measured GRIN lens to detector distance ($D$). Finally, the error values associated with other measured parameters were determined from the partial derivative of Eqs. (2) and (4)–(8).

Table 1. Measured Size and Velocity of the Tracked Particle Specified with the Yellow Arrow in Figs. 8(a) and 8(b)

View Table | View all tables in this article

Table 2. Measured Size and Velocity of the Tracked Particle Specified with the Yellow Arrow in Figs. 8(d) and 8(e)

View Table | View all tables in this article

Table 3. Measured Size and Velocity of the Tracked Particle Specified with the Yellow Arrow in Figs. 8(g) and 8(h)

View Table | View all tables in this article

Table 4. Summarized Size and Velocity Measurements of Two Tracked Released Particles Chosen from Each Particle Size Category^a

View Table | View all tables in this article

It is seen from Tables 1–3 that the axial position ($d$) of particles released from the MEMS device has slightly changed between the initial and final frames; therefore, the magnifications and effective pixel sizes of the in-line DHM are different. The averaged magnification or averaged effective pixel size of the initial and final frames is utilized in Eqs. (6) and (7) to estimate the size and average velocity of released particles reported in Tables 1–3. Finally, the measured size and velocity of both tracked released particles, selected from each size category, are summarized in Table 4. The experimental results given in Table 4 show that: (1) particles released from the MEMS device have a small velocity component in the horizontal direction and a large velocity component in the vertical direction; (2) the lateral average velocities of particles released from the MEMS device are of the order of a few tens of cm/s. It is worth noting that dry launching of silica nanoparticles in vacuum using a different release system/mechanism has resulted in the same order of magnitude for the measured velocity of launched silica particles (refer to [25] for further details). (3) Measured sizes of small particles are not covered within the expected 9–13 µm range. The main reason could be that particles selected for size measurement are a cluster of a few single 10 µm particles. These clusters may be formed when dry-loading and/or when distributing the silica particles onto the MEMS device. It is expected that the velocity and acceleration of particles released from the MEMS device depend on the size of the particles as well as their positions on the vibrating silicon membrane (e.g., the highest initial release velocity is provided for a particle located at the center of the circular resonating membrane when the membrane works at the second resonant mode). Therefore, to properly compare the average velocity of particles with different size ranges released from the MEMS device, it is necessary to measure the velocity distribution of all released particles that belong to a size range category. To do so, we should fully automate our proposed hybrid detection and localization method, so that it can automatically track all released particles and measure their velocity and size distribution. In addition, we are interested in setting up the MEMS device and the GRIN lens-based in-line DHM system (except for the detector array) inside a vacuum chamber and repeat the above-demonstrated experiments and also try to release and detect 10 µm silica particles in diameter. These steps form the subject of future work.

4. CONCLUSION

In summary, we have experimentally demonstrated that a MEMS device composed of a 50 µm thick flat circular ultrasonic membrane bonded to a commercially available annular PZT piezoelectric transducer is a promising system to store and launch silica particles, previously dry-loaded onto the membrane. Moreover, our recently developed GRIN-lens-based in-line DHM, which provides 1 µm imaging resolution within a mm-scale FOV, is proven to be a suitable setup for the detection of silica microparticles, within a 10–300 µm size range, released from the MEMS device. The active area of the MEMS device that launches particles is a few mm in diameter, and this microscopy technique provides images within such a mm-scale depth-of-field, thanks to the numerical refocusing capability associated with the digital holographic imaging method.

According to the experimental results shown here, we conclude that the proposed automatic hybrid detection and localization method can provide holographically reconstructed 3D images, in which all particles of the sample volume are in focus, and precisely determine depth/$z$ position of each particle, thanks to the variance of gray level distribution focus metric. Note that precise measurement of particle depth is crucial in a DHM with a degree of magnification, because the measured size and displacement/velocity of particle depend directly on the actual particle depth. In this research, this hybrid detection method was applied to create 3D videos that demonstrate particle release from the MEMS device and also to measure the size and velocity of released particles. The measurement results show that the velocity of released particles is of the order of a few tens of cm/s.

Funding

European Space Agency (4000131569/20/NL/BW); Irish Research eLibrary.

Acknowledgment

The authors acknowledge E. O’Connor and O. Ryan of University of Galway (UOG) for their technical assistance with the electronic part of the setup. The authors are also grateful for instrumentation support provided by G. M. O’Connor of UOG. Open access funding provided by Irish Research eLibrary.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

REFERENCES

1. M. K. Kim, “Principles and techniques of digital holographic microscopy,” SPIE Rev. 1, 018005 (2010).

2. S. Murata and N. Yasuda, “Potential of digital holography in particle measurement,” Opt. Laser Technol. 32, 567–574 (2000). [CrossRef]

3. L. Tian, N. Loomis, J. A. Domínguez-Caballero, et al., “Quantitative measurement of size and three-dimensional position of fast-moving bubbles in air-water mixture flows using digital holography,” Appl. Opt. 49, 1549–1554 (2010). [CrossRef]

4. E. Darakis, T. Khanam, A. Rajendran, et al., “Microparticle characterization using digital holography,” Chem. Eng. Sci. 65, 1037–1044 (2010). [CrossRef]

5. D. R. Guildenbecher, J. Gao, P. L. Reu, et al., “Digital holography simulations and experiments to quantify the accuracy of 3D particle location and 2D sizing using a proposed hybrid method,” Appl. Opt. 52, 3790–3801 (2013). [CrossRef]

6. A. Asundi, Digital Holography for MEMS and Microsystem Metrology (Wiley, 2011), Vol. 7.

7. L. Xu, X. Peng, J. Miao, et al., “Studies of digital microscopic holography with applications to microstructure testing,” Appl. Opt. 40, 5046–5051 (2001). [CrossRef]

8. G. Coppola, P. Ferraro, M. Iodice, et al., “A digital holographic microscope for complete characterization of microelectromechanical systems,” Meas. Sci. Technol. 15, 529–539 (2004). [CrossRef]

9. A. A. Khorshad and N. Devaney, “A comparison of the resolution obtained using two regimes of digital in-line holographic microscopy,” in Digital Holography and Three-Dimensional Imaging (Optica, 2022), paper W5A-24.

10. A. A. Khorshad and N. Devaney, “Grin-lens-based in-line digital holographic microscopy,” Appl. Opt. 62, D131–D137 (2023). [CrossRef]

11. T. Yeo, S. Ong, R. Sinniah, et al., “Autofocusing for tissue microscopy,” Image Vis. Comput. 11, 629–639 (1993). [CrossRef]

12. E. Krotkov, “Focusing,” Int. J. Comput. Vis. 1, 223–237 (1988). [CrossRef]

13. F. C. Groen, I. T. Young, and G. Ligthart, “A comparison of different focus functions for use in autofocus algorithms,” Cytometry 6, 81–91 (1985). [CrossRef]

14. L. Firestone, K. Cook, K. Culp, et al., “Comparison of autofocus methods for automated microscopy,” Cytometry 12, 195–206 (1991). [CrossRef]

15. F. Dubois, C. Schockaert, N. Callens, et al., “Focus plane detection criteria in digital holography microscopy by amplitude analysis,” Opt. Express 14, 5895–5908 (2006). [CrossRef]

16. P. Langehanenberg, B. Kemper, D. Dirksen, et al., “Autofocusing in digital holographic phase contrast microscopy on pure phase objects for live cell imaging,” Appl. Opt. 47, D176–D182 (2008). [CrossRef]

17. D. R. Guildenbecher, J. Gao, P. L. Reu, et al., “Digital holography reconstruction algorithms to estimate the morphology and depth of nonspherical absorbing particles,” Proc. SPIE 8493, 849303 (2012). [CrossRef]

18. D. R. Guildenbecher, L. Engvall, J. Gao, et al., “Digital in-line holography to quantify secondary droplets from the impact of a single drop on a thin film,” Exp. Fluids 55, 1–9 (2014). [CrossRef]

19. J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-8, 679–698 (1986). [CrossRef]

20. T. J. Naughton, E. Darakis, T. Khanam, et al., “Processing of digital holograms for size measurements of microparticles,” Proc. SPIE 7155, 715524 (2008). [CrossRef]

21. C. Cierpka, R. Segura, R. Hain, et al., “A simple single camera 3C3D velocity measurement technique without errors due to depth of correlation and spatial averaging for microfluidics,” Meas. Sci. Technol. 21, 045401 (2010). [CrossRef]

22. European Space Agency (ESA), “QPPF CDF study report: assessment of quantum physics payload platform,” CDF Study Report: CDF-183(C), https://sci.esa.int/documents/34923/36148/1567255001222-QPPF_CDF_study_report.pdf.

23. M. L. Terranova and E. Tamburri, Nanotechnology in Space (CRC Press, 2021).

24. J. Bateman, S. Nimmrichter, K. Hornberger, et al., “Near-field interferometry of a free-falling nanoparticle from a point-like source,” Nat. Commun. 5, 4788 (2014). [CrossRef]

25. A. Khodaee, K. Dare, A. Johnson, et al., “Dry launching of silica nanoparticles in vacuum,” AIP Adv. 12, 125023 (2022). [CrossRef]

26. U. Schnars and W. P. Jüptner, “Digital recording and numerical reconstruction of holograms,” Meas. Sci. Technol. 13, R85–R101 (2002). [CrossRef]

27. J. Lu, J. P. Fugal, H. Nordsiek, et al., “Lagrangian particle tracking in three dimensions via single-camera in-line digital holography,” New J. Phys. 10, 125013 (2008). [CrossRef]

28. O. Alexandrov, “Vibrations of a circular membrane,” https://en.wikipedia.org/wiki/Vibrations_of_a_circular_membrane.

Name	Description
Visualization 1	Typical surface profile of a vibrating circular membrane at first resonant mode.
Visualization 2	Typical surface profile of a vibrating circular membrane at second resonant mode.
Visualization 3	Typical recorded hologram of ~300um silica particles released from a MEMS device.
Visualization 4	Holographically reconstructed 3D video of ~300um silica particles released from a MEMS device.
Visualization 5	Holographically reconstructed 3D video of ~100um silica particles released from a MEMS device.
Visualization 6	Holographically reconstructed 3D video of ~10um silica particles released from a MEMS device.

Frame	$d^{'} (m m)$	$d (m m)$	$M$	$P_{e f f} (µ m)$	$(x, y)$
51	$(270 \pm 5)$	$(50.4 \pm 0.7)$	$(5.36 \pm 0.02)$	$(3.74 \pm 0.01)$	(1146,621)
76	$(290 \pm 5)$	$(51.1 \pm 0.7)$	$(5.68 \pm 0.02)$	$(3.52 \pm 0.01)$	(1183,255)
$Δ x (µ m)$	$Δ y (µ m)$	${\bar{v}}_{x} (\frac{c m}{s})$	${\bar{v}}_{y} (\frac{c m}{s})$	$\| {\bar{v}}_{l} \| (\frac{c m}{s})$	$s i z e (µ m)$
$134 \pm 1$	$1328 \pm 5$	$1.7 \pm 0.0$	$16.7 \pm 0.1$	$16.8 \pm 0.1$	$292 \pm 1$

Frame	$d^{'} (m m)$	$d (m m)$	$M$	$P_{e f f} (µ m)$	$(x, y)$
9	$(280 \pm 5)$	$(54.1 \pm 0.8)$	$(5.18 \pm 0.01)$	$(3.86 \pm 0.01)$	(324,634)
26	$(310 \pm 5)$	$(55.1 \pm 0.8)$	$(5.63 \pm 0.01)$	$(3.56 \pm 0.01)$	(303,53)
$Δ x (µ m)$	$Δ y (µ m)$	${\bar{v}}_{x} (\frac{c m}{s})$	${\bar{v}}_{y} (\frac{c m}{s})$	$\| {\bar{v}}_{l} \| (\frac{c m}{s})$	$s i z e (µ m)$
$(77.9 \pm 0.2)$	$(2155 \pm 5)$	$(1.2 \pm 0.0)$	$(33.8 \pm 0.1)$	$(33.9 \pm 0.1)$	$(56.0 \pm 0.2)$

Frame	$d^{'} (m m)$	$d (m m)$	$M$	$P_{e f f} (µ m)$	$(x, y)$
473	$(275 \pm 5)$	$(53.9 \pm 0.8)$	$(5.10 \pm 0.01)$	$(3.92 \pm 0.01)$	(541,729)
483	$(320 \pm 5)$	$(55.4 \pm 0.8)$	$(5.78 \pm 0.01)$	$(3.46 \pm 0.01)$	(532,571)
$Δ x (µ m)$	$Δ y (µ m)$	${\bar{v}}_{x} (\frac{c m}{s})$	${\bar{v}}_{y} (\frac{c m}{s})$	$\| {\bar{v}}_{l} \| (\frac{c m}{s})$	$s i z e (µ m)$
$(33.2 \pm 0.1)$	$(583 \pm 2)$	$(1.0 \pm 0.0)$	$(18.3 \pm 0.1)$	$(18.3 \pm 0.1)$	$(42.0 \pm 0.1)$

$S_{N} (µ m)$	(212–300)		$\leq 106$		(9-13)
${\bar{v}}_{x} (\frac{c m}{s})$	$(1.7 \pm 0.0)$	$(3.2 \pm 0.0)$	$(1.2 \pm 0.0)$	$(1.3 \pm 0.0)$	$(1.0 \pm 0.0)$	$(1.3 \pm 0.0)$
${\bar{v}}_{y} (\frac{c m}{s})$	$(16.7 \pm 0.1)$	$(15.0 \pm 0.1)$	$(33.8 \pm 0.1)$	$(35.6 \pm 0.4)$	$(18.3 \pm 0.1)$	$(15.8 \pm 0.1)$
$\| {\bar{v}}_{l} \| (\frac{c m}{s})$	$(16.8 \pm 0.1)$	$(15.3 \pm 0.1)$	$(33.9 \pm 0.1)$	$(35.6 \pm 0.4)$	$(18.3 \pm 0.1)$	$(15.8 \pm 0.1)$
$S_{M} (µ m)$	$(292 \pm 1)$	$(238 \pm 1)$	$(56.0 \pm 0.2)$	$(76.0 \pm 0.3)$	$(42.0 \pm 0.1)$	$(51.0 \pm 0.2)$

Application of GRIN-lens-based in-line digital holographic microscopy to automatic detection and localization of particles released from a MEMS device

Abstract

1. INTRODUCTION

2. METHODOLOGY

A. Automatic Hybrid Particle Detection and Localization Method

1. Segmentation

2. Localization

B. Procedure for Particle Size and Velocity Measurement

3. EXPERIMENT

A. MEMS Device

B. Experimental Setup

C. Automatic Detection of Released Particles

1. Large-Sized Particles

2. Medium-Sized Particles

3. Small-Sized Particles

D. Size and Velocity Measurement Results

4. CONCLUSION

Funding

Acknowledgment

Disclosures

Data availability

REFERENCES

Supplementary Material (6)

Data availability

Cited By

Figures (8)

Tables (4)

Equations (8)

Applied Optics