1. Introduction

Optical microscopy has been the workhorse of various fields including physical sciences, engineering, biology and medicine. However, existing optical microscopes are still relatively bulky and expensive, making them by and large restricted to laboratory settings. In recent years, however, there has been an extensive research effort to create alternative microscopy tools that are based on computational tools such as holographic imaging [‎1-‎14]. As a result of these efforts, several new microscopy modalities were introduced to create simpler, more compact and cost-effective designs that can even be used in remote locations and field settings [‎15-‎30].

A recent development toward the same goal is the lensfree super-resolution holographic microscope [‎31, ‎32] which achieves sub-micron spatial resolution over a large field-of-view of e.g., ~24 mm². It works based on partially-coherent lensfree digital in-line holography using multiple light sources (e.g., light-emitting diodes - LEDs) placed at ~3-6 cm away from the sample plane such that at a given time only a single source illuminates the objects, projecting in-line holograms of the specimens onto a CMOS sensor-chip. Since the objects are placed very close to the sensor chip (e.g., ~1-2 mm) the entire active area of the sensor becomes our imaging field-of-view, and the fringe-magnification is unit. As a result of this, these holographic diffraction signatures are unfortunately under-sampled due to the limited pixel size at the CMOS chip (e.g., ~2-3 µm). To mitigate this pixel size limitation on spatial resolution, several lensfree holograms of the same static scene are recorded as different LEDs are turned on and off, which creates sub-pixel shifted holograms of the specimens. By using pixel super-resolution techniques, these sub-pixel shifted under-sampled holograms can be digitally put together to resolve/sample much larger portion of the higher spatial frequency oscillations within the lensfree object hologram [‎31, ‎32].

This super-resolved (SR) in-line hologram, however, still suffers from twin-image artifact, which is common to all in-line hologram recording geometries [‎33]. In our earlier work, we demonstrated the use of an iterative object-support based phase recovery method [‎34, ‎35] to eliminate this twin-image artifact creating wide-field microscopic images of samples. This twin-image elimination method, however, requires as input the location estimations of the objects within the imaging field-of-view. For this end, a simple threshold or a segmentation algorithm can be used to automatically estimate the objects’ locations (creating the object-support) for relatively sparse samples. However, in denser specimens, this object support is difficult to estimate which can create challenges in removal of the twin-image artifact.

To overcome these object-support related imaging challenges for dense and connected specimens, here we demonstrate a new approach for pixel super-resolution holographic microscopy that uses multiple (e.g., 2-5) lensfree intensity measurements that are each captured at a different height (i.e., Z₂) from the detector-array (see Fig. 1 ). Stated differently, each lensfree super-resolved hologram is synthesized with ~30-70 µm change in the relative height of the object with respect to the detector-chip surface, after which they are digitally registered and aligned to each other to take into account possible rotations and shifts among these in-line holograms. These co-registered super-resolved holograms, corresponding to different object heights, are then iteratively processed [‎36] to recover the missing optical phase so that microscopic images of the specimens can be automatically reconstructed without the need for any spatial masking steps. Therefore, this multi-height holographic approach eliminates the need to estimate the object-support at the sample plane cleaning the twin-image artifacts of our in-line geometry over a large field-of-view of e.g., ~24 mm² even for dense and connected specimens.

 

Fig. 1 Schematic diagram of the multi-height pixel super-resolution based lensfree on-chip imaging set-up. A partially-coherent source (emanating from a 0.1 mm core fiber-optic cable) creates lensfree in-line holograms of the samples, which are sampled using a sensor-array. In order to reconstruct images of dense samples, multiple intensity measurements at different Z₂-distances (or heights) are captured (see the upper right inset). To reduce the effective pixel-size, a pixel-super resolution algorithm is utilized by source-shifting (see the upper left inset). Since Z₁ >> Z₂ the entire active area of the sensor-array becomes our imaging FOV (e.g., ~24 mm²).

Download Full Size | PDF

Compared to previously reported results, this work demonstrates the first implementation of maskless multi-height phase recovery in partially-coherent lensfree optical microscopy on a chip. Furthermore, this is the first time that pixel-super resolution has been implemented in multi-height phase recovery to digitally mitigate twin image artifact in lensfree in-line holography. A key to the success of pixel super-resolved multi-height phase recovery is actually the use of partially-coherent illumination (both spatially and temporally) as opposedto coherent illumination. Considering that under highly coherent illumination, speckle noise and multiple reflection interference artifacts would create fundamental challenges for digital registration of sub-pixel shifted lensfree holograms of a given height to each other as well as to lensfree holograms of different heights, our partially-coherent illumination scheme is rather important for enabling maskless reconstructions over a large field-of-view of ~24 mm². Based on its unique hologram recording geometry with unit fringe magnification (Fig. 1), the presented technique can work with a spectral illumination bandwidth of e.g., ~5-10 nm and a spatial coherence diameter of e.g., < 0.5 mm at the detector plane. As a result, speckle and multiple reflection interference artifacts can be minimized, which is the key for multi-height pixel super-resolved lensfree on-chip imaging over large field-of-views as demonstrated in this work. We validated the superior performance of this approach by imaging dense Papanicolaou smears (i.e., Pap smears or Pap tests, which are used to screen cervical cancer by detecting premalignant and/or malignant cells in the endocervical canal) as well as blood samples. Providing a light-weight and cost-effective design, this multi-height holographic on-chip imaging platform could be rather useful for wide-field microscopy and pathology needs in resource poor locations as well as in field conditions.

2. Overview of phase recovery methods

In digital holography, the optical phase information of the scattered object field cannot be directly measured and is actually encoded into intensity oscillations of the recorded hologram. Therefore, for reconstruction of images using digital holographic data, phase recovery is of paramount importance. There have been several different approaches to tackle this important problem, and depending on the hologram recording scheme and its complexity, the degree of success varies [‎36-‎42].

The goal of phase recovery in our context is to extract a complex-valued object function from the intensity of its diffraction pattern. Note that in our partially-coherent holographic microscopy scheme described in Fig. 1, since the specimens are placed rather close to the detector array (e.g., Z₂ ~0.7-1 mm), the object field-of-view roughly equals to the detector active area. And due to partial-coherence of illumination there is no longer a single Fourier transform relationship between the entire object and detector planes. In fact, using the transfer function of free-space one can digitally propagate back and forth between the object and detector planes through two successive Fourier transform operations. Therefore, the number of effective pixels (P) at the diffraction sampling plane equals to the number of useful pixels at the object plane. On a related note, the basic function of pixel super-resolution techniques in lensfree digital in-line holography is actually to increase the value of P beyond what the sensor chip can provide at the circuit level [‎31, ‎32].

As a result of this, for a complex-valued object function, phase recovery problem becomes undetermined by a factor of 2 since there are 2 × P pixels defining the object function (P pixels for the real part and P pixels for the imaginary part), whereas there are only P pixels in the measurement matrix [‎43, ‎44]. In order to solve this underdetermined phase recovery problem, new information about the object function needs to be acquired and incorporated as a constraint on the solution space.

To provide a solution to this important problem, numerous iterative phase recovery algorithms were devised [‎37, ‎45-‎50], where various different types of information about the object were used as constraints. These object constraints, together with the measured diffraction pattern were enforced at each step of the iterations, gradually converging to the missing 2D phase information [‎51]. The object-support constraint, which specifies the locations of the objects as a binary mask, is one of the most commonly used constraints [‎34], which also formed the basis of some of our earlier work [‎28, ‎29, ‎52]. This binary mask is a matrix, where a value of one is assigned to the binary mask wherever the object is located, and a zero is assigned wherever the background appears. This way, the object-support constraint or the spatial mask provides the coordinates for which the object function is zero, which effectively reduces the number of unknowns in our underdetermined phase recovery problem.

While quite effective [‎42, ‎53], object-support based iterative phase recovery techniques encounter practical challenges in digital in-line holography when the density of the sample is relatively high, which makes it difficult to create an appropriate binary mask that can effectively reduce the number of unknowns at the object plane. This issue can affect convergence of iterative phase recovery techniques, yielding sub-optimum reconstructions of dense object functions. To overcome these hurdles, several methods were devised which in practice are restricted to sparse objects and/or to relatively small imaging field-of-views [‎54-‎56].

In this work, we utilized an iterative phase recovery method together with a pixel super-resolution technique that uses multiple diffraction intensity measurements of the objects captured at different planes along the light propagation direction. Each additional lensfree diffraction measurement captured at a different object height brings a new set of equations to help us solve this underdetermined phase recovery problem without the use of any object-support constraint or spatial masks. In the literature, there are various methods that one can use to retrieve the lost phase information from two or more diffraction intensity measurements, such as transport-of-intensity equation (TIE) based methods [‎40, ‎41, ‎57, ‎58], iterative methods which use these diffraction measurements as successive amplitude constraints [‎36, ‎59] and other non-iterative methods [‎60, ‎61]. Furthermore, multiple defocused images are also utilized in phase diversity methods, where phase aberrations of incoherent imaging systems can be characterized [‎62-‎64]. In this work, we followed the iterative method presented in reference [‎36] due to its simplicity and robustness. Refer to Sections 4 and 5 for further details of implementation.

3. Pixel super-resolution (PSR) based multi-height lensfree holographic imaging

Our pixel super-resolved multi-height imaging set-up is illustrated in Fig. 1. It is composed of a partially-coherent light source (Xenon lamp attached to a monochromator, with bandwidth of ~5 nm), that is coupled to a multi-mode fiber (100 µm core diameter). The fiber-tip to sample distance (Z1) is ~10 cm, while the sample to sensor distance (Z2) is ~0.7-1 mm (Fig. 1). The detector is a CMOS chip (Aptina MT9P031, 5 Mega-pixel) with a 2.2µm pixel size. Since we employ Z1 >> Z2, the imaging field-of-view (FOV) is only limited by the CMOS sensor’s active area (i.e., ~24 mm² in our case) [‎55].

As discussed in our introduction, in this hologram recording geometry, the main limitation on resolution is the limited pixel size (2.2 µm) at the detector-array, which causes the higher frequency oscillations in our lensfree in-line holograms to be undersampled. To overcome this undersampling issue, we employ source-shifting [‎31, ‎32] to create sub-pixel shifted replicas of the same holographic pattern on the CMOS array such that by using PSR we can digitally synthesize an effectively much smaller pixel size [‎32]. Here we should emphasize that without employing any source shifting or pixel super-resolution techniques, a sub-pixel spatial resolution can already be achieved using our unit fringe magnification hologram recording geometry shown in Fig. 1 (see e.g., references ‎29 and ‎52). In this respect, despite its unit magnification, our holographic imaging set-up has significant advantages compared to a hypothetical “perfect” near-field sampling experiment that utilizes the same pixel size at the detector array to sample the object field [‎29, ‎52]. The main idea behind PSR to further improve spatial resolution is to incorporate few lower-resolution (LR) images of the same object into one SR image [‎65-‎67]. In our multi-height implementation, a slight translation of the fiber-tip in the plane parallel to the CMOS sensor will result in a sub-pixel translated image, during which a single LR lensfree hologram is captured. By translating the source multiple times, an image stack is created, where each lensfree image in this stack is slightly shifted with respect to the others (see Fig. 1). Note also that since Z₁ >> Z₂ there is a large demagnification factor (e.g., >100) between the source-shifting plane and the detector plane so that relatively large shifts at the source plane translate into much smaller shifts at the detector plane, which makes our multi-height imaging approach rather convenient to operate without the need for any fine mechanical alignment or scanners. From this acquired stack of LR lensfree holograms, an SR hologram is synthesized (individually for each object height), which is effectively equivalent to recording the same object hologram with a smaller pixel size detector-array [‎31, ‎32].

4. Image registration among multi-height lensfree SR holograms

In our multi-height imaging based iterative reconstruction scheme, we synthesize M PSR holograms, one for each object height (Z₂) - see Fig. 2 . However, among these M super-resolved holograms there might be spatial discrepancies since the object might have shifted and/or rotated between the acquisitions of each lensfree image stack. Therefore, we perform a digital registration step after the synthesis of these SR holograms to ensure that each hologram essentially looks at the same object field-of-view. This registration procedure is based on arbitrary selection of three points (control-points) in one hologram (i.e., the reference hologram, typically the one with the lowest Z₂-distance) and finding the matching three points in the other M-1 lensfree holographic SR images. These control points should be close to the corners of the image in order to achieve small registration errors (~2 μm or less) over relatively large distances (e.g., ~5 mm). Moreover, these control points should better be circularly symmetric since normalized-correlation, which is used to find the shift of each control-point between the images, is sensitive to rotation. Quite conveniently, the acquired lensfree images are in-line holograms, and an ergo isolated object (like a dust particle) in the object plane resembles an airy function in the hologram domain and exhibits circular symmetry. Therefore, in-line lensfree holograms of these dust particles or substrate defects, which are common in most samples, are desirable candidates to serve as control points. If needed, isolated spatial marks (in the form of e.g., dots) can also be placed to the corners of a sample slide to serve as predetermined control points.

 

Fig. 2 (a) A schematic portraying the image processing that is conducted after acquiring M LR lensfree image stacks at different Z₂-distances. Each image stack is used to create one super-resolved lensfree hologram. These high-resolution holograms are then registered to each other and multi-height iterative phase recovery algorithm is applied. After 1-70 iterations, amplitude and phase images of dense specimens can be reconstructed.

Download Full Size | PDF

After selection of the control points, a small area (e.g., ~30 × 30 μm) around each control point is cropped and digitally interpolated (~4-6 times) to serve as a normalized correlation template. Furthermore, for accurately finding the coordinate shift of each control point among M images, lensfree holographic images have to be positioned in the same Z₂-distance. Therefore, the difference in the Z₂-distance between lensfree holograms acquired at different heights is evaluated by an auto-focus algorithm [‎68, ‎69], which permits us to digitally propagate the selected correlation templates to the same Z₂-distance, where normalized correlations are calculated to find the coordinate shifts between the control points in each image. After each control-point’s coordinates are known in every image, M-1 affine transformations are built to register these lensfree holographic images to the reference image, such that all the lensfree holograms acquired at M different heights can be digitally registered to each other.

5. Phase recovery using M PSR lensfree holograms

In order to retrieve the phase information and consequently eliminate the twin image artifact in our multi-height lensfree imaging approach, a modified Gerchberg-Saxton (GS) algorithm is utilized [‎59]. This modified GS algorithm is based on M intensity measurements that are taken at different Z₂ planes (see Fig. 1). The lowest measurement plane is typically taken with a Z₂ value of ~700 µm (which will be referred as intensity measurement #1). Note that to achieve a measurement with a lower Z₂ distance, the sensor’s protective glass will have to be removed. The highest intensity measurement is typically taken with a Z₂ value of ~1 mm (referred as intensity measurement #M). The distance (∆Z) between two sequential intensity measurements is ~50 µm. We achieved this ∆Z value by using glass cover slips of different thicknesses.

In our iterative phase recovery process, we used the transfer function of free-space [‎29] to propagate the fields back and forth, where the amplitudes of the M intensity measurements serve as constraints at each plane. To initiate the phase recovery process, a zero-phase is assigned to the object intensity measurement. One iteration during this phase-recovery process can be described as follows: Intensity measurement #1 is forward propagated (with zero initial phase) to the plane of intensity measurement #2 (see Fig. 2 green arrow). Then, the amplitude constraint in measurement #2 is enforced while the calculated phase resulting from forward propagation remains unchanged. The resulting complex field is then forward propagated to the plane of intensity measurement #3, where once again the amplitude constraint in measurement #3 is enforced while the calculated phase resulting from forward propagation remains unchanged. This process continues until reaching the plane of intensity measurement #M. Then instead of forward propagating the fields of the previous stages, back propagation is used (see Fig. 2 yellow arrow). The complex field of plane #M is back propagated to the plane of intensity measurement #M-1. Then, the amplitude constraint in measurement #M-1 is enforced while the resulting phase remains unchanged. The same iteration continues until we reach the plane of intensity measurement #1. When one complete iteration is achieved (by reaching back to the plane of intensity measurement #1), the complex field that is derived in the last step will serve as the input to the next iteration. Typically, 1-70 iterations are required for satisfactory results, and for final object reconstruction step the acquired complex field of any one of the measurement planes can be back propagated to the object plane to retrieve both phase and amplitude images of the dense specimens on the chip.

6. Sample preparation

Blood smear samples were prepared using whole blood (UCLA Blood Bank, USA), where the samples were diluted (×2) with RPMI (Thermo Scientific, Catalog #: SH3002701) in room temperature. Then 5μL of the diluted blood was dropped on a type-one glass cover slip (Fisher Scientific Catalog # 12-548-A). The blood droplet was then smeared by a second cover slip by applying a constant force. The sample was then left to dry in air for ~10 minutes before being fixed and stained by HEMA 3 Wright-Giemsa staining kit (Fisher Diagnostics). The Papanicolaou smear (Pap smear) was prepared using a standard SurePath (BD Inc.) procedure [‎70].

7. Experimental results and discussion

A schematic diagram of our lensfree multi-height imaging set-up can be seen in Fig. 1. The set-up is composed of a partially-coherent light source (~5 nm bandwidth centered at 550 nm), glass cover slips with different thicknesses and a CMOS detector-array. The set-up is rather simple to operate without any complicated alignment. For reconstructing dense samples where object-support based phase-recovery approaches face challenges, different lensfree intensity measurements of the sample are acquired at different Z₂ distances.

The Z₂ distance is controlled by placing glass cover slips with different thicknesses between the sample and the sensor-chip. The thicknesses of our glass cover slips varied between 50 µm and 250 µm, hence the corresponding Z₂ distances varied between ~0.7 mm and ~1 mm. Each lensfree intensity measurement is sampled by our CMOS sensor with 2.2 µm pixel size. This relatively large pixel size can cause undersampling issues; therefore, a PSR method is applied in order to effectively decrease the detector pixel size [‎31, ‎32]. For each Z₂-distance a lower-resolution (LR) image stack is captured, where each image in this stack is sub-pixel shifted with respect to the other images in the stack. These sub-pixel shifts are achieved by a slight translation of the fiber-tip position between two sequential images (see Fig. 1. upper left inset).

Figure 2 depicts the image processing steps after image acquisition. For each Z₂-distance, one SR hologram is synthesized from the LR image-stack (typically 16 images in each stack). Then, these M SR images are registered to each other (see Section 4) to account for rotation, translation and shear that may occur during placing/replacing the glass cover slips between each lensfree measurement. For the reconstruction procedure (iterative phase recovery, see Section 5), these M SR intensity measurements are utilized as amplitude constraints (typically 1-70 iterations are required for satisfactory results). Once the phase is iteratively recovered, any one of these M SR images is low-pass filtered (Butterworth 2nd order) to eliminate high frequency noise and the resulting complex image is back propagated to the object plane, retrieving both phase and amplitude images of the specimens on the chip (see Fig. 2).

Figure 3 illustrates the benefits of using the above outlined multi-height lensfree imagingapproach for a blood smear sample. Figure 3(a) shows a full FOV (~24 mm²) LR lensfree hologram as captured by the CMOS sensor. The green dashed rectangle focuses on an area that is rather dense; however the blood cells are still organized as a mono-layer, suitable for imaging. The reconstruction results of this dense blood smear using five different Z₂-distances (711μm, 767μm, 821μm, 876μm and 946μm) are shown in Fig. 3(b). These five Z₂-distances/heights are automatically evaluated by using an auto-focus algorithm [‎68, ‎69]. The reconstruction results of Fig. 3(b) provide a decent agreement to a 10 × microscope objective comparison image shown in Fig. 3(c). Figures 3(d), 3(e) and 3(f) provide images of zoomed areas (taken from the dashed blue rectangle in Fig. 3(b)) of single height back propagation image, multi-height reconstruction image and a 20 × microscope objective comparison image, respectively. The back propagated single height image (Fig. 3(d)) has lower contrast, and it is hard to evaluate the locations of the RBCs for spatial masking purposes. Therefore support-based phase recovery would not be effective in this case. On the other hand, the multi-height amplitude image (Fig. 3(e)) has significantly improved contrast, and individual RBCs can be identified and resolved even in dense clusters. It is important to emphasize that these multi-height reconstruction images shown in Figs. 3(b,e) are obtained without the use of any spatial masking or any other prior information regarding the sample.

 

Fig. 3 (a) Full FOV, LR hologram. (b) Multi-height based PSR lensfree amplitude image of a dense RBC smear is shown. This lensfree image was reconstructed using five different heights (λ = 550nm). The FOV corresponds to the green dashed rectangular in (a). (c) A 10 × objective lens (0.25NA) microscope image is provided for comparison. (d) A single height back propagated PSR amplitude image. The image FOV corresponds to the dashed blue rectangular in (b) and (c). (e) Multi-height based PSR lensfree amplitude image acquired using five different heights is shown. This FOV corresponds to the same FOV as in (d). (f) A 20 × objective lens (0.4 NA) microscope image is also provided for comparison purposes.

Download Full Size | PDF

After these blood smear experiments, next we imaged Pap smears (based on SurePath automated slide preparation [‎70]) using the same multi-height imaging set-up. Figure 4 summarizes our imaging results for this Pap smear. Because of the density of the specimen, the reconstruction of this image is a challenging task for any phase recovery method. Figure 4(a) shows the multi-height phase image, which is recovered using lensfree measurements from five different heights (754μm, 769μm, 857μm, 906μm and 996μm - these Z₂-distances were automatically determined using an auto-focus algorithm [‎68, ‎69]). Figures 4 (c,h) and 4(d,i) show zoomed images of the same Pap smear sample, for amplitude and phase channels, respectively. In these reconstructed multi-height images the cell morphology is clear and their boundaries can clearly be seen and separated from the background. Moreover, minor overlaps among the cells do not constitute a limitation in this method. As a comparison, Fig. 4(b) depicts a single height back propagated phase image corresponding to one of the Z₂ measurements (the FOV is the same as in Fig. 4(a)). It is evident that distinguishing the cells from the background is a difficult task in this dense reconstructed image. To better provide a comparison, Figs. 4 (f,k) and (g,l) also show zoomed images of the same Pap smear sample, for phase and amplitude channels, respectively, calculated using back propagation of a single height image. Compared to Figs. 4(d,i) and 4(c,h), these single height back projection images show significant spatial distortion due to the density of the cells. Figures 4(e) and 4(j) also provide 40 × objective lens (0.65NA) microscope comparison images for the same zoomed regions, clearly providing a decent match to our multi-height reconstruction results shown in Figs. 4(d,i) and (c,h). Especially note the enhanced contrast of the cell boundaries in our phase images (Figs. 4(d,i)), which is complementary to the spatial information coming from our amplitude images (Figs. 4(c,h)). This complementary set of information that is conveyed by the amplitude and phase images might facilitate detection of abnormal cells within a Pap test that are characterized for instance by a high nuclear-cytoplasmic ratio. It is also important to note that all the phase images reported in our manuscript are wrapped; hence, for the multi-height reconstructed phase images, phase jumps should be expected in absorbing areas of the cells (e.g. nuclei), where phase would not be properly defined. Contrary to the phase images, these absorbing areas will be of high contrast in their corresponding lensfree amplitude images, which once again emphasizes the complementary nature of phase and amplitude lensfree image channels.

 

Fig. 4 (a) Multi-height based PSR lensfree phase image of a Pap test is shown. This image was reconstructed using five heights. 36 iterations were used during phase recovery (λ = 550nm). (b) Single height back propagated PSR phase image is shown. (c) and (d) are multi-height based PSR lensfree amplitude and phase images, respectively, of the green dashed rectangle shown in (a). The absorbing nuclei of the cells are clearly visible in the amplitude images, while the cell’s boundaries are more visible in the phase image. The corresponding 40x (0.65NA) microscope image is provided for comparison in (e). (f) and (g) are the corresponding single height based back propagated phase and amplitude images respectively. (h) and (i) are multi-height based PSR lensfree amplitude and phase images, respectively, of the blue dashed rectangle in (a). The corresponding 40 × (0.65NA) microscope image is also provided for comparison in (j). (k) and (l) are the corresponding single height based back propagated phase and amplitude images respectively. All the phase images in the figure are wrapped since we did not employ phase unwrapping algorithms.

Download Full Size | PDF

Next we investigated how the number of intensity measurements used in our iterative reconstruction process affects the image quality (see Fig. 5 ). To provide a fair comparison (i.e., to better isolate the source of improvement in image quality), a total of 144 Fourier transform pairs were used in each case, regardless of the number of intensity measurements employed in the multi-height based phase recovery. Moreover, all the phase images are wrapped and the same global phase was assigned to all of the images to avoid different phasejumps in different images. Figure 5(a) shows a single height back propagated phase image. When a second intensity measurement is added, multi-height based iterative phase recovery approach can be utilized. Consequently, the recovered phase image after 72 iterations (see Fig. 5(b)) looks significantly better than the phase image of Fig. 5(a). A further improvement in image quality is achieved by adding a third intensity measurement to the multi-height phase recovery process (Fig. 5(c)). After 36 iterations (i.e., corresponding to a total of 144 Fourier transform pairs as before), the cells that were hidden in the noisy background are now visible (see white arrows in Fig. 5(c)). A moderate improvement is noticed in the image contrast when adding more intensity measurements, as can be seen in the reconstructed multi-height phase images from four and five heights (Fig. 5(d) and 5(e), respectively). Note that in these two cases, 24 and 16 iterations were used, respectively, so that the total number of Fourier transform operations remains the same in all reconstructions shown in Figs. 5(b-e), which helps us to isolate the source of the phase reconstruction improvement and relate it to multiple height measurements rather than the number of back-and-forth digital propagation operations. Figure 5(f) shows a microscope comparison image (10 × , 0.25NA) for the same region of interest. Note also that the cell’s boundaries are more visible in our phase images, while the absorbing nuclei of the cells are better visualized in our amplitude images as illustrated in Fig. 4.

 

Fig. 5 Pap smear reconstruction results acquired for different number of lensfree diffraction intensities. (a) Back propagated image from one PSR lensfree hologram. (b), (c), (d) and (e) Multi-height based PSR lensfree phase images from two, three, four and five heights, respectively (same color bar as in (a)). For fair comparison 144 Fourier transform pairs were used in each reconstruction case. (f) 10 × objective lens (0.25NA) microscope image is provided for comparison purposes. The cell’s boundaries are more visible in our phase images, while the absorbing nuclei of the cells are better visualized in our amplitude images as illustrated in Fig. 4.

Download Full Size | PDF

After validating the usefulness of pixel super-resolved multi-height based phase recovery approach with dense blood smears and Pap tests, we experimentally tested its impact on the reconstructed image quality. An important question that we aimed to address with this additional experiment was whether or not the digital cross registration process among different Z₂ lensfree holograms results in spatial smearing of our reconstructed images. Therefore we compared the imaging performance of our multi-height reconstruction results against a single back-propagated super resolved hologram. For this end, we imaged an isolated ‘UCLA’ pattern that was etched on a glass slide using focused ion beam (FIB) milling, where the letters ‘U’ and ‘C’ are ~1 µm apart. We emphasize here that for such an isolated object multi-height based image reconstruction is not necessary. Since we aim to understand the impact of multi-height cross registration related issues, in this final experiment we chose an isolated object (‘UCLA’) so that the back-propagation result of a single height SR hologram could work for comparison purposes. This is quite different from the dense objects/specimens reported in Figs. 3-5, where back-propagation of a single height lensfree PSR hologram fails, requiring the use of multiple height measurements.

For this final experiment, the single height back propagated SR holographic image is shown in Fig. 6(a) , where the letters ‘U’ and ‘C’ are clearly separated. The ‘UCLA’ pattern is spatially isolated from nearby objects, and therefore for this small isolated FOV phase recovery is not necessary as emphasized earlier. Figures 6(b), 6(c), 6(d) and 6(e) show multi-height based reconstructed amplitude images, for two, three, four and five different heights, respectively (λ = 490nm). For fair comparison among these recoveries, once again the number of Fourier transform pairs was kept constant in each case, as a result of which each reconstruction used a different number of iterations (60, 30, 20 and 15 iterations, respectively). It is evident that the letters ‘U’ and ‘C’ are clearly separated in all of these images, which is an indication of our success in cross registration of different height super-resolved holograms to each other so that spatial smearing affects due to possible inconsistencies among different Z₂ lensless holograms are minimized. A microscope comparison image of the same “UCLA” pattern can also be seen in Fig. 6(f), acquired using a 40x objective lens (0.65 NA).

 

Fig. 6 Adding intensity measurements from different Z₂ distances does not degrade the image resolution. An important question that we aimed to address with this additional experiment was whether or not the digital cross registration process among different height lensfree holograms results in spatial smearing of our reconstructed images. Therefore we compared the imaging performance of our multi-height reconstruction results against a single back-propagated super resolved hologram. (a) Single height based back propagated PSR amplitude image. (b), (c), (d) and (e) are multi-height based PSR lensfree amplitude images, where two, three, four and five different heights were used in the reconstruction process, respectively (λ = 490nm). For fair comparison the number of Fourier transform pairs was equal in each case, such that each reconstruction used different number of iterations. In all of these reconstructed images, the letters “U” and “C”, with a spacing of ~1 µm, are clearly separated. (f) Microscope comparison image of the same sample (40 × objective lens; 0.65 NA).

Download Full Size | PDF

8. Conclusions

We demonstrated that by incorporating pixel super-resolved partially-coherent lensfree imaging with a multi-height phase recovery approach, dense and connected samples can be reconstructed over a large FOV (~24 mm²). The multi-height iterative phase recovery approach uses multiple super-resolved intensity measurements, which are acquired at different sample-to-sensor distances. Unlike mainstream object-support based phase recovery methods, the multi-height based phase recovery approach does not need a spatial mask for objects. As a proof of concept, this pixel super-resolved partially-coherent multi-height lensfree imaging approach was tested on Pap smears and whole blood samples. Color imaging results are not shown here, however using the same multi-height lensfree imaging approach at three different wavelengths (e.g., λ = 450 nm, 550 nm and 650 nm) and combining these lensfree images to generate color images of the specimens should be straightforward (see e.g., Ref. [‎71]). Finally, we should also mention that translating the same lensfree microscopy platform into reflection geometry is also feasible, although a reduction in imaging field of view would be expected as demonstrated in Ref. [‎72].