1. Introduction

Integral imaging [1] permits the recording of the spatial and the angular information of 3D scenes under incoherent illumination. In recent years, the interest in this technology has experienced a spectacular growth, and has been renamed in other ways, such as lightfield technology [2], or plenoptics technology [3]. This technology can be utilized, for example, to compute depth maps [4], to remove occlusions [5] or to perform tracking of micrometer-sized specimens [6]. Biomedical applications are also remarkable, such as the use of plenoptic technology in otoscopy [7], ophthalmology [8], endoscopy [9], and for deep-tissue inspection [10,11].

An application of integral imaging that deserves special attention is 3D microscopy [12]. There exist three different modes for applying the Lippmann concept to microscopy. The first mode was proposed first by Levoy et al. [13] and is based on inserting a microlens array (MLA) just at the image plane of the host microscope. The integral microscope mounted according to this operation mode will be named here as IMic. The main advantage of this IMic is that it provides a high number of perspective images. The main drawback is that the spatial resolution of the views is determined by the competence between the MLA pitch, and the size of the point-spread function (PSF) of the host microscope. As a result, up to now IMic has demonstrated capacity for obtaining views with a spatial resolution that in the best case is about 10 times worse than that of the host microscope [14,15].

The second mode, denoted as integral-imaging microscopy is based on the so-called plenoptic 2.0 concept [16]. In this case, the MLA is inserted at an intermediate plane within the image space of the host microscope. The main advantage of this design is that it is possible to obtain views with a resolution that is more close to that of the host microscope [17,18]. The main drawback is that the number of views is small and therefore the 3D effect is strongly limited. In the third mode, the MLA is inserted at the Fourier plane of the host microscope [19–21]. As in the second mode, this microscope provides views with good resolution, but at the cost of reducing the number of views. The main advantage of this design is the longer depth of field of views images.

Focusing our attention in the IMic mode, it is remarkable that in the state of art it is assumed that it can work accurately only when the PSF of the host microscope is significantly smaller than the MLA pitch. This has limited strongly the resolution of the views provided by IMic [13–15]. Note however that in the past, no systematic study has been performed in order to find which is the acceptable ratio between the host PSF and the MLA pitch. Thus, no research has been focused in exploring which is the real resolution frontier of IMic; i.e., how much can the resolution of view images to the host resolution approach? In this context, the aim of this paper is to obtain perspective views with the best resolution ever achieved in integral microscopy. To get this aim we design an ad hoc experiment that permits to explore and find such frontier. As results of this experiment we demonstrate that integral microscopy can provide perspective images with resolution close to the one provided by 2D conventional microscopy.

2. Basic theory of IMic

To build an IMic, the microscope objective (MO) and the tube lens (TL) of the host microscope are placed in telecentric configuration. Then, a MLA is placed at the image plane, where the camera sensor was originally located, and the latter is shifted by a distance that is equal to the focal length of the MLA, $f_{\text{ML}}$ (see Fig. 1

 

Fig. 1 Scheme of an IMic.

Download Full Size | PDF

With this configuration, an IMic records an integral image that is comprised by a set of microimages that carry the 3D information of the sample. The number of microimages is equal to the number of usable microlenses in the array. To avoid any overlapping between the recorded microimages, the numerical aperture in the image space of the MO, NA’, and the numerical aperture of the microlenses, NA_ML, should be equal. This condition can be written as

The spatial resolution limit of the computed perspective views is the result of the competence between two factors [20,21]. One factor is due to diffraction and the other is due to the sensor pixilation. This competence can be expressed as

At this moment, it is convenient to define the dimensionless parameter

The depth of field of the view images can be calculated, also in terms of the coupling parameter [20], as

3. Exploring the validity of resolution formula

IMic is a hybrid technique in which we can distinguish two very different stages. The first stage, the capture of microimages, is a purely optical process. As result, the information held by the microimages is strongly influenced by optical experimental factors, such as aberrations, vignetting or diffraction. The second stage is the calculation of the perspective images from the microimages. This is a purely computational procedure in which it is assumed that each pixel holds the spatio-angular information of a single light ray that passes through the center of the microlens. In the state of art of IMic it is stated that this assumption is acceptable provided that the size of the image host PSF, ${\text{ρ}}_{\text{hst}}^{\text{'}}$, is much smaller that the MLA pitch [15,24,25]. In other words, the state of art assumes that distinguishable perspective views with the resolution predicted by Eq. (5) can be obtained only when $\text{η}<0.2$. Thus, one should expect that higher values of $\text{η}$ would give rise to a strong conflict between the wave nature of microscopic imaging and the para-geometric nature of IMic computations. As result of this conflict, the higher the value of $\text{η}$, the smaller the number of distinguishable perspectives. Thus, it is commonly assumed that, it has no sense to build an integral microscope with $\text{η}>0.2$. For this reason, no high-resolution IMic has ever been built. Nevertheless, a remarkable fact is that for values of $\text{η}>0.2$ there is no any experimental research that explores the capacity of IMic for providing distinguishable perspectives with the resolution predicted by Eq. (5) and the expected parallax. Thus, and in order to fill this gap, we consider necessary to perform a rigorous and systematic experimental study in which many IMic with $\text{η}>0.2$ are implemented and the perspective images are analyzed.

To this end, an IMic like the one sketched in Fig. 2

 

Fig. 2 Scheme of the experimental setup for determining the validity of resolution and DoF formulae.

Download Full Size | PDF

In the first experiment, we used as an object an USAF 1951 resolution target, which was placed at the focal plane of the IMic, and was illuminated with the white light proceeding from a fiber bundle. We captured the set of microimages for six different values of η. In Fig. 3

 

Fig. 3 Results of the experiment for η = 0.25. (a) Microimages recorded by the camera sensor; (b) View images computed from the microimages; (c) and (d) show intensity profiles along resolved (Element 7.6) and non-resolved (Element 8.1) elements, respectively, of the central view image.

Download Full Size | PDF

In particular, we illustrate what is the aspect of the microimages captured optically with the IMic, and of the view images computed from them. We show, as well, the procedure for evaluation of the resolution limit. Specifically, for determining if one element of the USAF target is resolved we integrate the intensity profile along the element and check for the existence of an intensity dip of at least 25%.

The experimental results are summarized in Table 1

Table 1. Summary of Results

View Table

In the first column we write the focal length of the tube lens. This value affects the effective magnification of the host microscope, which can be calculated as

In the second column we write the diameter of the diaphragm. The value ϕ_D = 2.20 mm corresponds to the case in which the diaphragm is totally open and therefore the system aperture is limited by the MO aperture stop. In the last three rows it was necessary the use of small diameters of the diaphragm.

In the third column we write the effective NA of the system. When the diaphragm is open the NA_eff does not change. However, as the diaphragm is set smaller and smaller, the effective NA decreases proportionally.

In the fourth column we have written the host resolution limit, which is calculated with Eq. (3). In column fifth we write the size of the PSF at the image plane. It is calculated as ρ’_hst = M_eff ρ_hst. In the sixth column we write the value of the coupling parameter η.

Finally, in the seventh and eight columns we have written the theoretical, Eq. (5), and measured value of the resolution limit of the view images.

From Table 1 we confirm that for low values of $\eta$ the spatial resolution of the calculated view images is much worse than the host resolution. However, as the value of 𝜂 increases, the difference between the two values of resolution is shrunk. For values of $\eta$ close to, $2.0$ the two resolutions are comparable. But the most important outcome is that the obtained resolutions are the ones predicted by Eq. (5). Thus we can confirm that the validity of such equation is not restricted to values of η < 0.2, but at a much wider range.

4. Confirming the existence of distinguishable perspective views

In the previous Section, we have demonstrated that IMic can produce view images whose resolution is not restricted to ${\rho }_{\text{View}}>10 {\rho }_{\text{hst}}$. However, our work is not finished yet, since we still need to demonstrate that the high-resolution view images have the expected parallax and can be combined to produce high-quality refocused images.

To demonstrate this, we performed a new microscopy experiment, but using now as the object a 3D microscopic sample stained with a fluorescent ink. As the sample we used regular cotton, which from microscopic point of view, is an almost hollow specimen with large depth range and composed by long fibers with thin structure. The cotton was stained with fluorescent ink proceeding from a marking pen, and illuminated with the light proceeding from a laser of wavelength ${\lambda }_{\text{exc}}=532 nm$. The emission wavelength was around ${\lambda }_0=600 nm$. Thus, a chromatic filter (${\lambda }_{\text{c}}=550 nm$) was used to reject the laser light. For this experiment we used the IMic setup corresponding to $\eta =0.50$. Note that we could no use for this experiment higher values of η, because they require (for the MLA available in our Lab.) for the insertion of a diaphragm that reduces the effective NA of the system and therefore the captured parallax. Anyway, even working with this value of η the IMic can produce view images with, by far, the best resolution (${\rho }_{\text{View}}=2.2 \mu m$) ever achieved in integral microscopy.

The captured microimages are shown in panel (b) of Fig. 4

 

Fig. 4 Practical IMic operating at η = 0.50. (a) Central view image of USAF target, together with intensity profile (Element 8.6, corresponding to ρ_View = 2.2 μm, is resolved); (b) Microimages of the cotton sample; (c) Calculated view images; and (d) Central view image.

Download Full Size | PDF

Additionally, we have prepared Visualization 3 and Visualization 4 with the images obtained with η = 0.25. Note that in this case resolution is worse than with η = 0.50, but the DoF is longer, as expected from Eq. (6).

5. Conclusions

In summary, in this letter we have researched for the spatial resolution capability of IMic. Contrarily to what it is assumed in the IMic community, we have demonstrated that IMic can produce view images with high resolution, and that these view images have the expected parallax and the capacity of depth refocusing. Apart from the proof-of-concept experiment, we have implemented an IMic that has produced view images with the best resolution ever achieved in IMic.

It is remarkable that in the past some interesting research was addressed to design new digital processing algorithms, see for example [24], for improving the resolution of refocused images calculated from low-resolution view images. Although the design or use of such algorithms is not the aim of our research, naturally this kind of computational tools could be applied as well to the high-resolution views obtained with IMic operating with high η.