Open Access
Add to BibSonomy
Abstract
An integral imaging microscopy (IIM) system with improved depth-of-field (DoF) using a custom-designed bifocal polarization-dependent liquid-crystalline polymer micro lens array (LCP-MLA) is proposed. The implemented MLA has improved electro-optical properties such as a small focal ratio, high fill factor, low driving voltage, and fast switching speed, utilizing a well-aligned reactive mesogen on the imprinted reverse shape of the lens and a polarization switching layer. A bifocal MLA switches its focal length according to the polarization angle and acquires different DoF information of the specimen. After two elemental image arrays are captured, the depth-slices are reconstructed and combined to provide a widened DoF. The fabricated bifocal MLA consists of two identical polarization-dependent LCP-MLAs with 1.6 mm and f/16 focal ratio. Our experimental results confirmed that the proposed system improves the DoF of IIM without the need for mechanical manipulation.
© 2017 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Optical microscopy magnifies micro objects and presents an enlarged visualization of a specimen to the observer. It is used extensively in various fields, especially in the medical and educational fields [1]. Advances in this technology have attempted to optimize the amount of information acquired per sample, including three-dimensional (3D) information. Among them, integral imaging microscopy (IIM) acquires true 3D information of the specimen with lateral and axial visualizations [2, 3]. The IIM system is based on an integral imaging technique [4] in which a micro lens array (MLA), located in front of the camera, captures an elemental image array (EIA).
Several methods have been proposed for IIM resolution enhancement [5, 6]. An important consideration in this regard is the depth-of-field (DoF) of the IIM. Lim et al. showed that IIM has markedly extended DoF versus typical two-dimensional (2D) optical microscopy approaches, but is still insufficient for specimens with wide depth information [7]. Double display devices [8] and non-uniform lens array-based time-multiplexing [9] have been applied to enhance the DoF of IIM; however, these methods are more suitable for improving the DoF of the integral imaging display than IIM. Recently, Kwon et al. proposed a DoF-enhanced IIM system based on spatial and peristrophic multiplexing methods. In the spatial multiplexing method, two identical cameras capture the EIAs through two MLAs with different focal lengths that focus on different depth planes of the specimen [10]. For the peristrophic (rotational) multiplexing method, two focal distances of the MLA are recorded onto a single holographic optical element, capturing two EIAs according to the rotation of the MLA [11]. These methods improve the DoF of the IIM system markedly. However, the mechanical rotation and imperfect matching of the devices limits the use of IIM in real applications.
The switchable liquid-crystal (LC) lens has been extensively studied for 2D/3D switchable displays, integral imaging 3D displays, and integral imaging endoscopes [12, 13]. To develop an IIM system using a switchable MLA, the switchable MLA should be implemented with improved optical properties that include a high fill factor, small f-number, small focal ratio, and square-type 2D MLA shape. One method for implementing the LC-based switchable lens is to use a gradient refractive index (GRIN) lens profile to control the LC director via patterned electrodes [14–17]. However, it is difficult to obtain a 2D MLA shape suitable for electro-optic display devices with these switchable GRIN lens types, due to the limitation of the refractive index properties of LC molecules. Notably, most of the GRIN lenses are customized to the specific application.
Another method to implement a switchable lens utilizes aligned LC molecules on the imprinted reverse shape of the lens [18–20]; this approach provides a small f-number and a high fill factor compared with the GRIN lens type. However, the desired properties of a low driving voltage and fast switching speed limit this approach due to the necessity of switching the thick LC layer. Recently, Kim et al. demonstrated a polarization-dependent switchable liquid-crystalline polymer (LCP) lens array developed with improved electro-optical properties, specifically a small f-number, high fill factor, low driving voltage, and fast switching speed, using well-aligned reactive mesogen (RM) [21] on the imprinted reverse shape of the lens and a LC-based polarization switching layer [22]. The developed lens can be implemented on a flexible substrate as a stack structure with several RM lenses. It has been confirmed that this type of lens is particularly suitable for microdisplays.
In this research, we implement a DoF-enhanced IIM system using a bifocal switchable polarization-dependent LCP-MLA containing an aligned RM that is capable of switching its focal length according to the polarization axis of incident light via a LC-based polarization switching layer. This approach addresses the limitations of existing LC-based MLA methods, in which the conventional method is supplemented with a more practical bifocal LCP-MLA.
2. Proposed system
Figure 1 presents a schematic configuration of the proposed IIM using the bifocal LCP-MLA and LC-based electrical polarization switching system. The main structure consists of the objective and tube lenses (i.e., an infinite optical system), the bifocal LCP-MLA with electrical polarizer, and a camera sensor.
Fig. 1 Schematic diagram of the optical layout of the proposed integral imaging microscopy (IIM) system with switchable bifocal liquid-crystalline polymer micro lens array (LCP-MLA).
The basic structure and principle of operation of the polarization-dependent bifocal LCP-MLA is illustrated in Fig. 2. The bifocal LCP-MLA is designed as two switchable MLAs separated by an index-matching layer. To switch the focal mode of the bifocal LCP-MLA, the LC-based polarization switching layer is utilized. If the polarization switching layer applies 0 voltage, polarized light passes through the polarization switching layer at 0° and is transformed into linearly polarized light at θin = 90°, as shown in Fig. 2(a). When linearly polarized light (θin = 90°) is incident, the polarization-dependent MLA1 focuses at the fLA1 distance, the focal length of the first mode, due to the mismatch between the extraordinary refractive index of the RM1 layer of MLA1, ne1 = 1.68, and the refractive index of the optically isotropic layer of MLA1, np1 = 1.51 (presented as OIL in Fig. 2). In this case, MLA2 cannot act as a lens because the ordinary refractive index of the RM2 layer of MLA2, no2 = 1.51, and the refractive index of the optically isotropic layer of MLA2, np2 = 1.51, are identical.
Fig. 2 Operational principle of the implemented polarization-dependent bifocal LCP-MLA according to the polarization direction of incident light: (a) focusing on the effective focal plane fLA1 with θin = 90ο and (b) focusing on the effective focal plane fLA2 with θin = 0ο.
On the other hand, if the polarization switching layer applies voltage of 10 V, the 0° polarized light passes through the polarization switching layer and is maintained as linearly polarized light without changing the polarization angle (θin = 0°), as shown in Fig. 2(b). Thus, when linearly polarized light (θin = 0°) is incident, the light rays travel without refraction though MLA1, because the ordinary refractive index of the RM1 layer of MLA1, no1 = 1.51, and np1 are equal in MLA1. However, MLA2 operates as an MLA due to the mismatch between the extraordinary refractive index of the RM2 layer of MLA2, ne2 = 1.68, and np2. Light rays passing through MLA2 are then focused at a distance of fLA2.
Thus, polarization-controlled incident light rays pass through the bifocal LCP-MLA with two focal distances along the z-axis, depending on the characteristics of the LCP-MLA. The two MLAs have identical focal lengths, but can focus on different depth planes of the specimen due to the index-matching layer between the MLAs. Note that it is much difficult to overcome the trade-off relationship between the shorter focal length and response time of lens only by the passive MLAs where the thickness of the LC layer should be increased to achieve the short focal length; but the response time of the lens becomes slower when the LC layer becomes thicker. But it can be overcoming with the polarization-dependent bifocal LCP-MLA of the proposed IIM system.
As mentioned above, conventional LC-based switchable lenses cannot provide the combined features of a short focal length (up to 1 mm), high fill factor (up to 100%), low driving voltage (up to 10V), and fast response time (up to 10 ms), especially for small display panel and imaging applications. Also, the controllable distance of the gap between the two MLAs is limited. Note that the custom-designed bifocal LCP-MLA can be manufactured on a flexible substrate as a slim, laminated structure (e.g., for endoscopy applications) as opposed to LC lens arrays that require a rigid substrate, such as glass. Thus, the LCP-MLA can be applied to a wide range of applications, including those that are curvature-based.
Figure 3(a) shows a schematic illustration of the depth ranges that can be acquired through the bifocal active MLA. Given that there are two focal planes, the two depth ranges may be separated or they may overlap. Figure 3(b) shows the bifocal LCP-MLA that satisfies the overlap condition, in which the total DoF is described as:
where ΔzIIM1 and ΔzIIM2 are the depth ranges in both focal modes; z1 and z2 are the central depth planes that can occur in both modes; d is the overlapped part of both ranges, given by d = (3z2-z1)/2M2; λ is the wavelength of the illumination; g is the gap between the bifocal LCP-MLA and camera sensor; NALA is the numerical aperture of each elemental lens of a bifocal LCP-MLA; PS is the pixel pitch of the camera sensor, and M is the magnification of the objective lens. When d = 0, Δztot achieves the maximum value corresponding to the linear sum of the two depth ranges, as follows:Note that in the above calculation, we considered that the focal length of each MLA is identical; however, the g parameters differ in that g for MLA2 is considered as “g + the width of the index-matching layer”. If d is negative in value, the two ranges do not overlap (i.e., they are completely separated). Under the separated condition, the DoF of the IIM system cannot be optimized.Fig. 3 (a) Different focusing of bifocal LCP-MLA according to the polarizing direction switching and (b) two depth-of-field (DoF) ranges generated by both focal modes.
Based on Eqs. (1) and (2) described above, we analyzed the expected enhancement of the DoF of the proposed IIM system. Figure 4 shows the simulation results of a typical 2D microscope, a conventional IIM system (DoFs of each MLA of bifocal LCP-MLA), and the proposed IIM system using a bifocal LCP-MLA (illumination wavelength, λ = 532 nm; magnification: 5 × ; numerical aperture, NA = 0.14; fLA = 1.6 mm; g1 = 2 mm; g2 = 2.7 mm; width of the index-matching layer: 0.7 mm). In Fig. 4, the DoF of the proposed IIM (Δztot) system is illustrated as a red line with circle symbols; the DoF of a conventional IIM system with MLA1 and MLA2 (ΔzIIM1 and ΔzIIM2, respectively) is illustrated by blue and magenta lines with triangle symbols, respectively, and the DoF of a typical 2D microscope (ΔzO) is illustrated as a black line. Note that the DoF of the 2D microscope is fixed by a specification objective lens [3, 7]. The DoF information of the proposed and conventional IIM systems increased with the z value, as shown in Fig. 4. Since the DoF can be less or equal with the DoF of conventional 2D microscope in the conventional IIM case with z (z1 or z2) is less than 4.5 mm basically [7], the proposed IIM system using a bifocal LCP-MLA has a higher DoF value than the conventional IIM system. Here, the z parameter was measured over the range of 2 to 10 mm in 0.5-mm increments, as z should be more than fLA or 1.6 mm. Also, Δztot cannot be more than twice the value of ΔzIIM1 or ΔzIIM2, because z1 > z2 in every case, due to the difference between g1 and g2 given by g2 = g1 + 0.7, where g1 ≤ 3 mm constitutes the longer z, as shown in Fig. 3(a). Figure 4 confirms that the proposed method can improve the DoF of the IIM system, with further enhancement expected with adjustment to the bifocal LCP-MLA.
Fig. 4 Analysis for the expected improvement of DoF through the proposed IIM system using an electro-switchable bifocal LCP-MLA.
3. Experimental results
Figure 5 shows a photograph of our IIM prototype, a bifocal LCP-MLA, and an electrical-switching LC polarizer. To verify the DoF enhancement of the IIM prototype, we manufactured a bifocal LCP-MLA using a square MLA template with a pitch of 100 μm, a focal length of 800 µm, and a f/16 focal ratio; notably, 700 µm of the index-matching layer was used as spacing between the MLAs. The fabricated bifocal LCP-MLA exhibited shifting of the focal plane according to the incident polarized light from θin = 90° and θin = 0°. Note that the focal length of the reference square MLA, 800 µm, was doubled during fabrication of the bifocal LCP-MLA. fLA1 was 1.6 mm, and fLA2 was 2.3 mm (1.6 mm of the original focal length + 700 µm corresponding to the width of the index-matching layer). The fabricated bifocal LCP-MLA was applied to the IIM system, as shown in Fig. 5 (wavelength, λ: 532 nm; numerical aperture of the objective lens: 0.14; magnification: 5 × ; switching speed of polarization switching layer: 5 ms). The gap between the bifocal LCP-MLA and camera sensor was configured as g ≈3 mm. The resolution of captured EIAs were 2048 × 2048 pixels as same as the resolution of camera sensor where the pixel pitch of camera sensor is 6 µm.
Fig. 5 Photographs of the prototype DoF-enhanced IIM system using a bifocal LCP-MLA and liquid-crystal (LC)-based electro-switching polarizer.
3.1 Analysis for a bifocal active MLA
Figure 6 shows the optical properties of the manufactured bifocal MLA. As described above, a bifocal LCP-MLA was fabricated such that the focal plane was shifted by 700 µm from MLA1 to MLA2, based on the incident polarization state. To verify the optical properties of the fabricated bifocal LCP-MLA, we captured images from the specific focal planes using a polarization optical microscope. Figure 6(a) shows the image taken by changing the incident polarized light from 0° and 90° at the effective focal plane fLA1 position when θin = 90°. At the effective focal plane fLA1 position, we verified that 90°-polarized light with a short focal distance was focused where the 0°-polarized light with long focal distance was defocused. Figure 6(b) shows the image taken by changing the incident polarized light from 0° and 90° at the effective focal plane fLA2 position when θin = 0°. In the effective focal plane fLA2 position, the 90°-polarized light with short focal length was defocused, and the 0°-polarized light with long focal distance was focused. Figures 6(c) and 6(d) show the intensity of each position along the white dashed lines of Figs. 6(a) and 6(b). As verified earlier, the intensity of the 90° and 0° polarizations were focused at the fLA1 and fLA2 positions, respectively, and high-intensity and sharp beam shapes were observed. Because the custom-designed switchable bifocal LCP-MLA has a fast response time, we expected that the DoF-enhanced IIM was capable of measuring a moving object.
Fig. 6 Optical properties of the implemented bifocal active MLA: (a, b) polarization optical microscope images of fabricated bifocal LCP-MLA with different focal position according to the change of polarization direction and (c, d) intensity profiles corresponding to the white dotted lines in Figs. 6(c) and (d).
3.2 Image acquisition and reconstruction
In the implemented IIM system, the difference between the focal modes was measured as approximately 50 μm by changing the focal length of the bifocal LCP-MLA, according to the on/off operation of the polarizer. Figure 7 shows the captured EIAs and corresponding reconstructed depth slices for the fruit fly specimen. The entire size of the fruit fly is approximately 400 µm, and approximately 250 µm area is captured. From Figs. 7(a) and 7(b), the MLA focused on the wing (bottom portion) or the eye (upper portion), according to the on/off switch of the polarizer. When the depth-slices from two EIAs were reconstructed, DoF information for the specimen was acquired through the bifocal active MLA. Here, the depth-slices with the 461 × 461 pixels were reconstructed from 0 to 56 µm at intervals of 8 µm. Note that the area marked by the white circles in each depth-slide are the well-focused portions.
Fig. 7 Captured elemental image arrays (EIAs; EIA1 and EIA2) and reconstructed depth slices for a fruit fly specimen: from (a) MLA1 and (b) MLA2 (Visualization 1).
Figures 8(a) and 8(b) show the additional captured EIAs and the corresponding reconstructed depth slices from MLA1 and MLA2, respectively. Here we chose a surface-mounted resistor as a specimen which has same size with a specimen “fruit fly”, approximately 250 µm, where entire size is approximately 350 µm. Similar to Fig. 7, we marked the well-focused areas as white circles with dashed lines. From the reconstructed depth slices, the two modes of the manufactured bifocal active MLA capture different DoF information of the specimen; therefore, different DoF visualizations can be reconstructed.
Fig. 8 Two captured EIAs and reconstructed depth slices for a surface-mounted resistor: from (a) MLA1 and (b) MLA2 (Visualization 2).
From the experimental results presented in Figs. 7 and 8, the proposed IIM system using a bifocal LCP-MLA has a wider DoF than the conventional IIM case. Especially, the proposed system presents sufficient enhancement of the DoF of IIM when all of the optical devices are fixed, i.e., mechanical movement and rotation (in the previous cases using passive MLAs) are not required with our approach). Thus, the proposed configuration provides an efficient means to implement IIM with wide DoF information, especially in practical applications.
4. Summary
In this paper, a DoF-improvement method for IIM using a bifocal LCP-MLA and electro-switching LC polarizer is proposed. The basic structure of the proposed IIM system is similar to the design of conventional IIM systems, except that the common MLA is replaced with a polarization-dependent switchable bifocal LCP-MLA. All of the optical devices are fixed; thus, mechanical manipulation, such as shifting or rotation, is not required. The bifocal LCP-MLA, a planar-convex-oriented photo-curable liquid-crystalline polymer RM, switches the focal length according to the polarization angle, θin = 0° or θin = 90°, to capture two EIAs. The custom-designed bifocal LCP-MLA can be fabricated on a free form-type film substrate. Given its small f-number and fast response time, this system is expected to be applicable to fields that require real-time imaging. Our experimental results confirm that the proposed method enhances the DoF of IIM by approximately two or more times. Thus, our proposed method provides a simple and effective way to enhance the DoF of IIM. For the lateral resolution, it is almost similar with previously proposed IIM systems due to the same camera and similar MLA is used in the experiment. Additionally, the reconstructed image quality can be improved when the higher-resolution camera (larger number and smaller size of the pixels) and smaller pitch of microlenses are utilized. Further research will consider the enhancement of the reconstructed image quality.
Funding
Korea government (GK17C0200); Korea government (NRF-2017R1A2B4012096).
Acknowledgments
This work was supported by 'The Cross-Ministry Giga KOREA Project' grant funded by the Korea government (No. GK17C0200, Development of full-3D mobile display terminal and its contents) and supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (NRF-2017R1A2B4012096).
References and links
1. D. B. Murphy and M. W. Davidson, Fundamentals of Light Microscopy and Electronic Imaging, 2nd ed. (Wiley-Blackwell, 2012).
2. J.-S. Jang and B. Javidi, “Three-dimensional integral imaging of micro-objects,” Opt. Lett. 29(11), 1230–1232 (2004). [PubMed]
3. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” Proc. SIGGRAPH06, 924–934 (2006).
4. N. Kim and M.-U. Erdenebat, 3-D Integral Photography (SPIE Press, 2016).
5. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009). [PubMed]
6. K.-C. Kwon, J.-S. Jeong, M.-U. Erdenebat, Y.-T. Lim, K.-H. Yoo, and N. Kim, “Real-time interactive display for integral imaging microscopy,” Appl. Opt. 53(20), 4450–4459 (2014). [PubMed]
7. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Analysis on enhanced depth of field for integral imaging microscope,” Opt. Express 20(21), 23480–23488 (2012). [PubMed]
8. S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. 42(20), 4186–4195 (2003). [PubMed]
9. J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes,” Opt. Lett. 28(20), 1924–1926 (2003). [PubMed]
10. K.-C. Kwon, M.-U. Erdenebat, M. A. Alam, Y.-T. Lim, K. G. Kim, and N. Kim, “Integral imaging microscopy with enhanced depth-of-field using a spatial multiplexing,” Opt. Express 24(3), 2072–2083 (2016). [PubMed]
11. K.-C. Kwon, Y.-T. Lim, C.-W. Shin, M.-U. Erdenebat, J.-M. Hwang, and N. Kim, “Enhanced depth-of-field of an integral imaging microscope using a bifocal holographic optical element-micro lens array,” Opt. Lett. 42(16), 3209–3212 (2017). [PubMed]
12. Y.-C. Chang, T.-H. Jen, C.-H. Ting, and Y.-P. Huang, “High-resistance liquid-crystal lens array for rotatable 2D/3D autostereoscopic display,” Opt. Express 22(3), 2714–2724 (2014). [PubMed]
13. M.-K. Park, H. Park, K.-I. Joo, H.-D. Jeong, J.-C. Choi, and H.-R. Kim, “Continuous viewing angle distribution control of liquid crystal displays using polarization-dependent prism array film stacked on directional backlight unit,” J. Opt. Soc. Korea 20, 799–806 (2016).
14. M. Ye, B. Wang, and S. Sato, “Liquid crystal lens with focus movable in focal plane,” Opt. Commun. 259, 710–722 (2006).
15. M. Ye, B. Wang, and S. Sato, “Realization of liquid crystal lens of large aperture and low driving voltages using thin layer of weakly conductive material,” Opt. Express 16(6), 4302–4308 (2008). [PubMed]
16. C.-T. Lee, Y. Li, H.-Y. Lin, and S.-T. Wu, “Design of polarization-insensitive multi-electrode GRIN lens with a blue-phase liquid crystal,” Opt. Express 19(18), 17402–17407 (2011). [PubMed]
17. J. Kim, J. Kim, J.-H. Na, B. Lee, and S.-D. Lee, “Liquid crystal-based square lens array with tunable focal length,” Opt. Express 22(3), 3316–3324 (2014). [PubMed]
18. H. Ren, Y.-H. Fan, and S.-T. Wu, “Liquid-crystal microlens arrays using patterned polymer networks,” Opt. Lett. 29(14), 1608–1610 (2004). [PubMed]
19. H. T. Dai, Y. J. Liu, X. W. Sun, and D. Luo, “A negative-positive tunable liquid-crystal microlens array by printing,” Opt. Express 17(6), 4317–4323 (2009). [PubMed]
20. S.-J. Hwang, Y.-X. Liu, and G. A. Porter, “Improvement of performance of liquid crystal microlens with polymer surface modification,” Opt. Express 22(4), 4620–4627 (2014). [PubMed]
21. C.-J. Yun and J.-K. Song, “Functional films using reactive mesogens for display applications,” J. Inf. Disp. 18, 119–129 (2017).
22. K.-B. Son, M. Kim, M.-K. Park, and H.-R. Kim, “Polarization-dependent microlens array using reactive mesogen aligned by top-down nanogrooves for switchable three-dimensional applications,” J. Opt. Soc. Korea 19, 265–271 (2015).
