Open Access
Add to BibSonomy
Abstract
A micro-plenoptic system was designed to capture the three-dimensional (3D) topography of the anterior iris surface by simple single-shot imaging. Within a depth-of-field of 2.4 mm, depth resolution of 10 µm can be achieved with accuracy (systematic errors) and precision (random errors) below 20%. We demonstrated the application of our micro-plenoptic imaging system on two healthy irides, an iris with naevi, and an iris with melanoma. The ridges and folds, with height differences of 10~80 µm, on the healthy irides can be effectively captured. The front surface on the iris naevi was flat, and the iris melanoma was 50 ± 10 µm higher than the surrounding iris. The micro-plenoptic imaging system has great potential to be utilized for iris disease diagnosis and continuing, simple monitoring.
© 2017 Optical Society of America
Corrections
15 September 2017: Typographical corrections were made to the author listing and the funding section.
1. Introduction
The iris is the most anterior component of the uveal tract of the eye. It is made of blood vessels, sphincter and dilator smooth muscles, connective tissues, and pigment epithelial cells [1]. The iris serves as a demarcation between the anterior and posterior chambers of the eye. The iris musculature regulates the size of the pupil and thus controls the amount of light reaching the retina.
The iris tissue can be distorted as a result of trauma, infections, anatomic variations, or tumors. Changes to iris morphology can have significant visual consequences for patients [2]. Iris morphologic changes can occur from ocular pathology or from systemic disease. For example, advanced diabetes mellitus can cause iris neovascularization, also known as rubeosis iridis, in which small fine blood vessels develop on the iris [3]. In addition, medications, such as those for benign prostatic hypertrophy (BPH), can change iris muscular tone, causing iris structure change and complications during ocular surgeries [4].
Techniques capable of measuring the structure of patient’s iris can provide valuable information for accurate evaluation of disease. Currently, existing techniques to evaluate iris structure include anterior segment optical coherence tomography (AS-OCT) and ultrasound biomicroscopy (UBM) [5]. AS-OCT was first demonstrated in 1994 [6] and became commercially available in 2001 [7]. AS-OCT is based on low-coherence interferometry measuring the time delay and intensity of back-reflected light from tissue structures at various depths. It is a noncontact method for imaging the anterior segment using an infrared super-luminescent light-emitting diode (1310 nm), achieving a resolution of 18 μm and a penetration of 3 to 4 mm. This method allows a detailed assessment of the anterior chamber structures with high-resolution cross-sectional images.
UBM [8] is a well-established imaging technique to assess ocular structures. UBM rapidly records successive axial measurements at different transverse locations. UBM for anterior segment imaging achieves a resolution of 25 μm with a penetration of 5 to 6 mm. However, UBM requires the patient’s eye to be anesthetized because the probe directly contacts the eye using a water eyebath. The patient must lie in a reclined position during imaging. Furthermore, a sub-specialized ophthalmic photographer is required because of difficulty of the imaging technique. Because of the technical difficultly, UBM is not widely available in clinical practices.
Three-dimensional (3D) imaging the human eye has great potential value for ophthalmologists who must assess three dimensions for surgical planning. Currently, 3D eye imaging is realized by combining a sequence of 2D cross-sectional OCT [9] or ultrasound images [10]. However, the iris is a dynamic tissue and its shape and configuration may be altered with eye motion; consequently, iris anatomy can change during recording of the multiple 2D cross-sectional images required. Therefore, a simple-to-use, rapid and non-invasive technique to image 3D iris patterns in a single photograph may have clinical value.
Plenoptic cameras (also called light-field cameras) enable capturing the origin and direction of each light ray in a single image by mounting a micro-lens array (MLA) in front of the camera sensor. The resulting image can then be processed to obtain the 3D information of the object. The basic concept of plenoptic imaging was proposed a century ago by Lippmann [11] and was then advanced further by Chutjian [12] with the first digital imaging device. Adelson et al. [13] proposed a plenoptic camera design that significantly reduced the correspondence problem in stereo imaging. Raytrix GmbH introduced the first commercial plenoptic camera in 2010. Detailed theory and methods of plenoptic photography can be found in [14–17].
In this study, we demonstrate a micro-plenoptic 3D imaging technique for 3D iris imaging with a single-exposure image. The purpose is to create a simple diagnostic technique to map the 3D iris pattern and to monitor changes over time for iris pathology.
2. Micro-plenoptic imaging system
Figure 1 depicts the micro-plenoptic system for in vivo human iris 3D imaging. The system was designed and built by mounting a plenoptic camera onto a standard slit lamp (Topcon SL-3D) [18]. The observation system of the slit lamp was removed, and the lighting source was retained to provide side illumination for the iris. As shown in Fig. 1, a 29-megapixel (MP) color plenoptic camera (R29, Raytrix GmbH), an extension tube and Nikon 105 mm lens were assembled to image the eye. The extension tube was set at 150 mm, and the main lens was adjusted for the imaging system to have a magnification of 2.3, which was determined based on the size of the camera sensor and common human iris size. The field-of-view (FOV) of the system is 15.8 × 10.5 mm2, and depth-of-field (DOF) was 2.25 mm as computed by the RxOptics software from Raytrix GmbH [19]. The f-number of the main lens was fixed at 2.8 to match the aperture of the micro-lenses [19]. The f-number was set so the micro images would not overlap and would not have gaps between neighboring micro images, both of which have negative consequences on 3D reconstruction accuracy. The depth resolution, accuracy and precision were experimentally assessed as described in the following section.
The imaging system was connected to a computer by a Camera Link cable to enable camera control and data transfer and was operated with commercially available software (RxLive 3.1, Raytrix GmbH) to record, process, and display the images. Iris images were recorded at 5 frame/second. An exposure time of 100 milliseconds was selected to use the maximum dynamic range of the camera sensor without saturating the camera. Imaging required approximately 10 seconds to generate a sequence of images (~50) for each eye.
The micro-plenoptic imaging system was calibrated for each camera or lens setting. First, the reconstruction software was calibrated for the geometry of the MLA; this process creates a calibration grid using homogeneous illumination. Then, the 3D physical dimensions were determined through the spatial calibration procedure [20, 21] which records a sequence of target images at different orientations. The calibration target, which has a dot array with 0.3 mm dot diameter and 0.5 mm dot spacing, was placed at an oblique angle to the camera axis with the goal of covering the entire desired depth-of-field. After these two calibrations, the plenoptic imaging system and reconstruction software could produce 3D images of approximately 7 MP size, reduced from the full-sensor dimensions due to the trade-off between spatial and angular resolution inherent in plenoptic cameras [22].
3. Characterization of micro-plenoptic imaging system
The lateral resolution of the system is 18 µm for the whole depth of field, which was determined using a 1951 USAF resolution test target. To quantify the accuracy and precision of depth measurement for the micro-plenoptic imaging system, an experiment was designed as shown in Fig. 2. A printed paper card target was fixed on a micrometer translation stage (Newport M-UMR12.80 and BM32.80). The translation stage offers a resolution of 10 µm, sensitivity of 1 µm, and travel range of 80 mm. This setup allowed moving the target accurately in depth (z) direction to create a true displacement between two images. The goal was to quantify 10 µm displacements measured over a range of distances from the camera. This criterion is matched to the dimensions and the overall depth range that needs to be resolved for human iris imaging [2].
Fig. 2 A setup to experimentally assess the accuracy and precision of depth measurement (along z direction) of the micro-plenoptic imaging system. A paper target was utilized as the imaging target and fixed on a translation stage. The farthest plane that can be imaged by the system was defined as z = 0 mm.
The farthest focal plane was determined and defined as reference plane at z = 0 mm, and the direction from imaging target to the camera was defined as the positive z-direction. Then, image pairs with displacements of 10 µm in the z-direction were recorded for each position from z = 0 mm to z = 4.8 mm in increments of 0.2 mm. 50 image pairs were recorded at each z-position. The z-direction position of a letter on the paper target was measured by the micro-plenoptic imaging system for all 50 image pairs. The z-direction displacement between each image pair was then computed. The average and standard deviation (shown as error bar) of the 50 measurements were plotted, along with the true displacements, in Fig. 3(a). At z<1 mm, i.e. the far field, the measured average displacement is inaccurate and the standard deviation shows the measurement within this range is not reproducible. Within the near field (z>3.8 mm), the true displacement was underestimated. In the center field (1 mm ≤ z ≤ 3.8 mm), the difference between plenoptic measurement and true displacement is within 2 µm, and the standard deviation is approximately 2 µm. The different behavior in the above three regions is a result the geometry of the main lens, MLA, and imaging sensor. In the far field, only a few micro-lenses can image the same object from a narrow angle, leading to unreliable 3D reconstruction. In the near field, many more micro-lenses can image the same object, but the signal-to-noise ratio of the image is low once the imaging range goes beyond the depth of field of the micro-lenses [23]. Therefore, only the center field retains a sufficiently wide imaging angle and good signal-to-noise ratio, which allows good 3D reconstruction. A more detailed discussion of these effects was presented in the context of 3D flow measurements, but is equally applicable here [24].
Fig. 3 (a) Comparison between measured z-direction displacement with the true displacement. The average ± one standard deviation are shown. (b) The depth measurement accuracy (quantified as measurement error) and precision (quantified by standard deviation divided by the average of 50 measurements) of the micro-plenoptic system.
Accuracy and precision are quantified as shown in Fig. 3(b). The accuracy is expressed as the percent difference between measurement value and the true value, with positive values meaning that the measurement is larger than the actual value and vice versa. The precision is the spread of several repeated measurements of the same quantity, and it is quantified as the standard deviation divided by the average from 50 measurements, expressed as a percentage. As shown in Fig. 3(b), the accuracy and precision are both within 20% in the range of 1.4 mm ≤ z ≤3.8 mm. At both z <1.4 mm and z >3.8 mm, the depth measurement is deemed too inaccurate for the purposes of iris imaging, and the repeatability is dramatically reduced. Therefore, we conclude that the desired 10 µm depth resolution can be achieved within a 2.4 mm depth-of-focus (DOF) for the micro-plenoptic imaging system. The 2.4 mm DOF agrees with the built-in calculation from the camera operating software (RxOptics).
4. Clinical results and discussion
This pilot study was conducted at the W.K. Kellogg Eye Center in the Department of Ophthalmology and Visual Sciences at the University of Michigan in Ann Arbor. Institutional review board approval was obtained from the University of Michigan Medical School for the protection of human subjects.
Figure 4 depicts the plenoptic results from an in vivo healthy human iris. Figure 4(a) is the computationally re-focused image with the focus at iris, i.e. the equivalent of a conventional 2D photograph. The eyelashes were out-of-focus in this image, but blocked part of the iris. No 3D iris structure can be obtained for regions with eyelashes. The bright spot on the right side in Fig. 4(a) is the direct reflection from the illumination source, and hence no 3D reconstruction was obtained for that location, due to the image sensor saturation. Appropriate 3D reconstruction was achieved in the center region of iris. Figure 4(b) shows the 3D iris topography of this healthy iris structure in the central 3.9 mm × 4.9 mm area. The detailed fine-structured radial folds in this region were measured. The geometry along the z direction, the height along line MN (defined in Fig. 4(b)) is plotted in Fig. 4(c). The farthest focus plane is defined as z = 0 mm, the direction from the iris towards the camera is the positive z-direction. Peaks and valleys with height differences of 20 ~80 µm along line MN are noted. The observed depth modulation is bigger than the experimentally quantified accuracy and precision errors (see Section 3), thus giving confidence in the instantaneously measured topography of the iris.
Fig. 4 (a) A computationally re-focused iris image. (b) The 3D iris pattern viewed at an angle, and (c) peaks and valleys on the iris can be identified. z = 0 mm is defined at the focus plane that farthest away from camera.
Figure 5 depicts the result of a healthy iris from the right eye of a second participant. Figure 5(b) utilizes color-coded depth mapping to visualize the relative height of iris regions on the iris surface. A depth point field was first computed by the 3D reconstruction procedure, unreliable 3D reconstruction was filtered out, and then a depth-fill interpolation algorithm was applied to create a smother “depth map.” More details regarding the processing procedure can be found in [25]. The farthest focal plane is defined as z = 0 mm, and the out-of-plane direction is defined to be the positive z-direction. Two radial ridges and part of the circular folds, as highlighted by the solid and dashed white lines, respectively, in Fig. 5(a), are clearly captured with height variation of approximately 40 µm. Figure 6 illustrates the iris from the left eye of this second participant, who had an iris naevi [2] in the 4-5 o’clock region of the iris. As shown in Fig. 6, the topography of the iris has been altered by the iris naevi, which affects the surrounding anatomy. The iris is stretched unevenly and the pupil is more oval in shape. In addition, the surface on the iris navei is flat. As shown in Fig. 6(c), the height varies less than 2 µm in the naevi region. Even though peaks and valleys can still be observed at the neighboring healthy region, the height of the peak in this region is reduced from typically up to 50 µm to approximately 15 µm.
Fig. 5 (a) Computation re-focused iris image for the healthy left eye, with rectangular highlights the 3D reconstruction region; (b) The height on iris represented by color (The focal plane that farthest away from the camera is defined as z = 0. Out-of-plane direction is positive z direction.)
Fig. 6 (a) Computation re-focused iris image for right eye, with rectangular highlights the 3D reconstruction region and dashed triangle highlights the iris naevi; (b) the color-coded image shows the height on iris, the naevi region is highlighted by dashed line; (c) the height of the iris structures along line AB clearly shows the structural differences at the naevi.
Figure 7 is a plenoptic image of the left eye of a patient with iris melanoma [2]. The melanoma leads to significant pupillary distortion, with a long diameter (line AB) 30% larger than the short diameter (line CD). Unfortunately, the large melanoma at the lower part of the iris, highlighted by the black dotted oval, could not be reconstructed due to the shadowing artifact of the eyelashes in this exposure. The small melanoma in the upper right, highlighted by the rectangle, with a length in both x- and y-direction of approximately 1 mm, was captured clearly. As shown in Fig. 7(b), this iris melanoma is 50 ± 10 µm (standard deviation based on the initial system evaluation) higher than its surrounding iris.
Fig. 7 A ~1 mm iris melanoma, where the depth is 50 ± 10 µm higher than the surrounding iris, is recognized.
5. Summary and future work
A simple-to-use single-camera micro-plenoptic system was designed to measure the 3D human iris topography with a non-contact single-shot imaging approach. The imaging setup was calibrated, and it was experimentally shown that 10 µm depth resolution can be achieved within a depth range of 2.4 mm, in which the accuracy and precision for depth measurement are both within 20%. The imaging system was used to photograph the 3D iris structure of patients, demonstrating that plenoptic imaging provides the opportunity to evaluate iris topography and monitor iris abnormalities and diseases.
Results of 3D images of four human irides were presented here: two normal irides, an iris with naevi, and an iris with melanoma. The height of the anterior iris surface, including ridges and folds, could be measured in all cases. For the healthy irides, peaks and valleys with height difference of 10-80 µm were recognized. In the region of an iris navei, there was a relatively flat topography with small height differences within 2 µm. As iris naevi is the precursor of iris tumor, this type of iris naevi certainly needs to be monitored frequently. The micro-plenoptic 3D imaging technique offers a simple promising way to realize this monitoring. An iris melanoma with dimension about 1 mm along the x- and y-direction showed an elevation above its surroundings of 50 ± 10 µm.
The prototype micro-plenoptic 3D imaging system shows promise in these preliminary investigations but has limitations to be improved upon by optimizing existing technology. First, currently a 100 ms exposure time is required to ensure high signal-to-noise ratios in the raw images. With such long exposure, any eye motion blurs the image and leads to unreliable 3D reconstruction. Replacing the slit lamp with a brighter short-pulsed flash light illumination source will allow decreasing the exposure time and thus freeze the eye movement to address this issue. Second, the current algorithm relies on high-contrast features in the image for the 3D reconstruction, and therefore it does not work for translucent objects (i.e. cornea, etc.). New reconstruction software needs to be developed to enable 3D measurement of the translucent objects in the eye [26]. Third, the calibration of the micro-plenoptic imaging system was conducted using a high-contrast dot-array target [19–21] without the cornea and aqueous humor between the imaging system and the target. This calibration method could reduce the accuracy of the depth computation when measuring the depth on human irides. A new calibration procedure, which employs an iris-like target and mimics the real imaging situation, needs to be developed to understand and quantify these potential limitations. In addition, the current system can be made more compact, or even hand-held, using more recent commercial plenoptic 3D imaging systems. This miniaturization will enable widespread penetration of this novel technique into clinical use.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
Funding
This work was funded through the University of Michigan MCubed program: “3D Plenoptic Imaging in Medical Diagnostics.” Some of this material is based upon work supported by the National Science Foundation under Grant Number CBET 1402707 and also supported by the National Eye Institute under Grant Number K23EY023596.
Acknowledgments
The authors would like to acknowledge Ms. Elaine Lok at the University of Michigan W.K. Kellogg Eye Center for recruiting patients for this study.
References and links
1. L. M. Levine, V. S. Brar, M. H. Goldstein, A. Kahana, W. R. Katowitz, S. K. Law, and D. A. Mackey, “Basic and Clinical Science Course (BCSC) Section 2: Fundamentals and Principles of Ophthalmology,” (American Academy of Ophthalmology, San Francisco, CA, 2016).
2. R. C. Eagle Jr., “Iris pigmentation and pigmented lesions: an ultrastructural study,” Trans. Am. Ophthalmol. Soc. 86, 581–687 (1988). [PubMed]
3. S. Gartner and P. Henkind, “Neovascularization of the iris (rubeosis iridis),” Surv. Ophthalmol. 22(5), 291–312 (1978). [CrossRef] [PubMed]
4. D. F. Chang and J. R. Campbell, “Intraoperative floppy iris syndrome associated with tamsulosin,” J. Cataract Refract. Surg. 31(4), 664–673 (2005). [CrossRef] [PubMed]
5. C. Bianciotto, C. L. Shields, J. M. Guzman, M. Romanelli-Gobbi, D. Mazzuca Jr, W. R. Green, and J. A. Shields, “Assessment of anterior segment tumors with ultrasound biomicroscopy versus anterior segment optical coherence tomography in 200 cases,” Ophthalmology 118(7), 1297–1302 (2011). [PubMed]
6. J. A. Izatt, M. R. Hee, E. A. Swanson, C. P. Lin, D. Huang, J. S. Schuman, C. A. Puliafito, and J. G. Fujimoto, “Micrometer-scale resolution imaging of the anterior eye in vivo with optical coherence tomography,” Archives of Ophthalmology 112, 1584–1589 (1994).
7. S. Radhakrishnan, A. M. Rollins, J. E. Roth, S. Yazdanfar, V. Westphal, D. S. Bardenstein, and J. A. Izatt, “Real-time optical coherence tomography of the anterior segment at 1310 nm,” Archives of Ophthalmology 119, 1179–1185 (2001).
8. C. J. Pavlin, K. Harasiewicz, M. D. Sherar, and F. S. Foster, “Clinical use of ultrasound biomicroscopy,” Ophthalmology 98(3), 287–295 (1991). [CrossRef] [PubMed]
9. J. J. Liu, A. J. Witkin, M. Adhi, I. Grulkowski, M. F. Kraus, A. H. Dhalla, C. D. Lu, J. Hornegger, J. S. Duker, and J. G. Fujimoto, “Enhanced vitreous imaging in healthy eyes using swept source optical coherence tomography,” PLoS One 9(7), e102950 (2014). [CrossRef] [PubMed]
10. D. B. Downey, D. A. Nicolle, M. F. Levin, and A. Fenster, “Three-dimensional ultrasound imaging of the eye,” Eye (Lond.) 10(1), 75–81 (1996). [CrossRef] [PubMed]
11. G. Lippmann, “Epreuves reversibles, photographies integrales,” Comptes Rendus de l’Académie des Sciences 146, 446–551 (1908).
12. A. Chutjian and R. J. Collier, “Recording and Reconstructing Three-Dimensional Images of Computer-Generated Subjects by Lippmann ntegral Photography,” Appl. Opt. 7(1), 99–103 (1968). [CrossRef] [PubMed]
13. E. H. Adelson and J. Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 99–106 (1992). [CrossRef]
14. M. Levoy, “Light Fields and Computational Imaging,” Computer 39(8), 46–55 (2006). [CrossRef]
15. A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Computational Photography (ICCP),2009IEEE International Conference on, 2009), 1–8. [CrossRef]
16. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light Field Photography with a Hand-Held Plenoptic Camera,” (Stanford University Computer Science Tech Report CSTR, 2005).
17. M. Levoy, R. Ng, A. Adam, M. Footer, and M. Horowitz, “Light Field Microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006). [CrossRef]
18. H. Chen, V. Sick, M. Woodward, and D. Burke, “Human Iris 3D Imaging using a micro-Plenoptic Camera,” in Optics in the Life Sciences Congress, OSA Technical Digest (online) (Optical Society of America, 2017), BoW3A.6. [CrossRef]
19. C. Perwaß and L. Wietzke, “Single lens 3D-camera with extended depth-of-field,” in Human Vision and Electronic Imaging XVII, 2012), 829108.
20. C. Heinze, S. Spyropoulos, S. Hussmann, and C. Perwass, “Automated Robust Metric Calibration Algorithm for Multifocus Plenoptic Cameras,” IEEE Trans. Instrum. Meas. 65(5), 1197–1205 (2016). [CrossRef]
21. O. Johannsen, C. Heinze, B. Goldluecke, and C. Perwaß, “On the Calibration of Focused Plenoptic Cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications: Dagstuhl 2012 Seminar on Time-of-Flight Imaging and GCPR 2013 Workshop on Imaging New Modalities, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer Berlin Heidelberg, Berlin, Heidelberg, 2013), pp. 302–317. [CrossRef]
22. T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoffs in integral photography,” in Proceedings of the 17th Eurographics conference on Rendering Techniques, (Eurographics Association, Nicosia, Cyprus, 2006), pp. 263–272.
23. N. Zeller, F. Quint, and U. Stilla, “Depth estimation and camera calibration of a focused plenoptic camera for visual odometry,” ISPRS J. Photogramm. Remote Sens. 118, 83–100 (2016). [CrossRef]
24. H. Chen and V. Sick, “Three-Dimensional Three-Component Air Flow Visualization in a Steady-State Engine Flow Bench Using a Plenoptic Camera,” SAE Int. J. Engines 10(2), 625–635 (2017). [CrossRef]
25. H. Chen, P. M. Lillo, and V. Sick, “Three-dimensional spray–flow interaction in a spark-ignition direct-injection engine,” Int. J. Engine Res. 17(1), 129–138 (2016). [CrossRef]
26. H. Nien, J. A. Fessler, and V. Sick, “Model-based image reconstruction of chemiluminescence using a plenoptic 2.0 camera,” in ICIP 2015 IEEE International Conference on Image Processing, (IEEE, Québec City, Canada, 2015). [CrossRef]
