Open Access
Add to BibSonomy
Abstract
Adaptive optics flood illumination ophthalmoscopy (AO-FIO) allows imaging of the cone photoreceptor in the living human retina. However, clinical interpretation of the AO-FIO image remains challenging due to suboptimal quality arising from residual uncorrected wavefront aberrations and rapid eye motion. An objective method of assessing image quality is necessary to determine whether an AO-FIO image is suitable for grading and diagnostic purpose. In this work, we explore the use of focus measure operators as a surrogate measure of AO-FIO image quality. A set of operators are tested on data sets acquired at different focal depths and different retinal locations from healthy volunteers. Our results demonstrate differences in focus measure operator performance in quantifying AO-FIO image quality. Further, we discuss the potential application of the selected focus operators in (i) selection of the best quality AO-FIO image from a series of images collected at the same retinal location and (ii) assessment of longitudinal changes in the diseased retina. Focus function could be incorporated into real-time AO-FIO image processing and provide an initial automated quality assessment during image acquisition or reading center grading.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
The array of therapeutic options available to clinicians for treating retinal diseases is growing. With these advances comes the need for high-resolution non-invasive imaging of the retina to enable detection of disease progression and measurement of therapeutic efficacy at a cellular level [1]. Current clinical retinal imaging systems, such as optical coherence tomography [2], scanning laser ophthalmoscopy [3] and fundus photography [4] do not have sufficient resolution to image single retinal cells. The imperfections of the eye’s optics limits the resolution of these devices. However, it is now possible to compensate for the eye’s aberrations and provide near diffraction-limited resolution by using adaptive optics (AO) technology [5]. AO can be incorporated into each of the above-mentioned retinal imaging technologies to improve image resolution and visualization of the individual retinal photoreceptor cells, retinal blood vessel walls and capillaries and bundles of ganglion cell axons within the living human retina [6–8].
AO combined with fundus photography is called adaptive optics flood illumination ophthalmoscopy (AO-FIO) and this is the first commercially available imaging modality to image the retina at a cellular level in vivo. The imaging resolution of AO-FIO is approximately 2 µm and the images delivered by the instrument cover a field of view of 4° × 4° (~1.2 × 1.2 mm2) on the retina. With these imaging capabilities, AO-FIO allows the longitudinal analysis of change in cone photoreceptors integrity to infer photoreceptor cell number and geometry, facilitating the study of normal aging, disease progression and treatment effect [9,10]. As with all modalities of imaging, AO-FIO image quality can be reduced by defocus and the presence of cellular debris or other cell types (e.g., retinal pigment epithelium cells) in the photoreceptor layer due to acceptance of out-of-focus back reflected light that reduces signal to noise ratio. In addition, loss of the photoreceptor cells may interfere with its wave-guiding property reducing sharpness of the image [10]. It has been reported that outer segment reflectivity images vary considerably during the day, as well as from day to day. This variation in reflectivity is not completely understood and may arise from multiple factors, including natural morphological variation within cells, refractive index changes associated with photocurrent dynamics, and the coherence properties of the imaging light [11–13]. Moreover, residual uncorrected wavefront aberrations (both low and higher order) and rapid eye motion may also impact on image quality. All these factors contribute to the challenges in clinical interpretation of AO-FIO images and longitudinal analysis [14,15]. Therefore, caution should be taken when interpreting AO-FIO images and it is desirable that AO-FIO images are correlated with other imaging modalities for a more complete understanding of the structural state of the retinal tissue [16]. Since AO-FIO imaging of the photoreceptors is gaining wider acceptance as an imaging tool to monitor disease [9,17,18], there is an unmet clinical need to have a robust method for assessing the quality of post-processed AO-FIO images prior to trial endpoint analysis.
One of the software solutions to improve AO-FIO image quality is to apply image deconvolution to remove the residual uncorrected wavefront aberrations as proposed by a number of authors [19–21]. AO-FIO images can be also pre-processed using a wavelet approach to correct for uneven illumination and then assessed for image quality using a range of variables, from which variance and contrast provided the best results [19]. Similarly, Zhou and colleagues [22] proposed a post-processing method to improve the lack of confocality of an AO system using a HiLo method, while Lazareva [23] proposed a method based on the Hessian-LoG filtering for enhancement and detection of cone photoreceptors reflexes in AO retinal images. Optimizing cone visibility can be also undertaken by band-pass filtering to enhance cone reflexes [24]. Sophisticated deep convolutional neural network methods have also been used to restore degraded AO images [25]; the authors showed that blur can have a significant negative impact on the cone detection performance and visibility. Although these procedures improve the image quality, it is preferable to capture a high quality image in the first instance, rather than to manipulate a poor quality image.
The aim of this work is, therefore, to identify an objective image-quality grading function that can be applied to AO-FIO of retinal photoreceptors. Upon validation, these functions can be used to guide the design of new operators of photoreceptor cell characterization and analysis. Automated techniques for cone detection/counting is an area of active research [24,26–28]. Methods proposed in this paper could be incorporated into real-time AO-FIO image processing and provide an initial automated quality assessment during image acquisition or reading center grading.
2. Materials and methods
2.1. Adaptive optics instrumentation
Retinal photoreceptor images were acquired using flood illumination adaptive optics ophthalmoscopy (AO-FIO) (rtx1, Imagine Eyes, Orsay, France). The instrument is comprised of two parts. The first is a non-contact, en face reflectance retinal imaging device, employing a non-coherent flood illumination light source, with a central wavelength of 850 nm and a low-noise CCD camera. The second part is the AO control loop that measures and simultaneously corrects the ocular aberrations. The apparatus directs a small beam of light generated by a superluminescent diode with a central wavelength of 750 nm into the eye, which then backscatters off the retina. The scattered light leaving the eye is aberrated by the eye's optics before being recorded by the imaging components of the Shack-Hartmann wavefront sensor. In addition, a corrective element – a deformable mirror and a control system, are used to provide real-time closed-loop correction the eye's wavefront aberrations.
A defocused image of the photoreceptor cell mosaic can be achieved by manually changing the focusing depth in the acquisition software. For example, 0 µm depth is set by the device based on the highest reflecting layer in the retina, which is assumed to be the retinal pigment epithelium (RPE). Manually adjusted focus at approximately 40 µm above the RPE is recommended by the manufacturer for imaging of the outer segment portion of the photoreceptor cells located between 2° and 6° from the center of the fovea. Whereas, focusing at 250 µm above the RPE is recommended for imaging of the superficial retinal vessels.
Each AO-FIO image acquisition process records 40 consecutive AO-FIO frames over 4 seconds. The visibility of the cone photoreceptors reflex is improved by the increased signal-to-noise ratio of the AO-FIO image reconstructed from co-registration of individual AO-FIO frames using a cross-correlation method (registration of X/Y and rotation) and averaging. The raw images, which show artefacts due to eye blinking and saccades, are automatically eliminated by the acquisition software during this process. Unless specified, this post-processed, aligned and averaged, AO image is the one used for further analysis. The final AO image corresponds to a 4° × 4° (750 × 750 pixels, oversampled to 1,500 × 1,500 pixels) region in the retina. In linear dimensions, this is approximately 1.2 × 1.2 mm2. The resolution of the system is 250 line pairs per mm, thus limiting the ability to distinguish cone photoreceptors structures that are 2 µm or less. Therefore, the images acquired from this system are not suitable for visualizing cone photoreceptor outer segments within 2.5° from the foveal center nor the outer segments of the rod photoreceptor cells [29].
2.2. Study subjects
All research procedures described in this work followed the tenets of the Declaration of Helsinki. The research protocol was approved by The University of Western Australia Human Research Committee. In a retrospective study, AO-FIO images from the Lions Eye Institute image database was examined (RA/4/1/7662). Clinical records were reviewed for retinal diagnosis as determined by a retinal specialist (FKC). Healthy controls were recruited in a prospective study that investigates the spatial variation of cone density (RA/4/1/7226 and RA/4/1/7457). Eye drops (tropicamide) were used to induce pharmacological pupil dilation and ensure a pupil diameter that was larger than 4 mm, and these drops also cause a partial paralysis of accommodation.
2.3. AO-FIO image analysis to identify optimal image focus measure operator
Poor AO-FIO image quality is predominantly due to uneven illumination and target defocus. Uneven brightness of the image occurs naturally due to shadow artefact from superficial retinal vessels. Cone photoreceptors signal defocus occurs naturally due to the curvature of the retina within the imaging field or it can be simulated by acquiring images outside the recommended optimal focusing depth settings. In this study, we characterized image quality by using an image focus measure operator. A Matlab (MathWorks, Natick, MA, USA) function based on the previous work by Pertuz et al. [30] was adapted for this study. This function implements a number of focus measure operators to measure the relative degree of focus (defocus) of an image. The function includes a large number of methods; for further details on each focus measure operator the reader is referred to [30] and the references therein. For the purpose of this study specific focus measure operators have been chosen as described in section 3.1. Throughout the paper, we refer to the quality and focus of images as the same property.
The image analysis section is further subdivided into three objectives. The first objective was to evaluate the behavior of the different operators in characterizing the image under controlled conditions of image focus or defocus (acquired at different focal depths) and two representative focus measure operators were selected. The second objective was to evaluate the performance of the focus operators at various retinal locations or eccentricities from the fovea. Finally, clinical applications of the method are discussed.
2.3.1. Performance of the image focus measure operators at different focal depths
To characterize and select the most representative focus operator to quantify AO-FIO image quality, a data set of images taken at various focal depths was used. Since the focusing depth of each AO-FIO image is known, the behavior of the different focus functions can be modelled against the extent of defocus from the optimal range. The AO-FIO images were captured at 8 different depths of focus, at −60, −40, 0, + 40, + 80, + 120, + 160 and + 200 μm. The reference depth (0 µm) was set automatically by the instrument based on the participant’s refractive error entered, negative values denote depth below the RPE (into the choroid) while positive values denote depth above the RPE (into the neural retina). The RPE layer is a band of tissue 20 µm thick, so errors in the reference depth position (0 µm) will induce some variability in the instrument’s focusing depth, which can also affect the variably of the acquired image focus. There is also strong reflection originating from the junction between inner and outer segments of cones and cone outer segment tips which may influence the reference depth setting of the instrument. A total of 30 AO images were available at each of the eight focus depths taken at a location 5° temporal to the foveal center in a 35-year-old healthy female subject. Since eye movements can alter focusing depth, a comparison of defocus matrix from 30 consecutive AO-FIO images taken between + 60 µm below the RPE to + 200 µm above the RPE should reduce the impact of saccades or fixation drifts on image focus value. A representative set of images captured at the different focal depths is shown in Fig. 1. To facilitate comparison and visualization of the cone photoreceptor distribution, a zoomed and cropped region (approximately 160 μm2) of the image is shown for each depth at the same retinal location.
Fig. 1 Example of the zoomed region of the acquired images at different focal depths including: (A) −60 μm, (B) −40 μm, (C) 0 μm, (D) + 40 μm, (E) + 80 μm, (F) + 120 μm, (G) + 160 μm and (H) + 200 μm. The reference depth (0 µm) is at the level of the RPE. Negative values denote depth below the RPE (into the choroid) and positive values denote depth above the RPE (into the neural retina).
It should be noted that shifting the focus depth from −60 to + 40 µm improves visibility of the cone photoreceptors segment reflexes [Fig. 1(A), 1(B)]. Clarity of outer photoreceptors is lost when the focusing depth control is moved up from + 120 to + 160 μm [Fig. 1(F), 1(G)]. Therefore, images taken below + 40 µm and above + 120 µm are suboptimal and unsuitable for quantitative analysis of cone photoreceptor cell distribution.
A range of image focus measure operators were used to quantify the quality of focus using the same data set [30]. After the image focus values were calculated for each image (a single focus value characterizes the entire image) across the different focus depths, a Gaussian function was used to model the focus values profile with respect to depth as derived from each function. Although the true shape of the focus function is still under active investigation, the Gaussian function model provides an idealized focus function that corresponds to a smooth bell-shaped peak whose maximum coincides well with the position of best focus [31].
2.3.2. Performance of the image focus measure operator at different locations from the fovea center
The cone distribution and density is known to change across the retina. Several studies have shown that cone spacing increases with increasing eccentricity up to 10° away from the foveal center [10]. To explore the effect of eccentricity on the focus measure operators, we analyzed a total of 680 images from each of the 35 healthy subjects (mean age, 53 years; range 21 to 75 years) enrolled in our studies. There were 21 males and 14 females, mean axial lengths were 24.05 (range 22.56 to 25.59) mm. The images were collected at various retinal locations within 7° from the foveal center. A total of 20 different retinal locations were imaged, with a mean number of 33 images per location (range 28 to 43). The focus values of AO-FIO images at each retinal location were calculated. These AO-FIO images were captured by an experienced operator following the instrument’s recommendations. The focus depth was manually adjusted during image acquisition to improve image quality at the various retinal locations. The frequency images acquired through a focusing depth at −40, 0 and + 40 µm were 40%, 22%, 23% respectively. Other 15% includes single used of focusing depths such as: + 50 µm, + 60 µm, + 80 µm, + 120 µm.
2.4. Application of the image focus measure operators
To test the potential application of the selected focus measure operators, we examined a set of four consecutive images acquired at a single retinal location. The variability of image quality between consecutive AO-FIO images was quantified by comparing the focus values. Moreover, the potential application of the selected focus measure operators in disease was studied using longitudinal data of a patient with acute macular neuroretinopathy.
3. Results and discussion
The behavior of the various operators for measuring image quality is demonstrated in section 3.1. The performance of the focus measure operators at the various retinal locations from the fovea (eccentricity) is reported in section 3.2. Finally, a potential clinical application of the technique for optimizing quality of the AO-FIO image-montage for grading and selection process for cone photoreceptor distribution is demonstrated in section 3.3 and the use of the operator to quantify change in pathology images is presented in section 3.4.
3.1 Validation of focus measure operators against defocused images
Amongst 20 operators tested, 5 failed to capture the predicted behavior of the data (i.e. those methods didn’t provide a clear separation between the different focal depths) and thus were not considered further. A clear separation was defined as a statistically significant difference between focus values across the 8 levels of focusing depth with a one-way ANOVA test. The remaining majority of the operators (15 in total) showed a focus value profile that mimics the expected outcome from subjective grading of the example images. These operators provide a clear separation in focus values between the different depths of focus.
For all focus measure operators, the peak normalized focus values occurred between + 73 and + 97 μm (mean of 85 μm above the RPE), whilst the standard deviation of the Gaussian function was 95 to 128 μm (mean: 103 μm). The Gaussian distribution plots for each of the operators (left side) as well as the parameters for the distribution (spread of the mean and standard deviation) of those functions (right side) are shown in Fig. 2. To evaluate the fit, an R-squared value was calculated for every image focus operator. The group mean R-squared value was 0.6247 [range: 0.5763 to 0.6922]. Given the similar curves between the 15 operators, a strong correlation coefficient between these focus measure operators was found with a group mean value of 0.938 [range 0.796 to 0.998]. Given this strong correlation, the similarities between the Gaussian models and the need to reduce the complexity, 2 of the 15 operators were selected for further analysis. The specific methods were; BREN: Brenner function that uses the first difference between a pixel and its neighboring two points [32,33] following the Eq. (1):
and LAPE: the energy of Laplacian, which convolves a discrete Laplacian mask with the input image, to provide a focus measure operator [34] following the Eq. (2):Where I(i, j) is the gray level of pixel I located at coordinates i, j and M and N are the image dimensions and m = 2 for the Brenner’s function. The two selected focus functions sit close to the group mean as shown in Fig. 2, right side. A set of notched box-plots [Fig. 3] are used to illustrate the behavior of the two selected image focus operators for our healthy retina data set, which contains 30 AO images at each focusing depth. To facilitate comparison, the focus values are normalized between 0 (worst focus) and 1 (best focus). Overall, all the operators show that optimal image focus occurs at 40, 80 and 120 μm above the RPE. However, different operators showed a slightly different median value at different depths, with the 80 and 120 μm consistently displaying a superior performance. This is in agreement with the Gaussian model, which places the peaks of the bell-shaped curves (position of the best focus) between 73 and 97 μm for the various operators.Fig. 2 Gaussian fit for the different considered focus measure operators (left). Mean and standard deviation of the Gaussian probability density function (PDF) parameters for the all the image focus functions with a highlight for the three focus functions used in this study (right). BREN = Brenner's function, CURV = image curvature, GDER = Gaussian derivative, GLLV = gray-level local variance, GLVN = normalized GLLV, GRAE = energy of gradient, GRAT = thresholded gradient, GRAS = squared gradient, LAPE = energy of Laplacian, LAPM = modified Laplacian, LAPV = variance of Laplacian, LAPD = diagonal Laplacian, SFRQ = spatial frequency, TENG = Tenengrad value, TENV = Tenengrad variance.
Fig. 3 Notched box-plots illustrating the normalized focus value at each level of focusing depth. The solid central horizontal line indicates the median change, and the box extends between the 25th and 75th percentile, width of notches in each box represent the 95% confidence interval of the median, whiskers extend to 1.5 times the interquartile range of the data. Possible outliers are display with a circle.
It is worth noting the large variation of normalized focus values in AO-FIO images acquired at the same focus depth. For example, the BREN operators calculated normalized focus values that span 0 to 0.7 in AO-FIO images taken at + 200 μm (above the RPE), Fig. 3. Eye movements, which result in misalignment during AO-FIO image acquisition, are likely to contribute to the occurrence of outliers in focus values. Since the operators calculates the quality of image focus, these extreme values should not be interpreted as a pitfall, but rather the ability of this approach to detect and quantify AO-FIO image variability of the AO-FIO image due to variable illumination. For example, Fig. 4 shows the actual images associated with the extreme focus values. For the lowest extreme focus value (worst focus, left panel) the image shows a blurry mosaic of cone photoreceptors reflexes, while in the highest focus value (best focus for this depth, right panel), most of the cone photoreceptors reflexes are visible. This example illustrates the significant variation in the quality of AO-FIO images taken under the same standard conditions in the same imaging session at the same retinal location and depth. It highlights the need to improve the post-processing of the images to ensure the clinical interpretations are not confounded by these artifacts.
Fig. 4 Example of two images (zoomed region covering about 160 μm2) acquired at + 200 μm (above the RPE). The left side panel shows an example of the poorest quality AO-FIO images (focus features) and right panel presents an example of good image quality (sharp details) chosen from a group of 30 consecutive images taken at the same retinal location.
3.2 Effect of retinal location on normalized focus value
We applied the two selected image focus measure operators to the AO-FIO images collected from healthy controls at various retinal locations within 7° from the foveal center. The global focus values, averaged across the entire image and mapped to its center location were calculated Fig. 5 (left panel) and notched box-plots were used to illustrate the variability of these values grouped into each eccentricity Fig. 5 (right panel).
Fig. 5 Mean values per location (left) and notched box-plots (right) illustrating the normalized focus measure value at different locations away from the fovea. For notched box-plots, the solid central horizontal line indicates the median change, and the box extends between the 25th and 75th percentile, width of notches in each box represent the 95% confidence interval of the median, whiskers extend to 1.5 times the interquartile range of the data. Possible outliers are displayed with a circle. S-superior, I-inferior, N-nasal, T-temporal. Asterisk (*) indicates statistically significant differences (p<0.01) between eccentricity rings.
To illustrate the variability of peak focus values across retinal locations, we present notched box-plots where the x-axis is retinal locations (distance in degrees from the foveal center) and the y-axis is normalized focus values. The focus values decreased (poorer focus) with increasing distance from the foveal center. Focus values were averaged within each eccentricity for each subject and a repeated-measures analysis of variance was carried out to examine the variations in focus values over the four different eccentricities. For both operators, the analysis revealed a significant effect of eccentricity on the focus values. Except for the 1° and 3° rings, the rest of the pair-wise comparison between the rings of eccentricity showed statistically significant differences (p<0.01). This finding was consistent across the two focus measure operators. The variation in focus value with retinal eccentricity could be due to physiological changes in the spacing and number of cones with distance from the foveal center. Additionally, the increasing number of rod photoreceptors with eccentricity, though unable to be resolved by AO-FIO, may also reduce the sharpness of the image. Representative images at each eccentricity are shown in Fig. 6. The lower density of the cone mosaic with increasing eccentricity results in a decrease in the focus value, since there are fewer well-defined structures (i.e. cones) in the image.
Fig. 6 Example of four images (zoomed region covering about 160 μm2) acquired at different locations away from the fovea. Degrees away from the fovea; 1° (A), 3° (B), 5°(C) and 7° (D).
3.3. Application of the focus measure operators in image selection
To demonstrate the potential clinical application of the focus measure operators in the selection of optimal image, a series of AO-FIO images were acquired at the same retinal location and depth. In some participants, repeated images showed a range of focus values. Figure 7 illustrates an example of four consecutively acquired images from a healthy subject. The top row presents these images in the order they were acquired. After calculation of the global focus value for each image using the LAPE focus operator, these images were ranked from worst to the best focus (bottom row). Results from BREN operator (not shown here) produced the same order of the images. The ranking by focus value concurs with subjective evaluation. Looking at each image from Fig. 7, images 2 and 3 showed a close overall global focus values (0.5% difference) while 1 and 4 showed a 4% and 10% decrease in the focus value respectively, when compared to the other images. From a practical perspective, the image with the best focus is also the one most suitable for image segmentation and analysis. This application of the proposed focus measure operator is particularly useful when the image has a good focus across the entire frame.
Fig. 7 Four consecutive images captured at the same retinal location, sorted by the order they were captured and labelled 1-4 (top). These same images were sorted by the global focus values (bottom). The image with the highest focus value (far-right image in the bottom row) has the sharpest cone reflex across the entire frame.
In some cases, all of the acquired images from the data set may have suboptimal quality across some portion of the image frame; in other words there is not a single image from the data set that displays sharp focus across the entire image. It may then be best to apply a similar principle (best image selection based on the focus measure operator) only to a sub-region of the image. Thus, the sub-regions with the highest focus values can be selected for creating a final cone photoreceptors image. For example, the focus values (LAPE) were calculated for consecutive windows of 32 × 32 pixels (corresponding to 25.6 μm2) across the image frame and for all the images at the same retinal location. The windows with the best focus were used to create an “optimal” AO-FIO image. Results from BREN (not shown here) showed an 82% agreement on the selected regions of best focus. A Gaussian low pass function that removes any subtle changes between windows can be used to smooth the resultant image for optimal visualization. Pyramid blending methods [35] or other more sophisticated image fusion techniques [36] may also be used in future iterations.
Figure 8 shows an example for a healthy subject with four consecutively acquired images at the same retinal location. The left panel presents the data labeled in the order that the images were acquired. It is clear that only a portion of each of the overlapping images is in good focus (i.e. distinct cone pattern). We applied the sub-region selection of the best focused image and blended these together to create a composite image which has clearly defined features across the entire frame (right panel).
Fig. 8 Four consecutive images captured at the same retinal location, sorted by the order they were captured (left panel). The three best focus sub-region and the final blended image are shown in the right panel.
3.4. Application of the focus measure operators to support analysis of pathology
Longitudinal AO-FIO images from a 49-year-old female with acute macular neuroretinopathy were used to explore the impact of photoreceptor pathology on focus measure operator. AO-FIO, spectral domain optical coherence tomography and microperimetry (a technique to measure subjective contrast threshold in a localized region of the retina) were performed on 4 consecutive clinic visits over an eighteen months period. Figure 9 (top panel) shows the mean focus values for the AO-FIO images collected at different retinal location within 3° from the foveal center. Figure 9 (middle panel) shows a set of AO images centered at a particular retinal location (1°T, 1°S) collected during the different visits, while the bottom panel shows the en face map of interdigitating zone (mean reflectivity map generated using the OCT images) centered at the fovea, also collected at the different visits.
Fig. 9 Longitudinal focus value data of a patient with acute macular neuroretinopathy. Top row presents the maps for the mean focus values at each retinal location across 4 clinic visits. Changes in the focus value correspond with changes in the AO-FIO image (middle panel) and the corresponding en face OCT image of the interdigitation zone (bottom row). AO-FIO images illustrate the change in cone distribution and appearance, with an increased cone visibility in the last two visits. The en face map shows a more homogenous reflectivity distribution at the interdigitation zone over time. The outline of the AO-FIO image (dotted) corresponds with the outline in the focus value maps (top row) centered at the retinal location (1°T, 1°S). The OCT en face map (bottom row) covers the field of view of the AO mean focus values (top row).
AO images at different visits show an increased visibility of the cone mosaic. There was also a trend for increasing focus values over time in several retinal locations. This trend corresponded to recovery of the interdigitation zone as visible on OCT en face maps of interdigitation zone and improving retinal function on microperimetry. For example, improved image focus at location (1°T, 1°S) Fig. 9 (middle row) with an normalized focus values of 0.26, 0.37, 0.38 and 0.45 across the different visits correlated with resolving defects seen in the en face OCT image of the interdigitation zone Fig. 9 (bottom row) and increased subjective retinal sensitivity from 7 to 23 dB (visit 1 to 4). The specific AO location was selected since it covers most of the area of changes observed in the OCT en face maps, however it is worth noting that in the current analysis the AO mean focus values provides a macro characterization of the tissue (across a 4° x 4° area) while compared to the more localized microperimetry values.
4. Discussion and conclusions
In this study, we evaluated the use of focus measure operator to assess cone photoreceptors visibility in AO-FIO images. AO-FIO images of the retina may provide fundamental knowledge of cone density and distribution in both- health and disease [9,10]. Due to the narrow focal depth of the technique, AO-FIO image quality (focus) can fluctuate significantly during image acquisition. This variability in focus is likely to be linked to a range of factors, including: (i) saccadic or fixational eye movements during image acquisition, (ii) uncorrected wavefront aberrations, (iii) incorrect manual adjustment of the image plane during image acquisition and (iv) curvature of the retina precluding simultaneous focus at the level of cone photoreceptors across the whole image frame. It is worth noting that, although participants had eye drops instilled to increase pupil size, the accommodation was not completely paralyzed to eliminate the effect of the unwanted residual accommodation. Given that the AO closed-loop system operated at 9.6 fps, some residual accommodation or microfluctuations of accommodation cannot be ruled out. Regardless of the underlying reason, the resultant image is suboptimal (out of focus) and this limits the accuracy of cone density and distribution measurements. Although a number of post-processing techniques have been proposed to improve the quality of AO-FIO images [19–24], it is preferable to capture high quality image in the first instance than to manipulate a poor quality image. Due to the complexity of cone AO imaging procedure, the photographer needs an objective and rapid measure of the AO image quality at the time of image acquisition to decide if additional images are required at the same region before moving to the next region of interest. In this study, we concentrated on images of the cone mosaic. However, the AO system can also be used to image other structures of the posterior segment of the eye. The application of the proposed method to those different tissues could be considered in future work.
The operators proposed for a surrogate measure of AO image quality may inform the photographer, in real-time, and provide a tool to ensure the quality of the captured images is maximized based on an objective measure. In a practical clinical scenario, the focus measure operator could be embedded into the image acquisition software to inform the photographer at the time of image acquisition whether more images should be taken, or alternatively, to aid in the selection of the best portions of a set of images taken at the same location to create the highest quality composite image. This application of the techniques has been demonstrated in this study, to both (i) grade the image quality from a set of AO images and (ii) generate an image mosaic from portions of multiple overlapping images to improve visibility of cones photoreceptors reflexes across the frame. In this study, the use of the focus measure operator to assess the impact of pathology was also presented. The operator was used to quantify images of a patient with acute macular neuroretinopathy. This condition leads to loss of cone photoreceptor and its spontaneous restoration over several months. We demonstrated the effect of pathology on reducing focus values (appears to be more defocused), and correlation between recovery of function and cone photoreceptor visibility during follow-up. However, for this particular example only a single AO measure was collected per retinal location and visit, so measurement variability is also likely to affect the reported findings. It should be noted, however, that the motivation of this work is to assess the use of the focus measure operator as a surrogate measure of image quality rather than disease severity. Although the method may be used in the future to assess disease severity, it is important, first, to understand the effect of photoreceptor disease on the focus measure operators. We are currently monitoring a cohort of patients with photoreceptor degeneration. We plan to investigate the clinical utility of these operators as a biomarker of disease or disease progression in this natural history cohort.
The proposed image focus measure operator provides a focus/defocus value, which could be used by a clinician as a surrogate indicator of image quality. Despite the strong correlation between all the evaluated methods, given the lack of a standard reference of image quality for this particular application, it was not possible to assess if the chosen two methods are optimal. This is a limitation of the current method. Values below a certain threshold should be interpreted as potentially “inadequate quality for analysis” and as a result, additional capture of the image in the same region of the eye is advisable. The aim here is to ensure artefacts are recognized early and removed if possible during image acquisition to reduce their impact on clinical interpretation. The work developed in this study analyzed the AO-FIO images after they have been post-processed (registered and averaged) to improve the image quality and reduce noise. However, the methods proposed in the study could also be incorporated into this processing step and may enhance the results from the post-processing. Quantification of image quality is a fundamental step prior to image analysis. In this study, a range of focus measure operators were used as a surrogate marker of image quality, and focus values can be used to inform the photographer regarding potential issues pertaining to the image that has been acquired, or can be used in a post-processing scheme to select the best quality portion(s) of the image to be analyzed or stitched in a montage. The ultimate goal is to maximize the quality of the data acquired from the AO-FIO instrument so that quantification of image features (cones photoreceptors reflexes) can be performed. This should also reduce potential misinterpretation of the data in clinical trials and routine clinical care settings.
Funding
Ophthalmic Research Institute of Australia (DMS, FKC); Bayer Global Ophthalmology Awards Program (FKC); National Health and Medical Research Council (APP1142962, FKC); Telethon – Perth Children’s Hospital Research Fund (DAC, DMS, FKC).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References and links
1. J. Carroll, D. B. Kay, D. Scoles, A. Dubra, and M. Lombardo, “Adaptive optics retinal imaging-clinical opportunities and challenges,” Curr. Eye Res. 38(7), 709–721 (2013). [CrossRef] [PubMed]
2. D. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, “Optical coherence tomography,” Science 254(5035), 1178–1181 (1991). [CrossRef] [PubMed]
3. R. H. Webb and G. W. Hughes, “Scanning laser ophthalmoscope,” IEEE Trans. Biomed. Eng. 28(7), 488–492 (1981). [CrossRef] [PubMed]
4. G. A. Williams, I. U. Scott, J. A. Haller, A. M. Maguire, D. Marcus, and H. R. McDonald, “Single-field fundus photography for diabetic retinopathy screening: a report by the American Academy of Ophthalmology,” Ophthalmology 111(5), 1055–1062 (2004). [CrossRef] [PubMed]
5. J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. A 14(11), 2884–2892 (1997). [CrossRef] [PubMed]
6. A. Uji, S. Ooto, M. Hangai, S. Arichika, and N. Yoshimura, “Image quality improvement in adaptive optics scanning laser ophthalmoscopy assisted capillary visualization using B-spline-based elastic image registration,” PLoS One 8(11), e80106 (2013). [CrossRef] [PubMed]
7. S. Zayit-Soudry, J. L. Duncan, R. Syed, M. Menghini, and A. J. Roorda, “Cone structure imaged with adaptive optics scanning laser ophthalmoscopy in eyes with nonneovascular age-related macular degeneration,” Invest. Ophthalmol. Vis. Sci. 54(12), 7498–7509 (2013). [CrossRef] [PubMed]
8. A. Pinhas, M. Dubow, N. Shah, T. Y. Chui, D. Scoles, Y. N. Sulai, R. Weitz, J. B. Walsh, J. Carroll, A. Dubra, and R. B. Rosen, “In vivo imaging of human retinal microvasculature using adaptive optics scanning light ophthalmoscope fluorescein angiography,” Biomed. Opt. Express 4(8), 1305–1317 (2013). [CrossRef] [PubMed]
9. J. L. Duncan, Y. Zhang, J. Gandhi, C. Nakanishi, M. Othman, K. E. Branham, A. Swaroop, and A. Roorda, “High-resolution imaging with adaptive optics in patients with inherited retinal degeneration,” Invest. Ophthalmol. Vis. Sci. 48(7), 3283–3291 (2007). [CrossRef] [PubMed]
10. D. Merino, J. L. Duncan, P. Tiruveedhula, and A. Roorda, “Observation of cone and rod photoreceptors in normal subjects and patients using a new generation adaptive optics scanning laser ophthalmoscope,” Biomed. Opt. Express 2(8), 2189–2201 (2011). [CrossRef] [PubMed]
11. A. Pallikaris, D. R. Williams, and H. Hofer, “The reflectance of single cones in the living human eye,” Invest. Ophthalmol. Vis. Sci. 44(10), 4580–4592 (2003). [CrossRef] [PubMed]
12. K. S. Bruce, W. M. Harmening, B. R. Langston, W. S. Tuten, A. Roorda, and L. C. Sincich, “Normal perceptual sensitivity arising from weakly reflective cone photoreceptors,” Invest. Ophthalmol. Vis. Sci. 56(8), 4431–4438 (2015). [CrossRef] [PubMed]
13. O. P. Kocaoglu, Z. Liu, F. Zhang, K. Kurokawa, R. S. Jonnal, and D. T. Miller, “Photoreceptor disc shedding in the living human eye,” Biomed. Opt. Express 7(11), 4554–4568 (2016). [CrossRef] [PubMed]
14. M. Lombardo, S. Serrao, P. Ducoli, and G. Lombardo, “Influence of sampling window size and orientation on parafoveal cone packing density,” Biomed. Opt. Express 4(8), 1318–1331 (2013). [CrossRef] [PubMed]
15. M. Bidaut Garnier, M. Flores, G. Debellemanière, M. Puyraveau, P. Tumahai, M. Meillat, C. Schwartz, M. Montard, B. Delbosc, and M. Saleh, “Reliability of cone counts using an adaptive optics retinal camera,” Clin. Experiment. Ophthalmol. 42(9), 833–840 (2014). [CrossRef] [PubMed]
16. C. Bowes Rickman, M. M. LaVail, R. E. Anderson, C. Grimm, J. Hollyfield, and J. Ash, “Retinal degenerative diseases: Mechanisms and Experimental Therapy,” Springer, 2015.
17. S. S. Choi, N. Doble, J. L. Hardy, S. M. Jones, J. L. Keltner, S. S. Olivier, and J. S. Werner, “In vivo imaging of the photoreceptor mosaic in retinal dystrophies and correlations with visual function,” Invest. Ophthalmol. Vis. Sci. 47(5), 2080–2092 (2006). [CrossRef] [PubMed]
18. J. I. Wolfing, M. Chung, J. Carroll, A. Roorda, and D. R. Williams, “High-resolution retinal imaging of cone-rod dystrophy,” Ophthalmology 113(6), 1014 (2006). [CrossRef] [PubMed]
19. G. Ramaswamy and N. Devaney, “Pre-processing, registration and selection of adaptive optics corrected retinal images,” Ophthalmic Physiol. Opt. 33(4), 527–539 (2013). [CrossRef] [PubMed]
20. J. C. Christou, A. Roorda, and D. R. Williams, “Deconvolution of adaptive optics retinal images,” J. Opt. Soc. Am. A 21(8), 1393–1401 (2004). [CrossRef] [PubMed]
21. L. Blanco and L. M. Mugnier, “Marginal blind deconvolution of adaptive optics retinal images,” Opt. Express 19(23), 23227–23239 (2011). [CrossRef] [PubMed]
22. X. Zhou, P. Bedggood, and A. Metha, “Improving high resolution retinal image quality using speckle illumination HiLo imaging,” Biomed. Opt. Express 5(8), 2563–2579 (2014). [CrossRef] [PubMed]
23. A. Lazareva, P. Liatsis, and F. G. Rauscher, “Hessian-LoG filtering for enhancement and detection of photoreceptor cells in adaptive optics retinal images,” J. Opt. Soc. Am. A 33(1), 84–94 (2016). [CrossRef] [PubMed]
24. D. M. Bukowska, A. L. Chew, E. Huynh, I. Kashani, S. L. Wan, P. M. Wan, and F. K. Chen, “Semi-automated identification of cones in the human retina using circle Hough transform,” Biomed. Opt. Express 6(12), 4676–4693 (2015). [CrossRef] [PubMed]
25. X. Fei, J. Zhao, H. Zhao, D. Yun, and Y. Zhang, “Deblurring adaptive optics retinal images using deep convolutional neural networks,” Biomed. Opt. Express 8(12), 5675–5687 (2017). [CrossRef] [PubMed]
26. D. Cunefare, R. F. Cooper, B. Higgins, D. F. Katz, A. Dubra, J. Carroll, and S. Farsiu, “Automatic detection of cone photoreceptors in split detector adaptive optics scanning light ophthalmoscope images,” Biomed. Opt. Express 7(5), 2036–2050 (2016). [CrossRef] [PubMed]
27. D. Cunefare, L. Fang, R. F. Cooper, A. Dubra, J. Carroll, and S. Farsiu, “Open source software for automatic detection of cone photoreceptors in adaptive optics ophthalmoscopy using convolutional neural networks,” Sci. Rep. 7(1), 6620 (2017). [CrossRef] [PubMed]
28. C. Bergeles, A. M. Dubis, B. Davidson, M. Kasilian, A. Kalitzeos, J. Carroll, A. Dubra, M. Michaelides, and S. Ourselin, “Unsupervised identification of cone photoreceptors in non-confocal adaptive optics scanning light ophthalmoscope images,” Biomed. Opt. Express 8(6), 3081–3094 (2017). [CrossRef] [PubMed]
29. M. N. Muthiah, C. Gias, F. K. Chen, J. Zhong, Z. McClelland, F. B. Sallo, T. Peto, P. J. Coffey, and L. da Cruz, “Cone photoreceptor definition on adaptive optics retinal imaging,” Br. J. Ophthalmol. 98(8), 1073–1079 (2014). [CrossRef] [PubMed]
30. S. Pertuz, P. Puig, and M. A. Garcia, “Analysis of focus measure operators for shape-from-focus,” Pattern Recognit. 46(5), 1415–1432 (2013). [CrossRef]
31. S. Pertuz, P. Domenecg, and M. A. Garcia, “Reliability measure for shape-from-focus,” Image Vis. Comput. 31(10), 725–734 (2013). [CrossRef]
32. J. F. Brenner, B. S. Dew, J. B. Horton, T. King, P. W. Neurath, and W. D. Selles, “An automated microscope for cytologic research a preliminary evaluation,” J. Histochem. Cytochem. 24(1), 100–111 (1976). [CrossRef] [PubMed]
33. A. Santos, C. Ortiz de Solórzano, J. J. Vaquero, J. M. Peña, N. Malpica, and F. del Pozo, “Evaluation of autofocus functions in molecular cytogenetic analysis,” J. Microsc. 188(3), 264–272 (1997). [CrossRef] [PubMed]
34. M. Subbarao and J. K. Tian, “Selecting the optimal focus measure for autofocusing and depth-from-focus,” IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 864–870 (1998). [CrossRef]
35. E. H. Adelson, C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden, “Pyramid methods in image processing,” RCA Engineer 29(6), 33–41 (1984)
36. A. A. Goshtasby and S. Nikolov, “Image fusion: Advances in the state of the art,” Inf. Fusion 8(2), 114–118 (2007). [CrossRef]
