1. Introduction

In 1953, Horace Babcock, an astronomer at the Mount Wilson Observatory in California, proposed a method to correct the dynamic aberrations introduced by the turbulent atmosphere lying between the stars and ground-based telescopes [1]. Babcock’s plan was to sense the atmosphere’s aberrations while correcting them in real time with electrostatic modulation of the thickness of a layer of oil on the surface of the telescope’s mirror. Unfortunately, he was unable to secure the funding to realize his idea and it would take another two decades for a team lead by John Hardy at Itek Corporation, with the financial backing of the US military, to develop the first closed loop adaptive optics (AO) system [2]. It would be another two decades before closed loop AO and the ophthalmoscope were married, a marriage that in the quarter century since continues to evolve, producing progeny both in the form of engineering advances and scientific discovery.

The first attempt to translate deformable mirror technology into the eye was made by Andreas Dreher and colleagues at the University of Heidelberg [3]. Using an active mirror, they demonstrated the correction of defocus and astigmatism obtained through a conventional refraction of the eye. Junzhong Liang, working with Josef Bille, demonstrated the first objective wavefront sensor for the eye, a key requirement for the development of a fully functional AO system [4]. Liang, Williams, and Miller at the University of Rochester [5] then demonstrated the first closed loop AO system for correcting the aberrations of the eye. AO control loops have two major components, one for measuring aberrations, typically a wavefront sensor, and another for correcting them, typically a deformable mirror. While modern AO systems measure and correct the eye’s aberrations at near video rates and higher, the wavefront sensor in this early instrument was not fully automated, often requiring human intervention of several minutes each loop to adjust the centroids of the wavefront sensor spots. Fully automated wavefront sensing was developed shortly afterward, greatly decreasing the time, not to mention the tedium, of correcting the eye’s aberrations [6,7]. Over the 25 years since, there has been impressive growth in the application of AO beyond astronomy, not only to the eye, but also to microscopy, especially to correct aberrations in the tissue between the objective and the image plane (see review in this special issue). The success of AO at correcting the eye’s higher order aberrations with a deformable mirror inspired the combination of wavefront sensing with static aberration correction technologies including contact lenses [8,9], intraocular lenses [10], and refractive surgery [11], though this burgeoning field is beyond the scope of this review.

Figure 1 shows the evolution of AO applied to the eye, with, by our count, a total of 1266 papers having been published in the 25 years since it was introduced. There has been steady and rapid growth, especially from 2004 to 2016, reaching a current rate of about 100 publications per year. Figure 1(A) divides these publications into those that are predominantly concerned with instrumentation development and AO theory and those that are mainly about applications of the technology. These data suggest that the rate of technological innovation over the last 15 years has been relatively constant compared with the growth in applications of the technology, which continues to increase as the technology becomes more widely accessible.

 

Fig. 1. Adaptive optics papers published per year between 1997 and 2021. Only peer-reviewed articles that either applied a closed loop AO system to the eye or were intended to augment such a system were included. Review papers, conference proceedings, abstracts, articles that used wavefront sensors without real-time wavefront correction, and papers that offer computational alternatives to AO were excluded. The publication list was based on a search of the Dimensions database (Digital Science & Research Solutions Ltd, Cambridge, MA), using a search string that captured 100% of the publications meeting our inclusion criteria of a test set of 7 scientists, clinicians, and engineers actively engaged in the field. This string was then used to capture the entire field as represented in the Dimensions database. Papers that did not meet our criteria were deleted manually. (A) The growth of publications over this quarter century was largely driven by applications of the technology with developments in instrumentation and AO theory for the eye holding relatively steady. (B) The growth of retinal imaging, vision, and papers that involve both applications over the past 25 years. (C) The growth in predominantly clinical publications, basic science publications, and publications involving both over the past 25 years. (D) The growth in publications over the past 25 years that combine AO with various additional imaging modalities including reflectance scanning laser ophthalmoscopy, flood reflectance, OCT, fluorescence, and phase contrast. “Other” refers to publications that use other modalities including those that combine multiple modalities in a single instrument.

Download Full Size | PDF

 

Fig. 2. A 25-year timeline of selected technical and scientific advances associated with AO applied to the eye.

Download Full Size | PDF

Liang et al. [5] showed that, thanks to the reversibility of optics, AO can simultaneously correct the aberrations for light both entering and exiting the eye, improving the quality of the image delivered to the retina as well as images of the retina. AO applied to the eye has subsequently evolved into two relatively independent fields. In the first, reviewed by Marcos et al. [12] in this same issue, the aberrations of the eye are either corrected as in the Liang instrument or modified to explore their impact on vision [13,14]. Applications include studies of optical and neural factors in spatial vision and also neural adaptation to the eye’s aberrations [15]. These devices are also valuable in designing contact lenses, intraocular lenses, and spectacles because the visual impact of a particular design can be assessed prior to its fabrication. In the second, AO is combined with a wide variety of retinal imaging modalities with the goal of improving image contrast and resolution for retinal measurements. Some studies capitalize on both capabilities, especially those invested in understanding the relationship between visual performance and the microscopic anatomy of the living retina, specifically the cone mosaic [5,16,17]. As shown in Fig. 1(B), the use of AO for vision applications has been relatively constant over the past 15 years, averaging about 15 papers per year. Retinal imaging applications have risen to roughly 80 papers per year and are largely responsible for the increased use of AO over time.

As shown in Fig. 1(C), much of the growth in retinal imaging papers is attributable to the increasing use of AO in clinical research. The publication rate for papers directed at basic science applications have held roughly constant over the past 15 years at about 30 per year, whereas in recent years purely clinically relevant papers have surpassed those focused only on basic science, with almost twice as many in 2021. While the complexity of AO technology is at least partly responsible for slowing its dissemination compared with technologies such as OCT, these data show that AO is nonetheless penetrating clinical research, presumably due to increasing commercial availability of AO-equipped ophthalmoscopes and increasingly better AO. We anticipate that this trend will continue. Morgan et al. [18] review the clinical applications of AO for retinal imaging in this same issue.

In a typical human eye, the image quality that can be obtained with AO is, in an absolute sense, not greatly improved over that which can be obtained without it. However, the resolution gain straddles the important spatial scale corresponding to the dimensions of the cells that comprise the retina, taking us from retinal images in which single cells can rarely be seen, to retinal images in which most of the major cell classes in the retina can now be imaged. Microscopic histological examination of the retina, formerly conducted only in fixed tissue, or fresh retina excised from the eye, is now possible in living eyes. AO is not essential for cellular resolution in many cases. Indeed, just prior to the introduction of AO to vision science, Miller et al. [19] were able to obtain images of photoreceptors in the living human eye without the use of AO, as many others have since. Nonetheless, the gains achievable with AO have inspired a whole host of new strategies to push the image quality of the ophthalmoscope to new theoretical limits. Some of these strategies have involved minimizing residual aberrations generated internal to the AO ophthalmoscope itself [20–22], as well as its scanning accuracy [23], while many others have improved the various parts of AO in order to better match performance to the spatial and temporal properties of the human ocular aberrations: examples include optimizing wavefront correctors [24,25] and sensor parameters [26,27], characterizing correctors [28], improving correction with the woofer-tweeter concept [29–31] and multiconjugate corrector designs [32,33], achieving faster control loop speeds to better track aberration dynamics [34–36], developing budget error analyses to characterize and reduce sources of residual wavefront error in the AO [37,38], streamlining AO hardware using, for example, wavefront sensorless implementations [39,40], and improving performance in the presence of the optical errors common in older eyes [41].

Another strategy is to combine AO with one or more additional imaging modalities. One of the fortunate features of AO is that there is no optical imaging technology we can think of that cannot in principle be combined with it. Each combination has created a new imaging platform that has allowed us to extract new kinds of information about the living retina. The last quarter century has therefore been punctuated by a number of technological milestones in ophthalmoscopy in which AO, originally demonstrated in a flood-illuminated ophthalmoscope [5], has been combined with confocal imaging [42], optical coherence tomography [43], phase contrast imaging (here referred generically as methods sensitive to contrast that arises from interactions of light at multiple surfaces in the eye) [44], and fluorescence imaging [45,46]. Increasingly, these modalities are being combined in single instruments, providing alternative, simultaneous views of the same region of interest [47–51], even in handheld versions [52]. Figure 1(D) shows that the most widely used single technology is AOSLO reflectance imaging. AO flood imaging is not far behind, presumably as a result of the commercial dissemination of flood reflectance AO ophthalmoscopes. AO fluorescence imaging, AO-OCT, and phase contrast imaging are less widely disseminated to date. The major technical advances and important scientific discoveries are illustrated in the timeline in Fig. 2. The remainder of this review highlights a selection of these with an emphasis on the two primary application areas, photoreceptors and vasculature.

2. Imaging platforms

2.1 AO flood illumination ophthalmoscope

The first closed loop AO system for the human eye [5] illuminated the retina with light from a krypton flash while a CCD array acquired the retinal image. As shown in Fig. 3, images of cone photoreceptors could be obtained that were improved in contrast over what had been possible before. Flood AO ophthalmoscopes offer the advantage of simplicity, requiring no scanning mirrors. Because they acquire an image of the entire region of interest simultaneously, each frame is free of the distortion that eye movements create in scanning systems. Furthermore, current point scanning technology essentially limits frame rates to video rates, whereas flood systems can operate at rates several times higher to keep eye movement artifacts to a minimum. The most widely used commercial instruments currently use flood illumination and this platform has seen a resurgence in recent years [36,53]. A disadvantage of flood AO ophthalmoscopes is that they lack the optical sectioning and transverse resolution advantages of scanning ophthalmoscopes described below.

 

Fig. 3. First published images of the photoreceptor mosaic obtained with an AO ophthalmoscope from David Williams’ group at the University of Rochester [5] marking the transition from ophthalmoscopy to in vivo retinal microscopy. The flood instrument had a 37-actuator Xinetics deformable mirror and operated at a closed loop rate of ∼0.01 Hz). (A) Without AO compensation. (B) With AO compensation. Scale bar = 25 microns.

Download Full Size | PDF

2.2 AO scanning light ophthalmoscope

An imaging modality that benefits tremendously from AO is the scanning light (or laser) ophthalmoscope (SLO). An SLO is essentially a scanning laser microscope [54] with the important distinction that the optics of the eye serve as the objective lens and the retina is the sample [55]. In an SLO, each image is recorded over time, pixel-by-pixel, by measuring the light returning from a small, focused spot as it is scanned across the retina - usually in a raster pattern. In all forms of AOSLO, the resolution is primarily governed by the sharpness of the focused spot on the retina and they therefore benefit from AO-correction like other ophthalmic modalities [42]. The major advantage of an SLO is its flexibility in how the detected light is generated and collected.

2.2.1 Confocal AOSLO

In a flood-illuminated camera, all the scattered light from the retina that exits the pupil contributes to the image, but SLOs can reject light from everywhere other than the region around the focus by passing the returned light through a confocal pinhole (i.e. a pinhole prior to the detector that is optically conjugate to the focused spot on the retina). This yields images with higher contrast than that of flood illumination systems and also confers an optical sectioning capability, albeit not with the axial resolution available with OCT (see later section). Correcting the blur with AO also makes it possible to use tiny confocal pinholes without significant signal loss. Small pinholes can improve lateral resolution by up to the square root of 2 [56] and this was best shown by Dubra et al. [57], who demonstrated that in a well-designed AOSLO system with optimal confocal pinhole diameters, features as small as single rods and the smallest cones in the foveal center could routinely be resolved (Fig. 4). Confocal AOSLO has made a significant impact on the study of human vision in health and disease, but the AOSLO platform enables a host of other non-confocal imaging modalities, as well as the ability to combine imaging and functional measurements which will be described in the following sections.

 

Fig. 4. Confocal AOSLO images of a human retina. (A) Foveal cone mosaic. (B) mosaic of rods and cones in the peripheral retina. A similar version of this figure is published by Carroll et al. [58]. Scale bar = 50 microns. Original image courtesy of Joe Carroll and Alfredo Dubra.

Download Full Size | PDF

 

Fig. 5. Images obtained using different forms of phase contrast imaging with AOSLO. (A) Single frame image of human arterioles and capillaries showing individual red blood cells. Cells in the larger vessels are distorted due to being sampled at slightly different times during the progressive scan [44]. Scale bar = 50 microns. (B) Averaged AOSLO image using an offset aperture showing individual mural cells, presumably pericytes lining the wall of the arteriole [73]. Scale bar = 25 microns. (C) Image of cone inner segments using split detection [65]. Scale bar = 25 microns. (D) Image of ganglion cells in a non-human primate using multi-offset imaging [66]. Scale bar = 25 microns.

Download Full Size | PDF

 

Fig. 6. (A) AO autofluorescence image of the human RPE mosaic, excitation 532 nm (adapted from Granger et al. [75]). Scale bar = 50 microns. (B) AO Rhodamine fluorescence image of monkey retinal ganglion cells, Scale bar = 5 microns. (C) AO two-photon image of the monkey rod and cone mosaic (courtesy of Christina Schwarz). Scale bar = 50 microns. (D) AO fluorescence lifetime image of the monkey photoreceptor mosaic, where color variations reveal the cones which have shorter fluorescence lifetimes than the rod regions between them. 730 nm excitation Scale bar = 25 microns (adapted from Walters et al. [76]).

Download Full Size | PDF

2.2.2 Multiply scattered light and phase contrast AOSLO

While a confocal pinhole conjugate with the point source illuminating the retina in an AOSLO is effective at recording light that is directly backscattered from the location of the focused spot on the retina, it is insensitive to light that is scattered in other directions by the presence of index of refraction gradients at that location. However, this deviated light, which is the result of local phase shifts in the wavefront created by the refractive index profile, can be captured by collecting the light that is backscattered to the detection plane but arrives at locations other than that normally occupied by the on-axis confocal pinhole [59]. This light contains valuable information about the local refractive index gradients, and hence of the phase, at the point of illumination, which can be revealed by choosing a retinal conjugate aperture that preferentially collects light that is not directly backscattered. This technique of generating contrast in an SLO was originally proposed during the development of the SLO [60], and was developed by Elsner and others [59,61,62] who showed that this multiply-scattered light is very sensitive to disruptions to the normal clarity of the retina arising from disease. With the advent of AOSLO imaging, this technique gained the ability to image very local changes in the index of refraction, such as at cell boundaries [44]. This is because the scattered beam is very small at the focus of the imaging system, and relatively large at the other scattering locations, so only the focal region of the scanned beam contributes substantially to the formation of a spatial non-confocal image, and thus provides high resolution. The amount and direction of light scattering is sensitive to both size and index differences [63] but a particularly salient feature of images obtained with this approach is the brightness gradient across each cell body, one side being bright and the other dark. This asymmetry is generated because cell bodies behave like tiny lenses, deviating the scanned beam in opposite directions as it illuminates opposite sides of the cell [64]. This deviated light is then backscattered from deeper retinal layers and a confocal pinhole conjugate with this layer but spatially offset in one direction then collects more or less light depending on which side of the cell is illuminated, generating the asymmetric brightness gradient across the cell.

This approach to generating images has diversified into multiple aperture configurations, including split-aperture imaging [65], multiple aperture imaging [66] and even more complex electronic [67] and fiber-optic-array based [68,69] approaches. Selected example images from these technologies are shown in Fig. 5. These different aperture configurations can be equated to different imaging modes in microscopes. For instance, obtaining an image where light is collected from all directions of scatter, other than the directly backscattered light, is equivalent to dark-field microscopy, and quantitatively comparing scattering in different directions is equivalent to phase contrast microscopy. In practice, many different aperture configurations are actively used in AOSLO systems, some looking at light in orthogonal directions, and some comparing over space, and since the actual angular dependence of the light distribution depends on the combined optical properties of layer boundaries, cell borders, and organelle distributions, no single approach is likely to be sensitive for all tissue targets.

While the use of multiply scattered light for imaging has been widely exploited in SLO-based systems, it is important to note that non-scanning systems can also collect all the light returning from the retina, and thus contain a full range of information. For these types of AO systems, the challenge is to increase the contrast of the portions that are of primary interest for a particular application. This has been achieved with multiple approaches [70,71], including transscleral delivery of the illumination [72].

2.2.3 Fluorescence AOSLO

AOSLOs can also be readily equipped with fluorescence imaging capabilities to provide novel information about retinal structure and function at a cellular spatial scale (Fig. 6). Fluorescence AO flood illumination imaging is also possible, but we are not aware of its use in the eye despite its common use in microscopy [74]. While fluorescence signals from the retina are weaker than those obtained with reflectance imaging at comparable levels of retinal irradiance, typically roughly 2 orders of magnitude in the case of single photon fluorescence and 4 orders of magnitude in the case of two photon fluorescence, there are many circumstances where the contrast benefits of suppressing background and increasing signal from structures of interest offset the cost of a reduced signal. Moreover, fluorescence AO offers opportunities to monitor retinal functions that are difficult to achieve with other imaging modalities.

Endogenous fluorescence: Many applications of fluorescence AO ophthalmoscopy rely only on endogenous fluorophores that are intrinsic to the retina, which offers the distinct advantage that no invasive injection of an exogenous fluorescent agent is required. Endogenous fluorescence applications are therefore much more easily and safely undertaken in humans. Several groups have used the FAOSLO to study the mosaic of RPE cells based on lipofuscin and melanin autofluorescence [45,46,75,77,78]. Lipofuscin is a collection of a large number of fluorescent compounds that reside in the cytoplasm of retinal pigment epithelial cells. Because lipofuscin is distributed nonuniformly, excluded for example from some organelles such as the nucleus, the mosaic of RPE cells is revealed (Fig. 6(A)). This has allowed in vivo characterization of the RPE mosaic in the normal as well as the diseased eye, paralleling the characterization of the cone mosaic that reflectance AO has enabled. This is especially valuable in humans where ex vivo histological examination is often not possible. This capability has also engendered a new way of conducting light safety experiments in animal models, because light damage to the photoreceptors and RPE can be imaged in the living eye, obviating the need for ex vivo histology [79].

One of the challenges of quantifying changes in retinal fluorescence over time is variability due to a host of factors including alignment of the instrument with the eye and eye movements, encouraging the development of alternatives that are immune to intensity fluctuation. One of these is fluorescence lifetime ophthalmoscopy (FLIO). Fluorescence lifetime refers to the time required following excitation of a fluorophore for the fluorescence to decay to 1/e and is independent of intensity but provides valuable information about the underlying fluorescent compound(s). FLIO has recently been successfully combined with AO (AOFLIO) and offers considerable promise for identifying fluorescent compounds in the eye at a cellular scale [76,80,81] (Fig. 6(D)).

The use of two-photon fluorescence opens up a whole host of endogenous fluorophores in the retina that are excited by short wavelengths that are not accessible with single photon fluorescence [82,83]. In a two-photon FAOSLO, a femtosecond source is used to illuminate the retina with a flying spot consisting of intense and brief pulses, increasing the probability that two photons are absorbed by the same fluorescent molecule within a time window that causes them to behave like a single photon of roughly twice the energy, or half the wavelength. Capitalizing on this nonlinearity in light absorption allows an infrared femtosecond source, for which the eye’s cornea and lens are highly transmissive, to excite fluorophores normally excited by wavelengths in the violet or near ultraviolet, wavelengths that are too strongly absorbed by the cornea and lens for safe single photon fluorescence. Endogenous fluorophores that can be reached in the intact eye with this trick include retinol, NADH, and FAD. As one example, retinol autofluorescence, revealed through two-photon excitation, provides an alternative to reflectance imaging of rods and cone photoreceptors, with the advantage that the kinetics of retinol inside the retinoid cycle of photopigment bleaching and regeneration can be monitored in vivo through changes in retinol fluorescence in photoreceptors [84–86] (Fig. 6(C)). While the concentration of photons at the focal point of the excitation beam provided by AO offers a substantial increase in the fluorescence signal, a limitation of two-photon AO imaging is that very high light levels are required [83]. Moreover, Schwarz et al. [87] has shown in the monkey that, for reasons that are not yet clear, two-photon excitation damages the short-wavelength sensitive cones at light levels much lower than would be expected from their spectral sensitivity. Until techniques are developed to use this method at lower light levels, its utility will probably be restricted to animal models. Multiphoton excitation with AO is of interest not only for imaging but also for retinal ablation highly localized in all three spatial dimensions. Dhakal et al. [88] used this dual approach to selectively ablate photoreceptors in a monkey model of retinal degeneration without damaging the overlying inner retina.

Exogenous fluorescence: Another approach to expanding the repertoire of fluorescence AO imaging is to introduce exogenous fluorophores to the retina to label specific cells. Due to limitations on the specific fluorophores that can be used in human eyes, this approach is especially valuable in animal models, though some human applications do exist [89,90]. Indeed, the use of exogenous fluorescent markers has proven useful at examining several retina structures from ganglion cells [45,91,92] to the pericyte structure surrounding capillaries [93]. There are a number of viable methods to insert fluorophores into the living retina, including intravitreal and subretinal injection of viral vectors preloaded with payloads that produce fluorophore expression once the virus has integrated with the retinal cells. This method has the advantage that the infected cell’s genetic machinery is permanently altered providing long term fluorescence in each infected cell. By introducing functional fluorophores such as GCaMP, which allows calcium imaging, it is now possible to monitor neural activity in single retinal neurons in the intact animal [94–96]. For the study of retinal ganglion cells, it is also possible to introduce fluorophores or viral vectors for them into brain regions to which the ganglion cells project. Retrograde transport of these constructs back to the retina successfully labels these cells after several weeks, allowing high resolution imaging of retinal ganglion cells and their dendritic arbors in mouse, rat, and monkey (Fig. 6(B)). A distinct advantage of this approach is that, by restricting labeling only to those RGC classes that project to a brain region, the role of that brain region in vision can be more readily assessed. AO imaging based on exogenous fluorescence has provided a way to track photoreceptor precursor cells injected subretinally as a therapy for blindness in retinal degeneration [97]. It has also proven very useful in identifying the cellular constituents flowing through blood vessels in vivo [98] and fluorescently tagged glial cells in the mouse retina [99].

2.3 AO optical coherence tomography

Distinctly different in operation from the flood illuminated retina camera and SLO, OCT is the optical analogy of ultrasonography, measuring the time of flight of light, rather than sound, to generate cross-sectional and volume images of the retina. OCT accomplishes this using a technique called low-coherence interferometry. A major advantage of this approach is that axial resolution is set by the light source’s coherence length and independent of the system’s numeric aperture, which determines axial resolution in conventional optical systems such as the flood illuminated retina camera and SLO. Axial resolution for ophthalmic OCT systems is typically quite small (3-10 microns). OCT also has the ability to detect both phase and amplitude of the imaging light and this enables subwavelength optical path length changes, which are orders of magnitude smaller than OCT’s axial resolution, to be detected in the retina. This ability has become immensely valuable in measuring cellular-level physiological processes in the retina. Like flood and SLO, OCT’s lateral resolution suffers from ocular aberrations, hence the benefit of AO.

AO-OCT was first applied to the human eye in the early to mid-2000s [43,100–102], and its performance has steadily improved since then. Early AO-OCT developments were often framed by what AO-OCT can achieve after all of its technological problems are overcome—capturing pristine 3D views of the cellular retina that are not possible using other imaging modalities. A sustained effort on AO-OCT was required to realize this vision, primarily by three groups at University of California at Davis, Medical University of Vienna, and Indiana University. Critical AO-OCT advancements in high-speed OCT image acquisition, image reconstruction and display, and 3D image registration finally thrust AO-OCT over its formidable technological barriers. As an example from 2004 to 2014, AO-OCT image acquisition speed increased by 4,000X with A-scan rates of 250 Hz [100], 18 KHz [101], 75 KHz [103], 120 KHz [104] and 1 MHz [105]. State-of-the-art AO-OCT is now able to visualize cells and cell components across the full retinal thickness and measure cell processes manifested as micrometer- and nanometer-scale optical changes [106–111].

AO-OCT development has benefited greatly from advances in the underlying OCT technology. There are three broad categories of OCT: time-domain OCT (TDOCT), spectral-domain OCT (SDOCT), and swept-source OCT (SSOCT), that can be realized with any of the three broad schemes of illumination and detection: point-scanning, line field, and full field [112]. Hardware-based AO combined with point-scanning SDOCT is the most mature AO-OCT configuration and has provided some of the best images to date of the cellular retina in humans [106,113,114](see Fig. 7). However, other AO-OCT implementations continue to be investigated and in recent years some have demonstrated exciting performance gains in speed that have reached A-scan rates of several megahertz and beyond. These include point scanning AO-SSOCT [115], line field AO-SDOCT [116], and full-field AO-OCT [117,118]. Each of these approaches requires performance trade-offs among axial and lateral resolution, sensitivity, and image acquisition speed, and the relative importance of these depends strongly on the retinal application.

 

Fig. 7. AO-OCT imaging reveals cellular details across the full thickness of the retina of the living human eye. (A) B-scan cross section with depth layers labeled. (B-J) En face images are shown selected from the AO-OCT volume and labeled by retinal depth as denoted in the cross-sectional slice on the left. Image was acquired in the parafovea. Key: RNF, retinal nerve fiber; GCL, ganglion cell layer; IPL, inner plexiform layer; INL, inner nuclear layer; ONL, outer nuclear layer; outer plexiform layer; IS/OS, inner segment - outer segment junction; RPE, retinal pigment epithelium. Scale bar = 100 microns. Figure adapted with permission from Miller and Kurokawa [106].

Download Full Size | PDF

2.4 Retinal-image-based eye tracking

Distortion removal: As is evident from the previous sections, to date scanning technologies are the most widely adopted modality for AO retinal imaging. For SLO systems, the scanning enables confocal, non-confocal and fluorescence imaging, which vastly improves the contrast of retinal images or enables other detection options as discussed earlier. Scanning for OCT enables similar confocal advantages, but more importantly makes it convenient to record depth information at each scanned location, giving rise to a volume image. But the cost of these scanning technologies are unique distortions within each scanned frame resulting from ever-present motion of the eye, even when a subject is asked to fixate. Distortions caused by eye movements in scanned retinal images have always been present but are especially problematic in AO systems because distortions increase with magnification. Even eye drift causes distortions on the scale of the resolved cells in a typical high-magnification AO image. Early on, it was recognized that any attempt to improve the signal-to-noise ratio of images by multiple frame averaging, or any effort to track individual cells over time necessitated a removal of the distortion. A solution to the problem, specifically for raster and line scanning systems was articulated years before by Mulligan [119] and was first applied to AOSLO images by Stevenson and Roorda [120] and by Vogel et al. [121]. These software-based approaches to correcting image distortion, is a distinct alternative to hardware approaches [20,122–125] which, despite offering some advantages, are typically more cumbersome to implement. The 2D algorithms were extended to effectively remove eye motion distortions from AO-OCT volumes followed thereafter [107,126]. Today, multiple versions of distortion-correction algorithms are being used routinely by those doing all forms of AO scanning imaging. The ultimate aim to generate distortion-free ‘ground truth’ images from scanned videos has yet to be entirely met, but there are several promising new approaches [127–132].

Eye tracking: It has long been recognized that registering SLO images frame to frame created a record of eye movements [133]. It was subsequently recognized that an important by-product of removing eye-motion-caused distortions within each frame was an even higher bandwidth record of the eye motion that caused it [134]. The effort to remove distortion in high resolution AO imaging ended up creating the world’s most accurate eye tracker. Two features of such eye tracking qualify that statement. First, the tracking rate is computed at the frame rate multiplied by the resolution of the distortion correction within each frame, so a 30 frame per second video corrected with 30 strips per frame yields an eye motion trace of 900 Hz, faster than most eye trackers. Second, once the aim to remove distortion on the scale of the resolved cells is achieved, the accuracy is unambiguously confirmed to be a small fraction of one arcminute. While this eye tracking capability has limits - primarily in its dynamic range - it has been used to produce new knowledge about fine eye movements and how they manifest on the retinal image [135].

Targeted stimulus delivery: With appropriate hardware and software, image-based eye tracking can be done in real time. There are several motivations for real-time tracking, but it mainly serves to guide the placement of light delivered to the retina for imaging or vision testing purposes. In a pioneering demonstration of this, Arathorn et al. [136] showed that real-time eye tracking can be used to guide the placement of a light stimulus to the retina via the AOSLO raster scan with 0.26 arcminute accuracy or 1.3 micrometers*. (*Note that the accuracy of targeted light delivery is always worse than the tracking accuracy because of the small latency between the eye movement event and the effort to correct for it.) This led to a host of applications where AO-corrected light patterns could be delivered to targeted retinal locations with cell-level accuracy. In one example, it enabled a study by Ratnam et al. that showed how eye movements can be leveraged to improve visual acuity [137], and in another it was used by Tuten et al. to very carefully measure light sensitivity thresholds at and near the border of retinal lesion in patients with choroideremia [138].

3. Scientific discoveries enabled by AO retinal imaging

In this section we highlight scientific discoveries enabled by AO in two keys areas that have been the dominant focus of AO eye research to date: photoreceptors and vasculature.

3.1 Photoreceptors

The first line of scientific investigation with AO was the study of cone photoreceptors. There are several reasons for this: Photoreceptors represent the first stage in vision after image formation; they are the easiest cells to image optically owing to their high intrinsic contrast and retro-reflective properties (see below); and there are many fundamental, unresolved questions about their structure and function in health and disease. For example, though it has been known since the time of Thomas Young (1802) that the human trichromatic retina comprised three cone types [139], their distribution on the retina, specifically that of the L and M cones which are optically, genetically and morphologically very similar, had never been mapped in humans or any other trichromatic primates to any appreciable extent. So, mere months after Liang, Williams and Miller completed the first AO ophthalmoscope [5], a project was initiated to map the trichromatic photoreceptor mosaic. Using a method coined single cone densitometry, Roorda and Williams succeeded in mapping the three cone types in two trichromats and one protanope as a control [140](Fig. 8(A)). This was followed up later by Hofer et al. who added another 8 subjects [141]. Still today, only a small number of humans have had their cones classified and only over limited regions, but that has begun to change as interferometric techniques in an AO-OCT system have recently proven to be a much more efficient and accurate method [53,142,143].

 

Fig. 8. (A) Trichromatic cone mosaic from Roorda and Williams [140]. (B) Plots of color percepts elicited by small spot stimulation of targeted retinal cones from Sabesan et al. [17]. The annular ring indicates the fraction of color percepts elicited when that cone was stimulated. Scale bar for both panels = 5 arcminutes.

Download Full Size | PDF

Color Vision: Maps of the trichromatic mosaic in live humans enabled researchers for the first time to assess visual performance and the circuitry of the retina in eyes for which the trichromatic mosaic is known. A logical launch point for investigation of color vision was on a single cone scale. AO offered a way to increase the likelihood of single cone stimulation. Hofer et al. recorded subjects’ color percepts elicited by small, dim, narrowband green (550 nm), AO-corrected flashes of light against a black background. Subjects required up to 8 different colors (including white) to describe the flash, dispelling any notion that under these conditions, subjects ought to only see two colors (the wavelength was too high to reliably stimulate S-cones) as per Helmholtz’ elemental sensation hypothesis [16]. The development of targeted stimulus delivery (see above) enabled Sabesan et al. to target small flashes of light to individual cells [17]. In this case, against a white background (by technical necessity) the elemental sensation hypothesis largely bore true - stimulated L-cones were likely to elicit sensations of red, and M-cones sensations of green regardless of the stimulating wavelength (Fig. 8(B)). However, the variability in perceived saturation was high, with many cones eliciting nearly achromatic percepts. In any case, the exact mechanisms by which one transitions from a complete inability to judge color via single photoreceptors to immediate and accurate judgements of color appearance for large stimuli remains largely unknown, and tools that use AO will continue to enable discoveries in this area.

Preview of Powerful Clinical Applications: One of the most potent early examples of clinically relevant research using an AO ophthalmoscope was published by Carroll et al. [144] who revealed two unique forms of color blindness cause by different mutations in the part of the gene array that encode the L/M cone pigments. One form of color blindness manifested as a complete, contiguous array devoid of M-cones (Fig. 9(A)), whereas the other had a patchy array, where the locations of a proportion of the cones - presumed to be non-functioning M-cones - appeared dark in the image (Fig. 9(B)). Until this discovery, exactly how the dichromatic cone mosaic appeared was only a topic of debate, since there were no conventional functional tests that were able to reveal such striking anatomical differences. (Members of the same team later devised a functional test to reveal the differences [145]). Most importantly, this study revealed how powerful in vivo microscopy could be in the discovery of genotype-phenotype relations and to learn about eye disease in general. Since that landmark paper, many studies have used AO systems to reveal these relations in eye disease, recognizing the important role of high-resolution imaging for understanding and monitoring eye diseases in the effort to treat and cure them. For more examples, the reader should refer to the review by Morgan et al. [18] in this same issue.

 

Fig. 9. Modified from Carroll et al. [144]. Cone mosaic image of two individuals with different gene mutations that cause color blindness. Scale bar for both panels = 20 micrometers.

Download Full Size | PDF

Photoreceptor Structure: The ubiquity of 2D cone mosaic images in the AO literature – as for example those referenced in the previous section to characterize the cone mosaic – makes it easy to forget that cone cells are structurally more extensive and optically more complicated than the en face 2D slices suggest. Indeed, cone cells extend over a significant fraction of the retinal thickness, from cone outer segment tips (COSTs) (which are ensheathed by RPE microvilli) to synaptic terminals in the outer plexiform layer (OPL). While cone morphology is established, the detailed interaction of cones with light is often not. AO retinal imaging has been instrumental in expanding our understanding of where reflections arise in the inner and outer segments and is now being used to study cone nuclei and axon reflections. These reflections in turn have been used to characterize the morphology of individual cone components in the living normal and diseased eye. Using AO retinal imaging systems and OCT systems, cone cells have been found to generate an exceedingly wide range of reflectance. On one extreme are the bright reflections at the inner segment/outer segment junction (IS/OS, also referred to as the ellipsoid zone [146]) and cone outer segment tip (COST), which form the en face cone mosaic image and are some of the strongest reflections in the retina. On the other are the extremely weak soma reflections, which are responsible for the dimmest band (outer nuclear layer) in clinical OCT images. It is now recognized that, with the exception of the photopigment they contain, much of a cone cell is more transparent (less reflective) than the highly transparent neurons that compose the inner retinal layers.

Early AO-OCT users were the first to discover the bright IS/OS and COST reflections of individual cone cells and the prominent role that they played in forming the image of the cone mosaic [103]. These same reflections have also been used in AO studies to measure the axial extent of the cone’s light-capturing appendages, the inner and outer segments. While we know a good deal about the lengths of these appendages from histology and clinical OCT images, AO retinal imaging has allowed individual appendages to be measured and characterized in the same retinal patch [126,147–149]. These measurements have taken on significant clinical interest as local inter-cone differences in appendage lengths have been found to dramatically increase in diseased eyes, such as retinitis pigmentosa [150]. Using AO-OCT and AO phase contrast imaging methods, structures internal to these appendages – in particular in the ellipsoid compartment of the inner segment – are also beginning to be revealed.

Perhaps surprisingly, we still do not have anatomical consensus for all the major reflections that photoreceptors generate. Correct attribution is important as misinterpretation can cause incorrect diagnosis and scientific conclusions. A hallmark example is the second outer hyper-reflective band observed in clinical OCT images that correlates with the position of the photoreceptor inner and outer segments, but the exact axial location remains controversial. The most widespread view – based in part on comparing clinical OCT images to histology – attributes this band to the densely packed mitochondria that forms in the IS ellipsoid [146,151,152]. AO-OCT imaging studies of this same band, however, have not substantiated this interpretation [153,154]. AO-OCT images show that the second band thickness and depth do not align to the expected size and location of the IS ellipsoid and instead are consistent with the slightly deeper inner segment/outer segment junction. While attribution for this hyper-reflective band remains unsettled, AO imaging studies are playing an increasingly central role in providing quantitative information not possible with traditional structural methods (histology and clinical retinal imaging).

Photoreceptor Optics: The use of cone photoreceptor reflections has been instrumental in direct studies of the light capturing properties of cones in vivo. The directional sensitivity of the cones to light was first discovered in 1933 by Stiles and Crawford [155]. This antennae-like directional sensitivity plays an important role in photopic vision as it favors light entering through the eye’s pupil center (where the cone segments nominally point) over more peripheral pupil locations as well as non-image-forming light from the iris, sclera and fundus. The property also has significant clinical interest as normal photoreceptor directionality requires normal morphology. Prior to AO retinal imaging use, most of what we knew about the light capturing properties of cone photoreceptors – at least in humans – came from decades of experimental investigations that measured the ensemble response of hundreds if not thousands of cones, by both psychophysical and reflectometric methods. And parallel to these experiments, extensive theory was developed for modeling light propagation in photoreceptors based on the principles of optical waveguides, i.e., tiny optical fibers or light pipes [156–160]. While these experiments and theoretical predictions provided substantial information about the light capturing properties of photoreceptors, direct evidence of what actually occurs in the individual cone remained largely missing.

AO retinal imaging has played an important role in filling this gap. AO systems have been instrumental in obtaining direct measurements of light capture in single cones to assess: (1) the angular tuning properties of single cone cells and (2) the number and types of waveguide modes supported by single cone cells. For the first, we have known that photoreceptor orientation is actively controlled by a phototropic mechanism that aligns the photoreceptors to point toward the pupil center. But the precision to which individual photoreceptors can do this has been of considerable debate, leading to uncertainty in the amount of disarray that exists in the local alignment between cones. Psychophysical [161] and conventional reflectometry methods [162] inferred the amount of disarray using the tuning properties of the ensemble of cone photoreceptors, but it was only with AO retinal imaging that the first opportunity to directly measure this property of individual cones in vivo was possible. AO imaging showed unequivocally that disarray between cones was indeed small [163].

A more fundamental descriptor than directionality of light capture by photoreceptors is the use of waveguide modes. These modes are exquisitely sensitive to the physical properties of the photoreceptor inner and outer segments and thus have led to considerable interest in their use to define visual performance at the photoreceptor level and to detect perturbations in the photoreceptor properties associated with pathology. Elegant theoretical models have been used to predict the number and types of waveguide modes that can be supported by cone cells but the lack of experimental tests of their prediction have confined them to largely abstractions [164]. Multimode behavior was reported many decades ago in animal models in postmortem [165], but in humans, even the most basic prediction as to whether photoreceptors support more than a single waveguide mode had not been proven. It was only with AO retinal imaging that it became possible to directly test for modal behavior [166–169]. By separately analyzing the cone IS/OS and COST reflections, the inner and outer segments were found to waveguide light but did so differently depending on retinal location. Outer segments and narrow inner segments, such as in the fovea, behaved like single-mode fibers whereas wider inner segments acted more like multimode fibers with the multimode cutoff just outside the fovea [168].

Photoreceptor Physiology: The photoreceptor reflections that inform us about cell structure and cell optics have also been discovered to carry fundamental information about cell physiology when these reflections are tracked over time. In particular, the bright reflections that straddle the outer segment (i.e., IS/OS and COST) have created a powerful tool for measuring physiological processes in the photoreceptor’s outer segment that occur over a vast timescale from milliseconds to days. The outer segment is arguably the photoreceptor component of most scientific and clinical interest as it is the primary site of phototransduction. Introduced in 2007 by Jonnal et al. [170], numerous AO imaging studies have since used the interference of the IS/OS and COST reflections to convert the outer segment into a biological interferometer with exquisite sensitivity to small (<<λ) changes in outer segment length. Interferometry in this way has evolved along two distinct paths: (1) intensity-based AO imaging such as AO-SLO and AO-Flood that collects the ensemble of interfered light over the axial extent of the photoreceptor cell plus any retinal layers in front of or behind the cell and (2) phase-based AO imaging such as AO-OCT that axially confines the interference to the local IS/OS and COST reflections.

Early users of this interferometric approach discovered the optical signature of outer segment disc renewal [171,172], the continual assembly of new membranous discs at the end of the outer segment proximal to the inner segment. This exceedingly slow elongation (40-80 nm/hr) could also be observed with AO-OCT systems by tracking the slow descent over days of isolated reflections that sometimes occur inside the cone OS [173]. This elongation is offset by disc shedding, discrete diurnal events in which small packets of discs of one to two microns in size are pruned from the distal end of the outer segment. This pruning is observed as an abrupt shortening of the outer segment (decrease separation of IS/OS and COST) in AO-OCT images [174]. The physiological processes of renewal and shedding help maintain photoreceptor health, and their dysfunction is associated with numerous retinal diseases, yet little is known about them in the living human eye. With these new optical signatures, ongoing AO studies are filling this gap, targeting the diurnal pattern of these processes and their regulation by light exposure and circadian control.

The same interferometric approach and variants of it have been used to discover that light stimulation of photoreceptors generates optical path length changes in the photoreceptor’s outer segment, detected originally as interference changes [170](see Fig. 10). These changes occur on a much faster timescale (milliseconds to seconds) than that of outer segment renewal and shedding (minutes to days). We are just beginning to understand the relationship between these photostimulation-induced changes – now called the photoreceptor optoretinogram (discussed below) – and vision in both normal and diseased eyes. The cone optoretinogram has been investigated by numerous groups to: (1) elucidate stages of the phototransduction cascade and its downstream effects [115,143,170,175–181]; (2) spectrally classify cones to reveal how color vision phenotype and genotype manifest in individual cone cells [142,143]; (3) measure the human photopic luminosity function, the collective response of all cone spectral types [182]; (4) reveal functional responses of rods [175]; and (5) quantify cone dysfunction in retinitis pigmentosa [150], the first application of the cone optoretinogram to diseased eyes [18]. Perhaps not surprisingly, optical changes of cones in response to photostimulation are not limited to phase. Grieve and Roorda [183] reported more than a decade ago intensity changes that they attributed to small scattering increases in the cone and more recently Azimipour et al. [115] demonstrated intensity and movement changes of reflections in the photoreceptor, subretinal space, and RPE layers.

 

Fig. 10. The physiological response of a cone cell to light produces nanometer changes in its OS optical path length (ΔOPL) that can be measured and tracked with AO-OCT imaging. (A) Schematic shows the inner and outer segments of a cone cell and the underlying RPE cell that ensheathes it. The cone cell is stimulated with a visible flash during imaging and a phase change results between the two bright reflections at opposing ends of the cone OS, which are labeled IS/OS and COST. Plot shows the normalized sensitivity functions of the three cone types that are sensitive to short- (S), medium- (M), and long- (L) wavelength light [185]. Vertical dashed line depicts peak of example red stimulus. (B) Phase response of cones is biphasic with a rapid, shallow decrease in ΔOPL (inset) followed by a sustained increase whose amplitude varies with cone type (S, M, and L). The dashed gray line at 0 s represents the 5-ms stimulus flash. Individual traces of cone cells are randomly colored. (C) En face intensity image shows the cone mosaic in the parafovea and (D) the same mosaic is color coded on the basis of cone classification (S = blue; M = green; L = red). Scale bar for panels C and D = 50 microns. Figure adapted with permission from Zhang et al. [143].

Download Full Size | PDF

The term optoretinogram (introduced above) has garnered much attention in recent years and even a review dedicated to it [110]. The term was introduced in the literature three decades ago [184] but only recently has gained wide acceptance. However different definitions are in circulation [110,181,184] leading to confusion as to what optoretinogram means and what retinal changes it includes. We favor that it be defined as the optical measurements of changes in the retina in response to light stimulation, in the same way that the electroretinogram (ERG) is the measurement of the electrical changes of the retina in response to light stimulation. Thus, like ERG, the optoretinogram represents the raw response trace of the entire retina (neural plus non-neural components) for a given experimental paradigm. The trace can include any optical change, with the principal changes being intensity, phase, and polarization. As is the case for the ERG, this definition leaves open interpretation and attribution of the various parts of the response trace. As the field becomes better at interpreting and using more precise stimuli and more sensitive detectors, it will become clearer what parts are due to photoreceptors, other neural cells like bipolar and retinal ganglion cells, and non-neural structures such as vasculature, glia, and retinal pigment epithelium cells.

3.2 Retinal vascular structure and blood flow

The function of the eye and retina depend critically on the normal function of the retinal vasculature, and vascular diseases represent a major class of blinding disorders as well as overall mortality. Because the retina is part of the central nervous system, and is readily accessible for noninvasive optical measurements, it is an ideal tissue for developing biomarkers for vascular disease [186–189]. AO systems provide the unique ability to make cellular level measurements of the retinal vasculature, including both structural and functional measures, primarily in the form of blood flow regulation and represent potentially the best way to evaluate the health of vessels ranging in size from the smallest capillary to the larger retinal arteries and veins.

Structural measurements using AO have enabled precise measurements of the relation between vessel walls and lumen size for small vessels [190] and the impact of diseases such as systemic hypertension [190,191], and diabetes [192] as well as the changes in vessel branching patterns that are hypothesized to arise from size dependent changes in viscosity [193–195] and are markedly changed by diabetes [194]. While the retinal circulation provides the predominant oxygen supply for the inner retina, from the cone inner segments through the ganglion cells and nerve fiber layer, the mitochondrial concentration in the ellipsoids of the cones and rods represent the watershed between the choroidal and retinal circulation, and thus both circulations are needed for the normal function of vision. While traditional measurements such as fluorescein angiography and more recently OCTA could show areas of missing choriocapillaris, AO-OCT has been able to precisely map the choriocapillaris in vivo [196] and fluorescent AOSLO imaging has provided the first glimpses into a heterogeneity of the RPE monolayer and its relation with the choriocapillaris [90].

The retina has one of the highest metabolic demands of any tissue in the body and because of this high demand the retina balances high capillary density, which provides oxygen but also scatters light, with the ability to support high levels of activity. Thus, the retinal vascular system must actively regulate blood flow. While traditional non-AO techniques, such as laser Doppler velocimetry [197] have long enabled measurement of retinal blood flow, AO allows precise measurements in much smaller blood vessels as well as measurements of blood flow regulation in response to retinal demand [198–201]. A number of techniques have been developed to make these measurements in small blood vessels. Roorda and colleagues [42,202] showed that sparse cells or cell aggregates could be directly imaged moving through retinal capillaries and using the AOSLO were able to generate flow maps along a vessel [203,204] and show the impact of early stages of diabetic retinopathy. In addition, because of the resolution provided by AO imaging and the ability to quantify flow at the smallest scales it is now possible to map the pulsatility of blood flow all the way down to the smallest capillaries with high accuracy [205–207]. This ability to quantify blood flow based on individual elements of the blood has generated a number of new approaches, directly generating time vs space imaging, which involves halting the scan in one dimension [208], high frame speed imaging with a scientific camera [209], high speed line-scan imaging [210] and the use of multiple beams in an AOSLO to capture velocities that would otherwise undergo aliasing [211].

The capability to make these quantitative measurements has allowed us to better quantify the importance of control of flow early in diseases such as diabetic retinopathy [212] and in vessels that are smaller than can be measured using conventional techniques such as laser Doppler velocimetry. This capability of measuring across the entire range of blood vessel sizes is a unique advantage for AO approaches, since in several retinal vascular diseases [190,192] (also see Morgan et al. [18]), it appears that damage occurs first in the small vessels of the retina [213].

The ability of AOSLO to specifically select the light being used to provide measurements has also allowed optimizing the light collection for different aspects of the vascular system, from fairly small angle scattering for small changes in the index of refraction boundaries arising from the vessel walls, to larger offsets where moving red blood cells predominate the retinal image [67]. This has also allowed clear visualization of not only red blood cells and vascular walls, but of increasing numbers of immune system associated cells [99,214] which can often be seen in ischemic retinas [215].

4. Future opportunities for AO retinal imaging

The evolution of ophthalmic AO over the last quarter century has been punctuated by many technological advances, usually drawn from other disciplines, that have resulted in significant scientific discoveries. We speculate that this diversification of the AO platform is likely to continue, especially through the parallel evolution of the fields of microscopy, astronomy and OCT. For example, the field of astronomy, challenged by the ∼arc-second isoplanatic patch sizes typically encountered due to the thickness of the atmosphere, is embracing multiconjugate AO systems in which multiple guide stars and multiple deformable mirrors are used to correct different locations in the field of view simultaneously, expanding the useful field of view. This approach has been tested in retinal imaging [33,216,217], but has not been fully exploited in the eye despite the strong interest, especially from the clinical ophthalmology community, in enlarging the field of view. An intermediate approach to increasing the field of view, is to enhance the ability of AO systems to rapidly tile the view of the retina. Thus, systems using wide field steering, where the AO image can be rapidly moved from location to location on the retina, without requiring changes in fixation and the consequent movement of the pupil have been developed [20] and can now access up to 30 degrees of retina [122] using rapid serial imaging.

Astronomers also long ago demonstrated the ability to measure and correct the dynamic aberrations of atmospheric turbulence at kilohertz rates, whereas AO retinal imaging systems typically operate at rates that are about two orders of magnitude slower. This has always been a glaring difference, especially since ophthalmic wavefront sensing has three to four orders of magnitude more light for wavefront sensing than are typically available in astronomy. By using light more efficiently and increasing the bandwidth of the AO control loop compared to current ophthalmic AO systems, we expect to be able to improve retinal image quality, especially for clinical conditions that induce additional temporal-frequency aberrations.

Using AO to minimize the impact of the eye’s aberrations on imaging, the next battle is overcoming retinal image blur arising from diffraction at the pupil. The field of microscopy, which contributed confocal scanning to the SLO [60] and subsequently to the AOSLO [42], has more recently produced an explosion of superresolution technologies that are capable of pushing spatial resolution and contrast beyond what can be achieved with AO alone [218]. Reaching the ultimate limits of in vivo retinal imaging requires as much light as possible, especially in light-starved applications such as fluorescence imaging. Fortunately, summing multiple frames can now be achieved with residual eye motion blur that is less than that produced by diffraction, clearing the path for superresolution approaches to make accessible increasingly smaller subcellular features inside the living eye. Some superresolution approaches have already been combined with ophthalmic AO [68,219] and we can anticipate additional advances in this direction in the future. Superresolution offers the exciting possibility of resolving subcellular features at the scale of organelles and cellular arborizations, extending the opportunities for detailed in vivo histology.

We also expect the range of detection modalities to expand. AO retinal imaging shares the same underlying imaging modalities (e.g., flood, SLO, OCT, etc.) with other disciplines and this allows AO imaging to gain shared benefit as these technologies improve. OCT in particular continues to rapidly evolve and will result in more powerful systems that are faster and more sensitive, and we expect AO-OCT systems to flourish. Another area of promise is the use of polarization to better understand tissue properties. OCT has leveraged polarization measurements extensively [220,221], as have SLO measurements [222,223]. While polarization measurements using AOSLO [224] and AO-OCT [225] have begun, they are likely to expand since polarization imaging is sensitive to the fine structure and particularly to orientational anisotropy of retinal tissues. In addition, the development of highly precise image registration methods facilitates temporal spatial averaging of the polarization signal, which is typically weak. Temporal averaging, unlike spatial averaging, increases the signal-to-noise ratio without degrading the high resolution gained with AO.

But it is not only allied optical fields like OCT, microscopy and astronomy that are poised to enhance the future capabilities of the AO ophthalmoscope. The development of exogenous fluorescence agents, including advances in viral vector delivery, that can provide increasingly sophisticated information about retinal structure and function at a microscopic scale [226]. For example, fluorescence lifetime-based sensors have recently been developed that can measure glucose utilization and NADH metabolism [227,228]. In addition, fluorescence lifetime has the capability to be used with endogenous fluorophores in the retina, providing unique functional information [76,80,81]. The combination of these technologies promises to monitor specific metabolic pathways in both the normal and diseased eyes of animal models.

As a second example, much remains to be learned about how the retina codes information for transmission to the brain. Indeed, there are more than 20 independent classes of retinal ganglion cell, each of which forms an independent mosaic that tiles the visual field. Despite the plethora of microelectrode recordings over many decades from single neurons in the excised retina, the function of most of these pathways, even those with the largest populations of retinal ganglion cells such as the midget pathway, is often controversial. An important technical limitation on clarifying the visual significance of these parallel pathways to the brain is that the standard model for physiological investigation of the retina requires sacrificing the animal and removing the retina from the eye, which destroys the central connections that could otherwise be used to establish the functional role of each RGC class. This is just beginning to change with the advent of fluorescent calcium indicators such as GCaMP, expressed via intravitreal injection of a viral vector, which allows optical recording from retinal ganglion cells in response to visual stimuli in the eye of the intact animal [94–96,229]. It is now possible to record optically from large numbers of RGCs simultaneously and in the understudied fovea, a strategy with exciting promise for revealing their response properties. The engineering of improved fluorescent markers is moving at a rapid pace, with calcium indicators emerging that are both faster and more sensitive [230].

The progress and opportunity afforded by fluorescence AO imaging for functional imaging notwithstanding, the light levels required in humans with endogenous fluorophores and the regulatory challenges of using fluorophores expressed with viral vectors in human (but see Sahel et al. [231]), encourages the development of entirely noninvasive methods for imaging function. The rapidly developing field of optoretinography naturally begs the question of whether the opportunity exists to measure the responses of other retinal neurons with label-free optical methods in the living eye. It is known that small changes in the refractive index and structure of neurons occur as they generate action potentials [232,233], and it is conceivable that an interferometric approach could capture these subtle signals in the human eye. Encouragingly, Pfaffle et al. [234], using an off-line, computational approach for blur correction, has shown changes in inner retina although these are probably relatively slow osmotic changes that may accompany the bulk movement of ions, as in the ERG b-wave.

The applications of AO retinal imaging to non-neural portions of the retina will also continue to proliferate. To function properly, the retinal neurons require the normal function of not only the neurons, but also blood vessels, glial cells and support cells such as the retinal pigment epithelium. It is now possible to measure properties of all these components using AO, yet the understanding of how they inter-relate and communicate to maintain a healthy retina is just beginning.

There has also been a push to use the ability of OCT to measure the complex field of the light returning from the retina to correct for aberrations computationally. This approach in principle can avoid some of the complexities of AO hardware, since it takes advantage of standard OCT systems. The challenges for the computational approach to high resolution imaging involve the need to acquire all the required data, from all relevant depths in the retina, without contamination due to eye movements and loss of light due to defocus. Nevertheless there are excellent examples of cellular scale imaging and functional measurements [179,235], including combining computational correction with hardware-based AO to improve performance [236].

AO is also being applied to other areas of the eye than the retina. As mentioned above, the ability to measure choriocapillaris using motion contrast [196] is of great importance in diseases such as AMD [237]. The behavior of the trabecular meshwork and drainage structures of the eye is critical in understanding how the eye maintains its structural integrity and how glaucoma and glaucoma treatments impact this maintenance and the cells of the trabecular meshwork using AO techniques [238] and we can expect this expansion to other eye structures to continue in the future.

Advances in the hardware and software systems that are built around AO ophthalmoscopes will also continue to augment their abilities to interrogate function. To date, the repertoire of stimuli that could be delivered to the retina through an AO system has been largely limited to single letter optotypes and to one or several small spots. With a few exceptions (see Arathorn et al. [239]), retina-contingent stimuli have been primarily stabilized. This was mainly due to technical limitations. But in time, AO retinal imaging systems that are being used for visual psychophysics will begin to operate more like displays, tracking and delivering light to every part of the scanned field, providing controlled spatial and temporal stimuli aligned to the individual photoreceptors. Such a display would effectively put the experimenter in total control of all the early stages of vision: the sharpness of the image, the motion of the image, and even the ratios of excitation of the three cone types that inform color vision. The display would be able to generate sensory signals at the level of the photoreceptors that natural vision could never produce, even augmenting its sensory capabilities, for example by boosting or expanding its color signals. The advent of optogenetics also offers the exciting opportunity to insert visual stimuli at subsequent stages of the retina including the retinal ganglion cell mosaic, a capability that could have value in vision restoration [240]. The ability to optogenetically excite single cells while awake behaving primates are performing psychophysical tasks could provide an entirely new route to establish the roles of each cell class in vision. Such a capability would parallel the studies that are already ongoing mapping the visual experience resulting from stimulating single cones, but at the far less well understood level of the retinal output rather than the retinal input. The experimental opportunities are broad, ranging from psychophysical studies to infer properties of the retinal circuitry that shape the signals prior to sending them through the optic nerve (e.g., lateral inhibition), to those that gauge the factors that yield our percepts of scene (e.g., perception of motion and/or perception of color).

Finally, like most other fields, artificial intelligence is poised to play an increasing role in all areas of ophthalmic AO from image analysis and interpretation to optimizing the AO control loop. We expect that will soon be the focus of its own review paper.

5. Concluding statement

This special issue demonstrates the many ways in which AO has driven a paradigm shift in how we study the eye and the visual system in health and disease. While the technology surrounding AO will no doubt continue to evolve in ways we cannot anticipate, the effectiveness with which AO, unlike static optical components such as lenses and mirrors, can address the variability intrinsic to the biological optics of the eye ensures that it is here to stay. The field continues to move forward: AO is steadily being improved, finding an increasing number of novel clinical applications and continues to enable new scientific discoveries with no end in sight.