Open Access
Add to BibSonomy
Abstract
Passive motion correction methods for optical coherence tomography (OCT) use image registration to estimate eye movements. To improve motion correction, a multi-image cross-correlation that employs spatial features in different image types is introduced. Lateral motion correction using en face OCT and OCT-A projections on Lissajous-scanned OCT data is applied. Motion correction using OCT-A projection of whole depth and OCT amplitude, OCT logarithmic intensity, and OCT maximum intensity projections were evaluated in retinal imaging with 76 patients. The proposed method was compared with motion correction using OCT-A projection of whole depth. The comparison shows improvements in the image quality of motion-corrected superficial OCT-A images and image registration.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Optical coherence tomography (OCT) is a cross-sectional, three-dimensional imaging tool for biomedical tissues [1]. In ophthalmic applications of OCT, involuntary eye movement is one of the largest problems, causing artifacts, missing areas, and distortion. These effects disturb three-dimensional morphology and en face investigation such as OCT angiography (OCT-A) and thickness mapping [2,3].
Two approaches have been used to achieve motion-free imaging. One is the combination of an OCT scanner with an active retina-tracker. The successful motion-free imaging of fine structure can be obtained with image-based retinal-tracking systems [4–8]. However, this approach requires an additional fast imaging system to monitor the retinal motion accurately. Moreover, missing sampling locations cannot be recovered when retinal-tracking fails or the eye blinks. Hence, additional mechanisms to detect missing regions and re-scanning are necessary [5,9]. The other approach is a passive motion correction approach. The retina is scanned with a spatially redundant scan pattern and image-registration-based algorithms are employed to estimate the eye motion [10–14]. This approach does not require additional hardware modifications. When the redundancy of sampling is high, the method becomes more tolerant of missing data such as eye blinks [14].
Recently, we demonstrated a motion-artifact-free imaging technique based on the Lissajous scanning pattern and an advanced motion correction algorithm [14,15]. The proposed scanning pattern is used during image acquisition and the motion is corrected afterward. Although this method works to a certain degree, the residual eye movements are still problematic for clinical applications.
En face OCT-A of the retina visualizes retinal and choroidal vasculature from large vessels to capillaries [16–19]. Several algorithms including our previous one use OCT-A image as a feature for motion estimation and can correct motion well [12,13,15]. However, some patients exhibit low or no blood perfusion due to circulation abnormalities. In such cases, the features used for motion estimation will be decreased. In contrast, some abnormalities are visualized in OCT intensity images as specific spatial patterns, e.g., abnormalities of the retinal pigment epithelium (RPE), pigment epithelium detachment, edema, etc [20–30]. Hence, using OCT images includes more features other than those of OCT-A and may improve motion estimation performance.
In this study, we demonstrate lateral motion estimation using OCT-A and OCT for motion-free OCT imaging. The OCT signals are acquired with a modified Lissajous scanning pattern that enables OCT-A imaging [15]. The lateral motion is estimated from several types of en face OCT and OCT-A images using a motion estimation algorithm. The OCT-A average projection and OCT average amplitude, average logarithmic intensity, and maximum intensity projections are used (Section 2.2). The eye-motion-corrected images are then obtained based on the estimated eye movements. To confirm that the presented method improves motion correction, imaging of retinal disease was performed with the proposed method and the method using only an OCT-A image. The resulting motion correction performance was evaluated and compared.
2. Motion correction method
2.1 Multi-image registration
In rigid image registration, image cross-correlation is used to estimate the shifts between images. The peak location of the cross-correlation function between two images of the same image type at different time points indicates the amount of motion that occurs between these two time points. To improve motion correction performance, several image types are used to estimate this amount. The details of image types used in this study are described in Section 2.2 and an example of them is shown in Fig. 1. Multi-image cross-correlation using a pair of image sets estimates the shift between two image sets using the features of all image types. Each image set consists of several image types obtained at the same time point. The image normalized cross-correlation, $\rho$, obtained using two image sets of $n$ image types, { $f_1 (\mathbf {r}), \ldots , f_n (\mathbf {r})$ } and { $g_1 (\mathbf {r}), \ldots , g_n (\mathbf {r})$ }, is calculated using integrated covariance and variance functions over the image types as follows:
Further, $\mathbf {r}$ and $\mathbf {r'}$ are spatial location and displacement vectors, respectively, $i$ is the index of the image type, $n$ is the number of image types, and $M_{\mathbf {r'}}$ is the overlap region between images $f(\mathbf {r})$ and $g(\mathbf {r} - \mathbf {r'})$. The operation $\langle \rangle _{M_{\mathbf {r'}}}$ is the average over the overlap region $M_{\mathbf {r'}}$. The calculation of the cross-correlation is limited within the domain where the overlap area is greater than or equal to the overlap area without displacement ($\mathbf {r'} = \mathbf {0}$) multiplied by the coefficient $\alpha$ (0 $< \alpha \le$ 1). The cross-correlation [Eq. (1)] corresponds to calculating the correlation of a single image in which all image types are concatenated.
Fig. 1. Multiple-image types of en face images of an AMD patient. (a) OCT-A average projection of the whole depth and (b) OCT average amplitude, (c) average logarithmic intensity, and (d) maximum intensity projections. OCT projections exhibit spatial patterns due to morphological abnormalities of the retina.
To compute the cross-correlation of images with arbitrary shapes fast, Eqs. (2)– (4) are converted to forms with a convolution operation (using the Fourier transform) with masked images as in [14,31]. Multi-image registration [Eq. (1)] is then used to estimate eye movements, and the estimated movements are used to suppress motion artifacts. The details of the motion estimation are described in Section 2.4.
2.2 Image types for lateral motion estimation
In the method proposed in this study, OCT-A average projection along the entire depth, OCT average amplitude projection (AP), OCT average logarithmic intensity projection (LIP), and OCT maximum intensity projection (MIP) are used. An example of these image types is shown in Fig. 1, and these images have been motion-corrected using the method described in Section 2.4. Using multiple types of OCT projections increases the amount of information regarding the axial distribution of the OCT signal.
OCT-A obtained by noise-corrected complex correlation method [32] is used. En face OCT-A images are generated by averaging the three-dimensional OCT-A volume over the whole depth. The en face OCT-A projection visualize the vasculature with high contrast because the OCT-A method [32] suppresses de-correlation at low signal regions.
OCT AP, LIP, and MIP are calculated as followings:
where $A$ is the amplitude of OCT signal, $m$ is the index of a voxel in en face plane, and $n$ is the index of a voxel along depth. A total number of voxel along depth is $N$. OCT AP provides a similar image to that obtained by confocal scanning ophthalmoscope. In contrast, OCT LIP applies logarithmic scaling before depth averaging and enhances the contrast of structure at depths with weak backscattered light. OCT MIP represents the highest backscattering intensity. In posterior eye imaging, it represents the OCT intensity scattered at the RPE, because the RPE exhibits very strong backscattering. This projection is similar to a “shadow gram” [33], which is the projection image of the OCT signal extracted around the RPE. The shadows of high scattering and absorption materials such as retinal blood are visualized in this type of image.2.3 Experimental system
A clinical-grade prototype OCT system [34] was used for the experiment. It uses 1-$\mu$m swept-source OCT with polarization-diversity detection. The system has a scan rate of 100,000 A-lines per second and 6.2-$\mu$m axial resolution. The system was installed in a hospital and patients’ eyes were scanned.
The modified Lissajous scan pattern, which enables OCT and OCT-A imaging [15], was applied. An imaging range of 3 mm $\times$ 3 mm was scanned. During one fast scanning cycle of the Lissajous pattern, 722 A-lines are acquired. The fast scanning cycle was repeated twice to scan the same locations and enable OCT-A imaging. The OCT amplitude, $A_{m,n}$, is also obtained by averaging A-lines among repeated cycles. The time lag between the repeated fast cycles was 7.24 ms and, a total of 724 repeated-cycle-sets were acquired for one volume. The acquisition time of one volume was about 10 s.
2.4 Motion estimation and correction
The processing flow of the lateral motion estimation is shown in Fig. 2. The algorithm for lateral motion estimation presented in our previous papers [14,15] is used. Here, the algorithm is identical to that is used in Ref. [15], except for the image cross-correlation computation, which is computed by Eq. (1) in the present study.
En face images of multiple types are produced from the acquired volume and split into small strips. The data affected by eye blinks and significant eye motion are determined by detecting reductions in OCT intensity and high de-correlation noise in OCT-A, respectively. The OCT LIP is used to detect eye blinks. The OCT LIP was averaged in one repeat-cycle-set of Lissajous and subtracted by noise floor:
The strips are transformed into a Cartesian grid (361 $\times {}$ 361). The translation between a target strip and a reference image is estimated using the multi-image registration (Section 2.1). To equalize the influence of multiple image types, each image type is normalized by its standard deviation before splitting into strips as follows:
The motion-corrected en face images are obtained by correcting the translation of each sampling point using the estimated lateral motion and mapping them to the Cartesian grid. The data assigned to the same grid point are averaged. For slab OCT-A images, average projections of OCT-A at the segmented layers are calculated and mapped to the Cartesian grid using motion correction. This motion-corrected slab OCT-A has been enhanced by the contrast-limited adaptive histogram equalization (CLAHE) [35] and Gabor filtering [15,36].
Motion estimation processing was performed using Python 3.7 scripts with NumPy and CuPy (for using CuFFT to compute convolutions in the cross-correlation process). The retinal layers were segmented using The Iowa Reference Algorithms (Retinal Image Analysis Lab, Iowa Institute for Biomedical Imaging, Iowa City, IA) [37–39]. Image processing (Gabor filtering, CLAHE, and rolling ball filtering) was performed using scikit-image 0.16.2.
3. Motion correction evaluation
A total of 91 eyes of 76 subjects were scanned. The number of patients with each disease was 46 with age-related macular degeneration, 10 with polypoidal choroidal vasculopathy (PCV), 10 with retinal vein occlusion, 4 with pathologic myopia, 3 with diabetic macular edema (DME), and 2 with central serous retinopathy, 1 idiopathic choroidal neovascularization. A total of 116 volumes of all subjects were used for all evaluations presented in this study. The present study adhered to the tenets of the Declaration of Helsinki and was approved by the Institutional Review Boards of Tokyo Medical University. The nature of the present study and the implications of participating in this research project were explained to all study participants, and written informed consent was obtained from each participant before any study procedures or examinations were performed.
To evaluate the performance of motion correction using multiple-image types, two motion correction methods were compared. One uses the motion estimated using only the en face OCT-A projection (OCT-A method) and the other uses the estimation with OCT-A and OCT projections (OCT-A + OCT method). The imaging quality of motion-corrected image and motion correction performance were assessed by the metrics as described in the following subsections.
The assessed imaging quality was also compared with respect to the number of detected rapid eye movements and eye blinks (see Section 2.4). This enabled the dependency of the motion correction performance on eye motion and blinks to be compared for both methods.
3.1 Assessment of OCT-A image quality
The image quality of a motion-corrected image is an essential criterion for evaluating motion correction methods. Here, we performed manual grading to assess the image quality of motion-corrected images. A grading of vasculature visibility was conducted by a single senior ophthalmologist with a clinical experience of more than 20 years (grader 1). To verify the consistency of the grading, another grader (a senior engineer, grader 2) rated the same image set, and inter-grader consistency was assessed. OCT-A images of a superficial layer (superficial OCT-A), the slab from the inner limiting membrane to the boundary between the inner nuclear and plexiform layers, were graded based on the grading criteria of vasculature visibility [Table 1], similar to the previous OCT-A study [40]. Motion-corrected en face OCT-A images obtained by motion correction with the OCT-A method and OCT-A + OCT method were presented in a random sequence to graders. Hence, it was a blind grading.
Table 1. Grading criteria
3.2 Metrics of motion correction
Two metrics were used in this study. After the motion correction, sampling data along the Lissajous trajectory is remapped to Cartesian grid points. If several strips are not registered, it could result in blanks in the motion-corrected data, i.e., some Cartesian grid points that do not have assigned data. The ratio of filled data in the Cartesian grid represents the degree of the success of the motion-correction. Hence, the first of two metrics, “filling rate”, is defined as follows:
The second metric is the usage rate of A-lines, defined as follows:
If image registration is improved, the number of registered strips will increase and the number of A-lines used in the motion-corrected image will accordingly increase.
4. Motion correction and evaluation results
4.1 Example cases of motion-corrected retinal images
Motion-corrected images of some representative cases are described here. The cases that grading score was improved with OCT-A + OCT method over OCT-A method (Fig. 3 and Fig. 4), not changed (Fig. 5), and degraded (Fig. 6) are presented.
Fig. 3. Representative motion-corrected en face images of a case improved by the proposed image registration method. Images with the lateral motion estimated using (a–e) the OCT-A method (filling rate: 0.996, A-line usage: 0.747), and (f–j) OCT-A + OCT method (filling rate: 0.996, A-line usage: 0.918). Motion-corrected images of (a, f) OCT-A of whole depth, (b, g) OCT amplitude, (c, h) OCT logarithmic intensity, (d, i) OCT maximum intensity, and (e, j) superficial OCT-A projections are shown. The enlarged images (k, l) shows that elliptical artifacts are suppressed by the OCT-A + OCT method. (m–q) En face images without motion correction show blurred and ghost structures. The yellow box indicates the projection types that were used for motion estimation. A BRVO patient eye (number of blinks: 1, number of eye movements: 20) was scanned.
Fig. 4. Representative motion-corrected en face images of a case improved by the proposed image registration method. Images with the lateral motion estimated using (a–e) OCT-A method (filling rate: 0.996, A-line usage: 0.867), and (f–j) OCT-A + OCT method (filling rate: 0.999, A-line usage: 0.855). Motion-corrected images of (a, f) OCT-A of whole depth, (b, g) OCT amplitude, (c, h) OCT Log-intensity, (d, i) OCT maximum intensity, and (e, j) superficial OCT-A projections are shown. The enlarged images (k, l) show that the OCT-A + OCT method sharpens structures. (m–q) En face images without motion correction show blurred and ghost structures. The yellow box indicates the projection types that were used for motion estimation. A DME patient eye (number of blinks: 0, number of eye movements: 28) was scanned.
Fig. 5. Representative motion-corrected en face images of a case with improved by the proposed image registration method. Images with the lateral motion estimated using (a–e) OCT-A method (filling rate: 0.987, A-line usage: 0.893), and (f–j) OCT-A + OCT method (filling rate: 1.00, A-line usage: 0.970). Motion-corrected images of (a, f) OCT-A of whole depth, (b, g) OCT amplitude, (c, h) OCT Log-intensity, (d, i) OCT maximum intensity, and (e, j) superficial OCT-A projections are shown. The enlarged images (k, l) show that the OCT-A + OCT method eliminates the blank areas and duplicated blood vessels and sharpens some retinal capillaries. (m–q) En face images without motion correction show blurred and ghost structures. The yellow box indicates the projection types that were used for motion estimation. An AMD patient eye with CNV (number of blinks: 0, number of eye movements: 7) was scanned.
Fig. 6. Representative motion-corrected en face images of a case degraded by the proposed image registration method. Images with the lateral motion estimated using (a–e) OCT-A method (filling rate: 0.994, A-line usage: 0.672), and (f–j) OCT-A + OCT method (filling rate: 0.997, A-line usage: 0.672). Motion-corrected images of (a, f) OCT-A of whole depth, (b, g) OCT amplitude, (c, h) OCT Log-intensity, (d, i) OCT maximum intensity, and (e, j) superficial OCT-A projections are shown. The enlarged images (k, l) show that the OCT-A + OCT method blurs structures and generates duplication of blood vessels. (m–q) En face images without motion correction show blurred and ghost structures. The yellow box indicates the projection types that were used for motion estimation. A PCV patient eye (number of blinks: 0, number of eye movements: 20) was scanned.
Figure 3 shows the en face images of a case in which the grading score was improved from 3 to 4 and elliptic artifacts are suppressed. A branch retinal vein occlusion (BRVO) patient eye was scanned. Fine structures are restored in motion-corrected images (Figs. 3(a)–3(l)) comparing to images without motion correction (Figs. 3(m)–3(q)). Artifacts appear when the OCT-A method is used, as shown in superficial OCT-A image (Fig. 3(e)), as apparent in the enlarged image (inside the white ellipse of Fig. 3(k)). These artifacts disappear when the OCT-A + OCT method is used (Fig. 3(j) and Fig. 3(l)). This may occur because the multi-image cross-correlation discards the data with artifacts or the averaging with a higher number of sampling points suppress the artifacts. The A-line usage is increased from 0.747 to 0.918. The OCT-A + OCT method rescues 89,000 sampling points that are discarded when the OCT-A method is used. In contrast, the filling rate is the same because it is already very high (0.996) without using OCT projections for motion estimation. The elliptical artifacts disturb the visibility of the retinal capillaries. Hence, the grading score is increased as these artifacts are suppressed.
Another case in which the grading score was improved from 2 to 3 is a DME patient’s eye, as shown in Fig. 4. In both methods, the eye movements were corrected somehow (Figs. 4(a)–4(l)) by comparing with images without motion correction (Figs. 4(m)–4(q)). However, the boundaries of some low-intensity spots (Figs. 4(b)–4(d), and green, magenta, and blue panes in Fig. 4(k)) are sharpened more by the proposed image registration method (Figs. 4(g)–4(i) and green, magenta, and blue panes in Fig. 4(l)). Some retinal vessels in the motion-corrected en face superficial OCT-A with the OCT-A method (Fig. 4(e) and orange and brown panes in Fig. 4(k)) became sharp when the OCT-A + OCT method is used (Fig. 4(j) and orange and brown panes in Fig. 4(l)). The grading score is improved and the filling rate is also slightly increased. In contrast, the A-line usage decreases. This could be because some strips containing noise were discarded during the motion estimation using multi-image registration.
The motion-corrected images of a case in which the grading score does not change are shown in Figs. 5(a)–5(l). In this case, an AMD patient with choroidal neovascularization (CNV) was scanned. Motion correction with the OCT-A method leads to blank regions on the left side (green arrowheads in Fig. 5(k)). Moreover, some blood vessels are duplicated (magenta arrowheads in Fig. 5(k)) which is also appeared in images without motion correction (Fig. 5(m) and Fig. 5(q)). It suggests that the motion correction has failed for one microsaccade. The blank regions and duplicated vessels disappear when the OCT-A + OCT method is used (Figs. 5(f)–5(i) and green and magenta panes in Fig. 5(l)). Further, some of the retinal capillaries are sharpened (blue arrowheads in Fig. 5(l)). However, the grading score does not change because its criteria are based on only the visibility of the entire vasculature. Namely, the manual grading does not take into account the small blank areas and ghost vessels that do not decrease the contrast in the vessels. The filling rate increases from 0.987 to 1.00 because the blanks disappear, and the A-line usage improves from 0.893 to 0.970. These increases suggest that image registration performance is improved.
An apparent increase in motion correction errors appears in one of the cases in which the grading score decreased from 3 to 2. En face motion-corrected images (a PCV case) are shown in Figs. 6(a)–6(l). The images without motion correction (Figs. 6(m)–6(q)) indicate that there are several large microsaccades during data acquisition. When the OCT-A + OCT method is used, some vessels are duplicated (blue arrowheads in Fig. 6(l)), and several fine vessels (orange circles in Fig. 6(k)) become less visible (orange circles in Fig. 6(l)) with the OCT-A + OCT method. Several high scattering spots visualized in the OCT MIP (magenta arrowheads in Fig. 6(k)) are blurred by the multi-image registration (orange pane in Fig. 6(l)). The OCT projections (Figs. 6(b)–6(d)) do not have clear signatures in contrast to those of the improved cases (Figs. 3–5). Moreover, the OCT-A projection (Fig. 6(a)) is noisy. Hence, image registration using all of these image types might decrease motion estimation performance.
4.2 OCT-A image quality evaluation results
The imaging quality of all 232 motion-corrected superficial OCT-A images (116 volumes $\times$ 2 motion correction methods) was rated by two graders. The inter-grader consistency was assessed using a two-way, consistency, average-measures intraclass correlation coefficient (ICC) to assess the degree that graders provided consistency in their ratings of image quality across images [41]. The resulting ICC was in the excellent range, ICC = 0.896 (p-value = 3.82 $\times {} 10^{-56}$), indicating that graders had a high degree of agreement and suggesting that image quality was rated similarly across graders. The high ICC suggests that a minimal amount of measurement error was introduced by the independent graders. We used grader 1 scores for subsequent analyses because grader 1 has extensive clinical experience.
The grading results are summarized in Table 2. The average scores are 2.94 and 3.19 for the OCT-A method and OCT-A + OCT method, respectively. The grading score difference is statistically significant (two-tailed Wilcoxon’s signed-rank test, p-value = 7.55 $\times {} 10^{-4}$). These results show that motion estimation using multiple-image types improves the quality of motion-corrected en face imaging.
Table 2. Grading results. Frequencies of each score.
4.2.1 Effects of eye movements and blinks on imaging quality
The number of eye movements of each grading score is plotted in Fig. 7(a). The result shows that the number of eye motions is associated with the imaging quality of the motion-corrected superficial OCT-A image, i.e., the grading score. The association was evaluated by Kendall’s correlation test because the dependent variable (grading score) is ordinal data. Kendall’s tau-b is -0.324 (p-value = 4.78 $\times {} 10^{-6}$) and -0.309 (p-value = 1.17 $\times {} 10^{-5}$) for the cases of motion correction with the OCT-A and OCT-A + OCT methods, respectively. There is a significant negative association between the number of eye movements and the grading score. A large number of eye motions are a significant factor of low imaging quality in the motion-corrected image. The value of tau-b is slightly increased when the OCT-A + OCT method is used. This indicates that the association between the number of eye movements and the grading score is weakened when multi-image registration is used. Namely, the OCT-A + OCT method is more robust to eye motion.
Fig. 7. Distribution of the number of (a) eye motions and (b) blinks for each grading score. They are grouped by motion correction methods (blue dots for the OCT-A method and orange dots for the OCT-A + OCT method).
In contrast, the number of blinks seems to be unassociated with the grading score (Fig. 7(b)). The degree of this association was evaluated by Kendall’s correlation test. The value of $\tau _b$ were -0.0330 (p-value = 0.6879) and 0.000252 (p-value = 0.9975) for the cases obtained by motion correction using the OCT-A and OCT-A + OCT methods, respectively. There are no significant associations between imaging quality and eye blinks. Hence, neither motion-correction method is affected by the number of eye blinks.
4.3 Objective evaluation results of motion correction
The performance of the lateral motion estimation was evaluated by the filling rate of the imaging range (361 $\times {}$ 361 grids) in the motion-corrected image. A total of 116 volumes were included in this analysis.
The filling rate increases from 0.987 to 0.996 when additional OCT projections for the lateral motion estimation are used. However, there is no significant difference between motion correction methods (two-tailed Wilcoxon’s signed-rank test (Pratt), p-value = 0.05). The differences in the filling rate of the OCT-A and OCT-A + OCT methods with respect to the filling rate of the OCT-A method are plotted in Fig. 8(a). The filling rate is already very close to unity in almost all cases when the OCT-A method is used. Hence, the change in the filling rate is very small in these cases. However, there are large improvements in the cases that have a low filling rate.
Fig. 8. Differences in (a) filling rate and (b) A-line usage of the motion correction methods using the OCT-A and OCT-A + OCT methods.
The A-line usage also increased from 0.840 (mean) to 0.871 (mean). The difference is statistically significant (two-tailed Wilcoxon’s signed-rank test (Pratt), p-value = 2.41 $\times {} 10^{-10}$). Figure 8(b) shows the differences in the A-line usage of the OCT-A and OCT-A + OCT methods according to the A-line usage of the OCT-A method. Several cases exhibit significant improvements.
5. Discussion
5.1 Contribution of multiple projection images
In this study, we used the OCT-A and OCT en face projections for lateral eye motion estimation to employ features of vasculature and scattering distribution. Using two-dimensional en face images reduces the time and memory consumption of motion estimation; however, projection along the depth removes the features of axial vasculature and morphology. Using several projection types of a volume is a compromise that employs the information of the axial distribution in the lateral motion estimate. Several types of OCT projections, namely amplitude, logarithmic intensity, and maximum intensity projections from one OCT volume (Section 2.2) are used in this study. All of them exhibit different signatures, especially in the cases with retinal abnormalities as shown in Figs. 3–5. Using these different projection types might contribute to improvements in motion correction for images of retinal disease.
In contrast, if additional features are small as shown in Fig. 6, the adding image types will increase the effect of noise. In this paper, the simple normalization using variations of images (Section 2.4) was applied. More sophisticated weighting methods based on the amount of features of each image type will mitigate the drawback of adding image types.
Additional en face images for motion estimation would improve performance. Slab projections of OCT-A and OCT possess different features and will be promising image types for motion correction. Several previous studies used slab OCT-A [12,13]. However, retinal layer segmentation is required for slab projection, and pathologic cases can lead to segmentation errors. In addition, optimization for Lissajous data will be required, as discussed in the following (Section 5.5).
5.2 Processing time
By using multiple image types, the processing time of motion correction is increased. The OCT-A + OCT method used 4 image types; the data size used for the motion estimation is four times larger than that of the OCT-A method. The processing time of the lateral motion estimation per volume was roughly 149 seconds with the OCT-A method and 177 seconds with OCT-A + OCT method. The personal computer with a central processing unit (CPU: Core i9–7900X, Intel Corporation), 128 GB RAM, and a graphical processing unit (GPU: GeForce GTX 1080 Ti, Nvidia Corporation) has been used. The processing time did not increase more than 20 %. An image size of one en face image is not large. Hence, the increase of the processing time is approximately only due to the multi-image cross-correlation.
In this paper, only the cross-correlation was processed by the GPU. In the future, all the processing can be done by GPU to reduce the processing time.
5.3 Image quality assessment
The OCT-A image quality assessment has been done as shown in Section 4.2. Only a single grader has rated the image quality, hence, there will be a grader bias. The absolute score of rating in this study may not be appropriate for comparison with other studies. However, this bias is consistent across all grading and is not an issue in the current motion-correction study.
5.4 Metrics to evaluate motion correction
In this study, we used a subjective method to assess the image quality of motion-corrected images. The vasculature visibility in en face superficial OCT-A images were manually graded and used to evaluate the motion correction performance. For a more stable evaluation of motion correction performance, objective assessment methods of motion correction errors in the motion-corrected images are preferred.
The lateral motion of a sample causes blurring and duplication of the structure in Lissajous scan imaging [14]. Several metrics such as the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) [17,42–45] are used to assess OCT-A image quality. They can be used to assess the sharpness of vasculature in OCT-A images. Motion correction sharpens the structure, and hence the brightness and contrast of vasculature in OCT-A images will be increased. However, the separation of the signal and background is required to calculate the SNR or CNR. An automatic separation is challenging for an en face OCT-A image [43], and the manual demarcation of the background and vessel regions takes time for large amounts of data. The existing semi-automatic method [43] is not valid for images with motion correction errors.
An entropy-based metric obtained by calculating the information entropy by taking an image as a distribution was used to quantify the sharpness of the image [46,47]. However, this metric cannot be used to assess the quality of motion correction in this study. The normalized information entropy was calculated by taking the motion-corrected superficial OCT-A image as a distribution. This metric did not indicate an improvement in image sharpness in this study. The entropy-based metric increases from 0.993 to 0.994 when the proposed multi-image registration is used, i.e., image sharpness as quantified by this metric decreases. This increase in the entropy-based metric may be because more data were averaged. The appearance of fine structure in the image lowers the entropy-based metric, but pepper noise in the image also lowers this metric. Motion correction with multi-image registration increases the number of A-lines used in the motion-corrected image. Hence, the number of A-lines used in the averaging at a grid point is increased, and this reduces the noise. This tends to increase the entropy-based metric. Hence, the entropy-based metric is not suitable for evaluating the sharpness of the vasculature alone.
Vessel connectivity is another metric that is used to assess OCT-A images [17,44]. However, it is not suitable for imaging with a Lissajous scan because the motion-correction error does not provide discontinuous structures.
We used manual grading to assess the visibility of the vasculature. Although it is not an objective method and directly assesses the motion correction errors, the statistically significant increase in the grading score proves that the motion correction performance obtained with the OCT-A + OCT method is higher than that obtained with the OCT-A method. The metrics of motion correction used in this study, i.e., filling rate and A-line usage, were increased by the OCT-A + OCT method. This suggests that multi-image cross-correlation increases the number of successful registrations. An objective metric of the motion correction performance from the motion-corrected images will require some sophisticated assessment algorithms to be developed, and this will be a future task.
5.5 Residual artifacts in slab OCT-A
To generate slab OCT-A, a layer segmentation was applied before the motion correction and re-mapping of the data. The segmentation software [The Iowa Reference Algorithms (Retinal Image Analysis Lab, Iowa Institute for Biomedical Imaging, Iowa City, IA) [37–39]] was used. However, this is not optimized for Lissajous data. Axial eye motion causes discrepancies between the real boundaries and segmented lines because they are smoothed between adjacent frames. Segmentation errors cause elliptical artifacts in the slab OCT-A images. Hence, this image was not used for motion estimation. The grading score may be decreased by these artifacts. To remove these artifacts, axial eye motion correction before the layer segmentation or the segmentation in the motion-corrected OCT volume will be necessary.
6. Conclusion
A lateral motion correction that uses multi-image types was demonstrated in this study. The image quality of the motion-corrected en face OCT-A image, motion-corrected image size, and the number of registered A-lines were evaluated. Two motion correction methods, one with the OCT-A method, which uses only the en face OCT-A projection, and the OCT-A + OCT method, which uses the en face OCT-A and three OCT projections, were evaluated. The results of both methods indicate that image registration using multiple-image types improves the successful motion-free imaging of patient eyes. The proposed image registration method improves motion correction.
Funding
Japan Science and Technology Agency (JPMJMI18G8); Japan Society for the Promotion of Science (15K13371, 17K14121, 18H01893, 18K09460).
Disclosures
SM: Topcon (F), Tomey Corporation (F, P), Kao (F), Yokogawa (F), Nikon (F), Sky Technology (F); MM: Alcon (F), Novartis (F, R), Santen (F, R), Senju (F). SA: Topcon (E); TM: Topcon (E); TY: Topcon (E); YY: Topcon (F), Tomey Corporation (F, P), Kao (F), Yokogawa (F), Nikon (F), Sky Technology (F);
References
1. D. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, “Optical coherence tomography,” Science 254(5035), 1178–1181 (1991). [CrossRef]
2. A. Baghaie, Z. Yu, and R. M. D’Souza, “Involuntary eye motion correction in retinal optical coherence tomography: Hardware or software solution?” Med. Image Anal. 37, 129–145 (2017). [CrossRef]
3. G. Taibbi, G. C. Peterson, M. F. Syed, and G. Vizzeri, “Effect of motion artifacts and scan circle displacements on Cirrus HD-OCT retinal nerve fiber layer thickness measurements,” Invest. Ophthalmol. Visual Sci. 55(4), 2251–2258 (2014). [CrossRef]
4. Y. K. Tao, S. Farsiu, and J. A. Izatt, “Interlaced spectrally encoded confocal scanning laser ophthalmoscopy and spectral domain optical coherence tomography,” Biomed. Opt. Express 1(2), 431–440 (2010). [CrossRef]
5. K. V. Vienola, B. Braaf, C. K. Sheehy, Q. Yang, P. Tiruveedhula, D. W. Arathorn, J. F. de Boer, and A. Roorda, “Real-time eye motion compensation for OCT imaging with tracking SLO,” Biomed. Opt. Express 3(11), 2950–2963 (2012). [CrossRef]
6. B. Braaf, K. V. Vienola, C. K. Sheehy, Q. Yang, K. A. Vermeer, P. Tiruveedhula, D. W. Arathorn, A. Roorda, and J. F. de Boer, “Real-time eye motion correction in phase-resolved OCT angiography with tracking SLO,” Biomed. Opt. Express 4(1), 51–65 (2013). [CrossRef]
7. R. Zawadzki, A. Capps, D. Kim, A. Panorgias, S. Stevenson, B. Hamann, and J. Werner, “Progress on developing adaptive optics–optical coherence tomography for in vivo retinal imaging: Monitoring and correction of eye motion artifacts,” IEEE J. Sel. Top. Quantum Electron. 20(2), 322–333 (2014). [CrossRef]
8. M. Sugita, S. Zotter, M. Pircher, T. Makihira, K. Saito, N. Tomatsu, M. Sato, P. Roberts, U. Schmidt-Erfurth, and C. K. Hitzenberger, “Motion artifact and speckle noise reduction in polarization sensitive optical coherence tomography by retinal tracking,” Biomed. Opt. Express 5(1), 106–122 (2014). [CrossRef]
9. X. Wei, T. T. Hormel, Y. Guo, T. S. Hwang, and Y. Jia, “High-resolution wide-field OCT angiography with a self-navigation method to correct microsaccades and blinks,” Biomed. Opt. Express 11(6), 3234–3245 (2020). [CrossRef]
10. M. F. Kraus, B. Potsaid, M. A. Mayer, R. Bock, B. Baumann, J. J. Liu, J. Hornegger, and J. G. Fujimoto, “Motion correction in optical coherence tomography volumes on a per A-scan basis using orthogonal scan patterns,” Biomed. Opt. Express 3(6), 1182–1199 (2012). [CrossRef]
11. M. F. Kraus, J. J. Liu, J. Schottenhamml, C.-L. Chen, A. Budai, L. Branchini, T. Ko, H. Ishikawa, G. Wollstein, J. Schuman, J. S. Duker, J. G. Fujimoto, and J. Hornegger, “Quantitative 3D-OCT motion correction with tilt and illumination correction, robust similarity measure and regularization,” Biomed. Opt. Express 5(8), 2591–2613 (2014). [CrossRef]
12. P. Zang, G. Liu, M. Zhang, C. Dongye, J. Wang, A. D. Pechauer, T. S. Hwang, D. J. Wilson, D. Huang, D. Li, and Y. Jia, “Automated motion correction using parallel-strip registration for wide-field en face OCT angiogram,” Biomed. Opt. Express 7(7), 2823–2836 (2016). [CrossRef]
13. M. Heisler, S. Lee, Z. Mammo, Y. Jian, M. Ju, A. Merkur, E. Navajas, C. Balaratnasingam, M. F. Beg, and M. V. Sarunic, “Strip-based registration of serially acquired optical coherence tomography angiography,” J. Biomed. Opt. 22(3), 036007 (2017). [CrossRef]
14. Y. Chen, Y.-J. Hong, S. Makita, and Y. Yasuno, “Three-dimensional eye motion correction by Lissajous scan optical coherence tomography,” Biomed. Opt. Express 8(3), 1783–1802 (2017). [CrossRef]
15. Y. Chen, Y.-J. Hong, S. Makita, and Y. Yasuno, “Eye-motion-corrected optical coherence tomography angiography using Lissajous scanning,” Biomed. Opt. Express 9(3), 1111–1129 (2018). [CrossRef]
16. S. Makita, Y. Hong, M. Yamanari, T. Yatagai, and Y. Yasuno, “Optical coherence angiography,” Opt. Express 14(17), 7821–7840 (2006). [CrossRef]
17. Y. Jia, O. Tan, J. Tokayer, B. Potsaid, Y. Wang, J. J. Liu, M. F. Kraus, H. Subhash, J. G. Fujimoto, J. Hornegger, and D. Huang, “Split-spectrum amplitude-decorrelation angiography with optical coherence tomography,” Opt. Express 20(4), 4710–4725 (2012). [CrossRef]
18. R. K. Wang, L. An, P. Francis, and D. J. Wilson, “Depth-resolved imaging of capillary networks in retina and choroid using ultrahigh sensitive optical microangiography,” Opt. Lett. 35(9), 1467–1469 (2010). [CrossRef]
19. W. Choi, K. J. Mohler, B. Potsaid, C. D. Lu, J. J. Liu, V. Jayaraman, A. E. Cable, J. S. Duker, R. Huber, and J. G. Fujimoto, “Choriocapillaris and choroidal microvasculature imaging with ultrahigh speed OCT angiography,” PLoS One 8(12), e81499 (2013). [CrossRef]
20. M. L. Gabriele, G. Wollstein, H. Ishikawa, J. Xu, J. Kim, L. Kagemann, L. S. Folio, and J. S. Schuman, “Three dimensional optical coherence tomography imaging: Advantages and advances,” Prog. Retinal Eye Res. 29(6), 556–579 (2010). [CrossRef]
21. J. Wanek, R. Zelkha, J. I. Lim, and M. Shahidi, “Feasibility of a method for en face imaging of photoreceptor cell integrity,” Am. J. Ophthalmol. 152(5), 807–814.e1 (2011). [CrossRef]
22. M. Rispoli, J.-F. Le Rouic, G. Lesnoni, L. Colecchio, S. Catalano, and B. Lumbroso, “Retinal surface en face optical coherence tomography: A new imaging approach in epiretinal membrane surgery,” Retina 32(10), 2070–2076 (2012). [CrossRef]
23. R. P. Nunes, G. Gregori, Z. Yehoshua, P. F. Stetson, W. Feuer, A. A. Moshfeghi, and P. J. Rosenfeld, “Predicting the progression of geographic atrophy in age-related macular degeneration with SD-OCT en face imaging of the outer retina,” Ophthalmic Surg Lasers Imaging Retin. 44(4), 344–359 (2013). [CrossRef]
24. F. Mohammad, J. Wanek, R. Zelkha, J. I. Lim, J. Chen, and M. Shahidi, “A method for en face OCT imaging of subretinal fluid in age-related macular degeneration,” J. Ophthalmol. 2014, 1–6 (2014). [CrossRef]
25. R. F. Spaide, “Colocalization of pseudodrusen and subretinal drusenoid deposits using high-density en face spectral domain optical coherence tomography,” Retina 34(12), 2336–2345 (2014). [CrossRef]
26. M. F. Clamp, G. Wilkes, L. S. Leis, H. R. McDonald, R. N. Johnson, J. M. Jumper, A. D. Fu, E. T. J. Cunningham, P. J. Stewart, S. J. Haug, and B. J. Lujan, “En face spectral domain optical coherence tomography analysis of lamellar macular holes,” Retina 34(7), 1360–1366 (2014). [CrossRef]
27. D. Ferrara, K. J. Mohler, N. Waheed, M. Adhi, J. J. Liu, I. Grulkowski, M. F. Kraus, C. Baumal, J. Hornegger, J. G. Fujimoto, and J. S. Duker, “En face enhanced-depth swept-source optical coherence tomography features of chronic central serous chorioretinopathy,” Ophthalmology 121(3), 719–726 (2014). [CrossRef]
28. M. C. Savastano, M. Rispoli, A. Savastano, and B. Lumbroso, “En face optical coherence tomography for visualization of the choroid,” Ophthalmic Surg Lasers Imaging Retin. 46(5), 561–565 (2015). [CrossRef]
29. K. B. Schaal, A. D. Legarreta, G. Gregori, J. E. Legarreta, Q. Cheng, P. F. Stetson, M. Cai, M. Laron, M. Durbin, and P. J. Rosenfeld, “Widefield en face optical coherence tomography imaging of subretinal drusenoid deposits,” Ophthalmic Surg Lasers Imaging Retin. 46(5), 550–559 (2015). [CrossRef]
30. R. F. Spaide, L. Yannuzzi, K. B. Freund, R. Mullins, and E. Stone, “Eyes with subretinal drusenoid deposits and no drusen: Progression of macular findings,” Retina 39(1), 12–26 (2019). [CrossRef]
31. D. Padfield, “Masked object registration in the Fourier domain,” IEEE Trans. Image Process. 21(5), 2706–2718 (2012). [CrossRef]
32. S. Makita, K. Kurokawa, Y.-J. Hong, M. Miura, and Y. Yasuno, “Noise-immune complex correlation for optical coherence angiography based on standard and Jones matrix optical coherence tomography,” Biomed. Opt. Express 7(4), 1525–1548 (2016). [CrossRef]
33. S. Jiao, R. Knighton, X. Huang, G. Gregori, and C. A. Puliafito, “Simultaneous acquisition of sectional and fundus ophthalmic images with spectral-domain optical coherence tomography,” Opt. Express 13(2), 444–452 (2005). [CrossRef]
34. S. Makita, T. Mino, T. Yamaguchi, M. Miura, S. Azuma, and Y. Yasuno, “Clinical prototype of pigment and flow imaging optical coherence tomography for posterior eye investigation,” Biomed. Opt. Express 9(9), 4372–4389 (2018). [CrossRef]
35. K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in Graphics Gems IV, P. S. Heckbert, ed. (Academic Press, 1994), pp. 474–485.
36. H. C. Hendargo, R. Estrada, S. J. Chiu, C. Tomasi, S. Farsiu, and J. A. Izatt, “Automated non-rigid registration and mosaicing for robust imaging of distinct retinal capillary beds using speckle variance optical coherence tomography,” Biomed. Opt. Express 4(6), 803–821 (2013). [CrossRef]
37. K. Li, X. Wu, D. Chen, and M. Sonka, “Optimal surface segmentation in volumetric images-A graph-theoretic approach,” IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 119–134 (2006). [CrossRef]
38. M. Garvin, M. Abramoff, X. Wu, S. Russell, T. Burns, and M. Sonka, “Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images,” IEEE Trans. Med. Imaging 28(9), 1436–1447 (2009). [CrossRef]
39. B. Antony, M. D. Abràmoff, L. Tang, W. D. Ramdas, J. R. Vingerling, N. M. Jansonius, K. Lee, Y. H. Kwon, M. Sonka, and M. K. Garvin, “Automated 3-D method for the correction of axial artifacts in spectral-domain optical coherence tomography images,” Biomed. Opt. Express 2(8), 2403–2416 (2011). [CrossRef]
40. G. Anastassiou, A. Elmajie, C. Smok, and S. Kremmer, “Real-life performance of swept-source optical coherence tomography angiography,” Spektrum Augenheilkd. 31(6), 245–250 (2017). [CrossRef]
41. K. A. Hallgren, “Computing inter-rater reliability for observational data: an overview and tutorial,” Tutor. Quant. Methods Psychol. 8(1), 23–34 (2012). [CrossRef]
42. A. Mariampillai, M. K. K. Leung, M. Jarvi, B. A. Standish, K. Lee, B. C. Wilson, A. Vitkin, and V. X. D. Yang, “Optimized speckle variance OCT imaging of microvasculature,” Opt. Lett. 35(8), 1257–1259 (2010). [CrossRef]
43. A. Lozzi, A. Agrawal, A. Boretsky, C. G. Welle, and D. X. Hammer, “Image quality metrics for optical coherence angiography,” Biomed. Opt. Express 6(7), 2435–2447 (2015). [CrossRef]
44. Zhang Anqi, Zhang Qinqin, Chen Chieh-Li, and K. Wang Ruikang, “Methods and algorithms for optical coherence tomography-based angiography: A review and comparison,” J. Biomed. Opt. 20, 1–13 (2015). [CrossRef]
45. I. Gorczynska, J. V. Migacz, R. J. Zawadzki, A. G. Capps, and J. S. Werner, “Comparison of amplitude-decorrelation, speckle-variance and phase-variance OCT angiography methods for imaging the human retina and choroid,” Biomed. Opt. Express 7(3), 911–942 (2016). [CrossRef]
46. Y. Yasuno, Y. Hong, S. Makita, M. Yamanari, M. Akiba, M. Miura, and T. Yatagai, “In Vivo high-contrast imaging of deep posterior eye by 1-μm swept source optical coherence tomography and scattering optical coherence angiography,” Opt. Express 15(10), 6121–6139 (2007). [CrossRef]
47. Y. Nakamura, S. Makita, M. Yamanari, M. Itoh, T. Yatagai, and Y. Yasuno, “High-speed three-dimensional human retinal imaging by line-field spectral domain optical coherence tomography,” Opt. Express 15(12), 7103–7116 (2007). [CrossRef]
