Open Access
Add to BibSonomy
Abstract
Maxwellian display, as an effective solution to the vergence accommodation conflict in near-eye displays (NEDs), has demonstrated its unique advantages in many aspects, such as the ability to provide sharp images within a certain depth of field (DOF) without being affected by the eye's focus. In recent years, the appearance of holographic Maxwellian displays has addressed the shortcomings of traditional Maxwellian displays, meeting the demands for flexible control parameters, aberration-free designing, and expanded eyebox. Nonetheless, the human eye's requirement for immersion still leaves room for a significant improvement in terms of the field-of-view (FOV). In this paper, we propose a large FOV holographic Maxwellian display based on spherical crown diffraction. The proposed spherical-crown holographic Maxwellian display theoretically can cover the full FOV required by the human eyes without complex optical paths and has flexible control of performance parameters such as DOF and image quality. We have successfully demonstrated the feasibility of the spherical crown diffraction model in lensless holographic Maxwellian displays, and it is expected to have practical applications in the field of holographic Maxwellian NEDs in the future.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
In the past few decades, the development of augmented reality (AR) devices has received increasing attention because it enables users to view and interact with virtual content overlaid onto the real world. Especially near-eye displays (NEDs) can offer better immersion and portable designs. However, early NED devices only provided 2D images, inevitably causing the vergence-accommodation conflict (VAC) [1]. Maxwellian displays, also known as retinal projection displays, can be used to mitigate VAC, which can provide sharp images within a certain depth of field (DOF) range, unaffected by the eye's focus.
Traditional Maxwellian displays utilize a lens system to control light. They suffer from issues of a limited field-of-view (FOV) and eyebox, while parameters like depth of field (DOF) and image quality are fixed and lack flexible control. Furthermore, the use of lenses contributes to the bulkiness and higher costs, and lens aberrations can compromise image quality, especially for color imaging where chromatic aberration caused by lenses is fatal. Later on, holographic optical elements (HOEs) replace lenses to create more compact systems with larger eyebox [2–6]. However, micro-lens structures on some HOEs still lead to significant aberrations, and HOEs lack flexibility since they cannot be changed once manufactured. Despite the use of HOEs, achieving the necessary FOV for human eyes remains a challenge for displays. As a result, many attempts have been made to expand the FOV of Maxwellian displays. In [7], time-division multiplexing is used to improve the integrated FOV of Maxwellian view displays, achieving a FOV of 22.6°. In [8], HOEs are manufactured and used as off-axis lens arrays to expand the FOV of NEDs. With a micro-projector module as the display source, a 60° wide FOV and 10 mm × 10 mm eyebox are successfully achieved. In [9], although the vertical FOV is sacrificed, a doubled horizontal FOV is successfully demonstrated on a waveguide-type Maxwellian NED, compared to the input image.
In recent years, a wave-optics-based holographic Maxwellian NED has been proposed, and the position and width of the beam convergence point can be flexibly controlled through the expected wavefront generated by a planar spatial light modulator (SLM), which is used to extend the eyebox and DOF [10,11]. As a result, the system is compact and has no lens aberrations. Subsequently, a hybrid holographic Maxwellian NED [12] and a color display scheme for holographic Maxwellian NED [13] are proposed based on this. However, for the planar holographic Maxwellian displays, the FOV is limited by the shape and pixel interval of the planar spatial light modulator (SLM), and cannot meet the requirements of a large FOV in high immersion applications [14]. The method of stitching multiple SLMs to increase the FOV of holographic displays is not only complex but also difficult to achieve seamless stitching [15,16]. Curved holographic images are an effective method to overcome these constraints, and spherical holography can achieve the omnidirectional reconstruction of the light field and theoretically provide a larger FOV, which has great significance for immersion. In recent years, there have been many studies based on spherical holography. M. Tachiki et al. propose a fast convolution algorithm based on FFT to calculate spherical holographic images quickly [17]. B. Jackin and T. Yatagai propose a spherical wave spectrum method based on wave propagation defined in spherical coordinates [18]. Y. Sando et al. propose a calculation method based on spherical harmonic transformation [19]. In addition, Liu et al. theoretically derive and verify the spherical crown diffraction model [20]. These studies provide a solid theoretical foundation for the application of spherical holography.
In this paper, we propose a large FOV holographic Maxwellian display based on spherical crown diffraction. Firstly, we derive the diffraction process of the spherical-crown holographic Maxwellian display model using the Rayleigh-Sommerfeld formula, and then demonstrate the different focal depths can provide high-quality imaging at far distances and extension of DOF by controlling the size of the image points. We then compare the proposed model to the planar holographic Maxwellian display and demonstrate the advantages of our model in FOV extension, theoretically covering the full range of human vision without lens aberrations. Through numerical simulation and optical experiments, we have shown the feasibility of the proposed method and the applicability of spherical crown diffraction in lensless holographic Maxwellian display. This method is promising for practical applications in the field of holographic Maxwellian NEDs in the future.
2. Principle
2.1 Holographic Maxwellian display based on spherical crown diffraction
Figure 1(a) shows the principle of spherical-crown holographic Maxwellian display, where the pupil is equivalent to a small spherical surface, and the object sphere, hologram sphere, and pupil sphere are all concentric spherical surfaces with a common center point O. The radii of the object sphere, hologram sphere, pupil sphere are R, rh, re, in which re is relatively small compared to R. The complex amplitude distribution on the object sphere is Us (R,$\varphi $s,$\theta $s), on the hologram sphere is Uh (rh ,$\varphi $h ,$\theta $h), and on the pupil sphere is Ue (re,$\varphi $e,$\theta $e). The eye can be approximated as a spherical surface passing through the center point O, with a front-back diameter of feye. As shown in Fig. 1(b), $\varphi $ represents the angle in the latitude direction, and $\theta $ represents the angle in the longitude direction.
Fig. 1. Holographic Maxwellian display based on spherical crown diffraction. (a) Model of spherical-crown holographic Maxwellian display. (b) Coordinate schematic diagram of the model. (c) Filtering schematic diagram of the model.
According to Rayleigh-Sommerfeld (RS) diffraction formula, the complex amplitude of the diffraction field in the hologram sphere can be expressed as:
Similarly, the complex amplitude distribution on the pupil sphere Ue can be expressed as:
where d is the distance between the source point Us(R,$\varphi $s,$\theta $s) and the destination point Ue (re,$\varphi $e,$\theta $e), and cos$\beta {\; }$ is the obliquity factor.Figure 1(c) illustrates the filtering process on the pupil sphere, where the fields outside a certain aperture on the pupil sphere are set to zero. Here, the size of the filtering aperture refers to its diameter.
After the filtering process, Ue is diffracted back to the hologram sphere, resulting in the desired hologram, which can be expressed as:
To study Eq. (2), we know from Ref. [20] that the obliquity factor corresponding to the spherical crown diffraction model of outside in propagation (OIP) is:
where ${\; }\Delta \varphi $ and $\Delta \theta $ are the sampling intervals on the object sphere, which are both very small. re is very small relative to R, so that d${\approx} $R - re . Thus, cos$\beta $ can be approximated as 1. The formula can be simplified as: according to the geometric relationship of Fig. 1(a), the distance can be expressed by:As k is usually a large value, even slight variations in d can cause phase changes greater than 2$\mathrm{\pi }$. Therefore, we substitute Eq. (7) into the exponent of Eq. (5). Since re is very small compared to R and the denominator is not significantly affected by the variation of d, we can approximate d${\approx} $R - re. By performing the necessary operations on Eq. (5), we can obtain the complex amplitude distribution on the pupil sphere:
2.2 DOF of proposed method
Following the DOF analysis method presented in Ref. [12], we consider a one-point image on the object sphere, which means that we have determined the coordinates of a point on the object sphere as Us (R,$\varphi ,\theta $). The point can be represented by a delta function 1/R2sin$\theta $ $\delta $($\varphi - \varphi $0, $\theta - \theta $0)$.{\; }$ Substituting this expression into Eq. (9), we can express the wavefront of the point on the pupil sphere as:
Equation (12) shows that the wavefront of the point image can be decomposed into a constant phase and an inclined plane wave. The direction cosine of the plane wave is [-x0/(R-re), -y0/(R-re), -z0/(R-re)], which depends on the location of the point. Therefore, each point generates a specific plane wave that converges toward the pupil sphere and superimposes to form a point. If we ignore diffraction due to the limited aperture, the light ray will maintain its width. The width of the light beam is determined by the spatial frequency of the target point and can be written as 2λ$\nu $(R-re), where $\nu $ is the spatial frequency.
As shown in Fig. 2(a), assuming that the diameter D of the spot on the pupil sphere is smaller than the size of the pupil, and ignoring the influence of the spherical curvature due to the small sampling interval $\Delta \theta $ on the object sphere, the maximum size p of an image spot of a pixel on the object sphere at the retina can be expressed as:
where l is the focus depth of the eye. The smaller the value of p, the smaller the maximum size of the image point on the retina, and the clearer the imaging. Equation (13) indicates that as the focus depth l of the eye increases, the spot size D becomes smaller and the image quality improves.Fig. 2. Calculation and analysis of DOF. (a) Schematic diagram of DOF calculation. (b) Curve chart of retinal pixel size corresponding to different filtering apertures.
In order to more intuitively demonstrate the DOF range and the effect of filtering, we calculate and plot the function curve of the size p of image points as a function of reconstruction depth for different aperture sizes, as shown in Fig. 2(b), with the following parameters set: $\Delta \theta $=2${\times} {10^{ - 4}}$ rad, R = 450mm, rh = 300mm, λ=635nm, and feye = 18mm. The blue solid line represents the variation of p at different focus depths without filtering. The results show that as the focus depth increases, the maximum size of the image point on the retina decreases, leading to improved reconstructed image quality, indicating that the DOF of this model is within a relatively far depth range. By filtering on the pupil sphere, the spot size D can be reduced. The green and red lines in Fig. 2(b) respectively show two spot sizes of 2 mm and 1 mm. It can be seen that as the spot size decreases, the DOF is slightly expanded, but the image quality will be reduced due to the loss of high-frequency information.
2.3 Large FOV of proposed method
As shown in Fig. 3(a), according to the principle of planar holographic Maxwellian display [10–13], the complex amplitude distribution of the object plane generally is the product of the image intensity and the convergent spherical wave. The convergent center of the convergent wave is at the center of the pupil. And the object plane, hologram plane, and pupil plane all involve Fresnel diffraction. If we want to increase the FOV, we can only make the SLM expand longitudinally by splicing it. Figure 3(b) shows the principle of spherical-crown holographic Maxwellian display, which has irreplaceable advantages in the improvement of FOV, as it can reduce the display cost and obtain better performance.
Fig. 3. Comparison of planar and spherical-crown holographic Maxwellian displays. (a) Planar holographic Maxwellian display. (b) Spherical-crown holographic Maxwellian display.
Figure 4(a) intuitively illustrates the difference between the hologram plane and the hologram sphere. When the FOV is $\omega $ and the hologram plane is located at a distance of R from the center O (pupil center/equivalent pupil sphere center), the side length of the planar SLM required is L1 = 2R$\cdot $tan($\omega $/2), corresponding to the curve f(x)=tan(x) in Fig. 4 (b), and the side length of the spherical SLM required is L2 = R$\omega $=2R$\cdot $($\omega $/2), corresponding to the curve f(x)=x in Fig. 4(b). Therefore, in one-dimensional space, the length difference between the planar SLM and the spherical SLM is $\Delta L$=L1-L2 = 2R$\cdot $[tan($\omega $/2)-($\omega $/2)]. From Fig. 4(b), it can be seen that the larger the FOV, the more significant the difference between the two. With a large FOV, the reduction of the SLM by the spherical model is significant, or in other words, for the same SLM consumption, the spherical model has a significant improvement in FOV. When discussing two-dimensional space, the difference in area between the planar SLM and the spherical SLM will be more significant.
Fig. 4. Comparison between the hologram plane and the hologram sphere. (a) Comparison of SLM length required for the same FOV and comparison of FOV formed by SLM of the same length. (b) Function chart of length comparison for the same FOV.
3. Numerical simulations
3.1 Verification of model correctness
To verify the correctness of the proposed diffraction model, we use two relatively simple images and simulated their holograms and reconstructed images. The complex amplitude of the diffraction field on the hologram is generated by using the diffraction formula of the spherical crown diffraction model, and the reconstructed image is produced by using the point source method. The radius of the object sphere is 450mm, and the hologram sphere and pupil sphere radii are 300 mm and 5 mm, respectively. The reconstruction depth is 1.25 m, and the wavelength λ is 12 ${\mathrm{\mu} \mathrm{m}}$. The horizontal and the vertical FOVs are selected as -$\mathrm{\pi }/3\sim \mathrm{\pi }/3$ and -$\mathrm{\pi }/6\sim \mathrm{\pi }/6$, respectively. A target image resolution of 2048${\times} $1024 chosen to satisfy the Nyquist sampling criterion [20].
There are two simple images on the surface of the sphere (Fig. 5(a1)), and its reconstruction result (Fig. 5(c1)) matches the reality. The hologram of the letter image (Fig. 5(a2)) is shown in Fig. 5(b2), and Fig. 5(c2) indicates that it can also be correctly reconstructed.
Fig. 5. Verification of model correctness. (a1)(a2) Original image. (b1)(b2) Diffraction pattern on the hologram sphere with 5mm filtering. (c1)(c2) Reconstructed image.
As shown in Fig. 6, we demonstrate the large FOV of the spherical-crown holographic Maxwellian display. The information on the surface of the sphere can be seen at far focus depths, allowing corresponding objects to cover a large FOV, thus enhancing the immersive experience. By combining with pupil tracking technology, corresponding objects can be seen when the viewer looks up, down, left, or right, achieving a borderless display.
3.2 Verification of the DOF
In section 2.2, we analyze the DOF of the spherical-crown Maxwellian holographic display and find that the deeper the focusing depth, the clearer the imaging and the smaller the image points on the pupil, resulting in an expanded DOF. To demonstrate this effect more intuitively, we perform numerical simulations. The parameters are set as follows: Δ$\theta $=Δ$\varphi $=5.3${\times} {10^{ - 5}}$rad, considering the horizontal FOV, the number of samples in the horizontal direction of the image is set to 20480, the radii of the object sphere and hologram sphere are 450mm and 300mm. The equivalent pupil radius is 5mm, and λ=635nm. To ensure the accuracy of the results, the number of samples selected here not only satisfies the sampling theorem of the spherical diffraction [20] but is also 25% higher than the minimum value calculated by the sampling theorem. The peak signal-to-noise ratio (PSNR) is used to evaluate image quality [12].
The original image can be horizontally stitched together from 20 1024${\times} $1024 sub-regions. We randomly select 5 different sub-regions of the original image as targets and calculate the PSNR of the corresponding reconstructed image for each of these regions. When the radius of the object sphere is 450mm, Fig. 7(a) shows the numerical reconstruction results for different reconstruction depths and different filter aperture sizes for one of the 1024 × 1024 regions. The average PSNR values for multiple regions at the same reconstruction depth and filter aperture size are calculated and displayed in the line chart in Fig. 7(b). As the reconstruction depth increases, the image quality improves. Meanwhile, as the size of the pupil spot decreases, the DOF increases, but the image quality decreases due to the loss of high-frequency information, which is consistent with the theoretical derivation in section 2.2. To further verify the conclusion about DOF, we also conduct numerical simulations with object sphere radii of 400mm and 500mm, respectively, and the results are shown in Fig. 7(c) and Fig. 7(d). Obviously, when the object sphere is close to the hologram sphere, the image quality will improve at closer reconstruction depths and reach a relatively stable PSNR value earlier. Conversely, when the object sphere is far from the hologram sphere, the image quality will decrease at closer reconstruction depths and reach a relatively stable PSNR value later.
Fig. 7. Verification of the DOF. (a)Image comparison of reconstruction quality at different reconstruction depths when the radius of object sphere is 450mm. Line charts of average PSNR of images when the radius of object sphere is 450mm(b), 400mm(c), 500mm(d).
4. Optical experiments
Due to the current unavailability of a spherical SLM, we can only perform approximate experiments using a planar spatial light modulator for the near-axis region with a very small FOV. Based on the theory of diffraction calculation, we have preliminarily verified the correctness of the conclusion on DOF. The experimental optical setup is shown in Fig. 8(a). A series of toys are used as reference objects, and a camera is positioned close to the aperture. The input image size on the object sphere is 7.2mm${\times} $7.2mm, the object sphere radius is 450mm, the hologram sphere radius is 300mm, the pupil sphere radius is 5mm, λ=635nm, and the FOV is very small which is 7.2/450${\approx} $0.9°, located in the near-axis region. We load the spherical computer-generated hologram (SCGH) of the near-axis image onto the planar SLM and add a spherical wave phase to the hologram surface during programming [21].
Fig. 8. Verification of DOF with an optical experiment. (a) Optical path diagram. At a reconstruction depth of (b)0.45m, (c) 0.75m, (d) 1.05m, (e) 1.25m, (f) 2.05m.
As shown in Fig. 8, by filtering the obtained light information and adjusting the camera focus, we capture images at different reconstruction depths for the objects “fox”, “dice”, “tree”, “monster”, and “toy car” located at 0.45 m, 0.75 m, 1.05 m, 1.25 m, and 2.05m, respectively. A SLM with a pixel pitch of 8 µm and a resolution of 1920${\times} $1080 is used to load the hologram. The filtering aperture size in the hologram calculation is set as 5 mm. Observing Fig. 8(b)-(f), it can be seen that when the reconstruction depth is 0.45 m, the letters are very blurry. The farther the reconstruction depth, the clearer the letters and the better the image quality. This result is consistent with the theoretical analysis and numerical simulation results, which preliminarily proves the conclusion about DOF.
5. Conclusion
In this paper, we propose a large FOV holographic Maxwellian display based on spherical crown diffraction. The proposed method of spherical-crown holographic Maxwellian display theoretically covers the entire visual field required by the human eye, without complex optical paths and lens aberrations, and can maintain high image quality over a large range of DOF by controlling the size of the image points. Furtherly, the applicability of the spherical crown diffraction model in lensless holographic Maxwellian displays is preliminarily verified through numerical simulation and optical experiments. Although there are currently no optical devices for spherical holography to provide accurate optical experimental evidence, the proposed spherical-crown holographic Maxwellian display greatly enriches the field of holographic Maxwellian display. With the development of metasurface holography technology [22] and flexible materials [23], this technology might be gradually moved from theory to practice and achieve holographic Maxwellian three-dimensional display with a larger FOV in the future.
Funding
National Natural Science Foundation of China (62275178); Chengdu Science and Technology Program (2022-GH02-00016-HZ).
Acknowledgments
The authors wish to thank the anonymous reviewers for their valuable suggestions.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” Journal of Vision 8(3), 33 (2008). [CrossRef]
2. T. Ando, K. Yamasaki, M. Okamoto, T. Matsumoto, and E. Shimizu, “Evaluation of HOE for Head Mounted Display,” in Conference on Practical Holography XIII (San Jose, Ca, 1999), pp. 110–118.
3. T. Ando, K. Yamasaki, M. Okamoto, T. Matsumoto, and E. Shimizu, “Retinal projection display using holographic optical element,” in Conference on Practical Holography XIV and Holographic Materials VI (San Jose, Ca, 2000), pp. 211–216.
4. H. Takahashi, Y. Ito, S. Nakata, and K. Yamada, “Retinal projection type super multi-view head-mounted display,” in Conference on Engineering Reality of Virtual Reality (San Francisco, CA, 2014).
5. X. L. Shi, J. Liu, Z. Q. Zhang, Z. J. Zhao, and S. J. Zhang, “Extending eyebox with tunable viewpoints for see-through near-eye display,” Opt. Express 29(8), 11613–11626 (2021). [CrossRef]
6. M. H. Choi, K. S. Shin, J. Jang, W. Han, and J. H. Park, “Waveguide-type Maxwellian near-eye display using a pin-mirror holographic optical element array,” Opt. Lett. 47(2), 405–408 (2022). [CrossRef]
7. J. S. Lee, Y. K. Kim, M. Y. Lee, and Y. H. Won, “Enhanced see-through near-eye display using time-division multiplexing of a Maxwellian-view and holographic display,” Opt. Express 27(2), 689–701 (2019). [CrossRef]
8. X. Xinxing, G. Yunqing, Y. Furong, M. Yu, Z. Huadong, and Y. Yingjie, “Eyeglasses-Style Maxwellian-View Near-eye Display with Lens-Array-Based Holographic Optical Element,” SID Symp. Dig. Tech. Pap. (USA) 52(1), 394–397 (2021). [CrossRef]
9. W. Han, J. M. Jeon, M. H. Choi, and J. H. Park, “Lightguide type Maxwellian near-eye display with enlarged horizontal field of view by optical reconfiguration of input image,” J. Inf. Disp. 23(3), 201–211 (2022). [CrossRef]
10. Y. Takaki and N. Fujimoto, “Flexible retinal image formation by holographic Maxwellian-view display,” Opt. Express 26(18), 22985–22999 (2018). [CrossRef]
11. C. L. Chang, W. Cui, J. Park, and L. Gao, “Computational holographic Maxwellian near-eye display with an expanded eyebox,” Sci. Rep. 9(1), 18749 (2019). [CrossRef]
12. Z. Wang, X. Zhang, G. Q. Lv, Q. B. Feng, H. Ming, and A. T. Wang, “Hybrid holographic Maxwellian near-eye display based on spherical wave and plane wave reconstruction for augmented reality display,” Opt. Express 29(4), 4927–4935 (2021). [CrossRef]
13. Z. Wang, X. Zhang, K. F. Tu, G. Q. Lv, Q. B. Feng, A. T. Wang, and H. Ming, “Lensless full-color holographic Maxwellian near-eye display with a horizontal eyebox expansion,” Opt. Lett. 46(17), 4112–4115 (2021). [CrossRef]
14. Y. Z. Liu, X. N. Pang, S. J. Jiang, and J. W. Dong, “Viewing-angle enlargement in holographic augmented reality using time division and spatial tiling,” Opt. Express 21(10), 12068–12076 (2013). [CrossRef]
15. J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express 16(16), 12372–12386 (2008). [CrossRef]
16. F. Yaras, H. Kang, and L. Onural, “Circular holographic video display system,” Opt. Express 19(10), 9147–9156 (2011). [CrossRef]
17. M. L. Tachiki, Y. Sando, M. Itoh, and T. Yatagai, “Fast calculation method for spherical computer-generated holograms,” Appl. Opt. 45(15), 3527–3533 (2006). [CrossRef]
18. B. J. Jackin and T. Yatagai, “Fast calculation of spherical computer generated hologram using spherical wave spectrum method,” Opt. Express 21(1), 935–948 (2013). [CrossRef]
19. Y. Sando, D. Barada, B. J. Jackin, and T. Yatagai, “Spherical-harmonic- transform-based fast calculation algorithm for spherical computer-generated hologram considering occlusion culling,” Appl. Opt. 57(23), 6781–6787 (2018). [CrossRef]
20. C. M. Liu, J. Wang, Y. Wu, X. L. Lei, P. D. Wang, H. Han, and C. Chen, “Spherical crown diffraction model by occlusion utilizing for a curved holographic display,” Opt. Express 30(18), 31685–31700 (2022). [CrossRef]
21. H. Han, J. Wang, and Y. Wu, “Spherical crown holographic display with planar SLM,” Opt. Commun. 537, 129452 (2023). [CrossRef]
22. A. C. Overvig, S. Shrestha, S. C. Malek, M. Lu, A. Stein, C. X. Zheng, and N. F. Yu, “Dielectric metasurfaces for complete and independent control of the optical amplitude and phase,” Light-Science & Applications 8(1), 92 (2019). [CrossRef]
23. S. Dal Kim, B. Lee, T. Byun, I. S. Chung, J. Park, I. Shin, N. Y. Ahn, M. Seo, Y. Lee, Y. Kim, W. Y. Kim, H. Kwon, H. Moon, S. Yoo, and S. Y. Kim, “Poly(amide-imide) materials for transparent and flexible displays,” Sci. Adv. 4(10), eaau1956 (2018). [CrossRef]
