Open Access
Add to BibSonomy
Abstract
Pancake optics (also known as polarization-based catadioptric system) has been widely used as the imaging lens for virtual reality (VR) and mixed reality (MR) headsets because of its compact formfactor and excellent image quality. However, such a folded pancake optics not only dramatically lowers the optical efficiency to 25% because of the utilization of a half mirror, but also suffers from ghost images due to the stray light from multiple surface reflections and imperfect polarization control inside the optical system. In this paper, the origins including the light paths of the ghost images are explored by experiment and then analyzed by simulation. The effect of different incident angles on the intensity of each ghost is also investigated.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Video see-through virtual reality (VR) headsets (also known as mixed reality (MR)), such as Apple Vison Pro, Meta Quest 3, and Pico 4, providing more direct interactions with digital information, have the potential to revolutionize the ways we perceive and interact with the digital world [1,2]. Offering fully immersive experiences, VR is one of the key enablers for metaverse, spatial computing, and digital twins that have found useful applications in education [3], healthcare [4], manufacturing [5], and construction [6], just to name a few.
The optical layout of a VR system is relatively simple: a viewing optics to magnify the digital images from the microdisplay panel. To ensure compact formfactor and excellent image quality, the pancake optics system (also known as polarization-based catadioptric optical system) has been widely used in VR displays [7–9]. Figure 1(a) shows the pancake VR system. It consists of a microdisplay light engine, which is usually a high-resolution density liquid crystal display (LCD) or a micro-OLED (organic light-emitting diode) display [2,10–12], a circular polarizer (CP) to convert the incident linearly polarized light to circular polarization, and a pancake lens group to generate magnified virtual images. Figure 1(b) depicts the optical paths inside the pancake lens. Without losing generality, let us assume the emitted light from the microdisplay panel is right-handed circularly polarized (RCP) light. It passes through a half mirror (HM) with 50% transmittance and 50% reflectance. The remaining 50% RCP light (denoted by the solid blue arrow) is then incident on a quarter-wave plate (QWP) and a reflective polarizer (RP), which can be carefully aligned to become a polarization-sensitive reflector for the RCP light while transmitting the left-handed circularly polarized (LCP) light. The reflected 50% RCP light hits the HM for the second time and only 25% of the remaining light is converted to LCP (denoted by red arrow) and reflected toward the QWP and RP. The surface of the HM is usually curved, which provides a much larger optical power during reflection, as compared to its transmission counterpart, enabling a small f-number. The QWP converts the LCP to the pass-state light of the RP so that the remaining 25% light can finally pass through the pancake system. If the microdisplay emits an unpolarized light, e.g., micro-LED, then the maximum pancake optical efficiency would drop to 12.5%. Such a low optical efficiency will undoubtedly shorten the battery operating time and raise the temperature of the VR headset. A photo in Fig. 1(c) shows the low transmittance of a pancake lens. Several approaches have been proposed to improve the optical efficiency of the pancake lens [13–15].
Fig. 1. (a) System configuration of a pancake VR system. (b) Light paths inside the pancake optics. (c) A photo showing the low transmittance of a pancake lens. (d) Captured signal and ghost images of a pancake lens. (e) Effect of a real ghost image on visual experience.
Besides above-mentioned low optical efficiency, ghost images (caused by stray lights) also degrade the contrast ratio and resolution of a pancake VR headset [16–18]. Inside the folding pancake optics system, the light polarization states have been modified several times by the QWP during multiple reflections inside the pancake system. Therefore, the polarization state could be disturbed when the light is incident on the RP, especially at a large angle [19]. As a result, some unwanted light leakage shown by the dashed gray arrows in Fig. 1(b) occurs and finally becomes the ghost images. Moreover, the optical efficiency and extinction ratio of the reflective polarizer also affect the stray light formation. In this paper, we will analyze the origins of ghost images and then propose some approaches to mitigate the ghost images. The angular dependence of the ghosts will be also discussed.
2. Experiments and simulations
To analyze the ghost images inside the pancake VR system, we captured the images of a letter ‘E’, generated by an OLED light engine and the results are presented in Fig. 1(d). The camera was intentionally placed far away from the designed location for eye pupil to record all the ghost images, which is the reason that the signal image exhibits severe aberrations. In total, seven ghost images were observed in the experiment, and they can be divided into two groups depending on the orientations. The upright ghost images shown in Fig. 1(d) are virtual images that can be observed when our eyes are located near the designed pupil location. The inverted ghost images in the diagonal locations are real images and cannot form clear images when observing from the eye pupil location. In addition to image orientation, these two types of ghost images also affect the visual experience in different ways as shown in Fig. 1(e). The camera was located near the designed eye position and the captured image is more like what the viewer could see. The virtual ghost images may become blurry depending on the focused position of the human eye. On the other hand, the real ghost images may occupy a large area as shown in the dashed circle in Fig. 1(e). This is because the real ghost images generate real displayed image at the pupil side of the pancake lens, while the virtual ghost images result in virtual image of the display at the display side of the pancake lens. The focused real ghost images are then projected to a large area on retina after passing through the eye lens.
We first investigated the origins of the virtual ghost images as they are more noticeable on the retina. Figure 2(a) shows the captured signal and 3 virtual ghost images. A smartphone with OLED panel was employed as the light engine and the emitted light was converted to circular polarization using a CP. The origin of the virtual ghost 1 is the light leakage at the first incidence onto the RP, which can be confirmed by observing the subsequent movement of ghost 1 when moving the display. The optical path of ghost 1 is illustrated in Fig. 2(f). Besides, we found that ghosts 2 and 3 originated from the reflection of the CP because their positions changed accordingly when the CP was tilted by a small angle. The optical efficiency of the pancake system is lower than 25% so that most of the wasted light is reflected to the CP and the display side, which causes ghosts 2 and 3. To validate this analysis, we captured the virtual images using CPs without and with anti-reflection coating and the results are shown in Fig. 2(b) and 2(c), respectively The reflectance of these two CPs was measured to be 5.9% and 1.2%. Two photos were captured using the same exposure time and ghost 2 was almost invisible after using the CP with anti-reflection coating (Fig. 2(c)), which confirmed our analysis for the ghost image formations. The color difference in Fig. 2(b) and 2(c) originated from the wavelength dependent polarization conversion efficiency of the employed circular polarizers.
Fig. 2. (a) Captured images focused on signal. Captured images using CPs without (b) and with (c) an AR coating. (d) Captured images focused on ghost 1. (e) Relative locations of the virtual ghost images and signal on the optical axis. Light paths of ghost 1 (f), 2 (g), and 3 (h).
To further study the light paths of ghosts 2 and 3, we captured another image with the camera focusing on ghost 1 as Fig. 2(d) shows. Because ghost 2 shows a clearer image compared to ghost 3, it indicates the image plane of ghost 2 is located closer to that of ghost 1 due to the limited depth of field (DOF) of the employed camera. Alternatively, ghost 3 shows a clearer image in Fig. 2(a) where the camera was focused on the signal image, indicating that the image plane of ghost 3 is nearer to that of the signal image. According to our simulation, the image plane of ghost 1 is the closest to the pancake lens position while the signal image plane is expected to locate at infinity. The relative locations of the virtual images and signal are illustrated in Fig. 2(e). After comparing the locations with our simulation results, we draw the light paths of ghosts 2 and 3 in Fig. 2(g) and 2(h). We also noticed that ghost 3 exhibits a stronger illuminance than ghost 2 in the captured images. The underlying reason is that the two reflections of ghost 2 occurs when the light remains circular polarization and therefore, the polarization state of the stray light is first switched to the opposite circular polarization and then switched back to the original circular polarization. Then, the stray light is converted to the block state of the RP by the QWP. In contrast, only one of the two reflections happens when the stray light is circularly polarized, which turns the polarization state of ghost 3 to the passing state of the RP at its second incidence onto the RP. Consequently, ghost 3 shows a stronger illuminance as the RP provides a much higher transmission for the pass-state light.
After understanding the origins of the virtual ghost images, we built a pancake system based on the structure in Fig. 1(b) to investigate the real ghost images and the results are shown in Fig. 3(a). The origins of the real ghost images are the multiple reflections inside the pancake lens. Ideally, all the light that exhibits the pass-state polarization can pass through the RP. However, there is still some reflections from the RP due to the imperfect polarization states and the non-ideal RP performance. Then, the reflected light will propagate inside the pancake cavity and become stray light. Here, a blue laser diode (488 nm) was employed as the light source for quantitative analysis. The optical components in the experiment were tilted intentionally to only present the interested images as there can be many more noticeable ghost images due to the high laser power, while those ghost images were not observable in VR systems using a flat panel as the light engine. Limited by the size of the optical elements, only 5 spots corresponding to the emitted light at 5° incident angle are presented, because other spots would escape from the cavity with a larger incident angle. These spots are designated as the 0th, 1st, 2nd, 3rd, and 4th from right to left and the order is determined by the reflection from the HM. For example, the 0th spot corresponds to the light travelling through the whole system without any reflection at the optical surfaces as shown in Fig. 3(b), which is exactly the virtual ghost 1 discussed earlier. Similarly, the 1st spot refers to the signal light with only one reflection on the HM as illustrated in Fig. 3(c). The intensity of the 4th spot is too weak to be detected. Thus, we mainly consider the remaining 4 spots. Only 2nd and 3rd spots shown in Fig. 3(d) and 3(e) correspond to the real ghosts mentioned before. To investigate the effect of RP on the stray light, two RPs with different extinction ratios were applied to analyze their influence on the signal and ghost spots.
Fig. 3. (a) Captured signal and ghost spots. Light paths of the 0th (b), 1st (also the signal) (c), 2nd (d), and 3rd (e) spots. (f) Adding a LP to reduce stray light. Blue arrows: RCP; red arrows: LCP; green arrows: block state of RP and LP; black arrows: pass-state of RP and LP.
The first multilayer RP with alternating birefringent/isotropic films, also known as dual brightness enhancement film (DBEF), exhibits an extinction ratio of 32.3 while the second one is a wire-grid polarizer (WGP) with a higher extinction ratio of 513.3 at 488 nm. Here, the purpose is to compare the performance of RPs with different extinction ratios and to understand how they affect the stray light. The measured data are summarized in Table 1. It should be mentioned that an improved DBEF reflective polarizer with a much higher extinction ratio for VR applications has been developed [20,21].
Table 1. Measured spot power with two different reflective polarizers.
According to Table 1, the 0th spot, also the virtual ghost 1 in earlier discussion, is the strongest stray light. It’s worth noting that although WGP exhibits a higher extinction ratio, its pass-state transmittance (78.7%) is lower than that (89.2%) of DBEF RP in our experiment. Therefore, the signal of the DBEF system is brighter than that of the WGP system. However, WGP with a higher extinction ratio leads to a better signal-to-noise ratio (SNR) of 29.0 when comparing the 1st signal spot and the 0th ghost spot, while the DBEF system only provides a SNR of 15.8. Moreover, the power of the 2nd and 3rd ghost spots in the WGP system is too weak to be detected. To further reduce the power of ghost spots, a linear polarizer (LP) was deployed after the DBEF RP system, and its orientation was carefully aligned with the RP orientation to absorb the light with unwanted polarization states. Results are shown in Table 2.
Table 2. Suppressing the ghost images using a LP.
To record the spot power more accurately, we increased the laser output power. Therefore, the SNR in Table 2 is slightly different from that in Table 1. The LP transmittance for the pass-state light is about 86.1%. The power of 0th spot decreases dramatically after adding the LP because the polarization of the 0th light is orthogonal to the axis of LP, as shown in Fig. 3(f). Green arrows represent the light with block-state polarization of RP and LP, which are expected to be reflected by the RP. However, some block-state light can still pass through the RP due to the non-ideal RP performance. An additional LP helps eliminate the 0th and 2nd stray lights according to our analysis. Unfortunately, we didn’t see much reduction in the 2nd stray light. To figure out the reason, a polarimeter is used to visualize the polarization states of the stray light and the measured polarization ellipses are shown in Fig. 4.
Fig. 4. Polarization ellipses. 0th spot before (a) and after (b) RP. 2nd spot before (c) and after (d) RP.
Linear polarization along the horizontal direction represents the block state of the RP while the vertical direction is the pass state. According to the measured results, the 0th spot remains good block-state polarization before the RP, and the horizontal component still dominates the output light after RP. In contrast, the light of 2nd spot exhibits nearly equal horizontal and vertical components before RP. Then, most of the horizontal component is reflected by the RP due to its different optical responses for a linearly polarized light. Therefore, the 2nd spot after RP has more vertical component and cannot be eliminated by the added LP. The origins of 2nd and 3rd stray light spots should be attributed to the reflection of 1st spot on the RP. Ideally, all the light with the pass-state polarization should pass through the RP without being reflected. However, the RPs in our experiment are films that need to be laminated to glass substrates, which reflects the pass-state light and causes further stray light. Therefore, the 2nd and 3rd stray light spots should be much weaker compared to the measured results.
In addition to experimentally investigating the stray light, we also conducted comprehensive simulations to study the angular dependence of the ghost images as shown in Fig. 5(a). Rays with different colors indicate different light paths that finally reach the retina. Figure 5(b) illustrates the simulated illuminance on retina. The reflection at CP surface is 0.5% in our simulation. The spectral and angular responses of RP and QWP for simulation are presented in Fig. 5(c)-(g). The RP shows an excellent extinction ratio over the visible spectral range at normal incidence. However, increasing the incident angle gradually degrades the optical performance for the pass-state light while the block-state light transmittance remains small even at a large incident angle. Similarly, the phase retardation also changes quickly as the incident angle increases, which deteriorates the polarization state and decreases the contrast ratio. We investigated the SNR of the pancake model, and the results are plotted in Fig. 5(h). The system exhibits the lowest SNR when the point source is located at Y = 0 mm (central location) as expected since most of the stray light can finally reach the pupil and retina. Some ghosts will disappear and cannot be received when the source is located at different positions. The SNR (defined as the flux ratio of signal over ghosts) gradually increases when Y becomes larger and then decreases when the source is located near the edge of the display. We also found that blue light shows a slightly inferior performance to the green and red colors, which can be attributed to the spectral response of RP as shown in Fig. 5(c).
Fig. 5. (a) Simulation model of the pancake VR system. (b) Illuminance on retina when Y = 20 mm (corresponds to an incident angle of about 50 degrees at pupil). (c) Spectral response of the RP. Angular response of RP for (d) pass-state light and (e) block-state light. (f) Phase retardation of the employed QWP in the visible spectral range. (g) Phase retardation of the QWP at different incident angles. (h) System SNRs at different point source positions.
Apart from considering the effect of the overall ghost, we also conducted simulation to understand how different source positions (Y) (or incident angles) affects the flux of each ghost. Results are plotted in Fig. 6(a-f). Figure 6(a) illustrates the total flux of the signal image on retina. Less light can enter the pupil as the source gets further away from the center of the display, especially at the edges. As for the stray light, virtual ghost 1 dominates the pancake system at a larger Y as Fig. 6(b) depicts, while the 2nd ghost degrades the contrast more severely when the source is located at the center. Fortunately, the 2nd ghost gradually disappears as Y increases (Fig. 6(e)) because the stray light cannot be received by the pupil. Other stray light also shows dependence on the source positions. The virtual ghosts 2 and 3 in Fig. 6(c) and 6(d) resulted from the reflections of the CP generate a relatively large flux as the incident angle keeps increasing. The total flux of virtual ghost 3 is higher than that of virtual ghost 2, which agrees well with our captured images in Fig. 2(b). The flux of the 3rd ghost related to the multiple reflections inside the pancake cavity gradually decreases when the point source moves away from the center to the edge as Fig. 6(f) shows. Their impacts to the overall performance are much weaker than those two ghosts discussed earlier.
Fig. 6. Effect of different source position on the (a) flux of signal (also the 1st spot) on retina, and (b) normalized flux of virtual ghost 1 (also the 0th spot), (c) virtual ghost 2, (d) virtual ghost 3, (e) 2nd ghost spot, and (f) 3rd ghost spot. The ghost flux is normalized by the corresponding signal flux.
3. Conclusion
In summary, we have investigated the ghost images inside the pancake system. The origins and corresponding light paths of ghost images are well understood. The observed ghost images are divided into virtual and real ones depending on their image’s locations. The intensity of each ghost from different source positions (incident angles) is also investigated by simulations. The dominating stray light is the virtual ghost 1 (or the 0th stray light spot) originating from the block-state leakage of the RP, which can be reduced by using a RP with a higher extinction ratio and adding an extra LP after the pancake lens. Laminating a QWP layer after the LP also helps eliminate the reflection from the user’s eye pupil. Other virtual ghosts are related to the reflection of wasted light from the CP. Therefore, anti-reflection coating on the surfaces is essential to eliminate this part of stray light. Moreover, the 2nd ghost also degrades the contrast ratio at central area. This real ghost image originates from the multiple reflections inside the pancake lens, which also requires a higher extinction ratio reflective polarizer for better viewing quality.
Disclosures
The authors declare no conflicts of interest.
Data Availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. B. C. Kress, Optical architectures for augmented-, virtual-, and mixed-reality headsets (SPIE Press, 2020).
2. J. Xiong, E. L. Hsiang, Z. He, et al., “Augmented reality and virtual reality displays: emerging technologies and future perspectives,” Light: Sci. Appl. 10(1), 216 (2021). [CrossRef]
3. W. P. Zhang and Z. Wang, “Theory and practice of VR/AR in K-12 science education—a systematic review,” Sustainability 13(22), 12646 (2021). [CrossRef]
4. A. A. Rendon, E. B. Lohman, D. Thorpe, et al., “The effect of virtual reality gaming on dynamic balance in older adults,” Age 41(4), 549–552 (2012). [CrossRef]
5. S. Choi, K. Jung, and S. D. Noh, “Virtual reality applications in manufacturing industries: past research, present findings, and future directions,” Concurr. Eng. 23(1), 40–63 (2015). [CrossRef]
6. X. Li, W. Yi, H.-L. Chi, et al., “A critical review of virtual and augmented reality (VR/AR) applications in construction safety,” Autom. Constr. 86, 150–162 (2018). [CrossRef]
7. J. A. LaRussa and A.T. Gill, “The holographic pancake window TM,” Proc. SPIE 162, 120–129 (1978). [CrossRef]
8. F. Peng, Y. Geng, J. Wang, et al., “Liquid Crystals for Virtual Reality (VR),” SID Symp. Dig. Tech. Pap. 52(1), 427–430 (2021). [CrossRef]
9. Y. Geng, “Viewing optics for immersive near-eye displays: pupil swim/size and weight/stray light,” Proc. SPIE 10676, 5 (2018). [CrossRef]
10. Y. H. Wu, C.-H. Tsai, Y.-H. Wu, et al., “5-2: Invited Paper: High dynamic range 2117-ppi LCD for VR displays,” SID Symp. Dig. Tech. Pap. 54(1), 36–39 (2023). [CrossRef]
11. Y. Motoyama, K. Sugiyama, H. Tanaka, et al., “High-efficiency OLED microdisplay with microlens array,” J. Soc. Inf. Disp. 27(6), 354–360 (2019). [CrossRef]
12. E. L. Hsiang, Z. Yang, Q. Yang, et al., “AR/VR light engines: perspectives and challenges,” Adv. Opt. Photonics 14(4), 783–861 (2022). [CrossRef]
13. N. Usukura, K. Minoura, and R. Maruyama, “Novel pancake-based HMD optics to improve light efficiency,” J. Soc. Inf. Disp. 31(5), 344–354 (2023). [CrossRef]
14. Z. Luo, Y. Ding, Y. Rao, et al., “High-efficiency folded optics for near-eye displays,” J. Soc. Inf. Disp. 31(5), 336–343 (2023). [CrossRef]
15. Y. Ding, Z. Luo, G. Borjigin, et al., “Breaking the optical efficiency limit of virtual reality with a nonreciprocal polarization rotator,” Opto-Electron. Adv. 7(3), 230178 (2024). [CrossRef]
16. Q. Hou, D. Cheng, Y. Li, et al., “Stray light analysis and suppression method of a pancake virtual reality head-mounted display,” Opt. Express 30(25), 44918–44932 (2022). [CrossRef]
17. E. L. Hsiang, Z. Yang, and S. T. Wu, “Optimizing microdisplay requirements for pancake VR applications,” J. Soc. Inf. Disp. 31(5), 264–273 (2023). [CrossRef]
18. B. Hao, G. Tan, S. T. Wu, et al., “Ghost analysis of a polarized catadioptric system for virtual reality,” Proc. SPIE 12449, 12 (2023). [CrossRef]
19. H. Chen, G. Tan, and S. T. Wu, “Ambient contrast ratio of LCDs and OLED displays,” Opt. Express 25(26), 33643–33656 (2017). [CrossRef]
20. T. L. Wong, Z. Yun, G. Ambur, et al., “Folded optics with birefringent reflective polarizers,” Proc. SPIE 10335, 103350E (2017). [CrossRef]
21. J. Le, B. Hao, D. Aastuen, et al., “High resolution reflective polarizer lens for catadioptric VR optics with accommodating eye box design,” Proc. SPIE 12449, 9 (2023). [CrossRef]
