Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Aerial floating 3D display device with lenticular-type naked-eye 3D display and a crossed-mirror array

Open Access Open Access

Abstract

The depth feeling of a naked-eye 3D display is determined by the combination of the display and the lenticular lens, and because the parallax of the displayed image cannot be made stronger, providing a more stereoscopic effect is difficult. In this study, we propose a system that provides a clearer stereoscopic effect by enabling the naked-eye 3D display to be observed as an aerial stereoscopic image without the framework of the display. By comparing it with the conventional aerial display method, we demonstrate that it is possible to maintain the same level of resolution as the method without a crossed-mirror array and that natural stereoscopic view is possible.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. INTRODUCTION

Display technologies have been developing rapidly in the past few years [1]. This technological development has contributed remarkably to the practical application of three-dimensional (3D) displays, which are next-generation displays that facilitate a realistic visual experience and immersion for users. The 3D displays enable stereopsis via active shutter glasses [2] or passive polarizing glasses [3] to observe images from different viewpoints with both eyes, thereby enabling a 3D experience. The initial problem of reduced luminance due to the polarizing plate has been overcome by increasing the luminance of 2D displays [4]. However, studies show that users generally find it burdensome to wear special glasses for stereoscopic viewing and that the active shutter method strains the eyes more than 2D displays do [5]. Consequently, naked-eye 3D displays have been commercialized in recent years [6,7].

There are two mainstream display methods for naked-eye 3D displays: the parallax barrier method [8] and the lenticular lens method [9]. In the parallax barrier method, a filter with a narrow slit, which is placed on the display, blocks one of the images so that different images can be observed depending on the viewing position. In the parallax barrier method, more than half the area of the display is blocked by the filter, which reduces the brightness of the display to less than half. To solve this problem, a lenticular-lens-based method was developed. A lenticular lens is a sheet of thin, semi-cylindrical convex lenses arranged in countless numbers. By passing through the convex lens, the image placed under the lens changes depending on the viewing angle. This method enables stereoscopic vision by giving different images to the left and right eyes as the visible image changes. Both the parallax barrier and the lenticular method require the images to be transferred to the left and right eyes at the same time, and the resolution of the left and right images displayed is half the resolution of the display. The recent advances in 2D display technology have enabled the display of images with a resolution of UHD 4 K or UHD 8 K [10], which is higher than the HD resolution for the left and right eyes, respectively.

In a naked-eye 3D display, when an observer views the display naturally, the left and right eyes ideally receive the corresponding images so that stereopsis can be achieved without any visual discomfort. In other words, if images with an angular mismatch between convergence and adjustment are provided to each eye, stereopsis cannot be achieved because of the extreme discomfort. This is because the angle of convergence required for observing the binocular disparity naturally does not match that of the images, thus causing a strong sense of discomfort [11]. Therefore, the depth of a naked-eye 3D display is determined by the combination of the display and lenticular lens, and it is difficult to give a more stereoscopic effect because the parallax of the displayed image cannot be made stronger.

An aerial floating display method is used to obtain a 3D effect by forming a display image in the air. A common aerial floating display method uses a mirror array [12]; the light path of the display is changed, and the image is formed in the air. However, in the method, the display is only shown in the air, and the aerial image is flat. To achieve a 3D effect, the aerial image needs to be displayed in three dimensions. To solve this problem, the integral display method is used [13]. The integral method, which is referred to as a ray reproduction method, enables 3D display by storing and reproducing rays of light from various viewpoints. However, the problem associated with the integral method is that the number of rays that can be reproduced is small when the display is used to reproduce rays; this results in a decrease in the resolution of the displayed aerial image and the number of parallaxes.

In this paper, we propose a system that yields a high stereoscopic effect to a naked-eye 3D display by eliminating the framework of the display and having it observed as an aerial stereoscopic image. The system enables high-definition parallax display on a naked-eye 3D display and enables the naked eye to observe 3D images displayed in the air.

The remainder of this paper is organized as follows. Section 2 describes the underlying concepts of naked-eye 3D displays and mirror arrays as well as conventional methods for aerial displays. Section 3 details the proposed system. Section 4 describes the experiments conducted and the results obtained. Section 5 presents a discussion on the experimental results. Section 6 summarizes the conclusions of this study.

 figure: Fig. 1.

Fig. 1. Example of how a parallax image looks owing to the rays of a naked-eye 3D display.

Download Full Size | PDF

2. CONVENTIONAL TECHNOLOGY

A. Naked-Eye 3D Display

In this section, we explain the naked-eye 3D display of the lenticular method. Figure 1 shows the 3D display structure of the lenticular method. In Fig. 1, the lenticular lens is stretched on the surface of the display. The display also shows a parallax image in the area under the lens. The light rays through the lens are focused at different positions depending on the observation position, and different images are provided to the left and right eyes, thus enabling stereoscopic vision. Figure 1 shows that the position where the light is focused from the fixed lens. This distance is called the optimal observation distance and is necessary to see the correct image.

 figure: Fig. 2.

Fig. 2. Structural drawing of the crossed-mirror array.

Download Full Size | PDF

B. Crossed-Mirror Array

A crossed-mirror array consists of multiple mirror arrays of thin glass with mirrored surfaces, stacked orthogonally, and it is used to project airborne images. Figure 2 shows an example of a crossed-mirror array. An example of the reflection of a ray of light at a corner mirror [14] is illustrated in Fig. 3, wherein the mirrors are initially arranged at right angles. In Fig. 3, the light reflected by the first mirror is reflected at the same angle as the incoming light by specular reflection. When the reflected light passes to the other mirror, the angle of incidence is ($90-\theta$)° because the other mirror is tilted by 90°. The light reflected by this mirror is also reflected at the same angle as the angle of incidence, and thus the final reflection is parallel to the incident light. A device that consists of two layers of one-dimensional mirror arrays arranged at right angles to each other [15] is a type of orthogonal mirror. Given that only the surface portions are mirrored, the light can pass through the glass and resin portions. Therefore, light rays can be incident on the top and bottom.

 figure: Fig. 3.

Fig. 3. Example of incident and reflected light rays when mirrors are placed at right angles.

Download Full Size | PDF

 figure: Fig. 4.

Fig. 4. Principle image of the proposed method.

Download Full Size | PDF

Assuming that the horizontal direction is the $x$ axis, the vertical direction is the $y$ axis, and the front-back axis is the $z$ axis, the conversion equation of the vector by specular reflection is shown in Eq. (1), wherein ${\rm Mirror\_IN}\_[X | Y | Z]$ is the incident light and ${\rm Mirror\_OUT}\_[X | Y | Z]$ is the reflected light. In specular reflection, the incident light and the reflected light are reflected at the same angle; thus, when the incident angle is $\theta$, the angle of the vector rotates by $2\theta$. Further, the incident light reverses its direction, thus making the vector a reverse vector.

$$\begin{split}\left[{\begin{array}{*{20}{c}}{{\rm Mirror\_OUT}\_X}\\{{\rm Mirror\_OUT}\_Y}\\{{\rm Mirror\_OUT}\_Z}\end{array}} \right] &= - \left[{\begin{array}{*{20}{c}}{\cos 2\theta}&\quad0&\quad{\sin 2\theta}\\0&\quad1&\quad0\\{- \sin 2\theta}&\quad0&\quad{\cos 2\theta}\end{array}} \right]\\&\quad\times\left[{\begin{array}{*{20}{c}}{{\rm Mirror\_IN}\_X}\\{{\rm Mirror\_IN}\_Y}\\{{\rm Mirror\_IN}\_Z}\end{array}} \right].\end{split}$$

The vector conversion equation for a crossed mirror array in the $x-z$ plane is shown in Eq. (2). In the crossed-mirror array, the vector transformation of specular reflection described above is performed twice, at incident angles $\theta $ and ($90-\theta$)°. Here, ${\rm IN}\_[X | Y | Z]$ is the incident light, and ${\rm OUT}\_[X | Y | Z]$ is the reflected light.

$$\begin{split}\left[{\begin{array}{*{20}{c}}{{\rm OUT}\_X}\\{{\rm OUT}\_Y}\\{{\rm OUT}\_Z}\end{array}} \right] &= - \left[{\begin{array}{*{20}{c}}{\cos 2(90 - \theta)}&\,\,\,0&\,\,\,{\sin 2(90 - \theta)}\\0&\,\,\,1&\,\,\,0\\{- \sin 2(90 - \theta)}&\,\,\,0&\,\,\,{\cos 2(90 - \theta)}\end{array}} \right]\\&\,\,\,\times \left({- \left[{\begin{array}{*{20}{c}}{\cos 2\theta}&\,\,\,0&\,\,\,{\sin 2\theta}\\0&\,\,\,1&\,\,\,0\\{- \sin 2\theta}&\,\,\,0&\,\,\,{\cos 2\theta}\end{array}} \right]\left[{\begin{array}{*{20}{c}}{{\rm IN}\_X}\\{{\rm IN}\_Y}\\{{\rm IN}\_Z}\end{array}} \right]} \right)\\ &= \left[{\begin{array}{*{20}{c}}{- {{({{\sin}^2}\theta + {{\cos}^2}\theta)}^2}}&\,\,\,0&\,\,\,0\\0&\,\,\,1&\,\,\,0\\0&\,\,\,0&\,\,\,{- {{({{\sin}^2}\theta + {{\cos}^2}\theta)}^2}}\end{array}} \right]\left[{\begin{array}{*{20}{c}}{{\rm IN}\_X}\\{{\rm IN}\_Y}\\{{\rm IN}\_Z}\end{array}} \right]\\& = \left[{\begin{array}{*{20}{c}}{- 1}&\,\,\,0&\,\,\,0\\0&\,\,\,1&\,\,\,0\\0&\,\,\,0&\,\,\,{- 1}\end{array}} \right]\left[{\begin{array}{*{20}{c}}{{\rm IN}\_X}\\{{\rm IN}\_Y}\\{{\rm IN}\_Z}\end{array}} \right].\end{split}$$

By solving Eq. (2), we observe that the $x$ and $z$ vectors are multiplied by $-1$. In this process, there is no change in the light ray with respect to the $y$ vector. Therefore, the light incident from below forms an image in the air with plane symmetry with respect to the crossed-mirror array and thus enables aerial display.

C. Conventional Method for Aerial Display

Among aerial display methods, the integral and combination methods enable stereoscopic viewing of aerial images. Because aerial display is based on the ray reproduction method, it is not necessary to photograph each parallax corresponding to the left and right eyes. By photographing an object (photographed object) through a two-dimensional lens array, the images from multiple viewpoints are stored. By passing the photographed image through the 2D lens array again, the light rays are reproduced and the object is displayed in 3D. In this method, if the number of rays stored by the 2D lens array is large, a high resolution can be maintained. In reality, however, the number of rays is limited by the resolution of the display. The resolution of the display also depends on the lens pitch of the 2D lens array. Using a finer lens array can help to reproduce a clearer three-dimensional image. However, it reduces the resolution of the image entering under the lens and thus involves a trade-off. In addition, crossed-mirror arrays and two-sided corner reflector arrays for aerial display also have a mirror pitch that causes a reduction in resolution. Therefore, the resolution of both the two-dimensional lens array and the two-sided corner reflector array is reduced; consequently, a high-resolution display cannot be achieved via the conventional method.

 figure: Fig. 5.

Fig. 5. Example of disparity image selection for the left and right eyes using a lookup table.

Download Full Size | PDF

 figure: Fig. 6.

Fig. 6. Examples of the proposed method: (left image) without the correction method; (right image) with the correction method.

Download Full Size | PDF

In recent years, owing to the impact of the COVID-19 pandemic, there has been a growing demand for aerial displays of information boards as non-contact displays [16]. However, one reason why it has not become popular is that the displayed images are of low resolution and the displayed text is difficult to read. The proposed method can solve the resolution problem as well as providing three-dimensional display of the information board (three-dimensional operation panel buttons) by using a naked-eye 3D display.

3. PROPOSED METHOD

In a naked-eye 3D display, the combined displayed disparity image must correctly provide different disparity images to each of the observer’s eyes to achieve naked-eye stereopsis. Therefore, the aerial image displayed by the crossed mirror array should be spectrally correct. In the proposed method, it is considered that a virtual display is at the optimum observation distance from the naked-eye 3D display, and an aerial floating naked-eye 3D display system is constructed by installing the virtual display and the crossed mirror array in a state that enables aerial display. Figure 4 shows a schematic diagram of the proposed method.

A naked-eye 3D display consisting of a display, and a lenticular lens displays disparity images, and each disparity image is spectrally divided at the optimum viewing distance. The image that can be observed at the optimal viewing distance is called the virtual display position (VDP) and can be displayed in the air by using the crossed mirror array. By observing each disparity displayed in the air, the observer can achieve naked-eye stereopsis in the air.

 figure: Fig. 7.

Fig. 7. Experimental device and its dimensions.

Download Full Size | PDF

Tables Icon

Table 1. Specifications of the Naked-Eye 3D Display and Crossed-Mirror Array Used in the Experiment

In a normal naked-eye 3D display, each disparity image is shown on the display. In other words, each disparity image is superimposed on the other, thus resulting in a convergence angle when observed with both eyes. However, with the proposed display method, the spectral disparity images observed at the VDP are shifted by the disparity distance set by the naked-eye 3D display. The disparity image whose display position is shifted is displayed in the air with the disparity distance shifted by the crossed mirror array; thus, stereopsis is possible only with parallel viewing. To solve this problem, the observer’s eyes are tracked by a color camera, and the disparity image is deformed according to the disparity entering each eye. First, either the left or right eye checks the parallax image displayed in the air. At the same time, a color camera is used to capture the face image and calculate the left and right coordinates, respectively. The color and coordinates of the observed disparity image are obtained, and a lookup table is generated to identify the disparity image observed from the coordinates of the eyes. In the naked-eye 3D display used in this study, the disparity image changes in response to left and right movements. For this reason, the disparity images to be observed are mapped to the $x$ coordinates. Figure 5 shows an example of disparity image selection corresponding to the left and right eyes using a lookup table.

Generally, the interocular distance of Asians is about 65 mm [17]. The lookup table gives the angle of convergence by moving the displayed disparity image 32.5 mm to the right if the observer is left-eyed and 32.5 mm to the left if the observer is right-eyed. To move the image by 32.5 mm, we only need to change the position of the display, and the number of pixels to be moved ($\Delta x$) is determined by Eq. (3). The number of pixels varies depending on the size and resolution of the display. Figure 6 shows an example of when and when not to perform the correction process.

$$\Delta x[{\rm pixel}] = 32.5\;[{\rm mm}]\frac{{\rm Display\_W\_pixels[pixel]}}{{\rm Display\_width[mm]}}.$$

The aforementioned correction method involves eye tracking. It is also possible to provide motion parallax to the observer by changing the position of the camera observing the object according to the viewpoint position.

4. RESULTS

A. Experimental Device

An assembled view of the proposed device is shown in Fig. 7. The dimensions are also indicated in Fig. 7. The specifications of the naked-eye 3D display and the crossed-mirror array used in the experiments are shown in Table 1.

In the proposed system built with the devices shown in Table 1, the viewing angle is the range of the 3D display seen from the crossed mirror array. Consequently, the viewing angle of the proposed method is calculated to be 6° left and right, which is within the viewing angle of the crossed-mirror array. In Fig. 7, the display is used vertically, but it can be expanded to 11° left and right by using it horizontally. The viewing angle of the proposed system depends on the display. Because the distance relationship between the crossed-mirror array and the VDP sets the distance between the VDP and the observer, the observation distance can be set freely depending on the placement of the device.

 figure: Fig. 8.

Fig. 8. Parallax images generated by Unity from four viewpoints with varying background colors.

Download Full Size | PDF

 figure: Fig. 9.

Fig. 9. Display image synthesized from the disparity image in Fig. 8 for display on a naked-eye 3D display.

Download Full Size | PDF

B. Experimental Methods

We performed an experiment to verify whether participants were able to observe different parallax images depending on their observation position relative to a parallax image shown on the autostereoscopic display. First, we created a disparity image to be displayed on the naked-eye 3D display. The disparity image was created with a 3D object in Unity wherein four virtual cameras were moved in parallel. The background of each camera was changed to a different color to easily check if the spectra were correct. Figure 8 shows the disparity image used in the experiment. In this naked-eye 3D display, four disparities were used. The synthesized disparity image to be displayed on the naked-eye 3D display is shown in Fig. 9. The blurring of the edges of the objects in Fig. 9 confirms the presence of parallax. By displaying the image in Fig. 9 on the naked-eye 3D display and shooting the aerial image with a camera from the position where it is supposed to be displayed in the air, we could confirm whether the image is spectrally displayed in the air. As the image displayed by the crossed-mirror array is inverted, the image on the 3D display is also inverted.

For eye tracking, we used the face landmark in Dlib to detect the contours of the eyes [18]. Six contours were detected around the eyes in Dlib. The coordinates of the eyes were obtained by determining the center of gravity from the detected contours of the left and right eyes, respectively. Equation (4) was used to calculate the center of gravity:

$$\textit{EyePos}(x,y) = \left({\frac{{\sum\limits_i^n {{x_i}}}}{n},\frac{{\sum\limits_i^n {{y_i}}}}{n}} \right),$$
where $n$ is the number of feature points of the contour, and $({x_i},{y_i})$ are the $x$ and $y$ coordinates of the feature points.
 figure: Fig. 10.

Fig. 10. Results of spectroscopic experiments of naked-eye 3D displays imaged in the air. (Sony ILCE-7M3/Sony FE 50 mm: $f$-number: $f/5$, exposure: 1/13 s, ISO: 250).

Download Full Size | PDF

 figure: Fig. 11.

Fig. 11. (Left) image observed by left eye; (right) image observed by right eye. (Zed mini: $f$-number: $f/2$, exposure: auto, other adjusting camera settings: default). (a) viewpoint a; (b) viewpoint b; (c) viewpoint c.

Download Full Size | PDF

In the experiment, we first acquired a picture of the disparity image displayed in the air with a color camera from the observation position to confirm that the spectra were correctly formed. Next, we confirmed that the disparity image was correctly deformed by eye tracking. To confirm that the image was deformed by the left and right eyes, a stereo camera (Zed mini, StereoLabs) was used, with a distance of 65 mm between the cameras. Additionally, to confirm that the image was displayed in the air, focus adjustment was performed on the aerial image and the crossed-mirror array to confirm that the image is displayed in different positions. Because the crossed mirror array is translucent, we placed the display next to the crossed mirror array and adjusted the focus.

Finally, we evaluated the resolution of the proposed method by using the contrast method to obtain the modulation transfer function (MTF) curve. In addition, we took pictures of the display with characters of different font sizes and analyzed the blurring of the characters.

C. Experimental Results

First, the results of a spectroscopic experiment in which the displayed aerial image was captured with a color camera are shown in Fig. 10.

Next, we tracked the position of the eyes and ensured that the lookup table moved the image into each eye. Figure 11 shows images of the three viewpoints captured by a stereo camera. Figure 12 also shows the result of superimposing the display objects of the left and right parallax by deformation using eye tracking.

 figure: Fig. 12.

Fig. 12. Overlaid left and right parallax images captured by stereo camera (50% transparency). (Zed mini: $f$-number: $f/2$, exposure: auto, other adjusting camera settings: default). (a) Without transformation using eye tracking and (b) with transformation using eye tracking.

Download Full Size | PDF

To confirm that the image is displayed in the air, we performed a shooting experiment when focusing on the aerial image and the crossed mirror array, respectively. Figure 13 shows the images taken by changing the focus. The left image in Fig. 13 is the aerial image, and the right image is the display placed at the same position as the crossed mirror array. Figure 13(a) shows the result of focusing on the aerial image, and Fig. 13(b) shows the result of focusing on the crossed mirror array.

 figure: Fig. 13.

Fig. 13. Examples of how the aerial image looks when the focus is adjusted. Baumer VCXU-32C ($f$-number: $f/4$, exposure: 1/12 s, gain: 1.0 dB). (a) Aerial image; (b) crossed-mirror array.

Download Full Size | PDF

Finally, we evaluated the resolution. The horizontal axis of the MTF graph shows the spatial frequency (cycles/millimeter), and the vertical axis shows the MTF value at each spatial frequency as a number from zero to 1.0. The MTF value is obtained by taking the ratio of the maximum contrast of the sine wave displayed on the naked-eye 3D display to the maximum contrast of the aerial display. The camera used to obtain the contrast was a Baumer VCXU-32C ($f$-number: $f/4$, exposure: 1/12 s, gain: 1.0 dB), and the exposure time and $f$-number were both fixed. The MTF curve graph is shown in Fig. 14. In addition, text with different font sizes was displayed to check the readability of the characters. The image displayed on the naked-eye 3D display is shown in Fig. 15(a). The dimensions of the displayed text are also shown in Fig. 15(a). The image of the characters displayed in the air by the crossed-mirror array is shown in Fig. 15(b).

 figure: Fig. 14.

Fig. 14. MTF curve obtained by the contrast method.

Download Full Size | PDF

 figure: Fig. 15.

Fig. 15. Character image display experiment. (a) Display results on a naked-eye 3D display and display dimensions (mm). (Sony ILCE-7M3/Sony FE 50 mm: $f$-number: $f/5.6$, exposure: 1/160 s, ISO: 2000); (b) aerial display results and display dimensions (mm). (Sony ILCE-7M3/Sony FE 50 mm: $f$-number: $f/4$, exposure: 1/160 s, ISO: 5000).

Download Full Size | PDF

5. DISCUSSION

From the images of the experimental results in Figs. 10 and 13, it was confirmed that each of the spectral disparity images of the naked-eye 3D display was visible as an aerial image. The disparity images were projected in order as per the principle of the proposed device, and it was confirmed that stereopsis was possible by observing each disparity with the left and right eyes. However, owing to the principle of the proposed device, it was necessary to observe the disparity with parallel vision because the image plane was the same as the virtual display plane formed at the distance between the left and right eyes. Although naked-eye stereopsis is still possible in this condition, it is not a comfortable stereoscopic view because of the angle of convergence. To solve this problem, it is necessary to move the position of the objects on the display.

From the results from each viewpoint shown in Fig. 11, we can confirm that the abovementioned problem was solved by tracking the eye position and translating it with the lookup table. The right-eye image of viewpoint a and the left-eye image of viewpoint $b$ have the same yellow parallax, but the objects move according to the observing eye. The left-eye image of viewpoint $b$ and the right-eye image of viewpoint c, which are indicated in green, also correspond to the same disparity. From the result of Fig. 12, it was confirmed that it is possible to provide a convergence angle when observing the displayed object, thus enabling natural stereopsis.

From the MTF curve graph in Fig. 14, it is seen that the MTF for the low frequency of 0.25 cycles/mm is 0.57, and the MTF for the high frequency of 1.5 cycles/mm is 0.29. As the transmittance of the crossed-mirror array used in this experiment is considered to be about 50%, we can say that the contrast ratio is about 0.5, which is the maximum value [15]. From this, it can be said that displays with MTF values of 0.25 cycles/mm to 0.3 cycles/mm, which are above 0.5, are almost free of degradation. As for the MTF of the display, the conventional MTF of OLEDw (screen size: ${305 \;\rm mm} \times {543 \; \rm mm}$ , resolution: $1080 \times 1920$, pixel pitch: 0.283 mm) is approximately 0.76 at 0.75 cycles/mm, and that of LEDw (screen size: ${320 \;\rm mm} \times {427 \; \rm mm}$, resolution: $1536 \times 2048$, pixel pitch: 0.207 mm) at 0.75 cycles/mm is reported to be approximately 0.90 [19]. The MTF of the proposed method at 0.75 cycles/mm is 0.33, which is approximately 0.66 even when the transmissivity of the crossed-mirror array is taken into account. Hence, we can say that the resolution of the proposed system is 0.86 times higher than that of OLEDw. This problem can be attributed to the fact that the naked-eye 3D display requires an optimum viewing distance to observe the correct parallax and, therefore, the distance between the light source (display) and the crossed mirror array is too far. The far distance between the light source and the crossed mirror array causes unnecessary light to enter the crossed mirror array owing to light diffusion and also causes blur due to light attenuation by diffusion. To provide a higher-resolution aerial image, the optimal observation distance of the naked-eye 3D display should be shortened.

From the character display results shown in Fig. 15 it can be confirmed that the limit of the size of the characters that can be identified when they are displayed by the proposed system is up to 40 mm. The 20 mm characters are blurred and can be seen by some people, but they are not clearly visible. Because of the distance between the display and the crossed mirror array, the character size displayed on the display and the character size in the aerial image differ. For example, a 40 mm character on the display will be 6.631 mm in the aerial image. In other words, although contactless displays are attracting attention owing to the COVID-19 pandemic, it is possible to use a character size of 7 mm or larger for information displays if this proposed system is introduced. In addition, it is possible to show buttons in three dimensions, which is expected to give a more intuitive operation feeling.

Figure 16 shows the results of an aerial display using the integral method, which has been recently reported [20].

 figure: Fig. 16.

Fig. 16. 3D image with the optical tilt effect {Ref. [13], Fig. 14(b)}.

Download Full Size | PDF

Because the integral method is a ray reproduction method, the image changes depending on the observation position. In Fig. 16, however, the image changes with the movement of the observation position but is dimly lit. Furthermore, the image from the other viewpoint is reflected and can be observed with blurred contours. Using the proposed method, no cross talk, (such as blurred contours) occurs even when the viewpoint changes. An aerial multiview system using a half-mirror has also been proposed by Takaki et al. [21]. The method assumes that the driver is driving and displays a 3D image at the back of a half-mirror [21]. They report that the resolution in pixels of the image displayed in 3D is $320 \times 200$ [21]. In contrast, as our proposed method is based on crossed-mirror arrays for aerial display, it is possible to form disparity images in the air. Furthermore, the resolution of the aerial 3D image can be improved beyond that of their method because our method provides a resolution of $540 \times 1280$.

6. CONCLUSION

We developed a device that produces a 3D aerial image by combining a naked-eye stereoscopic display, which allows stereoscopic viewing even in a stationary state, and a crossed-mirror array. The following aspects were confirmed by experiments using the proposed device:

  • 1. We conducted a spectroscopy experiment and confirmed that the device provides different disparity images in air to observers.
  • 2. It was confirmed that the disparity image could be deformed and the convergence angle could be provided to the observer’s eyes by understanding the disparity image given to each eye through eye tracking.
  • 3. The proposed method provides aerial 3D images with higher resolution than conventional methods.

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

REFERENCES

1. M. Kimura, T. Ishinabe, S. Oka, T. Shimizu, H. Tsuji, T. Usui, T. Tsuzuki, H. Yamakita, H. Okumura, K. Kajiyama, K. Hashimoto, Y. Uraoka, and S. Yoshida, “Research trend on information display,” J. Inst. Image Inf. Telev. Eng. 75, 226–238 (2021).

2. T. L. Turner and R. F. Hellbaum, “LC shutter glasses provide 3-D display for simulated flight,” Inf. Disp. Mag. 2(9), 22–24 (1986).

3. J. L. Fergason, S. D. Robinson, C. W. McLaughlin, B. Brown, A. Abileah, T. E. Baker, and P. J. Green, “An innovative beamsplitter-based stereoscopic/3D display design,” Proc. SPIE 5664, 488–494 (2005). [CrossRef]  

4. H. K. Shin, J. H. Lee, J. W. Kim, T. H. Yoon, and J. C. Kim, “Fast polarization switching panel with high brightness and contrast ratio for three-dimensional display,” Appl. Phys. Lett. 98, 063505 (2011). [CrossRef]  

5. Y. Sato, S. Yamaguchi, S. Funaki, A. Kurihara, Y. Kumagai, S. Nakamura, T. Ueno, and Y. Bao, “Quantitative evaluation of work efficiency and eye strain for remote control construction equipment using 2D/3D displays,” Open Constr. Build. Technol. J. 14, 133–138 (2020). [CrossRef]  

6. G. Chen, T. Huang, Z. Fan, X. Zhang, and H. Liao, “A naked eye 3D display and interaction system for medical education and training,” J. Biomed. Inform. 100, 103319 (2019). [CrossRef]  

7. A. Cao, L. Xue, Y. Pang, L. Liu, H. Pang, L. Shi, and Q. Deng, “Design and fabrication of flexible naked-eye 3D display film element based on microstructure,” Micromachines 10, 864 (2019). [CrossRef]  

8. G. J. Lv, Q. H. Wang, W. X. Zhao, and J. Wang, “3D display based on parallax barrier with multiview zones,” Appl. Opt. 53, 1339–1342 (2014). [CrossRef]  

9. Y. Takaki, K. Tanaka, and J. Nakamura, “Super multi-view display with a lower resolution flat-panel display,” Opt. Express 19, 4129–4139 (2011). [CrossRef]  

10. Y. Shishikui and Y. Sawahata, “Effects of viewing ultra-high-resolution images with practical viewing distances on familiar impressions,” IEEE Trans. Broadcast. 64, 498–507 (2018). [CrossRef]  

11. J. Park, S. Lee, and A. C. Bovik, “3D visual discomfort prediction: vergence, foveation, and the physiological optics of accommodation,” IEEE J. Sel. Top. Signal Process. 8, 415–427 (2014). [CrossRef]  

12. S. Markon, S. Maekawa, A. Onat, and H. Furukawa, “Interactive medical visualization with floating images,” in ICME International Conference on Complex Medical Engineering (CME) (2012), pp. 20–23.

13. J. Wena, X. Yana, and X. Jiang, “Comparative study on light modulation characteristic between hexagonal and rectangular arranged macro lens array for integral imaging based light field display,” Opt. Commun. 466, 125613 (2020). [CrossRef]  

14. H. Kim and B. Lee, “Optimal design of retroreflection corner-cube sheets by geometric optics analysis,” Opt. Eng. 46, 094002 (2007). [CrossRef]  

15. Askanet Co. Ltd., “ASKA3D,” 2018, https://aska3d.com/en/technology.html.

16. E. Miller, N. Banerjee, and T. Zhu, “Smart homes that detect sneeze, cough, and face touching,” Smart Health 19, 100170 (2021). [CrossRef]  

17. Y. J. Woo and J. S. Yoon, “Changes in pupillary distance after fat versus bony orbital decompression in Graves’ orbitopathy,” Canad. J. Ophthalmol. 52, 186–191 (2017). [CrossRef]  

18. K. Vahid and S. Josephine, “One millisecond face alignment with an ensemble of regression trees,” in IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 1867–1874.

19. A. Yamazaki, C. L. Wu, W. C. Cheng, and A. Badano, “Spatial resolution and noise in organic light-emitting diode displays for medical imaging applications,” Opt. Express 21, 28111–28133 (2013). [CrossRef]  

20. S. Choi, S. Park, and S. W. Min, “Design of ghost-free floating 3D display with narrow thickness using offset lens and dihedral corner reflector arrays,” Opt. Express 28, 15691–15705 (2020). [CrossRef]  

21. Y. Takaki, Y. Urano, S. Kashiwada, H. Ando, and K. Nakamura, “Super multi-view windshield display for long-distance image information presentation,” Opt. Express 19, 704–716 (2011). [CrossRef]  

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1.
Fig. 1. Example of how a parallax image looks owing to the rays of a naked-eye 3D display.
Fig. 2.
Fig. 2. Structural drawing of the crossed-mirror array.
Fig. 3.
Fig. 3. Example of incident and reflected light rays when mirrors are placed at right angles.
Fig. 4.
Fig. 4. Principle image of the proposed method.
Fig. 5.
Fig. 5. Example of disparity image selection for the left and right eyes using a lookup table.
Fig. 6.
Fig. 6. Examples of the proposed method: (left image) without the correction method; (right image) with the correction method.
Fig. 7.
Fig. 7. Experimental device and its dimensions.
Fig. 8.
Fig. 8. Parallax images generated by Unity from four viewpoints with varying background colors.
Fig. 9.
Fig. 9. Display image synthesized from the disparity image in Fig. 8 for display on a naked-eye 3D display.
Fig. 10.
Fig. 10. Results of spectroscopic experiments of naked-eye 3D displays imaged in the air. (Sony ILCE-7M3/Sony FE 50 mm: $f$ -number: $f/5$ , exposure: 1/13 s, ISO: 250).
Fig. 11.
Fig. 11. (Left) image observed by left eye; (right) image observed by right eye. (Zed mini: $f$ -number: $f/2$ , exposure: auto, other adjusting camera settings: default). (a) viewpoint a; (b) viewpoint b; (c) viewpoint c.
Fig. 12.
Fig. 12. Overlaid left and right parallax images captured by stereo camera (50% transparency). (Zed mini: $f$ -number: $f/2$ , exposure: auto, other adjusting camera settings: default). (a) Without transformation using eye tracking and (b) with transformation using eye tracking.
Fig. 13.
Fig. 13. Examples of how the aerial image looks when the focus is adjusted. Baumer VCXU-32C ( $f$ -number: $f/4$ , exposure: 1/12 s, gain: 1.0 dB). (a) Aerial image; (b) crossed-mirror array.
Fig. 14.
Fig. 14. MTF curve obtained by the contrast method.
Fig. 15.
Fig. 15. Character image display experiment. (a) Display results on a naked-eye 3D display and display dimensions (mm). (Sony ILCE-7M3/Sony FE 50 mm: $f$ -number: $f/5.6$ , exposure: 1/160 s, ISO: 2000); (b) aerial display results and display dimensions (mm). (Sony ILCE-7M3/Sony FE 50 mm: $f$ -number: $f/4$ , exposure: 1/160 s, ISO: 5000).
Fig. 16.
Fig. 16. 3D image with the optical tilt effect {Ref. [13], Fig. 14(b)}.

Tables (1)

Tables Icon

Table 1. Specifications of the Naked-Eye 3D Display and Crossed-Mirror Array Used in the Experiment

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

[ M i r r o r _ O U T _ X M i r r o r _ O U T _ Y M i r r o r _ O U T _ Z ] = [ cos 2 θ 0 sin 2 θ 0 1 0 sin 2 θ 0 cos 2 θ ] × [ M i r r o r _ I N _ X M i r r o r _ I N _ Y M i r r o r _ I N _ Z ] .
[ O U T _ X O U T _ Y O U T _ Z ] = [ cos 2 ( 90 θ ) 0 sin 2 ( 90 θ ) 0 1 0 sin 2 ( 90 θ ) 0 cos 2 ( 90 θ ) ] × ( [ cos 2 θ 0 sin 2 θ 0 1 0 sin 2 θ 0 cos 2 θ ] [ I N _ X I N _ Y I N _ Z ] ) = [ ( sin 2 θ + cos 2 θ ) 2 0 0 0 1 0 0 0 ( sin 2 θ + cos 2 θ ) 2 ] [ I N _ X I N _ Y I N _ Z ] = [ 1 0 0 0 1 0 0 0 1 ] [ I N _ X I N _ Y I N _ Z ] .
Δ x [ p i x e l ] = 32.5 [ m m ] D i s p l a y _ W _ p i x e l s [ p i x e l ] D i s p l a y _ w i d t h [ m m ] .
EyePos ( x , y ) = ( i n x i n , i n y i n ) ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.