Open Access
Add to BibSonomy
Abstract
Autostereoscopic displays based on the moiré effect in LCD have the advantage of being able to display 3D animations without reversed vision, but their practical use is greatly limited due to the low and fixed pixel density of the LCD. We propose a 3D display device with LCD, which resolves this issue by generating a periodic pattern with high resolution as the virtual intermediate image. The final image is obtained by selecting and rearranging specific pixels of this virtual image. Experiments reveal that 3D images with almost the same image quality and parallax as a printed matter with abundant pixels could be displayed.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Autosteoroscopy, i.e., naked eye three-dimensional (3D) display technology that does not require glasses, has garnered significant attention recently in a wide range of fields, such as video equipment, telemedicine, remote controls and advertisement. An example is a 3Ddisplay system in which two different parallax images are displayed, one for each eye of the observer using a parallax barrier [1–3] or a lenticular lens [4–6]. However, a significant drawback of these systems is a narrow angular range, i.e., reversed vision occurs when the observer moves from the observation position.
To this end, a naked eye autostereoscopic display device based on moiré interference has been proposed, which does not cause reversed vision and therefore, has a wider angular range [7–9]. Moiré interference is the superposition of two close lying periodic patterns with slightly different intervals to generate a new pattern. This method can be used to display a simple stripe pattern as well the image of an arbitrary 3D object by using an image synthesized from the parallax images of the original object as the first periodic pattern and a lenticular lens as the second periodic pattern [10].
Such 3D display systems use moiré interference between two printed periodic patterns [11–12]. In the 3D displays based on moiré, it is extremely important to control the number of pixels that can be displayed under each lens of the lenticular lens because the width or the depth of the displayed moiré is determined by the widths of the two periodic patterns constituting the moiré. Therefore, an arbitrary width has to be specified for the two patterns in order to adjust the width or depth of the displayed moiré. In the previous research, we set arbitrary width to two patterns by adjusting print resolution (DPI). This technology we invented which use printed materials has already been commercialized by a company as “Wedys II” [13–14]. However, a display device that uses printed material is not suitable for a moving object; it can only generate a very simple animation according to the movement of an observer by synthesizing a periodic pattern from each frame image constituting the moving image.
LCDs can display a more complicated animation by continuously showing the frame images constituting the moving image. However, the pixel density of LCD is much lower as compared to that of the printed matter; for example, the pixel density of a regular ink jet printer is in the order of thousands DPI, while that of the regular LCD is few hundred DPI. Further, the pixel density can be arbitrarily set in a printer; however, it assumes a static value in a LCD screen. Therefore, LCDs cannot be used to generate the moiré from periodic patterns having arbitrary pitches. Although LCD screens are widely used in several fields, the above issues limit their advanced practical applications. To this end, we propose a 3D display device using moiré interference, which facilitates an arbitrary pixel width for both lenticular lens and the periodic pattern to be displayed on a LCD screen by generating a periodic pattern with higher resolution as a virtual intermediate image. This is followed by a thinning-like process, where some pixels are picked out from this virtual image at intervals of pixels on the LCD screen, and a new periodic pattern is finally generated by rearranging the selected pixels.
2. Principle
The basic principle of the 3D autostereoscopic display based on moiré interference is described as follows. Here, two neighboring periodic patterns with slightly different periods are overlapped with each other to generate the moiré pattern. Assuming that the periods of the two patterns corresponding to the front and rear view of the observer are “p” and “w”, respectively, the moiré pattern is displayed at the rear side for p > w, as shown in Fig. 1. It is displayed at the front side for p < w, as shown in Fig. 2. These two simple periodic patterns (Figs. 1 and 2) like a parallax barrier comprise of simple stripes with continuous black lines. However, this method cannot be used to display complex objects, such as a triangular pattern. To this end, if the periodic patterns at the front and back sides are replaced with a lenticular lens and the reduced (resized) image of an arbitrary object, respectively, arbitrary 3D objects can be displayed, which are shown in Figs. 3 and 4. Further, if the periodic pattern at the back is synthesized from the parallax images of the object, 3D display with both binocular and motion parallax can be realized.
In these displays, p and w mean the ratio of the width of a single lens of a lenticular lens to the width of a single cycle of a periodic pattern. Although p and w can be expressed in mm, since the periodic pattern that constitutes moiré is a Bitmap image, the width of p and w is expressed as num of pixels, not mm at the stage of generating the periodic pattern. For example, p is 100 if 100 pixels are arranged under a single lens. Since w is also expressed in the number of pixels, w is not a decimal but an integer close to p like 98, 99, 101, 102, etc. That is, in this conventional method, the individual lenses of the lenticular lens act as a second-order periodic pattern with a period of pitch p governed by the resolution of the print or the pixel density of the liquid crystal.
For example, if you want to use a printed matter and set p = 100, these pixels can be arranged by adjusting the print resolution. To the contrary, such an arrangement is difficult if LCD is used because the number of pixels that can be arranged depends on the pixel density of the LCD. In other words, LCDs cannot be used to the display periodic patterns with arbitrary widths.
3. Method
The proposed autostereoscopic display technique, which is based on moiré interference, includes the generation of a periodic pattern with a higher resolution that is not limited by the finite pixel density and selecting specific pixels from this high resolution image to reduce the number of pixels in such a way that all the pixels can be displayed on the LCD. In the conventional method, as described in the previous section, p is fixed to the number of actual pixels of the LCD lined under the single lens. On the other hand, in this proposed method, by generating a virtual high-resolution pattern as an intermediate image, an arbitrary number of virtual pixels are arranged under the single lens instead of the actual pixels whose number is fixed. Therefore, any value can be specified for p. Even with this proposed method, decimals can not be directly specified for w. However, because any integers can be specified for p and w, for example, if you want to specify 10 as p and 9.5 as w, by setting 20 as p and 19 as w, the ratio of p to w will be 10:9.5. In this way, it is also possible to assign decimals to w using the ratio.
The procedure for creating this virtual pattern is shown in Fig. 5. Figure 5 is the example where p = 8 and w = 7. This pattern is used as a prototype to demonstrate the 3D display. We obtain the parallax image of the object to be displayed and resize it such that its width is equal to the pitch w of periodic pattern. This resized image is called a reduced image. The pixels in each row of the reduced image are then arranged at pixel intervals of the lens pitch p. This procedure is repeated for all the parallax images to create a periodic pattern. Here, the pixel rows from each parallax image are arranged at intervals of p, and a periodic pattern with width equal to w is generated, as shown in Fig. 5. This virtual image is called the virtual second pattern (VSP).
For a LCD screen, if 8 pixels are arranged under each lens of the lenticular lens, a parallax image is displayed by seven lenses when the lenticular lens (with pitch p) is superimposed on the VSP displayed on the LCD. Further, when the observer moves towards the left and right direction with respect to the front side of the lens, the n-1th and n + 1th sheet parallax images are displayed, respectively. As different parallax images can be obtained based on the movement of the observer in the horizontal direction, 3D display is possible by displaying these images on left and right eyes. However, in practical devices, it is extremely rare that p pixels fit under a lens of the lenticular lens.
We now describe the procedure for converting the VSP into a periodic image, which is finally displayed on LCD. Figure 6 shows an example, where p is 8, w is 7, the width of each LCD pixel is 2.08, and the width of a single pixel on the VSP is 1. Here, the number of pixels arranged under a single lens, which is expected to be 8, is actually 3.85. Therefore, in order to reduce the number of pixels on the periodic pattern so that they can be displayed on the LCD, we created a new periodic pattern by removing and rearranging the pixel rows of the VSP, which is located at the center of each pixel in the LCD. The pixel rows are arranged in the order of n-2, n-4, and n-6 from the left. This image is defined as the actual second pattern (ASP).
ASP has a different cycle width w’ for each cycle; w’ = 3, 4, 3, and 3 in order from the left (Fig. 6). These values of w’ can be chosen based on the number of pixels of ASP in each cycle. The ratio of w’ and the actual number of pixels under the single lens of the lenticular lens can be used to select arbitrary values of p and w regardless of the product specifications, i.e., this selection is possible even if the LCD display and the lenticular lens have several pixel densities and widths, respectively. Finally, the lenticular lens is superimposed on the LCD screen to display the ASP, which generates the moiré pattern.
4. Experiment
4.1 Experimental details
We used our method to display a 3D image, composed of two periodic patterns with arbitrary p and w, on a LCD screen and compared it with that obtained using conventional printed matter. The specifications of the display device, printer, lenticular lenses, and the computer used are given in Tables 1–4, respectively.
Table 1. Specifications of the display device
Table 2. Printer specifications
Table 3. Specifications of the lenticular lenses
Table 4. Computer specifications
When the lenticular lens1 was overlaid on the LCD screen, 98.4 pixels were arranged under a single lens by dividing the dots per inch (DPI) of the LCD with the lines per inch (LPI) of the lenticular lens. The number of actual pixels placed under a single lens is defined as q. The computer was used for pixel rearrangement A ray tracing program (POV-Ray) was used for capturing the parallax images, and C++ and OpenCV programs were used for image processing, which included the synthesis of VSP from the captured parallax images and converting it into ASP.
In regular LCDs such as those used in this experiment, a pixel does not emit light in a specific color; it consists of three sub-pixels of red, green, and blue. Therefore, LCDs have sub-pixel rows of respective colors arranged vertically. Then, when the lenticular lens is overlaid on the LCD screen, each subpixel acts as a periodic pattern, and the moiré interference between the lenticular lens and these subpixels creates a new pattern called color moiré [15–16]. It is necessary to remove this color moiré as it ruins the 3D image. To this end, the periodic pattern is tilted obliquely and the lenticular lens is superimposed obliquely on the LCD screen so that the subpixels and the lens are not parallel (Fig. 7). When the lenticular lens is obliquely placed by the one pixel in the horizontal direction for every l pixels along the vertical direction, as shown in Fig. 7, the distance between the center of the lenses arranged along the horizontal direction of the LCD screen is different from the actual lens width q. The angle tilted is expressed by tan-1 (1/l). The modified width q′ is expressed as Eq. (1), which is based on the assumption that the two triangles t1 and t2 in Fig. 7 are identical. In this experiment, the pixels are arranged by shifting one pixel in the horizontal direction for every eight pixels along the vertical direction of the ASP. If the lens is tilted by tan-1(1/8), the modified width q′ of the lens used in the pixel arrangement is 99.17, which is obtained using Eq. (1). For the display that uses printed matter, color moiré does not occur even if the lens is overlaid parallel to the surface of the paper; however, the lens is titled by the same angle in this case to match the experimental conditions with that described above.
4.2 Results and discussion
We used the object shown in Fig. 8 for our analysis, which was imaged by POV-Ray. The VSP and the ASP of lenticular lens 1 were obtained under three conditions: p = 150, w = 140; p = 150, w = 135, and p = 200, w = 190. The VSP and ASP are shown in Figs. 9 and 10, where p = 150 and w = 140. Figure 11 shows the image obtained by overlaying the lenticular lens on the VSP printed at 3.364×150 DPI. Figure 12 shows the image obtained by overlaying the lens on the ASP displayed on the LCD with 1:1 pixel mapping. The images of the generated moiré patterns for p = 150, w = 135 and p = 200, w = 190 are shown in Figs. 13,14 and Figs. 15,16, respectively.
It is evident in these figures that the quality of images obtained using the proposed method are at par with that obtained using printed matter for all the three cases. As the pixel information is reduced in the conversion of the VSP to the ASP, as shown in Fig. 6, the generated pattern is composed of multiple parallax images from different observation points, instead of a single parallax image. Further, it is evident in Figs. 11–16 that the quality of images obtained from the ASP are not degraded as compared to that from the VSP. This may be attributed to the fact that the parallax images are obtained at fine angular intervals; therefore, the difference between the neighboring images is negligible. Further, it can also be confirmed from these figures that the contrast of the images obtained using the proposed method is much higher. This may be attributed to the presence of backlights in the LCD screen, which illuminate the screen.
We now investigate the efficacy of the proposed method with respect to the observer position. Figures 17 and 18 show the images of the object, which are obtained when the observer moves by a distance of 15 cm towards the left and right directions, respectively. From Figs. 17 and 18, in the image of the leftmost gear surrounded by the yellow frame, you can see only the side of the gear at the left observation point. On the other hand, from the observation point on the right side, it can be seen that not only the side of the gear but also the front is displayed by parallax. Next, in the second gear from the left surrounded by a red frame, it can be seen that the area of the front of the gear is larger in the image of (b) observed from the right side than in the image of (a) observed from the left side. This is because the gear rotated in the opposite direction due to parallax as the observer moved from left to right. From these results, it is evident that the images exhibit the same parallax when the observer moves by the same distance. The number of parallax frames corresponding to the movement of the observer is larger if printed matter is used. However, this difference could not be confirmed in the experiment which may be attributed to the high pixel density of the 4k display.
Next, an experiment was conducted with another lenticular lens whose pitch is 20LPI. The specifications of this lens are shown in Lens 2 of Table 3. VSP and ASP at p = 85, w = 84 are shown in Figs. 19 and 20. Figures 21 and 22 show 3D images generated by overlaying a lens on displayed VSP and ASP. The focal length of this lenticular lens is 0 mm. Therefore, when the LCD of Table 1 is used, each pixel cannot be arranged exactly at the focal length due to the thickness of the touch panel and the glass on the surface. It is known that when a plano-convex lens constituting a lenticular lens is used upside down, its focal length is extended. In this experiment with LCD, the focal length was adjusted by stacking this lens upside down.
In Fig. 21, the whole displayed image is blurred. This seem to be because the width of each printed pixel row in Fig. 21 is much smaller than when a 3.364 LPI lens is used. In the case of Fig. 11 using a lens with an LPI of 3.364, 3.364 lenses are arranged in one inch, and 150 pixels are arranged under a single lens. On the other hand, with a 20 LPI lens, 20 lenses are arranged in one inch, and 85 pixels are arranged under a single lens. Therefore, the width of the printed single pixel row has a width of only about (3.364 ×150) / (20 × 85) × 100 ≒ 30% as compared with the case where a 3.364 LPI lens is used. Thus, it is considered that a plurality of adjacent pixel rows are simultaneously seen by the observer, and such blurring has occurred.
In the image shown in Fig. 22, this blurring was reduced becaus the number of pixels actually displayed under the single lens was thinned out and the display width of single pixel rows was increased. But the edge of the gear was jagged in some places. This is presumed to be due to the fact that the actual number of pixels arranged under the single lens greatly decreased. Figures 23 and 24 are enlarged details of the VSP and ASP when the LPI is 3.364LPI and 20LPI, respectively. In Fig. 23(a) and (b), only about (150 - 331/3.364) / 150 × 100 = 34% of pixels are thinned out, whereas in Fig. 24(a) and (b) In (85 - 331/20) / 85 × 100 = 81% of pixels are thinned out. Moreover, in Fig. 24(b), because the actual pixel width of the gear was compressed to 5 pixels, it was not possible to maintain sufficient information to restore the original image. In this experiment, the lenticular lenses were used upside down in order to extend the focal length of a lens having a focal length of 0 mm. In general, it is known that when a plano-convex lens is used upside down, spherical aberration increases. Therefore, it is presumed that this spherical aberration also causes deterioration in image quality.
In the case of using a 20 LPI lens, the thinning of many pixels caused the image quality of the displayed image to deteriorate, while the moiré display cycle was similar in Figs. 21 and 22. This is because the proposed method works correctly even when a lens with a small pitch is used, and maintains the periodicity of the periodic pattern while reducing the number of pixels. Therefore, when a lens with a small pitch is used, it can be expected that 3D images having the same quality can be displayed on the printed matter and the LCD while suppressing a decrease in image quality by using a display having a higher pixel density.
5. Conclusion
We have proposed a 3D naked eye autostereoscopic display device based on moiré interference in LCD. Although, LCD is more suitable than printed matter for displaying the images of moving objects, the pixel density of the LCD screen is fixed which limits its usage for periodic patterns with arbitrary periods. To resolve this issue, our method provides arbitrary widths to both the lenticular lens and the displayed periodic pattern by creating a periodic pattern with higher resolution as a virtual image, which is not limited by the finite pixel density. This is followed by thinning of the image at an interval corresponding to the actual display. We compared the moiré pattern displayed from both printed matter and the LCD for three different conditions on the pitch of the lenticular lens and periodic pattern to find that the quality of the images is similar in both the cases. The moiré pattern is generated from multiple neighboring parallax images in the proposed method. Further, we confirmed that the effect of the movement of observer is similar for both the display devices based on LCD and printed matter. In another experiment using a finer pitch lenticular lens, it was confirmed that the moiré cycle and the width of the generated 3D object were the same when using printed matter and when using an LCD. This result is consistent with the previous experiment. On the other hand, the image quality of the displayed moiré was degraded due to the use of a lens having a small pitch. This is because the use of a thin lens reduces the number of pixels existing under a single lens, so that more pixels must be thinned out. In the previous experiment using a lens with a large pitch, this image quality problem did not arise because the number of pixels to be thinned was small. Therefore, if you use a lens with a fine pitch, you can solve the problem of image quality by using an LCD with a sufficiently high DPI.
Although, “Wedys” has already been established in the advertisement field as a practical demonstration of moiré based autostereoscopic display using printed materials, our method facilitates the use of LCD for displaying more complicated animation and eye catching 3D visual effects without causing reversed vision. Further, in principle, this method is applicable to almost all the commercially available LCDs as well as the recently popular organic light emitting diodes (OLEDs). Therefore, we believe our study boosts the practical applications of autostereoscopic displays based on moiré interference. However, this method is limited in the sense that it can be used to display only periodic images due to the intrinsic property of the moiré. We expect to resolve this issue in future work to develop a more versatile 3D display, which can be used for advertising as well as in various other fields.
Acknowledgments
We thank for my colleagues for helpful discussions and comments on the manuscript. We would like to thank Editage (www.editage.com) for English language editing.
Disclosures
The authors declare no conflicts of interest.
References
1. G. J. Lv, F. Wu, R. Wu, B. C. Zhao, J. Fan, and J. J. Lv, “High resolution parallax barrier 3D display based on a sub display panel with transparent slits,” Optik 127(7), 3569–3571 (2016). [CrossRef]
2. D. H. Kang, B. S. Oh, J. H. Oh, M. K. Park, H. J. Kim, S. M. Hong, J. H. Hur, J. Jang, S. J. Lee, K. H. Lee, and K. H. Park, “Auto-stereoscopic TFT-LCD with LC parallax barrier on wire grid polarizer,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 40(1), 344–347 (2009). [CrossRef]
3. T. Peterka, R. L. Kooima, D. J. Sandin, A. Johnson, J. Leigh, and T. A. DeFanti, “Advances in the Dynallax solid-state dynamic parallax barrier autostereoscopic visualization display system,” IEEE Trans. Visual. Comput. Graphics 14(3), 487–499 (2008). [CrossRef]
4. M. Lambooij, K. Hinnen, and C. Varekamp, “Emulating autostereoscopic lenticular designs,” J. Disp. Technol. 8(5), 283–290 (2012). [CrossRef]
5. Y. Takaki, K. Tanaka, and J. Nakamura, “Super multi-view display with a lower resolution flat-panel display,” Opt. Express 19(5), 4129–4139 (2011). [CrossRef]
6. W. X. Zhao, Q. H. Wang, A. H. Wang, and D. H. Li, “Autostereoscopic display based on two-layer lenticular lenses,” Opt. Lett. 35(24), 4127–4129 (2010). [CrossRef]
7. S. G. Wang, “Moiré-based auto-stereoscopic images by duplex printing on transparencies,” Patent Application Publication US 2007/0097111 A1.
8. C. Yamada and H. Isono, “Experiments on pseudoscopic 3d display using moiré method,” Terebijon Gakkaishi 62(3), 420–428 (2008). [CrossRef]
9. Y. Bao and K. Nagasaki, “A Moire Type of 3D Display Providing Variable Depth Feeling,” Journal of IIEEJ 38(3), 290–295 (2009). [CrossRef]
10. N. Oishi and Y. Bao, “Periodic stereo image which removes viewing zone boundary by using a feature of moiré interference,” Journal of IIEEJ 39(6), 1074–1087 (2010). [CrossRef]
11. N. Oishi, “A study of periodic stereo image which uses a feature of moiré interference,” ITE Technical Report 14(24), 15–18 (2010).
12. N. Oishi, “Periodic stereo image which has periodically changing depth,” ITE Technical Report 35(15), 51–54 (2010).
13. Grapac Japan Co., Inc., “Explanation of Wedys” On-line shop for Wedys parts. http://www.wedys.net/free7.php?lang_id=en.
14. grapacjapan, “Wedys II –RUNNER- Evacuation Guidance POP,” Online video. YouTube, https://www.youtube.com/watch?v=mtMYM8zlaTY.
15. Y. Kim, G. Park, J. Jung, J. Kim, and B. Lee, “Color moiré pattern simulation and analysis in three-dimensional integral imaging for finding the moiré-reduced tilted angle of a lens array,” Appl. Opt. 48(11), 2178–2187 (2009). [CrossRef]
16. Y. Kim, G. Park, S. W. Cho, J.-h. Jung, B. Lee, Y. Choi, and M. G. Lee, “Integral imaging with reduced color moiré pattern by using a slanted lens array,” in Proceedings of Stereoscopic Displays and Applications XIX 6803, 68031L (2008).
