Abstract
A microlens array (MLA) based see-through, front projection screen, which can be used in direct projection head-up displays (HUD), color teleprompters and bidirectional interactive smart windows, is evaluated for various performance metrics in transmission mode. The screen structure consists of a partially reflective coated MLA buried between refractive-index-matched layers of epoxy as reported in Ref [1]. The reflected light is expanded by the MLA to create an eyebox for the user. The brightness gain of the screen can be varied by changing the numerical aperture of the microlenses. Thus, using high gain designs, a low-power projector coupled with the screen can produce high-brightness and even 3D images as the polarization is maintained at the screen. The impact of the partially reflective coatings on the transmitted light in terms of resolution and modulation transfer function associated with the screen is studied. A condition similar to the Rayleigh criteria for diffraction-limited imaging is discussed for the microlens arrays and the associated coating layers. The optical path difference between the light transmitted from the center and the edges of each microlens caused by the reflective layer coatings should not exceed λ/4. Furthermore, the crosstalk between the front and rear projected images is found to be less than 1.3%.
© 2013 Optical Society of America
1. Introduction
Transparent displays have been of great interest as they create a magical feeling by displaying information as if it were hanging in the air. They have many applications like head-up displays (HUD), teleprompters, transparent computer/laptop displays, holographic displays and augmented reality wearable displays [1–4].
In this paper we present a partially reflective microlens array (MLA) based, see-through, front projection screen that can be projected onto from two sides simultaneously as seen in Fig. 1. Projecting two different images on either sides of a single screen and maintaining the transparency at the same time allows new kinds of interactive displays. A partial reflective MLA sandwiched between refractive-index-matched epoxy layers forms the basis of our see-through screen technology [1]. Due to this structure, the screen creates a bright display in the reflection mode (due to high gain) while maintaining excellent transparency. Ref [5]. focused on the design methodology for screens with gains on the order of 100, using rotated microlens designs. The focus of this paper is the analytical and experimental characterization of the reflective coating’s impact on the transmittance function of the screen, which was first reported in ref [1].
In Section 2, the basic operation principle and fabrication steps are explained, and the eyebox related issues are discussed using ray optics simulations. The transmitted light through the MLA screen passes through the layered media with non-flat microstructures and layers of coatings. The impact of this micro-optical element on the resolution and modulation transfer function (MTF) is studied in detail in Section 3 and a condition similar to Rayleigh criteria for diffraction limited imaging is derived.
2. See-through MLA screen
A good diffuser screen, i.e. Lambertian screen, expands the light in every direction. Thus the projected image can be seen from anywhere within the semi-sphere centered on the screen. In many applications the user has limited mobility in front of the screen, such as computers and HUDs; that’s why expanding the incident light in every direction is inefficient. A screen that concentrates the diffused light in a useful area, i.e. an eyebox, has a gain compared to a Lambertian screen. An eyebox can be described as a virtual box suspended in the air where, if the users' eyes are within that box, they can see the image on the screen [5]. The brightness gain of the screen makes it possible to be used with low-lumen projectors and it is particularly important with transparent displays where a considerable portion of the light is lost to transmission. We used a reflective MLA to diffuse the light in a controlled manner to achieve high brightness and a large enough eyebox for comfortable operation.
A partially-reflective coated MLA is sandwiched between refractive-index-matched layers of glass and epoxy, as illustrated in Fig. 2. As a result, the transmitted light sees a phase object with negligible phase variation across the MLAs, whereas the reflected light gets expanded by the MLA and creates an eyebox for the viewer. In other words, for the reflected light the screen behaves like a bright screen with a limited viewing window and for the transmitted light it behaves essentially like ordinary glass. Although the whole structure is index matched, the thickness of the partially-reflective coating introduces some phase function to the transmitted light, whose impact is discussed later in this paper.
A quartz substrate is micro-fabricated to create the MLA mold, which is used as the master for making many screens. Using the mold and a UV-curable resin (n2 in Fig. 2), an MLA is fabricated on a glass or plastic substrate (n1 in Fig. 2). The surface of the MLA is coated with a thin-film, partially reflective coating. Choices for this coating are discussed in greater detail below. Once the coating has been applied, a second glass or plastic substrate is mounted onto the MLA using the same UV-curable resin.
One key advantage of MLAs compared to diffractive optical elements is that the eyebox size is the same for all the wavelengths [6, 7]. As a result, the color balance across the overlapping eyeboxes is very good. Moreover, as the MLA surface is very smooth, the angular expansion is very uniform and there is virtually no speckle on the screen, which is particularly important when using it with laser projectors.
Since the MLA is a periodic structure, it creates diffraction orders. The MLA pitch (Λ) and wavelength (λ) determines the angular separation (θ) of diffraction orders as given by the grating equation, sin(θ) = λ/ Λ. The MLA pitch should be smaller than the pixel size of the display so that each pixel is expanded fully, but it should also be large enough to set the diffraction order spacing smaller than the minimum pupil size of the human eye, which is assumed to be 3mm. As an example, for HUD applications, the typical distance between the screen and the user is about 0.5m-1m, so the angular separation between the diffraction orders should be at most 3mrad. With these constraints, 300μm is a good choice for the MLA pitch for the screen distances in the range 0.5m-1m.
The screen with the parameters above has been modeled and simulated using Zemax software. The simulation layout can be seen in Fig. 3(a). The radius of curvature of the microlenses is set at 625μm, which is optimized using Zemax to yield the desired eyebox size of about 65cm at the user’s position. As illustrated in Fig. 3(b), a total of 50 equidistant sample points were chosen on 5 equidistant rows across the screen of size 87.5mm x 175mm. The screen size is based on what is currently available for the experiments. The model represents using a scanned laser projector to illuminate the MLA screen. Each incident beam results in a hexagonal eyebox at the driver’s position. The eyeboxes shift laterally as the scan angle increases. The sum of eyeboxes in all their shifted positions for the 50 points in the Zemax model can be seen in Fig. 3(b). The bright white region at the center of the Fig. 3(b) shows where all of the individual eyeboxes overlap that is the useable full viewing window, where every pixel on the screen can be seen by the user. The width of the full viewing window is ± 18°, which corresponds to about 65cm at the user’s position that is 1m away from the screen. Due to the rectangular shape of the screen, the maximum incidence angle in the horizontal direction is twice as much as the maximum incidence angle in the vertical direction. As a result the eyeboxes shift more in the horizontal direction, thus their overlapping region is reduced and the overlapped eyebox becomes an elongated hexagon, as seen in Fig. 3(b).
Since the screen has a symmetrical structure it can be used from both sides simultaneously. The only difference is the curvature of the microlenses, i.e. for one side it is concave and for the other side it is convex. This difference does not have an impact on the screen operation because for the concave side the image is formed less than 1mm in front of the screen and for the convex side the image is formed less than 1mm behind the screen. The different image locations cannot be perceived by the human eye. The crosstalk between the two sides of the screen when used by two people, as in Fig. 1, has been measured by projecting a white page from one side only. The luminance both on the projected side and the back side are measured. The average luminance on the back side is divided by the average luminance on the projected side to find the cross-talk ratio. The experiment was also repeated for the other side. The crosstalk is measured to be less than 1.3% as shown in Table 1.
The screen has a gain of about 3 compared to a Lambertian scatterer. The gain is calculated using Zemax simulations where a 100% reflective Lambertian scatterer and 100% reflective MLA screen are compared. The average intensity in the overlapped eyebox for the MLA screen is divided by the average intensity for the Lambertian scatterer to calculate the gain. By rotating the microlenses towards the user, we can improve the efficiency of the screen substantially and can offer brightness gains on the order of 100, as illustrated in Fig. 4. However, the fabrication of such a screen still remains as a major challenge [5].
3. Partial reflective coatings
3.1. Coating design
Many choices are available for the partially reflective coating. We have tried thin metal coatings and two different designs for wavelength selective notch coatings as the reflector layer on the MLA. A single layer metal coating is the simplest for our demonstrators. The thickness of the metal controls the reflectance of the screen and in our experiments: 40Å aluminum coating resulted in average values across the visible band of about 35% reflectance and 50% transmittance, and 15% absorption [8]. With metal coatings there is a trade-off between the transmittance and the reflectance, so the thickness of the coating should be optimized for specific applications. A thin metal coating is a good choice for broadband sources like LED based projectors. If a laser projector is used, more advanced coatings are possible, such as a notch coating that reflects nearly 100% of the RGB laser wavelengths and transmits nearly 100% of the visible spectrum outside of the reflective notches. We designed the notch coating shown in Fig. 5 for a laser pico-projector that has the RGB wavelengths of 645nm, 532nm and 445nm.
The transmission bands of the fabricated screen was measured with a grating spectrometer and found to be shifted from the original coating specifications as seen in Fig. 5. This resulted in some error in the transmittance and reflectance values and the coloration of the screen. Figure 6(a) shows the imaging setup to test the coated screens. While the metal coated screen appears in the correct color, the notch coated screen has a pinkish hue, which is primarily due to the measured transmittance characteristics in Fig. 5. Improving the coating process can in principle eliminate this problem. While the metal coated screen produces a sharp image, the notch-coated screen degrades the resolution of the transmittance as seen in Fig. 6(b). The blurring effect is quantified by measuring the MTF of the screens as discussed below.
3.2. MTF measurements
We used the slanted edge technique to measure the MTF of the MLA screens [9]. An experimental setup similar to Fig. 6(a) was used, where the resolution chart was replaced by an LCD computer monitor. A slanted edge with a 5° angle was displayed on the LCD located 80cm from the MLA screen, followed by a CCD camera at 1m distance to the MLA screen. Figure 7(a) shows the slanted edge image on the camera. First the Canny edge detection algorithm is applied to find the edge [10]. The angle of the slanted edge is subsequently computed using Principal Component Analysis (PCA) [11]. The image is up-sampled by a factor of four and the edge is straightened by an affine transformation of the whole image using the computed angle of the edge. The resultant image is shown in Fig. 7(b). The average of the columns of Fig. 7(b) results in the oversampled edge profile, which is the 1D edge spread function (ESF) of the system, as shown in Fig. 7(c). Figure 7(d) shows the derivative of the ESF, which is the point spread function (PSF) of the system. MTF is obtained by calculating the modulus of the Fourier Transform of the PSF and normalizing the resultant transfer function.
With the method described above, the MTF curves for the thin metal and notch-coated screens are obtained, as illustrated in Fig. 8. The free space MTF shows the MTF of the imaging system without any screen for reference. The camera lens diameter and f/# were adjusted to obtain a cut-off frequency of around 30cyc/deg in the experiments to make it consistent with the performance of the 20/20 vision for human eye. As seen in Fig. 8, the metal coating and the ‘no screen’ MTF curves match very well, showing that the index-matched structure behaves as expected. For MTF50, that is the MTF falling to 50%, the bandwidth of the notch coated screen is reduced almost by half, compared to the ‘no screen’ MTF. The reduced bandwidth explains the blurring observed in Fig. 6(b).
3.3. Coating thickness effect and the Strehl ratio
With the index matched screen structure that was shown in Fig. 2, the screen should not have any effect on the transmitted light. However, as seen from the MTF curves, thick coatings can degrade the resolution. Since the notch coating is thick, it introduces some phase function across each microlens and as the MLA structure is periodic, the phase function results in diffraction orders. To test the presence of diffraction orders emanating from the screen, the screen was illuminated with a 3mm collimated beam from a laser diode. The diffraction orders are faint compared to the central spot and only visible when the logarithm of the image intensity is displayed as seen in Fig. 9(a). Since MLAs are packed in a hexagonal fashion, there are six 1st order diffraction spots surrounding the central 0th order. Figure 9(b) shows the physical optics simulations for the same scenario. The details in between the diffraction orders observed in logarithmic scale in Fig. 9(b) are mostly due to interference and are missing in Fig. 9(a) due to the limited dynamic range of the camera. To quantify the noise due to diffraction and scattering, the encircled intensity plot of the experimental PSF is calculated as shown is Fig. 9(c), which is the integral of the intensity inside a circle with an increasing radius. If the phase variations due to the partially reflective coating were negligible and there were no noise due to scattering and diffraction, we would expect to see a step function with a smooth transition from zero to one. In the real case we have diffraction orders due to the coating thickness, thus we observe two steps in the encircled intensity plot. The variations between those steps are mainly due to diffraction and scattering noise.
The main problem associated with the MTF degradation is due to the thickness of the coating layers. As shown in Fig. 10(a), even though the coating thickness is uniform, the lens curvature introduces an optical path difference (expressed as Δ in Eq. (1)) between the light transmitted through the center and the edges of the lenses, where p(x,y) is the length of the ray path inside the coating, which varies between t and tmax from the center to the edge of each microlens, n2 and n3 are the refractive indices of the medium and the coating, respectively. A parametric plot of the Δ created by the coating is given in Fig. 10(b). The phase function associated with a single microlens is shown in Eq. (2) and the phase function for an array of microlenses can be expressed as in Eq. (3), where ** denote 2D convolution and dx and dy denote the pitch of the MLAs in each axis.
We performed physical optics simulations to see the effect of this phase function using MATLAB [12]. In our code, a 3mm diameter area on the screen is illuminated with a collimated, monochromatic light with λ = 550nm. The screen is a hexagonally packed MLA with 300μm pitch, where each hexagon is filled with the phase function exp(j2πΔ/λ), as shown in Fig. 11. After the light passes through the screen, it propagates 1m and is focused by a 3mm-diameter lens. The resulting intensity is compared to the diffraction-limited system, i.e. uniform phase function, to find the Strehl ratio of the actual system.
We simulated three different cases to see the relationship between the Δ and the Strehl ratio, as seen in Fig. 12. In each subsection of Fig. 12 the real part of the phase function exp(j2πΔ/λ) is shown on the left and the Strehl ratio for the corresponding Δ is shown on the right. In Fig. 12(a) the Strehl ratio is 0,91 and the real part of the phase function is non-negative. In Fig. 12(b) the Strehl ratio is 0,79 and the real part of the phase function starts to show negative values at the corners of the hexagon. In Fig. 12(c) the Strehl ratio is 0,31 and the negative values gets more dominating in the real part of the phase function. It is generally assumed that the human eye can differentiate the aberration effects for Strehl ratios less than 0,8 [13]. From the simulations we conclude that the real part of the phase function should be greater than zero to satisfy this condition. This means cos(2πΔ/λ) ≥ 0, so Δ ≤ λ/4 to eliminate aberration artifacts introduced by the coating. This is essentially identical to the well known Rayleigh criteria. The metal-coated screen has a film thickness of about 40Å and a refractive index of about 1.09 in the visible band [14], which creates a peak-to-valley OPD of 0.0006λ and results in a Strehl ratio of 0.998. However, the notch coating has more than hundred layers of coatings with refractive index of about 2,5 to 3 for some coating layers, which violates the condition Δ ≤ λ/4.
4. Conclusion
In this paper the design methodology, fabrication steps and the partial reflective coating analysis for the see-through screen were discussed. We demonstrated that a partial reflective MLA sandwiched between index-matched layers can be used as a see-through screen that can be projected onto from both sides simultaneously, as seen in Fig. 1. The large eyebox creates a comfortable operating zone for the user. The screen can be used in any application where the desired information needs to be superimposed onto the real world scene such as direct projection HUDs and teleprompters. The effect of coating selection and its impact on the screen transparency has been discussed. A limiting condition for the thickness and refractive index of the partially reflective coating has been demonstrated that can be used to ensure diffraction limited performance of the see-through screen.
Acknowledgments
We would like to thank and acknowledge Microvision Inc. for their support in this research. Mark Freeman, the second author of this paper, was the Director of Advanced Development and HUD at Microvision when this research was performed. We also would like to thank Aref MostafaZadeh for his technical contribution to the MTF calculation algorithm.
References and links
1. M. K. Hedili, M. O. Freeman, and H. Urey, “Microstructured head-up display screen for automotive applications,” Proc. SPIE 8428, 84280X1–84280X-6 (2012). [CrossRef]
2. P. Görrn, M. Sander, J. Meyer, M. Kröger, E. Becker, H. H. Johannes, W. Kowalsky, and T. Riedl, “Towards see‐through displays: fully transparent thin‐film transistors driving transparent organic light‐emitting diodes,” Adv. Mater. 18(6), 738–741 (2006). [CrossRef]
3. A. Olwal, C. Lindfors, J. Gustafsson, T. Kjellberg, and L. Mattsson, “ASTOR: an autostereoscopic optical see-through augmented reality system,” Mixed and Augmented Reality, Proceedings. Fourth IEEE and ACM International Symposium on. IEEE (2005). [CrossRef]
4. J. P. Rolland and H. Fuchs, “Optical versus video see-through head-mounted displays in medical visualization,” Presence (Camb. Mass.) 9(3), 287–309 (2000). [CrossRef]
5. M. K. Hedili, M. O. Freeman, and H. Urey, “Microlens array-based high-gain screen design for direct projection head-up displays,” Appl. Opt. 52(6), 1351–1357 (2013). [CrossRef] [PubMed]
6. H. Urey and K. D. Powell, “Microlens-array-based exit-pupil expander for full-color displays,” Appl. Opt. 44(23), 4930–4936 (2005). [CrossRef] [PubMed]
7. H. Urey, “Diffractive exit-pupil expander for display applications,” Appl. Opt. 40(32), 5840–5851 (2001). [CrossRef] [PubMed]
8. G. Hass and J. E. Waylonis, “Optical constants and reflectance and transmittance of evaporated aluminum in the visible and ultraviolet,” JOSA 51(7), 719–722 (1961). [CrossRef]
9. M. Estribeau and P. Magnan, “Fast MTF measurement of CMOS imagers using ISO 12233 slanted edge methodology,” Proc. SPIE 5251, 243–252 (2004). [CrossRef]
10. J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986). [CrossRef] [PubMed]
11. K. Pearson, “LIII. On lines and planes of closest fit to systems of points in space ,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2.11, 559-572 (1901).
12. D. G. Voelz, Computational Fourier Optics: A Matlab Tutorial (SPIE, 2011).
13. W. J. Smith, Modern Optical Engineering (Tata McGraw-Hill Education, 2000).
14. A. D. Rakić, “Algorithm for the determination of intrinsic optical constants of metal films: application to aluminum,” Appl. Opt. 34(22), 4755–4767 (1995). [CrossRef] [PubMed]