Open Access
Add to BibSonomyAbstract
A three-dimensional (3D) structure designed by our proposed algorithm can simultaneously exhibit multiple two-dimensional patterns. The 3D structure provides multiple patterns having directional characteristics by distributing the effects of the artefacts. In this study, we proposed an iterative algorithm to improve the image quality of the exhibited patterns and have verified the effectiveness of the proposed algorithm using numerical simulations. Moreover, we fabricated different 3D glass structures (an octagonal prism, a cube and a sphere) using the proposed algorithm. All 3D structures exhibit four patterns, and different patterns can be observed depending on the viewing direction.
© 2016 Optical Society of America
1. Introduction
Volumetric displays, which provide natural three-dimensional (3D) images, have received significant attention [1, 2]. For example, volumetric displays of strings [3], fog [4, 5] and water drops [6] have been demonstrated. Volumetric displays have actual 3D architectures; thus, we can observe their 3D images from any viewpoint.
Volumetric displays represent multiple types of unique 3D structures and simple 3D images. For example, one such unique 3D structure was shown on the cover of a book written by Hofstadter (1979) [7]. This structure is a famous shadow art that casts three shadow patterns in the form of the letters G, E and B in different directions. Moreover, a method that optimizes the structures of shadow art objects has been recently reported [8]. Shadow art exhibits 3D structures that simultaneously contain multiple two-dimensional (2D) patterns. However, the exhibited patterns are limited to only binary patterns, such as characters, and increasing the number of patterns is difficult.
In a previous study, we developed an algorithm to design novel 3D structures that contain multiple 2D patterns [9]. The 3D structures designed by the algorithm exhibit multiple arbitrary patterns by distributing the effects of the artefacts. Unlike conventional shadow art, the exhibited patterns are not limited to binary information. Using numerical simulations, we demonstrated that up to six patterns can be included in the 3D structures. Moreover, we demonstrated an actual prototype that exhibits three different patterns using the laser-induced damage (LID) technique as shown in Fig. 1. Furthermore, we developed a quantum-dot-based volumetric display that can exhibit three multi-color patterns [10]. Because these structures can independently and simultaneously provide multiple 2D patterns with directional characteristics, we believe that they can be applied to information service systems such as digital signage.
Note that no previous prototype that exhibits more than three patterns has been reported in the literature [9, 10]. Therefore, in this study, we discuss actual prototypes of 3D structures that exhibit four 2D patterns to demonstrate the extendibility of the algorithm. Upon fabrication of the prototypes, we proposed and used a new iterative-computation-based algorithm to improve the image quality of the exhibited patterns because we found that increasing the number of patterns deteriorated the image quality. Moreover, this study presents the results of a numerical simulation of the 3D structures to verify the effectiveness of the proposed algorithm.
2. Algorithm
We used the LID technique to fabricate 3D structures as different-shaped glass objects (i.e. an octagonal prism, a cube and a sphere). In this technique, small cracks induced by a laser in transparent solid glass can represent any 3D structure. In this study, the number of cracks per unit volume, which we refer to as volume element (voxel) values, is determined by the proposed algorithm. Further details of the algorithm are discussed in the following sections. The actual coordinate points of the cracks are determined randomly in unit volume. The created coordinate data were sent to the glass-processing firm and the glass objects were fabricated there. All 3D structures were designed to contain and exhibit four patterns, which are shown in Figs. 2(a)–(d), viewable from different directions. We determined four exhibiting directions at 45° intervals. Therefore, the shapes in which the cracks are induced are octagonal prisms. Here, the original patterns are 8-bit grayscale images.
2.1. Original algorithm
The proposed algorithm is based on a previous algorithm [9]. With Fig. 3(a), we describe how the voxel values V(x, y, z) of 3D structures using the original algorithm are determined. The pixel values of the patterns exhibited from the 3D structure along the w-axis, P(u, v), are determined as follows:
The projection coordinates (u, v, w) are given by rotating the world coordinates (x, y, z). These coordinates comply with the right-hand rule. Here, the u-, v- and w-axes represent the horizontal component, vertical component and the exhibition direction of the exhibited pattern, respectively. Ideally, the voxel values V(x, y, z) should be determined to satisfy the following equation to create a 3D structure with input patterns Ii(ui, vi): Here, i takes values 1, …, N. Note that we can use digital images, such as those shown in Fig. 2, as input patterns. In the original algorithm, V(x, y, z) are determined as follows: For simplicity, we consider a 3D cubic structure that exhibits three different patterns to three different orthogonal directions, as shown in Fig. 3(a). The projection coordinates of the patterns are given as (u1, v1, w1) = (x, y, z), (u2, v2, w2) = (−z, y, x) and (u3, v3, w3) = (x, z, y). From Eq. (3)V(x, y, z) are determined as follows: Therefore, the pixel values of the three exhibited patterns Pi(ui, vi) are expressed as follows: Here, Pi(ui, vi) are expressed as a product of the input patterns Ii(ui, vi) and background noise (e.g. ∑z[I2(−z, y)I3(x, −z)] in Eq. (5)).
Fig. 3 Schematic of the algorithms: (a) determination of voxel values and (b) description of the iterative computation in the proposed algorithm.
2.2. Proposed algorithm
In the proposed algorithm, the image quality of the patterns is improved by iterative computation. Figure 3(b) shows the procedure of the improved algorithm. The procedure is divided into the following three steps:
The voxel values V(x, y, z)(k) are determined by the N input patterns Ii(ui, vi)(k) as follows:
Here, i takes values 1, …, N, and k denotes the k-th loop. Note that N ideal images, which are used as the input patterns in the original algorithm, are used as the initial input patterns Ii(ui, vi)(0). In the proposed algorithm, voxel values are determined using the geometric mean of the input patterns rather than the product. This slows the change of V(x, y, z)(0) with each iteration in order to facilitate convergence.In the k-th loop, the exhibited patterns Pi(ui, vi)(k) from the 3D structure with voxel values V(x, y, z)(k) are expressed just like Eq. (2) as follows:
Here, Pi(ui, vi)(k) are normalized to be comparable to Ii(ui, vi)(0). Ideally, Pi(ui, vi)(k) should have the same values as the initial input patterns Ii(ui, vi)(0).The input patterns used in the next (k + 1-th) loop are updated as follows:
For example, if a pixel value of the exhibited pattern Pi(ui, vi)(k) is greater than the ideal pixel value of the input pattern Ii(ui, vi)(0), the pixel value of the input pattern in the next step is updated to be lesser than the previous value. Therefore, the pixel value of Pi(ui, vi,)(k+1) is corrected to be lesser that that of Pi(ui vi)(k). As a result, the exhibited patterns would be improved with respect to the iteration number.The proposed algorithm is more time-consuming than the original one because of the iterations. In the case when M iterations are needed, the proposed algorithm needs approximately 2M times as much as the original one because iteration computation includes the projection and determination. Although time consumption of the algorithm is not concerned in this study because the 3D structures are static, it should be concerned when regarding volumetric displays exhibiting dynamic 3D structures.
3. Results
3.1. Image quality improvement
To evaluate the proposed algorithm, we performed numerical simulations of the 3D structures designed by the original and proposed algorithms. The obtained results were then compared. First, we designed 3D octagonal prism structures that exhibit four patterns shown in Figs. 2(a)–(d) using the two algorithms. In the simulations, we ignored the effect of refraction at the surface of glass in order to compare the algorithms themselves. As shown in Fig. 4(a), each exhibiting direction of the pattern is perpendicular to a lateral face of the prism. Figure 4(b) shows the exhibited patterns of the 3D structure designed using the original algorithm. Although we can recognize the exhibited patterns as the input images, significant noise is evident. To obtain the output, the exhibited patterns were normalized to have the same averages and standard deviations as the original images. Similarly, Fig. 4(c) shows each pattern of the 3D structure designed using the proposed algorithm. Here, the number of iterations was ten. As can be seen, the noise levels in the latter patterns are reduced compared with those of the original algorithm.
Fig. 4 Numerical simulation of the 3D octagonal prism structures exhibiting four patterns: (a) Relationship between the exhibiting directions, (b) patterns obtained using the original algorithm and (c) patterns obtained using the proposed algorithm.
We performed similar simulations to confirm the effectiveness of the proposed algorithm with six exhibited patterns. As shown in Fig. 5(a), we designed 3D dodecahedron structures with six patterns shown in Figs. 2(a)–(f). Here, each exhibiting direction is perpendicular to each side of the dodecahedron. Figures 5(b) and 5(c) show the exhibited patterns from the 3D structures designed using the original and proposed algorithms, respectively. In the proposed algorithm, the number of iterations was set to five. As shown in Fig. 5(b), increase in the number of exhibited patterns deteriorates the image quality. In contrast, the proposed algorithm suppresses the deterioration of image quality as shown in Fig. 5(c).
Fig. 5 Numerical simulation of the 3D dodecahedron structures exhibiting six patterns: (a) Relationship between the exhibiting directions, (b) patterns obtained using the original algorithm, (c) patterns obtained using the proposed algorithm.
We quantitatively evaluated the image quality of the patterns by using the structural similarity (SSIM) method [11]. To calculate the SSIM values, we used the input images shown in Fig. 2 as references. SSIM values provide an indication of the image quality of the patterns. Table 1 and 2 show the SSIME values of the exhibited patterns in Fig. 4 and Fig. 5. For example, the SSIM values for the leftmost patterns in Fig. 4(b) and 4(c) are 0.506 and 0.645, respectively. This indicates that the proposed algorithm improves image quality. Figure 6(a) shows the average SSIM values obtained with different numbers of patterns. The results of the evaluation confirm the effectiveness of the proposed algorithm.
Fig. 6 Results of quantitative evaluations. (a) Average SSIM values obtained with three, four and six patterns. (b) The changes of MSE with respect of the iteration number in the case when the four patterns are exhibited.
Table 1. SSIM values of the four exhibited patterns from the 3D octagonal prism structures.
Table 2. SSIM values of the six exhibited patterns from the 3D dodecahedron structures.
Figure 6(b) shows the curves of mean square error (MSE) between exhibited patterns and the original images with respect of the iteration number in the case when a 3D structure exhibits four patterns. Here, the solid and dashed curves show the MSE of the proposed and the original algorithm, respectively. We confirmed that three-fourths of the exhibited patterns (patterns A, B and C) converge whereas pattern D does not. It is required to stop the iteration at the appropriate iteration number for the most effective improvement.
3.2. Implementation
Figure 7(a) shows a prototype that exhibits the four patterns. The bottom face of the glass is an octagon (each side is 41 mm long and 80 mm high). A total of 485,000 cracks were induced in the actual space (50 mm × 50 mm × 50 mm) in the glass. A 3D structure that exhibits the four patterns in perpendicular directions to each lateral face of the prism is shown in Fig. 7(a) (red arrows). Figure 7(b) shows images of the prototype taken from the different exhibiting directions. The patterns recorded in the 3D structure are visible from the appropriate viewpoints. Note that mirror-reversed images are exhibited on parallel faces. Thus, we can confirm that an octagonal prism exhibits a single pattern per lateral face. However, the information of the exhibited patterns at the sides are lost as shown in Fig. 7, whereas the results of the simulations show the complete images. This is because the exhibited images are larger than the faces of the octagon. Therefore, the part of images is exhibited to different directions due to the refraction which was not concerned in the simulations.
Fig. 7 3D structure exhibiting four patterns implemented on a glass octagonal prism (see Visualization 1): (a) 3D structure and (b) images obtained from the exhibiting directions.
Figure 8(a) shows another prototype that exhibits the four patterns. The glass object is a cube (80 mm × 80 mm × 80 mm), in which 485,000 cracks were induced in the space (70 mm × 70 mm × 70 mm). Note that two patterns are observable per lateral face, as shown in Fig. 8(b); however, the exhibiting directions are inconsistent with the directions that were previously determined; moreover, some parts of the patterns are not exhibited well because of the refractive effect on the sides. Despite this refraction issue, the exhibiting directions are not limited to viewing angles perpendicular to the object’s sides.
Fig. 8 3D structure exhibiting four patterns implemented in a glass cube (see Visualization 2): (a) 3D structure and (b) images obtained from the exhibiting directions.
Figure 9(a) shows another prototype that exhibits the four patterns. This prototype is a glass sphere with a diameter of 100 mm, in which 485,000 cracks were induced in the space (50 mm × 50 mm × 50 mm). Here, the glass sphere are cut to have flat surface for the LID processing. The four patterns are recognizable; however, they are significantly distorted because of the refractive effect.
Fig. 9 3D structure exhibiting four patterns implemented in a glass sphere (see Visualization 3): (a) 3D structure and (b) images taken from the exhibiting directions.
4. Discussion
In this study, we have shown the results of a spherical prototype that ultimately paves the way to developing a system that can exhibit an arbitrary number of patterns. However, the refractive effect remains an ongoing issue with the spherical system. In designing such 3D structures, we must consider refraction of the exhibiting directions relative to the given glass surface. The proposed algorithm should be modified to address such issues, which depend on the implemented devices such as light emitting diodes (LEDs), strings and glasses.
Here, we discuss limitations of 3D structures. We performed numerical simulations of 3D cylinder structures exhibiting from two to twenty patterns at regular intervals. At current resolution (64 × 64 pixels), up to twelve patterns can be recognized as the input images with the proposed algorithm whereas only up to six patterns with the original one. We confirmed that the dense set of the exhibiting directions increases the noise level of the patterns because the patterns are influenced by each other. It is difficult to quantitatively evaluate the limitations because of their dependence on the combination of the input images. Therefore, we would like to report about the limitations after exhaustive consideration of an evaluation method.
We demonstrated the 3D structures on the glasses, which can represent only monochromatic and static images herein. These would be applied to interior fixtures, anniversary gifts and so forth. Moreover, we believe that color dynamic volumetric displays could expand the application fields of the 3D structures; for example, advertising, amusement and security systems; because the exhibited patterns have directional characteristics. Actually, in the previous study [9], we demonstrated 3D structures that exhibit multiple color dynamic patterns using two types of volumetric displays: one based on the LEDs and the other based on an array of strings on which an image is projected. Furthermore, a volumetric display system that can represent color 3D images with laser-induced cracks in a transparent block using a projector has been proposed [12]. We assume that the algorithm proposed in this study could improve the image quality of the color patterns of these volumetric displays by applying the algorithm to each primary color (red, green and blue). In near future, we will implement the algorithm with such volumetric displays to demonstrate a new type of information service system. Herein, the results obtained in this study have an impact on the field of the volumetric displays.
5. Conclusion
We proposed the iterative algorithm and verified the effectiveness of it using visual qualitative assessments and quantitative evaluations. Moreover, we have fabricated three prototypes, i.e. an octagonal prism, a cube and a sphere, which exhibit four 2D patterns to demonstrate the extendibility of the proposed algorithm. The results obtained using the proposed iterative algorithm and a spherical prototype provide a development path for a system that can display an arbitrary number of patterns.
Acknowledgments
This work was partially supported by the Japan Society for the Promotion of Science Grant-in-Aid No. 15J07684 and No. 25240015.
References and links
1. B. G. Blundell, A. J. Schwarz, and D. K. Horrell, “Volumetric three-dimensional display system: their past, present and future,” Eng. Sci. Ed. J. 2, 196–200 (1993). [CrossRef]
2. D. L. MacFarlane, “Volumetric three-dimensional display,” Appl. Opt. 33, 7453–7457 (1994). [CrossRef] [PubMed]
3. M. Parker, “Lumarca,” ACM SIGGRAPH ASIA 2009 (ACM, 2009) 77.
4. A. Yagi, M. Imura, Y. Kuroda, and O. Oshiro, “360-degree fog projection interactive display,” ACM SIGGRAPH ASIA 2011 (ACM, 2011), pp. 19.
5. A. Sand and I. Rakkolainen, “A hand-held immaterial volumetric display,” Proc. SPIE 9011, 90110Q (2014). [CrossRef]
6. P. C. Barnum, S. G. Narasimhan, and T. Kanade, “A multi-layered display with water drops,” Proc. ACM SIGGRAPH 2010 29, 76, Los Angeles, CA, USA. [CrossRef] (2010, Jul. 26).
7. D. R. Hofstadter, Godel, Escher, Bach: an Eternal Golden Braid, (Basic Books, New York, 1979).
8. N. J. Mitra and M. Pauly, “Shadow Art,” ACM T. Graphic 28, 156 (2009). [CrossRef]
9. H. Nakayama, A. Shiraki, R. Hirayama, N. Masuda, T. Shimobaba, and T. Ito, “Three-dimensional volume containing multiple two-dimensional information patterns,” Sci. Rep. 3, 1931 (2013). [PubMed]
10. R. Hirayama, M. Naruse, H. Nakayama, N. Tate, A. Shiraki, T. Kakue, T. Shimobaba, and T. Ito, “Design, Implementation and characterization of a quantum-dot-based volumetric display,” Sci. Rep. 5, 8472 (2015). [CrossRef] [PubMed]
11. A. Wang, A. C. Bovik, H. R. Sheikh, and H. R. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004). [CrossRef] [PubMed]
12. S. K. Nayar and N. A. Vijay, “Projection volumetric display using passive optical scatters,” Computer 40(7), 54–63 (2007). [CrossRef]
