Open Access
Add to BibSonomy
Abstract
Human eyes are often “cheated” by an optical illusion (or visual illusion) so that the perceived image differs from the physical reality. But various optical illusions have been seldom investigated for technological applications such as image processing and optical display in the past. As a unique attempt of combining information technology with optical illusion, we propose a novel image steganography scheme based on a color assimilation illusion. A synthesized image containing a grayscale background and a saturated color line (or point) grid can be perceived as a color image, with external secret data hidden simultaneously.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Optical illusion (or visual illusion) refers to a phenomenon that the image perceived by human vision is different from the physical reality. Human eyes are “cheated” when a visual illusion occurs. Many different types of visual illusions have been found in the past [1,2] and three simple examples are shown in Fig. 1.
Fig. 1. Three simple examples of visual illusions: (a) the line below appears to be longer than the line above but they actually have the same length; (b) the two parallel red lines appear to be curves; (c) the colors of two arrows appear to be different but actually they are identical.
Visual illusions receive much attention in vision science, psychology, creative arts and popular culture. But visual illusions have been seldom investigated for practical technological applications such as image processing, computer vision, and optical imaging & display. For example, it is reported that very few works take the visual illusion issue into account in the popular deep learning research field [3]. Recently Øyvind Kolås [4] demonstrates a new visual illusion based on the color assimilation (or chromatic assimilation) effect. We propose a novel image steganography (or information hiding) scheme based on this visual illusion. Our work is a very unique attempt of combining information technology with visual illusion.
The various optical illusions can be roughly classified into six categories: luminance and contrast, motion, geometric or angle illusions, 3D interpretation (size constancy and impossible figures), cognitive/gestalt effects and color [5]. Color illusions [6] can be further classified into color constancy, color assimilation, visual completion, visual scission etc. The color assimilation illusion has been discussed in many literatures in the past few decades such as [7–10]. When a color area is partly occluded by a grating with another color, the perceived color of this area by human eyes can be distorted. Munker illusion, dungeon illusion, dotted color illusion, and De Valois-De Valois illusion are proposed for one-dimensional grid, two-dimensional grid, dot grid and checkboard grid respectively. The cognitive mechanism behind this kind of illusion has been extensively investigated by vision researchers at different levels [11]. Explanations have been proposed in terms of retinal receptive fields and cortical receptive fields at the low level, belongingness cued by T-junctions at the middle level and experience with “natural contexts” at the high level. The illusion proposed by Øyvind Kolås is similar to these traditional color assimilation illusions. But it has a unique feature that the grayscale background pixels are assimilated into color pixels.
Image steganography allows external information to be secretly embedded (or hidden) into a host image, which can be applied for secure communication, copyright protection and ownership identification. The change of visual appearance in the host image before and after data embedding is usually minimal so that the steganography operation is not easily noticed or detected. Steganography techniques can be implemented by computer algorithms or physical means [12–26]. Current digital image steganography techniques are mainly divided into two categories [27], spatial domain image steganography and transform domain image steganography. In the former category, the pixel intensity values of the host image are slightly modified based on the hidden secret data and the modification is not easily noticed. The data embedding in spatial domain can be implemented based on least significant bit (LSB), pixel value differencing (PVD), histogram shifting, difference expansion, multiple bit-planes, palette, pixel intensity modulation and quantization. In the latter category, the secret data is inserted into certain transformed spectrum components. Various image transforms can be utilized including discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete wavelet transform (DWT), integer wavelet transform (IWT), complex wavelet transform (CWT), dual-tree complex wavelet transform (DT-CWT) and compressive sensing (CS). In addition, adaptive image steganography techniques with machine learning are emerging recently. Even though there are many different existing digital and optical image steganography methods, visual-illusion-based steganography has been rarely reported in the past literatures.
2. Visual illusion based on a color assimilation grid
In the visual illusion demonstrated by Øyvind Kolås [4], a color image is first down-sampled by a grid consisting of lines or points. Then the saturation of the color pixels on the grid will be enhanced. The color pixels outside the grid will be converted to grayscale pixels. Finally, the majority of pixels are grayscale and only a small percentage of pixels are colored in the synthesized image. However, such a synthesized image will be perceived by human eyes as being very similar to the original color image. The fact that most pixels are grayscale will not be evidently noticed. It can be considered that the saturated color pixels and neighboring grayscale pixels in the image are “averaged” in the human vision mechanism.
An example of Øyvind Kolås’ illusion is shown in Fig. 2. The original color image is shown in Fig. 2(a). A saturated color grid consisting of horizontal lines (sampling ratio:1/3) or points (sampling ratio:1/4) is overlaid on the corresponding grayscale background image (Fig. 2(b)). The synthesized images are shown in Fig. 2(c) and Fig. 2(d) respectively and they appear to be similar to the original color image. The existence of grayscale pixels is not evident unless a small image block in the synthesized image is significantly enlarged, shown in the right side of Fig. 2(c) and Fig. 2(d)
Fig. 2. Examples of Øyvind Kolås’ visual illusion: (a) color image; (b) grayscale image; (c) synthesized image with a color line grid; (d) synthesized image with a color dot grid.
Mathematically, the saturation of each color grid pixel can be adjusted in the following way [28]. R, G and B denote the pixel intensity in red, green and blue channels respectively. Max and Min refer to the maximum one and minimum one of the three values. Then the luminance L and saturation S are given by Eqs. (1) and (2). The image saturation is enhanced by a ratio η (0≤η≤1) and the saturation is maximum when η=1. The R, G and B values will be adjusted as R′, G′ and B′ according to Eqs. (3,6). The ratio η needs to be manually tuned adaptively for each different image to achieve optimal quality. η is typically set to be between 0.5 and 1.
The similarity between color assimilation grid illusion, image dithering and chroma subsampling is that the image quality can still be preserved to certain extent after some image information is lost (or the image information is compressed in some way). However, the working principles of these three schemes are different. Image dithering maintains the color depth by properly distributing pixels spatially when the color quantization levels are reduced. Chroma subsampling maintains the image quality when the chromatic channels have much lower resolution than the luminance channel. But it shall be noticed that none of the color pixels are converted to grayscale pixels in chroma subsampling. Color assimilation grid illusion maintains the image color perception when the chromatic information of many pixels is lost and only the luminance channel is fully preserved.
In comparison, image dithering and chroma subsampling have been widely used in various technology fields. For example, image dithering has been applied in optical imaging [29,31], printing [32,33] and display [34–36]. Chroma subsampling plays a key role in image and video compression [30,37,38]. On the other hand, the application potential of color assimilation grid illusion has not received enough attention. When a color image is converted to a synthesized image with color assimilation grid illusion, the amount of color information will be significantly compressed. It is anticipated that this feature can be explored for various applications such as color sensing, printing, display, data compression and digital arts. Like a dithered image, a synthesized image with color assimilation grid illusion can be considered as a special type of image format.
3. Proposed image steganography schemes
For a synthesized image based on Øyvind Kolås’ visual illusion, the lines and points are not necessarily distributed with equal spacing on the grid. Therefore, we propose that the synthesized image is used as the host carrier and the positions of lines or dots can be encoded based on the embedded secret information for image steganography.
For a grid with horizontal lines, the spacing between every two neighboring sampling lines can be adjusted as 2 rows or 1 rows. Every 5 rows can be encoded in two different ways to represent 1 binary bit of data, shown in Fig. 3.
Fig. 3. Binary encoding of grid lines for representing embedded data in our proposed scheme (green cell: one row of saturated color pixels; white cell: one row of grayscale pixels).
For a grid with points, the position of 2 sampling points in each window of 2×2 pixels can be arranged in two different ways to represent 1 binary bit of data, shown in Fig. 4. In the synthesized image, half of the pixels will be grayscale and the other half of the pixels will be colored. The two schemes stated above are similar to one-dimensional barcode and two-dimensional matrix barcode respectively.
Fig. 4. Binary encoding of grid points for representing embedded data in our proposed scheme (green cell: saturated color pixel; white cell: grayscale pixel).
The hidden data retrieval in these two schemes is straightforward. The task is to simply determine each pixel is grayscale or colored. For a grayscale pixel, the red, green and blue intensities are equal. For a color pixel, the three intensities usually have variations. The variation σ can be calculated by Eqs. (7) and (8). When all the color pixels are found based on the pixel-wise chromatic variation in the synthesized image, the color grid can be recovered and the hidden secret data can be thus retrieved.
The security level and robustness of our proposed scheme can be further enhanced by including scrambling encryption and error correction coding. The overall flowchart of our proposed enhanced image steganography scheme is shown in Fig. 5. The secret message to be hidden is first encoded as a binary bit sequence. The order of binary bits is scrambled with a key K. The original bit sequence can only be correctly recovered when the key K is known. This operation can significantly enhance the security. After that, error correction coding is applied to the scrambled bit sequence and some additional error check bits (marked in red in Fig. 5) are added. Error correction coding allows the automatic detection and correction of bit errors at the expense of including redundant non-message bits. The robustness of our proposed steganography scheme can be enhanced. Reed-Solomon error correction coding is adopted in this work, which is also used in quick-response (QR) codes [39–41]. Then the bit sequence with error correction capability is embedded into the synthesized visual illusion image. They can be retrieved by the steps described above as well. Some bits in the retrieved bit sequence may be wrong (marked in blue in Fig. 5) due to noise contamination or other factors. As long as the percentage of bit errors is within the error correction limit, all the wrong bits can be corrected and the original scrambled bit sequence can be recovered. Finally, the secret hidden message can be obtained if the scrambling key K is known. Only random message can be obtained if K is not known.
One major advantage of our proposed image steganography scheme over conventional schemes is that information hiding and color information compression are performed by simple down-sampling simultaneously. As stated above, a synthesized image with color assimilation grid illusion can be potentially utilized in different applications as a special image format since it’s favorable that the chromatic information is compressed. For example, the color pixels can be replaced with low-cost grayscale pixels in the design of sensor, printer and display devices. The steganography scheme proposed above is tailor-made for copyright protection, ownership identification and other information security schemes with this image format. Steganography techniques based on compressive sensing [42] can perform host image compression and external information hiding simultaneously as well. But it requires computational recovery of missing information. The recovery of missing chromatic information in our proposed scheme is performed naturally by human eyes.
4. Experimental results and discussions
First, there are many different ways to design the color saturation grid for synthesizing the visual illusion image. For example, the thickness of lines, the diameter of dots, the angle orientation of the grid and the down-sampling ratio can all be adjusted. In this work, we only consider two basic types of grids, horizontal line grid and point grid. The thickness of lines and the diameter of point is set to be one pixel. The synthesized results with different down-sampling ratios are shown in Fig. 6. A subjective image quality rating experiment is performed by 40 volunteers who are not investigators of this research work. The volunteers are asked to give a score in the range between 0 and 10 for each image, regarding to the overall quality and color fidelity. As references, the grayscale image in Fig. 2(b) is the baseline for a score of 0 and the original image in Fig. 2(a) is a baseline for the score of 10. The average rating score for the six images in Fig. 6 are 3.4, 4.9, 7.3, 2.7, 5.0 and 7.7 from Fig. 6(a) to Fig. 6(f) respectively. The results indicate that both sparse and dense grids can produce color assimilation illusion to certain extent. But the perceived quality will be generally better as the percentage of saturated color pixels in the image increases. Figure 6(c) and Fig. 6(f) receive high scores since their sampling ratio both reach 1/2. Under the same sampling ratio, a point grid generally performs better than a line grid.
Fig. 6. Synthesized image with color assimilation illusion by a line grid: (a) 1/4 sampling ratio; (b) 1/3 sampling ratio; (c) 1/2 sampling ratio; by a point grid: (d) 1/9 sampling ratio; (e) 1/4 sampling ratio; (f) 1/2 sampling ratio.
Our proposed image steganography scheme is tested with two color images (96×96 pixels) shown in Fig. 7(a) and Fig. 8(a). They are converted to synthesized visual illusion images by line grids [Fig. 7(c), Fig. 7(d), Fig. 8(c) and Fig. 8(d)] and by point grids [Fig. 7(e), Fig. 7(f), Fig. 8(e) and Fig. 8(f)] respectively. Due to the visual illusion, all the synthesized images are perceived as color images by observers and appear to be similar to the original color image shown in Fig. 7(a) or Fig. 8(a). It is hard to evaluate the similarity by an objective metric since the visual illusion does not exist physically. But common subjective feelings reveal that the synthesized images have acceptable visual quality with some minor artifacts. The average subjective rating score for images with point grid is around 8 and the average subjective rating score for images with line grid is around 5. Different external data are embedded into these synthesized images. For line grids, 19 bits can be hidden by our proposed scheme. Two sets of totally different binary bits are embedded in Fig. 7(c) and Fig. 7(d) respectively. But the visual appearances of Fig. 7(c) and Fig. 7(d) are very close. This reveals that the data hiding is not easily noticed by one attacker. Similar observations can be found in Fig. 8(c) and Fig. 8(d). For point grids, 2304 (48×48) bits can be hidden by our proposed scheme, shown in Fig. 7(e), Fig. 7(f), Fig. 8(e) and Fig. 8(f). Similarly, the host images with different data hidden appear to be similar as well. The feasibility of our proposed scheme is verified by these experimental results.
Fig. 7. (a) Original color image; (b) Corresponding grayscale image; (c-d) Synthesized visual illusion images by line grids with secret data hidden (right); (e-f) Synthesized visual illusion images by point grids with secret data hidden (right two-dimensional barcode with black square dots denoting “0” and white square dots denoting “1”).
Fig. 8. (a) Original color image; (b) Corresponding grayscale image; (c-d) Synthesized visual illusion images by line grids with secret data hidden (right); (e-f) Synthesized visual illusion images by point grids with secret data hidden (right two-dimensional barcode with black square dots denoting “0” and white square dots denoting “1”).
Our proposed enhanced scheme with better security and robustness is verified by experimental results as well. In the Reed-Solomon coding, every eight bits are grouped as one byte. If any single bit within a byte is wrong, the entire byte will be considered wrong. For n bytes, k bytes contain the message and (n−k) bytes contain error correction bits. At most, t=(n−k)/2 wrong bytes can be corrected. There are many different options for the values of n and k. In this work, the setting is n=255, k=165 and t=45. The secret hidden message is a binary image with 36×36 pixels shown in Fig. 9(a) and the total number of bits is 1296 bits (162 bytes, slightly less than 165 bytes).
Fig. 9. (a) Original secret message: a binary pattern with 36×36 pixels (enlarged version); (b) Retrieved message with the correct scrambling key; (c) Retrieved message with the wrong scrambling key.
After scrambling encryption and error correction coding, the 2040 (255×8) binary bits are embedded into a synthesized point-grid host image containing 92×92 pixels shown in Fig. 10(a) with our proposed scheme described in Fig. 5. The hidden data bits can be retrieved from Fig. 10(a) in a reverse manner and finally the secret message (binary pattern) can be recovered. The error correction coding mechanism allows the stored message to be recovered correctly even if the host image is damaged to certain extent. All the wrong bits can be corrected automatically as long as the percentage of bit errors is within the pre-defined threshold. In Fig. 10(b)-(d), the synthesized host image is damaged with scratch lines, cropping and random dots respectively. The corresponding recovered secret binary patterns are shown in Fig. 9(b) if the scrambling key is known, as intact as Fig. 9(a). However, if the key is wrong, the recovered message will be random, shown in Fig. 9(c). The results show that our proposed image steganography scheme based on color assimilation illusion will have both good security strength and robustness after scrambling encryption and error correction coding are adopted.
Fig. 10. (a) Synthesized host image by a point grid containing secret message with our proposed scheme; Synthesized host image damaged in different ways by: (b) scratch lines; (c) cropping; (d) random noise.
5. Conclusion
In summary, we propose a novel image steganography scheme based on color assimilation grid illusion in this work. This scheme differs from conventional digital and optical image hiding methods by utilizing visual illusion for hiding binary bits in an image. The feasibility of this scheme is verified by experimental results. In future works, the potential of connecting various visual illusions with technology applications can be more extensively explored.
Funding
National Natural Science Foundation of China (61805145).
Disclosures
The authors declare no conflicts of interest.
References
1. A. G. Shapiro and D. Todorovic, “The Oxford compendium of visual illusions” (Oxford University, 2016).
2. S. Coren and J. Girgus, “Seeing is deceiving: The psychology of visual illusions” (Routledge, 2020).
3. A. Gomez-Villa, A. Martin, J. Vazquez-Corral, and M. Bertalmío, “Convolutional neural networks can be deceived by visual illusions,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12309–12317 (2019).
5. M. Bach and C. M. Poloschek, “Optical illusions,” Visual Neurosci. 6(2), 20–21 (2006).
6. A. Kitaoka, “A brief classification of colour illusions,” Colour: Design & Creativity 5(3), 1–9 (2010).
7. S. Anstis, “White’s effect in lightness, color, and motion,” Seeing spatial form, 91 (2006).
8. R. L. De Valois and K. K. De Valois, “Spatial vision,” Annu. Rev. Psychol. 31(1), 309–341 (1980). [CrossRef]
9. P. Bressan, “Explaining lightness illusions,” Perception 30(9), 1031–1046 (2001). [CrossRef]
10. M. White, “The assimilation-enhancing effect of a dotted surround upon a dotted test region,” Perception 11(1), 103–106 (1982). [CrossRef]
11. M. White, “The early history of White’s illusion,” JAIC-Journal of the International Colour Association 5 (2010).
12. E. T. Lin and E. J. Delp, “A review of data hiding in digital images,” PICS 299, 274–278 (1999).
13. S. Jiao, C. Zhou, Y. Shi, W. Zou, and X. Li, “Review on optical image hiding and watermarking techniques,” Opt. Laser Technol. 109, 370–380 (2019). [CrossRef]
14. B. Wu, Z. Wang, B. J. Shastri, M. P. Chang, N. A. Frost, and P. R. Prucnal, “Temporal phase mask encrypted optical steganography carried by amplified spontaneous emission noise,” Opt. Express 22(1), 954–961 (2014). [CrossRef]
15. C. Zhang, W. He, B. Han, M. Liao, D. Lu, X. Peng, and C. Xu, “Compressive optical steganography via single-pixel imaging,” Opt. Express 27(9), 13469–13478 (2019). [CrossRef]
16. S. Jiao, J. Feng, Y. Gao, T. Lei, and X. Yuan, “Visual cryptography in single-pixel imaging,” Opt. Express 28(5), 7301–7313 (2020). [CrossRef]
17. T. Shimobaba, Y. Endo, R. Hirayama, D. Hiyama, Y. Nagahama, S. Hasegawa, M. Sano, T. Takahsahi, T. Kakue, M. Oikawa, and T. Ito, “Holographic microinformation hiding,” Appl. Opt. 56(4), 833–837 (2017). [CrossRef]
18. H. Hamam, “Digital holography-based steganography,” Opt. Lett. 35(24), 4175–4177 (2010). [CrossRef]
19. X. Li, M. Zhao, X. Zhou, and Q. H. Wang, “Ownership protection of holograms using quick-response encoded plenoptic watermark,” Opt. Express 26(23), 30492 (2018). [CrossRef]
20. S. Jiao, D. Zhang, C. Zhang, Y. Gao, and X. Yuan, “Data hiding in complex-amplitude modulation using a digital micromirror device,” Opt. Lasers Eng. 138, 106455 (2021). [CrossRef]
21. M. Sharifzadeh, M. Aloraini, and D. Schonfeld, “Adaptive batch size image merging steganography and quantized gaussian image steganography,” IEEE Trans.Inform.Forensic Secur. 15, 867–879 (2020). [CrossRef]
22. X. Zhang, F. Peng, and M. Long, “Robust coverless image steganography based on DCT and LDA topic classification,” IEEE Trans. Multimedia 20(12), 3223–3238 (2018). [CrossRef]
23. A. S. Ansari, M. S. Mohammadi, and M. T. Parvez, “A multiple-format steganography algorithm for color images,” IEEE Access 8, 83926–83939 (2020). [CrossRef]
24. D. Hu, H. Xu, Z. Ma, S. Zheng, and B. Li, “A spatial image steganography method based on nonnegative matrix factorization,” IEEE Signal Proc. Lett. 25(9), 1364–1368 (2018). [CrossRef]
25. J. Tao, S. Li, X. Zhang, and Z. Wang, “Towards robust image steganography,” IEEE Trans. Circuits Syst. Video Technol. 29(2), 594–600 (2019). [CrossRef]
26. Z. Wang, X. Zhang, and Z. Qian, “Practical cover selection for steganography,” IEEE Signal Proc. Lett. 27, 71–75 (2020). [CrossRef]
27. I. Kadhim, P. Premaratne, P. Vial, and B. Halloran, “Comprehensive survey of image steganography: techniques, evaluations, and trends in future research,” Neurocomputing 335, 299–326 (2019). [CrossRef]
28. M. K. Agoston and M. K. Agoston, Computer graphics and geometric modeling (Springer, 2005), pp. 301–304.
29. J. Rapp, R. M. Dawson, and V. K. Goyal, “Dithered depth imaging,” Opt. Express 28(23), 35143–35157 (2020). [CrossRef]
30. A. Wong and W. Bishop, “Practical content-adaptive subsampling for image and video compression,” Eighth IEEE International Symposium on Multimedia (ISM'06), 667–673 (2006).
31. J. Sun, C. Zuo, S. Feng, S. Yu, Y. Zhang, and Q. Chen, “Improved intensity-optimized dithering technique for 3D shape measurement,” Opt. Lasers Eng. 66, 158–164 (2015). [CrossRef]
32. M. R. Gupta and J. Bowen, “Ranked dither for high-quality robust printing,” J. Opt. Soc. Am. A 25(6), 1454–1458 (2008). [CrossRef]
33. V. Ostromoukhov, P. Emmel, N. Rudaz, I. Amidror, and R. D. Hersch, “Dithering algorithms for variable dot size printers,” Proceedings of 3rd IEEE International Conference on Image Processing1, 553–556 (1996).
34. J. B. Rodriguez, G. R. Arce, and D. L. Lau, “Blue-noise multitone dithering,” IEEE Trans. on Image Process. 17(8), 1368–1382 (2008). [CrossRef]
35. S. W. Lee and H. Nam, “A new dithering algorithm for higher image quality of liquid crystal displays,” IEEE Trans. Consumer Electron. 55(4), 2134–2138 (2009). [CrossRef]
36. H. J. Choi and M. Park, “A time-sequential autostereoscopic 3D display using a vertical line dithering for utilizing the side lobes,” Holography, Diffractive Optics, and Applications, International Society for Optics and Photonics9271, 92710 T (2014).
37. W. J. Yang, K. L. Chung, W. N. Yang, and L. C. Lin, “Universal chroma subsampling strategy for compressing mosaic video sequences with arbitrary RGB color filter arrays in H. 264/AVC,” IEEE Trans. Circuits Syst. Video Technol. 23(4), 591–606 (2013). [CrossRef]
38. C. Babu, D. A. Chandy, and P. Karthigaikumar, “Novel chroma subsampling patterns for wireless capsule endoscopy compression,” Neural Comput & Applic 32(10), 6353–6362 (2020). [CrossRef]
39. J. F. Barrera, A. Mira, and R. Torroba, “Optical encryption and QR codes: Secure and noise-free information retrieval,” Opt. Express 21(5), 5373–5378 (2013). [CrossRef]
40. J. F. Barrera, A. Vélez, and R. Torroba, “Experimental scrambling and noise reduction applied to the optical encryption of QR codes,” Opt. Express 22(17), 20268–20277 (2014). [CrossRef]
41. S. Jiao, W. Zou, and X. Li, “QR code based noise-free optical encryption and decryption of a gray scale image,” Opt. Commun. 387, 235–240 (2017). [CrossRef]
42. Y. Zhang, L. Y. Zhang, J. Zhou, L. Liu, F. Chen, and X. He, “A review of compressive sensing in information security field,” IEEE Access 4, 2507–2519 (2016). [CrossRef]
