Open Access
Add to BibSonomy
Abstract
Visible light communication (VLC) can be integrated into a liquid crystal display (LCD) by modulating its backlight while normally showing pictures. Received by ordinary cameras, such integrated display and communication (IDAC) systems are promising for the Internet of Things and Metaverse. However, in the premise of unaffected display function, the capacity of current IDAC systems is limited, with data rates of very few kbps. This work proposes a new architecture: multiple-input, multiple-output (MIMO) VLC integrated into a mini-LED LCD, whose many backlight segments act as multiple transmitters. A camera utilizes the rolling shutter effect with independent pixel columns to form multiple outputs. The communication capacity is thus significantly multiplied by the backlight column number. In addition, local dimming, which is favorable for an LCD’s contrast and power consumption, is exploited to achieve efficient signal modulation. We built a mini-LED LCD prototype with 8-by-20 backlight segments for experimental verification. The backlight segments multiplex a video-rate signal for local dimming and a high-frequency (∼34 kHz) signal modulated through multi-pulse position modulation (MPPM) for VLC. By taking photographs with a camera 1.1 m away from the screen, a record-high rate of 201.6 kbps (approximately ten times faster than current IDAC systems) was experimentally achieved with a bit error rate satisfying the forward error correction. Improved image contrast due to local dimming was also observed.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
In the coming era highlighted by the Internet of Things (IoT) and Metaverse represented by virtual reality (VR) and augmented reality (AR) [1,2,3], display and communication are cornerstones of paramount importance. It is desirable to enable displays to be the transmitter of visible light communication (VLC) [4–5] while normally showing pictures [6–23], i.e., an emerging concept this study focuses on—integrated display and communication (IDAC). IDAC merges two crucial functions in a single device to simultaneously deliver information to users and machines. On the receiver side, compared with dedicated sensors (e.g., photodiodes), the popular CMOS-based camera is a very convenient choice.
Operating on the visible band, IDAC features high bandwidth, unlicensed spectrum, negligible electromagnetic interference, and high security [24], opening possibilities for novel applications. For example, public information displays (PIDs) deliver real-time meta-information to passersby; position information is embedded in indoor displays to facilitate indoor navigation/location; AR glasses and monitors share data through high-speed VLC interfaces. Moreover, the high directionality of visible light highlights the value of IDAC in security-sensitive scenarios, such as military communications and data centers [5].
Though high-speed VLC integrated into illumination sources is emerging [5,25–29], integrating VLC into a display is still a big challenge because the display function must not be affected. Current IDAC systems with an ordinary camera as the receiver mainly adopt two schemes: light source modulation and visual MIMO, as reviewed below. However, they can only achieve a transmission rate of very few kbps and an unqualified bit error rate (BER).
- (i) The light source modulation scheme temporally modulates the backlight of a liquid crystal display (LCD) or a direct-view LED display [6–12] at a high frequency invisible to the human visual system (HVS). When a CMOS camera is used as the receiver, the camera’s frame rate limits the data rate. Regarding the issue, the rolling shutter effect (RSE) [30], meaning pixel rows of a CMOS sensor are activated sequentially, has been exploited. As a result, the transmitted signal is recorded as dark and light stripes in one picture frame, which are then parsed to recover data. For example, DisCo [6] separated the high-frequency flicker and the video-rate display pattern by capturing two images with different exposures. Though the display function was seamlessly integrated into DisCo, the data rate was only 237 bps. Various stripe demodulation schemes based on machine learning or neural networks were recently proposed by Chow et al. to improve the data rate of RSE-based VLC [7–10,31–33]. Of the works, the LSTM-NN scheme [33] achieved the highest data rate of 14.4 kbps. In addition, almost all current light source modulation studies achieve a BER low enough to satisfy the forward error correction (FEC) requirement.
The light source modulation scheme allows for easy VLC-display integration because the two functions use distinct frequencies to guarantee a low BER regardless of image content. However, the backlight is used as a single input, and the frame rate of a CMOS camera is only 30 or 60 frames per second (fps), limiting the data rate to approximately ten kbps.
- (ii) Based on the spatial resolution difference between the HVS and cameras, visual MIMO modulates a display’s pixel values to embed data; then, the embedded information is retrieved by inspecting recorded frames. In visual MIMO, different pixel arrays act as independent transmitters to increase the communication capacity. For example, InFrame++ [14] used spatiotemporally complementary frames to carry data bits to achieve a bit rate of 19.26 kbps. Nguyen et al. proposed TextureCode [15] to attain flicker-free communication by exploiting the low sensitivity of the HVS on texture-rich regions. The highest data rate was 16.52 kbps under a display frame rate of 120 fps. Chen et al. addressed 2D barcodes with unobtrusive embedding in the blue channel and achieved a data rate of 34.33 kbps [16]. In addition to the spatial domain, the frequency domain of images is also used for information hiding. Kim et al. [17] achieved flicker-free transmission with a data rate of 9.5 kbps by embedding data bits after Hermitian symmetry into the frequency sub-bands of images.
Visual MIMO generally has a higher rate than light source modulation because multiple pixel arrays work independently. However, retrieving data from captured pictures is sensitive to grayscale fluctuation, inducing a BER too high to satisfy the FEC. In addition, exploiting the resolution gap between the HVS and the camera depends on image content. For example, embedding data is more challenging in solid-color pictures than in rich textures. The slow frame rate of displays also limits the capacity of visual MIMO.
Table 1 summarizes data rates, BERs, and communication distances of several IDAC studies (Supplement 1 will discuss the distance). The light source modulation family is superior in BER but not good at the rate, while visual MIMO is the opposite.
Table 1. Performance of existing IDAC systems
Regarding the dilemma, this study, for the first time, considers the emerging mini-LED LCD [34] as a new platform for VLC. Mini-LEDs enable LCDs with a thinner backlight module, a higher contrast ratio, and lower power consumption [35–37], thus emerging for high-end display products. Current source modulation-based IDAC uses the LCD backlight as a single transmitter. Nevertheless, mini-LEDs bring about numerous backlight segments, which can be independently controlled not only for precise local dimming but also for multiple VLC transmitters. This study exploits these backlight segments as multiple VLC transmitters, along with a CMOS camera as multiple outputs. As a result, a high-capacity MIMO IDAC system is demonstrated with a record-high data rate of 201.6 kbps and a qualified BER (see the last row of Table 1), significantly surpassing previous studies in both rate and BER.
The micro-LED is also a promising display technology [38], while current VLC based on a small-scale micro-LED array can achieve Gbps-level rates. However, high-resolution, large-sized, and full-color displays are still challenging for micro-LEDs. In addition, as a self-emissive display, locally driving micro-LEDs with display and VLC signals in a MIMO manner is complex. Thus, this study does not consider micro-LEDs for IDAC.
2. Method
2.1 System overview
The proposed IDAC system consists of an LCD with a mini-LED backlight as the transmitter (Tx), an RSE-based CMOS camera as the receiver (Rx), and a user watching the screen, as shown in Fig. 1.
On the Tx side, backlight segments are modulated through pulse modulation (see details in Sec. 2.3) at a high frequency (∼30 kHz), enveloped by video-rate waveforms (∼60 Hz) determined by local image content, known as local dimming [34–37]. Local dimming effectively boosts the contrast ratio and reduces the power consumption of an LCD by locally dimming the backlight under dark image areas. Compared with edge-lit, a direct backlight can easily adjust local luminance. In the past, the long optical distance hindered the direct backlight until the fine-pitch mini-LED emerged, enabling a thickness comparable to edge-lit and improved contrast due to finer segments [34–37]. Along with the intrinsic high peak brightness and high reliability, the mini-LED LCD has become a strong competitor to OLED displays. At the same time, VLC modulation requires dimming light sources [39], so display and VLC performance can be simultaneously improved through local dimming.
On the Rx side, the user, which holds a critical flicker frequency of around 60 Hz [40], perceives the video-rate backlight penetrating the LC panel as normal pictures. The camera utilizes the RSE to capture rich stripes in picture frames. The RSE is critical in IDAC because it breaks the limitation set by the native frame rate of a camera by exposing the CMOS sensor progressively rather than simultaneously (i.e., global exposure). Ideally, the exposure time of a pixel row coincides with the bit rate and the readout delay between adjacent rows, so a row can represent one bit, as Fig. 2(a) shows. In reality, a row is activated without waiting for the previous row to complete the exposure; i.e., a row’s exposure time is longer than the readout delay, as shown in Fig. 2(b). In this case, several consecutive rows are merged to record a bit.
Fig. 2. The RSE’s working principle: (a) ideal and (b) real case, where Tep and Ts denote a row’s exposure time and the readout delay between adjacent rows, respectively. (c) A temporal waveform from a global-dimming backlight recorded as 1D stripes through the RSE. (d) Waveforms independently transmitted by the columns of a local-dimming backlight recorded as 2D stripes through the RSE, forming 1D MIMO. (e) Waveforms independently transmitted by the backlight’s rows and columns recorded by a highspeed sensor array, constituting 2D MIMO.
In conventional IDAC based on a global backlight, the RSE converts the Tx’s temporal resolution to the CMOS sensor’s vertical spatial resolution (i.e., row number), as Fig. 2(c) shows. However, the data rate is still limited by the sensor’s vertical resolution, while the lateral resolution is wasted. To exploit the sensor’s lateral resolution, the proposed system independently modulates the backlight segments in the column direction to let every column transmit an independent bit sequence, as Fig. 2(d) shows. From a MIMO point of view, the multiple inputs and outputs are achieved by partitioning the backlight array and the CMOS sensor in the column direction. Supplement 1 demonstrates that the inter-channel crosstalk (ICC) hardly affects our MIMO. Therefore, we can multiply the communication capacity by the backlight column number.
The mini-LED backlight can also be independently modulated in the column and row directions, implying that the MIMO can be two-dimensional for a capacity gain of the total segment number. This study only exploits the column number because the RSE occupies the camera’s vertical resolution. Suppose a 2D array of highspeed Rx corresponds to the backlight segments; MIMO with a gain of the total segment number is possible, as Fig. 2(e) shows. Nevertheless, this study will not cover such 2D MIMO because the practicability will be much lower than the commercial camera-based system. Note that the video signal constantly manipulates the backlight in a 2D manner to form a standard local-dimming LCD, regardless of the VLC part.
Figure 3 illustrates the data flow in the proposed system, whose inputs include an image and a bit sequence. According to the input image, a specific local dimming algorithm determines each backlight segment’s dimming level and compensates for pixel grayscales, which will not be affected by the VLC part. After a series-parallel conversion, bit sequences independently modulate the backlight columns to multiply the bit rate. The modulated backlight signals are transferred to the corresponding LEDs, while the compensated grayscales are transmitted to the LC panel. On the Rx side, the RSE-based camera captures stripe patterns, which, after demodulation, can be compared with the input bits to estimate the BER. The above data flow includes three phases: local dimming, modulation, and RSE-based demodulation, which are discussed in detail in the following sections. Note that this study focuses on proposing a new MIMO architecture for capacity-multiplied IDAC but does not limit the specific methods used for the three phases. Hence, we adopt popular approaches for the three phases while improved ones can be employed instead.
Fig. 3. Data flow in the proposed IDAC system: from an image (Input 1) and a bit sequence (Input 2) to the LC panel, the backlight, and then received by the camera (ignore the human user). P/S and S/P denote conversions between series and parallel.
2.2 Local dimming
Figure 4 shows the flow chart of a typical local dimming process. In dimming level determination, the luminance level of each backlight segment is obtained per the image content above it with a specific algorithm. This study adopts the typical “max” algorithm; i.e., the backlight level equals the image block’s maximum luminance. Next, the actual backlight distribution is calculated by convoluting the dimming level map with the light spread function, which can be modeled using the Gaussian function. Finally, as the backlight is dimmed, grayscales are compensated for preserving the image luminance. More detailed considerations of local dimming can be found in [41–44].
Fig. 4. Flow chart of the local dimming process and an example with 9-by-16 segments: (a) an input image and its luminance map; (b) the dimming level map; (c) the actual backlight distribution; (d) the output image, generated by the actual backlight penetrating the LC panel with compensated gray levels, and the output luminance map.
In local dimming, the halo effect usually appears at the edges of a bright object surrounded by a dark background [45]. We apply VLC through pulse modulation while keeping the dimming level determined by the local dimming algorithm. Thus, the integration of VLC does not affect the halo effect. On the other hand, the halo effect suggests the RSE stripes at the edges of a backlight segment may be confused by adjacent segments for an increased BER. Hence, developed solutions for halos are suggested for the proposed IDAC system [45,46].
2.3 Modulation
Carrying data bits via visible light requires the light source to be dimmed. Favorably, a mini-LED backlight is already locally dimmed on request by the display performance. Of various intensity modulation methods in the VLC area, we adopt the widespread multi-pulse position modulation (MPPM) [39]. In MPPM, each symbol duration is partitioned into m chips. The transmitter sends w optical pulses during one symbol duration. We fix m and vary w for dimming control according to the dimming level. We define the dimming factor as γ = w/m. In this study, m is ten, and the optical pulse’s maximum number, w, is nine, considering VLC is unavailable when the backlight is fully lit (meaning a peak luminance decreased to 90%). Thus, γ ranges from 0.1 to 0.9. Figure 5(a) shows two examples of MPPM symbols. For a given γ, the number of light pulses emitted in each symbol period is 10γ. Different positions of the emitted pulses represent different symbols, so the symbol number is given in Eq. (1)
Fig. 5. (a) Two examples of MPPM symbols (γ = 0.5 and 0.6). (b) The data packet structure, where a packet (blue), repeated three times to be contained in one image frame, comprises three sections (purple).
From Eq. (1), every symbol can transmit $k = \lfloor{{{\log }_2}C} \rfloor$ bits. Therefore, when the symbol period is fixed, different dimming levels result in varying data rates. Generally, a duty cycle of around 50% brings about the highest rate. Under the 10-level chips, previous studies [39] demonstrated that the highest data rate occurs when the dimming level is 0.4, 0.5, and 0.6.
Regarding the processing time gap, each packet is transmitted three times to guarantee a complete VLC packet captured in one image frame. In addition, self-synchronization is necessary to perform demodulation at the Rx end. Hence, a header, a piece of dimming-level information, and a payload constitute a data packet, as shown in Fig. 5(b).
2.4 RSE-based demodulation
A captured image contains distinct stripes overlapping the normal picture, partitioned in the column direction in compliance with the backlight’s column number. The flow chart in Fig. 6 shows that the captured image experiences pre-processing and demodulation for data retrieval. First, a distortion correction algorithm is applied to each frame, followed by a region detection algorithm delineating the stripe partitions. A complete packet is confirmed by recognizing two headers, and the payload data is located between the two headers.
After pre-processing, threshold determination converts the grayscale values to binary. The accuracy of threshold determination depends on image content and transmission rate. Dark image content and inter-symbol interference (ISI) induced by a high transmission rate reduce stripe contrast, resulting in a high BER. To improve robustness, we use a convolutional auto-encoder (CAE) [47–50] that excels in image denoising, feature extraction, well-represented data generation, etc., to reconstruct the payload. The CAE is trained to remove noises from the input data for better robustness. Our dataset includes experimental images and synthetic images generated by modeling the rolling shutter acquisition process. CAE treats image content and ISI in the striped image as “noise,” while the original input is binary striped images corresponding to the transmitted bits. In this manner, CAE can reconstruct stripe features of each payload through feature extraction. After thresholding grayscales, a complete data packet is acquired. The dimming level is decoded from the symbol between the frame header and payload. Knowing the dimming level, the MPPM demodulation, which adopts a detector based on the Maximum Likelihood (ML) criterion, obtains a bit sequence.
3. Verification and result
3.1 Experimental setup
This section performs a proof-of-concept experiment to verify the proposed system, as Fig. 7(a) shows. We built a mini-LED backlight module in-house. The backlight contains 8-by-20 segments, whose columns are separately driven by arbitrary waveform generators (AWGs, DG4162 from Rigol; sampling rate: 1Gsa/s; bandwidth: 70 MHz). Note that mini-LED backlights with more segments are mature in the display industry (e.g., iPad Pro from Apple). A 14-inch LC panel covered the mini-LED array to form a complete LCD. More parameters of the LC panel (LP140WH4-TCL1 from LG Display) and the backlight are shown in Table 2. On the Rx end, a commercial CMOS camera (X-S10 from Fujifilm) took photographs of the screen 1.1 m away. The camera’s “electronic shutter” was on to enable the rolling shutter. Table 3 shows detailed configurations of the camera.
Table 2. Parameters of the LCD for experimental verification
Table 3. Camera configurations
Figure 7(b) shows the data flow in the experiment. When inputting an image, a computer performs the “max” local dimming algorithm to get the segments’ dimming levels. Next, according to the dimming levels, the computer converts twenty bit sequences to twenty MPPM-modulated waveforms with a time slot frequency of 34.6 kHz, sent to the corresponding backlight columns through the AWGs. The compensated grayscales acquired by the local dimming algorithm are transmitted to the LC panel via an HDMI port. Finally, image files captured by the camera are sent to the computer for demodulation.
The experiment began with a typical image sky shown in Fig. 8(a). The dimming level map is shown in Fig. 8(b). To validate the maximum communication rate in our experiments, we imprinted the maximum brightness to 50% of that before dimming, i.e., the dimming level of MPPM is 0.5. Figure 8(c) shows an example of bit sequence modulation. Section 2.1 mentions that the backlight segments are independently modulated in the column direction, so the backlight segments in the same column must flash at the same frequency, causing all segments in a column to have the same dimming level. To break this limitation, we changed the feedback resistor for each backlight segment to adjust the brightness per the dimming level map.
Fig. 8. (a) The input image sky in the experiment. (b) The dimming level map obtained through the max algorithm. (c) An example of bit sequence modulation using MPPM.
3.2 Experimental results
For the image sky, the camera captured a motion video. Twenty columns of partitioned stripes could be observed in every frame. Figure 9 shows three frames corresponding to different stripe widths for the same picture. The stripe widths correspond to 42, 92.4, and 201.6 kbps transmission rates by setting the time slot frequency to 8.65, 17.30, and 34.60 kHz, respectively. Figure 10 shows the main pre-processing procedures for each frame and the training output of CAE. To know the maximum available data rate, we measured the BER as a function of data rate, as shown in Fig. 11, indicating that the BER can still satisfy the FEC threshold when the bit rate reaches 201.6 kbps.
Fig. 9. RSE-induced stripe patterns captured in a camera frame when the data rate is (a) 42, (b) 92.4, and (c) 201.6 kbps.
Fig. 10. Main demodulation procedures under the data rate of 201.6 kbps: (a) Data packets extracted between two headers; (b) data packets after thresholding and denoising through CAE.
In addition, an image flower with more black zones was adopted to test the dependence on image content, as shown in Fig. 12. When the black area increases, the stripe area decreases, leading to a reduced communication rate of 100.08 kbps when satisfying the FEC. Indeed, more black areas will further lower the rate. Nevertheless, a frame usually contains a certain bright content in different backlight columns, so the proposed system must be much faster than the conventional system based on a single backlight source. Future studies will optimize the data packet protocol for varying image content, while this study focuses on the hardware architecture.
Fig. 12. RSE-induced stripe patterns captured in frames of the image flower with more black areas, which bring about data rates of (a) 21, (b) 42, and (c) 100.08 kbps.
In addition to the VLC function, the display function of the system was verified. By adjusting the camera’s exposure to 1/60 s to imitate the HVS, the camera photographed the display screen, as shown in Figs. 13 and 14. To verify the benefit of local dimming, Fig. 13(a) and Fig. 14(a) show full-on backlights and front-of-screen images, while Fig. 13(b) and Fig. 14(b) correspond to local dimming enabled, which is seen to improve the contrast of the black area at the bottom; i.e., the light leakage caused by the full-on backlight was addressed by local dimming.
Fig. 13. Image flower captured by the camera: (a) full-on and (b) local dimming backlights. The first row shows photographs of the backlight, and the second provides front-of-screen pictures. The image brightness in the third row is intentionally increased by 60% to observe the light leakage more easily.
4. Discussions
In Sec. 2, we mentioned that the boost in communication capacity comes from the direction of the column. In the experiment, we built an LCD with 8-by-20 segments, so twenty Tx were employed. Currently, mainstream mini-LED LCDs can reach up to 100 columns [34–37], suggesting a data rate higher than 1 Mbps, as Table 4 estimates. Another way to increase the communication capacity is to reduce the number of packet transmissions, such as loss compensation for packets [51] and beacon-jointed packet reconstruction [52], reflected in the last row of Table 4.
Table 4. Data rate of the proposed system and its expansion
The communication distance in our experiments was 1.1 m, which is, however, not the limit of our systems. Two practical measures can increase the distance of 1.1 m. One is to adopt a lens with a longer focal length proportional to the communication distance. The other is to use a larger-sized LCD while fixing the focal length (see details in Supplement 1).
This study adopts regular LEDs as the backlight. In the future, light sources with a higher modulation bandwidth, such as quantum-dot LEDs and more miniatured LEDs [53–55], can be utilized due to potentially shorter carrier lifetime. By combining a high-speed photodiode array as receivers, the proposed IDAC architecture can directly multiply the communication capacity through the 2D MIMO scheme in Fig. 2(e).
5. Conclusions
Combining VLC and display functions in a single device, IDAC opens new horizons to human-machine and machine-machine interaction. Using a camera as the receiver further adds convenience. However, existing IDAC systems suffer from low data rates and high BERs because the VLC function has to compete with the display function for the limited capacity of a display’s light source. Regarding this, we multiplied the capacity by considering the backlight segments of a mini-LED LCD work independently, proposing a brand-new MIMO VLC architecture. A proof-of-concept experiment was built using an LCD with 8-by-20 backlight segments. The experiment demonstrated a record-high VLC rate of 201.6 kbps at a communication distance of 1.1 m, approximately ten times faster than current IDAC systems. The BER was low enough to meet the FEC requirement. Meanwhile, source dimming required by VLC enabled local dimming, improving the LCD’s contrast. By adopting more backlight segments in the future, the proposed system’s communication capacity can be easily expanded, and the display performance can be improved through more precise local dimming.
Funding
National Key Research and Development Program of China (2022YFB3602803, 2021YFB2802300); Natural Science Foundation of Guangdong Province (2021A1515011449); Basic and Applied Basic Research Foundation of Guangdong Province (2023B1515040023).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. J. Xiong, E.-L. Hsiang, Z. He, et al., “Augmented reality and virtual reality displays: Emerging technologies and future perspectives,” Light: Sci. Appl. 10(1), 216 (2021). [CrossRef]
2. P. H. Pathak, X. Feng, P. Hu, et al., “Visible light communication, networking, and sensing: A survey, potential and challenges,” IEEE Commun. Surv. Tutorials 17(4), 2047–2077 (2015). [CrossRef]
3. T. Koonen, “Indoor optical wireless systems: Technology, trends, and applications,” J. Lightwave Technol. 36(8), 1459–1467 (2018). [CrossRef]
4. T.-W. Lu, Y. Huang, S.-Q. Lai, et al., “Full-duplex visible light communication system based on single blue Mini-LED acting as transmitter and photodetector simultaneously,” J. Lightwave Technol. 41(9), 1–11 (2023). [CrossRef]
5. Z. Xu, W. Niu, Y. Liu, et al., “31.38 Gb/s GaN-based LED array visible light communication system enhanced with V-pit and sidewall quantum well structure,” Opto-Electron. Sci. 2(5), 230005 (2023). [CrossRef]
6. K. Jo, M. Gupta, and S. K. Nayar, “DisCo: Display-camera communication using rolling shutter sensors,” ACM Trans. Graph. 35(5), 1–13 (2016). [CrossRef]
7. C.-W. Chow, R.-J. Shiu, Y.-C. Liu, et al., “Secure mobile-phone based visible light communications with different noise-ratio light-panel,” IEEE Photonics J. 10(2), 1–6 (2018). [CrossRef]
8. K.-L. Hsu, Y.-C. Wu, Y.-C. Chuang, et al., “CMOS camera based visible light communication (VLC) using grayscale value distribution and machine learning algorithm,” Opt. Express 28(2), 2427–2432 (2020). [CrossRef]
9. Y.-C. Chuang, C.-W. Chow, Y. Liu, et al., “Using logistic regression classification for mitigating high noise-ratio advisement light-panel in rolling-shutter based visible light communications,” Opt. Express 27(21), 29924 (2019). [CrossRef]
10. C.-W. Chow, Y. Liu, C.-H. Yeh, et al., “Display light panel and rolling shutter image sensor based optical camera communication (OCC) using frame-averaging background removal and neural network,” J. Lightwave Technol. 39(13), 4360–4366 (2021). [CrossRef]
11. X. Li, B. Hussain, J. Kang, et al., “Smart µLED display-VLC system with a PD-based/camera-based receiver for NFC applications,” IEEE Photonics J. 11(1), 1–8 (2019). [CrossRef]
12. X. Bao, J. Pan, Z. Cai, et al., “Real-time display camera communication system based on LED displays and smartphones,” Opt. Express 29(15), 23558–23568 (2021). [CrossRef]
13. T. Li, C. An, X. Xiao, et al., “Real-time screen-camera communication behind any scene,” in Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (ACM, 2015), pp. 197–211.
14. A. Wang, Z. Li, C. Peng, et al., “InFrame++: Achieve simultaneous screen-human viewing and hidden screen-camera communication,” in Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (ACM, 2015), pp. 181–195.
15. V. Nguyen, Y. Tang, A. Ashok, et al., “High-rate flicker-free screen-camera communication with spatially adaptive embedding,” in 35th Annual International Conference on Computer Communications (IEEE, 2016), pp. 1–9.
16. C. Chen, W. Huang, L. Zhang, et al., “Robust and unobtrusive display-to-camera communications via blue channel embedding,” IEEE Trans. on Image Process. 28(1), 156–169 (2019). [CrossRef]
17. B. W. Kim, H.-C. Kim, and S.-Y. Jung, “Display field communication: Fundamental design and performance analysis,” J. Lightwave Technol. 33(24), 5269–5277 (2015). [CrossRef]
18. S.-Y. Jung, J.-H. Lee, W. Nam, et al., “Complementary color barcode-based optical camera communications,” Wirel. Commun. Mob. Comput. 2020, 1–8 (2020). [CrossRef]
19. X. Zhang, J. Liu, Z. Ba, et al., “MobiScan: An enhanced invisible screen-camera communication system for IoT applications,” Trans. Emerg. Telecommun. Technol. 33(4), e4151 (2022). [CrossRef]
20. L. R. Ximenes, B. A. Laredo, and R. Arthur, “Integrated data detection and video restoration for optical camera communications,” Digit. Signal Process. 141, 104192 (2023). [CrossRef]
21. T. Fujihashi, T. Koike-Akino, T. Watanabe, et al., “DNN-based simultaneous screen-to-camera and screen-to-eye communications,” in Global Communications Conference (IEEE, 2019), pp. 1–6.
22. L. D. Tamang and B. W. Kim, “Deep D2C-Net: Deep learning-based display-to-camera communications,” Opt. Express 29(8), 11494–11511 (2021). [CrossRef]
23. N. Maharjan, L. D. Tamang, and B. W. Kim, “Dense D2C-Net: dense connection network for display-to-camera communications,” Opt. Express 31(19), 31005–31023 (2023). [CrossRef]
24. K. Wang, T. Song, Y. Wang, et al., “Evolution of short-range optical wireless communications,” J. Lightwave Technol. 41(4), 1019–1040 (2023). [CrossRef]
25. S. Zhu, P. Qiu, X. Shan, et al., “High-speed long-distance visible light communication based on multicolor series connection micro-LEDs and wavelength division multiplexing,” Photonics Res. 10(8), 1892–1899 (2022). [CrossRef]
26. T. Wu, Y. Lin, Y.-M. Huang, et al., “Highly stable full-color display device with VLC application potential using semipolar µLEDs and all-inorganic encapsulated perovskite nanocrystal,” Photonics Res. 9(11), 2132–2143 (2021). [CrossRef]
27. Y.-M. Huang, C.-Y. Peng, W.-C. Miao, et al., “High-efficiency InGaN red micro-LEDs for visible light communication,” Photonics Res. 10(8), 1978–1986 (2022). [CrossRef]
28. G.-R. Lin, H.-C. Kuo, C.-H. Cheng, et al., “Ultrafast 2 × 2 green micro-LED array for optical wireless communication beyond 5 Gbit/s,” Photonics Res. 9(10), 2077–2087 (2021). [CrossRef]
29. W.-T. Huang, C.-Y. Peng, H. Chiang, et al., “Toward high-bandwidth yellow-green micro-LEDs utilizing nanoporous distributed Bragg reflectors for visible light communication,” Photonics Res. 10(8), 1810–1818 (2022). [CrossRef]
30. C.-W. Chow, C.-Y. Chen, and S.-H. Chen, “Visible light communication using mobile-phone camera with data rate higher than frame rate,” Opt. Express 23(20), 26080–26085 (2015). [CrossRef]
31. K. Liang, C.-W. Chow, and L. Yang, “Mobile-phone based visible light communication using region-grow light source tracking for unstable light source,” Opt. Express 24(15), 17505–17510 (2016). [CrossRef]
32. C.-W. Chen, C.-W. Chow, Y. Liu, et al., “Efficient demodulation scheme for rolling-shutter-patterning of CMOS image sensor based visible light communications,” Opt. Express 25(20), 24362–24367 (2017). [CrossRef]
33. C.-W. Peng, C.-W. Chow, D.-C. Tsai, et al., “Mitigation of PAM4 rolling shuttered pattern grayscale ambiguity in demodulation utilizing long short term memory neural network (LSTM-NN) in optical wireless communication systems,” Opt. Commun. 532, 129260 (2023). [CrossRef]
34. Y. Huang, E.-L. Hsiang, M.-Y. Deng, et al., “Mini-LED, Micro-LED and OLED displays: present status and future perspectives,” Light: Sci. Appl. 9(1), 105 (2020). [CrossRef]
35. E. Chen, J. Guo, Z. Jiang, et al., “Edge/direct-lit hybrid mini-LED backlight with U-grooved light guiding plates for local dimming,” Opt. Express 29(8), 12179–12194 (2021). [CrossRef]
36. Y. Huang, G. Tan, F. Gou, et al., “Prospects and challenges of mini-LED and micro-LED displays,” J. Soc. Inf. Disp. 27(7), 387–401 (2019). [CrossRef]
37. W. Zhang, Y. Chen, J. Cai, et al., “Uniformity improvement of a mini-LED backlight by a quantum-dot color conversion film with nonuniform thickness,” Opt. Lett. 48(21), 5643–5646 (2023). [CrossRef]
38. E. Chen, Z. Fan, K. Zhang, et al., “Broadband beam collimation metasurface for full-color micro-LED displays,” Opt. Express 32(6), 10252–10264 (2024). [CrossRef]
39. K. Lee and H. Park, “Modulations for visible light communications with dimming control,” IEEE Photon. Technol. Lett. 23(16), 1136–1138 (2011). [CrossRef]
40. A. Eisen-Enosh, N. Farah, Z. Burgansky-Eliash, et al., “Evaluation of critical flicker-fusion frequency measurement methods for the investigation of visual temporal resolution,” Sci. Rep. 7(1), 15621 (2017). [CrossRef]
41. H. Chen, R. Zhu, M.-C. Li, et al., “Pixel-by-pixel local dimming for high-dynamic-range liquid crystal displays,” Opt. Express 25(3), 1973–1984 (2017). [CrossRef]
42. G. Tan, Y. Huang, M.-C. Li, et al., “High dynamic range liquid crystal displays with a mini-LED backlight,” Opt. Express 26(13), 16572–16584 (2018). [CrossRef]
43. F.-C. Lin, Z. Qin, K.-T. Teng, et al., “Color breakup suppression based on global dimming for field sequential color displays using edge information in images,” Opt. Express 27(3), 2335–2343 (2019). [CrossRef]
44. G. Zou, Z. Wang, Y. Liu, et al., “Deep learning-enabled image content-adaptive field sequential color LCDs with mini-LED backlight,” Opt. Express 30(12), 21044–21064 (2022). [CrossRef]
45. E.-L. Hsiang, Q. Yang, Z. He, et al., “Halo effect in high-dynamic-range mini-LED backlit LCDs,” Opt. Express 28(24), 36822–36837 (2020). [CrossRef]
46. S. J. Song, Y. I. Kim, J. Bae, et al., “Deep-learning-based pixel compensation algorithm for local dimming liquid crystal displays of quantum-dot backlights,” Opt. Express 27(11), 15907–15917 (2019). [CrossRef]
47. V. Jain and S. Seung, “Natural image denoising with convolutional networks,” Advances in neural information processing systems 21, 769–776 (2008).
48. L. Gondara, “Medical image denoising using convolutional denoising autoencoders,” in 16th International Conference on Data Mining Workshops pp. 241–246 (IEEE, 2016).
49. M. Nishio, C. Nagashima, S. Hirabayashi, et al., “Convolutional auto-encoder for image denoising of ultra-low-dose CT,” Heliyon 3(8), e00393 (2017). [CrossRef]
50. V. Turchenko, E. Chalmers, and A. Luczak, “A deep convolutional auto-encoder with pooling-unpooling layers in caffe,” arXiv, arXiv:1701.04949 (2017). [CrossRef]
51. Liqiong Liu, Rui Deng, and Lian-Kuan Chen, “47-kbit/s RGB-LED-based optical camera communication based on 2D-CNN and XOR-based data loss compensation,” Opt. Express 27(23), 33840–33846 (2019). [CrossRef]
52. W. C. Wang, C. W. Chow, Chen, et al., “Beacon jointed packet reconstruction scheme for mobile-phone based visible light communications using rolling shutter,” IEEE Photonics J. 9(6), 1–6 (2017). [CrossRef]
53. Y. Yan, Y. Xiao, J. Cai, et al., “Quantum-dot color wheel for projection displays,” Optica 10(11), 1559–1566 (2023). [CrossRef]
54. S. Mei, X. Liu, W. Zhang, et al., “High-bandwidth white-light system combining a micro-LED with perovskite quantum dots for visible light communication,” ACS Appl. Mater. Interfaces 10(6), 5641–5648 (2018). [CrossRef]
55. M. H. Memon, H. Yu, H. Jia, et al., “Quantum dots integrated deep-ultraviolet micro-LED array toward solar-blind and visible light dual-band optical communication,” IEEE Electron Device Lett. 44(3), 472–475 (2023). [CrossRef]
