1. Introduction

Oxygen saturation (SO₂) is a measure of the relative concentration of oxygenated hemoglobin with respect to the total amount of hemoglobin. The human body tissue demands and auto-regulates a precise and specific balance of oxygen content within blood circulation throughout various organs and tissue types. Oxygen saturation, including peripheral SO₂ (SpO₂), is one of key physiological indices commonly used to indicate the physical and medical conditions of a person. For instance, an abnormal level of oxygen saturation is often associated with severe medical condition, e.g., hypoxia, chronic obstructive pulmonary disease [1] and obstructive sleep apnea [2]. More importantly, in the current devastating COVID-19 pandemic, SpO₂ is a vital parameter to monitor as its decrease may reflect a compromised oxygen intake through the respiratory system, thus be alarming for a suspected infection of the coronavirus [3]. Therefore, regular measurement and monitoring of SpO₂ is of great importance for at-home health monitoring and clinical practices in dealing with various medical conditions and the current pandemic of COVID-19.

Current clinical gold standard for SpO₂ measurement is blood gas analysis with invasive blood sampling [4,5]. It is not until the early 1980s [6]can SpO₂ be continuously measured non-invasively with a contact-mode light-based pulse oximetry, which has revolutionized the way we monitor the blood oxygenation in clinical practice as well as in-hospital monitoring. The detecting principle of pulse oximetry is based on the distinct absorption difference between oxygenated (HbO₂) and deoxygenated hemoglobin (Hb) in the visible and near infrared wavelength range. Such basic fact is being continually utilized in the popular development of remote, non-contact measurements of SpO₂ using imaging photoplethysmography (iPPG) [7–10], by leveraging the advances in area array sensors, for example CCD cameras. The ability to remotely assess SpO₂ information could benefit both clinical and research efforts such as in the intensive care unit [11] and sleep studies [12]. In 2005, Wieringa [13] first introduced the idea of contactless imaging of arterial SpO₂ distribution within the body tissue by employing multiple monochromatic CCD cameras, each targeted to receive the reflectance images sensitive to specific wavelengths that are needed to deduce SpO₂. Immediately after this study, Humphreys et al. [14] presented a transmission iPPG system and demonstrated the practicality of iPPG in providing the information of SpO₂. Some system arrangement employed a set of two CCD cameras, each with a narrow band-pass filter (at 660 nm and 520 nm, respectively) [15]. In addition to the efforts of developing monochromatic camera-based devices, there are intense research activities over the recent years focusing on developing devices and algorithms for estimating SpO₂ from the RBG-channel signals provided by a color CCD camera. A Monte Carlo simulation of light transport was employed in an attempt to convert the RGB signals obtained into a relative concentration of oxygen saturation [16]. Ufuk Bal [17] used dual tree complex wavelet transform algorithm to reduce artifacts in SpO₂ calculation.

The success of almost all the prior studies was essentially based on the observed relationship of the oxygen saturation to the ratio of AC/DC ratios between two wavelengths of interest, which was derived from the Beer-Lambert law [18]. However, when using the Beer-Lambert law, only the absorbances from the chromophores within skin tissue were considered in the derivation, for example skin pigmentation, reduced and oxygenated hemoglobin. The effect of the variation caused by light scattering on the measured reflectance images was unfortunately neglected, which may cause considerable errors or misinterpretation of the measured SpO₂. According to biomedical optics, the light scattering properties of the heterogeneous skin tissue and the absorption and scattering strength of the effective blood volume would inevitably affect the appearance of reflectance images recorded by the CCD camera. The change in reflectance depends mainly on the effective blood volume, which may be determined by many physiological factors, such as local temperature, cardiac index and peripheral vasoconstriction [7,19,20]. Therefore, even under a stable illumination condition, the reflectance variation due to the changes in scattering and absorption can lead to a poor estimation of the relationship of oxygen saturation to the ratio of AC/DC ratios, giving rise to a considerable deviation of measured SpO₂ from the true value. While this complication can be minimized by the careful control of measuring environment through strictly maintaining a stable physiological condition, such approach is not practical in clinical practice and daily check-ups. Alessandro [9,21] proposed an approach of using the baseline changes in the color of the skin from one of the RGB channels to improve the estimation of oxygen saturation. However, the study did not describe how to use the color of the skin to amend the oxygen saturation results.

While camera-array sensor approach has been the main stay for the development of remote SpO₂ monitoring, there is an increasing interest in the development of smartphone-based approach simply because of its ever-growing accessibility and affordability in the community. In this regard, effort has been paid to develop pulse oximeter in which a contact light sensor is connected to smartphone-based mobile devices [22]. Mobile devices offer many advantages, such as user-friendly customer interfaces. However, the most efforts so far are based on contact-mode [23,24], requiring hardware attachments and a relative long data-recording period. This attribute makes smartphone-based contact method less attractive compared to the traditional pulse oximeter. To increase its competitiveness, an alternative approach is to utilize the color camera built in the today’s smartphone to record the reflectance images remotely, from which to deduce the SpO₂. However, as far as we are aware, such an approach has not been reported until now in the literature.

In this paper, we propose a non-contact method to estimate SpO₂ using smartphone-enabled imaging photoplethysmography. Following the relationship between ratio of ratios of the reflectance images recorded in RGB channels and oxygen saturation, we introduce a method of multiple linear regression (MLR) algorithm to minimize the effect of changes in the light scattering due to the variations of physiological conditions on the final SpO₂ estimation. We demonstrate our smartphone-based remote measurement of SpO₂ by imaging volunteers’ hands conditioned by a manual blood pressure cuff to provide oxygen saturation between 90% and 100%. In parallel, the SpO₂ of the conditioned hands is also simultaneously measured by a medical grade contact mode pulse oximeter at fingers for training the proposed multiple linear regression algorithm as well as for comparison.

2. Method

2.1 Theory

Oxygen saturation is used to indicate the extent to which hemoglobin is saturated with oxygen, which is defined as

During propagation within skin tissue, the light intensity would be progressively reduced due to the presence of absorbing chromophores, e.g., skin pigmentation, bones, the arterial and venous blood, following the Beer-Lambert law. The heartbeat would also lead to the pulsatile variation in the blood volume. Thus, the total absorbance represented at the reflectance captured by the remote cameras contains DC component and AC component. The DC component is due to the absorbance caused by venous blood, a constant amount of arterial blood and other non-pulsatile components such as skin pigmentation. And the AC component is due to the pulsatile nature of the blood volume within the light interrogated skin tissue volume.

At diastole, the diameter of the arterial vessels is minimal and therefore the absorbance due to the hemoglobin in arterial blood is minimal, leading to higher reflectance recorded by the camera (I_H). On the other hand, the arteries contain more blood during systolic phase, and therefore, the optical path length in the arteries increases. The amount of light being absorbed reaches maximum, giving rise to a minimal reflectance (I_L). According to the Beer-Lambert law, I_H and I_L could be expressed as,

In order to eliminate the influence of DC components, the ratio of absorbances at two wavelengths (defined as ratio of ratios) could be expressed as

Assuming that the optical path length $\; d$ is the same for the oxygenated hemoglobin (${d_{Hb{O_2}}}$) and reduced hemoglobin (${d_{Hb}}$) and the optical path lengths for the two wavelengths are equal, using Eqs. (2) and (3), Eq. (6) can be reduced to

In this form, the ratio of ratios (R) is not a function of the optical path length and can be derived from the oxygen saturation instead of the concentration of the hemoglobin in the blood.

Equation (7) can be rewritten in a form where SO₂ is a function of the measured and calculated ratio R,

Equation (8) is the calibration equation used for almost all the prior literatures to estimate oxygen saturation based on the ratio of ratios method. However, the derivation procedures described above are valid based on several assumptions. The most important one is that the Beer-Lambert law assumes no effect of light scattering on the images recorded by the remote CCD sensors, which is however not true in practice. It has been demonstrated that the influence of light scattering on the measurement cannot be removed by simply subtracting the DC signals [25]. In this case, the total absorbance due to whole blood volume within the interrogated tissue volume should be expressed as the sum of absorbance as described by the Beer-Lambert law (A_t) and a second term representing the changes in light scattering (ΔA_t). Thus, according to the non-linear variation of pulsatile blood volume in Eq. (6) and using Eq. (8), the SO₂ is actually the function of R and ΔR:

Fine and Weinreb indicated the influence of scattering on pulse oximeter readings [20]. They demonstrated that the ratio of ratios is a function of the effective blood volume and the concentration of hemoglobin, which relate to the subsequent $\Delta R$ and R. To account for scattering effect, Smith [26] introduced an optimal set of three wavelengths in the implementation of oximetry where two isosbestic wavelengths are used to estimate the contribution by optical scattering, and a third wavelength is used for SO₂ sensitivity. While three-wavelength oximetry may offer improved accuracy over two-wavelength oximetry by accounting for scattering, the choice of potential imaging wavebands is rather specific and in fact limited, which would not be suitable for the development of smartphone-based remote pulse oximetry.

As stated, the effective blood volume within the light interrogated tissue volume is influenced by many physiological factors such as the local temperature, cardiac index, peripheral vasoconstriction etc. In the meantime, the variations in effective blood volume due to its pulsatile nature from the heartbeats cause a change in the light scattering in addition to light absorption, which subsequently alters the appearance of light reflectance recorded by the CCD camera. This change in the reflectance images would be represented by a change in the gray value of images recorded at all the RGB channels. Therefore, $\Delta R$ may be mainly represented by the gray value variation recorded by the RGB sensors, which can be utilized to improve the SpO₂ estimation.

Incorporating the reflectance changes at the RGB channels into the SpO₂ estimation in addition to the ratio of ratios, the problem may be taken as a multifactorial process. To deal with this problem, we approximate the multifactorial process using multiple linear regression (MLR) method in which the oxygen saturation is expressed by

In this case, the determination can be achieved with a calibration process, given a set of reference measurements of the oxygen saturation, y. In the calibration process, the target sample is conditioned to provide a range of known SpO2 and in the meantime, it is sampled by the smartphone to give the measured signals at the RGB channels. This process gives rise to the training datasets. For each measurement in the training, the SpO₂ can be indicated by

Given the N measurements, the Eq. (11) can be expressed in a matrix form:

Each element in the matrix X is directly determined from the measurements at the RGB channels in the smartphone. Since ${\boldsymbol X} \in {{{\mathbb C}}^{N \times 5}}$ is not a square and invertible form when $N \ne 5$, singular value decomposition [27] can be used to calculate ${\boldsymbol \beta }$. X could thus be expressed as

Finally, the optimized ${\boldsymbol \beta }$ at minimal sum of squared errors can be obtained by

After calibrating with the training dataset, ${\boldsymbol \beta }$ is then obtained and stored for later use in the multiple linear regression of Eq. (10) for each smartphone imaging session to estimate the SpO₂ of the target sample.

2.2 Consideration of smartphone

Conventional contact-sensor based pulse oximeter generally adopts the wavelengths of 660 nm (Red) and 940 nm (Infrared) [28,29]. The purpose of current study is to use the intact and unmodified smartphone to provide the oximetry function without employing additional add-on hardware. However, the use of the standard red and infrared LEDs for illumination on a consumer smartphone is not possible without modification of the smartphone hardware. In addition, the RGB sensors in the smartphone present very low QE (Quantum Efficiency) at infrared waveband. Therefore, the selection of wavelengths for the smartphone-based non-contact oximetry must consider the characteristics of the built-in RGB sensors and the use of built-in flashlight for illumination.

When recording the reflectance images emerging at the sample illuminated by the built-in flashlight, the end-product would be a combined effect among the power spectral distribution of the flashlight, the spectral sensitivity of each RGB sensor at the smartphone camera and the absorption spectra of the chromophores of interest presented in the sample. In order to simulate the response of HbO₂ and Hb at specific RGB channels of smartphone camera with built-in flashlight illumination, we take an iPhone X from Apple Inc as an example. Figure 1(a) shows the spectral power distribution of its flashlight across the wavelength range from 400nm to 700nm. Figure 1(b) illustrates the spectral response curves of the red, green and blue channels of its built-in camera, respectively. Figure 1(c) gives the absorption spectra of the HbO₂ and Hb, respectively, where HbO₂ and Hb has similar absorption coefficient at green channel whereas they exhibit a relatively large difference between blue and red channels. Therefore, the end-product of the camera that records the reflectance images due to the absorbance of HbO₂ or Hb is the product of the three spectral curves presented in Fig. 1 (a)∼(c), respectively. The result of which is shown in Fig. 1(d).

 

Fig. 1. selection of wavelengths for oxygen saturation calculation. (a) Spectral power distribution of the smartphone flashlight used in this study. (b) Relative spectral sensitivity of Red, Green and Blue channels of the smartphone camera. (c) Absorption spectra of oxyhemoglobin and hemoglobin. (d) Combined response curves of flashlight, hemoglobin and camera RGB sensors when recording the reflectance images.

Download Full Size | PDF

The wavelength selection of dual-wavelength oximeter should obey the following requirements [30]. First, the absorption coefficients of HbO₂ and Hb should differ greatly at one wavelength. Second, there should be approximately equal absorption coefficient in terms of HbO₂ and Hb at the other wavelength. In this case, the difference of integral value between HbO₂ and Hb at Red channel is about 2×10⁴, whereas the corresponding difference at Green channel is about 1×10³. Therefore, the red channel would approximately satisfy the first requirement, whereas the green channel meets the 2^nd requirement. In addition, because the flashlight illumination presents relatively low spectral power intensity at blue channel (425∼500nm), it inevitably exhibits relatively low signal to noise ratio (SNR), thus should be avoided for the evaluation of the ratio of ratios. Consequently, the reflectance images recorded at green and red channels are chosen for calculating the ratio of ratios, which, together with the averaged gray values obtained at the RGB channels, are used in the MLR algorithm (Section 2.1) to estimate the oxygen saturation of the target sample.

2.3 Experimental setup

Figure 2 illustrates a schematic diagram of our experimental setup used to provide remote measurement of SpO₂ at the dorsal site of the volunteer’s hand, consisting of a smartphone (iPhone X, Apple Inc, USA), a reference contact-mode sensor-based pulse oximeter (PC-66H Handheld Pulse Oximeter, CMI Health, USA), and a manual blood pressure cuff (Aneroid sphygmomanometer, ParaMed, Canada). The video reflectance images from the skin surface illuminated by the built-in flashlight in the iPhone were recorded by the built-in rear camera. The iPhone X was used for demonstration purpose only, but any other types of smartphones can be used. The distance between the smartphone camera and the dorsal skin surface of the volunteer’s hand was kept at about 30 cm. The frame rate for video recording was set at 60 frames per second. We also placed a polarizer sheet at the output of flashlight and an analyzer sheet in front of the camera. The polarizer and analyzer were orthogonally oriented to minimize the specular reflection at the skin surface so that the reflection signal from within the tissue sample could be maximized.

 

Fig. 2. Schematic of calibrating oxygen saturation with smartphone and commercial oximeter.

Download Full Size | PDF

The medical grade contact-mode sensor-based pulse oximeter was used in parallel with the camera video recording, which provided the reference SpO₂ measurements that were used in the training process of the MLR algorithm to obtain the needed set of weighting coefficients (i.e., Eq.15). The pulse oximeter was also used for the comparison of the measured SpO2 derived from the conventional ratio of ratio method and the proposed MLR method here.

The standard medical grade manual pressure cuff was used to condition the oxygen saturation in the volunteer’s hand to provide the SpO₂ values ranging from 90% to 100%, to which the contact-mode pulse oximeter and the remote-mode smartphone measure. It should be noted that Fig. 2 illustrates the calibration (or training) process of smartphone based on a commercial oximeter. During measuring step, only smartphone is used.

For experimental demonstration, five healthy volunteers were enrolled in this study, with subject information shown in Table 1 including the skin color (indicated by Fitzpatrick scale [31]). All measurements were performed in a standard lab environment. All room windows were covered so as to minimize the amount of stray light from any source other than the flashlight of smartphone. Volunteers were instructed to sit comfortably and quietly on a chair. The experiment began after allowing 10-min accommodation of the lab environment. The manual blood pressure cuff was placed on the left arm that was finely controlled to realize the SpO2 at the hand ranging from 90% to 100%. The reference SpO₂ data was recorded using the contact-mode pulse oximeter positioned on the little finger, with a sampling time of 2-4 seconds and a reported accuracy of 2∼3% between 70% and 100% of the SpO₂. In parallel, the video reflectance images from skin surface of the volunteer were recorded by the smartphone.

Table 1. Demographic information of the volunteers including skin color type (Fitzpatrick scale)

View Table | View all tables in this article

Multiple trials (n = 3 to 5 depending on conditions during data recording) were conducted on each subject following the same procedure as described above. Each trial consisted of multiple levels of SpO₂ from 90% to 100% at the fingertip as measured by the pulse oximeter and controlled by the manual blood pressure cuff on the upper arm. At each level of SpO₂, the video images recorded by the iPhone were approximately 10-15 seconds in duration, but truncated to multiple 2-second non-overlapping segments (i.e., dataset) for later processing.

The experiments described in this study followed protocols reviewed and approved by the Institutional Review Board of Medical Sciences Subcommittee at the University of Washington, Seattle. The tenets of the Declaration of Helsinki and Health Insurance Portability and Accountability Act were followed. Informed consent forms were obtained from all subjects before participation.

2.4 Analyses of the recorded data

To obtain more accurate estimation of oxygen saturation, valid data should obey the following two criterions. Firstly, during data recording, the reference oxygen saturation read at the pulse oximeter should maintain stable for more than 10 seconds. Secondly, the AC components of iPPG signal should not excess three times the average baseline of AC component. The datasets after passed this quality check were made available for further processing and evaluation. Listed in Table 2 are the numbers of valid datasets from all the volunteers participated in this study, with each dataset corresponding to its unique SpO₂ reading by the pulse oximeter in parallel.

Table 2. Number of valid datasets from the participating volunteers

View Table | View all tables in this article

In the data processing to extract the time series of reflectance signals, a region of interest (ROI) was first selected in the video image, which covered most of the hand backside, as shown in Fig. 3(a). At each time frame, a mean value was obtained through averaging all the pixel values within the ROI at each color channel to provide the time varying reflectance signals, representing red, green and blue channels, respectively (as shown in Fig. 3(b), where red and green channels are given). The time series of reflectance signals exhibited low-frequency respiration signal, high-frequency heartbeat signal and noises. The fast Fourier transform (FFT) result of the signal from Green channel in Fig. 3(b) is shown in Fig. 3(c), where the heartbeat frequency around 1 Hz is clearly contrasted. A band-pass filter with a passband between the physiologically relevant frequency band of [0.5-5 Hz] was employed to acquire AC components (Fig. 2(d)). Similarly, the DC components were computed by passing time-traced signals through a low-pass filter with a cutoff frequency of 0.3 Hz. At this point, the ratio of ratios between red and green channels can then be calculated using Eq. (6).

 

Fig. 3. The extraction process of iPPG signals from a video sequence recorded by the remote smartphone at 60 fps. (a) A video sequence is illustrated along the time axis where the ROI is marked with dotted line box for evaluation. (b) Evolution of the spatially-averaged values within the ROI, i.e., the time series of reflectance signal, during the recorded video at Red (top) and Green (bottom) channels, respectively. (c) The Fourier spectra of the time series signal at the green channel, where it clearly shows the heartbeat frequency around 1 Hz. Band-pass filter (gray-marked between 0.5∼5 Hz) is employed to extract AC components from the time series reflectance signals. (d) The AC components of Red (top) and Green (bottom) channels.

Download Full Size | PDF

To estimate the mean gray values from each RGB channel in Eq. (10) or Eq. (11), the 2-second videos were volume-averaged within the ROI to represent the reflectance values at the skin surface recorded by the red-, green- and blue-array sensors in the built-in smartphone camera when illuminated by the flashlight.

To obtain the weighting coefficients of Eq. (15), about 50% of the datasets captured from the first 3 volunteers were randomly selected for the calibration process of the multiple linear regression algorithm. All the rest datasets were used to test and validate the algorithm to provide the SpO₂ readings through comparing these readings with those provided by the pulse oximeter using the well-known Bland and Altman method [32]. The total numbers of datasets used for training and cross-validation are also given in Table 2 for information. Note that the datasets from the volunteers 4 and 5 were not participated in the training.

3. Results

Before presenting the results of the proposed MLR method to measure the SpO₂, we first demonstrate the relationships between the reference SpO₂ and the variables that were evaluated from the smartphone recordings, including the ratio of ratios and the reflectance values as exhibited in red, green and blue channels. The dependence of the ratio of ratios on the reference SpO₂ from two representative volunteers are shown in Fig. 4, where the data shown were from three repeated trials. As expected, the ratio of ratios increases gradually when the oxygen saturation is decreased. However, volunteer 1 (Fig. 4(a)) has better coefficient of determination (with r² = 0.87) whereas volunteer 2 (Fig. 4(b)) presents worse correlation (with r² = 0.52) between the ratio of ratios and the oxygen saturation. The relatively poor linearity shown in Fig. 4 (b) may be caused by the variation of effective blood volume during data recording.

 

Fig. 4. The relationship between the reference SpO₂ and the calculated ratio of ratios. Results were obtained from three repeated trial tests. (a) volunteer 1. (b) volunteer 2.

Download Full Size | PDF

As discussed, the effective blood volume within the light interrogated tissue volume would directly impact the reflectance emerging at the tissue surface, represented as the change in the gray value of the images recorded by the smartphone camera. The relationships between the mean gray values from green, red and blue channels and the ratio of ratios were analyzed. The datasets from the first trial test shown in Fig. 4 were processed to show these relationships. The results of which are shown in Fig. 5.

 

Fig. 5. The relationship between the reference SpO₂ and the variables extracted from the smartphone recordings. The results shown are from the first trial test in Fig. 4. (a) and (c) The change in the ratio of ratios with the variation of the reference oxygen saturation. The dotted lines represent the linear fitting curves with r² values as shown. (b) and (d) The changes in the mean gray values of green, red and blue channels with the variation of the reference oxygen saturation. Linear fitted lines of three channels are also shown with dotted lines. (a) and (b) were from volunteer 1. (c) and (d) were from volunteer 2.

Download Full Size | PDF

The results from volunteer 1 presented an excellent coefficient of determination (r² = 0.98) between the ratio of ratios and the reference oxygen saturation (Fig. 5(a)). The same volunteer also exhibited a good correlation between the mean gray values at the green, red and blue channels and the oxygen saturation, albeit with an opposite trend (Fig. 5(b)). However, a much variation in the results was seen in the volunteer 2 with a relatively poorer correlation between reference oxygen saturation and the ratio of ratios (r² = 0.57) (Fig. 5(c)). While the linear relationships between the oxygen saturation and the mean gray values at the RGB channels exist (Fig. 5(d)), the coefficients of determination were relatively worse than that of volunteer 1. We tabulated the coefficients of determination of the ratio of ratios, green channel, red channel and blue channel from 4 trial tests in Table 3, where it indicates that when the ratio of ratios presents a good coefficient of determination, the relationships between the SpO₂ and the mean gray values at the green, red and blue channels also show a good linearity, and vice versa.

Table 3. Coefficients of determination resulted from linear fitting

View Table | View all tables in this article

After observing the behaviors of the extracted variables from the smartphone recordings with the change in the SpO₂, we then test and demonstrate the improvement that the proposed MLR algorithm would provide to estimate the SpO₂. While there were more datasets available for training the MLR algorithm to obtain the needed weighting coefficients as expressed in Eq. (15), about 50% of valid datasets from the volunteers 1, 2 and 3 were randomly selected for the training and calibration. The rest datasets were used to cross-validate the MLR algorithm (See Table 2 for information).

After obtaining weighting coefficients from the training process, we first tested the MLR algorithm for the volunteer 2 (i.e., intraclass testing because volunteer 2 was participated in the training). The Bland-Altman plots of the reference SpO₂ readings from the pulse oximeter and the estimated SpO₂ values from the ratio of ratios method and the proposed MLR method are shown in Fig. 6. In this case, the measurement error of SpO₂ is 0 ± 2.38% (μ ± 1.96σ) for the ratio of ratios method, showing a relatively big deviation relative to the reference oximeter (Fig. 6(a)). However, the accuracy of estimation based on MLR achieves 0 ± 1.07% (Fig. 6(b)), indicating an excellent 55% improvement compared to that of the ratio of ratios method.

 

Fig. 6. The test of the trained MLR algorithm on an intraclass scenario of Volunteer 2 against the reference oximeter readings. (a) Bland-Altman plot of estimated SpO₂ based on linear regression with the ratio of ratios method against the reference oximeter reading. The blue line and two red dotted lines at either side show μ ± 1.96σ = 0 ± 2.38. (b) Bland-Altman plot of estimated SpO₂ based on the MLR algorithm. The blue line and two red dotted lines at either side show μ ± 1.96σ = 0 ± 1.07.

Download Full Size | PDF

Next, we tested the MLR algorithm on the SpO₂ estimation on all the available validating datasets (Table 2), including those from the volunteers that were not included in the training. In this case, the cross-validation of leave-one-out technique [33,34] was used. The resulted SpO₂ compared with those of reference oximeter are summarized in Fig. 7, for the conventional ratio of ratios method (Fig.7a) and the proposed MLR algorithm (Fig. 7(b)). The overall estimated error between the conventional ratio of ratios method and the reference oximeter was μ ± 1.96σ = 0.008 ± 2.008%, whereas that for the MLR algorithm was μ ± 1.96σ = 0.029 ± 1.141%, indicating a 43% improvement for the proposed method. A degradation in the performance was observed compared to the intraclass testing, probably because of the mixed skin colors across the volunteers.

 

Fig. 7. The test of the trained MLR algorithm on the datasets available from all the volunteers against the reference oximeter readings. (a) Bland-Altman plot of estimated SpO₂ based on linear regression with the ratio of ratios method against the reference oximeter reading. The blue line and two red dotted lines at either side show μ ± 1.96σ = 0.008 ± 2.008. (b) Bland-Altman plot of estimated SpO₂ based on the MLR algorithm. The blue line and two red dotted lines at either side show μ ± 1.96σ = 0.029 ± 1.141.

Download Full Size | PDF

4. Discussion

The results presented above show that compared to the conventional ratio of ratios method, the multiple linear regression (MLR) algorithm results in much better estimation of the SpO₂, delivering a very good agreement with the readings from a commercial medical grade pulse oximeter sensor. This improvement is attributed to the additional consideration of the changes in light scattering and absorption that are induced by the effective blood volume within the light interrogated skin tissue, giving rise to the changes in the reflectance images recorded at the RGB channels. We note that when dealing with the ratio of ratios method, some literature suggested to carefully select the regions of interest (ROI) by calculating signal-to-noise ratios for each ROI [20] so that the best ROI was selected for improved estimation of the oxygenation saturation. Even with this complicated procedure, the ratio of ratios method still exhibits relatively poor coefficient of determination with oxygen saturation, affecting the accuracy of the final estimated SpO₂. In this study, we found that the mean gray values from three RGB channels of smartphone camera are correlated with the observed oxygen saturations. Taking these relationships as additional variables into the consideration for estimating SpO₂, the accuracy is demonstrated with great improvement using the proposed MLR algorithm. When compared to that of the ratio of ratios method, the measurement error for intraclass testing is reduced from 0 ± 2.38(μ ± 1.96σ) to 0 ± 1.07, indicating a ∼55% improvement. The overall improvement by including the volunteers not participated in the training is observed at 43%. The second advantage of the MLR method is a short sampling period. Non-contact detection of oxygen saturation has always required longer sampling time to acquire the datasets with good SNR. Most prior remote algorithms require 10∼12 seconds sampling time [15,17,21]. However, the sampling time-period of the proposed MLR algorithm in this paper was reduced to 2 seconds, which is similar to the fastest sampling time of commercial pulse oximeters. This merit is important because it could have great potential to improve the practicality of the non-contact smartphone-based approach to estimate the oxygen saturation in human subjects. Thirdly, because of the attributes of remote and non-contact measurement, the proposed MLR method delivers additional advantages of rapid survey and avoiding cross infection, which is particularly important for the current epidemic outbreak of COVID-19.

The reference standard used in this study was a commercial and medical grade pulse oximeter rather than blood-gas analysis. This could certainly introduce possible errors when assessing the accuracy of this new algorithm because the algorithm relies on the absolute accuracy of the pulse oximeter. If necessary, the MLR algorithm can be calibrated with the gold standard blood-gas analysis for improved accuracy. On the other hand, the infrared light used in pulse oximeter has deeper penetration depth relative to the visible light employed in remote smartphone method. Therefore, the pulse oximeter provides the arterial oxygen saturation (SaO₂) measurement, whereas the remote smartphone method is more likely to measure the peripheral tissue oxygen saturation (StO₂) [35]. Although Benaron et al. [36] indicated that SaO₂ and StO₂ are correlated over limited ranges, no study has been conducted on humans so far. Large fluctuation on the ratio of ratios may be explained by the fact that, venous and capillary oxygen saturation would more likely be influenced by ambient variation relative to arterial oxygen saturation.

Considering the ever-growing utility of the smartphones in the community, mobile health (mHealth) is playing an increasingly important role in nowadays medical market, even for personal amusements. Heart/respiration rate monitoring [37], blood pressure surveillance [38], morphological features analysis and hemodynamics monitoring [39,40] are all successful applications of mHealth to date. More opportunities of medical applications by leveraging the advances in the development of smartphones would help improve the medical environment in undeveloped areas, making it possible for remote monitoring of health conditions and realizing household health monitoring economically. Integration of accurate non-contact oxygen saturation measurement in the smartphone would thus accelerate the developments of mHealth.

5. Conclusion

We have demonstrated the feasibility of using intact and unmodified smartphone to realize the remote measurement of peripheral oxygenation saturation (SpO2). A multiple linear regression (MLR) algorithm has been proposed to accommodate the changes in light scattering and absorption due to the effective blood volume within the light interrogated skin tissue volume, for which the changes in the reflectance images recorded at RGB channels in the smartphone were used as a surrogate in the formulation of the algorithm. We have shown that the proposed MLR algorithm delivers a significant improvement (55% improvement for intraclass testing, and 43% for overall testing) in the estimation of SpO2 when compared with that of the conventional ratio of ratios method. Since the results were demonstrated with the built-in color camera and the built-in flashlight within a smartphone with a sampling time duration similar to the standard pulse oximeter, we expect this development to represent a significant advance to the current effort for developing accessible and cost-effective mHealth, particularly benefiting the populations living in the underserved and rural areas and in the developing countries.