Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Skin tissue perfusion mapping triggered by an audio-(de)modulated reference signal

Open Access Open Access

Abstract

Spatial mapping of skin perfusion provides essential information about physiological processes that are often hidden from the eyes of the examining physician. The perfusion map quality depends on several key factors, such as the camera system type, frame rate, sensitivity, or signal-to-noise ratio. When investigating physiological parameters, the reference signal allows for increasing the spatial resolution of the photoplethysmography imaging (PPGI) system. On the other hand, it increases the system complexity and the synchronization prerequisites. Our solution is a hardware device that modulates the reference biosignal into the audio frequency band. This signal is connected to the mic input of a digital camera or a smartphone, enabling the transformation of such a device into a PPGI measurement system even in the case of compressed video recording using lock-in amplification technique. It also brings the possibility of synchronous recording of PPGI and another reference signal such as conventional photoplethysmogram or electrocardiogram.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Perfusion of subcutaneous tissue is a valuable source of information about the physiological or pathophysiological state of the human body. On the one hand, perfusion changes are related to the cyclic activity of the heart muscle (heart rate related). However, on the other hand, they contain information from other associated processes, for example, regulating the autonomic nervous system [13]. In the past, and to a large extent still today, the most widely used method for investigating skin perfusion is contact photoplethysmography, which dates back to 1937 [4]. Its improved non-contact option is photoplethysmography imaging, introduced in 2000 by a research group around Prof Blazek [5]. This method had its limitations initially because the types of sensors used in camera systems either had low sensitivity or required external cooling to counteract the presence of noise. However, with the development of technology, more and more high quality and sensitive sensors are coming on the market, which can also be found in consumer electronics such as smartphones or digital cameras. These devices make it possible to record cardiac activity as long as a sufficient number of image pixels are averaged, which naturally increases the signal-to-noise ratio (SNR) [6]. However, this reduces the spatial resolution of maps created to investigate the distribution of perfusion changes.

The second problem is image compression when creating video or image data, reducing the quality of the detected PPGI signal [7]. Therefore, a number of algorithms have been developed that increase the SNR. Here we can mention the classical averaging of a certain number of pixels [8], PCA [9], ICA [10], CHROM [11], PBV [12], 2SR [13], POS algorithm [14], lock-in amplification (LIA) method [15]. The last-mentioned LIA method is a very effective tool that significantly increases the SNR. However, it requires a reference signal for its operation. The reference signal, in our case, means a diagnostic method capable of recording the biological activity of the organism associated with cardiac activity. Here we can include contact ECG or PPG. Such devices vary from complex laboratory systems to simple single-chip devices based on an analogue-front end. A frequent technical problem of sensing subcutaneous perfusion with PPGI while simultaneously acquiring other types of biosignals tends to be their synchronisation.

Our motivation is to develop a device that would link a PPGI system based on affordable imaging technology, such as a smartphone or a digital camera, with a reference signal. For example, if such a device has a microphone input, we can use this input to record another reference biosignal synchronously. However, for correct signal processing, it is first necessary to transform the signal into the audio frequency band. One option is to use a voltage to frequency converter [16]. Another way of modulation can be, for example, the switching modulation technique [17]. If the requirement is not to preserve the signal contours, then it is possible to acquire the signal with an adapted amplitude directly to the device's microphone input, e.g., in HRV detection from ECG signal [18]. Another way of transmitting the signal to a superior processing unit is to modulate it into the radio frequency band, enabling its wireless transmission [19]. Our chosen method of signal pre-processing was amplitude modulation (AM), which preserves all the characteristics of the sensed biosignal. This method of digitizing the signal using the microphone input of a PC sound card has been presented, e.g. by [20], where the analogue form of AM was used. An alternative method may be modulation using square wave signals [21].

The device's hardware should contain as few components as possible to make it affordable to a wide range of the public, from physicians to scientific research facilities. The ideal choice is a sufficiently powerful microcontroller that provides digitisation of the selected biosignal type, subsequent amplitude modulation into the audio band and transformation of the modulated signal back into the analogue form using a D/A converter. Such a device in combination with a camera can be used directly in the outpatient clinics of medical specialists, e.g. in allergology or neurology, or specialised departments dealing with the study of processes related to the regulation of the autonomic nervous system.

2. Materials and methods

2.1 Measurement protocol

In setting up the protocol for measurement, we were inspired by the work [22]. This work has evaluated cutaneous allergic reactions, focusing on their haemodynamic quantification using PPGI. The protocol involved four sequences of two minutes in length, during which a camera recording of the forearm was made. The time interval between each sequence was four minutes.

To induce a change in subcutaneous perfusion, we used two types of creams – cream 1 (C1), whose active ingredient is based on standardised semi-solid extract of capsicum (ratio 4-7/1), and cream 2 (C2), in which the two main vasodilators nonivamide and nicoboxil are present. Both creams are commonly available to the public without the need for a prescription. All experiments were conducted as self-experiments of the authors.

In our case, we also chose a time step of four minutes. However, we reduced the recording time for each sequence to 90 seconds. Thus, within the experiment, four PPGI sequences of 90 s duration were taken: shortly before application -> 4 min after application -> 8 min after application ->12 min after application -> 16 min after application -> 20 min after application -> 24 min after application. It means we recorded seven stages (7 × 90 s = 630 s in total).

Figure 1 shows the measurement setup, showing the region of interest - the forearm of the subject under investigation. Perfusion changes were recorded in two ways. The first method employed the PPGI method using a Canon 550D DSLR camera (Canon Inc., Tokyo, Japan). This camera had its firmware modified using Magic Lantern of version magiclantern-Nightly.2018Jul03.550D109, which allowed raw video recording in MLV format with a bit depth of 14-bits. The video sampling rate was set to 15 Hz, while the audio sampling rate was left at the default value of 44.1 kHz. Simultaneously with the camera recording, a reference signal was acquired using a contact PPG sensor (SparkFun Electronics, Niwot, Colorado, USA) placed on the index finger of the subject's forearm in the case of the first experiment. In the second case, we replaced this reference with an ECG analogue front-end based on AD8232 (Analog Devices, Inc., Wilmington, Massachusetts, USA) again on a prototype board from (SparkFun Electronics, Niwot, Colorado, USA). The ECG signal was recorded from the Einthoven lead I.

 figure: Fig. 1.

Fig. 1. Rendering of the measurement setup.

Download Full Size | PDF

To record the PPGI signal and the reference signal synchronously, we developed special hardware that allowed the modulation of the sensed biosignal into an audio band. Subsequently, we were able to connect the signal to the microphone input of the DSLR camera.

2.2 Hardware to modulate biosignal into the audio band

The main idea of amplitude modulation of the sensed biosignal into the audio frequency band is the possibility to synchronously record it with the video stream. It can then be used either as a reference or as different biosignal carrying information about different aspects of the physiological process, which is interconnected with the video-monitored biological process. In our case, the video-monitored process is tissue perfusion. Such a modulated signal can be connected to a commercial device's input (e.g. camera) with a microphone input, providing a valuable extension of its functionality. As synchronization of audio and video is key in the intended purpose of the device (e.g. recording lip-sync videos), we can expect the synchronization to be accurate. Thus, the camera can sense the perfusion of subcutaneous tissue and at the same time have a synchronised reference signal available.

The hardware design was based on the technical requirements for the sensed biosignal and its modulation to the audio band. It was necessary to ensure sufficient separation between the spectra of the sensed and carrier signals. If we consider, for example, a standard ECG, most of the diagnostic information is below 100 Hz. On the other hand, its high-frequency components can approach 250 Hz in infants [23]. To ensure sufficient spacing of the spectra, we considered a carrier frequency of 5 kHz in the design. At the same time, we emphasised that the device should be minimalistic with as few components as possible. The dsPIC33ck256mp202 (Microchip Technology Inc., Chandler, Arizona, USA) digital signal microcontroller (MCU) fulfilled all requirements. Its architecture is 16-bit, allowing for more complex calculations and is optimised for fixed-point operations that speed up program execution. This MCU also features a built-in 12-bit SAR A/D converter, a 12-bit D/A converter, and a pair of integrated operational amplifiers (OAs). This combination predisposes it to handle the operations required for analogue and digital signal processing before the modulated signal is fed to the microphone input of, in our case, a DSLR camera.

Thus, the microcontroller provides a series of steps necessary for modulating the signal into the audio band as shown in Fig. 2. The first step is digitising the analogue pre-processed biosignal (PPG or ECG). Amplitude modulation is then performed using a carrier frequency of 5 kHz. Mathematically, we can express the amplitude modulation of a biological signal ${{s}_{\textrm{bio}}}(t)$ and a carrier signal ${{s}_\textrm{c}}(t ) = C\textrm{sin} \omega _\textrm{c}t$ with frequency ${\mathrm{\omega }_\textrm{c}}$ as

$${s_{\textrm{mod}}}(t )= \; \left( {1 + \frac{{{s_{\textrm{bio}}}(t )}}{C}} \right)\sin {\omega _\textrm{c}}t,$$
where ${{s}_{\textrm{mod}}}(t)$ is the modulated signal and C is the amplitude of the carrier signal.

 figure: Fig. 2.

Fig. 2. Reference biosignal processing chain.

Download Full Size | PDF

This modulated signal is again converted into analogue form by a D/A converter. However, before the modulated signal is fed to the mic input of the DSLR camera, it is necessary to adjust the signal's AC and DC voltage level to prevent damage to this input. The signal is attenuated by an inverting amplifier and impedance separated from the microphone input via a voltage follower. The built-in operational amplifiers do not allow signal processing with a small amplitude, and therefore the modulated signal has been raised to a specific DC level (see Eq. (2)), which, however, does not pose a danger to the camera's mic input. The analogue processing using OAs is shown in Fig. 3 and the realized prototype can be seen in Fig. 4.

$$\begin{array}{c} {{V_{\textrm{out}}}\; = \; - \frac{{{R_2}}}{{{R_1}}}{V_{\textrm{in}}} + \left( {1 + \; \frac{{{R_2}}}{{{R_1}}}} \right){V_{\textrm{cc}}}\frac{{{R_4}}}{{{R_3} + {R_4}}}}\\ {{V_{\textrm{out}}}\; \cong \; 0.338 \cdot {V_{\textrm{cc}}}\; - \; 0.015 \cdot {V_{\textrm{in}}}} \end{array}, $$

 figure: Fig. 3.

Fig. 3. Signal conditioning before connecting to the mic input.

Download Full Size | PDF

 figure: Fig. 4.

Fig. 4. The realized device prototype.

Download Full Size | PDF

2.3 Demodulation technique

To demodulate the signal, complex demodulation is used. Let ${{s}_{\textrm{mod}}}[n ]$ be the discretized version of the modulated continuous signal ${{s}_{\textrm{mod}}}\textrm{(t)}$ with the discrete time n and the sampling frequency $f_{\textrm{samp}}$. First, the actual carrier frequency $f{\mathrm{^{\prime}}_\textrm{C}}$ is determined from ${{s}_{\textrm{mod}}}[n ]$ via Fast Fourier Transform (FFT) and peak detection. Next, the discrete complex demodulation signal is synthetized,

$${\textrm{s}}{\mathrm{^{\prime}}_{\textrm{comp}}}[n ]\textrm{ = exp}\left( {\textrm{j} \cdot n \cdot \mathrm{2\pi }\frac{{f{\mathrm{^{\prime}}_\textrm{C}}}}{{{f_{\textrm{samp}}}}}} \right),$$
where $\textrm{j}$ is the complex unit. The complex intermediate signal is obtained via sample-wise multiplication,
$${s}{\mathrm{^{\prime}}_{\textrm{interm}}}[n ]= {s}{\mathrm{^{\prime}}_{\textrm{comp}}}[n ]\cdot {{s}_{\textrm{mod}}}[n ].$$

To obtain a real-value signal, the absolute value is calculated,

$${{s}_{\textrm{interm}}}[n ]= |{{s}{\mathrm{^{\prime}}_{\textrm{interm}}}[n ]} |. $$

Note that ${{s}_{\textrm{interm}}}$ is positive and therefore contains a DC offset as well as unwanted high-frequency components. Since the DC offset is of no concern in this application, a 4th-order Butterworth lowpass filter with a cut-off-frequency of 100 Hz is applied to ${{s}_{\textrm{interm}}}$ to obtain ${{s}_{\textrm{demod}}}$.

2.4 Lock-in amplification technique

Lock-in amplification is an established technique for increasing the signal-to-noise ratio (SNR) in noisy signals. It is mainly used to measure signals with small amplitude and considerable noise, assuming that an external reference signal is synchronously recorded that is not affected by the noise. This technique aims to compress the selected signal band into a narrow frequency band by amplifying it while suppressing the other frequency components of the signal [2426].

The process itself consists of two steps: modulation and demodulation. Modulation describes the influence of, for example, environmental noise or motion artefacts on the signal. Demodulation describes the suppression of a useless signal by a reference signal. Subsequent filtering yields a DC component (slowly varying over time) corresponding to the amplitude of the output signal, which is in phase with the input signal [25].

Let's start from the equation for the input signal,

$${V_{\textrm{in}}}(t )= A\cos ({2\mathrm{\pi }{f_{\textrm{s}}}t + \varphi } ),$$
where ${A}$ is the amplitude of the input signal, $f_\textrm{s}$ is its frequency and $\varphi $ is the phase.

For reference signals such as $V_{\textrm{refX}}(t)$ and $V_{\textrm{refY}}(t)$ follows that they are phase shifted by $\frac{\mathrm{\pi }}{\textrm{2}}$,

$$\begin{array}{c} {{V_{\textrm{refX}}}(t )= \cos({2\mathrm{\pi }{f_\textrm{R}}t} ),}\\ {{V_{\textrm{refY}}}(t )={-} \sin ({2\mathrm{\pi }{f_\textrm{R}}t} ).} \end{array}$$

At the same time, we assume goniometric relations,

$$\begin{array}{c} {\cos(a )\cdot \cos(b )= \frac{1}{2}[{\cos({a + b} )+ \cos({a - b} )} ],}\\ {\cos (a )\cdot \sin (b )= \frac{1}{2}[{\sin ({a + b} )- \sin ({a - b} )} ].} \end{array}$$

Then, the products of the input signal ${V_{\textrm{in}}}$ and the reference signals ${V_{\textrm{refX}}}$ and ${V_{\textrm{refY}}}$ are

$$\begin{array}{c} {{V_\textrm{X}}(t )= {V_{\textrm{in}}}(t )\cdot {V_{\textrm{refX}}}(t )= \frac{1}{2}A\{{\cos [{2\pi ({{f_{s}} - {f_{s}}} )t + \varphi } ] {\; + \; \textrm{cos}[{2\pi ({{f_{s}} + {f_\textrm{R}}} )t + \varphi } ]} \},} }\\ {{V_\textrm{Y}}(t )= {V_{\textrm{in}}}(t )\cdot {V_{\textrm{refY}}}(t )= \frac{1}{2}A\{{\sin [{2\pi ({{f_{s}} - {f_\textrm{R}}} )t + \varphi } ]} {\; - \; \textrm{sin}[{2\pi ({{f_{s}} + {f_\textrm{R}}} )t + \varphi } ]} \}.} \end{array}$$

If ${f_{s}} = {f_\textrm{R}}$, then ${f_{s}} - {f_\textrm{R}} = 0$ and at the same time ${f_{s}} + {f_\textrm{R}} = 2{f_\textrm{R}}$. In this way, we obtain two signals ${V_\textrm{X}}$ and ${V_\textrm{Y}}$ having half amplitude but double frequency,

$$\begin{array}{c} {{V_\textrm{X}} = \frac{1}{2}A\{{\textrm{cos}(\varphi )+ \textrm{cos}[{2\pi ({2{f_\textrm{R}}} )t + \varphi } ]} \},}\\ {{V_\textrm{Y}} = \frac{1}{2}A\{{\sin (\varphi )- \sin [{2\pi ({2{f_\textrm{R}}} )t + \varphi } ]} \}.} \end{array}$$

Then, the low-pass filter suppresses the components of the signals ${V_\textrm{X}}$ and ${V_\textrm{Y}}$ with a frequency of $2{f_\textrm{R}}$ and we get

$$\begin{array}{c} {{V_{\textrm{outX}}}(t )= \frac{1}{2}\textrm{Acos}\varphi ,}\\ {{V_{\textrm{outY}}}(t )= \frac{1}{2}A\textrm{sin}\varphi .} \end{array}$$

Subsequently, by transforming the output signals into the polar coordinate system, we obtain expressions for amplitude and phase as functions of time [24],

$$\begin{array}{c} {A(t )= 2\sqrt {V_{\textrm{outX}}^2 + V_{\textrm{outY\; }}^2,} }\\ {\varphi (t )= \textrm{ta}{\textrm{n}^{ - 1}}({{V_{\textrm{outY}}}/{V_{\textrm{outX\; }}}} ).} \end{array}$$

In our case, the input signal (a PPGI curve) is a vector of average pixel values obtained from a moving kernel with a selectable dimension. Firstly, a high-pass filter filters this input signal to suppress the DC component of the signal. We used an equiripple FIR filter with a cut-off frequency of 0.6 Hz and an order of 71. As a reference signal, we will use the simultaneously sensed PPG signal from the finger or a harmonic signal with an amplitude of one and frequency given by the simultaneously measured ECG signal (I. Einthoven lead) for each heart cycle. These signals were transformed to a sinusoidal waveform with a normalized and stable amplitude, with the frequency of the reference signal varying with respect to the changing heart rate from beat to beat. Subsequently, we used a low-pass equiripple FIR filter with a cut-off frequency of 0.4 Hz and an order of 97 to extract the amplitude of the prefusion changes detected by PPGI.

2.5 Aims of analyses

We focused on the effect of moving kernel size, a comparison of methods for extracting the amplitude of perfusion changes, and most importantly, we sought to establish two measurement conditions related to image compression. The main idea is to compare the laboratory conditions with those that a user of a conventional digital camera may encounter in everyday practice without modification. First, we compressed the raw video obtained from the camera using H.264 standard, which is a common format used for recording video via consumer electronics. The parameter we set was constant rate factor CRF the value of which allows us to vary the compression rate from lossless (CRF = 0) to the compression commonly used in consumer electronics applications (CRF = 18) [7]. We used a terminal-based set of tools for video compression, namely FFmpeg [27]. A RAW video recording of 60 s length in the form of an uncompressed .avi file had an original size of 2.21 GB, whereas, after compression, the sizes for CRF = 0 and CRF = 18 were 607.9 MB and 11.7 MB, respectively.

3. Results

In the following, we present results on the overall system performance, perfusion mapping using FFT and LIA, as well as comparison of the extracted amplitudes.

3.1 Test of the system performance

We first tested the system performance on an artificial sine wave with a frequency of 1 Hz. This signal was generated directly by the MCU and fed to the output of its DAC. In this way, we wanted to verify the quality of the digital and analogue signal processing by the proposed HW and the system's robustness to interference, e.g. 50 Hz from the power grid. This test signal was then fed to the microphone input of a DSLR camera. In this case, we focused only on the audio input, leaving the camera lens covered by a protective cover. The signal was sampled at 44.1 kHz, and its modulated waveform is shown in top of Fig. 5. The middle part of Fig. 5 shows the signal waveform after demodulation, and the bottom part shows the power spectral density with the calculated SNR, which was 29.09 dB.

 figure: Fig. 5.

Fig. 5. Performance test. The top image shows the modulated test signal - a sinusoid with a frequency of 1 Hz. The middle part shows the signal after demodulation and the lower graph represents the SNR of the measurement system.

Download Full Size | PDF

Subsequently, we proceeded with the experiment, which protocol is described in the Materials and Methods section. The modulated and demodulated waveforms obtained during the two experiments are depicted in Fig. 6. On the left is the 5 s recording using the PPG sensor, and on the right is the recording via the ECG reference. The ECG signal contained noise from the power supply network, so this noise was removed using a wavelet transform.

 figure: Fig. 6.

Fig. 6. Test of capturing PPG (left) and ECG (right). The modulated signal is shown in blue. The red curve is the signal after demodulation and in the case of the ECG also after 50 Hz removal.

Download Full Size | PDF

3.2 Perfusion mapping using FFT and LIA

The next part was to create perfusion maps in order to assess the change in perfusion induced by vasodilator creams. The areas where the creams were applied were square, and we also marked them on the forearm of the subject for better orientation in map and image. The C1 was applied on the left part of the forearm, and the area with the applied C2 was located on the right. The area in the middle served as a reference with no assumed vasodilatory changes in subcutaneous tissue perfusion (see Fig. 1). We have created a sequence of images/maps as part of this analysis shown in Fig. 7. These maps were made 20 minutes after applying the external stimulus in the form of the creams mentioned above.

 figure: Fig. 7.

Fig. 7. Perfusion maps obtained for different kernel sizes from 3×3 px2 to 15×15 px2 and different compression levels (CRF = 0 on the left, CRF = 18 on the right). The coloured circles indicate the locations from which the time courses of perfusion changes were extracted for each site application: with cream 1 (purple), cream 2 (blue), and site without cream (green), see also Fig. 8. When comparing amplitudes, these reference points were always placed at the same location for all kernel sizes. The left column contains the maps obtained by FFT and the right column contains the maps generated by LIA.

Download Full Size | PDF

All maps were obtained from the green layer of the image. We gradually increased the kernel size from 3×3 px2 to 15×15 px2 with a step size of 3 px. When comparing the quality of the maps in terms of the methods used for extracting the PPG signal amplitude, the LIA performs much better in this compared to the FFT. Figure 7 also shows that image compression affects the quality of the resulting perfusion map, which is a well-known fact. With lossless compression (CRF = 0), subtle changes in map structure can be discerned even with the FFT method, especially from the 13×13 px2 kernel size (increased perfusion in the inner region with cream 2 applied). The LIA method reveals this trend already at a kernel of 5×5 px2. The maps show a difference when compressed with CRF = 18. While the FFT generated maps are almost identical with no change in detail for all kernel sizes, the LIA method is still able to discern and detect changes in the spatial distribution of perfusion starting from kernel 9×9 px2. The reference signal obtained from our HW connected to the camera's mic input plays a significant role here.

Furthermore, in Fig. 7, it can be noticed that the effect of the creams varies. The vasodilatory effect and thus the increase in tissue perfusion (pulsatile part depending on cardiac activity) of the C1 is significantly lower compared to the C2. At the same time, it is possible to see the impact of different PPGI signal amplitude extraction methods on the quality of the produced map. In the case of amplitude extraction from the FFT spectrum, the map contours and subtle changes in amplitude are blurred. The LIA technique makes it possible to create a map that offers more detail than the FFT spectrum mapping method. At the same time, it is possible to observe the effect of kernel size when looking in detail at perfusion changes. At a kernel size of 3×3 px2, the maps for both methods are similar in quality. However, already at a kernel size of 5×5 px2, the lower part of the C2 region for the LIA map is more finely resolved compared to the FFT map. We can see local perfusion changes approaching the normalised maximum at a kernel size of 11×11 px2 and the LIA map (red colour in the C2 region). On the FFT map, this information is not discernible.

3.3 Amplitude comparison

The RAW signals (only pixel averaging) in the time domain are shown in Fig. 8. Here it is possible to notice the changes in PPG signal amplitude depending on the location on the forearm. For example, the amplitude at the C2 application site is many times higher compared to the C1 application site or the reference site between the two regions.

 figure: Fig. 8.

Fig. 8. Time courses from 3 areas of size 15×15 px2 created from recording 20 minutes after application of creams. Colours of signals correspond to the marks in Fig. 7.

Download Full Size | PDF

To compare the quality of the PPG signal and its amplitude obtained by the FFT and LIA methods, we show in Fig. 9 the amplitudes from selected locations of the maps in Fig. 7. The perfusion changes show that the C2 cream has considerably larger vasodilating effects than C1. At the same time, amplitude mapping using the LIA method allows more sensitive capture of perfusion changes. The dashed (LIA) and dash dotted (FFT) lines show the trend of detecting perfusion at the site of its most pronounced change. Additionally, the amplitudes for different CRF levels are compared. The comparison shows that in the case of compression with CRF = 18, the LIA method still shows better detection of perfusion changes even for smaller kernel values. Furthermore, the curve describing the FFT detection trend is more flattened than LIA in this case.

 figure: Fig. 9.

Fig. 9. Comparison of amplitudes for different locations with cream 1 (purple), without cream (green) and cream 2 (blue). The colours correspond to the marks in Fig. 7. The individual locations are grouped to allow comparison of the amplitude for the same location but obtained by a different method. The left column of the same colour is the FFT, and the right is the LIA method. The dashed line shows the trend for the LIA method at the site of the most considerable increase in perfusion induced by cream 2. The dot-dashed line characterizes the trend of the change in perfusion of the same site but obtained by the FFT method.

Download Full Size | PDF

4. Discussion & conclusion

Our work focuses on an innovative single chip (OAs are also the part of the MCU) hardware device that allows the modulation of a selected biosignal into the audio frequency band. While combined with a commercial device such as a digital camera or a smartphone, a unique measurement system is created that allows simultaneous recording of perfusion changes with a selected reference, e.g., PPG placed on a fingertip or ECG measurement. Thus, the camera provides tissue perfusion imaging using the PPGI technique, whereby the reference signal can increase the signal-to-noise ratio, especially when using a technique such as LIA. In principle, LIA does not necessarily require an external reference signal. In that case, pixel averaging of the larger imaged area can be used [15]. Thus, Kamshilin et al. demonstrated the suitability of using a “self-reference” signal, introducing the “PPGI of high spatial resolution”. This method would also very likely work in our case because we recorded the signal at room temperature and from a location with relatively good perfusion. However, this may not be the case in a scenario where vasoconstriction mechanisms are involved, e.g. while using medication or cold application. Since Kamshilin et al. haven shown the feasibility of their method, we chose to focus on the advantages of an external reference obtained with our HW, which allows extending the capabilities of the PPGI measurement setup in a very affordable and convenient way.

When recording RAW data, despite its undeniable advantages, the RAW recording has a disadvantage too, namely archiving and the demands of processing large files. In the case of image or video compression, we should consider the degradation of the signal quality [7]. Here, creating a reference signal by averaging pixels even from a larger area can become problematic and we may not get a signal with the desired quality. Therefore, the appropriate solution is a compromise between the recording size and its quality on the one hand. On the other hand, we can use an external reference signal as in our case, whereby the compression of the video does not affect the resulting perfusion map in terms of the resolution of the spatial changes. At the same time, this approach significantly reduces the requirements for archiving the measured data, even in the case of long-term recordings. Another option is to monitor the pulse wave propagation velocity and calculate the pulse arrival time (PAT) if the ECG signal is used or pulse transit time (PTT) in the case of the PPG Ref. [28]. At the same time, ECG allows detection of changes in heart rhythm based on very established algorithms such as [29], which can help generate the reference signal more precisely. However, for some types of physiological parameter measurements using ECG, the modulation frequency of our device may not be sufficient. It is in the case of ventricular desynchrony assessment by ultra-high frequency ECG, when the sampling frequency can be up to 5 kHz per channel which contradicts use of our device for such a type of measurement [30].

Thus, with considerable space-saving, this method of tissue perfusion mapping is also suitable for recordings that will be made with a common and affordable technique, e.g. during an outpatient examination in an allergology office. The increase in signal quality and its spatial resolution makes it possible to deploy such a system for different applications in allergology [22] or applications dealing with the investigation of the state of the autonomic nervous system [31]. Another possible example is detecting heart rate variability (HRV) and simultaneous investigation of subcutaneous perfusion changes [32].

An outlook is the extension of the device with the possibility of simultaneous recording of multiple channels. Both the microcontroller and the carrier modulation frequency would allow this adaptation of the system.

Funding

University of Zilina (Grant system No. 1/2021/I-21-028-61).

Acknowledgment

Authors thank the Grant System of University of Zilina for their support.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. M. Huelsbusch and V. Blazek, “Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI,” in A. V. Clough and C.-T. Chen, eds. (2002), p. 110.

2. N. Blanik, A. K. Abbas, B. Venema, V. Blazek, and S. Leonhardt, “Hybrid optical imaging technology for long-term remote monitoring of skin perfusion and temperature behavior,” J. Biomed. Opt. 19(01), 1 (2014). [CrossRef]  

3. V. Perlitz, B. Cotuk, M. Lambertz, R. Grebe, G. Schiepek, E. Petzold, H. Schmid-Schönbein, and G. Flatten, “Coordination dynamics of circulatory and respiratory rhythms during psychomotor drive reduction,” Auton. Neurosci. 115(1-2), 82–93 (2004). [CrossRef]  

4. A. B. Hertzman, “Photoelectric plethysmography of the fingers and toes in man,” Exp. Biol. Med. 37(3), 529–534 (1937). [CrossRef]  

5. T. Wu, V. Blazek, and H. J. Schmitt, “Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes,” in (International Society for Optics and Photonics, 2000), 4163, pp. 62–70.

6. S. Zaunseder, A. Vehkaoja, V. Fleischhauer, and C. Hoog Antink, “Signal-to-noise ratio is more important than sampling rate in beat-to-beat interval estimation from optical sensors,” Biomedical Signal Processing and Control 74, 103538 (2022). [CrossRef]  

7. D. J. McDuff, E. B. Blackford, and J. R. Estepp, “The impact of video compression on remote cardiac pulse measurement using imaging photoplethysmography,” in (IEEE, 2017), pp. 63–70.

8. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16(26), 21434–21445 (2008). [CrossRef]  

9. M. Lewandowska and J. Nowak, “Measuring pulse rate with a webcam,” J Med Imaging Hlth Inform 2(1), 87–92 (2012). [CrossRef]  

10. M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011). [CrossRef]  

11. G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013). [CrossRef]  

12. G. De Haan and A. Van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913–1926 (2014). [CrossRef]  

13. W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016). [CrossRef]  

14. W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote PPG,” IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2017). [CrossRef]  

15. A. A. Kamshilin, S. Miridonov, V. Teplov, R. Saarenheimo, and E. Nippolainen, “Photoplethysmographic imaging of high spatial resolution,” Biomed. Opt. Express 2(4), 996–1006 (2011). [CrossRef]  

16. K. A. Reddy, J. R. Bai, B. George, N. M. Mohan, and V. J. Kumar, “Virtual instrument for the measurement of haemo-dynamic parameters using photoplethysmograph,” in 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings (IEEE, 2006), pp. 1167–1171.

17. Mingui Sun, P. A. Rochel, Jun Zhao, and R. J. Sclabassi, “Switching modulation for wireless transmission of biological waveforms using a cellphone,” in IEEE 30th Annual Northeast Bioengineering Conference, 2004. Proceedings of The (IEEE, 2004), pp. 17–18.

18. K. Maheshkumar, K. Dilara, K. Maruthy, and L. Sundareswaren, “Validation of PC-based sound card with Biopac for digitalization of ECG recording in short-term HRV analysis,” North Am J Med Sci 8(7), 307 (2016). [CrossRef]  

19. N. F. Güler and U. Fidan, “Wireless Transmission of ECG signal,” J. Med. Syst. 30(3), 231–235 (2006). [CrossRef]  

20. Xiaoli Quan, Nanquan Zhou, and Haotian Wu, “Design of sound card electrocardiosignal acquisition system based on LabVIEW,” in 2011 International Conference on Multimedia Technology (IEEE, 2011), pp. 282–285.

21. R. Stojanovic and D. Karadaglic, “An economical and feasible teaching tool for biomedical education,” in 2011 24th International Symposium on Computer-Based Medical Systems (CBMS) (IEEE, 2011), pp. 1–5.

22. C. Blazek and V. Blazek, “Selected Clinical Applications of Functional PPGI Perfusion Mapping in Dermatology,” in Studies in Skin Perfusion Dynamics, V. Blazek, J. Kumar V, S. Leonhardt, and M. Mukunda Rao, eds., Biological and Medical Physics, Biomedical Engineering (SpringerSingapore, 2021), pp. 207–221.

23. P. Kligfield, L. S. Gettes, J. J. Bailey, R. Childers, B. J. Deal, E. W. Hancock, G. van Herpen, J. A. Kors, P. Macfarlane, D. M. Mirvis, O. Pahlm, P. Rautaharju, and G. S. Wagner, “Recommendations for the Standardization and Interpretation of the Electrocardiogram: Part I: The Electrocardiogram and Its Technology: A Scientific Statement From the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society Endorsed by the International Society for Computerized Electrocardiology,” Circulation 115(10), 1306–1324 (2007). [CrossRef]  

24. S. DeVore, A. Gauthier, J. Levy, and C. Singh, “Development and evaluation of a tutorial to improve students’ understanding of a lock-in amplifier,” Phys. Rev. Phys. Educ. Res. 12(2), 020127 (2016). [CrossRef]  

25. P. Horowitz, W. Hill, and I. Robinson, The Art of Electronics (Cambridge University Press, 1989), 2.

26. R. E. Simpson, Introductory Electronics for Scientists and Engineers (Allyn & Bacon, 1974).

27. FFmpeg (FFmpeg developers, 2022).

28. M. Elgendi, R. Fletcher, Y. Liang, N. Howard, N. H. Lovell, D. Abbott, K. Lim, and R. Ward, “The use of photoplethysmography for assessing hypertension,” npj Digit. Med. 2(1), 60 (2019). [CrossRef]  

29. J. Pan and W. J. Tompkins, “A real-time QRS detection algorithm,” IEEE Trans. Biomed. Eng. BME-32(3), 230–236 (1985). [CrossRef]  

30. P. Jurak, J. Halamek, J. Meluzin, F. Plesinger, T. Postranecka, J. Lipoldova, M. Novak, V. Vondra, I. Viscor, L. Soukup, P. Klimes, P. Vesely, J. Sumbera, K. Zeman, R. S. Asirvatham, J. Tri, S. J. Asirvatham, and P. Leinveber, “Ventricular dyssynchrony assessment using ultra-high frequency ECG technique,” J Interv Card Electrophysiol 49(3), 245–254 (2017). [CrossRef]  

31. D. McDuff, I. Nishidate, K. Nakano, H. Haneishi, Y. Aoki, C. Tanabe, K. Niizeki, and Y. Aizu, “Non-contact imaging of peripheral hemodynamics during cognitive and psychological stressors,” Sci. Rep. 10(1), 10884 (2020). [CrossRef]  

32. Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1.
Fig. 1. Rendering of the measurement setup.
Fig. 2.
Fig. 2. Reference biosignal processing chain.
Fig. 3.
Fig. 3. Signal conditioning before connecting to the mic input.
Fig. 4.
Fig. 4. The realized device prototype.
Fig. 5.
Fig. 5. Performance test. The top image shows the modulated test signal - a sinusoid with a frequency of 1 Hz. The middle part shows the signal after demodulation and the lower graph represents the SNR of the measurement system.
Fig. 6.
Fig. 6. Test of capturing PPG (left) and ECG (right). The modulated signal is shown in blue. The red curve is the signal after demodulation and in the case of the ECG also after 50 Hz removal.
Fig. 7.
Fig. 7. Perfusion maps obtained for different kernel sizes from 3×3 px2 to 15×15 px2 and different compression levels (CRF = 0 on the left, CRF = 18 on the right). The coloured circles indicate the locations from which the time courses of perfusion changes were extracted for each site application: with cream 1 (purple), cream 2 (blue), and site without cream (green), see also Fig. 8. When comparing amplitudes, these reference points were always placed at the same location for all kernel sizes. The left column contains the maps obtained by FFT and the right column contains the maps generated by LIA.
Fig. 8.
Fig. 8. Time courses from 3 areas of size 15×15 px2 created from recording 20 minutes after application of creams. Colours of signals correspond to the marks in Fig. 7.
Fig. 9.
Fig. 9. Comparison of amplitudes for different locations with cream 1 (purple), without cream (green) and cream 2 (blue). The colours correspond to the marks in Fig. 7. The individual locations are grouped to allow comparison of the amplitude for the same location but obtained by a different method. The left column of the same colour is the FFT, and the right is the LIA method. The dashed line shows the trend for the LIA method at the site of the most considerable increase in perfusion induced by cream 2. The dot-dashed line characterizes the trend of the change in perfusion of the same site but obtained by the FFT method.

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

s mod ( t ) = ( 1 + s bio ( t ) C ) sin ω c t ,
V out = R 2 R 1 V in + ( 1 + R 2 R 1 ) V cc R 4 R 3 + R 4 V out 0.338 V cc 0.015 V in ,
s comp [ n ]  = exp ( j n 2 π f C f samp ) ,
s interm [ n ] = s comp [ n ] s mod [ n ] .
s interm [ n ] = | s interm [ n ] | .
V in ( t ) = A cos ( 2 π f s t + φ ) ,
V refX ( t ) = cos ( 2 π f R t ) , V refY ( t ) = sin ( 2 π f R t ) .
cos ( a ) cos ( b ) = 1 2 [ cos ( a + b ) + cos ( a b ) ] , cos ( a ) sin ( b ) = 1 2 [ sin ( a + b ) sin ( a b ) ] .
V X ( t ) = V in ( t ) V refX ( t ) = 1 2 A { cos [ 2 π ( f s f s ) t + φ ] + cos [ 2 π ( f s + f R ) t + φ ] } , V Y ( t ) = V in ( t ) V refY ( t ) = 1 2 A { sin [ 2 π ( f s f R ) t + φ ] sin [ 2 π ( f s + f R ) t + φ ] } .
V X = 1 2 A { cos ( φ ) + cos [ 2 π ( 2 f R ) t + φ ] } , V Y = 1 2 A { sin ( φ ) sin [ 2 π ( 2 f R ) t + φ ] } .
V outX ( t ) = 1 2 Acos φ , V outY ( t ) = 1 2 A sin φ .
A ( t ) = 2 V outX 2 + V outY\;  2 , φ ( t ) = ta n 1 ( V outY / V outX\;  ) .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.