Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Inferring deep-brain activity from cortical activity using functional near-infrared spectroscopy

Open Access Open Access

Abstract

Functional near-infrared spectroscopy (fNIRS) is an increasingly popular technology for studying brain function because it is non-invasive, non-irradiating and relatively inexpensive. Further, fNIRS potentially allows measurement of hemodynamic activity with high temporal resolution (milliseconds) and in naturalistic settings. However, in comparison with other imaging modalities, namely fMRI, fNIRS has a significant drawback: limited sensitivity to hemodynamic changes in deep-brain regions. To overcome this limitation, we developed a computational method to infer deep-brain activity using fNIRS measurements of cortical activity. Using simultaneous fNIRS and fMRI, we measured brain activity in 17 participants as they completed three cognitive tasks. A support vector regression (SVR) learning algorithm was used to predict activity in twelve deep-brain regions using information from surface fNIRS measurements. We compared these predictions against actual fMRI-measured activity using Pearson’s correlation to quantify prediction performance. To provide a benchmark for comparison, we also used fMRI measurements of cortical activity to infer deep-brain activity. When using fMRI-measured activity from the entire cortex, we were able to predict deep-brain activity in the fusiform cortex with an average correlation coefficient of 0.80 and in all deep-brain regions with an average correlation coefficient of 0.67. The top 15% of predictions using fNIRS signal achieved an accuracy of 0.7. To our knowledge, this study is the first to investigate the feasibility of using cortical activity to infer deep-brain activity. This new method has the potential to extend fNIRS applications in cognitive and clinical neuroscience research.

© 2015 Optical Society of America

1. Introduction

Functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS) are two popular neuroimaging techniques used to measure hemodynamic changes associated with neural activity. Functional MRI measures the blood oxygen level-dependent (BOLD) response that results from local concentration changes in paramagnetic deoxy-hemoglobin (deoxy-Hb), while fNIRS measures the concentration changes of both oxygenated and deoxygenated hemoglobin (oxy- and deoxy-Hb). Functional MRI can measure activity anywhere in the brain where T2* signal is obtained, but fNIRS can only measure activity at the local cortical surface. Nonetheless, fNIRS offers several advantages over fMRI. For instance, fNIRS has high temporal resolution (100ms or faster) and is inexpensive and portable. In addition, fNIRS can be used on some populations that cannot undergo fMRI scanning, such as infants, children with autism-related sensory hypersensitivity, pregnant women, and people with claustrophobia. Furthermore fNIRS allows measurement of hemodynamic activity in environments with greater ecological validity, an especially important advantage, as studies have shown that simulated tasks do not always generate the same brain activity as do their real-life counterparts [1]. These many advantages make fNIRS a potentially advantageous modality with which to investigate brain function and, in some life-like experimental paradigms, the only hemodynamics-measuring modality available.

Despite these many advantages, fNIRS does have a major drawback: limited sensitivity to hemodynamic changes occurring in brain regions below the cortical surface, including subcortical and inferior cortical brain regions (henceforth referred to as “deep-brain regions”). Limited ability to measure hemodynamic changes in deep-brain regions arises primarily from scattering and absorption of near-infrared light as it passes through biological tissues. Measurement depth is also a function of the distance between light emitters and detectors; increasing source-detector distance can increase penetration depth, but only at the expense of signal-to-noise ratio. Thus, scattering and absorption properties of the medium dictate the maximum allowable distance between emitters and detectors and therefore the maximum depth at which one can measure with usable signal [2]. As a result, fNIRS devices can typically detect hemodynamic changes that occur at a maximum depth of 2-3 cm in the adult brain – limiting fNIRS to the measurement of cortical surface activity [3].

Since fNIRS cannot measure brain activity below the cortical surface, researchers hoping to study hemodynamic activity in deep-brain regions must rely on fMRI, which, in addition to being costly, can limit experimental design. Even when cost is not an obstacle, fMRI is not ideal in some scenarios, such as investigation of brain activity in naturalistic environments (e.g., face-to-face communication or exercise) and simultaneous measurement of activity from two or more interacting participants (i.e., hyperscanning) [4], especially when such interaction involves complex social interaction [5]. If one were able to obtain deep-brain activity with fNIRS without increasing cost or forgoing experimental design freedoms, one could open a new range of clinical and experimental possibilities.

We propose a novel computational method to mitigate the limitations that arise from fNIRS' inability to directly measure activity in deep-brain regions and potentially extend fNIRS applications in cognitive and clinical neuroscience research. To achieve this goal, we rely on functional connectivity in the brain. Numerous studies have shown that many brain areas are connected both anatomically and functionally [6,7]. Activity in deep-brain areas is often correlated with activity in cortical areas. Such connectivity between deep brain areas and cortical surface areas suggests an intriguing possibility: the use of cortical surface activity to predict deep-brain activity.

We developed an approach to infer deep-brain activity from cortical activity, as measured by fNIRS detectors designed to quantify activity at the superior and lateral cortical surfaces. We used a support vector regression (SVR) learning algorithm [8–12] for the time series prediction. In the past decade, SVR has gained wide popularity in the neuroimaging community [13] and has been used to decode visual stimuli [14], classify brain states [15,16], and diagnose brain diseases [17–19].

To develop our prediction method, we simultaneously recorded brain activity with fNIRS and fMRI while participants performed three cognitive tasks. We then used fNIRS cortical surface signal to infer signal from twelve separate deep-brain regions. We tested our inferences against fMRI deep-brain signals. In addition, we used fMRI cortical surface signals, both from the entire cortex and from fNIRS channel locations, to infer signals from the same twelve deep-brain regions. We used this comparison as a benchmark to demonstrate the effectiveness of the method. We hypothesized that cortical surface activity, as measured either by fNIRS or fMRI, could be used to infer deep-brain activity.

2. Materials and methods

2.1 Participants

Eighteen healthy young adults initially participated in the study. One of the participants did not complete the study, and that individual’s data were excluded from analysis. The remaining seventeen participants included seven males and ten females. Their mean age was 26.8 years with an age range of 20-45. Written informed consent was obtained from all participants, and the study protocol was approved by the Stanford University Institutional Review Board.

2.2 Experimental procedure

Concurrent fMRI and fNIRS scans were performed while participants completed three cognitive tasks: go/no-go (nog), fearful/scrambled faces (fac) [20,21], and a complex visual task (comp-vis). These tasks are described in detail below and have been used by our research group in previous studies [3,22]. Participants were given task instructions verbally before entering the scanner. The order of the tasks was fixed: nog, fac, comp-vis. All experimental stimuli were presented using E-prime software (http://www.pstnet.com/).

2.3 Task descriptions

Go/no-go task (nogo)

The go/no-go task consisted of rest (R), go (G), and no-go (NG) epochs in the following order: R–G–NG–G–NG–G–NG–G–NG–G–NG–G–NG–R. Each rest epoch lasted 30 s, and each task epoch lasted 26 s. During rest epochs, participants passively viewed a blank screen. During go epochs, “Push for all letters” was first displayed for 2 s, then participants were presented with 1 of 12 different letters for 500 ms, followed by a 1.5-s inter-stimulus interval. Participants were instructed to press a button with their right index finger each time a letter appeared on the screen. During the no-go epochs, “Push for All Letters Except X” was displayed for 2 s, then again participants were presented with 1 of 12 different letters for 500 ms, followed by a 1.5-s inter-stimulus interval. In the no-go epochs, participants were instructed to push the button for all letters except X, for which they were to withhold their response. During no-go epochs, the letter X was presented on 50% of trials.

Faces task (faces)

The faces task consisted of rest (R), Fearful (F) face and Scrambled (S) face epochs, presented in alternation as follows: R-F-S-F-S-F-S-F-S-F-S-F-S-R. Each rest epoch lasted 30 s. Each epoch (both F and S) lasted 20 s and consisted of the presentation of pictures of five human faces for four seconds each. Participants were instructed to determine the sex of the depicted faces and press 1 for female, or 2 for male. In the F epoch, presented faces had fearful expressions. In the S epoch, faces were scrambled, offering a complex visual control stimulus. Participants were instructed to guess the sex of the face even in the S epochs, though accurate sex identification was impossible. Stimulus order was fixed for all participants.

Complex Visual task (comp-vis)

The complex visual task consisted of rest (R), fixation (F) and task (T) epochs in the following order: R-T-F-T-F-T-F-T-F-T-F-T-R. Each rest epoch lasted 30 s, each fixation epoch lasted 20 s, and each task epoch lasted 16 s. During the fixation epochs, a green “+” was presented in the center of a black screen. Participants were instructed to gaze at the “+”. During the task epochs, pictures of animals and non-animals were presented with the prompt: “Is it an animal? Press 1 for Yes, 2 for No.” Then four animal or non-animal stimuli were presented for 3 s each, with 1-s inter-stimulus intervals (blank screen). Animals and non-animals were ordered semi-randomly so a particular task epoch could contain, for example, one non-animal and three animals. No task epochs contained all non-animals or all animals. Stimulus order was fixed for all participants.

2.4 fNIRS data acquisition and preprocessing

The fNIRS apparatus used for data acquisition was an ETG-4000 Optical Topography system (Hitachi Medical, Japan), a multichannel, continuous-wave optical imaging system with 34 optodes, half of which served as illumination optodes and half of which served as collection optodes. Each illumination optode was connected to a pair of laser diodes generating two different wavelengths, 695 nm and 830 nm, for simultaneous assessment of oxy- and deoxy-hemoglobin levels. In addition, each laser diode was modulated at different frequencies. This frequency multiplexing technique allowed all illumination optodes to be on simultaneously and yet still be uniquely differentiated by multiple adjacent collection optodes. This arrangement allowed the machine to sample hemodynamic changes at 10 Hz. The distance between illumination and collection optode pairs was 3 cm. We used two “3x3” patches provided by Hitachi. Each patch held five illumination optodes and four collection optodes, resulting in 24 total measurement channels. MRI fiducial markers (MRIEquip, SKU: BP-103X, Model: BP-1015, 15 mm outer diameter, 3.5 mm thick, 5 mm central axis hole) were affixed to the inner side of the patches between illumination-collection optode pairs to indicate the location of measurement channels. The fNIRS measurement covered a 6x6 cm2 area of the left and right prefrontal brain region (Fig. 4(d)). The cap remained in the same position on each participant’s head for all three tasks.

The ETG-4000 was located in the control room of the MRI scanning facility. A set of ten-meter, magnet-safe optical fiber bundles (Hitachi Medical, Japan) was passed through a waveguide in the wall between the control room and the MRI scanner room. An E-prime stimulus program triggered both the ETG-4000 and fMRI scanner simultaneously via serial port.

The recorded changes in optical density were first converted to the concentration changes of oxy-hemoglobin (∆[HbO]) and deoxy-hemoglobin (∆[Hb]) using the modified Beer-Lambert law [23]. An fNIRS channel was determined to be a noisy channel by visual inspection. Specifically, a channel was determined to be noisy if the heartbeat was not identifiable in the corresponding wavelet transform map [24,25] or if the channel was excessively noisy with various spikes on visual inspection. Figures 1(a)-1(d) shows examples of noisy and good-quality fNIRS signals.) A subject was excluded from the SVR analysis if more than twelve channels (50% of total collected channels per subject) were determined to be excessively noisy (Fig. 1€). The concentration signal was bandpass filtered in a frequency range of 0.01-0.5 Hz to remove high-frequency instrument noise, heartbeat physiological noise, and low-frequency drift before it was inputted into the SVR model.

 figure: Fig. 1

Fig. 1 Noise determination. (a) The wavelet transform map of a poor-quality fNIRS signal; note that there is no heartbeat frequency band at 1 Hz. (b) The wavelet transform map of a good-quality fNIRS signal; note that there is a clear heartbeat frequency band showed at 1 Hz. (c) and (d) are the time series corresponding to (a) and (b). The blue curves represent the original signals, and the red curves represent the signals after bandpass filtering. (e) Number of noisy fNIRS channels for all subjects.

Download Full Size | PDF

2.5 fMRI data acquisition and preprocessing

Structural and functional MRI images were acquired using a 3.0T whole-body scanner (Signa Discovery 750, GE Healthcare Systems, Milwaukee, WI) with an eight-channel head coil. The acquisition parameters for axial-oblique 3D T1-weighted structural images were: fast spoiled gradient recalled echo (FSPGR) pulse sequence, inversion recovery preparation pulse T1 = 400 ms; repetition time TR = 8.5 ms; echo-time TE = 3.4 ms; flip angle = 15°; receiver bandwidth ± 32 kHz; slice thickness = 1.2 mm; number of slices = 128; number of excitations (number of signals averaged) = 1; field-of-view FOV = 22 cm; phase FOV = 0.75; acquisition matrix = 256 x 192. The acquisition parameters for functional images were: T2*–sensitive gradient echo spiral pulse sequence sensitive to BOLD contrast [26], TR = 2000 ms; TE = 30 ms; flip angle = 80°; FOV = 220 mm x 220 mm; number of slices = 31; voxel size 3.4x3.4x4 mm3.

Functional MRI images first had their physiological noise (such as respiration and heartbeat) removed with RETROICOR [27] and RVHRCOR [28] methods. They were then realigned, slice timing corrected, coregistered, and normalized to the standard template using SPM software (version 8, http://www.fil.ion.ucl.ac.uk/spm/). Temporal drift in the fMRI signal was removed by subtracting a moving average with a window size of 50 TRs (100 s).

2.6 Support vector regression analysis to infer deep-brain signals

We used SVR to infer deep-brain activity. As compared to other prediction approaches, such as the linear regression method, SVR is most suitable to our study. Specifically, as certain features in our data are highly correlated, linear regression could potentially assign a large weight to the one feature that correlates best with the predicted variable, while SVR assigns weights in a manner that better reflects contributions among features. A detailed description of SVR can be found elsewhere [29]. We briefly introduce the formulation here.

Given a set of training data, {(xi, yi)| xiRn, yiR1}, i = 1, …, m, where xi is the input and yi is the target output, the purpose of regression estimation with SVR is to find a vector w such that wT xi is close to the target values yi. The vector w is also known as the weight vector. In this study, we used ν-support vector regression (ν-SVR). It solves the following primal problem [30]:

where

minwf(w),f(w)12wTw+C(νε+1mi=1m(ξi+ξi*))
and
(wTϕ(xi)+b)yiε+ξi,yi(wTϕ(xi)+b)ε+ξi*,ξi,ξi*0,i=1,...,m,ε0.
The constant C>0 is the regularization parameter, and ξi is the so-called ε-insensitive loss function. The function ϕ maps the training data xi into a higher-dimensional space. The above problem can be solved in its dual formulation. And various kernels are introduced to improve the computational efficiency.

In this study, the ν-SVR was performed using the Libsvm software developed by Chang and Lin [31]. All ν-SVR here used the linear kernel kijϕ(xi)Tϕ(xj). Two parameters need to be determined for the SVR training process: ν and C. We used cross-validation to find the best ν and C.

We used the Pearson’s correlation coefficient to evaluate the performance of the prediction from the ν-SVR models. The Pearson’s correlation coefficient (r) measures the degree of association between the observed and predicted values. The higher the value of r, the better the performance of the model.

3. Data analysis

A diagram of the SVR algorithm for this study is shown in Fig. 2. We used three groups of input signals: the fNIRS signal measured at the prefrontal cortex (fNIRS-PFC); the fMRI surface signal extracted from the positions on the prefrontal cortex where the fNIRS channels were located (fMRI-PFC); and the fMRI surface signal extracted uniformly from the entire cortex (fMRI-EC). The target deep-brain signals were fMRI signals extracted from twelve representative deep-brain regions: bilateral fusiform, insula, hippocampus, parahippocampal gyrus, amygdala and caudate.

 figure: Fig. 2

Fig. 2 Diagram of SVR algorithm.

Download Full Size | PDF

The SVR analysis included two main steps: training and prediction. The SVR training step used the first two-thirds of each input and target signal to generate a model. The SVR prediction step then used this model and the last one-third of each input signal to predict the target signal and test the model.

3.1 Generating the fNIRS-PFC input signal

The fNIRS signals included the oxy- and deoxy- hemoglobin signals (HbO and Hb), both of which were used as input to the SVR pipeline after two preprocessing steps. First, we down-sampled the fNIRS signals (collected at 10 Hz) so that they contained the same number of data points as did the target fMRI signals (collected at 0.5 Hz). Second, we selectively utilized the fNIRS signals that best correlated with the fMRI signals at the same locations. Specifically, we utilized BOLD signals from prefrontal areas where fNIRS probes were located and studied the latency between fNIRS and BOLD signals.

Previous studies have reported the temporal latencies between oxyhemoglobin/deoxyhemoglobin signals in fNIRS and blood-oxygen-level dependent (BOLD) signals in fMRI [32–35]. These studies reveal an optimal lag between fNIRS and fMRI signals, that is, a latency at which the two signals are most highly correlated. Latencies vary with cortical location, task, task duration, and subject. To find the optimal latency in the prefrontal cortex, we did the following two steps: (1) we interpolated the fMRI signals to 10 Hz and temporally realigned them to the fNIRS signals using the task onset time; (2) we normalized both fMRI and fNIRS signals and performed a cross correlation of the normalized signals at each channel with a +/−20-second time lag (Matlab function: xcorr). Figure 3 shows an example of the cross correlation at four channels for a representative subject. The black curves represent the cross correlation between the HbO and BOLD signals that are positively correlated, and the red curves represent the cross correlation between the Hb and BOLD signals that are negatively correlated. The arrows indicate the latency times at which the cross correlations attained their peak/valley values. For instance, the closest peak to the origin of the black curve at channel 6 is at −3.8 seconds (Fig. 3(a)), which indicates that the BOLD signal is best correlated with the HbO signal that was collected 3.8 seconds prior to the BOLD signal. Supplementary Fig. 3(a)-3(d) shows that the optimal latencies varied with channel number and that, in all cases, optimal latencies were shorter than 4.0 seconds. In fact, optimal latencies were shorter than 4.0 seconds for all subjects and channels.

 figure: Fig. 3

Fig. 3 Cross correlation of fNIRS and fMRI signals for a representative subject at four representative channels. In each panel, the arrow indicates the optimum time lag (i.e. the one with the highest correlation) within a task cycle (36 seconds per cycle). The fMRI signal was interpolated to match the fNIRS sample rate (10Hz), and both fMRI and fNIRS signals were normalized before conducting cross-correlation. Notice that all optimum time lags were smaller than 4 seconds.

Download Full Size | PDF

We utilized the cross correlation results to generate a set of input signals for SVR. This set of input signals, named fNIRS-PFC-1, included 80 time-lag fNIRS signals from −4 to + 4 seconds in 0.1-second intervals—a range that included the signal with optimum latency. We performed this procedure on both oxy- and deoxy- hemoglobin signals for all 24 channels. Thus, the total number of input signals to SVR for each subject was 24 × 80 × 2 = 3840.

To demonstrate that SVR prediction using only fNIRS signals at the optimum latency time could also match the accuracy of SVR prediction using BOLD signal, we generated a second set of SVR input signals that included only those fNIRS signals at the optimum latency time. This set of input signals, named fNIRS-PFC-2, contained one HbO signal and one Hb signal for each of the 24 NIRS channels, for a total of 24 × 2 = 48 signals input to the SVR. We then compared prediction accuracy when using this SVR input signal subset against prediction accuracy when using BOLD signal.

3.2 Generating the fMRI-PFC input signal

To retrieve fMRI signals from the fNIRS channel locations on the PFC, we projected the channel markers from the scalp to the brain surface, then grew spherical regions around the projection points with radii of 5 voxels—about 2 cm. We removed any portion outside the brain mask. Functional MRI signals from these voxels were then extracted and averaged. The method has been thoroughly described previously [3].

In total, we obtained 24 fMRI-PFC signals (Fig. 4(c)), each corresponding to one of the 24 fNIRS channels. The signal set was then provided to the SVR pipeline for training and testing.

 figure: Fig. 4

Fig. 4 Process of extracting the fMRI signals from the cortex. (a) Location of uniformly distributed points on a spherical cap. (b) Location of retrieved voxels on the brain surface with uniform distribution. (c) Projection of twenty-four fNIRS channels on the brain. The fMRI signals were extracted from those voxels. (d) Two 3x3 patches were placed over the left and right prefrontal brain region. Source probes (not shown) were plugged in the red holes on the patches, and detector probes (not shown) were plugged in the blue holes.

Download Full Size | PDF

3.3 Generating the fMRI-EC input signal

To retrieve fMRI signals from the entire cortex, we began with the mathematical assumption that the cortex shape can be approximated by a spherical cap. We generated uniformly distributed points on the spherical cap (Stefan Stoll, matlabCentral, stoll@phys.chem.ethz.ch) and projected the points on the brain surface. Figures 4(a)-4(b) illustrates the process when using 565 points. We then extracted the time series signals at those points. These signals became the input to SVR. To evaluate the effect of the number of points on prediction accuracy, we repeated this process and generated predictions using ten different numbers of points: 4,225, 2,353, 1,621, 1,025, 565, 241, 173, 105, 41 and 25 (see also Fig. 11 in the appendix).

3.4 Extracting target signal from twelve deep-brain regions

The regional mask images were created in accordance with the Automated Anatomical Labeling (AAL) atlas [36]. The extracted signal was averaged for each region of interest (ROI) to obtain a single time series. The obtained target signal was used for all three groups of input signals—fNIRS-PFC, fMRI-PFC, and fMRI-EC—in the SVR analysis.

3.5 Statistical analysis to evaluate the prediction performance

Prior to the statistical analysis, the correlation values were converted to Fisher z-statistics using the formula z = tanh−1(r), where r is the correlation coefficient. The Fisher z-transformed correlation coefficients follow a normal distribution with a standard error of 1/N3, where N is the number of time points.

The statistical analysis included both between-group and within-group statistics. We performed two-way analysis of variance (ANOVA) to assess the effects of brain region choice and cognitive task on prediction performance for each input group. We performed the within-group comparisons using paired t-tests.

3.6 Weight vector analysis

Weight vectors were studied for the fMRI-EC case (4,225 fMRI signals from the entire cortex). Weight vectors indicate the importance of the cortical voxels in the prediction model. The higher the weight is, the more critical the associated voxel is for the prediction. To find out the critical voxels for prediction of each ROI in general subjects, we normalized the weight vector for each individual, collected the voxels with high normalized weight, and visualized them on a rendered brain using xjView (http://www.alivelearn.net/xjview).

4. Results

4.1 Prediction performance when using fMRI measurements from the entire cortex (fMRI-EC) to predict deep-brain activity

Figure 5(a) shows prediction performance for twelve different deep-brain regions when using 4,225 fMRI signals from the entire cortex in a single subject. The prediction performance varied across regions and tasks, with the highest correlation, r = 0.90, at left insula and the lowest, r = 0.31, at left parahippocampal gyrus—both during the complex visual task (see Fig. 10 in the appendix). The overall mean correlation was 0.73 across all regions and tasks. The paired t-tests showed that there were no significant differences in predictive ability between tasks in any regions.

 figure: Fig. 5

Fig. 5 Prediction performance at twelve deep-brain regions when using fMRI signals as input to SVR. (a) Prediction performance for a representative subject, and (b) prediction performance averaged across all subjects. The input signals to the SVR prediction model for (a) and (b) were 4,225 fMRI signals from the entire cortex (fMRI-EC signals). (c) The overall mean correlation between predicted and measured signals plotted as a function of the number of input fMRI-EC signals. The correlation coefficients in (c) were averaged across all subjects, regions, and tasks, and the error bars represent the standard deviation. (d) Prediction performance averaged across all subjects, where the input signals to the SVR prediction model were 24 fMRI signals from the prefrontal cortex (fMRI-PFC). The error bars represent the standard deviation.

Download Full Size | PDF

The averaged prediction performances for all seventeen subjects are shown in Fig. 5(b). The result shows an overall good prediction performance, with the highest mean of 0.80 at left fusiform and an overall mean of 0.67 across all regions and tasks. The output from a two-way ANOVA is displayed in Table 1. An examination of this output reveals a significant difference in prediction performance at different regions (F11,2 = 10.51; p < 0.0001); a marginally significant difference in the tasks (F11,2 = 2.88; p = 0.0569); and no significant region × task interaction effects (F11,2 = 0.73; p = 0.8086).

Tables Icon

Table 1. Two-way ANOVA: prediction performance versus deep-brain regions, for fMRI-EC input signal (4,225 fMRI signals extracted uniformly from the entire cortex).

Figure 5(c) illuminates the relationship between the overall prediction performance and the number of cortical signals provided to the SVR model. The mean correlation was obtained by averaging signal across all twelve brain regions, all three tasks, and all subjects. As can be seen, the prediction performance improved with an increasing number of input signals. The lowest mean correlation, r = 0.60, resulted from predictions made with 25 input signals, and the highest, r = 0.67, resulted from those made with 4,225 input signals. The plot of mean correlations exhibits a plateau phase from 241 input signals to 4,225 input signals. Because of the monotonic change of the performance, we only performed the paired t-test between the closest neighbors. The paired t-test showed that the differences in prediction performance were significant between: 25 and 41, 41 and 105, 105 and 173, and 173 and 241 input signals (df = 611, p < 0.05). The results indicate that cortical signals from 241 voxels are sufficient to accurately predict the activity of deep-brain regions (mean r = 0.67) and that inclusion of additional cortical surface signals will not significantly improve prediction accuracy.

4.2 Prediction performance when using fMRI measurements from the prefrontal cortex (fMRI-PFC) to predict deep-brain activity

The prediction performance, when using fMRI-PFC signal as input to the SVR, is presented in Fig. 5(d). The correlation coefficients were averaged across all subjects. The error bars represent the standard deviation. The overall prediction performance was r = 0.56 across all regions and tasks. The two-way ANOVA (Table 2) indicates that there was a significant difference in prediction performance at different regions (F11,2 = 8.53; p < 0.0001). There was neither a significant difference between tasks (F11,2 = 2.11; p = 0.1225) nor a significant region × task interaction (F11,2 = 0.38; p = 0.9958).

Tables Icon

Table 2. Two-way ANOVA: prediction performance versus deep-brain regions, for fMRI-PFC input signal.

4.3 Prediction performance when using fNIRS measurements from the prefrontal cortex (fNIRS-PFC) to predict deep-brain activity

Figure 1(e) shows the number of noisy fNIRS channels for all seventeen subjects. We excluded subjects 2, 3, 11, 12, 13 and 15 from SVR analysis because more than half of their channels were unacceptably noisy.

Figure 6 shows the prediction performance when using fNIRS-PFC-1 and fNIRS-PFC-2 signals as input to the SVR. The overall prediction performances were r = 0.43 and 0.44, respectively, averaged across all regions and tasks. A paired t-test showed that there was no significant difference between those two results (T-value = −1.30, p = 0.19, df = 395). However, both results were inferior to fMRI-PFC results (~23% lower).

 figure: Fig. 6

Fig. 6 Prediction performance when using fNIRS-PFC signals as input to SVR. The correlation coefficients were averaged across eleven subjects (those that remained after exclusion for noise). The error bars represent the standard deviation across the subjects. The top figure is the prediction performance when using the fNIRS-PFC-1 signals, which included 80 time-lag fNIRS signals from −4 to + 4 seconds in 0.1-second intervals (3,840 total signals per subject). The bottom figure is the prediction performance when using the fNIRS-PFC-2 signals, which included only the fNIRS signals at the optimal latency time, i.e., those with the highest correlation with corresponding BOLD signals (48 total signals per subject).

Download Full Size | PDF

4.4 Comparing prediction performance of fMRI and fNIRS input signals

Figure 7(a) shows the distribution of correlation coefficients between predicted and observed signals from the three groups of input signals: fMRI-EC, fMRI-PFC and fNIRS-PFC-1. We removed the subjects 2, 3, 11, 12, 13 and 15 from all three groups for comparison because their fNIRS signals were too noisy. We found a wide range of correlations for all three groups. The percentages of high correlation (higher than 0.5) were 83%, 63%, and 42% for fMRI-EC, fMRI-PFC, and fNIRS-PFC-1, respectively. The overall mean correlations are shown in Fig. 7(b). The fMRI signal extracted uniformly from the entire cortex (fMRI-EC) yielded the highest correlation, i.e. the best prediction performance among those three. The mean correlations from the top 60 of the 396 (3 tasks × 12 regions × 11 subjects = 396) values (15%) are also shown in Fig. 7(b). When averaged over the top 15%, all three groups had mean correlations higher than 0.7.

 figure: Fig. 7

Fig. 7 Distribution and mean of correlations between predicted and observed signals for three groups of input signals: fMRI-EC, fMRI-PFC, and fNIRS-PFC-1. (a) Distribution of correlations for the three groups of input signal. The white, vertical, dashed lines indicate 0.5-correlation. (b) Mean correlations for the three groups of input signals. Light blue bars indicate the averaged correlation coefficients across all subjects, regions and tasks, i.e., averaged across 396 values. Magenta bars indicate the mean correlations averaged over the top 15% of the 396 correlation values.

Download Full Size | PDF

We also compared the prediction performance achieved when using fMRI signals from the entire cortex against the prediction performance achieved when using fMRI signals from only the prefrontal cortex. To do this, we used the fMRI-EC data set with 25 signals in order to match the number of signals (24) in the fMRI-PFC data set. Mean correlations across all twelve regions were 0.60 and 0.56 for fMRI-EC and fMRI-PFC, respectively. A paired t-test showed a significant difference between the two cases (p = 2.43 × 10−13, df = 611, T-value = 7.49), with fMRI-EC data providing superior predictive ability than that provided by fMRI-PFC (95% CI for difference: 0.051~0.089).

4.5 Weight analysis to study the generalization of the model

Figures 8(a)-8(c) shows the normalized weight (> 0.6) associated with predicting activity in a representative brain region, the left fusiform, in a representative subject during all three tasks. In all three conditions, high-weight voxels were located in the visual cortex.

 figure: Fig. 8

Fig. 8 Weight maps associated with predicting: (a-c) left fusiform activity in a representative subject during nogo, faces, and complex-visual tasks; (d) left and (e) right fusiform activity, averaged over all subjects and all tasks; and the activity at (f) insula, (g) hippocampus, (h) parahippocampal gyrus, (i) amygdala and (j) caudate, averaged over all subjects and all tasks. The weights combine voxels for left and right predictions in (f-j). Red points mark the voxels with normalized weight > 0.6 in the top panel, and > 0.7 in the bottom panel.

Download Full Size | PDF

Figures 8(d)-8(e) shows the weight map associated with predicting activity in the fusiform (left and right), averaged over all subjects and all three tasks. The results indicate that visual cortex activity is predominant in predictions of fusiform activity.

Figures 8(f)-8(j) shows the weight maps associated with prediction of activity in all other deep-brain regions included in the study. For each of the six regions, prediction of left/right counterparts yielded similar weight maps, so weights associated with left and right predictions were plotted together. Only voxels with normalized weight > 0.7 are shown. Table 3 lists cortical regions contributing highly (according to those weight maps) to prediction of activity in each of the six deep-brain regions.

Tables Icon

Table 3. Summary of the cortical regions with large weight (> 0.7) in predictions of activity in the six deep-brain regions.

5. Discussion

This study is the first to investigate the feasibility of using superior and lateral cortical activity to infer deep-brain activity. We developed a prediction method based on an SVR learning algorithm. We demonstrate and validate the method for predicting the activity of twelve deep-brain regions using cortical surface activity, as measured either by fNIRS or fMRI. We developed this method to mitigate the limitations that arise from fNIRS’ inability to directly measure activity in deep-brain regions.

5.1 Cortical activity can be used to predict deep-brain activity

Our results confirm that one can use cortical activity to predict deep-brain activity. When using fMRI-measured brain activity from the entire cortex, our SVR algorithm inferred fusiform activity that correlated highly with actual fusiform activity (mean correlation: 0.8). Prediction performance across all regions was also strong (mean correlation with actual activity: 0.67). These high correlations demonstrate the feasibility of the developed method.

Although the predictive ability of the fNIRS-PFC signal was inferior to that of each fMRI signal and there was a wide range in correlation values (Fig. 7(a)), 42% of predictions made using fNIRS-PFC signal reached high accuracy (correlation r > 0.5). In addition, the top 15% of correlation values from fNIRS-PFC-based predictions achieved a mean correlation greater than or equal to 0.7. Taken together, these findings suggest great potential for fNIRS-based prediction of deep-brain activity.

5.2 Choice of brain region and cognitive task affects prediction performance

The results from two-way ANOVA reveal significant differences in prediction performance between different deep-brain regions and marginally significant differences in prediction performance between cognitive tasks (Table 1). Furthermore, the type of input signals to SVR affected which deep-brain activity was predicted most accurately. When using the signal from the entire cortex, predictions of fusiform activity were most accurate (Fig. 5(b)). When using signals from prefrontal cortex, predictions of insula activity were most accurate (Fig. 5(d)). A multi-comparison of means among regions (Matlab function: multcompare) also confirmed this observation.

Relative success of fusiform and insula activity predictions may arise from morphological properties of those regions. Of the twelve regions we included in our study, the fusiform has the largest volume, and the insula is closest both to the cortical surface and the prefrontal cortex. Accuracy of the predictions may be sensitive to size and location of both the target and input brain regions.

5.3 Utilizing the entire cortex yields higher overall prediction accuracy for multiple deep-brain regions than does utilizing partial cortex

We were better able to predict deep-brain activity when using fMRI-EC signals than we were when using fMRI-PFC signals, even when we restricted the fMRI-EC data to 25 signals, which is comparable to the number of signals (24) in the fMRI-PFC data set. In other words, 25 fMRI signals spread over the entire cortex enabled better prediction than did 24 fNIRS signals from the PFC. From this result, we expect that, if one were able to take fNIRS measurements from the entire cortex, the overall prediction accuracy for multiple brain regions would improve. We consider this a potential method to improve the prediction performance of fNIRS signal.

5.4 Predictions of activity in different deep-brain regions require different optode configurations

In order to achieve high prediction accuracy for a specific deep-brain region, one needs to include signals from certain cortical regions as input to the SVR prediction process. For instance, to infer fusiform activity, one should include signal from visual cortex (occipital) in the model [37]; and to infer insular activity, one should include signals from the temporo-parietal junction [38], inferior parietal cortex [39], and superior occipital cortex (Table 3).

Thus, we arrive at a heuristic for placing fNIRS optodes: if the region of interest includes a brain network or multiple brain regions, the fNIRS optodes should be uniformly distributed across the entire cortical surface to achieve overall good prediction performance; if the region of interest includes only one or two individual deep-brain regions, the fNIRS optodes should be placed on the cortical regions with large weights associated with SVR predictions.

5.5 Limitations and future directions

In order to improve prediction accuracy, one might further optimize the correlation between fMRI and fNIRS. Several factors likely influenced the correlation between fMRI and fNIRS signals at the cortex. For instance, the contrast mechanisms of fMRI and fNIRS are different, although they share the same contrast source, hemoglobin. In fMRI, the BOLD contrast is the difference in signal on T2*-weighted images as a function of the amount of deoxygenated hemoglobin [40]. Hence, fMRI is not sensitive to arterial blood. In fNIRS, both oxy- and deoxy- hemoglobin contribute to the measured signal change, so fNIRS is sensitive to more vascular dynamics than fMRI. Furthermore, the topographic fNIRS signal measured in the study contains superficial signals from the scalp and the skull, while BOLD signal does not contain this confound. The aforementioned and other fNIRS measurement confounds, such as vasomotion, may also have influenced correlations between fNIRS and fMRI in our experiments. Although it is hard to remove all of the physiological noise related to cardiac and respiratory processes from fMRI and fNIRS signals, we utilized RETROICOR [27] to extract and remove these noise components from the fNIRS signals. Figure 9 shows that the physiological noise was very small, and thus is unlikely to have influenced our results. Future experiments will involve the development of methods (such as utilizing independent component analysis [41] or adaptive filtering [42,43]) and instruments to reduce superficial artifacts in fNIRS signal and improve the correlation between fNIRS and fMRI cortical signals.

 figure: Fig. 9

Fig. 9 The fNIRS signal (HbO) change before and after respiratory denoising for a representative subject. The HbO signals were first down-sampled to match the fMRI BOLD signal sampling rate, then the standard RETROICOR method (Glover et al., 2000) was used to remove respiratory noise. The corresponding BOLD signal was used as a benchmark to identify the efficiency of the denoising. Specifically, the mean correlation between the HbO signal and the corresponding BOLD signal across all channels was calculated for both uncorrected and corrected HbO signals. A paired t-test between those two cases (utilizing Fisher z-transformed correlation coefficients) indicated that there was no difference between them (p = 0.13, df = 23, T-value = 1.56). The result implies that the global physiological noise was very small and thus was likely not a confound that influenced the findings or the resulting conclusions. (a) The HbO signals before (blue) and after (red) RETROICOR correction for one representative channel. (b) The mean correlation between HbO signal and the corresponding BOLD signal across all channels for both uncorrected and corrected HbO signals. The error bar represents the standard deviation.

Download Full Size | PDF

In order to apply our prediction method, we recommend the following steps: (1) perform a concurrent fMRI and fNIRS scan to generate the prediction model for each individual; (2) determine the optimal latency time between fNIRS and BOLD signals at the common cortical locations; (3) determine the SVR weights and generate the prediction model for each individual; (4) use the generated prediction model to infer deep-brain activity utilizing only fNIRS in subsequent studies that require repeated measurements, such as monitoring a treatment.

To further generalize the method, one might attempt to create a robust, non-subject-specific prediction model for each deep-brain region. In the current study, each prediction model was based on an individual data set. But interestingly, the SVR weight results (Fig. 8) revealed that high-weight voxels from individual weight maps were largely overlapping. Thus, further experimentation may reveal that SVR weights for a deep-brain region can be successfully trained on a group data set. If so one could generate a robust prediction model for an entire experimental group.

6. Conclusion

In this study we developed a novel computational method to infer subcortical and inferior cortical brain activity from superior cortical surface activity measured by fNIRS and fMRI. Our results demonstrate that fMRI-measured cortical signal can be used to infer deep-brain signal. The highest mean correlation across all subjects between predicted and observed signals reached 0.8 at left fusiform. Our results also show that fNIRS-measured cortical signal from the prefrontal cortex can be used to infer deep-brain signals. Although the predictive ability of the fNIRS-PFC signal was inferior to that of each fMRI signal, 42% of predictions made using fNIRS-PFC signal reached high accuracy (correlation r > 0.5). In addition, the top 15% of correlation values from fNIRS-PFC-based predictions achieved a mean correlation greater than or equal to 0.7. Further development of this method has potential to significantly extend the application of fNIRS in cognitive and clinical neuroscience research.

Appendix

 figure: Fig. 10

Fig. 10 Two representative cases with different prediction performance. The blue lines represent the measured BOLD signals (Target signals), and the green lines represent the predicted signals. (a) The correlation coefficient between the target signal and the predicted signal was very high: r = 0.99. (b) The correlation coefficient between the target signal and the predicted signal was moderate: r = 0.79.

Download Full Size | PDF

 figure: Fig. 11

Fig. 11 Location of uniformly distributed points on a spherical cap. (a) 241 points; (b) 565 points.

Download Full Size | PDF

Acknowledgments

This work was supported by the Stanford Institute for Neuro-Innovation and Translational Neurosciences (SINTN) fellowship (X.C. and A.L.R.) and S10 RR024657 (A.L.R.), P41 EB015891 (G.H.G.). We thank Albert Yu and Mary Bechmann Foundation and Autism Speaks (N.L.) for their financial support. We thank Drs. Paul K. Mazaika and Amy Garrett for useful discussion of data analysis. We also thank Hitachi Medical for the use of a MRI-compatible set of optodes and fibers. The authors have no conflicts of interest to declare.

References and links

1. M. Okamoto, H. Dan, K. Shimizu, K. Takeo, T. Amita, I. Oda, I. Konishi, K. Sakamoto, S. Isobe, T. Suzuki, K. Kohyama, and I. Dan, “Multimodal assessment of cortical activation during apple peeling by NIRS and fMRI,” Neuroimage 21(4), 1275–1288 (2004). [CrossRef]   [PubMed]  

2. S. Del Bianco, F. Martelli, and G. Zaccanti, “Penetration depth of light re-emitted by a diffusive medium: theoretical and experimental investigation,” Phys. Med. Biol. 47(23), 4131–4144 (2002). [CrossRef]   [PubMed]  

3. X. Cui, S. Bray, D. M. Bryant, G. H. Glover, and A. L. Reiss, “A quantitative comparison of NIRS and fMRI across multiple cognitive tasks,” Neuroimage 54(4), 2808–2821 (2011). [CrossRef]   [PubMed]  

4. P. R. Montague, G. S. Berns, J. D. Cohen, S. M. McClure, G. Pagnoni, M. Dhamala, M. C. Wiest, I. Karpov, R. D. King, N. Apple, and R. E. Fisher, “Hyperscanning: simultaneous fMRI during linked social interactions,” Neuroimage 16(4), 1159–1164 (2002). [CrossRef]   [PubMed]  

5. X. Cui, D. M. Bryant, and A. L. Reiss, “NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation,” Neuroimage 59(3), 2430–2437 (2012). [CrossRef]   [PubMed]  

6. B. B. Biswal, M. Mennes, X. N. Zuo, S. Gohel, C. Kelly, S. M. Smith, C. F. Beckmann, J. S. Adelstein, R. L. Buckner, S. Colcombe, A. M. Dogonowski, M. Ernst, D. Fair, M. Hampson, M. J. Hoptman, J. S. Hyde, V. J. Kiviniemi, R. Kötter, S. J. Li, C. P. Lin, M. J. Lowe, C. Mackay, D. J. Madden, K. H. Madsen, D. S. Margulies, H. S. Mayberg, K. McMahon, C. S. Monk, S. H. Mostofsky, B. J. Nagel, J. J. Pekar, S. J. Peltier, S. E. Petersen, V. Riedl, S. A. Rombouts, B. Rypma, B. L. Schlaggar, S. Schmidt, R. D. Seidler, G. J. Siegle, C. Sorg, G. J. Teng, J. Veijola, A. Villringer, M. Walter, L. Wang, X. C. Weng, S. Whitfield-Gabrieli, P. Williamson, C. Windischberger, Y. F. Zang, H. Y. Zhang, F. X. Castellanos, and M. P. Milham, “Toward discovery science of human brain function,” Proc. Natl. Acad. Sci. U.S.A. 107(10), 4734–4739 (2010). [CrossRef]   [PubMed]  

7. E. Bullmore and O. Sporns, “Complex brain networks: graph theoretical analysis of structural and functional systems,” Nat. Rev. Neurosci. 10(3), 186–198 (2009). [CrossRef]   [PubMed]  

8. H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V. Vapnik, “Support vector regression machines,” Adv. Neural Inf. Process. Syst. 9(9), 155–161 (1997).

9. K. R. Müller, A. J. Smola, G. Rätsch, B. Schölkopf, J. Kohlmorgen, and V. Vapnik, “Predicting time series with support vector machines,” in Artificial Neural Networks — ICANN'97, Gerstner W, Germond A, Hasler M, Nicoud J-D, ed. (Springer Berlin Heidelberg, 1997), pp. 999–1004.

10. A. J. Smola and B. Schölkopf, “A tutorial on support vector regression,” Stat. Comput. 14(3), 199–222 (2004). [CrossRef]  

11. V. Vapnik and A. Chervonenkis, “A note on one class of perceptrons,” Autom. Remote Control 25, 25 (1964).

12. V. Vapnik and A. Lerner, “Pattern recognition using generalized portrait method,” Autom. Remote Control 24, 774 (1963).

13. F. Pereira, T. Mitchell, and M. Botvinick, “Machine learning classifiers and fMRI: a tutorial overview,” Neuroimage 45(1Suppl), S199–S209 (2009). [CrossRef]   [PubMed]  

14. Y. Kamitani and F. Tong, “Decoding the visual and subjective contents of the human brain,” Nat. Neurosci. 8(5), 679–685 (2005). [CrossRef]   [PubMed]  

15. J. Mourão-Miranda, A. L. Bokde, C. Born, H. Hampel, and M. Stetter, “Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data,” Neuroimage 28(4), 980–995 (2005). [CrossRef]   [PubMed]  

16. R. A. Poldrack, Y. O. Halchenko, and S. J. Hanson, “Decoding the large-scale structure of brain function by classifying mental States across individuals,” Psychol. Sci. 20(11), 1364–1372 (2009). [CrossRef]   [PubMed]  

17. R. Chaves, J. Ramírez, J. M. Górriz, M. López, D. Salas-Gonzalez, I. Alvarez, and F. Segovia, “SVM-based computer-aided diagnosis of the Alzheimer’s disease using t-test NMSE feature selection with feature correlation weighting,” Neurosci. Lett. 461(3), 293–297 (2009). [CrossRef]   [PubMed]  

18. C. Ecker, A. Marquand, J. Mourão-Miranda, P. Johnston, E. M. Daly, M. J. Brammer, S. Maltezos, C. M. Murphy, D. Robertson, S. C. Williams, and D. G. Murphy, “Describing the Brain in Autism in Five Dimensions--Magnetic Resonance Imaging-Assisted Diagnosis of Autism Spectrum Disorder Using a Multiparameter Classification Approach,” J. Neurosci. 30(32), 10612–10623 (2010). [CrossRef]   [PubMed]  

19. S. Kloppel, C. M. Stonnington, C. Chu, B. Draganski, R. I. Scahill, J. D. Rohrer, N. C. Fox, C. R. Jack, J. Ashburner, and R. S. J. Frackowiak, “Automatic classification of MR scans in Alzheimers disease,” Brain 131(Pt 3), 681–689 (2008).

20. A. S. Garrett, A. L. Reiss, D. J. Miklowitz, T. K. Acquaye, V. E. Cosgrove, M. K. Singh, M. E. Howe, R. G. Kelley, D. Taylor, E. George, et al.., “Changes in Brain Activation Following Family-Focused Treatment in Youth At-Risk for Bipolar Disorder,” Biol. Psychiatry 69(9), 163S (2011).

21. B. W. Haas, F. Hoeft, Y. M. Searcy, D. Mills, U. Bellugi, and A. L. Reiss, “Individual differences in social behavior predict amygdala response to fearful facial expressions in Williams syndrome,” Neuropsychologia 48(5), 1283–1288 (2010). [CrossRef]   [PubMed]  

22. F. Hoeft, A. Hernandez, S. Parthasarathy, C. L. Watson, S. S. Hall, and A. L. Reiss, “Fronto-striatal dysfunction and potential compensatory mechanisms in male adolescents with fragile X syndrome,” Hum. Brain Mapp. 28(6), 543–554 (2007). [CrossRef]   [PubMed]  

23. D. T. Delpy, M. Cope, P. van der Zee, S. Arridge, S. Wray, and J. Wyatt, “Estimation of Optical Pathlength through Tissue from Direct Time of Flight Measurement,” Phys. Med. Biol. 33(12), 1433–1442 (1988). [CrossRef]   [PubMed]  

24. C. Chang and G. H. Glover, “Time-frequency dynamics of resting-state brain connectivity measured with fMRI,” Neuroimage 50(1), 81–98 (2010). [CrossRef]   [PubMed]  

25. A. Grinsted, J. C. Moore, and S. Jevrejeva, “Application of the cross wavelet transform and wavelet coherence to geophysical time series,” Nonlinear Process. Geophys. 11(5/6), 561–566 (2004). [CrossRef]  

26. G. H. Glover and C. S. Law, “Spiral-in/out BOLD fMRI for increased SNR and reduced susceptibility artifacts,” Magn. Reson. Med. 46(3), 515–522 (2001). [CrossRef]   [PubMed]  

27. G. H. Glover, T. Q. Li, and D. Ress, “Image-based method for retrospective correction of physiological motion effects in fMRI: RETROICOR,” Magn. Reson. Med. 44(1), 162–167 (2000). [CrossRef]   [PubMed]  

28. C. Chang, J. P. Cunningham, and G. H. Glover, “Influence of heart rate on the BOLD signal: The cardiac response function,” Neuroimage 44(3), 857–869 (2009). [CrossRef]   [PubMed]  

29. A. J. Smola and B. Scholkopf, “A tutorial on support vector regression,” Stat. Comput. 14(3), 199–222 (2004). [CrossRef]  

30. C. C. Chang and C. J. Lin, “Training nu-support vector regression: Theory and algorithms,” Neural Comput. 14(8), 1959–1977 (2002). [CrossRef]   [PubMed]  

31. C. C. Chang and C. J. Lin, “LIBSVM: A Library for Support Vector Machines,” ACM Trans. Intelligent Systems Technology. 2(3), 1–27 (2011). [CrossRef]  

32. A. Kleinschmidt, H. Obrig, M. Requardt, K. D. Merboldt, U. Dirnagl, A. Villringer, and J. Frahm, “Simultaneous recording of cerebral blood oxygenation changes during human brain activation by magnetic resonance imaging and near-infrared spectroscopy,” J. Cereb. Blood Flow Metab. 16(5), 817–826 (1996). [CrossRef]   [PubMed]  

33. L. Minati, E. Visani, N. G. Dowell, N. Medford, and H. D. Critchley, “Variability comparison of simultaneous brain near-infrared spectroscopy and functional magnetic resonance imaging during visual stimulation,” J. Med. Eng. Technol. 35(6-7), 370–376 (2011). [CrossRef]   [PubMed]  

34. J. Steinbrink, A. Villringer, F. Kempf, D. Haux, S. Boden, and H. Obrig, “Illuminating the BOLD signal: combined fMRI-fNIRS studies,” Magn. Reson. Imaging 24(4), 495–505 (2006). [CrossRef]   [PubMed]  

35. M. Wolf, U. Wolf, V. Toronov, A. Michalos, L. A. Paunescu, J. H. Choi, and E. Gratton, “Different time evolution of oxyhemoglobin and deoxyhemoglobin concentration changes in the visual and motor cortices during functional stimulation: a near-infrared spectroscopy study,” Neuroimage 16(3), 704–712 (2002). [CrossRef]   [PubMed]  

36. N. Tzourio-Mazoyer, B. Landeau, D. Papathanassiou, F. Crivello, O. Etard, N. Delcroix, B. Mazoyer, and M. Joliot, “Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain,” Neuroimage 15(1), 273–289 (2002). [CrossRef]   [PubMed]  

37. K. Grill-Spector, T. Kushnir, T. Hendler, S. Edelman, Y. Itzchak, and R. Malach, “A sequence of object-processing stages revealed by fMRI in the human occipital lobe,” Hum. Brain Mapp. 6(4), 316–328 (1998). [CrossRef]   [PubMed]  

38. J. Downar, A. P. Crawley, D. J. Mikulis, and K. D. Davis, “A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities,” J. Neurophysiol. 87(1), 615–620 (2002). [PubMed]  

39. S. Caspers, S. Geyer, A. Schleicher, H. Mohlberg, K. Amunts, and K. Zilles, “The human inferior parietal cortex: cytoarchitectonic parcellation and interindividual variability,” Neuroimage 33(2), 430–448 (2006). [CrossRef]   [PubMed]  

40. S. A. Huettel, A. W. Song, and G. McCarthy, Functional Magnetic Resonance Imaging (Sinauer Associates, Incorporated, 2009).

41. S. Kohno, I. Miyai, A. Seiyama, I. Oda, A. Ishikawa, S. Tsuneishi, T. Amita, and K. Shimizu, “Removal of the skin blood flow artifact in functional near-infrared spectroscopic imaging data through independent component analysis,” J. Biomed. Opt. 12(6), 062111 (2007). [CrossRef]   [PubMed]  

42. F. Tian, H. Niu, B. Khan, G. Alexandrakis, K. Behbehani, and H. Liu, “Enhanced functional brain imaging by using adaptive filtering and a depth compensation algorithm in diffuse optical tomography,” IEEE Trans. Med. Imaging 30(6), 1239–1251 (2011). [CrossRef]   [PubMed]  

43. Q. Zhang, G. E. Strangman, and G. Ganis, “Adaptive filtering to reduce global interference in non-invasive NIRS measures of brain activation: How well and when does it work?” Neuroimage 45(3), 788–794 (2009). [CrossRef]   [PubMed]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1 Noise determination. (a) The wavelet transform map of a poor-quality fNIRS signal; note that there is no heartbeat frequency band at 1 Hz. (b) The wavelet transform map of a good-quality fNIRS signal; note that there is a clear heartbeat frequency band showed at 1 Hz. (c) and (d) are the time series corresponding to (a) and (b). The blue curves represent the original signals, and the red curves represent the signals after bandpass filtering. (e) Number of noisy fNIRS channels for all subjects.
Fig. 2
Fig. 2 Diagram of SVR algorithm.
Fig. 3
Fig. 3 Cross correlation of fNIRS and fMRI signals for a representative subject at four representative channels. In each panel, the arrow indicates the optimum time lag (i.e. the one with the highest correlation) within a task cycle (36 seconds per cycle). The fMRI signal was interpolated to match the fNIRS sample rate (10Hz), and both fMRI and fNIRS signals were normalized before conducting cross-correlation. Notice that all optimum time lags were smaller than 4 seconds.
Fig. 4
Fig. 4 Process of extracting the fMRI signals from the cortex. (a) Location of uniformly distributed points on a spherical cap. (b) Location of retrieved voxels on the brain surface with uniform distribution. (c) Projection of twenty-four fNIRS channels on the brain. The fMRI signals were extracted from those voxels. (d) Two 3x3 patches were placed over the left and right prefrontal brain region. Source probes (not shown) were plugged in the red holes on the patches, and detector probes (not shown) were plugged in the blue holes.
Fig. 5
Fig. 5 Prediction performance at twelve deep-brain regions when using fMRI signals as input to SVR. (a) Prediction performance for a representative subject, and (b) prediction performance averaged across all subjects. The input signals to the SVR prediction model for (a) and (b) were 4,225 fMRI signals from the entire cortex (fMRI-EC signals). (c) The overall mean correlation between predicted and measured signals plotted as a function of the number of input fMRI-EC signals. The correlation coefficients in (c) were averaged across all subjects, regions, and tasks, and the error bars represent the standard deviation. (d) Prediction performance averaged across all subjects, where the input signals to the SVR prediction model were 24 fMRI signals from the prefrontal cortex (fMRI-PFC). The error bars represent the standard deviation.
Fig. 6
Fig. 6 Prediction performance when using fNIRS-PFC signals as input to SVR. The correlation coefficients were averaged across eleven subjects (those that remained after exclusion for noise). The error bars represent the standard deviation across the subjects. The top figure is the prediction performance when using the fNIRS-PFC-1 signals, which included 80 time-lag fNIRS signals from −4 to + 4 seconds in 0.1-second intervals (3,840 total signals per subject). The bottom figure is the prediction performance when using the fNIRS-PFC-2 signals, which included only the fNIRS signals at the optimal latency time, i.e., those with the highest correlation with corresponding BOLD signals (48 total signals per subject).
Fig. 7
Fig. 7 Distribution and mean of correlations between predicted and observed signals for three groups of input signals: fMRI-EC, fMRI-PFC, and fNIRS-PFC-1. (a) Distribution of correlations for the three groups of input signal. The white, vertical, dashed lines indicate 0.5-correlation. (b) Mean correlations for the three groups of input signals. Light blue bars indicate the averaged correlation coefficients across all subjects, regions and tasks, i.e., averaged across 396 values. Magenta bars indicate the mean correlations averaged over the top 15% of the 396 correlation values.
Fig. 8
Fig. 8 Weight maps associated with predicting: (a-c) left fusiform activity in a representative subject during nogo, faces, and complex-visual tasks; (d) left and (e) right fusiform activity, averaged over all subjects and all tasks; and the activity at (f) insula, (g) hippocampus, (h) parahippocampal gyrus, (i) amygdala and (j) caudate, averaged over all subjects and all tasks. The weights combine voxels for left and right predictions in (f-j). Red points mark the voxels with normalized weight > 0.6 in the top panel, and > 0.7 in the bottom panel.
Fig. 9
Fig. 9 The fNIRS signal (HbO) change before and after respiratory denoising for a representative subject. The HbO signals were first down-sampled to match the fMRI BOLD signal sampling rate, then the standard RETROICOR method (Glover et al., 2000) was used to remove respiratory noise. The corresponding BOLD signal was used as a benchmark to identify the efficiency of the denoising. Specifically, the mean correlation between the HbO signal and the corresponding BOLD signal across all channels was calculated for both uncorrected and corrected HbO signals. A paired t-test between those two cases (utilizing Fisher z-transformed correlation coefficients) indicated that there was no difference between them (p = 0.13, df = 23, T-value = 1.56). The result implies that the global physiological noise was very small and thus was likely not a confound that influenced the findings or the resulting conclusions. (a) The HbO signals before (blue) and after (red) RETROICOR correction for one representative channel. (b) The mean correlation between HbO signal and the corresponding BOLD signal across all channels for both uncorrected and corrected HbO signals. The error bar represents the standard deviation.
Fig. 10
Fig. 10 Two representative cases with different prediction performance. The blue lines represent the measured BOLD signals (Target signals), and the green lines represent the predicted signals. (a) The correlation coefficient between the target signal and the predicted signal was very high: r = 0.99. (b) The correlation coefficient between the target signal and the predicted signal was moderate: r = 0.79.
Fig. 11
Fig. 11 Location of uniformly distributed points on a spherical cap. (a) 241 points; (b) 565 points.

Tables (3)

Tables Icon

Table 1 Two-way ANOVA: prediction performance versus deep-brain regions, for fMRI-EC input signal (4,225 fMRI signals extracted uniformly from the entire cortex).

Tables Icon

Table 2 Two-way ANOVA: prediction performance versus deep-brain regions, for fMRI-PFC input signal.

Tables Icon

Table 3 Summary of the cortical regions with large weight (> 0.7) in predictions of activity in the six deep-brain regions.

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

min w f(w), f(w) 1 2 w T w+C( νε+ 1 m i=1 m ( ξ i + ξ i * ) )
( w T ϕ( x i )+b ) y i ε+ ξ i , y i ( w T ϕ( x i )+b )ε+ ξ i * , ξ i , ξ i * 0,i=1,...,m,ε0.
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.