1. Introduction

Blood flow is one of the paramount parameters monitored for diseases diagnosis or treatment follow-up in various medical fields. For instance, blood flow measurement is an indicator of the skin’s healing capability from burns, tissue’s viability during surgery or an impaired vasodilatory response in metabolic diseases such as diabetes [1].

Laser speckle contrast imaging (LSCI) has emerged as a non-invasive, contactless, and tracer-less technique for blood flow assessment in micro-vessels [2]. It is based on the reflectance imaging of tissues illuminated with coherent light. Speckle patterns arise from the interference of backscattered photons with different path lengths. Moving particles in the field of view, i.e., red blood cells (RBCs), cause spatiotemporal fluctuations of the speckle pattern in the image plane. In the spatial domain, these fluctuations can be seen as image blurring. In LSCI, the spatial speckle contrast (K) quantifies the extent of this spatial blurring and is defined as:

Despite its better accuracy compared to LSCI, MESI suffers from the need to acquire several exposure images followed by a pixelwise non-linear fit to a multiparametric model. MESI acquisition can be cumbersome and subsequent data analysis is computationally intensive, preventing real-time analysis. Therefore, most laboratory systems and all commercial speckle imaging systems either for pre-clinical or clinical applications still rely on LSCI. For fast MESI data acquisition, previous developments have used custom fast CMOS sensors associated to field programmable gate arrays (FPGA). These architectures allow MESI data recording in a synthetic exposure mode, strongly decreasing the duration required for acquisition of a set of multi-exposure frames [8,10]. Implementation of MESI synthetic exposure acquisition with a standard CMOS was also demonstrated provided that noise contributions are accurately evaluated and subtracted [11]. For fast computation of blood flow maps from MESI data, recent developments have focused on using neural networks to bypass the computer intensive burden of the non-linear fitting step. Most of the proposed networks were trained using simulated MESI speckle data [12,13] or using experimental LSCI speckle data obtained experimentally for flows in a single large diameter channel [14]. Surprisingly, most reported studies used simple artificial neural networks (ANNs), whereas convolutional neural networks (CNNs) are inherently expected to perform better for image analysis due to their ability to extract local relevant features automatically [15]. In recent years, CNN has gained considerable attention in image-to-image translation, such as denoising, super-resolution, and prediction [16] and has already found application in laser speckle reduction [17] and denoising of laser speckle imaging [18]. Recently, a study has proposed the use of CNN for analysis of speckle contrast imaging data. The network was trained and validated in vitro with experimental speckle data obtained using a disk covered with diffusive paint with controlled rotation speeds to emulate flows [19]. The aim of the present study is to evaluate the performances of two architectures of CNNs and compare them to the non-linear fit approach to derive blood flow maps from in vivo MESI data. First, we build an experimental dataset of MESI contrast images using microfluidic phantoms for a physiological range of blood vessels diameters and flows. Second, we use this dataset to train different CNN architectures. Third, we compare the performances of the CNN approach to the non-linear fit approach. Two residual CNN architectures (multiple-stream residual CNN and three-dimensional residual CNN) focusing on spatial and exposure times features were tested to predict blood flow maps from MESI data. Finally, the analysis of in vivo MESI data was carried out with the CNN architecture with the best performances and compared to the non-linear fit approach.

2. Materials and methods

2.1 MESI set-up and speckle contrast images database building

The imaging set-up is showed in Fig. 1. A 633 nm laser diode (SGL, Shanghai, China) is used. Images were acquired with a sCMOS camera (Orca Flash 2.8, Hamamatsu, Japan) through a stereomicroscope (Leica MZ16). Images are composed of 1920 × 1440 square pixels. MESI data were obtained for the following exposure times: 1, 2, 5, 10, 15, and 25 and 50 ms and for two magnification levels of the optics (×2.5 and ×6) corresponding to fields of view of 3 × 2.1 mm² and 1.1 × 0.9 mm² respectively. For all conditions, speckle contrast images K at different exposure times were generated using the MESI synthetic exposure technique as detailed in [11] with a 5 × 5 sliding window and Eq. (1).

 

Fig. 1. MESI synthetic exposure imaging set-up. A speckle contrast images database was acquired for different exposure times, flows and channels diameter using a microfluidic system (pressure flow control, flowmeter, and a microfluidic chip) with 2%-intralipid as flowing media. Insert: microfluidic chip including 40, 60, 80, 100, 300 and 500 µm diameter channels.

Download Full Size | PDF

The microfluidic set-up was described in detail previously [9]. Speckle contrast images were obtained for various flows in microfluidic channels with diameters of 40, 60, 80, 100, 300 and 500 µm. We used 2%-intralipid as flowing media as it provides a reduced scattering optical coefficient close to that of flowing blood [20,21]. For the database construction, flows of 1, 2, 3, 4, 5, 6, 7 and 8 µl/min have been used. These flows are representative of microcirculation blood velocities in the mice brain (speed ranging from 0.5 to 30 mm/s) [7,22]. All flow values have been used to generate the database except for the 40 µm channel where the highest achievable flow with reliable stability was 5 µl/min. Representative speckle contrast images from the database are shown in Fig. 2. The complete dataset is available on an institutional repository and referenced as Dataset 1 [23].

 

Fig. 2. Speckle contrast database sample images. Representative speckle contrast images are shown for Q = 2 µl/min and Q = 5 µl/min), 2 different channels diameters Φ = 100 µm and Φ = 300 µm and two magnification levels (×2.5 and ×6). In each case, contrast images are shown for three exposure times (T = 1 ms, T = 5 ms and T = 15 ms).

Download Full Size | PDF

2.2 Non-linear fitting on speckle contrast data for blood flow estimation

Contrast images are computed using Eq. (1). A sliding spatial window of 5 × 5 pixels is used for the calculation of the local contrast over the whole image. For exposure times greater than 1 ms, the speckle contrast is calculated after frames summation and noises correction as detailed in [11]. Equation (2) is fitted to MESI data as a function of exposure time.

2.3 Framework of CNN-based blood flow prediction from MESI data

2.3.1 Patch extraction and reference flow map preparation

The overall architecture of the proposed patch-based blood flow prediction from MESI data is shown in Fig. 3. CNN-based methods need large numbers of samples. This requirement cannot be easily met in practice, especially for pre-clinical optical imaging. In this study, we proposed to use randomly sampling patches in the speckle contrast images. The advantage of this strategy is that it detects the small features of local regions while increasing the number of samples considerably. For supervised training, input pairs of speckle contrast images and ground truth flow maps are required. Experimental speckle images are acquired from channels with controlled geometrical parameters and with controlled flows accurately measured in-line. Yet, this value is an average flow inside all the channel. The reference velocity maps accounting for velocities distributions inside the channels were generated by numerically solving the Navier-Stokes equation for fluid dynamics in laminar permanent flow based on the geometry and average flow rate of microchannels. A rigid transformation was then used to align the generated reference image with the channel position in the contrast images. During the network training phase, patches (128 × 128 pixels) containing the microfluidic channel are randomly selected from speckle contrast images, while the corresponding patches for the learning target are simultaneously extracted from the reference flow map at the same location. Data augmentation with image rotation was further applied to simulate different orientations of blood vessels within the speckle images. During the prediction phase, the speckle contrast image was spilt into uniform half-overlapped patches that were then fed into the CNN in sequential order. A full flow map is then assembled from the predicted flow patches using the 64 × 64 central parts of the 128 × 128 patches inferred. This allows to have no overlap on the output map, yet no visible border effects as well due to the use of overlapping inputs. A flowchart of patches inference and re-assembly is provided as supplementary Figure S2.

 

Fig. 3. The working pipeline of the proposed patch-based flow prediction method. Patches (128 × 128 pixels) for speckle contrast images and reference flow map were randomly extracted within the microfluidic channels. The datasets of the training pairs were used to train the proposed CNN architectures. For the prediction stage, each patch of the speckle contrast images was processed by the trained network. The predicted flow patches were then reassembled to form the output flow map.

Download Full Size | PDF

2.3.2 Network architectures

To predict flow maps from MESI images in an end-to-end fashion, we developed two neural network architectures based on residual neural network (Res-Net) model focusing on spatial features and the correlation information between frames acquired at different exposure times. Figure 4(a) presents the multiple-stream residual network (MSR-Net), which is divided into seven sub-streams, each corresponding to one exposure time. The speckle contrast patches along the dimension of exposure times are the inputs for each stream. Each stream of the network architecture contains B residual blocks with identical layout. The residual block comprises two-dimensional convolution (Conv) layers (8 feature channels, 3 ${\times} $ 3 filter with a stride of 1 and replication padding) with batch normalization (BN) and leaky rectified linear unit (leaky ReLU) layers. The features obtained at the output of each stream are concatenated and passed to the subsequent output stream. A shortcut connection within each residual block is applied to perform identity mapping and skip the two Conv layers to avoid the vanishing gradient problem [25]. The number B of residual blocks was progressively increased to evaluate the best trade-off between features extraction and network convergence and time required for training. In principle, the network learns the appropriate combinations of features from the seven streams to predict the final flow map.

 

Fig. 4. Architecture of the two residual neural networks proposed for blood flow prediction from multiple exposure speckle patches.

Download Full Size | PDF

Figure 4(b) presents the architecture of a residual 3D convolution neural network (Res3D-Net). The input of the network is a stack of speckle contrast patches with sequentially increasing exposure times. The proposed Res3D-Net is based on 3D residual blocks to learn the spatial-exposure features. The first B 3D residual blocks (8 feature channels, 3 ${\times} $ 3 ${\times} $ 1 filter with a stride of 1 and replication padding) serve as the joint-spatial feature extractor across multiple exposure speckle contrast images. At the end of the feature extractor, the generated features are then fed into three spatial-exposure residual blocks (8 feature channels, 3 ${\times} $ 3 ${\times} $ 3 filter with a stride of 1, replication padding along spatial dimension, and no padding along exposure time dimension). The strategy of no padding along the exposure time dimension allows for a gradual extraction of neighboring exposure features from local to global information while preserving input distribution along the borders of the exposure time dimension.

2.3.3 Network training

We used a five-fold cross-validation method to evaluate the performances of the two networks. This allows a robust estimation of the model performances bias and variances [26]. The patch dataset is composed of 18,000 patches obtained by random sampling of 200 patches from each of 90 MESI data sets corresponding to different experimental parameters (5 flows for the 40 µm diameter channel, 8 flows for all other 5 channel dimensions, all at two optical magnification). The patches pairs were obtained from experimental speckle contrasts images and associated ground truth flow maps as detailed in section 2.3.1. The entire dataset was used for part-to-part training and validation. The patches were randomly gathered into five groups each with the same number of patches. The patch from a single microfluidic parameter could be found only in one of the groups. Each group, therefore, contained data from a couple of microfluidic parameters (different flows, microfluidic diameters, and magnification levels), referred to as microfluidic parameter-wise cross-validation. The microfluidic parameter-wise cross-validation was implemented to test the generalization capability of the network for flow prediction under different microfluidic parameters. To compare the performance of our network with non-linear fitting methods, we further tested the network without five-fold cross-validation. Each speckle image was split in half horizontally, then the top half was used for training and the bottom half as testing. Each top half of the 90 speckle images is randomly sampled into 200 patches, resulting in a training dataset composed of 18,000 patches and their associated ground truth flow patches. In contrast with the cross-validation method described above, the patches in the training data set cover all microfluidic parameters. The resulting trained network was first applied to process testing data from the microfluidic flow phantoms, and second it was applied to process in vivo MESI data, and to compare the network results with the non-linear fitting method applied to the same data. Both network architectures were trained using MATLAB R2021a on a workstation (NVIDIA Quadro GV100 GPU, 32 GB RAM). Weights for convolution were initialized with the Xavier initializer. Mean squared error (or L2 loss) and Adaptive Moment Estimation (ADAM) optimizer were used for optimizing weights and biases, with a learning rate of 0.01, and momentum of 0.9. A mini batch of size 100 is adopted for model training. All networks were trained for 50 epochs, which was empirically determined through loss curves and consistent across all the folds. The trained model is converted to ONNX (Open Neural Network Exchange) format for deploying into different platforms.

2.4 Evaluation metrics

To evaluate the performances of the two networks, we measured the mean absolute error (MAE), root mean squared error (RMSE), peak signal-to-noise ratio (PSNR), mean absolute percentage error (MAPE), and structural similarity index metrics (SSIM) [27] between predicted flow image and the reference flow image.

2.5 In vivo experiments

All experiments were performed in accordance with the European Directive 2010/63/UE (Ethical project #CEB-03-2018) regarding the care and use of laboratory animals. MESI recordings were carried out in a 4-months old, anesthetized mouse (male, C57BL6, mean weight = 48 g) after a craniotomy was performed over the barrel cortex. The surgical protocol was described previously [9]. A whiskers’ stimulation protocol was applied to visualize the blood flow change in the medial cerebral artery (MCA) in particular. Mouse’s whiskers were stimulated unilaterally with a cotton swab attached to a servomotor (f = 2 Hz; ± 60°) driven and synchronized with acquisition start using an Arduino microcontroller. A trial recording consisted of a baseline phase (10s) followed by mechanical stimulation of the whiskers phase (30 s) and finally a rest phase (30 s). MCA images were acquired on the contralateral side of whiskers being stimulated. For the analysis of in vivo data both techniques (CNN method and non-linear fitting method) were compared on Python 3.9 using Intel Xeon Gold 6244 CPU @ 3.60 GHz 16/32 physical/logical cores.

3. Results

3.1 Speckle contrast database

Figure 5 shows representative speckle contrasts for the whole range of exposure times, for two channels, two different flows and two imaging magnifications. As expected, the speckle contrast decreases with increasing exposure times and with increasing flows. Identical flows lead to similar speckle contrast values for both channels diameters. More unexpected, using a higher optical magnification leads to significantly higher values of the speckle contrast for the same parameters of flow, exposure, and channel diameter. Calculation of the number N of camera pixels per speckle grain image (i.e. spatial sampling of each speckle grain at the sensor) leads to values between 2.3 and 4.5, above the Nyquist sampling criteria. Following Thompson et al [24], we also calculated the ratio of the window size used for contrast calculation to the speckle size. Values of 2.4 and 1.1 were found for magnification of ×2.5 and ×6 respectively. The value at ×2.5 is not optimal but close to the threshold advisable value. The low value at ×6 magnification clearly results in underestimation the speckle contrasts due to the lack of statistics in the calculation of σ. Normalization by an experimental estimate of beta calculated over a large field of view was carried-out but did not allow to fully recover the same K(T) curves for ×2.5 and ×6 magnifications (see supplementary Figure S3). We believe that further parameters modified by magnification like depth of field could affect the contrast [28,29]. This result needs to be further studied in our specific set-up, yet it underlines the relevance of using experimental images to train the network with data accounting for instrumental effects difficult to simulate.

 

Fig. 5. Speckle contrast as a function of exposure time. A) For a channel of diameter Φ = 100 µm B) For a channel of diameter Φ = 300 µm. Black symbols correspond to the 2 µl/min flow. Red symbols correspond to the 5 µl/min. Diamonds correspond to the ×2.5 magnification. Dots correspond to the x6 magnification.

Download Full Size | PDF

3.2 Performance evaluation of MSR-Net and Res3D-Net

Table 1 summarizes the performances evaluation metrics MAE, RMSE, SSIM, PSNR and MAPE (see section 2.4) for the predicted maps using MSR-Net and Res3D-Net with respect to reference maps. For both networks the number of residual blocks B was increased from 1 to 6 and as expected, the accuracy of the network increases with the increase of B. Adding more residual blocks also increase the difficulty to achieve convergence resulting in much longer training time. Considering this tradeoff, the optimal number of residual blocks was determined to be 5 for both networks. To evaluate the statistical differences between the predictions of Res3D-Net and MSR-Net for the optimal number of residual blocks, we performed a Wilcoxon signed rank test on the metrics from the five-fold predicted results. No significant difference was found in the predictive performance between the two networks, as indicated by p values greater than 0.05 for all metrics. However, Res3D-Net demonstrated lower levels of uncertainty in its point estimates of the metrics compared to MSR-Net, suggesting more robust predictions. As a result, the Res3D-Net approach was selected for use in subsequent experiments.

Table 1. Comparison of performances metrics for MSR-Net and Res3D-Net. The best performances for each metric are indicated in bold font. Values correspond to the mean metrics across all five sets of predicted results from five-fold cross-validation and the corresponding standard error.

View Table

3.3 Comparison of CNN and non-linear fitting methods

Inverse correlation times maps (ICT) are derived from the non-linear fitting procedure, while the CNN produces velocity maps in mm/s units. The exact relationship between velocity and ICT is not known, though there is a proportionality relationship between both [30,31]. We have compared maps normalized to their maximum. As a representative example, Figs. 6(A) and 6(B) show a comparison between normalized velocity maps generated by CNN and normalized ICT maps derived from pixelwise non-linear fitting at varying flow rates under two distinct optical magnifications (×2.5 and × 6) for the channel of diameter Φ = 300 µm. It is important to note that MESI data were acquired in a sub-optimal temporal sampling using a simplified acquisition scheme [11]. Especially, the minimal exposure time in the MESI data is 1 ms. This leads the non-linear fit procedure to fail to converge or to produce unrealistic extreme values for some pixels in the ICT map. To mitigate these effects, a 2D median filter (15 × 15) was applied to the ICT map prior to normalization. Figures 6(C) and 6(D) display the corresponding profiles within the central region (across 40 pixels) of both ICT and speed maps at a flow rate of 8 µl/min. The profiles across the channel predicted from both approaches present similar shapes. However, pixelwise non-linear fitting is highly susceptible to the noise, resulting in failure to converge during the fitting process and predicting blood flow fluctuations with many extreme values. In contrast, the CNN approach incorporates information from neighboring pixels during regional feature extraction through convolutional computation, resulting in clearer dynamics (excluding extreme values) and more spatially homogeneous predictions of relative blood speed changes. Interestingly the CNN approach also reduces the edge effects (higher values of predicted speeds on the edges of the channel) due to the speckle contrast spatial processing.

 

Fig. 6. Comparison of speed maps predicted by CNN and ICT maps derived from the non-linear fit under (A) × 2.5 magnification and (B) × 6 magnification and (C-D) their corresponding profiles extracted from both maps at a flow rate of 8 µl/min. The mean speeds varied from 0.74 to 5.93 mm/s (flows ranging from 1 to 8 µl/min) in 0.74 mm/s increments for the Φ = 300 µm channel.

Download Full Size | PDF

Figure 7 shows the relative changes of Res3D-Net predicted speeds and non-linear fit ICTs as a function of the flow in channels with increasing diameter (60, 80, 100 and 300 µm). All relative changes are normalized to a reference flow of 4.5 µl/min. In all channels, the relative changes estimated either by the CNN approach or by the non-linear fit scale linearly with the actual relative changes of the flow measured independently (by the microfluidic system inline flowmeter) in the channel. However, the CNN approach performs much better in terms of linearity, the R-square values for CNN range from 0.82 to 0.96, while those for non-linear fit range from 0.21 to 0.91. Except for smallest channel, the slope of the linear regression derived from the Res3D-Net predicted results ranges from about 0.19 to 0.23, which is very close to the slope of 0.22 for the reference flow. In contrast, non-linear fits are susceptible to noise interference, resulting in a slope range of approximately 0.09 to 0.23. In addition, the non-linear fit approach results in outliers such as the relative flow change for a 4 µl/min flow in the 80 µm channel or the relative flow change at 8 µl/min in the 100 µm channel. Overall, Res3D-Net exhibits a higher slope than non-linear fits, indicating a broader and more robust dynamic range in response to incremental increases in flow rate. In the case of the smallest channel, the slope of the regression line from CNN-predicted results deviates from the expected values (reference flow). This deviation may be attributed to the smaller number of pixels with flow in images of small channels compared to large channels, resulting in a limited number of training patches with flow and potentially insufficient training. Future work is required to address this issue by increasing the sample size for channels with smallest dimensions.

 

Fig. 7. Relative changes of speeds and ICTs (normalized values to flow of 4.5 µl/min) as a function of the actual flows in channels with increasing diameter (60, 80, 100 and 300 µm). Ideal reference is provided as black dots.

Download Full Size | PDF

3.4 Implementation of CNN analysis on in vivo data

Multiple exposure speckle contrast images acquired in vivo were processed using the non-linear fit MESI model and the Res3D neural network. Figure 8(A) shows a reference image of the mouse’s brain under incoherent illumination at 570 nm, where the MCA main branch is visible. The corresponding extracted ICT and Res3D-Net-derived speed maps from the same MESI data are shown on Figs. 8(B) and 8(C) respectively. On both maps, the smallest capillaries that are partly out of focus could not be analyzed but either the non-linear fit and CNN approaches. We believe that in our experimental conditions the fit failure is mostly related to the lack of data at exposure times lower than 1 ms. This minimal exposure time was chosen for synthetic exposure as a trade-off between sensitivity and amount of data required for in vivo dynamic studies (several 1-minute stimulation/recovery trials are recorded during a single imaging session, see Fig. 8(D)). Yet as the decorrelation times observed experimentally in vivo in the mice brain is mostly close or below 1 ms [32], the decorrelation is not fully captured thus leading to possible non convergence of the model. For the analysis of these sub-optimal MESI data, the CNN derived map shows a larger dynamic of speed values compared to the non-linear fit derived ICT map. In addition, it presents homogeneous values in neighboring pixels while the ICT map presents some null values corresponding to pixels where the non-linear fitting model did not converge.

 

Fig. 8. Derivation of relative flow maps and changes from data obtained in vivo during sensory stimulation of the brain in mice. (A) Reflectance image of the MCA under green incoherent illumination at 570 nm. (B) Non-linear fit ICT map. (C) Res3D-Net predicted speed map. (D) Normalized speed/ICT profile (using ROI #1) that was applied on the CNN predicted speed map and the ICT fitted map respectively. Both profiles were fitted to the laminar viscous flow model. The center of the channel coincides with the reference 0 µm distance. (E) Time course of the relative changes in the speed during whiskers stimulation extracted from the Res3D-Net prediction maps after image processing (using ROI #2).

Download Full Size | PDF

The region of interest ROI#1 on Fig. 8(A) was used to derive the profile of the normalized ICT and speed derived from Fig. 8(B) and 8(C) respectively. Figure 8(D) shows the corresponding profiles in plain lines. A parabolic profile fits the data very well in both cases (dashed lines). This corresponds to the laminar viscous flow following Poiseuille’s law in a cylinder-shaped channel, thus validating the choice of this model to define the target images for the training of the network. The profiles demonstrate again higher dynamics and lower noise in the map derived using the Res3D-Net approach compared to the non-linear fit approach.

The region of interest ROI#2 on Fig. 8(A) was used to derive the time course of the relative changes of the speed during a single short sensory stimulation trial showed on Fig. 8(D). An increase of the blood speed of around +2% is detected during the stimulation relatively to the baseline mean value indicating a functional activation of the underlying somatosensory cortex. This increase of blood speed is followed by a decay back to the baseline value once the stimulation ceases.

4. Discussion

In this work, we have proposed a novel CNN-based approach as an alternative to conventional non-linear algorithms to process MESI data to extract relative flow-speed changes. The method relies on a 3D CNN (Res3D-Net) end-to-end neural network for MESI, which is capable of mapping blood flow with better image contrast and signal-to noise ratio (SNR) comparing to existing algorithms.

4.1 Benefit and limits of real experimental data for network training

We have trained the CNN using speckle contrast images of flows in microfluidic channels obtained experimentally with the same MESI imaging set-up that was used to acquire in vivo speckle contrasts images to be analyzed by the neural network for blood low estimations. In comparison to training based on simulated speckle contrasts images, the experimental training database has the advantage of incorporating all the instrumental and experimental parameters that are very difficult to accurately account for in simulations. Indeed, it has been demonstrated that the experimental speckle contrast values are strongly affected by many experimental parameters, some of which are difficult to evaluate. This includes the spatial sampling of the speckle grain at the sensor level defined by the camera magnification, pixel dimensions of the sensor and the numerical aperture of optics [33], the laser source coherence length [34], polarization, and angle of incidence [4], the illumination intensity and distribution, the camera dynamics and pixel well-depth, the camera readout and dark noise (including fixed patterns noise) [35]. Here, the use of microfluidic phantoms with controlled flows obtained through pressure control and on-line feedback allowed to create target images with accurate flow values for network training. Care was also taken to train the network with images obtained on channels with dimensions and flows representative of the mouse brain physiology, which was not achieved in previous attempts at using machine learning for speckle imaging analysis [14,19]. Using experimental data to train the network has also limits compared to simulations. Practically, MESI data can be acquired for a finite number of experimental situations, thus not all possible diameters and flows within the physiological range were used. However, the analysis of in vivo data shows that the Res3D-Net was able to infer relevant values for blood vessels whose exact diameter and flows had not been presented to the network in the training phase (see Fig. 8 and supplemental Figure S4). We hypothesize that the network can interpolate for data in the range parameters of the data presented in the training phase. Further work is requested to investigate this issue and how the network would perform for data with parameters far from the range of those used in the training phase. Using simulated data has the intrinsic advantage of providing large amount of data for a vast range of parameters. Whether using experimental or simulated data is better to train CNN networks in the context of LSCI or MESI imaging remains an open question.

4.2 Comparison of MSR-Net and Res3D-Net

In this study, we proposed two networks for predicting blood flow distribution. The MSR-Net primarily predicts blood flow distribution based on the extracting speckle features independently at different exposure times. In contrast, the Res3D-Net performs convolution with 3D kernels, allowing it to extract features that simultaneously contain speckles and their correlations at different exposure times. Since the MESI model is based on the changes in K values at different exposure times, 3D feature extraction of the Res3D-Net is inherently closer to the assumptions of the MESI model. Despite this, our microfluidic experiments showed that both the MSR-Net and Res3D-Net were capable of accurately predicting blood flow. However, when performing five-fold cross-validation, the five sets of data predicted by the Res3D-Net exhibited less variation. Less variation in network prediction indicates a reduced likelihood of overfitting and improved generalization ability. We speculate that this may be due to the Res3D-Net's closer alignment with the assumptions of the MESI model, making it easier to identify correlations between input speckle images and blood flow distribution during training. It is worth noting that previous work used ANN methods to establish the relationship between changes in K values at different exposure times and flow velocity using simulated data [12]. However, in our study, our CNN architecture further incorporates spatial information from speckle images. By considering spatial information, one can better avoid convergence failure or extreme values that may arise when predicting on a pixelwise basis.

4.3 Comparison of CNN and non-linear fitting approaches

The comparison of the blood flow maps derived using the CNN approach and non-linear fitting approach shows similar distribution of the relative blood flows. In addition, the machine learning approach has several advantages compared to the model fitting approach applied to the MESI data obtained using synthetic exposure with a sCMOS camera. First, it does not require computationally intensive pixelwise fitting. In fact, processing a 700 × 700 pixels image with the Res3D-Net takes around 400 ms, reducing the processing time by a factor of 3115 compared to non-linear fitting in a python implementation using the same computer hardware configuration (see supplementary Figure S1). Indeed, CNN computation time falls below 1 s starting from 3 used cores which obviates the need of a complex hardware configuration to perform the necessary calculations. Consequently, this improvement paves the way for real-time MESI analysis. Second, the pixelwise non-linear fitting model is sensitive to data noise and to the choice of the initial parameters of the fit. Therefore, for multiple exposure data with suboptimal time sampling at the early times of decorrelation, the fit does not converge for all pixels leading to the need of defining large ROIs to average neighboring pixels contrast values. Contrarily, the CNN derived blood flow maps show consistent and homogeneous relative flow values for neighboring pixels. Third, the CNN approach does not rely on a priori model of how speckle contrast is related to the decorrelation time. Indeed, in the non-linear fitting approach, the model is most generally founded on the assumption of an exponential decay of the decorrelation function of the field g₁(t), assuming that the flow regime is either single scattering un-ordered (SU) or multiple scattering ordered (MO). However, using a very high frame rate camera (>20kHz) to calculate experimentally g₁(t) for each pixel of the exposed brain of a mouse, it was shown that various mixed flow regimes co-exist including multiple scattering unordered flow (MU) for capillaries and parenchyma or single scattering ordered flow (SO) for large vessels [32]. These findings are meaningful from a physiological point of view, since the flow regimes in arterioles, capillaries or the parenchyma are different. Consequently, the fitting function should be adapted for each pixel of the field of view considering the local flow regime. Changing the fitting functions applied to the experimental data ultimately leads to different estimations of the decorrelation time τ_c. Practically, as a single fitting function for all pixels must be set in advance, a trade-off is made, leading to significant errors in the estimations of the flow. Alternatively, a model free approach for speckle data analysis based on a new definition of the speckle contrast was recently proposed and validated provided that the ergodicity condition is satisfied [36]. This assumption is not true when static scatterers contribute significantly to the speckle contrast and the method is no longer model-free as it requires the introduction of a parameter to characterize the stiffness of g₁(t) decrease. The CNN machine learning approach proposed here is totally model-free by nature and is likely to encompass the complexity of speckle contrast formation through training with experimental data to adjust the network weights. As also supported by a recent study for in vitro data acquired for a rotating disk covered with a diffusive paint [19], a carefully trained CNN network with accurately sampled MESI data is likely to predict quantitative velocities