Open Access
Add to BibSonomy
Abstract
Fluorescence molecular tomography (FMT) serves as a noninvasive modality for visualizing volumetric fluorescence distribution within biological tissues, thereby proving to be an invaluable imaging tool for preclinical animal studies. The conventional FMT relies upon a point-by-point raster scan strategy, enhancing the dataset for subsequent reconstruction but concurrently elongating the data acquisition process. The resultant diminished temporal resolution has persistently posed a bottleneck, constraining its utility in dynamic imaging studies. We introduce a novel system capable of simultaneous FMT and surface extraction, which is attributed to the implementation of a rapid line scanning approach and dual-camera detection. The system performance was characterized through phantom experiments, while the influence of scanning line density on reconstruction outcomes has been systematically investigated via both simulation and experiments. In a proof-of-concept study, our approach successfully captures a moving fluorescence bolus in three dimensions with an elevated frame rate of approximately 2.5 seconds per frame, employing an optimized scan interval of 5 mm. The notable enhancement in the spatio-temporal resolution of FMT holds the potential to broaden its applications in dynamic imaging tasks, such as surgical navigation.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Fluorescence Molecular Tomography (FMT) is a macroscopic optical imaging technique that enables 3D visualization of fluorescence distribution in scattering biological tissues [1]. Compared to other established imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI), the non-invasive and non-ionizing radiation nature of FMT, along with its high sensitivity and relatively low cost, makes it a promising tool for biomedical research and clinical applications such as brain imaging, disease diagnostics, and surgical navigation [2–4].
Traditional FMT follows a point-by-point raster scanning approach. More specifically, during data acquisition, an excitation light source and a scanning device (e.g., a galvano-mirror scanner) are used to scan the object’s surface. A detector captures 2D images of the fluorescence emitted by the targeted fluorescence probes after receiving the excitation light [5]. Through a model-based reconstruction, this raster scanning approach can achieve a spatial resolution of 1-2 millimeters at a depth of several centimeters [6]. However, this conventional method is time-consuming, with each complete scan costing at least several minutes. The relatively low temporal resolution of FMT limits its applicability in real-time monitoring scenarios, such as surgical navigation and kinetics studies [7–9]. Moreover, the effective illumination area of each point-shaped light source is relatively small. Scanning more points at a small step size is necessary to cover the whole object, but this also makes the fluorescence images acquired from adjacent points highly correlated in information content. Such a redundancy in the raw dataset is not helpful to solve the ill-posed inverse problem of FMT, meanwhile it also further elongates data acquisition and increases computational load during reconstruction [10]. On the other hand, reducing illumination points leads to an insufficient dataset for reconstruction, which in turn compromises the spatial resolution [11]. Overall, the existing raster scanning is a bottleneck for further improvement of FMT speed. Although a few optimization theories of point-shaped illumination have been proposed [12], the design of illumination patterns is still rather ad-hoc and suboptimal.
Instead of point illumination, structured light can be alternatively used as the illumination source. Broadly speaking, structured light illumination (SLI) refers to the generation and application of custom light fields through a specially designed projection device [13]. Configurable parameters of SLI include structured intensity, polarization, and phase. Nevertheless, altering intensity patterns is arguably the most widely used SLI approach in optical imaging. SLI provides a much wider area of illumination, thus yielding more fluorescence information in a single scan and accelerating the whole measurement procedure [10]. The patterns of SLI encompass dot matrix [14], line [15], sinusoid [16,17], binary (checkerboard, Gray code) [18,19], wavelet [20], adjustable [21,22], and mixed patterns [23]. SLI has found extensive applications in biomedical imaging, particularly in super-resolution microscopy. The application of sine phase-encoded SLI leads to the well-known Structured Illumination Microscopy (SIM), which achieves a lateral resolution twice that of classical diffraction-limited microscopes [24,25]. In the last decades, the application of SLI in emerging macroscopic imaging techniques, such as diffuse optical tomography (DOT) and FMT, is rapidly evolving. Belanger S. et al. employed a binary-patterned illumination in DOT via digital micro-mirror devices (DMDs), achieving real-time reconstruction in phantoms at 2-Hz framerate [18]. Similarly, Vanegas M. et al. developed an SLI-based DOT system with Gray code pattern using a projector to reconstruct absorption maps in a compressed breast model [19]. Besides binary patterns, Ducros N. et al. used DMDs to generate wavelet-patterned illumination for FMT, significantly reducing both data acquisition and reconstruction times without sacrificing the accuracy compared with traditional raster scan [20]. However, it is worth mentioning that the hardware implementation of complex illumination patterns requires high-cost projection devices such as spatial light modulators (SLMs) and DMDs. The complexity and precision of the projected patterns also rely on the selection of projection devices. Another trend in SLI-based DOT and FMT is based on optimization theory. Liu Y. et al. developed a robust FMT reconstruction algorithm with an adaptive illumination pattern [22]. Dutta J. et al. optimized the illumination pattern by improving the conditioning of the Fisher information matrix [21]. Joshi A. et al. [26] performed FMT simulations using a variety of hybrid structured light and demonstrated its advantage. However, these studies based on the optimization theory were only demonstrated on synthetic data, which lacks experimental verification largely due to the high complexity of the algorithm itself.
Among the aforementioned patterns, line scan is a relatively simple illumination technique, which can concentrate high energy in a single line and maintain a higher signal-to-noise ratio (SNR) even under dim light conditions, compared to other complex patterns such as sinusoid and wavelet patterns [17,20]. Besides, line scan necessitates no expensive devices such as SLMs and DMDs. Lastly, line pattern has an additional benefit of extracting the 3D surface information, which is routinely used in machine vision [27]. Regardless of the existing works on SLI FMT using complex patterns, the power of line scan FMT is underestimated and its reconstruction theory is less explored. More interestingly, the combination of surface geometry and 3D fluorescence distribution has not been reported. Here we propose a new system (named ShanghaiTech Multicontrast Optical Tomography-duo, or SHMOT-duo), which employs programmable line scan and a novel dual-camera design, with a fluorescence camera for recording fluorescence signals and a bright-field camera for acquiring surface information. Combining line illumination and dual-camera detection, SHMOT-duo is capable of simultaneously retrieving 3D fluorescence distribution and surface geometry. In addition, we also elaborate on the reconstruction algorithm for SLI FMT. Finally, the imaging performance of the new system has been validated through both static and dynamic FMT experiments. The new system addresses the limitations of traditional FMT by replacing the time-consuming point-by-point raster scan with a rapid line scan, significantly improving its temporal resolution for dynamic imaging studies. Additionally, the novel integration of dual-camera detection allows simultaneous retrieval of 3D fluorescence distribution and surface geometry, which has not been reported previously.
2. Methods
2.1 System description
The schematic of SHMOT-duo is illustrated in Fig. 1(a). The light source of the system is a wavelength-tunable pulsed laser (SuperK EXTREME, NKT, Denmark), treated as a continuous wave (CW) source covering a spectral range from 400 nm to 860 nm with a selective filter (VARIA, NKT, Denmark). The output light is coupled into a multimode fiber (50/125 $\mu$m, FC/PC) via a coupler (PAF2P-15A, Thorlabs, New Jersey, USA) and then is collimated (F280FC-B, Thorlabs, New Jersey, USA) before entering a scanning galvanometer (basiCube 10, SCANLAB, Puchheim, Germany). The collimated laser beam is focused with a focal distance of 300 mm and directed onto the sample surface via a pair of mirrors (PFSQ05-03-G01, Thorlabs, New Jersey, USA), resulting in a light spot with a diameter of about 1 mm. The scanner is programmable and the scanning line has the same width as the diameter of the focused spot.
Fig. 1. (a) Schematic of the imaging system. SHMOT-duo employs a dual-camera detection configuration, which is capable of acquiring surface topology and internal fluorescence distribution simultaneously. (b) The workflow of surface extraction, where the image stack at the excitation wavelength is recorded by the bright-field camera to generate the surface point cloud. (c) The workflow of line scan FMT, where the image stack at the fluorescence wavelength is recorded by the fluorescence camera and is used for the reconstruction of 3D fluorescence map. Notably, the surface information extracted from (b) provides important prior knowledge for the synchronized FMT reconstruction in (c).
A dual-camera detection configuration is adopted in our system, comprising a fluorescence camera (Edge 4.2, PCO AG, Kelheim, Germany) and a bright-field camera (XW800, XianWeiJinGong, China), both mounted 30 cm above the sample stage. To ensure a similar field-of-view (FOV) of two cameras, the orientation of the fluorescence camera is vertical, whereas the bright-field camera is tilted by 10 degrees. For recording fluorescence signals at different wavelengths, a filter wheel (FW102C, Thorlabs, New Jersey, US) is placed between the fluorescence camera and a focusing lens (f = 50 mm, Nikon, Tokyo, Japan), offering a selective wavelength for difference fluorescence probes (630 nm, 640 nm, 660 nm, 680 nm, 700 nm). The fluorescence camera has a FOV of 65 $\times$ 65 mm with an image resolution of 2048 $\times$ 2048 pixels. In comparison, the bright-field camera uses a focusing lens (25 mm, computar, Japan) and offers a FOV of 60 mm $\times$ 100 mm with an image resolution of 3840 $\times$ 2160 pixels. During data acquisition, for each scanning line, a fluorescence image is captured at a specific emission peak according to probe selection, while a bright-field image is simultaneously recorded for surface extraction.
2.2 3D surface extraction and evaluation
3D surface extraction was implemented based on unified pinhole modeling, where both the detection and illumination modules are considered as a pinhole model [28]. SHMOT-duo inherits the function of surface extraction from the first-generation SHMOT system, where the procedure can be divided into three steps: 1) system calibration, 2) surface point extraction, and 3) occlusion filling and meshing. The tie between 2D measurement and the 3D surface is established via a pinhole model, which is given below:
After system calibration, we extract the surface points by analyzing the line stripes from the bright-field camera. The windowing technique localizes the central points. All extracted 3D surface points are obtained and form a point cloud of the top surface. After occlusion filling, the updated point cloud is then transformed into a mesh for the following FMT reconstruction (Fig. 1(b),(c)).
The accuracy of our surface extraction algorithm has been described systematically in [28]. In this work, we mainly explore the impact of the scanning intervals (SIs) between two adjacent lines, because the density of illumination lines can influence the accuracy of the extracted surface and also alter the time cost for scanning the whole surface. Using fewer lines can shorten the scanning time, but also results in less usable information for surface extraction. Nevertheless, it is interesting to know if this missing information can be recovered by interpolation to some extent. In the end, our goal is to reach a balance between imaging speed and resolution by optimizing the scanning pattern or more intuitively, the value of SI. For calibration purposes, we designed a 50 $\times$ 50 mm$^2$ plate containing concave and convex components with different geometries and orientations. Hausdorff distance was used to evaluate the accuracy of the extracted surface [27]. The calibration plate was 3D printed with a precision of 50 micrometers (Objet260, Stratasys Ltd., Minnesota, USA).
2.3 FMT reconstruction
2.3.1 Conventional raster scan FMT
FMT reconstruction is based on the Radiative Transport Equation (RTE), which can predict light propagation and its energy changes in both scattering and non-scattering regions in biological tissues [29]. In practice, the diffusion equation (DE) is typically used to approximate the RTE in CW-FMT:
where $\mu _{a}(r)$ and $\kappa (r)$ are the absorption coefficient and the diffusion coefficient at position $r$ in the object domain $\Omega$. Photon density distribution $\phi (r)$ can be calculated from Eq. (2), either using an analytical solution [30] or numerically [11]. A Robin type boundary condition is employed [31]: where $\zeta _{0}$ accounts for refractive index mismatch, $\partial \phi (r)$ is a position-related source term in the boundary $\partial \Omega$, and $\partial v$ is an outward normal on the boundary.The analytical solution to Eq. (2) is only applicable to simple geometries. To cope with more complicated objects, we adopt the Finite Element Method (FEM) [29]. The linear relationship between $\phi (r)$ and $q(r)$ is established using the Galerkin method [32], which converts the continuous operator into a discrete form [29]. The relationship between the nodal photon density $\mathbf {\Phi }$ and the corresponding light source $\mathbf {Q}$ can be derived from Eqs. (2) and (3):
where $\mathbf {S}$ is the stiffness matrix.Typically, a complete FMT experiment consists of two stages: excitation and emission. At the former excitation stage, images for each scan are recorded at the excitation wavelength, where the vector $Q$ in Eq. (4) is determined by the laser source. For the conventional raster scan FMT, $\mathbf {Q}^{p}_{x,i}$ stands for the source term of the $i$th illumination point. At the latter emission stage, the source of $Q$ in Eq. (4) refers to the excited fluorescence inside the object. Similarly, for the $i$th illumination point, we can obtain a unique source term $\mathbf {Q}^{p}_{m,i}$. Therefore, the two-stage FMT experiment can be expressed by a coupled DEs:
where $x$ and $m$ denote ’excitation’ and ’emission’ respectively. $i = 1,2,\ldots,N_{p}$, $N_{p}$ is the total number of point scans. $p$ stands for ’point raster scan’. The source term $\mathbf {Q}^{p}_{m,i}$ at the emission stage can be calculated by: where $C_{d}$ is the nodal value of fluorophore distribution, $\eta$ is determined by the fluorescence yield and the absorption cross section at the excitation wavelength. The operator $diag$ converts the column vector $C_{d}$ into a diagonal matrix.As we use a non-contact detection configuration in our system, the camera records only part of the photon density on the boundary. A transmission matrix $\mathbf {\Gamma }$ models light propagation from the surface to the camera [30]. At both two stages, the relationship between the measured power $\mathbf {M}$ and the optical field $\mathbf {\Phi }$ is given by:
Using Eq. (2) to Eq. (9), FMT reconstruction can be transformed into an optimization procedure by minimizing the difference between the simulated data and the measured data:
where the system matrix $\mathbf {A}$ is obtained using Eqs. (5) and (6), $b$ is calculated by $[\mathbf {M}_{x,1}/\mathbf {M}_{m,1},$ $\mathbf {M}_{x,2}/\mathbf {M}_{m,2},\ldots,$ $\mathbf {M}_{x,N_{p}}/\mathbf {M}_{m,N_{p}}]^{T}$. The additional regularization term $\mathbf {R}(C_{d})$ encompasses the prior information, which is tuned by $\lambda$, a hyperparameter. To solve Eq. (10), we use the L-BFGS-B algorithm [33] with $L2$-norm regularization.2.3.2 Line scan FMT
Unlike raster scan FMT, line structured light illuminates a wider area of the object, potentially accelerating the experimental procedure. Here we still use the classic two-stage FMT paradigm, but the excitation source $\mathbf {Q}$ is changed into a new form by assuming a line-pattern boundary condition. Now the coupled-DEs change to:
where $l$ refers to ’line scan’. $i = 1,2,\ldots,N_{l}$, $N_{l}$ is the total number of line scans. Similarly, at the emission stage, the source term depends on the internal fluorescence distribution and the excitation photon density.In numerical implementation, we obtain the line scan excitation source $\mathbf {Q}^{l}_{x,i}$ by summing up a finite number of point-shaped sources
where ${\rm M}$ is determined by the FEM mesh and object size. Considering Eq. (11) to Eq. (14), we can obtain a similar inverse problem formulation as Eq. (10). Again, we use L-BFGS-B and $L2$-norm regularization for inversion.In a conventional framework, the temporal resolution of FMT is determined by the number of 2D scans and the time duration of each scan. This principle also works for line scan FMT. For example, if $N_{l}$ lines are used for scanning the object surface with scan duration of $\Delta t$, a complete FMT measurement takes $N_{l}$ $\times$ $\Delta t$ in an ideal situation. Although line scan FMT is assumed to be faster than raster scan FMT, the temporal resolution may still not meet the demanding requirement of some dynamic imaging problems, such as monitoring the kinetics of bio-markers in drug development. To fully exploit the speed advantage of line scan FMT, we also designed a rolling-window mechanism for the reconstruction task. More specifically, the first loop of reconstruction is identical as the conventional framework which requires $N_{l}$ scans. In the second loop, instead of using the scans from $N_{l}+1$th to 2$N_{l}$th lines, a rolling window is applied by considering the dataset from $2$th to $N_{l}+1$th lines (Fig. 5(d)). Such a rolling-window framework can potentially improve the temporal resolution of FMT to $\Delta t$, compared to $N_{l}$ $\times$ $\Delta t$ in conventional framework.
2.4 Simulation and experimental studies for static imaging
To assess the performance of our system in objects with complex surface, we designed a semi-cylindrical phantom with two inclusions as a simulation model (Fig. 3(a)). We also aim to investigate the impact of SI on the reconstruction outcomes. SI not only affects surface extraction but also changes FMT reconstruction quality. Using few scanning lines, or equivalently a large SI value, makes FMT reconstruction more ill-posed. Optical coefficients for the target were set to similar values as biological tissues with an absorption coefficient $\mu _{a}$ = 0.023 mm$^{-1}$, a scattering coefficient $\mu _{s}$ = 0.97 mm$^{-1}$, and a refractive index of 1.4 [34]. We performed raster scan FMT with a grid of 11 $\times$ 11 and line scan FMT with different SI values. All simulation studies include noise with 50% level of the mean measurement intensity on the boundary.
We also fabricated a corresponding silicone phantom based on the design in simulation. A proportional mixture of polydimethylsiloxane, titanium dioxide, and carbon powder was used to mimic the optical properties of biological tissues [34,35]. For the fluorescence probe, we used a Cy5 analog [36], featuring an absorption peak at 640 nm and an emission peak at 670 nm. Two glass tubes with an outer diameter of 1.8 mm and an inner diameter of 1 mm were filled with the Cy5 solution, forming fluorescence inclusions with a length of 10 mm, 4 mm below the top surface. Similarly, raster scan and line scan FMT experiments were conducted for comparison. For all reconstruction tasks in simulations and real experiments, we used a modular reconstruction toolbox STIFT [11] and a desktop computer with an i9-10980XE (3 GHz) CPU and an RTX 3090 (10496 cores) GPU. To solve the inverse problem, we used the L-BFGS-B algorithm with a tuning parameter $\lambda = 10^{-6}$.
2.5 Feasibility study of dynamic imaging
Compared with raster scan, line scan FMT can rapidly record the dynamic events in living tissues, broadening the application of the FMT technology. Combined with a rolling-window scheme, line scan FMT can be further accelerated. In order to demonstrate the feasibility of FMT in dynamic imaging, we performed line scan to a phantom containing a moving fluorescent bolus (Fig. 5(a)). We used the same phantom as described in the previous section and inserted only one glass tube in the phantom. An injection pump (PHD ULTRA, Harvard Apparatus, USA) was adopted to push the Cy5 droplet into the tube and control its speed. A rubber pipe with an inner diameter of 1.8 mm was connected with the pump and the glass tube in the phantom. During the experiment, the speed of the injection pump was set to 10 $\mathrm{\mu}$l/min. The length of the bolus was about 4 mm (Fig. 5(a)). We performed a continuous scan to the phantom using a line pattern with SI = 5 mm. After data acquisition, we used both conventional stationary and rolling-window schemes to do FMT reconstruction. The object surface was extracted using the same scanning lines, which serves as the prior knowledge in FMT reconstruction.
3. Results
3.1 Evaluation of surface extraction
A 50 mm $\times$ 50 mm calibration plate containing curvatures with different geometries is used to test surface extraction (Fig. 2(a)). To evaluate the performance of surface extraction, We scanned the calibration plate with different SIs (0.25, 0.5, 1, 3, and 6 mm). Figure 2(d) to Fig. 2(f) depict the corresponding point cloud and error maps. As SI increases, substantial error appears along the edges of cylindrical geometries, while the error on the spherical geometries is mainly concentrated on right side. This phenomenon is known as the detection error, where the laser beam is obstructed by protrusions and recesses, leading to information gaps at these occluded regions [28]. For low SI values (e.g., SI $\leq$ 1 mm), interpolation can effectively recover data points in these missing areas. However, as SI rises, the extent of information loss expands, leading to incorrect interpolation.
Fig. 2. (a) A 50 mm $\times$ 50 mm calibration plate containing curvatures with different geometries is used to test surface extraction. (b) Normalized error distribution of extracted point cloud for different scanning intervals (SIs) of lines. (c) Mean and standard deviation of surface extraction error for different SIs. (d)-(h) Point cloud and error maps for different SI are visualized.
Figure 2(b) and Fig. 2(c) show the variations in the cumulative distribution and mean values of error. The difference among five cumulative curves is negligible in the range from 0% to 50%, but it becomes larger afterwards (Fig. 2(b)). Notably, the error increases from 0.056 mm to 0.499 mm for measurement at SI = 6 mm and SI = 0.25 mm respectively, showing the importance of optimizing the SI value. In Fig. 2(c), the mean error is 0.144 mm for SI = 0.25 mm. When SI reaches 1 mm, the mean error slightly increases to 0.161 mm, with an increment of only 0.017 mm. However, for SI = 3 mm, the error grows to 0.209 mm, showing a substantial growth of 0.048 mm which further accelerates to 0.102 mm for SI = 6 mm. Nevertheless, it is worth mentioning that even with a large value of SI, e.g., SI = 6 mm, the mean error remains at a relatively low level, with 90% of point cloud errors staying below 0.858 mm, demonstrating the robustness of our surface extraction algorithm. To sum up briefly, quantitative analysis confirms that scanning at SI = 1 mm can generate sufficiently accurate surface exaction with mean error = 0.161 mm, and also maintains the efficiency of scanning.
3.2 Numerical simulation of line scan FMT
We further evaluate the results of FMT reconstruction based on the surface information obtained from the previous step. In a numerical study, we used a semi-cylindrical phantom with two line-shaped fluorescence inclusions embedded inside (Fig. 3(a)). Both of the inclusions are cylindrical with diameter of 1 mm and length of 11 mm, 4 mm below the surface. The center-to-center distance between two inclusions is 10 mm. The problem setting and the definition of SI are given in Fig. 3(h). The reconstruction result obtained from raster scan with a grid illumination of 11 $\times$ 11 is shown in Fig. 3(b), which can clearly resolve the two embedded bars. A series of SI values are evaluated for line scan FMT (SI = 3, 5, 7, 9, 11 mm), and the corresponding reconstruction results are compared to that of raster scan (Fig. 3(c)-(g)). With increasing values of SI, the reconstruction quality deteriorates, which is mainly reflected in the missing gap in the stripe-shaped inclusions in the results for SI = 7, 9, 11 mm (Fig. 3(e),(f),(g)). For SI = 3 mm and 5 mm, the stripe-shaped inclusion is continuous. We can also observe the process of deterioration of reconstruction quality in 3D visualizations in Fig. 3(i). We also analyzed the profile drawing in the cross sections in xz-plane and zy-plane in Fig. 3(a-g), which agrees well with the visual evaluation that if SI $\leq$ 5 mm, the reconstruction artefact becomes evident. In Fig. 3(l), the quantitative analysis shows that lower SI leads to lower RMSE and higher Dice. SI $\leq$ 5 mm is appropriate for a satisfying reconstruction result when we replace raster scan with line scan in FMT data acquisition.
Fig. 3. Comparison of reconstruction results in a numerical simulation. (a) The ground truth of phantom structure and fluorescence distribution. (b) FMT reconstruction resulted from raster scan method using $11 \times 11$ points. (c)-(g) FMT reconstruction resulted from line scan method with different values of SI (SI = 3, 5, 7, 9, 11 mm). (h) Reconstruction problem setting and the definition of SI. (i) 3D visualization of line scan FMT with increasing SI values. (j),(k) Profile drawing in the cross sections in xz-plane and zy-plane in (a)-(g), indicated as two blue dashed lines. (l) Quantitative evaluation of reconstruction results.
3.3 Phantom experiment of static imaging
Following the numerical simulation, we conducted a similar phantom experiment. The phantom design and inserted fluorescence inclusions are identical as the simulation case. For comparison, we performed both raster scan FMT (a 11 $\times$ 11 grid) and line scan FMT (SI = 3, 5, 7, 9, 11 mm). We successfully reconstructed fluorescence distribution in a real FMT experiment using the conventional raster scan method (Fig. 4(b)). The deterioration of reconstruction quality with increasing SI values becomes even more obvious in experimental data, which is manifested in bot cross sections (Fig. 4(c)-(g)) and 3D rendering (Fig. 4(i)). The missing gaps in the fluorescence inclusions for SI $\ge$ 5 mm are substantially larger than that in simulation cases. For example, at SI = 11 mm, the reconstructed targets become two separate dots instead of stripes. Although line scan FMT shows a bit blurring in the reconstructed targets, it is still safe to use SI = 3 and 5 mm to achieve comparable FMT results compared with the time-consuming raster scan method. From the analysis of profiles in the zy-plane, the high intensity appears from y = 20 mm to 30 mm for SI $\leq$ 5 mm, which corresponds well to the simulation case (Fig. 4(k)). Due to the appearance of noise in a real experiment, the resulted quantitative metrics (RMSE and Dice) do not perfectly match the simulation, but the general trends agree well, especially for the Dice index in Fig. 4(l).
Fig. 4. Comparison of reconstruction results in a real phantom experiment. (a) The ground truth of phantom structure and fluorescence distribution. (b) FMT reconstruction resulted from raster scan method using $11 \times 11$ points. (c)-(g) FMT reconstruction resulted from line scan method with different values of SI (SI = 3, 5, 7, 9, 11 mm). (h) Reconstruction problem setting and the definition of SI. (i) 3D visualization of line scan FMT with increasing SI values. (j),(k) Profile drawing in the cross sections in xz-plane and zy-plane in (a)-(g), indicated as two blue dashed lines. (l) Quantitative evaluation of reconstruction results.
3.4 Phantom experiment of dynamic imaging
There is a trade-off between temporal resolution and spatial resolution for line scan FMT. Through the validation of both simulation and real experiments, we performed line scan FMT continuously with SI = 5 mm on a semi-cylindrical phantom containing a moving bolus. After data acquisition, we reconstructed the fluorescence distribution with two different schemes, i.e., a conventional stationary scheme and a rolling-window scheme. The variations of the gravity center of the bolus achieved from different methods are compared in Fig. 5(b). During the entire monitoring process of 143 s, a total frame number of 57 was obtained under the rolling-window scheme, in comparison to only 5 frames from the conventional scheme. After curve fitting, the trajectory depicted by more points can more accurately predict the real moving speed of the fluorescence bolus. The dashed lines in blue and red indicate the fitted curves of the conventional and rolling-window schemes respectively. As shown in Fig. 5(c), the recovered fluorescence signals are concentrated in the expected regions indicated by dashed lines, while the shape and location of the bolus are displayed without noticeable noise. In the end, we use line scan FMT for dynamic imaging at a framerate of 0.4 Hz.
Fig. 5. (a) SHMOT-duo is applied in a dynamic imaging task, where a moving fluorescent bolus inside a semi-cylindrical phantom will be tracked by line scan FMT. An injection pump controls the speed of the bolus. (b) Variations of gravity center of the reconstructed bolus under two different schemes. (c) Time slices of reconstruction result under a rolling-window scheme. (d) Comparison of rolling-window and conventional schemes.
4. Discussion and conclusion
Compared to traditional planar fluorescence imaging, FMT is capable of resolving the fluorescence signal in depth in living tissues, constituting a promising technique for drug development [3] and surgical navigation [7,37]. Currently, most FMT prototypes employ a raster scan strategy, which enriches the data sampling but prolongs data acquisition procedure. This has become a bottleneck in the FMT community as the low temporal resolution of FMT cannot fulfill the demanding requirement for many dynamic imaging tasks. We have previously developed the first-generation SHMOT system by integrating the functions of FMT and surface extraction into a single system using a compact hardware design [28]. Its high-accuracy imaging performance has been validated in both phantom and animal studies. However, it is important to note that the FMT part of SHMOT still relies on the traditional raster-scanning approach, resulting in overly long data acquisition and a large computational load of image reconstruction. In this work, we take a step further by replacing raster scanning with line illumination during FMT data acquisition. Compared with the traditional N $\times$ N raster scan FMT, line scan efficiently reduces both the data acquisition time and data volume by a factor of $\frac {1}{N}$, thus improving the temporal resolution of FMT. Additionally, the collected surface information can be further used as prior knowledge in the following FMT reconstruction to achieve higher spatial resolution. To optimize our scanning strategy, we investigated the impact of SI on reconstruction results in both simulation and phantom studies. The optimized SI value was then applied in a dynamic imaging experiment, where the moving bolus was recovered successfully by our method. This cannot be accomplished by using a raster scan method. In addition, we also introduced a novel rolling-window scheme to accelerate FMT reconstruction, enabling more accurate dynamic measures throughout the experiment.
It is worth mentioning that it is difficult to compare the effectiveness of different illumination patterns in FMT reconstruction, which is not the focus of the current work. The optimization of illumination patterns was mainly reported in simulation studies [21,26]. Validation through experiments necessitates the standardization of imaging procedure, noise level, phantom design, and fluorescence targets. There may exist other more effective illumination patterns than line scan. Nevertheless, line scan provides a high signal-to-noise ratio and a large illumination area. Besides, line scan has been widely used for surface extraction, which is another advantage compared with many other illumination patterns. To the best of our knowledge, our SHMOT-duo system is the first FMT prototype that combines both surface extraction and 3D fluorescence reconstruction after one measurement.
The similarity and regularity of the reconstruction artifacts observed in the xz plane is related with the ill-posedness of FMT inverse problem itself, particularly in the condition of undersampling as we reduced the size of the raw dataset from 11 $\times$ 11 (raster scan) to only 4 $\times$ 1 (line scan, SI = 11 mm) [21,22]. Such a problem is also observed in other undersampling imaging problems, such as sparse-view X-ray CT [38]. Similar to sparse-view CT, line scan FMT also seeks reconstruction results comparable to conventional raster scan FMT using a reduced number of projection data. Our numerical and experiments results indicate that with a SI $\leqslant$5 mm, the reconstruction results are close to that of raster scan FMT. Furthermore, the current reconstruction method uses $L2$-norm regularization, which is known to produce over-smooth effect, especially for extreme undersampling cases with a large SI value. Other regularization strategies, such as $L1$ or total variation, can significantly improve the reconstruction results by encouraging the sparsity in the final solution [39,40]. Recently emerging deep learning-based reconstruction is another alternative that has shown promising results in diffuse optical imaging [41].
Our method emphasizes acquiring surface information of the target object, differentiating it from FMT-CT [42], which captures both surface and inner structure. For a homogeneous setting, line scan FMT and FMT-CT are equivalent. If the inner structure of the object is highly heterogeneous, FMT-CT should be more advantageous, as the CT image can indicate space-varying optical properties and allow more accurate structural localization [42,43]. Nonetheless, the real value of optical properties inferred from CT remains debatable. A wrong optical property map may in turn deteriorate FMT reconstruction. A truly complete FMT process necessitates both the surface information of the target object and the distribution of its internal optical properties. An effective strategy to directly measure the latter is by adopting time-domain diffuse optical tomography technique, which enables accurate characterization of both scattering and absorption properties of the object [44].
There are still some limitations in our system. Firstly, the accuracy of surface extraction can be further improved. Currently, the laser beam is scanned from only one direction, causing occlusions at concave and protruding regions on the surface. Employing multiple scanning views is a reasonable solution, as point clouds acquired from different angles can complement each other. Secondly, although line scan FMT is much faster than raster scan method, the speed of our system is about 0.4 Hz for a FOV of 50 mm $\times$ 50 mm, which may not reach the requirement for real-time monitoring in vivo at second-level speed. More effective illumination patterns should be explored in the future. Last but not least, the penetration depth of current near-infrared region I (NIR-I) FMT is limited by the severe scattering. Emerging near-infrared region II (NIR-II) fluorescence imaging is advantageous as the tissues exhibit lower scattering and deeper penetration depth at this window [45–47]. Our line scan FMT can also be extended to the NIR-II window, although its reconstruction theory should be adapted to some extent as the Diffuse Approximation may not fit in this new observation window [32].
In summary, we developed a novel multi-contrast optical tomography system allowing for high-fidelity surface extraction and 3D fluorescence reconstruction simultaneously, which is attributed to a fast line scanning approach and dual-camera detection. The accuracy of FMT and the importance of line density were investigated comprehensively via simulation and phantom experiments. Furthermore, we offer a proof-of-concept study demonstrating quasi-real-time monitoring using our devised system. The implications of our approach are significant in enhancing the spatio-temporal resolution of FMT, thereby broadening its utility in dynamic imaging applications, particularly in the context of surgical navigation.
Funding
Science and Technology Commission of Shanghai Municipality (21YF429100); ShanghaiTech University; National Natural Science Foundation of China (12101406, 62105205).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. V. Ntziachristos, “Fluorescence molecular imaging,” Annu. Rev. Biomed. Eng. 8(1), 1–33 (2006). [CrossRef]
2. L. Li, Y. Du, X. Chen, et al., “Fluorescence molecular imaging and tomography of matrix metalloproteinase-activatable near-infrared fluorescence probe and image-guided orthotopic glioma resection,” Mol. Imaging Biol. 20(6), 930–939 (2018). [CrossRef]
3. W. Ren, S. Cui, M. Alini, et al., “Noninvasive multimodal fluorescence and magnetic resonance imaging of whole-organ intervertebral discs,” Biomed. Opt. Express 12(6), 3214–3227 (2021). [CrossRef]
4. W. Ren, L. Li, J. Zhang, et al., “Non-invasive visualization of amyloid-beta deposits in Alzheimer amyloidosis mice using magnetic resonance imaging and fluorescence molecular tomography,” Biomed. Opt. Express 13(7), 3809–3822 (2022). [CrossRef]
5. W. Hou and S. H. Thorne, “Noninvasive Imaging of Fluorescent Reporters in Small Rodent Models Using Fluorescence Molecular Tomography,” In Vivo Fluorescence Imaging: Methods and Protocols 1444, 67–72 (2016). [CrossRef]
6. F. Nouizi, H. Erkol, D. Nikkhah, et al., “Development of a preclinical CCD-based temperature modulated fluorescence tomography platform,” Biomed. Opt. Express 13(11), 5740–5752 (2022). [CrossRef]
7. K. Wang, Y. Du, Z. Zhang, et al., “Fluorescence image-guided tumour surgery,” Nat. Rev. Bioeng. 1(3), 161–179 (2023). [CrossRef]
8. Y. An, C. Bian, D. Yan, et al., “A Fast and Automated FMT/XCT Reconstruction Strategy Based on Standardized Imaging Space,” IEEE Trans. Med. Imaging 41(3), 657–666 (2021). [CrossRef]
9. H. Meng, K. Wang, Y. Gao, et al., “Adaptive Gaussian Weighted Laplace Prior Regularization Enables Accurate Morphological Reconstruction in Fluorescence Molecular Tomography,” IEEE Trans. Med. Imaging 38(12), 2726–2734 (2019). [CrossRef]
10. J. P. Angelo, S.-J. Chen, M. Ochoa, et al., “Review of structured light in diffuse optical imaging,” J. Biomed. Opt. 24(07), 071602 (2019). [CrossRef]
11. W. Ren, H. Isler, M. Wolf, et al., “Smart toolkit for fluorescence tomography: simulation, reconstruction, and validation,” IEEE Trans. Biomed. Eng. 67(1), 16–26 (2019). [CrossRef]
12. S. V. Patwardhan, S. R. Bloch, S. Achilefu, et al., “Time-dependent whole-body fluorescence tomography of probe bio-distributions in mice,” Opt. Express 13(7), 2564–2577 (2005). [CrossRef]
13. H. Rubinsztein-Dunlop, A. Forbes, M. V. Berry, et al., “Roadmap on structured light,” J. Opt. 19(1), 013001 (2017). [CrossRef]
14. H. Shang, C. Liu, and R. Wang, “Measurement methods of 3D shape of large-scale complex surfaces based on computer vision: A review,” Measurement 197, 111302 (2022). [CrossRef]
15. X. Xu, Z. Fei, J. Yang, et al., “Line structured light calibration method and centerline extraction: A review,” Results Phys. 19, 103637 (2020). [CrossRef]
16. Y. Wang, J. I. Laughner, I. R. Efimov, et al., “3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique,” Opt. Express 21(5), 5822 (2013). [CrossRef]
17. S. D. Konecky, A. Mazhar, D. J. Cuccia, et al., “Quantitative optical tomography of sub-surface heterogeneities using spatially modulated structured light,” Opt. Express 17(17), 14780 (2009). [CrossRef]
18. S. Bélanger, M. Abran, X. Intes, et al., “Real-time diffuse optical tomography based on structured illumination,” J. Biomed. Opt. 15(1), 016006 (2010). [CrossRef]
19. M. Vanegas, M. Mireles, E. Xu, et al., “Compact breast shape acquisition system for improving diffuse optical tomography image reconstructions,” Biomed. Opt. Express 14(4), 1579–1593 (2023). [CrossRef]
20. N. Ducros, C. D’andrea, G. Valentini, et al., “Full-wavelet approach for fluorescence diffuse optical tomography with structured illumination,” Opt. Lett. 35(21), 3676–3678 (2010). [CrossRef]
21. J. Dutta, S. Ahn, A. A. Joshi, et al., “Illumination pattern optimization for fluorescence tomography: theory and simulation studies,” Phys. Med. Biol. 55(10), 2961–2982 (2010). [CrossRef]
22. Y. Liu, W. Ren, and H. M. Ammari, “Robust reconstruction of fluorescence molecular tomography with an optimized illumination pattern,” Inverse Problems Imaging 14(3), 535–568 (2020). [CrossRef]
23. J. Geng, “Structured-light 3D surface imaging: a tutorial,” Adv. Opt. Photonics 3(2), 128–160 (2011). [CrossRef]
24. Y. Wu and H. Shroff, “Faster, sharper, and deeper: structured illumination microscopy for biological imaging,” Nat. Methods 15(12), 1011–1019 (2018). [CrossRef]
25. M. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” Proc. Natl. Acad. Sci. U. S. A. 102(37), 13081–13086 (2005). [CrossRef]
26. A. Joshi, W. Bangerth, and E. M. Sevick-Muraca, “Non-contact fluorescence optical tomography with scanning patterned illumination,” Opt. Express 14(14), 6516–6534 (2006). [CrossRef]
27. S. Zhang, “High-speed 3D shape measurement with structured light methods: A review,” Opt. Lasers Eng. 106, 119–131 (2018). [CrossRef]
28. S. Gao, J. Zhang, Y. Hu, et al., “Multifunctional Optical Tomography System With High-Fidelity Surface Extraction Based on a Single Programmable Scanner and Unified Pinhole Modeling,” IEEE Transactions on Biomedical Engineering, pp. 1–13 (2023).
29. S. R. Arridge and J. C. Schotland, “Optical tomography: forward and inverse problems,” Inverse problems 25(12), 123010 (2009). [CrossRef]
30. J. Ripoll, R. B. Schulz, and V. Ntziachristos, “Free-space propagation of diffuse light: theory and experiments,” Phys. Rev. Lett. 91(10), 103901 (2003). [CrossRef]
31. M. Schweiger, S. R. Arridge, M. Hiraoka, et al., “The finite element method for the propagation of light in scattering media: boundary and source conditions,” Med. Phys. 22(11), 1779–1792 (1995). [CrossRef]
32. S. Arridge, M. Schweiger, M. Hiraoka, et al., “A finite element approach for modeling photon transport in tissue,” Med. Phys. 20(2), 299–309 (1993). [CrossRef]
33. J. L. Morales and J. Nocedal, “Remark on ’Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization’,” ACM Trans. Math. Softw. 38(1), 1–4 (2011). [CrossRef]
34. B. W. Pogue and M. S. Patterson, “Review of tissue simulating phantoms for optical spectroscopy, imaging and dosimetry,” J. Biomed. Opt. 11(4), 041102 (2006). [CrossRef]
35. J. Baeten, M. Niedre, J. Dunham, et al., “Development of fluorescent materials for diffuse fluorescence tomography standards and phantoms,” Opt. Express 15(14), 8681–8694 (2007). [CrossRef]
36. S. H. Kim, J. R. Gunther, and J. A. Katzenellenbogen, “Nonclassical SNAPFL analogue as a Cy5 resonance energy transfer partner,” Org. Lett. 10(21), 4931–4934 (2008). [CrossRef]
37. Y. Zhao, S. Li, X. He, et al., “Liver injury monitoring using dynamic fluorescence molecular tomography based on a time-energy difference strategy,” Biomed. Opt. Express 14(10), 5298–5315 (2023). [CrossRef]
38. H. Kudo, T. Suzuki, and E. A. Rashed, “Image reconstruction for sparse-view CT and interior CT-introduction to compressed sensing and differentiated backprojection,” Quantitative imaging in medicine and surgery 3(3), 147 (2013). [CrossRef]
39. D. Zhu, Y. Zhao, R. Baikejiang, et al., “Comparison of Regularization Methods in Fluorescence Molecular Tomography,” Photonics 1(2), 95–109 (2014). [CrossRef]
40. J.-C. Baritaux, K. Hassler, and M. Unser, “An Efficient Numerical Method for General Lp Regularization in Fluorescence Molecular Tomography,” IEEE Trans. Med. Imaging 29(4), 1075–1087 (2010). [CrossRef]
41. P. Zhang, C. Ma, F. Song, et al., “A review of advances in imaging methodology in fluorescence molecular tomography,” Phys. Med. Biol. 67(10), 10TR01 (2022). [CrossRef]
42. D. E. Hyde, R. B. Schulz, D. H. Brooks, et al., “Performance dependence of hybrid x-ray computed tomography/fluorescence molecular tomography on the optical forward problem,” J. Opt. Soc. Am. A 26(4), 919 (2009). [CrossRef]
43. F. Stuker, J. Ripoll, and M. Rudin, “Fluorescence Molecular Tomography: Principles and Potential for Pharmaceutical Research,” Pharmaceutics 3(2), 229–274 (2011). [CrossRef]
44. W. Ren, J. Jiang, A. D. C. Mata, et al., “Multimodal imaging combining time-domain near-infrared optical tomography and continuous-wave fluorescence molecular tomography,” Opt. Express 28(7), 9860–9874 (2020). [CrossRef]
45. M. Cai, Z. Zhang, X. Shi, et al., “NIR-II/NIR-I Fluorescence Molecular Tomography of Heterogeneous Mice Based on Gaussian Weighted Neighborhood Fused Lasso Method,” IEEE Trans. Med. Imaging 39(6), 2213–2222 (2020). [CrossRef]
46. C. Cao, A. Xiao, M. Cai, et al., “Excitation-based fully connected network for precise NIR-II fluorescence molecular tomography,” Biomed. Opt. Express 13(12), 6284–6299 (2022). [CrossRef]
47. X. Shi, L. Guo, J. Liu, et al., “NIR-IIb Fluorescence Molecular Tomography of Glioblastomas Based on Heterogeneous Mouse Models and Adaptive Projection Match Pursuit Method,” IEEE Trans. Biomed. Eng. 70(8), 2258–2269 (2023). [CrossRef]
