Open Access
Add to BibSonomy
Abstract
Microwave-induced thermoacoustic (TA) imaging (MTAI) combines pulsed microwave excitation and ultrasound detection to provide high contrast and spatial resolution images through dielectric contrast, which holds great promise for clinical applications. However, artifacts caused by microwave dielectric effect will seriously affect the accuracy of MTAI images that will hinder the clinical translation of MTAI. In this work, we propose a deep learning-based method fully dense generative adversarial network (FD-GAN) for removing artifacts caused by microwave dielectric effect in MTAI. FD-GAN adds the fully dense block to the generative adversarial network (GAN) based on the mutual confrontation between generator and discriminator, which enables it to learn both local and global features related to the removal of artifacts and generate high-quality images. The practical feasibility was tested in simulated, experimental data. The results demonstrate that FD-GAN can effectively remove the artifacts caused by the microwave dielectric effect, and shows superiority in denoising, background suppression, and improvement of image distortion. Our approach is expected to significantly improve the accuracy and quality of MTAI images, thereby enhancing the diagnostic accuracy of this innovative imaging technique.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Microwave-induced thermoacoustic (TA) imaging (MTAI) is a hybrid imaging modality which combines microwave excitation with ultrasound detection [1–3]. In MTAI, pulsed microwaves provide excitation energy, which can be absorbed by biological tissues and generate ultrasound waves through thermal expansion to obtain physiological and pathological information about the tissues [4,5]. Based on these principles, MTAI has been widely used in a variety of biomedical applications, such as breast tumor screening [6–9], joint disease diagnosis [10], brain imaging [11–14], and MTAI endoscopy [15]. MTAI exploits the differences in electromagnetic properties, such as electrical conductivity and dielectric constant, between the lesion and the background biological tissue to provide high endogenous contrast for MTAI [16]. The longer wavelength of MTAI produces a relatively deeper penetration depth than laser-induced photoacoustic imaging [17–21]. MTAI combines the high contrast advantages of microwave imaging with the high-resolution advantages of ultrasound imaging, making it a great potential in the field of biomedical imaging [22–25].
When the size of an object is smaller than the microwave wavelength, we usually assume that the electric-field distribution of the object is uniform. However, when the size of the object is smaller than the microwave wavelength, artifacts caused by the microwave dielectric effect are present in the object, resulting in the accuracy of the reconstructed microwave thermoacoustic image results being compromised. The microwave dielectric effect refers to the interaction of matter with the microwave field, as the wavelength is inversely proportional to the square root of the dielectric constant, and most biological tissues have a high dielectric constant, resulting in much shorter wavelengths in the tissue than in the air, which leads to the emergence of equal sized standing wave currents on both sides of the tissue towards the middle, which either cancels out or rises up, thus forming inhomogeneous patterns of light and dark [26–28]. Under linearly polarized irradiation conditions, the imaged target shows split image artifacts [29,30], while under circularly polarized irradiation conditions, the imaged target TA response shows the hollow donut pattern [31]. When multiple phantoms with the same parameters are imaged under microwave irradiation, the artifacts caused by dielectric effect are manifested as the sensitivity of the MTAI image modes to the boundary conditions of the vector Helmholtz equation due to the interference effect, and the inhomogeneity of the TA response generated within multiple phantoms with the same parameters [32]. In addition, to reduce patient discomfort and panic in clinical applications, MTAI usually uses linear ultrasound transducer arrays, and its planar detection enables flexible localization in the human body. However, this leads to the generation of limited-angle artifacts, which manifest as curved streak features stretched on both sides of the reconstructed imaging target and missing part of the image information, leading to degradation of the reconstructed image quality and hindering the determination of the actual contour of the target [16,19]. Therefore, removing image artifacts and improving the quality of the reconstructed images facilitate the determination of the specific structure and actual contour of the region of interest, providing a bright future for the practical application of MTAI in the clinical setting.
In recent years, deep learning (DL) methods have achieved great success in medical imaging. DL-based methods, especially convolutional neural networks (CNN), have promising applications in MTAI. Xu et al. applied CNN to MTAI, which achieved better performance in terms of artifact removal and robustness [33]. Zhang et al. based on DL and employed a signal-to-image domain conversion mechanism that imposed two input signals, achieving a further breakthrough in imaging performance [34]. Li et al. addressed the adverse effect of the acoustic heterogeneity using deep-learning-enabled microwave-induced thermoacoustic tomography (DL-MITAT) for transcranial brain hemorrhage detection [35]. U-Net is a CNN architecture widely used to apply deep learning in sparse data image reconstruction, and its multi-level decomposition and multi-channel filtering are well suited for artifact removal but has its limitations for compensating for missing image information [36]. FD-UNet adds dense connection structure to U-Net, which makes CNN more compact and superior, and its ability to remove artifacts is better than standard U-Net after comparison [37]. Generative adversarial network (GAN) is one of the latest advances and most important breakthroughs in medical imaging, with two mutually adversarial models, the generator and the discriminator, capable of synthesizing realistic images with arbitrary inputs [38]. Applications of GAN in medical imaging include image reconstruction and segmentation, with competing models suitable for compensating for missing image information and correcting image distortions. However, the disadvantage of GAN is that the training is unstable and when a particular model is too powerful, it is prone to gradient disappearance [39].
In this work, we propose an improved CNN architecture, called fully dense generative adversarial network (FD-GAN), for removing artifacts caused by microwave dielectric effect in MTAI. Dense blocks are added by FD-GAN to the generator model of GAN, which mitigates the learning of redundant features, makes the generator structure more compact, and reduces gradient vanishing, as well as the mutual confrontation between the network models is suitable for compensating the missing image information. Then, we tested the imaging quality and artifact removal ability of FD-GAN through simulations and experiments. The quantitative and qualitative results show that the output image of FD-GAN can effectively remove the artifacts caused by the microwave dielectric effect, output a higher-quality image, and effectively compensate for the missing image information caused by the limited angle.
2. Materials and methods
2.1 Artifacts caused by microwave dielectric effect
2.1.1 Mode effect in MTAI
Figure 1(a) shows the TA response of different shapes of phantoms under homogenous field microwave irradiation, linearly polarized microwave irradiation and circularly polarized microwave irradiation. When the target diameter is smaller than the microwave wavelength, the artifacts under linearly polarized microwave irradiation are manifested as splitting artifact in the imaging target, which is due to the mode effect of the imaging target, where the polarized charges are aligned along the direction of polarization, generating a splitting distribution of currents in the horizontal cross-section, which ultimately produces a similar distribution of electric fields in the horizontal cross-section [32,40]. Unlike linearly polarized microwave irradiation, in which the electric field at a fixed point in space still points in a fixed direction, circularly polarized microwave irradiation consists of polarization planes rotating in a helical pattern, making a complete rotation at each wavelength, and thus circularly polarized can provide a more uniform irradiation than linearly polarized. However, in circularly polarized irradiation, samples with diameters smaller than the microwave wavelengths are still subjected to mode effect under microwave irradiation, which are manifested by a gradual decay of the TA response from the edges to the center, and the final TA response distribution shows the hollow donut pattern. Figure 1(b) shows the profile comparison of Fig. 1(a) homogenous field microwave irradiation, linearly polarized microwave irradiation and circularly polarized microwave irradiation along the direction of the red dashed line. It can be seen from the graphs that there are varying degrees of missing TA responses of linearly polarized microwave irradiation and circularly polarized microwave irradiation under the influence of artifacts caused by the microwave dielectric effect.
Fig. 1. (a) Schematic diagram of TA response with different phantoms under homogeneous field microwave irradiation, linearly polarized microwave irradiation and circularly polarized microwave irradiation. (b) Comparison of profiles of homogeneous field microwave irradiation, linearly polarized microwave irradiation and circularly polarized microwave irradiation along the red dashed line in (a).
2.1.2 Interference effect in MTAI
Figure 2 shows the TA response of one to several phantoms with the same parameters under homogenous field microwave irradiation, linearly polarized microwave irradiation and circularly polarized microwave irradiation. When the wavelength of the microwave is comparable to the size of the target, artifacts caused by microwave dielectric effect manifest as interference effect between multiple imaging targets, and imaging targets under circularly polarized microwave irradiation are more significantly affected by the interference effect. When interference effect is generated, the boundary conditions will play a key role in determining the shape of the image pattern, as evidenced by the fact that the internal TA response uniformity of the imaging target far from the center is affected by the interference of other targets. Imaging targets far from the center suffer from internal TA response uniformity under the interference of other targets. This phenomenon may be related to the microscopic current-induced Lorentz force [32].
Fig. 2. Schematic diagram of TA response of one to more phantoms with the same parameters under homogeneous field microwave irradiation, linearly polarized microwave irradiation and circularly polarized microwave irradiation.
2.2 FD-GAN and model structure
FD-GAN is built based on the GAN framework, which exists in two parts: the generator and the discriminator. The generator is used to generate high-quality TA images from the original TA images with artifacts, and the discriminator is used to distinguish the generated high-quality TA images from the ground truth images, as shown in Fig. 3. Optimization between the generator and discriminator is performed using adversarial training, where the goal of the generator is to minimize the difference between the generated image and the real image, while the goal of the discriminator is to maximize the accuracy of distinguishing the generated image from the real image, and their loss functions are designed to combine adversarial loss and mean square error (MSE). Specifically, our goal is to minimize
where x is the TA image with artifacts, $G(x)$ is the generator output, $D(.)$ is the discriminator prediction of an image which be generated high-quality TA image or ground truth image, and Z is the full-view image without artifacts used as ground truth. The definition of MSE is where N is the total number of image pixels. Although the generator learns to convert the original TA image with artifacts into a full-view high-quality TA image by fighting against losses, the inclusion of the MSE term ensures that the output image of the generator matches the input image exactly at the pixel level [41]. In addition, the MSE term helps stabilize the training process and smooth the loss curve, making it easier to converge the training process to an equilibrium state.Fig. 3. Architecture of FD-GAN: (a) generator network, (b) discriminator network. Hyperparameters for the shown architecture are ${k_1}$ = 8 and ${f_1}$ = 64 for an input image X of 256 × 256 pixels.
For the generator, the fully dense block based on FD-UNet is applied, consisting of five up-blocks and five down-blocks, as shown in Fig. 3(a). The input TA image is passed through multiple convolutional and downsampling layers for feature extraction, thus representing the input image as a smaller coding tensor. This coding tensor is then fed into multiple deconvolutional and upsampling layers, and the coding tensor is recovered into a high-resolution TA image through a process of gradual upsampling of these layers. Max Pooling layers are used to iteratively reduce the spatial dimension of the feature map, which allows the convolutional neural network to efficiently learn local and global features related to artifact removal and to learn information at different spatial scales. In this process, each deconvolution layer connects the upsampling feature maps of the previous layer with the channel of the encoded feature maps from the same resolution to better preserve the detailed information of the image.
In the generator, each spatial level s has a dense block with a growth rate ${k_s}$ for learning multiple feature maps ${f_s}$. ${k_s}$ and ${f_s}$ are defined as
where the initial values ${f_1}$ and ${k_1}$ are determined by the user-defined hyperparameters. To maintain computational efficiency, all dense blocks in FD-GAN have the same number of convolutional layers. In the dense block, the output of each layer is passed to subsequent layers through channel connections, thus enabling feature reuse. This strategy can effectively enhance the expressiveness of the network. The features learned in the previous layers are passed backward through connections, thus avoiding the network from learning redundant features and further promoting the diversity of feature learning. This feature reuse approach can also effectively improve the training speed and accuracy of the network while reducing the risk of overfitting. This encoder-decoder paired with the dense blocks’ architecture can learn feature information at different scales and levels in TA images and integrate them effectively to ultimately generate high-quality TA images.For the discriminator, as shown in Fig. 3(b), it consists of four convolutional blocks, which are finally output by sigmoid layers. Each convolutional block consists of a convolutional layer with stride set to 2, a batch normalization layer, and a ReLU layer. Among them, the convolutional layer is used to extract local features of the input image, the batch normalization layer is used to normalize the mean and variance of the input features to accelerate the training of the network and improve the stability of the model, and the ReLU layer is used to increase the nonlinearity of the network. The four convolution blocks serve to extract features from the input image at multiple levels to gradually reduce the spatial size and number of channels of the input image as well as to increase the level of abstraction of the image. The discriminator has fewer parameters, which can make the discriminator pay more attention to the details of the image and improve the realism of the image. In addition, the discriminator runs faster because the discriminator has a simpler structure and does not have too many parameters to be trained [42]. This is also an advantage of the discriminator part of the GAN model, which can make the training process more efficient and stable.
Input images of 515 × 515 pixels are used to train the generator and the discriminator, and the input images are converted into images of 256 × 256 pixels before being fed into the generator, with the batch size set to 2. In each iteration of training, the generator and the discriminator are updated with parameters at the same time, and both networks are randomly initialized and optimized using the adaptive moment estimation (Adam) optimizer with the learning rate of 1 × 10−4 [43]. Adam optimizer's purpose is to adaptively adjust the learning rate of each parameter during the training process to effectively train the model and accelerate the convergence. And the learning rate is determined by optimizing and adjusting during the training process. When the learning rate is 1 × 10−4, the loss functions of generator and discriminator tends to stabilize. This means that with each iteration, their parameters are adjusted to better fit the training data. All training for FD-GAN is performed on an NVIDIA RTX 3060 GPU using Keras 2.6.0 and TensorFlow 2.6.0.
2.3 MTAI experimental setup
The schematic diagram of the MTAI experimental system is shown in Fig. 4, which is used to generate the experimental training data and test data required for FD-GAN. In the MTAI system, the computer controls the microwave source to generate high frequency pulsed microwaves with a carrier frequency of 6 GHz and a pulse width of 500 ns, as well as to start the real-time imaging acquisition system for data acquisition. The microwave antenna is connected to the microwave source via a coaxial cable for irradiating the column phantom. We use a single line focus transducer with a center frequency of 2 MHz, a bandwidth of 100%, and a sensitivity of 20 dB to detect the generated acoustic signal, which is scanned at different angles around the column phantom with a scanning radius of 63 mm. The measured acoustic signals are amplified by a preamplifier, filtered and output data in a real-time imaging acquisition system. A similar operation is performed on mouse brains to generate experimental training data and test data to further explore the performance of FD-GAN.
2.4 Generation of simulation and experimental dataset
2.4.1 Simulation
COMSOL Multiphysics 5.6 (COMSOL Co. Ltd.) was used to construct polygonal models (cylindrical, trigonal and rectangular) with reference to properties of tumor. The tumor has a relative permittivity of ${\varepsilon _r}\textrm{ = 55,}$ a conductivity of ${\sigma _1}\textrm{ = 2 S/m,}$ and the relative density of $\rho \textrm{ = 1050 kg/}{\textrm{m}^3}$ [44,45]. And we set microwave and electric field outputs, set the output frequency to 3.05 GHz, and set the port mode to output a linearly polarized microwave field. Then we use the finite element method simulation to calculate the microwave specific absorption rate (SAR) distribution, which is defined as
The k-Wave toolbox was invoked to perform two sets of simulations on the SAR distribution images, each creating 1000 training sets and 200 test sets [46], corresponding to linearly and circularly polarized irradiation. We add Gaussian noise to the simulated TA signal with a signal-to-noise ratio (SNR) of 20 dB. The TA images are reconstructed from finite view detection and full view (360°) detection using the delay and sum (DAS) algorithm [47]. The image size is 515 × 515 pixels. To simulate the actual sample in the coupling medium more accurately in the simulation, the speed of sound (SOS) is 1400 m/s, and the density of the coupling medium is set to 900 kg/m3. In addition, to maintain the realism of the simulation, the whole area is set to have no acoustic loss to avoid the attenuation of the acoustic signal in the simulation.
The training set was enriched by changing some of our parameters. Specifically, considering that samples with different diameters have different SAR distributions under microwave irradiation, the sample diameters in each training set are varied from 5 mm to 10 mm in a random manner. Since the samples tend to exhibit different SAR distributions due to different placement, we randomize the sample positions in the training set and rotate the samples in a randomized manner. Moreover, we construct the simulation dataset using different polygonal samples. These operations greatly enhance the richness of the dataset.
2.4.2 Experiment
A custom MTAI system with a 2 MHz unit-line focusing sensor was used to acquire experimental TA images for two-dimensional FD-GAN training [48]. We constructed the experimental dataset by means of water-filled plastic tubes with diameters ranging from 2 mm to 5 mm randomly distributed in locations under 6 GHz linearly polarized irradiation, as well as mouse brains under 6 GHz circularly polarized irradiation. View cases of 30°, 60°, 90°, 180° and 360° for the plastic tube and the mouse brain are reconstructed from TA images by the DAS algorithm. The plastic tube datasets constructed in the experiments were all used for testing, mainly to verify whether the FD-GAN network trained by the simulation dataset could make accurate judgments and generalization ability of the output when applied to the data from the experiments. And the obtained 408 pairs of mouse brain datasets were used 80% for training data, 10% for validation data, and 10% for test data to further characterize the performance of FD-GAN.
3. Results
3.1 MTAI simulations test data
A comparison of the performance of FD-GAN with FD-UNet and GAN under line polarization microwave irradiation and circular polarization microwave irradiation at 90° view is shown in Fig. 5. For the different simulated polygon phantoms, as shown in Fig. 5(a), the FD-UNet method effectively reduces a large amount of background noise and artifacts, but the images are still distorted due to the interference caused by the splitting artifacts in the network model in the case of linearly polarized microwave irradiation. The artifacts in the GAN results and FD-GAN results are mostly reduced, and the splitting artifacts caused by the line polarization microwave irradiation are significantly reduced. However, GAN still has significant background noise compared with FD-GAN. For multiple simulated phantoms, as shown in Fig. 5(b), the farther the target phantom is from the sensor, the more disturbed it is, and the reconstructed image shows more artifacts and image distortion due to the limited field of view. For the phantom with the most interference (indicated by the yellow arrow), FD-GAN is able to extract enough useful features to faithfully reconstruct the phantom, and its reconstruction effect is better than that of GAN and FD-UNet. The simulation results of the simple model show that the FD-GAN method significantly reduces background noise and artifacts while preserving image details and improves the quality of the reconstructed image.
Fig. 5. Simulation data under 90° view angle linearly polarized microwave irradiation and circularly polarized microwave irradiation using different networks: (a) Different simulation polygon phantom imaging results. (b) Multiple same simulation phantom imaging results.
Figure 6 shows the results of peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM) comparison the input images [49], FD-UNet results, GAN results and FD-GAN results for different viewing angles (covering 15°, 30°, 60°, 90°, 180° and 360°) and linearly polarized microwave irradiation conditions. The results shows that the PSNR and SSIM of all images show an increasing trend with increasing detection angle. The quantitative metrics of PSNR and SSIM indicate that the FD-GAN results have the best image quality, fidelity, and contrast performance in all restricted field of view and linearly polarized microwave irradiation cases.
Fig. 6. Comparison of different DL methods applied to the input images with different covering angles and linearly polarized microwave irradiation conditions. (a) PSNR metrics. (b) SSIM metrics.
3.2 MTAI column phantom data
To evaluate the generalization ability of the network model, we tested the model using the MTAI dataset of plastic tubes filled with water under 6 GHz linear polarized microwave irradiation. Figure 7(a) shows the results of removing artifacts by different methods. In the case of the real experimental plastic tube, the TA image reconstruction does not perform as well as shown in Fig. 6, and the image distortion becomes more and more severe as the detection angle decreases. From Fig. 7, the GAN model trained on the simulated dataset cannot be effectively applied to the real experimental dataset, which has severe background noise and artifacts. Although FD-UNet removes the artifacts, the output image of FD-UNet model has some blurring and distortion due to the distortion and information loss caused by the finite angle and the splitting artifacts caused by the line polarization. As expected, the artifacts are best removed using FD-GAN, and the details and edges of the images are well preserved, which indicates that the FD-GAN model has good generalization ability and can be applied to experimental datasets and practical applications. Figure 7(b) is the comparison of the profiles of different networks along the red dashed line in Fig. 7(a), which shows that the FD-GAN output image fits the input image best and effectively improves the missing information caused by the splitting artifacts.
Fig. 7. (a) Experimental results of MTAI data for water-filled plastic tubes with different detection coverage angles under 6 GHz linearly polarized microwave irradiation using different networks. (b) Comparison of the profiles of the different networks along the red dashed line with the input image.
3.3 MTAI mouse brain data
To further demonstrate that our method can be applied to in vivo imaging and characterize the performance of network models, mouse brain TA images under 6 GHz circularly polarized microwave irradiation were studied for artifact removal. Compared with simple phantoms, mouse brain usually has a relatively complex structure and lower image quality, and the difference in dielectric properties between the coupling medium and mouse brain tissue can lead to inhomogeneous distribution of electric field energy in the brain tissue. Therefore, artifact removal processing of TA images of mouse brain is more challenging.
Figure 8 shows the output results of the network model under different detection angles. As can be seen from Fig. 8(b), the input image can only determine the approximate location of the mouse brain tissue in the limited detection angle result, and we cannot even identify the outline of the mouse brain from the 30° case. However, the output results from FD-GAN show that the TA images reconstructed by FD-GAN can still clearly identify the size outline and location of mouse brain tissues even with a detection angle of 30°. In addition to this, the output of FD-GAN can effectively reduce the black artifacts in the middle region of the mouse brain due to microwave dielectric effect and make the internal details of the image clearer compared to the input results. The complexity of mouse brain structures and the inhomogeneous distribution of electric fields in brain tissues lead to discrepancies with previous simulation data and simple phantom experimental data. However, the input images output TA reconstructed images by FD-GAN, whose edge artifacts and image distortion are alleviated, and it clearly shows the structure of mouse brain tissues with high contrast as the detection angle increases. Moreover, the performance of FD-GAN in removing artifacts is evaluated using contrast-to-noise ratio (CNR), which further considers the difference between the mean values of the signal region and background noise [50]. Figure 8(c) gives the quantitative results, where a high signal-to-noise improvement is obtained for FD-GAN compared to the input with artifacts. Figure 8(d) is the comparison of the profile along the red dashed line between the input image and the FD-GAN output image in Fig. 8(b), and it is easily seen that the FD-GAN effectively removes the artifacts caused by the dielectric effect under the circularly polarized irradiation and improves the missing TA information.
Fig. 8. Experimental results using FD-GAN for mouse brain data with different detection coverage angles under 6 GHz circularly polarized microwave irradiation. (a) Photo of the mouse brain. (b) Mouse brain data imaging results. For CNR calculations, the yellow box denotes the target region, and the green box denotes the background region. (c) Comparison of input images and FD-GAN results using CNR for different coverage angles. (d) Comparison of the profiles of the FD-GAN output image with the input image along the red dashed line.
4. Discussion
Although in this study we only show the application in two-dimensional conditions, the proposed FD-GAN method is also applicable to three-dimensional scenes. This is mainly since the basic procedure of the method is not limited by the applied dimensionality. Meanwhile, the 3D k-wave simulation allows us to construct the corresponding training set. However, potential 3D applications may face some challenges. The computational burden of acquiring the training set and training the network will be significantly higher compared to the two-dimensional case. In addition, the phenomenon of multiple reflections of microwaves at the target boundary need to be considered.
Nevertheless, there are some limitations of our approach. First, the effect of inhomogeneity of microwave sound speed is not incorporated in our forward model. In fact, microwave acoustic field inhomogeneity can lead to differences in SAR distribution within the sample. The currently built model has not been trained to optimize for the microwave sound field distribution. In future work, the speed of sound inhomogeneity will be considered in the simulation training set. Second, the DL network is target-specified and can only identify TA targets like those in the training dataset, and the network adaptive performance needs to be further enhanced. Therefore, the subsequent study will construct a general training dataset covering multiple types of TA targets. In addition, compared with U-Net, FD-GAN takes longer training and computation time, and is more time-consuming if 3D image reconstruction is performed. In many biomedical applications, there is an urgent need for fast and efficient methods to acquire real-time high-quality TA images. In the next work, the network structure will be optimized to reduce the training time and computation time to achieve the goal of fast elimination of TA image artifacts in real-time in clinical research. Finally, the method proposed in this study is only applicable when the TA target diameter is smaller than the microwave wavelength within the TA target. When the TA target diameter is larger than or close to the microwave wavelength, a more complex situation occurs for TA target imaging under linearly polarized microwave irradiation. The Mie-like scattering mode and dielectric effect exist stably in the cross-section [51]. In addition, the microwaves in the z-direction are reflected many times at the TA target boundary, so the obtained cross-section images show different inhomogeneous SAR distributions at different layer depths. And under circularly polarized irradiation, the TA target imaging appears an inhomogeneous SAR distribution with the middle bright and the surrounding dim. The imaging situation when the microwave wavelength is larger than the diameter of the imaging target will be considered in further research work to achieve the removal of TA image artifacts at multiple scales.
5. Conclusion
The FD-GAN network framework is proposed and developed for the removal of the artifacts caused by microwave dielectric effect and distortion caused by limited angle detection field of view and has demonstrated its practical application in the biomedical field. The generator uses a U-Net structure to extract multi-level image features and is connected by a fully dense block structure to improve artifact and image distortion removal performance. A GAN training strategy is used to generate high signal-to-noise ratio and high-quality images. The effect of linearly polarized microwave irradiation and circularly polarized microwave irradiation on the target is considered when constructing the simulated output dataset for training and testing the model. In addition, real experimental images from column phantoms and mouse brains are used to verify the feasibility of the network model in practical applications. The results show that compared with other DL methods (FD-UNet, GAN), our proposed method can remove the artifacts caused by microwave dielectric effect more effectively and has advantages in denoising, background suppression, and effective removal of image distortion. The DL method can capture complex SAR distributions of diseased tissues with arbitrary shapes and sizes to effectively discover and localize lesion areas, which is important for advancing and developing MTAI technology for clinical applications.
Funding
National Natural Science Foundation of China (62075066, 62375088); Basic and Applied Basic Research Foundation of Guangdong Province (2023A1515010824).
Acknowledgments
Thanks to the Third Affiliated Hospital of Sun Yat-Sen University for the support of the biological sample.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. Z. Chi, Y. Zhao, L. Huang, et al., “Thermoacoustic imaging of rabbit knee joints,” Med. Phys. 43(12), 6226–6233 (2016). [CrossRef]
2. S. Li, E. Fear, and L. Curiel, “Breast tissue mimicking phantoms for combined ultrasound and microwave imaging,” Phys. Med. Biol. 66(24), 245011 (2021). [CrossRef]
3. S. Zhang, W. Li, X. Chen, et al., “Manganous-manganic oxide nanoparticle as an activatable microwave-induced thermoacoustic probe for deep-located tumor specific imaging in vivo,” Photoacoustics 26, 100347 (2022). [CrossRef]
4. C. C. Johnson and A. W. Guy, “Nonionizing electromagnetic wave effects in biological materials and systems,” Proc. IEEE 60(6), 692–718 (1972). [CrossRef]
5. R. A. Kruger, K. K. Kopecky, A. M. Aisen, et al., “Thermoacoustic CT with Radio Waves: A Medical Imaging Paradigm,” Radiology 211(1), 275–278 (1999). [CrossRef]
6. M. Ren, Z. Cheng, L. Wu, et al., “Portable Microwave-Acoustic Coaxial Thermoacoustic Probe With Miniaturized Vivaldi Antennas for Breast Tumor Screening,” IEEE Trans. Biomed. Eng. 70(1), 175–181 (2023). [CrossRef]
7. F. Ye, Z. Ji, W. Ding, et al., “Ultrashort Microwave-Pumped Real-Time Thermoacoustic Breast Tumor Imaging System,” IEEE Trans. Med. Imaging 35(3), 839–844 (2016). [CrossRef]
8. B. Wang, Z. Guo, Z. Zhao, et al., “Microwave and Induced Thermoacoustic Dual Imaging for Potential Breast Cancer Detection,” in 2018 IEEE Asia-Pacific Conference on Antennas and Propagation (APCAP), (IEEE, 2018), pp. 175–178.
9. L. Wu, Z. Cheng, Y. Ma, et al., “A Handheld Microwave Thermoacoustic Imaging System With an Impedance Matching Microwave-Sono Probe for Breast Tumor Screening,” IEEE Trans. Med. Imaging 41(5), 1080–1086 (2022). [CrossRef]
10. Z. Chi, L. Huang, S. Ge, et al., “Technical Note: Anti-phase microwave illumination-based thermoacoustic tomography of in vivo human finger joints,” Med. Phys. 46(5), 2363–2369 (2019). [CrossRef]
11. L. Lanbo, H. Kuang, and V. W. Lihong, “Transcranial ultrasonic wave propagation simulation: skull insertion loss and recovery,” Proc. SPIE 6437, 64370X (2007). [CrossRef]
12. M. Xu and L. V. Wang, “Analytic explanation of spatial resolution related to bandwidth and detector aperture size in thermoacoustic or photoacoustic reconstruction,” Phys. Rev. E 67(5), 056605 (2003). [CrossRef]
13. A. Yan, L. Lin, C. Liu, et al., “Microwave-induced thermoacoustic tomography through an adult human skull,” Med. Phys. 46(4), 1793–1797 (2019). [CrossRef]
14. X. Yuan and L. V. Wang, “Rhesus monkey brain imaging through intact skull with thermoacoustic tomography,” IEEE Trans. Ultrason., Ferroelect., Freq. Contr. 53(3), 542–548 (2006). [CrossRef]
15. X. Liang, H. Guo, Q. Liu, et al., “Thermoacoustic endoscopy,” Appl. Phys. Lett. 116(1), 013702 (2020). [CrossRef]
16. Q. Liu, X. Liang, W. Qi, et al., “Biomedical microwave-induced thermoacoustic imaging,” J. Innov. Opt. Health Sci. 15(04), 2230007 (2022). [CrossRef]
17. J. Lv, Y. Xu, L. Xu, et al., “Quantitative Functional Evaluation of Liver Fibrosis in Mice with Dynamic Contrast-enhanced Photoacoustic Imaging,” Radiology 300(1), 89–97 (2021). [CrossRef]
18. X. Tang, J. Fu, and H. Qin, “Microwave-induced thermoacoustic imaging with functional nanoparticles,” J. Innov. Opt. Health Sci. 16(02), 2230014 (2023). [CrossRef]
19. H. Zhang, M. Ren, S. Zhang, et al., “Microwave-induced thermoacoustic imaging for biomedical applications,” Phys. Scr. 98(3), 032001 (2023). [CrossRef]
20. J. Zhang, X. Sun, H. Li, et al., “In vivo characterization and analysis of glioblastoma at different stages using multiscale photoacoustic molecular imaging,” Photoacoustics 30, 100462 (2023). [CrossRef]
21. Z. Fan, X. Jiang, T. Sun, et al., “In vivo visualization of tumor-associated macrophages re-education by photoacoustic/fluorescence dual-modal imaging with a metal-organic frames-based caspase-1 nanoreporter,” J. Colloid Interface Sci. 659, 48–59 (2024). [CrossRef]
22. X. Wang, D. Bauer, R. Witte, et al., “Impact of microwave pulses on microwave-induced thermoacoustic imaging applications,” in 2013 USNC-URSI Radio Science Meeting (Joint with AP-S Symposium), (IEEE, 2013), pp. 210.
23. G. Ku and L. V. Wang, “Scanning microwave-induced thermoacoustic tomography: Signal, resolution, and contrast,” Med. Phys. 28(1), 4–10 (2001). [CrossRef]
24. L. Zhang, H. Qin, F. Zeng, et al., “A stimulated liquid–gas phase transition nanoprobe dedicated to enhance the microwave thermoacoustic imaging contrast of breast tumors,” Nanoscale 12(30), 16034–16040 (2020). [CrossRef]
25. B. Wang, Y. Sun, Z. Wang, et al., “Three-Dimensional Microwave-Induced Thermoacoustic Imaging Based on Compressive Sensing Using an Analytically Constructed Dictionary,” IEEE Trans. Microwave Theory Tech. 68(1), 377–386 (2020). [CrossRef]
26. C. Gabriel, S. Gabriel, and E. Corthout, “The dielectric properties of biological tissues: I. Literature survey,” Phys. Med. Biol. 41(11), 2231–2249 (1996). [CrossRef]
27. C. M. Collins, W. Liu, W. Schreiber, et al., “Central brightening due to constructive interference with, without, and despite dielectric resonance,” J. Magn. Reson. Imaging 21(2), 192–196 (2005). [CrossRef]
28. A. G. Webb and C. M. Collins, “Parallel transmit and receive technology in high-field magnetic resonance neuroimaging,” Int. J. Imaging Syst. Technol. 20(1), 2–13 (2010). [CrossRef]
29. Y. He, Y. Shen, X. Feng, et al., “Homogenizing microwave illumination in thermoacoustic tomography by a linear-to-circular polarizer based on frequency selective surfaces,” Appl. Phys. Lett. 111(6), 063703 (2017). [CrossRef]
30. A. Yan, L. Lin, S. Na, et al., “Large field homogeneous illumination in microwave-induced thermoacoustic tomography based on a quasi-conical spiral antenna,” Appl. Phys. Lett. 113(12), 123701 (2018). [CrossRef]
31. C. Li, M. Pramanik, G. Ku, et al., “Image distortion in thermoacoustic tomography caused by microwave diffraction,” Phys. Rev. E 77(3), 031923 (2008). [CrossRef]
32. W. Chao, W. Qi, Y. Gong, et al., “Electromagnetic Wave-Induced Vectorial Thermoacoustic Bioimaging,” IEEE Trans. Microwave Theory Techn. 72(2), 1266–1279 (2024). [CrossRef]
33. Q. Xu, Z. Zheng, and H. Jiang, “Deep learning for image reconstruction in thermoacoustic tomography,” Chinese Phys. B 31(2), 024302 (2022). [CrossRef]
34. J. Zhang, C. Li, W. Jiang, et al., “Deep-Learning-Enabled Microwave-Induced Thermoacoustic Tomography Based on Sparse Data for Breast Cancer Detection,” IEEE Trans. Antennas Propag. 70(8), 6336–6348 (2022). [CrossRef]
35. C. Li, Z. Xi, G. Jin, et al., “Deep-Learning-Enabled Microwave-Induced Thermoacoustic Tomography Based on ResAttU-Net for Transcranial Brain Hemorrhage Detection,” IEEE Trans. Biomed. Eng. 70(8), 2350–2361 (2023). [CrossRef]
36. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, (Springer, 2015), pp. 234–241.
37. S. Guan, A. A. Khan, S. Sikdar, et al., “Fully Dense UNet for 2-D Sparse Photoacoustic Tomography Artifact Removal,” IEEE J. Biomed. Health Inform. 24(2), 568–576 (2020). [CrossRef]
38. I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., “Generative Adversarial Nets,” Adv. Neural Inf. Process. Syst. 27, 8 (2014).
39. J. Peng, T. Guan, F. Liu, et al., “MND-GAN: A Research on Image Deblurring Algorithm Based on Generative Adversarial Network,” in 2023 42nd Chinese Control Conference (CCC), (IEEE, 2023), pp. 7584–7589.
40. X. Liang, Q. Liu, Z. Sun, et al., “Investigation of artifacts by mapping SAR in thermoacoustic imaging,” J. Innov. Opt. Health Sci. 14(05), 2150011 (2021). [CrossRef]
41. T. Lu, T. Chen, F. Gao, et al., “LV-GAN: A deep learning approach for limited-view optoacoustic imaging based on hybrid datasets,” J. Biophotonics 14(2), e202000325 (2021). [CrossRef]
42. P. Isola, J. Y. Zhu, T. Zhou, et al., “Image-to-Image Translation with Conditional Adversarial Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (IEEE, 2017), pp. 5967–5976.
43. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv, arXiv:1412.6980 (2014). [CrossRef]
44. M. Lazebnik, D. Popovic, L. McCartney, et al., “A large-scale study of the ultrawideband microwave dielectric properties of normal, benign and malignant breast tissues obtained from cancer surgeries,” Phys. Med. Biol. 52(20), 6093–6115 (2007). [CrossRef]
45. M. Lazebnik, L. McCartney, D. Popovic, et al., “A large-scale study of the ultrawideband microwave dielectric properties of normal breast tissue obtained from reduction surgeries,” Phys. Med. Biol. 52(10), 2637–2656 (2007). [CrossRef]
46. B. E. Treeby and B. T. Cox, “k-Wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields,” J. Biomed. Opt. 15(2), 021314 (2010). [CrossRef]
47. D. Feng, Y. Xu, G. Ku, et al., “Microwave-induced thermoacoustic tomography: Reconstruction by synthetic aperture,” Med. Phys. 28(12), 2427–2431 (2001). [CrossRef]
48. X. Chen, S. Zhang, J. Liu, et al., “Controlling dielectric loss of biodegradable black phosphorus nanosheets by iron-ion-modification for imaging-guided microwave thermoacoustic therapy,” Biomaterials 287, 121662 (2022). [CrossRef]
49. Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, (IEEE, 2003), pp. 1398–1402.
50. A. Rodriguez-Molares, O. M. H. Rindal, J. D’hooge, et al., “The Generalized Contrast-to-Noise Ratio: A Formal Definition for Lesion Detectability,” IEEE Trans. Ultrason., Ferroelect., Freq. Contr. 67(4), 745–759 (2020). [CrossRef]
51. H. Nan, T. C. Chou, and A. Arbabian, “Segmentation and artifact removal in microwave-induced thermoacoustic imaging,” in Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (IEEE, 2014), pp. 4747–4750.
