1. Introduction

Spectral imaging, in fields spanning astronomy to medicine, has proven to be a valuable tool in practical applications such as chemical detection, food sorting, non-invasive diagnosis and many others [1–3]. The ability to identify the spectrum of each pixel in the image of a scene provides rich data sets that can be used to detect different materials and processes. The data collected represent three-dimensional matrices where the x and y axes depict spatial information while wavelengths are plotted in the z-direction. When the number of wavelengths is large enough to reconstruct a continuous spectrum, this imaging modality is commonly referred to as hyperspectral imaging (HSI). On the other hand, if the number of collected spectral wavelengths is discrete and these bands are narrow and/or separated, it is then called multispectral imaging (MSI).

A trade off exists between the dynamic range of the image, its spatial resolution, and the time of acquisition. Faster integration times decrease the amount of light collected and lead to lower signal quality. Reducing the number of spectral bands on the other hand limits the amount of information available for post-acquisition classification. The ultimate goal of this paper is to find quantitative measures that will allow the transition from a HSI to a MSI mode without a significant decline in the ability to recognize target tissue. Such transition can benefit the two main classes of hyper and multi-spectral imagers available to date. In scanning systems, decreasing the number of bands needed results in faster scans. In non-scanning or snapshot systems reducing the number of channels yields higher spatial resolution of subfield images. Rapid spectral imaging is particularly important for dynamic targets or processes that require quick feedback, examples being surgical procedures and food sorting.

Our group is interested in using spectral approaches for endoscopic visualization of damaged heart tissue [4–8], including ablation lesions within the upper chambers of the heart. Such lesions are created using radiofrequency (RF) energy to destroy sources of abnormal electrical activity leading to atrial fibrillation. While this is a commonly used clinical procedure, repeated admissions are often reported among patients receiving RF treatment for atrial fibrillation [9]. The procedure’s high recurrence rate can be improved if surgeons were able to directly visualize necrotic tissue formation during the procedure. Yet, endoscopic visualization of RF lesions is challenging since visual differences in the color and contrast between lesions and non-ablated atrial tissue are negligible [10–12]. To solve this challenge, advanced spectral imaging approaches such as HSI can be helpful. Indeed, we have recently shown that autofluorescence-based HSI can reveal outlines of RF lesions even in a highly collagenous human left atria [4]. These bench experiments were conducted using HSI camera outfitted with a wide aperture, high numerical aperture objective and in absence of any motion. To implement this approach clinically, HSI has to be done via percutaneous access, which dramatically decreases the amount of light both delivered and collected. In addition, imaging of a beating heart requires fast acquisition to minimize contraction artifacts. Both constraints severely limit the amount of photons to be split between multiple spectral bands.

In this report we examine several strategies for choosing wavelength ranges and combinations with the overall goal to minimize the number of spectral bands and to enable faster acquisition. As a case study we used a library of 14 samples of left atrial tissue from adult pigs each having 2 to 5 endocardial ablation lesions. We started by examining the effect of increasing the size of the spectral step. We then considered binning spectral ranges and how this can affect classification power and signal quality. Finally, we investigated whether HSI cubes containing only two, three, or four bands can reveal the lesions. We compared the quality of the resulting component images to the ones extracted from high spectral resolution HSI cubes.

The metrics presented can be used to select a limited number of wavelengths towards in vivo imaging. Finding combinations of few spectral bands that are sufficient for lesion identification, would allow for simultaneous acquisition using the latest advances in fast scanning and high spatial resolution non-scanning imaging [13–17]. Overall, our studies offer several unbiased quantitative strategies as to how to limit the number of acquisition wavelengths while recognizing the target tissue with high accuracy. Insights from this work can be used in a wide range of biomedical and optical applications.

2. Methods

2.1 Tissue sources and ablation procedures

Ablations were performed on the endocardial surface of left atrial tissue freshly excised from pigs of mixed gender (Yorkshire, 5-6-month old, 180-260lb). Cardiac tissue samples were collected and used immediately after surgical training at the Washington Institute of Surgical Education at the George Washington University or from a local abattoir as described previously [4]. To expose the endocardial surface of excised hearts, the left and right atria were dissected along the atrioventricular groove, and along the interatrial septum to allow for the inversion of the tissue. The atria were then spread out for imaging. Atria were kept on ice throughout the experiments to minimize changes in the levels of chromophores and fluorophores. Radiofrequency energy was delivered with either non-irrigated (EP Technologies, Boston Scientific) or irrigated (LuxCath catheter, Nocturnal Product Development LLC, Cary, NC) ablation catheters. We have previously shown that spectral changes are non-significant between the two groups [6]. The 4 mm non-irrigated ablation tip was placed perpendicularly to the endocardial surface of porcine atrial tissue with ablation durations varying from 5 to 30 seconds and tip temperatures ranging between 50 and 70°C.

2.2 Hyperspectral image acquisition and analysis

Samples were illuminated with a 365nm UVA LED source (Mightex, Pleasanton, CA) placed ~10 cm from the tissue surface. A CCD camera (Nuance FX, PerkinElmer/CRi) combined with a liquid crystal tunable filter (LCTF) and outfitted with a Nikon AF Micro-Nikkor 60mm f/2.8D objective enabled spectral scanning through acquisition wavelengths yielding a stack of images, here on referred to as hypercube (Fig. 1(A)). The Nuance FX system is capable of capturing wavelengths from 420 to 720 nm at a spatial resolution of 1392x1040 pixels with spectral steps ranging from 1 to 100nm. To derive the quantitative and qualitative conclusions described in this paper, two independent sets of data were collected. The first set consisted of three HSI cubes acquired at 2nm steps, each cube containing 151 images. The second set included 14 cubes acquired at 10nm spectral steps, each cube containing 31 images. Smaller cubes were derived by downsampling the initial 2nm hypercubes.

 

Fig. 1 Hyperspectral Image acquisition and manual unmixing. A. Imaging set up comprised of Nuance FX camera and LCTF filter. Arrow points to an example of a hypercube. Example of spectral information of pixel is sketched in red. A raw image on the far right shows the difficulty of visualizing lesions with UV illumination. Average spectra of pixels from two regions of interests shown in red and green were used for the unmixing protocol. B. Example of corrected and normalized spectra for non-ablated and ablated cardiac tissue with an example of corresponding lesion component images (LCIs) used in the analysis presented this paper.

Download Full Size | PDF

Extracted spectra were corrected for the spectral sensitivity of the LCTF and the CCD chip and normalized as previously described [6]. HSI cubes of varying sizes were analyzed via the Nuance FX proprietary software using manual unmixing protocols [18], in which the user manually chooses two regions of interest corresponding to lesions and unablated tissue (Fig. 1(A)). Examples of component images for lesion site and unablated tissue are presented in Fig. 1(B). In this report, we use the lesion component images (LCIs) as the reference for quality assessment. Similar results were obtained using unablated tissue component (data not shown).

2.3 Image quality assessment

To quantify the image quality of various spectral ranges and wavelength combinations, we used several image quality descriptors. First, we computed the signal-to-noise ratio (SNR) of the spatial intensity profile across LCIs. A linear region-of-interest was placed across the center of the lesion and the resulting profile was then separated into signal and noise by means of wavelet-based decomposition [19,20]. SNR was then computed as: SNR = P_signal/P_noise, where P_signal and P_noise are the average power of the signal and noise respectively.

We also compared the LCIs derived from downsampled cubes to those found in the reference LCI which was derived from the 2nm hypercube. Towards this goal, four different classical image descriptors were used: the Mean Square Error (MSE), the Peak Signal to Noise Ratio (PSNR), the Structural Similarity Index Map (SSIM) and the Accuracy Index (Accu). MSE is computed by averaging the squared difference in intensity between the tested and reference images and represents a cumulative error measure. PSNR represents a measure of the peak error and was obtained by computing the ratio between the maximum power of possible signal and the power of the noise between a target and reference image according to the following equation for 8-bit gray images: $PSNR=10{\log }_{10}(\frac{{255}^2}{MSE}).$

The third index, SSIM, was developed as a measure of the changes in the image attributes independent of the luminance and the contrast of the image [21].

The fourth index, called Accuracy (Accu), was based on binary, pixel-to-pixel matching between pixels that were identified as being part of the lesion sites within the reference image and the image to be tested. First, the reference LCI was converted from gray-value to a binary (bi-reference) image using a threshold value defined by the user. Second, we applied similar user defined threshold to convert the LCI obtained from gray-value to binary (bi-test). Accuracy index (Accu) was then defined as: Accu = 1- Diff/N, where N is the total number of pixels identified as lesions in both images and ‘Diff’ was calculated by XOR operator to identify differences in both images. From the view of set theory, if I_r and I_t are the point’s sets for lesions in bi-reference and bi-test image, the Accuracy is: $Accu=\frac{2\left|I_r\cap I_t\right|}{\left|I_r+I_t\right|}.$

2.4 Fast unmixing to test the best 2, 3 and 4 cube combinations

Snapshot imaging depends on a limited number of bands to achieve results similar in SNR levels and accuracy of classification to the high spectral resolution HSI cubes. Towards this goal, we sought to compare the quality of component images generated from all possible random combinations of cubes with 2-, 3- and 4-spectral bands. Combinations were determined using binomial functions in Matlab (Appendix, Fig. 1). Corresponding coefficients were used to generate new data vectors and appropriately sized reference spectra. Matrix reshaping was applied to transform each cube from a [W, L, M] matrix into WxL = N distinct vectors each having M length. Each vector represents the spectrum of a pixel within the image. This matrix (size: NxM) was the observation (O). Reference spectra for lesions and unablated tissue were computed from the mean corresponding spectra of 14 studies (Fig. 1(B)). Spectra were then used to compute the component images as described previously by Xu and Rice [22]. This group of reference spectra forms another matrix (G, size: 2xM) having two rows, which stand for unablated and lesion spectra. C is the contribution matrix (Nx2) where each row represents the two composition weights of two reference spectra to form one row (vector) in the O. Therefore, it can be defined as a linear least squares (LS) problem: $O=CG+E$, where E is the error matrix (N x M). The goal is to find a C to minimize the error: $e={\Vert E\Vert }^2={\Vert CG-O\Vert }^2$, hence, $\frac{d{\Vert CG-O\Vert }^2}{dC}=0\Rightarrow CG=O$. Since G may not be a square matrix, we cannot get its inverse directly. So: $CG=O\Rightarrow CG{G}^T=O{G}^T\Rightarrow$$C=O{G}^T{\left(G{G}^T\right)}^{-1}$. For each column (N length) in C, we reshaped it to a [W, L] matrix. One of the two matrices shows the lesion component and another the unablated tissue component.

2.5 Statistical analysis

Values are presented as mean ± SEM unless otherwise noted. Statistical significance was determined using ANOVA. When significance was found, a Tukey post-hoc test for multiple comparisons was performed for pairwise comparisons. Significance was defined at the p < 0.05 level. Evaluation of the linearity between image quality measures and the different wavelengths was determined by calculating the R² coefficient of multiple correlations.

3. Results

3.1 Reduction of HSI cube size by increasing the spectral step

To quantify how reducing cube size by lowering spectral resolution can affect the signal quality of the LCI, we first looked at the noise in the image. Data were downsampled to create smaller cubes at 10 nm, 20 nm, 50 nm and 100 nm spectral steps, corresponding to 31, 16, 7 and 4 bands in each cube, respectively. The LCI obtained from unmixing the full, 2nm-step hypercube was used as a reference (Fig. 2). As expected, the line profile across the lesion sites reveal that SNR follows a decreasing linear trend with increased spectral step size (R² = 0.91) (Fig. 2(C)).

 

Fig. 2 Effect of step size on SNR of ablated component image. A. Ablated component images of porcine left atria showing 5 lesions. Component images were obtained by unmixing hypercubes with spectral step sizes of 2nm, 10nm, 20nm, 50nm and 100nm. The red dotted line indicates the line of pixels corresponding to spectra shown in B. B. Line profile of ablated component image obtained using component images shown in A). C. Quantified and Normalized Signal-to-Noise Ratio (SNR) of spectra shown in B (N = 3, n = 8). A decreasing linear trend between SNR and step size was found significant (R² = 0.91).

Download Full Size | PDF

To further quantify how the quality of the LCI changes with increased spectral steps, we compared four standard image quality assessment descriptors as outlined in the Methods. Analysis of the MSE, PSNR and SSIM indices revealed similar, statistically significant linear trends (Fig. 3(A)-3(C)). An ascending linear relation was identified for MSE (R2 = 0.91, p<0.05), while both PSNR and SSIM indices show a decreasing linear relation with increased step size (R2 = 0.91 and 0.98 respectively). Accuracy index, on the other hand, revealed significant differences between LCIs only when spectral step increased to 100 nm while LCIs obtained using 50nm and 100nm steps showed no statistical significance despite a sharp decrease in Accu from a mean of 0.89 at 50nm to a mean of 0.67 at 100nm (Fig. 3(D)).

 

Fig. 3 Effect of spectral step size on the quality of ablated component images. A-D. Mean Standard Error, Peak Signal to Noise Ratio (PSNR), Structural Similarity Matrix (SSIM) and Accuracy indices were computed to compare the quality of LCIs obtained from unmixing under sampled hypercubes (10, 20, 50 and 100 nm spectral step size) vs LCIs obtained by unmixing 2nm step size hypercubes. Linear trends were observed for MSE, PSNR and SSIM indices (R² = 0.91, 09.1, and 0.98 respectively). (E). Example images of SSIMs showing the differences in the quality of LCIs as we increase the step size compared to the reference component image F. Difference images between LCIs and reference image.

Download Full Size | PDF

To better illustrate what constitutes the decreased SSIM index, we show an example of sample SSIM maps (Fig. 3(E)) and difference images for various spectral steps (Fig. 3(F)). Darker areas represent a larger difference between LCIs vs reference LCI obtained from the full cube, reflecting the degradation in the lesion detection accuracy with decreased spectral resolution.

3.2 Binning of spectral bands

We then examined the effect of binning several spectral bands together. First, the 420-720nm spectrum was divided into five 60nm-wide spectral ranges, each encompassing fifteen 2nm-spaced bands (Fig. 4). We then used these 60nm spectral ranges to generate LCIs. For all four quantitative measures we obtained the highest LCI quality for the 480nm – 538nm spectral range (i.e., it yielded the lowest MSE and the highest PSNR, SSIM and Accu indices, Fig. 4(A)). Further binning of this most effective range (i.e., dividing it into smaller, 30-nm wide ranges) led to a decline in all measures as shown in Fig. 4(B), pointing to a limit as to how much the spectral range can be narrowed.

 

Fig. 4 Effect of binning multiple wavelengths on the quality of ablated component images. A. Results of Peak SNR (PSNR), Mean Standard Error and Structural Similarity Matrix index (SSIM index) for 60nm binned ranges (2nm spectral resolution) Example LCIs are shown on the left. B. Results of all measures from smaller bin sizes (gray) shown in comparison to the full range (black). Example LCIs are shown on the right.

Download Full Size | PDF

3.3 Minimization of the cubes to two, three, and four bands

Next, we looked at the possibility of a random combination of different wavelengths. To do so, we used custom Matlab algorithms based on Xu & Rice [22] to test all possible 2-, 3- and 4-band combinations. These combinations were derived from the second set of experiments that acquired 14 HSI cubes, each containing 31 spectral bands 10nm apart. The numbers of cube groupings tested were 465, 4495, and 31465 for 2-, 3-, and 4-band combinations, respectively. To speed-up the processing of such a large number of data sets, we used a fast unmixing protocol based on matrix multiplication (Details in Methods and Appendix Fig.7). Although this protocol yields slightly different numerical values, it provides similar trends and conclusions when compared to the more computationally costly unmixing approach that uses linear regression analysis-based unmixing [18]. The direct comparison of the two methods is shown in Appendix Fig.8. As shown in Fig. 5, SSIM index and PSNR values revealed a pattern of increased PSNR for the 4-band cubes compared to the 3- and 2-band cubes, although the SSIM indices remain relatively low for all cases. Moreover, the frequency of combinations with increased PSNR and SSIM values increase with an increased number of bands in the unmixed cubes as shown in Fig. 5(B). Accuracy results for all 14 cases studied on the other hand (Fig. 6), clearly show improved accuracy values for the 4-band case compared to the 3- and 2- band cases. Although some individual cases from the 3- and 2-band cubes analyzed had accuracy values higher than 80%, the mean values remained lower than 50%. One can also note an increased deviation of the accuracy values for the 2-band cubes compared to the 3- and 4- band cubes. This is even more evident in box plots of individual studies presented in Appendix Fig.9.

 

Fig. 5 Four, three and two band cubes for speedy detection of RF lesions. A. Relation between SSIM and PSNR is non-linear and bimodal. B. Histogram of SSIM indices and PSNR for all combinations of 2, 3 and 4 band cubes. Frequency of combinations with higher SSIM indices and/ or PSNR increases with increased size cubes.

Download Full Size | PDF

 

Fig. 6 Four, three and two band cubes for speedy detection of RF lesions. Accuracy results of the fast unmixing using 2-, 3-, 4- band cubes sorted in decreasing order. The mean accuracy value from 14 experimental studies is depicted by the thick grey line with black lines showing standard deviation.

Download Full Size | PDF

Highest accuracy values for 2-, 3- and 4-band combinations are presented in Table 1. The best cube combinations based on accuracy are comparable to the ones based on SNR, SSIM or MSE, pointing to an overall robustness of the methods used.

Table 1. Top accuracy values for two-, three- and four-band combinations.

View Table | View all tables in this article

4. Discussion

4.1 Accurate detection using a limited number of wavelengths

The results presented in this report show the feasibility of using a very limited number of specific wavelengths to achieve accurate differentiation between ablated and unablated atrial tissue. We also show that degradation of signal quality does not necessarily relate to the ability to classify tissue accurately. Specifically, when reducing the spectral steps, we revealed a linear trend of degradation in all three image quality measures (PSNR, MSE and SSIM index). Meanwhile the accuracy remained significantly indifferent up to 100nm resolution (4 wavelengths: 420, 520, 620 and 720) compared to component images obtained using the full visible spectra at 2nm resolution (Fig. 2).

As presented in Eq. (1), PSNR is a function of MSE. Although the information provided by both indices differ quantitatively, they remain dependent as shown in Eq. (1). On the other hand, the SSIM index and accuracy are independent of each other and all other quantitative image quality measures. With this knowledge, a comparison of the results using all four measures, allows the user to better assess signal and image quality. Examples are presented in Figs. 2 and 3 where PSNR, MSE and SSIM indices support significant quality degradation in the target image compared to the reference component image, while the accuracy results remain higher than 80%. These results suggest a minimal deviation in the ability of the algorithm to differentiate between lesion and unablated tissue when compared to lesion outlines identified from the reference image.

Our results also support the possibility of significantly reducing the range of acquisition (the latter, of course, is an application specific conclusion). In our case, the ability to differentiate between lesions and unablated tissue was not significantly changed when spectral range was narrowed five-fold (i.e., from 300 to 60nm). Specifically, component images derived from cubes in the specific range (480-538 nm) showed the highest PSNR, SSIM index and accuracy values while maintaining the lowest MSE. This range was the best across all ranges tested. It also yielded LCIs comparable to the ones obtained by unmixing hypercubes that encompass the entire visible range (Fig. 4).

This observation can be explained by our current knowledge of biological fluorophores. In cardiac tissue, the most common endogenous structural fluorophores are collagen and elastin, while the main fluorophores involved in cellular metabolism are nicotinamide adenine dinucleotide (NADH) and flavin adenine dinucleotide (FAD) [23]. When excited with UV light, collagen, elastin and NADH all exhibit broad autofluorescence within 400-550nm range. NADH has a wide peak close to 460-480nm which dramatically declines during the ablation. On the other hand, increased light scattering at the lesion site leads to a red shift in the normalized autofluorescence spectrum [6]. Combination of these factors leads to maximal spectral differences between lesions and unablated tissue to be observed within 480-540 range.

Finally, we studied whether random combinations of discreet wavelengths grouped in 2-, 3- and 4-band cubes, can also reveal ablation induced lesions (Figs. 5&6). It is interesting to note that for the best 25 cases, the four-band cube combinations shared the first wavelength. Furthermore, wavelengths in the three remaining bands, were within a 20nm window from each other (Table 1). These results support the notion of a minimal number of optimally selected wavelengths that can consistently reveal small variations between the spectra of ablated and unablated tissue. Furthermore, these results support the possibility of on-the-fly real-time analysis of cardiac tissue while it is being surgically ablated.

For the latter application it is important to consider heart contraction artifacts. Typical resting heart rates in humans are between 60 and 100 beats per minute. The best way to avoid motion which can distort cube acquisition would be to gate the acquisition to occur during diastole (estimated between 0.3– 0.4 sec). In vivo implementation of the results presented in this paper can include snapshot hyperspectral imaging devices [24–26] or use of ECG gating to acquire frames at sequential cycles.

4.2 Post-processing techniques used for wavelength selection

Adaptive wavelength selection has been reported for multiple applications in the near infrared and visible range spectroscopy. Examples of such techniques include partial least-squares, principal component regression, least-square approaches, genetic algorithms, simulated annealing, or stepwise elimination [27–29]. Optimizing these algorithms, can be computationally expensive [30–32]. In our study, we show how application of standard image qualifiers and an accuracy measure that we have developed, can be effective in yielding high accuracy LCIs by selecting band combinations using as few as 2-, 3-, or 4- bands.

4.3 Measurement stability and subject variations

Subject-to-subject variations play an important role when considering the best combination of acquired bands for specific application. Results presented in Table 1 show that the variations between subjects decreased with increased number of wavelengths. Minimum and maximum standard deviations of accuracy in the 4-band case were 0.09 and 0.31 respectively. These values increased to 0.13 and 0.39 in the case of 3-band and 0.39 and 0.41 in the 2-band case.

An additional way to reduce subject variability can be acquisition of a pre-treatment hypercube to be used as an internal calibration standard. One can also explore alternative post-acquisition processing approaches. For example, we have recently reported the use of K-means clustering as an unsupervised learning method enables one to accurately differentiate between ablated and non-ablated tissue without a priori knowledge of spectral signatures [33].

It is important to note that unlike absolute oxygen saturation measurements previously reported [17,34–36], our measurements rely on detecting the differences between the two states of the tissue under study (ablated vs. unablated), both visible in the field of view. As such, the need to perform calibration measurements would have less of an effect on the unmixing protocol since detected changes are relative differences between these two states of the tissue in the same subject.

4.4 Survey of current systems suitable for dynamic imaging

Dynamic applications of hyperspectral technology require fast acquisition and processing times. The results presented here, can benefit the two major classes of spectral imagers, i.e, scanning and snapshot based systems. In scanning systems that use acousto-optical tunable filters [37,38], line-scanning [39], point-by-point scanning [40] or liquid tunable filters [26], decreasing the number of bands needed, results in faster scans for the same spatial resolution. Faster acquisition times in such systems can also be achieved through narrowing the range of acquisition while maintaining the same spectral resolution as detailed in section 3.2 and Fig. 4.

For our particular application however, some of the most promising devices to achieve fast temporal spectral imaging include snapshot multispectral imagers. The latter acquire a limited number of wavelengths usually projected on the same chip detector. Compared to scanning based systems, snapshot imagers provide longer dwell times per pixel. As such, these imagers are better suited for applications with dim and dynamic targets such as endocardial fluorescent imaging [13].

A number of commercial products and recently reported snapshot based custom systems have shown the possibility of video rate spectral imaging with tunable spectral resolutions and varied integration times (few recent examples are listed in Table 2). Commercially available cameras such as IMEC can acquire up to 25 channels at 16 fps, while systems from Pixelteq can acquire 3-9 wide band channels simultaneously at rates up to 30 fps. Other examples include custom fitted systems for real-time spectral imaging, such as the image mapping spectrometry (IMS). The system is capable of simultaneously acquiring 40 channels at a resolution of 350x350 pixels [17]. More recently, the same group reported using Lenslet array based tunable snapshot imaging spectrometry (LATIS) for high resolution fluorescence imaging with reported resolutions of 200x200 at 27 spectral channels and integration time of less than a second [13]. Other recently reported in-vivo systems include 4D snapshot hyperspectral video-endoscope [41], fiber-based prototype imaging spectrometer for oxygen saturation measurements [14] and Lyot filters based multispectral imagers [42].

Table 2. Example of current high resolution, high speed spectral imagers

View Table | View all tables in this article

Additionally, limiting the number of acquired channels would allow the design of systems that have wider slits per channel. This is particularly important in maximizing the number of photons acquired in low light applications such as ours.

4.5 Scope of applications

Our findings can be useful for the design of optical devices aimed at live imaging of ablated cardiac tissue. They also can be applied more broadly to measurements of oxidative state [43], scar detection, edema or vessel visualization, just to name a few [8]. Spectral imaging coupled with deep learning algorithms for texture identification, fiber orientation measurements and other metabolic metrics, can provide a wealth of information about cardiac function and disease as well as guide surgeons in operating room settings.

5. Conclusion

Applications of metrics presented in this report can benefit a variety of hyper and multispectral imaging systems ranging from snapshot to scanning by providing objective means for the optimal selection of bands for maximal accuracy and least variations across subjects. We have also shown that a limited number of channels and/or simplified unmixing algorithms can be used to reveal ablated atrial tissue; this paves the way for on-the-fly in vivo RF lesion visualization. Such visualization can significantly improve the affordability and the success rate of the surgical procedures that are being used to treat atrial fibrillation.

Appendix Figures

 

Fig. 7 Fast unmixing protocol described in Methods. The algorithm was adopted from Ref. [21] (Xu et al) and optimized for speed as depicted above.

Download Full Size | PDF

 

Fig. 8 Comparing the quality assessment measures (MSE, PSN and SSIMs) between Nuance unmixing protocol [11] and fast unmixing protocol adapted from [21]. Sample 1. (B). Sample 2.

Download Full Size | PDF

 

Fig. 9 Left: Histogram distribution of Accuracies of all 14 studies used in this report. Right. Boxplot results for all 14 studies obtained using the fast unmixing protocol to reveal the distribution of Accuracies in the 2, 3 and 4 band cases. All possible combinations were tested thru the fast unmixing protocol to reveal lesion using only 2- or 3- or 4-band cases.

Download Full Size | PDF

Funding

NHLBI R42HL120511 award.

Acknowledgements

Authors thank Jeremy Otridge for editorial assistance.

Disclosures

None.

References and links

1. M. A. Calin, S. V. Parasca, D. Savastru, and D. Manea, “Hyperspectral Imaging in the Medical Field: Present and Future,” Appl. Spectrosc. Rev. 49(6), 435–447 (2014). [CrossRef]  

2. Y.-Z. Feng and D.-W. Sun, “Application of hyperspectral imaging in food safety inspection and control: a review,” Crit. Rev. Food Sci. Nutr. 52(11), 1039–1058 (2012). [CrossRef]   [PubMed]  

3. E. K. Hege, D. O’Connell, W. Johnson, S. Basty, and E. L. Dereniak, “Hyperspectral imaging for astronomy and space surviellance,” in S. S. Shen and P. E. Lewis, eds. (International Society for Optics and Photonics, 2004), Vol. 5159, p. 380.

4. N. Muselimyan, L. M. Swift, H. Asfour, T. Chahbazian, R. Mazhari, M. A. Mercader, and N. A. Sarvazyan, “Seeing the Invisible: Revealing Atrial Ablation Lesions Using Hyperspectral Imaging Approach,” PLoS One 11(12), e0167760 (2016). [CrossRef]   [PubMed]  

5. L. Swift, D. A. Gil, R. Jaimes 3rd, M. Kay, M. Mercader, and N. Sarvazyan, “Visualization of epicardial cryoablation lesions using endogenous tissue fluorescence,” Circ Arrhythm Electrophysiol 7(5), 929–937 (2014). [CrossRef]   [PubMed]  

6. D. A. Gil, L. M. Swift, H. Asfour, N. Muselimyan, M. A. Mercader, and N. A. Sarvazyan, “Autofluorescence hyperspectral imaging of radiofrequency ablation lesions in porcine cardiac tissue,” J. Biophotonics 10(8), 1008–1017 (2017). [CrossRef]   [PubMed]  

7. M. Mercader, L. Swift, S. Sood, H. Asfour, M. Kay, and N. Sarvazyan, “Use of endogenous NADH fluorescence for real-time in situ visualization of epicardial radiofrequency ablation lesions and gaps,” Am. J. Physiol. Heart Circ. Physiol. 302(10), H2131–H2138 (2012). [CrossRef]   [PubMed]  

8. L. M. Swift, H. Asfour, N. Muselimyan, C. Larson, K. Armstrong, and N. A. Sarvazyan, “Hyperspectral imaging for label-free in vivo identification of myocardial scars and sites of radiofrequency ablation lesions,” Hear. Rhythm 15, 564 (2017).

9. L. Roten, N. Derval, P. Pascale, D. Scherr, Y. Komatsu, A. Shah, K. Ramoul, A. Denis, F. Sacher, M. Hocini, M. Haïssaguerre, and P. Jaïs, “Current hot potatoes in atrial fibrillation ablation,” Curr. Cardiol. Rev. 8(4), 327–346 (2012). [CrossRef]   [PubMed]  

10. A. Schade, J. Krug, A.-G. Szöllösi, M. El Tarahony, and T. Deneke, “Pulmonary vein isolation with a novel endoscopic ablation system using laser energy,” Expert Rev. Cardiovasc. Ther. 10(8), 995–1000 (2012). [CrossRef]   [PubMed]  

11. S. R. Dukkipati, F. Cuoco, I. Kutinsky, A. Aryana, T. D. Bahnson, D. Lakkireddy, I. Woollett, Z. F. Issa, A. Natale, and V. Y. Reddy, “Pulmonary Vein Isolation Using the Visually Guided Laser Balloon,” J. Am. Coll. Cardiol. 66(12), 1350–1360 (2015). [CrossRef]   [PubMed]  

12. N. Muselimyan, M. A. Jishi, H. Asfour, L. Swift, and N. A. Sarvazyan, “Anatomical and Optical Properties of Atrial Tissue: Search for a Suitable Animal Model,” Cardiovasc. Eng. Technol. 8(4), 505–514 (2017). [CrossRef]   [PubMed]  

13. J. G. Dwight and T. S. Tkaczyk, “Lenslet array tunable snapshot imaging spectrometer (LATIS) for hyperspectral fluorescence microscopy,” Biomed. Opt. Express 8(3), 1950–1964 (2017). [CrossRef]   [PubMed]  

14. Y. Wang, M. E. Pawlowski, and T. S. Tkaczyk, “High spatial sampling light-guide snapshot spectrometer,” Opt. Eng. 56(8), 081803 (2017). [CrossRef]   [PubMed]  

15. K. Dorozynska and E. Kristensson, “Implementation of a multiplexed structured illumination method to achieve snapshot multispectral imaging,” Opt. Express 25(15), 17211–17226 (2017). [CrossRef]   [PubMed]  

16. Y. Yuan, X.-M. Ding, L.-J. Su, and W.-Y. Wang, “Modeling and analysis for the image mapping spectrometer,” Chin. Phys. B 26(4), 040701 (2017). [CrossRef]  

17. J. G. Dwight, C. Y. Weng, R. E. Coffee, M. E. Pawlowski, and T. S. Tkaczyk, “Hyperspectral Image Mapping Spectrometry for Retinal Oximetry Measurements in Four Diseased Eyes,” Int. Ophthalmol. Clin. 56(4), 25–38 (2016). [CrossRef]   [PubMed]  

18. J. R. Mansfield, K. W. Gossage, C. C. Hoyt, and R. M. Levenson, “Autofluorescence removal, multiplexing, and automated analysis methods for in-vivo fluorescence imaging,” J. Biomed. Opt. 10(4), 041207 (2005). [CrossRef]   [PubMed]  

19. H. Asfour, L. Swift, N. Sarvazyan, M. Doroslovački, M. Kay, and M. Doroslovacki, “Preprocessing of fluoresced transmembrane potential signals for cardiac optical mapping,” Conf. Proc. IEEE Eng. Med. Biol. Soc. 2011, 227–230 (2011). [PubMed]  

20. H. Asfour, L. M. Swift, N. Sarvazyan, M. Doroslovački, and M. W. Kay, “Signal decomposition of transmembrane voltage-sensitive dye fluorescence using a multiresolution wavelet analysis,” IEEE Trans. Biomed. Eng. 58(7), 2083–2093 (2011). [CrossRef]   [PubMed]  

21. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004). [CrossRef]   [PubMed]  

22. H. Xu and B. W. Rice, “In-vivo fluorescence imaging with a multivariate curve resolution spectral unmixing technique,” J. Biomed. Opt. 14(6), 064011 (2009). [CrossRef]   [PubMed]  

23. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19(1), 010901 (2014). [CrossRef]   [PubMed]  

24. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef]   [PubMed]  

25. H. C. Hendargo, Y. Zhao, T. Allenby, and G. M. Palmer, “Snap-shot multispectral imaging of vascular dynamics in a mouse window-chamber model,” Opt. Lett. 40(14), 3292–3295 (2015). [CrossRef]   [PubMed]  

26. N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013). [CrossRef]  

27. C. B. Lucasius and G. Kateman, “Understanding and using genetic algorithms Part 1. Concepts, properties and context,” Chemom. Intell. Lab. Syst. 19(1), 1–33 (1993). [CrossRef]  

28. C. B. Lucasius and G. Kateman, “Understanding and using genetic algorithms Part 2. Representation, configuration and hybridization,” Chemom. Intell. Lab. Syst. 25(2), 99–145 (1994). [CrossRef]  

29. Y.-L. Chang, “A simulated annealing feature extraction approach for hyperspectral images,” Future Gener. Comput. Syst. 27(4), 419–426 (2011). [CrossRef]  

30. J. M. Brenchley, U. Hörchner, and J. H. Kalivas, “Wavelength selection characterization for NIR spectra,” Appl. Spectrosc. 51(5), 689–699 (1997). [CrossRef]  

31. P. Bajcsy and P. Groves, “Methodology for Hyperspectral Band Selection,” Photogramm. Eng. Remote Sensing 70(7), 793–802 (2004). [CrossRef]  

32. S. DeBacker, P. Kempeneers, W. Debruyn, and P. Scheunders, “A Band Selection Technique for Spectral Classification,” IEEE Geosci. Remote Sens. Lett. 2(3), 319–323 (2005). [CrossRef]  

33. S. Guan, M. Loew, H. Asfour, N. Sarvazyan, and N. Muselimyan, “Lesion detection for cardiac ablation from auto-fluorescence hyperspectral images,” in SPIE Medical Imaging (SPIE, 2018), pp. A10578–A10579.

34. A. H. Kashani, G. R. Lopez Jaime, S. Saati, G. Martin, R. Varma, and M. S. Humayun, “Noninvasive assessment of retinal vascular oxygen content among normal and diabetic human subjects: a study using hyperspectral computed tomographic imaging spectroscopy,” Retina 34(9), 1854–1860 (2014). [CrossRef]   [PubMed]  

35. T. R. Choudhary, D. Ball, J. Fernandez Ramos, A. I. McNaught, and A. R. Harvey, “Assessment of acute mild hypoxia on retinal oxygen saturation using snapshot retinal oximetry,” Invest. Ophthalmol. Vis. Sci. 54(12), 7538–7543 (2013). [CrossRef]   [PubMed]  

36. T. Chen, P. Yuen, M. Richardson, Z. She, and G. Liu, “Wavelength and model selection for hyperspectral imaging of tissue oxygen saturation,” Imaging Sci. J. 63(5), 290–295 (2015). [CrossRef]  

37. A. B. Castrillo and L. L. Randeberg, “A Hyperspectral Imaging System using an Acousto-Optic Tunable Filter,” thesis (2015).

38. I. Kurtz, R. Dwelle, and P. Katzka, “Rapid scanning fluorescence spectroscopy using an acousto-optic tunable filter,” Rev. Sci. Instrum. 58(11), 1996–2003 (1987). [CrossRef]  

39. Y. J. Hsu, C.-C. Chen, C.-H. Huang, C.-H. Yeh, L.-Y. Liu, and S.-Y. Chen, “Line-scanning hyperspectral imaging based on structured illumination optical sectioning,” Biomed. Opt. Express 8(6), 3005–3016 (2017). [CrossRef]   [PubMed]  

40. E. L. P. Larsen, L. L. Randeberg, E. Olstad, O. A. Haugen, A. Aksnes, and L. O. Svaasand, “Hyperspectral imaging of atherosclerotic plaques in vitro,” J. Biomed. Opt. 16(2), 026011 (2011). [CrossRef]  

41. H.-T. Lim and V. M. Murukeshan, “A four-dimensional snapshot hyperspectral video-endoscope for bio-imaging applications,” Sci. Rep. 6(1), 24044 (2016). [CrossRef]   [PubMed]  

42. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010). [CrossRef]   [PubMed]  

43. T. S. Blacker, Z. F. Mann, J. E. Gale, M. Ziegler, A. J. Bain, G. Szabadkai, and M. R. Duchen, “Separating NADH and NADPH fluorescence in live cells and tissues using FLIM,” Nat. Commun. 5, 3936 (2014). [CrossRef]   [PubMed]