Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Multicolor fluorescence microscopy for surgical guidance using a chip-scale imager with a low-NA fiber optic plate and a multi-bandpass interference filter

Open Access Open Access

Abstract

In curative-intent cancer surgery, intraoperative fluorescence imaging of both diseased and healthy tissue can help to ensure the successful removal of all gross and microscopic diseases with minimal damage to neighboring critical structures, such as nerves. Current fluorescence-guided surgery (FGS) systems, however, rely on bulky and rigid optics that incur performance-limiting trade-offs between sensitivity and maneuverability. Moreover, many FGS systems are incapable of multiplexed imaging. As a result, clinical FGS is currently limited to millimeter-scale detection of a single fluorescent target. Here, we present a scalable, lens-less fluorescence imaging chip, VISION, capable of sensitive and multiplexed detection within a compact form factor. Central to VISION is a novel optical frontend design combining a low-numerical-aperture fiber optic plate (LNA-FOP) and a multi-bandpass interference filter, which is affixed to a custom CMOS image sensor. The LNA-FOP acts as a planar collimator to improve resolution and compensate for the angle-sensitivity of the interference filter, enabling high-resolution and multiplexed fluorescence imaging without lenses. We show VISION is capable of detecting tumor foci of less than 100 cells at near video framerates and, as proof of principle, can simultaneously visualize both tumors and nerves in ex vivo prostate tissue.

© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

A hallmark of a successful curative-intent cancer surgery is the complete removal of all disease, both gross and microscopic, with minimal damage to neighboring healthy tissue. Surgical guidance, however, is largely accomplished through visual and tactile feedback, which lacks adequate contrast and sensitivity, often leading to residual disease in the patient [1]. Thus, positive surgical margins (PSMs)—where cancer cells are detected along the margin of the excised tissue post-operation, suggesting an incomplete resection—occur at significant rates across all common cancers and are particularly prevalent in breast, prostate, and head and neck cancers [2]. In prostate cancer, the focus of this work, PSM rates exceed 20%, the highest PSM rate for men among the ten most common solid organ cancers in the US [2]. Studies show that PSMs are highly correlated to recurrence and mortality and require adjuvant treatment, incurring added costs, toxicity, and burden to the patient [2,3].

Across cancer types, surgeons face a common challenge—without intraoperative imaging to assist with identifying microscopic disease, they are confronted with the difficult decision of empirically removing wide tissue margins that may include undetected disease but risk damage to unseen critical structures. For example, in prostate cancer treatment, lack of visualization results in iatrogenic damage to tumor-adjacent nerves, which remains a persistent problem in radical prostatectomies despite advances in surgical technique [4]. Even in minimally invasive, robot-assisted radical prostatectomies, the prevalence of erectile dysfunction, caused by damage to the cavernous nerves, is posited to range from 10-46% at 1-year post operation [5]. Another challenge lies in the accurate assessment of nodal involvement, which is strongly correlated to aggressive disease and, if detected and removed, can both treat disease and guide adjuvant therapy [6]. Overall, these challenges underscore the need for surgical guidance systems that offer both the molecular-level contrast, sensitivity, and resolution necessary for real-time detection of microscopic disease in addition to multiplexed sensing of diseased and healthy tissue for effective surgical planning.

Among intraoperative imaging modalities, fluorescence imaging is uniquely suited to these needs [7,8]. Targeted fluorescent probes offer molecular specificity and contrast that is straightforward to extend to multiple cellular targets using different color fluorophores. As a result, a variety of contrast agents and fluorescence-guided surgery (FGS) systems are in clinical development [911] and fluorescence probes exist for the detection of many common cancer types [9] and, more recently, for critical structures such as nerves [12].

While several commercially available FGS systems allow for the clinical application of such agents, these systems suffer from performance-limiting trade-offs between sensitivity and physical maneuverability, as they rely on bulky and rigid optical components. The resolution and collection efficiency (sensitivity) of an optical system are dependent on both the aperture size and proximity to the sample. Highly sensitive microscopic detection, therefore, requires large optics placed near the tissue. Thus, conventional high-resolution systems, such as surgical microscopes [13] (Fig. 1(a)), employ bulky optics that obstruct the surgical workflow, limit visibility to line-of-sight imaging, and are incompatible with laparoscopic techniques that are standard of care for radical prostatectomies [14]. While fiber optic endoscopes [15,16] offer microscopic detection within a smaller form factor, they are fundamentally limited by tradeoffs between flexibility and field of view (FoV). This can lead to the use of flexible fibers with sub-millimeter FoVs, requiring long scan times to cover the full resection cavity. Finally, laparoscopes (Fig. 1(b)), seek to enhance maneuverability and visibility by using miniaturized optics that cover a wide FoV [17]. However, miniaturized optics have small apertures that limit sensitivity and resolution. Moreover, to cover and maintain focus across a large and depth-varying surgical FoV, a long working distance and a large depth-of-field—both inversely related to resolution and collection efficiency—are required, resulting in further performance limitations. An extended quantitative discussion of laparoscopes is presented in Supplement 1 section 1.1. Thus, while many practical FGS systems are effective at detecting millimeter-scale tumor foci, they are incapable of identifying microscopic disease on the order of hundreds of microns or smaller [18]. Compounded with the absence of multiplexed fluorescence capabilities on many systems [19], this lack of sensitivity prevents conventional FGS systems from taking full advantage of the molecular specificity and multicolor contrast offered by fluorescent probes.

 figure: Fig. 1.

Fig. 1. Comparison of conventional FGS systems with VISION. (a) Surgical microscopes have limited maneuverability and visibility and are incompatible with minimally invasive surgical techniques. (b) Laparoscopes use miniaturized optics that cover a wide depth-varying FoV, resulting in reduced sensitivity. (c) Conceptual diagram of VISION. VISION eliminates bulky optical lenses through contact imaging to allow for both a compact form factor and high collection efficiencies necessary for microscopic detection. Moreover, the optical frontend presented in this work enables multicolor fluorescence imaging for simultaneous visualization of both tumor and nerves.

Download Full Size | PDF

To address these needs in next-generation FGS, we introduce a lens-less and chip-based fluorescence imaging platform, VISION (Versatile Imaging Sensor for Intra-Operative Navigation) (Fig. 1(c)), capable of sensitive and multiplexed intraoperative imaging within a compact form factor. In contrast to conventional FGS systems, VISION takes advantage of contact imaging to eliminate bulky optical lenses and their associated tradeoffs. In contact imaging, the image sensor is placed in direct contact with the tissue to capture fluorescence emissions before they diverge [20]. This approach allows for adequate resolution without lenses and high collection efficiencies necessary for microscopic detection. Furthermore, CMOS technology enables full sensor integration within a single chip and an inherently array-like structure which can be scaled to accommodate FoVs on the order of 1 cm2 without significant trade-offs in resolution [20]. Therefore, the full imaging system can have a planar form factor on the scale of single chip, converting the optical image to an electronic one at the point of imaging, and connected to external hardware through only a few flexible wires. This facilitates eventual integration on the surface of surgical tools and manipulation of the sensor within the resection cavity including to image non-line-of-sight regions.

Despite these advantages, fluorescence contact imaging comes with two unique optical design challenges: (1) integrating high-performance, customizable, multicolor fluorescence emission filters on chip and (2) achieving microscopic resolution without lenses. In this work, we improve on our prior work in chip-based FGS [2123], introducing a novel optical frontend design that addresses both challenges. Our design consists of a multi-bandpass thin-film interference filter directly coated on a low-numerical-aperture fiber optic plate (LNA-FOP). The LNA-FOP is strongly absorptive of off-axis light and acts as a planar collimator, both to compensate for sensitivity the interference filter and to substantially improve imager resolution by eliminating divergent emissions. This innovation enables the use of high-performance interference filters without conventional focusing optics or lower-performance absorption filters, opening the door to multiplexed fluorescence microscopy on-chip with VISION. Fabricating this optical frontend on our custom-designed CMOS image sensor, we show that VISION achieves a resolution of 110 µm and can detect individual cell clusters of less than 100 cells at near video frame rates. Moreover, we image (ex vivo) resected patient prostate tissue illustrating three important clinical scenarios in FGS: (1) detection of microscopic disease at the resection margin, (2) simultaneous identification of tumor and nerves, and (3) assessment of spread beyond the primary site.

2. Challenges in lens-less fluorescence image sensor design

Chip-based contact imagers require thin, planar alternatives to conventional fluorescence emission filters and focusing optics. Due to the small absorption cross-section and optical properties of conventional fluorophores, fluorescence emissions are often 4 to 6 orders of magnitude weaker than the excitation light in intensity and red-shifted only 10-100 nm in wavelength from their absorption peak [24]. As a result, high-performance optical filters are needed for the detection of weak fluorescence signals from an intense excitation background.

2.1 Challenges with interference filters for lens-less fluorescence imaging

For conventional microscopes, thin-film interference filters are the gold standard for fluorescence filtering. They can be engineered to have high out-of-band rejection, sharp cut-off transitions, and near-total passband transmittance for any visible or near-IR spectral band as well as multiple non-overlapping passbands for multiplexed imaging [24]. Interference filters, however, are highly sensitive to angle of incidence (AOI) and only maintain performance specifications for normal incident light. At off-axis AOIs, the filter passband shifts to shorter wavelengths, rapidly increasing excitation bleed-through. This property complicates on-chip integration as without collimating lenses it is difficult to ensure that excitation light remains perpendicular to the filter [25]. This is especially true for in vivo contact imaging applications where the excitation light must be introduced at an angle between the tissue and the sensor.

2.2 Limitations of prior on-chip fluorescence filter designs

As a result, previously published on-chip fluorescence filter designs rely primarily on absorption filters, which are inherently angle-insensitive but have limited design versatility and inferior performance. Absorption filters can be fabricated using a colored dye [2629] or semiconducting material [30,31] that selectively absorbs excitation light while passing emitted fluorescence. As the absorption spectra is material dependent, different materials and/or fabrication processes must be used to image a new fluorescence wavelength, limiting design flexibility. Moreover, absorption filters have gradual cut-off transitions and significant passband losses that reduce fluorescence collection efficiencies. Finally, dye-based absorption filters often provide inadequate out-of-band rejection due to autofluorescence while semiconductor-based absorption filters are innately long-pass, reducing contrast by passing out-of-band tissue autofluorescence. Hybrid filters combining absorption and interference filters can improve out-of-band rejection, but still retain the poor pass-band characteristics and limited versatility of absorption filters [32,33]. As an alternative, approaches that avoid conventional filters altogether have also been proposed, such as using on-chip nanoplasmonic structures [34] or time-resolved imaging [23,35], but rely on unconventional inorganic fluorophores with either sufficiently long Stokes shifts or long relaxation times, respectively, to achieve adequate rejection. Inorganic fluorophores lack precedence for FDA approval and are often large (10s-1000s of nm), potentially hindering biodistribution [36].

Multicolor fluorescence imaging on chip has proven an even more difficult design challenge. Prior multicolor filter designs [3740] rely on a hybrid approach stacking both dye-based absorption filters and interference filters to achieve multiple passbands. Such designs require elaborate multi-step fabrication processes and generally suffer from the limitations in versatility and performance inherent to absorption filters.

2.3 Resolving optics for lens-less imaging

A second challenge in lens-less imager design is maintaining resolution. Without lenses, resolution is governed by the distance between the sample and the sensor over which incident light can diverge. For practical in vivo imaging, this distance cannot be arbitrarily small as a separation distance between the imager and the tissue must be maintained to introduce the excitation light. To improve resolution, planar collimating structures can deblur the image by eliminating divergent emissions at the cost of reduced collection efficiency. These structures can be integrated on-chip as angle-selective gratings (ASGs) fabricated in the metal layers [41,42] or introduced externally with a fiber optic plate [33,38]. Alternatively, coded apertures in combination with computational reconstruction algorithms can be used to improve resolution as well as to recover depth information, but require complex calibration routines, hand-optimized reconstruction algorithms, and constraints on the image sparsity and signal-to-noise ratio (SNR), limiting the robustness of these methods in low-latency in vivo applications [37,43,44].

3. Materials and methods

3.1 Optical frontend design

Our optical frontend design (Fig. 2(a)) harnesses the complementary characteristics of a high-performance multi-bandpass interference filter and an LNA-FOP to achieve both versatile multicolor fluorescence filtering and enhanced imager resolution. To compensate for the sensitivity of the interference filter to off-axis excitation, we use the LNA-FOP as a strongly absorptive angle filter. While FOPs and interference filters have been used together in prior on-chip fluorescence filter designs [33,38], these FOPs have NAs close to 1 and are used primarily as a substrate for filter fabrication and to improve imager resolution, not for rejection of off-axis excitation. Instead, in these works, dye-based absorption filters are used in combination with interference filters to provide excitation rejection across all AOIs. Absorption filters introduce less-ideal filter characteristics and limit design versatility such as the incorporation of multiple bandpass regions necessary for multiplexed fluorescence imaging without sacrificing sensitivity through pixel-level patterning of color filters. In this work, the collimating properties of the low-NA FOP eliminate the need for additional absorption filters.

 figure: Fig. 2.

Fig. 2. VISION optical frontend design. (a) Optical frontend design consisting of a low-NA fiber optic plate (LNA-FOP) and a multi-bandpass interference filter. (b) Micrograph of the LNA-FOP. (c) Prototype of VISION with the optical frontend (OF) epoxied on top of our custom CMOS image sensor chip mounted on a PCB for testing.

Download Full Size | PDF

The LNA-FOP (Shenzhen Laser, LTD) has a NA 0.15 and is composed of a matrix of optical fibers embedded in black, absorptive glass (Fig. 2(b)). It is less than 500 µm thick, contains fibers with a 12 µm outer diameter and 10 µm core, and has 40% transmittance at normal incidence. Light incident within the acceptance angle of the fibers is transmitted by total internal reflection through the fibers. Excitation light incident beyond the acceptance angle, that would otherwise pass through the interference filter, is attenuated in the absorptive sidewalls. As the LNA-FOP absorbs indiscriminate of wavelength, this technique enables the use of any interference filter with single or multiple passbands, allowing for a versatile design adaptable to any visible and NIR fluorophore and, importantly, for multicolor fluorescence imaging. In this work, specifically, we demonstrate dual-color imaging with a dual-bandpass interference filter (ETFITC/Cy5m, Chroma Technology Corp). The LNA-FOP also improves resolution by acting as a collimator.

3.2 Custom image sensor and optical frontend fabrication

We fabricate the proposed optical frontend design on our custom CMOS image sensor (Fig. 2(c)) presented in [21]. The 2.5 × 5 mm chip contains a 4.4 × 2.2 mm image sensor with an 80 × 36 active pixel array of 44 × 44µm2 pixels at a 55 µm pitch. The sensor achieves 8.2 V s-1 pW-1 sensitivity at 700 nm and uses correlated double sampling to attain shot-noise-limited performance in exposure times as short as 50 ms.

The optical frontend can be fabricated in two ways: (1) from commercially available parts (for rapid prototyping) including a 250 µm LNA-FOP and the dual-bandpass interference filter deposited on a 1mm-thick glass substrate and (2) by directly coating the interference filter on a 500µm-thick LNA-FOP to reduce the overall thickness. In (2) a 500µm-thick FOP is used to minimize the risk of fracturing during deposition, but a 250µm-thick FOP could likely be used with reduced yield. Direct deposition (2) reduces overall thickness and has the additional benefit of reducing imaging artifacts near the edge of the sensor (Supplement 1 section 1.2).

The optical component(s) are epoxied directly onto the chip. First, all surfaces are cleaned with 99% isopropyl alcohol. Next, the optical epoxy (SYLGARD 184, Dow Chemicals) is mixed and degassed for 20 minutes in a vacuum chamber. The optical component(s) are then epoxied on the chip and cured in a controlled temperature chamber (45 minutes at 65°C). To prevent optical leakage and protect the bond wires, the edges of the imager are sealed with black epoxy (EP1046FG Black, Resinlab), cured at 85°C for 90 minutes.

Due to scattering through the FOP, the order that the FOP and interference filter are placed on the chip effects filter performance under different illumination scenarios. Following the reasoning in Supplement 1 section 1.3, in this work, the filter is placed below the FOP to block scattered excitation under oblique illumination.

3.3 Preparation of cell cultures

PC3-PIP cells, which overexpress prostate-specific membrane antigen (PSMA), are cultured on monolayer plates. To form sparse clusters of closely spaced cells, the cells are trypsinized from monolayer cultures and plated on a commercially available matrix. Cells from the matrix cultures were fixed onto glass slides with care to ensure that they remain in small (10-1000 cell) clusters. Separate sets of slides are stained for imaging in each fluorescent channel with an anti-PSMA rabbit primary antibody (D4S1F, Cell Signaling Technology) and either an IRDye680LT goat anti-rabbit (LI-COR) or an AlexaFluor488 goat anti-rabbit secondary antibody (Invitrogen).

3.4 Preparation of prostate tissue samples

Banked resected patient prostate tissue samples are obtained from UCSF (IRB 15-16090). The paraffin-embedded tissue blocks are sectioned at 4 µm and mounted onto glass slides. One representative slide from each block is stained with hemoxylin and eosin (H&E) for histological analysis. The remaining slides are used for immunofluorescence staining. To label prostate cancer cells, an anti-PSMA rabbit primary antibody (D4S1F, Cell Signaling Technology) is used with an IRDye680LT goat anti-rabbit secondary antibody (LI-COR). For nerve labeling, an anti-S100 mouse primary (4C4.9, Abcam) is used with an AlexaFluor488 anti-mouse secondary (Invitrogen).

3.5 Imaging with VISION

A custom printed circuit board (PCB) is used to supply power and bias currents for the imaging chip as well as to read out and digitize each captured image frame. All timing and control signals necessary for image acquisition are generated by a field-programmable gate array (XEM6010, Opal Kelly), which interfaces between the chip and a laptop. A custom software user interface is used to set the imaging parameters and to visualize and save image data.

Two fiber-coupled lasers with wavelengths at 488 nm and 633 nm (QFLD-488-20S and QFLD-633-20S, QPhotonics) are used for fluorescence excitation. They are driven by a TEC-controlled driver (6310 Combo Source, Arroyo Instruments) and are operated with an optical intensity of 28 mW/cm2 for imaging. The lasers are collimated and transilluminate the sample at approximately 70°. While transillumination is difficult in vivo, a glass separator or light guide plate can be used in future work to couple light introduced from the side for epi-illumination between the sensor and tissue [21,45]. In this work, a 1mm-thick glass separator is fixed on top of the sensor to accurately reflect the reduction in resolution and sensitivity with the added thickness of such a device.

All imaging is performed in a light-proof box to minimize background light. For routine imaging, the sample is placed directly on the surface of the sensor and excitation light is introduced through the back of the slide. To simultaneously image the same FoV with VISION and a benchtop fluorescence microscope, we use an inverted microscope (Leica DM-IRB). The sample slide is placed with the coverslip facing away from the microscope objective lens. VISION is inverted and placed on the opposite side of the slide as the microscope objective. Excitation light is introduced at an angle between the objective and slide.

For each region of interest (ROI), 100 images are acquired with VISION at 50-75 ms exposure time to be used for image analysis or averaging. Signal-to-noise ratio (SNR) is reported as the maximum single pixel SNR within the ROI and is calculated as the mean pixel value divided by the standard deviation of the pixel intensity across 100 images. Image post-processing including thresholding and false coloring is performed in ImageJ (NIH).

4. Results and discussion

4.1 Optical frontend characterization

Figure 3(a) shows the measured angular transmittance of the 250µm-thick LNA-FOP in air at 488 nm compared with that of the on-chip angle-selective gratings (ASGs) presented in our prior work [21]. Angular transmittance measurements were performed using an optical power meter (PM100D with S120C photodiode, ThorLabs) and the same fiber-coupled lasers used for imaging. Narrow-band interference filters for each laser line (ZET488/10x and ET645/30x, Chroma Technologies Corp) eliminate any out-of-band emissions from the lasers. The sample is clamped into a motor-controlled rotating stage (HDR50, ThorLabs) to measure the transmitted optical power at 1° increments.

 figure: Fig. 3.

Fig. 3. Optical frontend characterization. (a) Measured angular transmittance of the LNA-FOP compared with that of the on-chip angle-selective gratings (ASGs) presented in our prior work [21]. (b) Transmittance spectra of the dual-bandpass interference filter. (c) Angular transmittance of the LNA-FOP at 488 nm and 633 nm plotted on a log scale. (d) Angular transmittance of the 488 nm and 633 nm excitation lasers through both the interference filter alone and the interference filter with the LNA-FOP.

Download Full Size | PDF

Compared with the ASGs, that are fabricated using reflective metal and have aspect ratios restricted by CMOS process design rules, the LNA-FOP achieves significantly sharper angle-selectivity and stronger rejection of oblique light with minimal added thickness. Accordingly, the measured full width at half maximum (FWHM) of the FOP is 12.65° in air, almost 3x narrower than that of the ASGs (36°). For a lens-less imager with collimators, the FWHM of the collimator is directly related to the resolution of the system.

Figure 3(b) shows the normal incidence transmittance spectra of the dual-bandpass interference filter (data is reproduced with permission from Chroma Technologies Corp). The filter has less than 10−6 transmittance at both 488 nm and 633 nm, the excitation wavelengths for AlexaFluor488 (AF488) and IRDye680LT (IR680LT) which are the fluorophores selected for our imaging studies. Moreover, the sharp band-edge and high passband transmittance (>95%) of the filter allow for efficient collection of fluorescence emissions from each fluorophore. However, as noted previously, excitation bleed-through increases sharply with AOI. Angular transmittance measurements of the filter at both excitation wavelengths (Fig. 3(d)) show a roughly 100x increase in transmittance at 24° for 488 nm and 36° for 633 nm. By 33° and 48°, respectively, the filter is practically transparent to the excitation, passing nearly all incident light.

We compensate for this behavior with the LNA-FOP, which is strongly absorptive of off-axis light. Figure 3(c) shows the measured angular transmittance of the FOP on a log scale. At 45°, the FOP achieves less than 10−5 transmittance of both wavelengths. The rejection increases for AOIs larger than 45°. Therefore, as shown in Fig. 3(d), when both the FOP and interference filter are used together, the resulting optical frontend maintains less than 10−5 transmittance at 488 nm and less than 10−6 transmittance at 633 nm across all AOIs.

To illustrate the potential performance improvements over absorption filters gained through the improved passband characteristics of the interference filter, we compare the VISION optical frontend with the amorphous silicon (a-Si) absorption filter presented in our prior work [31]. As discussed in Supplement 1 section 1.4, we measure a 9.3x improvement in excitation rejection and a 4.3x improvement in fluorescence emission collection efficiency with the new design.

4.2 Resolution measurements

To measure the resolution of VISION, we image a standard USAF-1951 resolution test target as shown in Fig. 4(a). For fluorescence imaging, a negative USAF target patterns excitation light onto a layer of Cy5.5 dye (Sulfo-Cyanine5.5 NHS ester, Abcam) dissolved in DMSO at 100 µg/mL. The dye is contained on the target with a 150µm-thick glass coverslip and is placed on VISION for imaging. A 1mm-thick glass separator separates the sensor from the sample. For practical in vivo imaging, the glass separator would be used to introduce excitation light to the tissue.

 figure: Fig. 4.

Fig. 4. Resolution measurements. (a) Experimental set up for resolution measurements. (b) Reference image of the fluorescent test target from a benchtop fluorescence microscope using a 2.5x objective. Green box is around group 2. (c) Image taken with VISION of group 2 on the test target, which spans VISION’s resolution limit. (d) Comparison of representative images of individual 3-bar elements from the target taken with a sensor with the ASGs presented in our prior work [21] and with VISION. (e) Plot of the measured contrast of each 3-bar element as function of line width from images taken with VISION and the sensor with ASGs.

Download Full Size | PDF

A reference image of the target taken with a benchtop fluorescence microscope (Fig. 4(b)) can be compared qualitatively to an image taken with VISION of group 2 on the target (Fig. 4(c)), which spans the resolution limit of our system. By group 2, element 6 (line widths of 70 µm), the lines become nearly indistinguishable with VISION.

To quantify the resolution improvement achieved with the LNA-FOP, we image individual 3-bar elements on the target both with VISION and an imager with ASGs presented in our prior work [21] and measure the image contrast. As the line widths approach the resolution limit of the sensor, the blur reduces the contrast between light and dark bars. Contrast is calculated as

$$({I_{max}} - {I_{min}})/({{I_{max}} + \; {I_{min}} - 2{I_{back}}} )$$
where ${I_{max}}$ is the maximum pixel value in the bright bars on the target, ${I_{min}}$ is the minimum pixel value in the dark bars, and ${I_{back}}$ is the background (the average pixel value when the excitation source is off). Figure 4(d) shows representative images from this measurement along with the line scans through each element used to calculate contrast.

The plot in Fig. 4(e) shows measured image contrast as a function of line width for both imagers. These measurements show that with the LNA-FOP, VISION can resolve line widths of 110 µm at 20% contrast, while with the ASGs, the imager is only capable of line widths of 330 µm at 20% contrast, representing a roughly 3-fold improvement in resolution. As expected, this result corresponds with the approximately 3x difference in FWHM between the LNA-FOP and the ASGs shown in Fig. 3(a). The resolution can be further improved by reducing the 1 mm separation distance between the sample and imager, but some separation must be preserved to introduce the excitation light to the tissue.

To be sure, this improvement in resolution gained with the increased angle selectivity of the FOP comes with a reduction in fluorescence signal seen by the sensor. However, with increased resolution, the background at each pixel due to tissue autofluorescence or nonspecific staining is also reduced improving the achievable signal-to-background ratio (SBR) for tumor foci smaller than the resolution of the system [18,46]. In FGS, the SBR often plays a dominant role in detection.

4.3 Sensitivity measurements

The identification and removal of microscopic tumor foci (<1mm3 or <105-106 cells) remains a critical barrier to the success of curative-intent cancer surgeries. Imaging of microscopic foci requires both efficient collection of fluorescence emissions and sufficient resolution to achieve high SNRs and signal to background ratios necessary for detection [18,46]. Towards these ends, VISION utilizes a contact imaging approach to achieve high collection efficiencies for microscopic detection.

We quantify the sensitivity of VISION by imaging slides containing differently sized clusters of fluorescently labeled PC3-PIP prostate cancer cells. The slides are imaged using excitation intensities of 28 mW/cm2 and exposure times of 50 ms and 75 ms for the AF488 and IR680LT channels, respectively. Figure 5(a).i shows a representative image of a AF488-stained sample taken with VISION compared with a reference image from benchtop fluorescence microscope (Fig. 5(a).ii). From 20x microscope images (Fig. 5(b).i-ii), the cell clusters labeled A and B are determined to contain approximately 100 and 50 cells, respectively. These clusters are imaged on VISION with maximum single pixel SNRs of 22x (13 dB) and 10x (10 dB), respectively.

 figure: Fig. 5.

Fig. 5. Sensitivity measurements. (a) (i) A representative image taken with VISION and (ii) a microscope reference of the PC3-PIP cell culture slides fluorescently stained with AF488. Clusters marked with A and B contain ∼100 and ∼50 cells and are imaged on VISION with SNRs of 22x and 10x, respectively. (b) 20x microscope images of clusters (i) A and (ii) B used for counting the number of cells in each cluster. (c) Plot of measured maximum single pixel SNR vs. cluster size for PC3-PIP cell cultures stained with AF488 and IR680LT.

Download Full Size | PDF

This procedure is repeated for a set of samples containing clusters ranging in size from 1-1000 cells. Figure 5(c) shows a plot of the maximum measured single-pixel SNR versus the cell cluster size for each fluorescence channel. VISION can detect cell clusters as small as 100 cells in a single 50 ms or 75 ms exposure with SNRs greater than 10x (10 dB) and 4x (6 dB) in the AF488 and IR680LT channels, respectively. Cells stained with AF488 are inherently brighter due to higher quantum yield of the dye and excitation closer to the absorption peak. Sensitivity can be further increased through longer exposure times, higher excitation intensities, and hardware optimizations (see Conclusion) advancing VISION closer to single-cell detection.

4.4 Ex vivo prostate tissue imaging

To demonstrate imaging in clinically relevant scenarios in prostate cancer resection with VISION, we image ex vivo banked, resected patient prostate tissue interspersed with tumor and nerves (Fig. 6). The tissue is fluorescently stained for both prostate cancer with an anti-PSMA-IR680LT (red) and nerves anti-S100-AF488 (green). While anti-S100 is an intracellular probe and, therefore, is not suitable for in vivo imaging, it is used here as a proof-of-principle for multiplexed imaging and can be replaced by existing nerve-specific intraoperative probes [12] in future experiments. To verify staining, slides are scanned with a fluorescence microscope (Zeiss Axio Observer) and compared against full-slide H&E scans by a trained pathologist (Fig. 6(a).iii-iv).

 figure: Fig. 6.

Fig. 6. Ex vivo imaging of resected prostate tissue with tumor foci and nerves. (a) (iii & iv) Full-slide microscope scans of immunofluorescence (IF) and H&E stained slides of resected prostate tissue. (i) Simultaneous detection of both tumors and nerves with VISION is possible by overlaying separate exposures of each channel. (ii & v) VISION can clearly identify major tumor and nerves in the sample with microscopic resolution. (vi) VISION detects a microscopic focus (<100 cells) along the inked margin. (b) Images of extra-prostatic extension into fibroadipose tissue. (i) IF and (ii) H&E full slide scans of sample with ROI for imaging in green. (iv) VISION detects tumor foci and nerves present in (iii) IF and (v) H&E references. (c) Images of metastatic tumor in a lymph node. (i) IF and (ii) H&E full slide scans of sample with ROI for imaging in green. (iv) VISION detects a small focus of metastatic tumor visible in (iii) IF and (v) H&E references.

Download Full Size | PDF

Relevant areas of each slide are then imaged in each fluorescence channel with VISION. Exposure times of 50 ms and 75 ms are used for the AF488 and IRDye680LT channels, respectively. To best demonstrate the resolution and image accuracy of VISION, the SNR of the images is enhanced by 10 dB by averaging 100 individual frames. While this level of averaging is used for presentation of the results and, in practice, would result in long imaging times, a single frame is necessary for detection of the labeled cells within the samples. To this point, single frame maximum at-pixel SNRs are reported in the following section for each image and are within the range of those measured with the PC3-PIP cell cultures. In addition, Fig. S4 compares single frame images with the averaged images presented in Figs. 6(a).ii and 6(a).v and illustrates the increase in SNR with averaging.

Figure 6(a).i-ii and 6(a).v-vi show a side-by-side comparison of images captured with VISION and a benchtop fluorescence microscope alongside H&E images for an histological reference. As shown in Fig. 6(a).i, dual-color imaging of both channels is achieved with our dual-bandpass filter by taking a separate exposure with each laser and then overlaying the images. In Figs. 6(a).ii and 6(a).v, VISION identifies major clusters of both tumor cells (SNR = 9.3 dB) and nerves (SNR = 10-25.5 dB), respectively. To emphasize the sensitivity of VISION, Fig. 6(a).vi shows VISION detecting a microscopic tumor focus of less than 100 cells (SNR = 7.3 dB) located along the inked margin of the sample. Figures 6(b) and 6(c) illustrate scenarios in which VISION can detect and characterize tumor spread beyond the primary tumor site. In Fig. 6(b), VISION visualizes extra-prostatic extension of tumor into fibroadipose tissue interspersed with nerves and, in Fig. 6(c), VISION detects a small lymph node metastasis (SNR = 8.8 dB). Overall, these images illustrate the potential use of VISION to intraoperatively assess resection margins for microscopic residual disease, provide critical information regarding the spatial proximity of tumor and nerves for surgical guidance, and identify potential sites of metastatic spread.

It should be noted that line-like artifacts are visible along the edges of some images (especially in Fig. 6(a).i). As discussed in Supplement 1 section 1.2, these artifacts occur when using the optical frontend design with the commercial interference filter and are caused by excitation light reflected off the bond-pads of the sensor and guided through the 1mm-thick fused silica substrate. The artifacts can be eliminated by directly depositing the filter on the FOP. As a result of these artifacts as well as the lower collection efficiency and use of trans-illumination compared with the microscope, the background signal due to the excitation bleed-through is more pronounced in the VISION images.

To be sure, after successful demonstration of ex vivo imaging, there are still challenges to address before transitioning to in vivo imaging. The cell cultures used for sensitivity measurements in this study represent an idealized scenario in which there is no background signal from surrounding tissue. Additionally, in vivo targets may be obscured by thin layer of blood or tissue, which attenuate signals through scattering and absorption. For these reasons, poorer sensitivity and reduced SNRs may be encountered in vivo, but can be compensated for through the strategies discussed in the preceding section. On the other hand, the cell cultures contain no more than 4 layers of cells and the tissue sections are less than a single layer thick (∼4 µm). In contrast, in vivo sub-millimeter tumor foci are 3D structures consisting of tens to hundreds of layers of cells, possibly leading to more concentrated signals.

5. Conclusion

Fluorescence-guided surgery technologies show potential to significantly improve clinical outcomes in curative-intent cancer surgery by helping surgeons better detect and eliminate residual and microscopic disease while avoiding damage to neighboring critical tissues. However, current FGS systems using conventional optics have restricted maneuverability, are limited to macro-scale detection, and for the most part are not designed for multiplexed imaging. Therefore, we have developed VISION, a fluorescence contact-imaging chip capable of both multiplexed and microscopic detection without sacrificing maneuverability. Specifically, VISION takes advantage of lens-less contact imaging and CMOS technology to: (1) achieve efficient collection of fluorescence emissions and resolution necessary for microscopic detection; (2) allow for a scalable design for imaging FoVs up to centimeter dimensions with little trade-off in resolution; and (3) enable a compact and planar form factor for eventual integration with laparoscopic tools and that can be manipulated to image surfaces throughout the resection cavity.

Moreover, in this work, we demonstrated a lens-less optical frontend combining an interference filter and LNA-FOP to provide both high-performance multicolor fluorescence filtering and enhanced resolution. We showed that the LNA-FOP effectively compensates for the angle-sensitivity of the filter, achieving less than 10−5 transmittance of excitation light across all AOIs. Critical to lens-less imaging applications, our design is thin, planar, and inherently scalable in size. By coating the filter directly on the FOP, the total thickness is just over 500 µm. Most importantly, compared with prior work—which primarily focused on absorption filters—our optical frontend harnesses the inherent advantages of interference filters: sharp cut-offs and near 100% passband transmittance, a versatile design adaptable for any visible or near-infrared fluorophore, and easy extension to multicolor imaging with organic fluorophores. While we demonstrated dual-color fluorescence imaging, three- and four-band interference filters are widely available such that our design can be straightforwardly adapted for complex multicolor fluorescence imaging schemes. The LNA-FOP also improves imager resolution. With the 250 µm LNA-FOP, VISION can resolve linewidths of 110 µm with 20% contrast at 1 mm separation distance from the sample, representing a 3x improvement over our prior work.

The advantages of our optical frontend design have relevance beyond intraoperative imaging. Other applications of fluorescence contact imaging include miniaturized implantable imagers for functional brain imaging in free-moving animals [35,47,48] and for in vivo monitoring of therapeutic response [49,50] as well as compact lab-on-chip devices for high-throughput molecular screening and diagnostics [5154]. In particular, the versatility of our optical frontend allows for transferrable sensor design: adapting a sensor to a new fluorescence wavelength involves epoxying on a new interference filter and changing the excitation source. We believe this design flexibility can help to bring about new applications which leverage novel optical probes and multiplexed imaging techniques.

Relevant to intraoperative imaging, we showed that VISION can detect microscopic tumor foci of less than 100 cells with SNRs near or above 10 dB at exposure times of 50 ms, close to acceptable video frame rates. These results show the potential for VISION to help surgeons identify microscopic residual disease below the mm-scale sensitivity limit of currently available intraoperative imaging platforms. Moreover, to emphasize clinical relevance for FGS, we demonstrated that VISION, enabled by the optical frontend presented in this work, can simultaneously identify microscopic tumor foci and nerves in ex vivo resected prostate tissue. We illustrated three important clinical scenarios in which VISION can improve surgical outcomes: (1) detection of a microscopic tumor foci of less than 100 cells along the surgical margin, (2) simultaneous localization of residual tumor and nerves, and (3) identification of a metastatic lymph node.

Future improvements to VISION can further this potential. Sensitivity can be increased through hardware optimizations. First, the optical frontend can be optimized with a custom interference filter with optimal passbands for the fluorophores. The FWHM of the FOP can also be increased to improve collection efficiency (albeit, at the cost of resolution) while still compensating for the angle-sensitivity of the interference filter which can maintain rejection up to 20-35°. Second, the quantum efficiency of our photodiodes (currently 13% at 530 nm and 18% at 700 nm) can be enhanced through use of a more optimized optical CMOS process. Building on our proof-of-concept platform, VISION can be adapted for practical use in the clinic. While our prototype sensor has a FoV 4.4 × 2 mm, CMOS image sensors are inherently scalable [55], enabling larger FoVs with the same resolution (pixel size) for faster raster scanning over the surgical cavity. In the future, for example, the sensor size could be increased to 1 × 1 cm, imaging a 1cm2 area in <1s while still fitting through a 12 mm laparoscopic port. In addition, CMOS technology enables the integration of power management, control, readout, and image sensor circuits all on a single mm-scale chip. Thus, a future system including the PCB and housing can be integrated at the scale of the chip and connected to external hardware only by a few flexible wires as shown in the conceptual Fig. 1(c). Finally, a new multi-bandpass filter can be integrated in the optical frontend to extend VISION to tri-color imaging with multiple NIR bands at 700 nm and 800 nm to cover a wide array of available FGS probes [9].

In conclusion, this work shows and furthers the promise of chip-based fluorescence contact imagers, such as VISION, in filling in the performance gaps left by conventional lens-based FGS systems in terms of maneuverability, sensitivity, and multiplexed imaging. With these advantages and further improvements, we envision chip-based intraoperative imagers like VISION improving clinical outcomes by extending surgical vision to hard-to-access areas and to microscopic and multiplexed imaging of both diseased and healthy tissue.

Funding

John V. Carbone Jr. Pancreas Cancer Research Memorial Fund; National Institute of Dental and Craniofacial Research (DP2DE030713).

Acknowledgments

Additional support for this work was provided by philanthropic contributions from the John V. Carbone Jr. Pancreas Cancer Research Memorial Fund. The authors thank Chroma Technologies Corp. for assistance with custom filter depositions and the sponsors of BSAC (Berkeley Sensors and Actuators Center) and TSMC for chip fabrication. Publication made possible in part by support from the Berkeley Research Impact Initiative (BRII) sponsored by the UC Berkeley Library.

Disclosures

Micah Roschelle and Mekhail Anwar applied for a patent entitled “Apparatus, Systems, and Methods for Fluorescence Imaging on Planar Sensors” related to this paper.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 for supporting content.

References

1. K. R. Tringale, J. Pang, and Q. T. Nguyen, “Image-guided surgery in cancer: A strategy to reduce incidence of positive surgical margins,” Wiley Interdiscip Rev Syst Biol Med 10, e1412 (2018). [CrossRef]  

2. R. K. Orosco, V. J. Tapia, J. A. Califano, et al., “Positive Surgical Margins in the 10 Most Common Solid Cancers,” Sci. Rep. 8, 5686 (2018). [CrossRef]  

3. J. L. Wright, B. L. Dalkin, L. D. True, et al., “Positive Surgical Margins at Radical Prostatectomy Predict Prostate Cancer Specific Mortality,” J. Urol. 183(6), 2213–2218 (2010). [CrossRef]  

4. A. Kumar, V. R. Patel, S. Panaiyadiyan, et al., “Nerve-sparing robot-assisted radical prostatectomy: Current perspectives,” Asian J. Urol. 8(1), 2–13 (2021). [CrossRef]  

5. T. F. N. Lima, J. Bitran, F. S. Frech, et al., “Prevalence of post-prostatectomy erectile dysfunction and a review of the recommended therapeutic modalities,” Int. J. Impotence Res. 33(4), 401–409 (2021). [CrossRef]  

6. L. Cheng, H. Zincke, M. L. Blute, et al., “Risk of prostate carcinoma death in patients with lymph node metastasis,” Cancer 91(1), 66–73 (2001). [CrossRef]  

7. I. S. Alam, I. Steinberg, O. Vermesh, et al., “Emerging Intraoperative Imaging Modalities to Improve Surgical Precision,” Mol. Imaging Biol. 20(5), 705–715 (2018). [CrossRef]  

8. F. J. Voskuil, J. Vonk, B. van der Vegt, et al., “Intraoperative imaging in pathology-assisted surgery,” Nat. Biomed. Eng. 6(5), 503–514 (2022). [CrossRef]  

9. C. W. Barth and S. Gibbs, “Fluorescence image-guided surgery: a perspective on contrast agent development,” Proc. SPIE 11222, 112220J (2020). [CrossRef]  

10. B. Zhu and E. M. Sevick-Muraca, “A review of performance of near-infrared fluorescence imaging devices used in clinical studies,” Br. J. Radiol. 88(1045), 20140547 (2015). [CrossRef]  

11. T. Nagaya, Y. A. Nakamura, P. L. Choyke, et al., “Fluorescence-guided surgery,” Front. Oncol. 7, 314 (2017). [CrossRef]  

12. L. G. Wang, C. W. Barth, C. H. Kitts, et al., “Near-infrared nerve-binding fluorophores for buried nerve tissue imaging,” Sci. Transl. Med. 12(542), eaay0712 (2020). [CrossRef]  

13. L. Ma and B. Fei, “Comprehensive review of surgical microscopes: technology development and medical applications,” J. Biomed. Opt. 26(01), 010901 (2021). [CrossRef]  

14. A. J. Costello, “Considering the role of radical prostatectomy in 21st century prostate cancer care,” Nat. Rev. Urol. 17(3), 177–188 (2020). [CrossRef]  

15. G. Walker, A. P. Wang, P. Z. McVeigh, et al., “The Scanning Fiber Endoscope: A Novel Surgical and High-Resolution Imaging Device for Intracranial Neurosurgery,” Operative Neurosurgery 23(4), 326–333 (2022). [CrossRef]  

16. D. Shin, M. C. Pierce, A. M. Gillenwater, et al., “A fiber-optic fluorescence microscope using a consumer-grade digital camera for in vivo cellular imaging,” PLoS One 5(6), e11218 (2010). [CrossRef]  

17. A. V Dsouza, H. Lin, E. R. Henderson, et al., “Review of fluorescence guided surgery systems: identification of key performance capabilities beyond indocyanine green imaging,” J. Biomed. Opt. 21(8), 080901 (2016). [CrossRef]  

18. B. W. Pogue, “Perspective review of what is needed for molecular-specific fluorescence-guided surgery,” J. Biomed. Opt. 23(10), 1–9 (2018). [CrossRef]  

19. F. van Beurden, D. M. van Willigen, B. Vojnovic, et al., “Multi-Wavelength Fluorescence in Image-Guided Surgery, Clinical Feasibility and Future Perspectives,” Mol. Imaging 19, 153601212096233 (2020). [CrossRef]  

20. A. Greenbaum, W. Luo, T. W. Su, et al., “Imaging without lenses: Achievements and remaining challenges of wide-field on-chip microscopy,” Nat. Methods 9(9), 889–895 (2012). [CrossRef]  

21. E. P. Papageorgiou, B. E. Boser, and M. Anwar, “Chip-Scale Angle-Selective Imager for in Vivo Microscopic Cancer Detection,” IEEE Trans. Biomed. Circuits Syst. 14(1), 91–103 (2020). [CrossRef]  

22. H. Najafiaghdam, C. C. S. Pedroso, N. A. Torquato, et al., “Fully Integrated Ultra-thin Intraoperative Micro-imager for Cancer Detection Using Upconverting Nanoparticles,” Mol. Imaging Biol. 25(1), 168–179 (2023). [CrossRef]  

23. H. Najafiaghdam, E. Papageorgiou, N. A. Torquato, et al., “A 25 micron-thin microscope for imaging upconverting nanoparticles with NIR-I and NIR-II illumination,” Theranostics 9(26), 8239–8252 (2019). [CrossRef]  

24. J. Reichman, Handbook of Optical Filters for Fluorescence Microscopy (2000).

25. M. Dandin, P. Abshire, and E. Smela, “Optical filtering technologies for integrated fluorescence sensors,” Lab Chip 7(8), 955–977 (2007). [CrossRef]  

26. E. Carregal-Romero, C. Fernández-Sánchez, A. Eguizabal, et al., Development and Integration of Xerogel Polymeric Absorbance Micro-Filters into Lab-on-Chip Systems (2004), Vol. 96.

27. E. Yıldırım, Ç. Arpali, and S. A. Arpali, “Implementation and characterization of an absorption filter for on-chip fluorescent imaging,” Sens. Actuators, B 242, 318–323 (2017). [CrossRef]  

28. O. Hofmann, X. Wang, A. Cornwell, et al., “Monolithically integrated dye-doped PDMS long-pass filters for disposable on-chip fluorescence detection,” Lab Chip 6(8), 981–987 (2006). [CrossRef]  

29. Y. Dattner and O. Yadid-Pecht, “Low light cmos contact imager with an integrated poly-acrylic emission filter for fluorescence detection,” Sensors 10(5), 5014–5027 (2010). [CrossRef]  

30. J. Alex Chediak, Z. Luo, J. Seo, et al., “Heterogeneous integration of CdS filters with GaN LEDs for fluorescence detection microsystems,” Sens. Actuators, A 111(1), 1–7 (2004). [CrossRef]  

31. E. P. Papageorgiou, H. Zhang, B. E. Boser, et al., “Angle-insensitive amorphous silicon optical filter for fluorescence contact imaging,” Opt. Lett. 43(3), 354 (2018). [CrossRef]  

32. C. Richard, A. Renaudin, V. Aimez, et al., “An integrated hybrid interference and absorption filter for fluorescence detection in lab-on-a-chip devices,” Lab Chip 9(10), 1371–1376 (2009). [CrossRef]  

33. K. Sasagawa, A. Kimura, M. Haruta, et al., “Highly sensitive lens-free fluorescence imaging device enabled by a complementary combination of interference and absorption filters,” Biomed. Opt. Express 9(9), 4329 (2018). [CrossRef]  

34. L. Hong, H. Li, H. Yang, et al., “Fully Integrated Fluorescence Biosensors On-Chip Employing Multi-Functional Nanoplasmonic Optical Structures in CMOS,” IEEE J. Solid-State Circuits 52(9), 2388–2406 (2017). [CrossRef]  

35. J. Choi, A. J. Taal, W. L. Meng, et al., “Fully Integrated Time-Gated 3D Fluorescence Imager for Deep Neural Imaging,” IEEE Trans Biomed Circuits Syst 14(4), 636–645 (2020). [CrossRef]  

36. S. Bouccara, G. Sitbon, A. Fragola, et al., “Enhancing fluorescence in vivo imaging using inorganic nanoprobes,” Curr. Opin. Bioetechnol. 34, 65–72 (2015). [CrossRef]  

37. A. J. Taal, C. Lee, J. Choi, et al., “Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters,” Light: Sci. Appl. 11(1), 24 (2022). [CrossRef]  

38. N. Kulmala, K. Sasagawa, T. Treepetchkul, et al., “Lensless dual-color fluorescence imaging device using hybrid filter,” Jpn J. Appl. Phys. 61(7), 079401 (2022). [CrossRef]  

39. W. S. Hee, K. Sasagawa, A. Kameyama, et al., “Lens-free dual-color fluorescent CMOS image sensor for förster resonance energy transfer imaging,” Sens. Mater. 31(8), 2579–2594 (2019). [CrossRef]  

40. T. Treepetchkul, N. Kulmala, K. Sasagawa, et al., “Dual-Color Lensless Fluorescence Imaging by Using a Notch Interference Filter and Absorption Filters,” OSA Technical Digest, paper DM3A.6 (2021).

41. E. P. Papageorgiou, B. E. Boser, and M. Anwar, “An angle-selective CMOS imager with on-chip micro-collimators for blur reduction in near-field cell imaging,” in Proceedings of the IEEE International Conference on Micro Electro Mechanical Systems (MEMS) (Institute of Electrical and Electronics Engineers Inc., 2016), Vol. 2016-February, pp. 337–340.

42. K. Sugie, K. Sasagawa, M. C. Guinto, et al., “Implantable CMOS image sensor with incident-angle-selective pixels,” Electron. Lett. 55(13), 729–731 (2019). [CrossRef]  

43. J. K. Adams, D. Yan, J. Wu, et al., “In vivo lensless microscopy via a phase mask generating diffraction patterns with high-contrast contours,” Nat. Biomed Eng. 6(5), 617–628 (2022). [CrossRef]  

44. S. Moazeni, E. H. Pollmann, V. Boominathan, et al., “A Mechanically Flexible Implantable Neural Interface for Computational Imaging and Optogenetic Stimulation over 5.4 × 5.4 mm 2FoV,” in Digest of Technical Papers - IEEE International Solid-State Circuits Conference (Institute of Electrical and Electronics Engineers Inc., 2021), Vol. 64, pp. 288–290.

45. H. Shin, G.-W. Yoon, W. Choi, et al., “Miniaturized multicolor fluorescence imaging system integrated with a PDMS light-guide plate for biomedical investigation,” npj Flexible Electron. 7(1), 7 (2023). [CrossRef]  

46. A. Gharia, E. P. Papageorgiou, S. Giverts, et al., “Signal to Noise Ratio as a Cross-Platform Metric for Intraoperative Fluorescence Imaging,” Mol. Imaging 19, 1536012120913693 (2020). [CrossRef]  

47. R. Rebusi, J. P. Olorocisimo, J. Briones, et al., “Simultaneous CMOS-Based Imaging of Calcium Signaling of the Central Amygdala and the Dorsal Raphe Nucleus During Nociception in Freely Moving Mice,” Front. Neurosci. 15, 667708 (2021). [CrossRef]  

48. Y. Sunaga, Y. Ohta, Y. M. Akay, et al., “Monitoring neural activities in the VTA in response to nicotine intake using a novel implantable microimaging device,” IEEE Access 8, 68013–68020 (2020). [CrossRef]  

49. R. Rabbani, H. Najafiaghdam, B. Zhao, et al., “A 36×40 Wireless Fluorescence Image Sensor for Real-Time Microscopy in Cancer Therapy,” in Proceedings of the Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineers Inc., 2022), Vol. 2022-April.

50. R. Rabbani, H. Najafiaghdam, M. M. Ghanbari, et al., “Towards an Implantable Fluorescence Image Sensor for Real-Time Monitoring of Immune Response in Cancer Therapy,” in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (IEEE, 2021), pp. 7399–7403.

51. R. R. Singh, D. Ho, A. Nilchi, et al., “A CMOS/Thin-Film Fluorescence Contact Imaging Microsystem for DNA Analysis,” IEEE Trans. Circuits Syst. I 57(5), 1029–1038 (2010). [CrossRef]  

52. J. Balsam, M. Ossandon, Y. Kostov, et al., “Lensless CCD-based fluorometer using a micromachined optical Söller collimator,” Lab Chip 11(5), 941–949 (2011). [CrossRef]  

53. H. Takehara, M. Nagasaki, K. Sasagawa, et al., “Micro-light-pipe array with an excitation attenuation filter for lensless digital enzyme-linked immunosorbent assay,” in Jpn. J. Appl. Phys. (Japan Society of Applied Physics, 2016), Vol. 55.

54. J. Ohta, H. Takehara, K. Sasagawa, et al., “On-chip fluorescence detection system with high-density microchamber array based on CMOS image sensor,” in Proceedings - IEEE International Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers Inc., 2016), Vol. 2016-July, pp. 2867–2870.

55. R. Turchetta, N. Guerrini, and I. Sedgwick, “Large area CMOS image sensors,” J. Instrum. 6(01), C01099 (2011). [CrossRef]  

Supplementary Material (1)

NameDescription
Supplement 1       Supplemental document 1

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1. Comparison of conventional FGS systems with VISION. (a) Surgical microscopes have limited maneuverability and visibility and are incompatible with minimally invasive surgical techniques. (b) Laparoscopes use miniaturized optics that cover a wide depth-varying FoV, resulting in reduced sensitivity. (c) Conceptual diagram of VISION. VISION eliminates bulky optical lenses through contact imaging to allow for both a compact form factor and high collection efficiencies necessary for microscopic detection. Moreover, the optical frontend presented in this work enables multicolor fluorescence imaging for simultaneous visualization of both tumor and nerves.
Fig. 2.
Fig. 2. VISION optical frontend design. (a) Optical frontend design consisting of a low-NA fiber optic plate (LNA-FOP) and a multi-bandpass interference filter. (b) Micrograph of the LNA-FOP. (c) Prototype of VISION with the optical frontend (OF) epoxied on top of our custom CMOS image sensor chip mounted on a PCB for testing.
Fig. 3.
Fig. 3. Optical frontend characterization. (a) Measured angular transmittance of the LNA-FOP compared with that of the on-chip angle-selective gratings (ASGs) presented in our prior work [21]. (b) Transmittance spectra of the dual-bandpass interference filter. (c) Angular transmittance of the LNA-FOP at 488 nm and 633 nm plotted on a log scale. (d) Angular transmittance of the 488 nm and 633 nm excitation lasers through both the interference filter alone and the interference filter with the LNA-FOP.
Fig. 4.
Fig. 4. Resolution measurements. (a) Experimental set up for resolution measurements. (b) Reference image of the fluorescent test target from a benchtop fluorescence microscope using a 2.5x objective. Green box is around group 2. (c) Image taken with VISION of group 2 on the test target, which spans VISION’s resolution limit. (d) Comparison of representative images of individual 3-bar elements from the target taken with a sensor with the ASGs presented in our prior work [21] and with VISION. (e) Plot of the measured contrast of each 3-bar element as function of line width from images taken with VISION and the sensor with ASGs.
Fig. 5.
Fig. 5. Sensitivity measurements. (a) (i) A representative image taken with VISION and (ii) a microscope reference of the PC3-PIP cell culture slides fluorescently stained with AF488. Clusters marked with A and B contain ∼100 and ∼50 cells and are imaged on VISION with SNRs of 22x and 10x, respectively. (b) 20x microscope images of clusters (i) A and (ii) B used for counting the number of cells in each cluster. (c) Plot of measured maximum single pixel SNR vs. cluster size for PC3-PIP cell cultures stained with AF488 and IR680LT.
Fig. 6.
Fig. 6. Ex vivo imaging of resected prostate tissue with tumor foci and nerves. (a) (iii & iv) Full-slide microscope scans of immunofluorescence (IF) and H&E stained slides of resected prostate tissue. (i) Simultaneous detection of both tumors and nerves with VISION is possible by overlaying separate exposures of each channel. (ii & v) VISION can clearly identify major tumor and nerves in the sample with microscopic resolution. (vi) VISION detects a microscopic focus (<100 cells) along the inked margin. (b) Images of extra-prostatic extension into fibroadipose tissue. (i) IF and (ii) H&E full slide scans of sample with ROI for imaging in green. (iv) VISION detects tumor foci and nerves present in (iii) IF and (v) H&E references. (c) Images of metastatic tumor in a lymph node. (i) IF and (ii) H&E full slide scans of sample with ROI for imaging in green. (iv) VISION detects a small focus of metastatic tumor visible in (iii) IF and (v) H&E references.

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

( I m a x I m i n ) / ( I m a x + I m i n 2 I b a c k )
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.