1. Introduction

In the physical sciences, addressing limitations imposed by instrument response functions often presents opportunities for new observations and insights by advancing experimental methods. As an example, the frontier of spatial resolution in optical imaging systems had continually improved to the point of wavelength-limited imaging and pushed into the regime of sub-wavelength resolution through advanced image processing. The capability of sub-wavelength-resolved imaging has made possible revolutionary insights into the functioning of biological systems [1]. As we explore in this work, the field of high-energy x-ray imaging also stands to benefit from signal processing strategies that push resolution beyond the limit of detector pixelation, especially by adapting concepts successfully employed by the biological imaging community. We focus on the specific case of Bragg coherent x-ray diffraction imaging, for which finite detector pixelation presents an especially pressing challenge as the method is applied at ever-shorter wavelengths at diffraction-limited synchrotron storage rings that deliver highly coherent high-energy x-ray beams [2–4].

Bragg coherent diffraction imaging (BCDI) has tremendous potential for providing insight into the nanoscale structure and dynamics of crystalline materials [5], especially when implemented at highly penetrating x-ray energies amenable to accessing relevant materials environments. The BCDI method provides three-dimensional (3D) images of the distribution of lattice displacement fields within nanoscale crystals via numerical inversion of reciprocal-space coherent x-ray diffraction patterns into real-space images. This approach has been applied to the study of structural changes in nanocrystals in-situ under a variety of conditions and has yielded valuable insights into catalytic processes [6], dislocation dynamics [7], crystal phase transformations [8], and many other phenomena. To date, such studies have been conducted at intermediate hard x-ray energies ($\sim$10 keV), an energy range in which third generation synchrotron sources have high coherent flux. The advent of diffraction-limited synchrotron sources presents the possibility of extending BCDI to highly penetrating x-ray energies (>50 keV) that enable the study of materials in deeply embedded environments [9]. However, satisfying data sampling constraints of BCDI and related coherent diffraction measurements at these energies is not trivial, requiring the adoption of new approaches. In this work, we present a method that addresses this challenge by invoking sparse sub-pixel detection of individual x-ray photons.

In x-ray coherent diffraction imaging (CDI) experiments, including BCDI, ptychography, and other related methods, data are collected as two-dimensional far-field diffraction intensity patterns (sometimes referred to as speckle patterns) measured with a pixelated detector. In order to produce a real-space image of the diffracting sample from these data, iterative phase retrieval algorithms are utilized. A widespread requirement of these phase retrieval algorithms is that the measured intensity patterns be oversampled by at least a factor of two above the Nyquist sampling criterion. This criterion, as it pertains to a coherent diffraction pattern measured with a pixelated area detector is given by:

At highly-penetrating x-ray energies (>50 keV), designing a CDI experiment to interrogate sub-micron-scale scattering volumes ($D\sim 500$ nm) requires that the sampling criterion be satisfied via a reduction in $p_{det}$, an increase in $Z_{sd}$, a combination of these, or the development of some altogether different mitigation strategy. Recently, both the small-$p_{det}$ and large-$Z_{sd}$ approaches have been successfully employed to enable direct-beam ptychography at 33.6 keV [10,11] ($p_{det}=9.6$ $\mu$m) and BCDI of an individual grain within a polycrystalline film at 52 keV ($Z_{sd}=6.12$ m) [9] . Alternative approaches by which to relax the sampling requirement have also been explored. In ptychography, reconstructions were enabled from undersampled intensity patterns by compensating with a high degree of real-space oversampling [12–14]. Methods utilizing multiple detector exposures displaced by sub-pixel distances were developed that enabled the recovery of fine fringe and speckle detail at an effective pixel pitch finer than of the detector used for the measurement [15,16]. In the specific context of BCDI, adaptations to well-established phase retrieval algorithms were developed to explicitly account for data that are modestly undersampled (by a factor as high as two) in the detector plane [17]. BCDI performed via trained neural networks, as compared to iterative phase retrieval algorithms, have also demonstrated an ability to perform image reconstructions on undersampled data [18]. Generally, the research that address the challenges of high-energy/undersampled CDI has been relatively exploratory to date, and a consensus has not been reached as to how best to enable such experiments. Thus, continued exploration of methods towards this end is important to pursue.

In this work, we demonstrate a new approach to enabling high-energy BCDI that leverages the ability to determine the location of photons impinging on a detector at sub-pixel-scales in low-photon-density diffraction patterns, effectively providing a means by which to lower $p_{det}$. The approach is to collect many sequential “nearly-dark” exposures of a static Bragg coherent diffraction pattern from a nanocrystal with a fast-frame-rate charge-integrating detector and to analyze the point spread functions (PSFs) from individual detected x-ray photons in order to accumulate the fractional-pixel positions of thousands of photons amassed from thousands of exposures. This is possible under conditions when the x-ray PSF extends over multiple pixels and the exposure time is short enough that only a handful of non-overlapping photons are measured per frame. The resulting list of detector position coordinates of individual photon hits can then be used to assemble a single image with a pixel mesh that is finer than that of the detector used for data collection, thus enabling super-resolution of the diffraction pattern and emulation of $p_{det}$ smaller than what can be realized by physically pixelated detectors. In this manner, this approach can produce data suitable for standard BCDI phase retrieval algorithms that require oversampled diffraction data as well as other x-ray methods that in principle require very fine pixelation.

This concept of super-resolution through PSF analysis has enabled important advances in different scientific fields and has resulted in the development of accurate and robust analysis methods. In the RADAR signal-processing community, noise-limited localization of sparse events was pioneered in the 1950s [19] and led to dramatic improvements. In optical imaging of biological samples, this concept forms the foundation of photo-activated localization microscopy (PALM) [20,21] and stochastic optical reconstruction microscopy (STORM) [22], which provided Nobel-Prize-winning capabilities for sub-diffraction-limit optical fluorescence microscopy. The popularity and importance of this fluorescence imaging method has led to significant emphasis on development of efficient photon PSF fitting routines that approach noise-limited accuracy and that enable real-time analysis. Typically, these algorithms are built on rigorous statistical estimation of the locations and energies of individual photons in a detector image based on fitting a Gaussian-distributed PSF, and they promise new opportunities for high energy x-rays, as we explore here.

In this work, we adapt and apply a photon PSF fitting algorithm from optical PALM/STORM biological fluorescence imaging to a rapid-frame-rate synchrotron measurement of a static x-ray coherent Bragg diffraction pattern from a gold sample measured with a specialized CCD at an x-ray energy of 52 keV to achieve sub-pixel detector resolution. We describe our experiment; we describe the principles of XR-UNLOC—an efficient fitting routine adapted to hard x-rays from optical PALM/STORM; we demonstrate super-resolution with XR-UNLOC that allowed fringes to be resolved in a high-energy BCDI diffraction pattern and comment on its limits; we briefly compare our results to those achieved with a photon-localization method utilized in the x-ray community; and we discuss future opportunities in areas of x-ray science beyond BCDI.

2. Experimental

The experiment was performed at the Sector 1-ID-E experimental station at the Advanced Photon Source and entailed measuring a 111 Bragg reflection from an individual grain within a polycrystalline Au film with a sufficiently coherent high-energy x-ray beam. A schematic representation of the experiment is shown in Fig. 1. An upstream x-ray optical configuration using high-resolution monochromatization (bandwidth $\Delta E/E = 1\times 10^{-4}$) [23] and focusing saw-tooth refractive lens [24] delivered a 52-keV-energy vertically-focused x-ray beam with a cross-sectional size of $100\times 1.5$ $\mu$m at the sample. This configuration was found previously [9] to produce transverse horizontal and vertical coherence lengths of $\ge 350$ nm and a longitudinal coherence length of 75 nm, sufficient to produce discernible fringe patterns about a 111 Bragg reflection from $\sim 500$-nm-scale crystallites. The line-focused beam was used to illuminate a sample consisting of a 5-mm-diameter amorphous carbon disk (1 mm thickness) that was coated with a $\sim 500$-nm thick polycrystalline gold layer with grain size comparable to the film thickness and grain orientation that was widely distributed. The orientation of the sample was such that it was nearly normal to the incident beam, with the Au-coated surface facing upstream. The amorphous carbon substrate was chosen as a rigid support for the Au film that would scatter weakly and transmit diffraction from the gold grains in the film to the detector. The sample used in this study was chosen as a means of testing our methodology with high-energy x-rays. While x-ray energies much lower than 52 keV could easily have been used to measure coherent diffraction patterns from this particular gold-film test sample, in future BCDI experiments of deeply buried individual grains within, for example, millimeter-scale bulk polycrystalline materials, this will not necessarily be the case.

 

Fig. 1. (a) A schematic of the experiment is shown. A line-focused x-ray beam illuminates a set of grains in the polycrystalline gold thin film sample, one of which is oriented to diffract into the detector which is positioned off of the direct beam path by $5.8^{\circ }$ in the horizontal plane (into the page). (b-d) Regions of interest of raw 20 ms FastCCD exposures of the Bragg peak are shown that are exemplary of the data set, which consists of 100,000 such images measured sequentially. Green circles indicate the areas in each frame where an x-ray photon hit and where a localized point-spread function is visible. The vertical strips in each image are due to detector dark background, or pedestal, which can be characterized and removed. (e, f) A sum of all 100,000 frames after pedestal subtraction is shown.

Download Full Size | PDF

The detector used in this work was the fastCCD, a specialized fast-frame-rate charge-coupled device (CCD) designed for time-resolved x-ray experiments [25]. The fastCCD has an active sensor area of $960 \times 960$ pixels with pixel size $p_{det}=30$ $\mu$m and is capable of running at full-chip frame readout rates up to 100 Hz. The FastCCD was operated with a 30 V sensor bias such that individual 52 keV energy x-rays produce PSFs that extend over multiple pixels. This spreading is critical in this experiment because it is necessary for sub-resolution via photon PSF analysis.

To obtain coherent Bragg diffraction signal in the fastCCD from an individual sub-micron grain of gold, the detector was placed in a position downstream of the sample and displaced from the direct beam by $\sim 5.8^{\circ }$ in the horizontal plane, so as to satisfy the (111) Bragg condition for crystalline gold at an x-ray energy of 52 keV. With the detector position fixed, isolated diffraction signal from a single grain in the sample was found by surveying the height of the sample in the beam as well as the angle of incidence of the sample in the beam, which is a common approach in BCDI measurements. The sample-to-detector distance for this experiment was $Z_{sd}=1.1$ m. This distance was chosen because it represents a typical $Z_{sd}$ for BCDI experiments performed on general purpose diffractometers. Importantly, with $Z_{sd}=1.1$ m, a typical grain in the film with $D \sim 500$ nm would not satisfy the inequality in Eq. (1) in this experiment, given that $\lambda = 0.2384$ Å for 52 keV energy x-rays.

With all other aspects fixed, 100,000 sequential detector images of the 111 Bragg peak from one grain in the sample were collected with the fastCCD operating at an image readout rate of 50 Hz, corresponding to an exposure time of 20 ms per frame. Immediately after the data were collected, a series of 1000 sequential dark images were collected under these conditions with the x-ray shutter closed. This exposure time was selected because it was sufficiently short to ensure a very low photon density per detector image, providing nearly-dark frames with almost exclusively non-overlapping photon events. Examples of raw exposures from this data set are shown in Fig. 1(b,c,d), which feature the $40\times 40$ pixel region of interest in the detector where the Bragg peak appears. The green circles in these images indicate the location where individual detected x-ray photons hit the sensor and produced PSFs in which the charge created by the absorbed photon (in units of analog-to-digital units, ADUs) spreads over several neighboring pixels. The vertical strip structure that is visible is due to variations in detector readout noise levels (also known as dark counts, a pedestal, or detector background). This detector background is easily estimated by averaging a series of dark exposures.

A summation of the set of 100,000 frames of the Au Bragg peak is shown in Fig. 1(e, f) (after pedestal subtraction and thresholding each frame). This summed image shows the characteristics of a coherent Bragg peak intensity distribution from a nanoscale crystal. In particular, it shows intensity modulation suggestive of coherent speckles and fringes. However, these fringes are not fully resolved with the native pixel pitch of the fastCCD camera, as was to be expected for the typical grain size ($D\sim 500$ nm) in this sample. As we describe below, a sub-pixel-resolved coherent diffraction fringe pattern can be extracted from these data via localization and PSF fitting of individual high-energy photons using an adaptation of the UNLOC algorithm from optical PALM/STORM. This approach provides a path to emulate fine-pitched detector pixels down to $<2$ $\mu$m, smaller than what can currently be manufactured for x-ray detectors.

3. Super-resolved detection with XR-UNLOC

The UNsupervised particle LOCalisation (UNLOC) strategy [26] was initially developed for the processing of PALM/STORM datasets of nearly-dark frames of activated fluorescence emitters in marked biological samples collected in an optical microscope. UNLOC seeks to identify the sub-pixel-resolved location of photons in a nearly-dark image frame and does so using an approach derived from standard statistical inference.

For a given frame, two successive steps are implemented with UNLOC:

The routines within UNLOC perform these operations very efficiently, and UNLOC is used regularly by biologists to perform PALM/STORM imaging. UNLOC can also handle the case of frames with photons with overlapping PSFs, but we limit the discussion in this work to the single-photon assumption that applies to the x-ray data under consideration.

Because UNLOC was specifically developed for optical imaging experiments, it cannot directly be utilized for hard x-ray data without adaptation. In UNLOC for optical image data sets, the statistical test employed within the photon decision routines is built upon the assumption that the photon PSF in the frame spreads over an area of $\sim 5\times 5$ pixels. This is a safe assumption for optical fluorescence microscopy, however, as shown Fig. 2(a), typical hard x-ray PSFs rarely extend beyond a $3\times 3$ pixel area. As a result, a different statistical test for the decision operation must be devised to adapt UNLOC for x-ray data with typically smaller PSF sizes. Such an adaptation will enable the power, accuracy, and speed of UNLOC that has so greatly benefited the biological community to also be utilized for x-ray science.

 

Fig. 2. (a) A single as-measured 20 ms exposure of the Bragg peak from the Au grain is shown. The dashed white line indicates the location of an ADU line-out, shown in (b), that cuts through the charge cloud created by a 52 keV x-ray. (c) A background-subtracted image of the frame in (a) is shown in which the vertical strips characteristic of a dark exposure of the fastCCD are not visible. This image is input in the CA-CFAR test to produce the $H_1$ map, shown in (d). The $H_1$ map consists of clusters of pixels that were determined by the CA-CFAR test to have an ADU count beyond what is expected with noise and background fluctuations, and cluster around the location of individual detected photons. The location of the photon that created each cluster can be estimated by fitting the PSF (as done with XR-UNLOC) or by a weighted center-of-mass calculation (as done in droplet analysis). Position estimates from both methods are overlaid as dots and show subtle differences.

Download Full Size | PDF

Below, we briefly describe the adapted decision test and registration operation, which remains almost unchanged from what is used in optical PALM/STORM. The new adapted approach we refer to from hereon as XR-UNLOC. We also provide in the Supplement 1 (SM) a comprehensive description of both the decision and registration operations of UNLOC and XR-UNLOC in order to provide readers a complete basis of comparison of the methods and details needed for implementation.

3.1 Decision task via CA-CFAR test

XR-UNLOC relies on a decision task performed in each pixel based on a method known as cell-averaged constant false-alarm rate (CA-CFAR) detection, which identifies any pixel in the detector with an ADU readout that exceeds typical noise fluctuations. The CA-CFAR test was adopted for XR-UNLOC because it was developed for signal identification in a very similar context in radar signal processing, i.e., a regime where the signal wavelength is comparable to signal binning (analogous to pixels in our experiment). Thus, when small photon PSFs are expected, this approach overcomes the limitations of the generalized likelihood-ratio test currently used in UNLOC for optical fluorescence imaging (described in detail in [26]).

Here, we present the key features of the CA-CFAR test and how it is used in XR-UNLOC. The test is applied to all pixels in the camera field of view to identify pixels that contain signal above what is expected from noise- and background-driven fluctuations in the detector. A model of the expected background and noise in the detector is required, as well as a straightforward means by which to estimate certain statistical noise parameters. For this experiment, the detector readout noise fluctuations are assumed to be Gaussian-distributed. Thus, for pixel $p$ in the detector, if the observed ADU count $x_p$ originates from detector noise only, the expected fluctuations can be modeled via a random variable $\epsilon _p$ which is Gaussian-distributed with mean and variance $\mu _p$ and $v_p$ respectively. The parameters of the probability density function (PDF) of $\epsilon _p$ that models noise fluctuations will generally vary as a function of pixel coordinate because of spatial variations in the response of the detector. In the case of detectors such as the fastCCD, one prominent spatially varying feature is that the mean dark response is “striped”, as shown in Fig. 2(a). In order to account for such prominent variations in the mean dark response of the detector, $\mu _p$ is expressed as a sum of two components,

The parameters $(b_p, m_p, v_p)$ are needed to model the random variable $\epsilon _p$, and the CA-CFAR method provides a means by which they can be estimated for a given pixel. As noted above, a pixel map of $b_p$ can be well approximated by measuring the average of a series of dark exposures, as shown by comparing (a) and (c) in Fig. 2. However, $m_p$ and $v_p$ are subject to variations between and within frames and need to be estimated from the local pixel environment about pixel $p$ in the experimental detector frame under consideration. The CA-CFAR approach addresses this problem by estimating $m_p$ and $v_p$ from a second-shell ring of pixels about the pixel $p$ of interest, as shown schematically in SM Fig. 1. We denote this set of second-ring pixels as $\Omega _p$. From $\Omega _p$, an empirical estimate of the mean $\widehat {m}_p$ and variance $\widehat {v}_p$ can be calculated according to:

Using the estimates of $(b_p, m_p, v_p)$ that parameterize the pixel noise model $\epsilon _p$, a statistical hypothesis test is performed at each pixel to categorize the observed ADU signal $x_p$. This hypothesis test evaluates $x_p$ in terms of two outcomes: $H_0$ corresponds to the case that the observed counts originate from noise fluctuations (the “no photon” case), and $H_1$ corresponds to the case that the observed pixel count originates from something other than noise (in this case from an x-ray photon). The test to distinguish these hypotheses is written (using notation of statistical inference) as:

In (4), the value $T$ plays an important role because it acts as a decision threshold between the photon/no-photon cases. When the numerator and denominator of $S_p$ are statistically independent (as applies here, see [28]), specific values of $T$ can be chosen that dictate a fixed probability of false alarm (PFA) for the decision test, leading to rigorous control of photon identification error. We utilize this feature to specify a PFA of $10^{-6}$ in this work. The relevant connection between $T$ and PFA is shown in SM, Eq. (20).

In each data frame, the test 4 is performed for each pixel, and this process produces a binary map of pixels satisfying $H_1$. An example of such an “$H_1$ map” with its corresponding data frame is shown in Fig. 2(d). In this figure, we clearly see that isolated incoming photons are usually associated with a small cluster of pixels for which the $H_1$ hypothesis test was satisified. From the $H_1$ map, a final pixel map is built consisting of the pixel within each cluster with the highest value of $S_p$. This latter pixel map $\widehat {{{{\mathcal {{H} }}} }}_1$ is known as a “detection map” which serves as a starting point for sub-pixel localization.

3.2 Sub-pixel localization

The sub-pixel localization approach used in XR-UNLOC is nearly unchanged from that used in UNLOC for optical wavelengths, as described in [26]. The salient elements of the approach are described below, highlighting a few differences that are required to minimize artifacts when smaller PSFs are considered.

The detection map $\widehat {{{{\mathcal {{H} }}} }}_1$ provides the locations of pixels in a given frame in which a photon has landed which requires sub-pixel localization. For each such photon event, this process starts by defining a square $5\times 5$ pixel window ${{{\mathcal {{W} }}} }$ about the pixel containing the photon. This window size was chosen to fully encompass a PSF in the data set considered here, but can in general be chosen differently depending on the observed characteristics of the PSF. Within a window about an identified photon event, the model considered for a photon impinging on the detector is given by:

In this expression, $\epsilon _p$ is the contribution of camera noise and background (as defined earlier), and ${{{\mathcal {{G} }}} }_p$ is a 2D Gaussian model of the instrumental PSF due to a photon. The Gaussian PSF is scaled by a multiplicative factor $\alpha$, and has a characteristic width given by $\sigma$ (as defined below). The variable ${{\boldsymbol {\delta }} }:=(\delta _x,\delta _y)$ is the sub-pixel shift that accounts for the exact position of the photon inside the pixel domain where the photon-hit was identified. With most x-ray area detectors, the typical PSF footprint corresponds to a characteristic width $\sigma$ that is smaller than one pixel, and for this reason the PSF model in XR-UNLOC, described in detail in the SM, differs from the one in UNLOC in that XR-UNLOC integrates the continuous 2D Gaussian distribution over the spatial domain of the pixels in the vicinity of the photon location.

The sub-pixel localization routine in XR-UNLOC estimates the quantities $\alpha$ (the total energy deposited in the detector by the photon in units of ADU), $\sigma$ (the size of the PSF), and ${{\boldsymbol {\delta }} }$. This estimation follows the standard maximum likelihood principle [29], which in this case simplifies to the minimization of the least-square criterion (SM Eq. (4)). This error metric takes into account the contributions of the photon PSF as well as the detector background response to quantify goodness of fit with respect to the raw ADU counts observed in the window ${{{\mathcal {{W} }}} }$. As in the decision step, the quantity $b_p$ is known, taken to be the mean dark counts (or pedestal) from a series of dark frames. The residual mean background contribution $m$ is presumed to be constant over the pixels of ${{{\mathcal {{W} }}} }$, and this value is estimated from the data along with the PSF. To efficiently and robustly estimate the unknowns $\alpha , m, \sigma , {{\boldsymbol {\delta }} }$, error minimization is implemented through an iterative second-order Gauss-Newton method. A detailed derivation of the error metric and its dedicated minimization strategy are given in the SM.

4. XR-UNLOC Applied to high-energy Bragg coherent diffraction

The XR-UNLOC photon analysis was applied to the 100,000 frames from the experiment described in Sec. 2. The threshold in the CA-CFAR test 4 was set to achieve a PFA of $10^{-6}$, meaning that on average one in $10^{6}$ tests result in a pixel being falsely associated with a photon event. The processing resulted in a list of 55,637 photon events detected in the data set, with their characteristics ($\alpha$, $\sigma$, ${{\boldsymbol {\delta }} }$) tabulated for further analysis. From this list, the 52 keV single photon events that were reliably fit were identified by the process described below so that they could be used to assemble a single diffraction image with emulated detector pixels $p_{det}$ smaller than what was used in the experiment.

The scatter plot in Fig. 3(a) shows how the energy of each detected photon $\alpha$ is distributed as a function of the PSF width $\sigma$. Along the $\alpha$ axis, photon events can be separated into three categories. The first type are background x-ray fluorescence photons characterized in the plot by an $\alpha \sim 350$ ADUs, numbering about 40% of the detected events. The primary origin of these fluorescence photons was not from the sample itself but from the copper backing plate used for cooling behind the fastCCD sensor which absorbed a portion of the diffracted beam (see Figure 8). The second type are single instances of 52 keV Bragg diffracted photons from the sample. These reside primarily in the sharp band centered at $\alpha \sim 2350$ ADUs depicted in Fig. 3(a, c) with red markers and make up the majority of the events. The third type consists of pairs of 52 keV photons with significantly overlapping PSFs in the same frame. These events correspond to $\alpha > 3510$ ADUs and are quite rare in our data set ($<1$%). For the purposes of our analysis, only the single 52 keV events will be considered (1170 ADU $< \alpha <$ 3510 ADU), which provides a means by which to exclude fluorescence photons that do not participate in coherent Bragg diffraction. The bounds chosen to select single photon events represent the mean 52 keV $\alpha$ of 2350 ADU multiplied by 0.5 and 1.5 respectively. This average 2350 ADU per photon signal, when compared to the standard deviation of the detector noise ($\sim 5$ ADU) determined from a totally dark frame, corresponds to a signal-to-noise ratio (SNR) of 53 dB, which is high enough that CA-CFAR can very reliably distinguish between noise-driven detector readout fluctuations and photon events (see Fig. 1 in the SM).

An additional event filtering step was needed to exclude 52 keV photon events with fitted PSF widths that yield photon localization error too high for a given pixel upsampling rate. For the upsampling rates applied in this work ($2\times$, $4\times$, $8\times$, and $16\times$), the corresponding PSF cutoff thresholds ($\sigma _{min}$) vary, as indicated in Table 1. The threshold of $\sigma _{min} = 0.2$ pixels, as applied to the $2\times$ and $4\times$ upsampling cases, originates from the fact that with very small PSFs, minimal charge-sharing occurs between neighboring pixels making the utilization of any localization fitting routine prone to significant error. Our observation was that photon events in our data with $\sigma _{min} \le 0.2$ pixels fitted with XR-UNLOC gave unreliable localization estimates, likely due to subtle features of readout noise or photon detection with the fastCCD camera that were difficult to capture with the PSF model. The thresholds of $\sigma _{min} = 0.22$ and $0.25$ pixels in the $8\times$ and $16\times$ upsampling cases are applied based on our analysis of the tradeoff between maximum photon localization error (inversely proportional to $\sigma _{min}$) and target upsampling rate. This analysis is described in Sec. V. As a result of the differences in $\sigma _{min}$, the number of photons available to form the upsampled images also varies, but comprise at least 55% of the total 52 keV photon events.

 

Fig. 3. (a) A scatter plot of all x-ray photon events identified with XR-UNLOC is shown in terms of the fitted PSF width $\sigma$ and the estimated intensity $\alpha$. The points in red represent x-rays designated as single-52-keV-photon events. Panel (b) shows a histogram of these black points binned by PSF size. The green, red, and blue vertical lines indicate $\sigma$ values of 0.20, 0.22, and 0.25 respectively, and these values were used as threshold values $\sigma _{min}$ to determine the populations of photons used to form the images in Fig. 4, as described in the text. Panel (c) shows a histogram of the single-52-keV-photon events binned by PSF intensity $\alpha$, indicating a peak at $\alpha \sim 2350$ ADU.

Download Full Size | PDF

Table 1. The variable parameters of the upsampled diffraction patterns in Figs. 4 and 5 are summarized. Note that in the images generated with XR-UNLOC fitting, the $8\times$ and $16\times$ upsampled images were generated with a $\sigma _{min}$ photon event threshold greater than 0.2 pixels. This is in accordance with the estimates of thresholding that should be applied for higher upsampling rates as discussed in Sec. V and in Fig. 6. The percentages reported are with respect to the total number of 52 keV x-ray photon events identified with each method, which was 48,618 with XR-UNLOC and 34,394 with the droplet analysis.

View Table

As shown in Fig. 4, sub-pixel resolved diffraction images were assembled from these populations of eligible events by binning the XR-UNLOC fitted photon positions to upsampled pixel grids corresponding to pitches of $p_{det}=$ 15$\mu$m, 7.5 $\mu$m, 3.75 $\mu$m and $1.875$ $\mu$m. In all cases, the coherent diffraction fringes in the pattern become well resolved as compared to the integrated image of the Bragg peak with the native $p_{det}=30$ $\mu$m of the fastCCD (Fig. 1(f)), successfully demonstrating the principle of using XR-UNLOC to enable the needed signal oversampling for a high energy BCDI experiment.

A relevant comparison to make is to a related method of photon localization known in the hard x-ray community as “droplet analysis” [30,31]. The droplet method produces a list of clusters of border-sharing pixels that make up isolated or slightly overlapping x-ray photon PSFs. These clusters are generated from pixels above a given threshold in a background-subtracted CCD frame. In the case of single-photon events, the droplet method typically obtains sub-pixel-resolved photon localization via an ADU-weighted center-of-mass (COM) calculation of the pixelated shape of a droplet, as compared to fitting explicitly. As in XR-UNLOC, upsampled diffraction patterns can be built by mapping these COMs onto a grid of arbitrary pixel-pitch. The 52 keV diffraction patterns shown in Fig. 5 were obtained after running the droplet analysis described in [31] on the fastCCD data. Droplets were identified as clusters of adjacent pixels with intensity above a threshold of 15 ADU, and small droplets consisting of fewer than three pixels were removed from consideration. As indicated in Table 1, the number of photons identified with XR-UNLOC and droplet analyses were comparable, but more photon events were eliminated from consideration in the droplet case because the thresholding step cuts the PSF tails and leads to one- and two-pixel droplets from which 2D localization estimates cannot be determined. The droplet COMs were then mapped to grids with effective pixel pitches of $p_{det} = 15$ $\mu$m and 7.5 $\mu$m. In the case of $p_{det}=15$ $\mu$m, the diffraction fringes are well resolved and are quite comparable to the $2\times$ upsampled diffraction pattern obtained with XR-UNLOC. However, a prominent checkerboard artifact is clearly visible with the droplet COMs are binned to a $p_{det} = 7.5$ $\mu$m detector grid, indicating a clear limit in the utility of the droplet COM approach for super-resolution.

 

Fig. 4. Upsampled images of Bragg diffraction patterns generated from XR-UNLOC analysis of individual photon events in the fastCCD camera with a native pixel pitch of $p_{det}=30$ $\mu$m. Images (a), (b), (c), and (d) correspond to pixel upsampling rates of $2\times$, $4\times$, $8\times$, and $16\times$ respectively, and were produced using the $\sigma _{min}$ PSF thresholds and total number of photons as indicated in Table 1. All images are shown in a logarithmic intensity color scale. Note that emulating finer pixels reduces the number of photons per pixel.

Download Full Size | PDF

 

Fig. 5. Upsampled images of Bragg diffraction patters generated from the “droplet” approach used in the x-ray correlation spectroscopy community. Sub-pixel photon locations are estimated in this method via an ADU-weighted center of mass (COM) calculation of clusters of pixels (i.e., droplets) associated with a photon event. The droplet method extracts clusters of adjacent pixels from background-subtracted frames with intensity above a threshold of 15 ADU. Droplets with fewer than 3 pixels are discarded, which necessarily cannot provide a location estimation in two dimensions. Single-52-keV-photon droplets are identified as those with integrated ADUs in the range of $[1170, 3510]$, the same criterion as applied in XR-UNLOC. The COMs of these droplets were gridded into different pixel arrays: (a) a $15\times 15$ $\mu$m mesh, providing upsampling by a factor of 2, and (b) a $7.5\times 7.5$ $\mu$m mesh, providing upsampling by a factor of 4, where prominent artifacts are visible. Logarithmic intensity are shown in units of photon counts.

Download Full Size | PDF

In light of this result, it should be noted that upsampling using the droplet method is outside of the context for which it was originally developed – low-count-rate x-ray photon correlation spectroscopy (XPCS). In this application it is critical to accurately determine, without any upsampling, which pixels contain photon events, a task for which droplet analysis is reasonably well suited. As shown in the SM, it may be the case that the improved photon localization capabilities of XR-UNLOC compared to the droplet analysis may provide benefits in the context of XPCS as well.

We also note that sparse-photon super-resolution in the x-ray regime has been demonstrated by related strategies of analyzing charge-sharing. Studies have made use of more sophisticated droplet COM determination by modeling the PSF imprint as circular (but not Gaussian) [32–34]. These studies achieve upsampling rates up to $4-10\times$ the original pixel pitch. Utilizing a different approach, sub-pixel photon locations were estimated by inferring sub-pixel positions from one-dimensional probability distribution functions of the observed nearest-neighbor charge sharing [35], achieving an upsampling rate of $5\times$ for the method. Sub-pixel photon localization has also been demonstrated in the x-ray regime by utilizing physical apertures (i.e., an array of micrometer-sized pinholes) that occlude all photons from the detector apart from those that pass through the aperture [36,37]. Though this strategy allows photon localization, it does not lend itself to upsampling a pattern because detected nearest-neighbor photon events are necessarily separated by the pitch of the physical array.

The main advantage of XR-UNLOC as compared to these methods is that each photon event is fitted independently and with joint estimation. XR-UNLOC is thus tolerant to inevitable variability in the PSF size (originating from differences in the depth in which the x-ray was absorbed in the sensor), to spatial variations of detector background, and to changes of the background characteristics that may evolve in time during data collection. Second, XR-UNLOC uses joint estimation to explicitly fit a 2D PSF function to the inherently two-dimensional photon signatures in the detector. Overall, these aspects afford a much higher degree of flexibility and accuracy as compared to methods that presume a uniform ensemble PSF response and unchanging detector readout characteristics. Nevertheless, a broad comparison of the performance of XR-UNLOC with different integrating charge detectors, under different operating conditions and x-ray energies, and with the different photon localization algorithms mentioned above will ultimately be needed to assess the best approach for different specific experimental contexts. Regarding BCDI specifically, future work will be needed to assess important questions of possible artifacts and downstream effects that may manifest in the resulting 3D image reconstructions from 3D diffraction data sets processed with XR-UNLOC, especially in low-signal regions of the data.

5. Conditions for success with XR-UNLOC

As we showed, binning the XR-UNLOC fitted photon positions to grids as fine as $16\times$ of the original pixel pitch of the CCD produced no apparent artifacts. Though, obtaining this result required careful selection of the minimum PSF size threshold $\sigma _{min}$, which determines the maximum position estimation error of ${{\boldsymbol {\delta }} }$. Qualitatively, for a given SNR, photon events with relatively large PSFs and a high $\sigma$ value share charge between neighboring pixels to a greater degree, leading to less aggregate estimation error in ${{\boldsymbol {\delta }} }$. On the other hand, events with small PSFs result in higher estimation error of ${{\boldsymbol {\delta }} }$ and therefore limit the degree of upsampling that can be achieved without artifacts. In this section, we present, via analysis of simulations of the experiment, a means by which to estimate the appropriate $\sigma _{min}$ to choose for a target upsampling rate by quantifying the mean-squared error of ${{\boldsymbol {\delta }} }$ (MSE) under different conditions. These estimates were used in generating the results in Fig. 4, and provide a guide for the design of future experiments.

To quantify the relationship between the MSE of ${{\boldsymbol {\delta }} }$ and the PSF size and SNR, we performed a series of simulations in which single photons impinged in known sub-pixel positions with a range of $\sigma$ and SNR conditions similar to our experiment. Photon events in the detector were repeatedly simulated at sub-pixel position increments of $1/16$ of a pixel using the model in Eq 6 applied within a $5\times 5$ pixel window ${{{\mathcal {{W} }}} }$. For each sub-pixel position, many instantiations of Gaussian noise were generated so as to introduce statistical variability in the ADU counts. These simulations were fitted with XR-UNLOC to determine the MSE. Two example simulations that highlight the impact of $\sigma$ in the high SNR regime (53 dB) are shown in Fig. 6(a, b). The two scatter plots in Fig. 6(a, b) depict the fitted photon locations in the detector pixel with $\sigma = 0.3$ and $\sigma = 0.2$ pixels respectively. The two PSF cases considered are particularly relevant because $\sigma =0.3$ is the mean PSF size found in the experiment and $\sigma = 0.2$ pixel is the cutoff below which we excluded photon events from consideration.

 

Fig. 6. Simulations of photon events that land within a pixel were performed, and results from two of the conditions tested are shown in (a,b), corresponding to $\sigma = 0.3$ (a) and $\sigma = 0.2$ (b) at a SNR of 53 dB. The scattered points about $1/16$-pitch fractional positions are the results of XR-UNLOC fitting of simulated data at each of these positions. The degree of scatter becomes more pronounced in (b) as compared to (a), resulting in a higher $\sqrt {\textrm {MSE}_{\textrm {max}}}$ metric, as described in the text. The contour plot of $\sqrt {\textrm {MSE}_{\textrm {max}}}$ as a function of SNR and the PSF width $\sigma$ is shown in (c). These iso-contours provide an estimate of the maximum upsampling achievable with XR-UNLOC under given conditions. As described in the text, the red markers at 53 dB indicate the different $\sigma _{min}$ threshold applied to obtain the images in Fig. 4.

Download Full Size | PDF

Two sources of location estimation error can be observed in these plots that respectively depend on the PSF size and the location of photon events within the pixel. First, decreasing the PSF increases the scatter of the XR-UNLOC fit, as evidenced by the broadening of the scatter of fitted locations In Fig. 6(b) as compared to (a). Second, location estimation of photon events near pixel corners tend to be biased towards the pixel center, as evidenced by the apparent distortion of the original grid of photon test locations. For a given condition, the “worst case” localization error, be it from scatter or near-corner estimation bias, should be used to determine the upsampling limit.

To quantify the worst-case error for each condition, we calculated $\sqrt {\textrm {MSE}_{\textrm {max}}}$, the square root of the maximum value of MSE among all sub-pixel locations tested. Figure 6(c) shows the isocontours of this quantity as a function of SNR and $\sigma$, which can be used to directly estimate the maximum degree of upsampling that can be achieved for a given condition by matching the upsampled pixel grid to $\sqrt {\textrm {MSE}_{\textrm {max}}}$. The red circle on this plot corresponds to the SNR of 53 dB observed in the experiment and the $\sigma _{min}=0.2$ pixels threshold we applied to the $2\times$ and $4\times$ upsampled images. The location of the red circle near the $\sqrt {\textrm {MSE}_{\textrm {max}}}=7$ contour suggests that a lower $\sigma _{min}$ could have been selected for the $2\times$ and $4\times$ upsampled images, which would have included more photons in the image. However, we found obvious gridding artifacts when setting $\sigma _{min} < 0.2$, indicating that some aspect of data collection and/or background that impacts low signal levels in the tails of the PSF with the fastCCD is not accounted in the idealized simulations. Thus, though it marks a conservative bound in Fig. 6(c), we adopted $\sigma _{min} = 0.2$ as the threshold for the $2\times$ and $4\times$ upsampled images in Fig. 4 (a, b). For the $8\times$ and $16\times$ upsampling cases, we utilized the $\sqrt {\textrm {MSE}_{\textrm {max}}}$ contour plot to guide the choice of the $\sigma _{min}$ threshold for creating images. The red triangle and red square positioned at $\sigma =0.22$ and $\sigma =0.25$ along the 53 dB line correspond to $\sqrt {\textrm {MSE}_{\textrm {max}}}$ values of 11 and 20. Thus, using $\sigma _{min}=0.22$ and $\sigma _{min}=0.25$ for the $8\times$ and $16\times$ upsampled images in Fig. 4(c, d) assured that the estimated worst-case sub-pixel positioning errors would not introduce artifacts. From the standpoint of the design of future experiments, it is clearly beneficial to collect data under conditions that create high SNR and large PSF sizes.

6. Conclusions and Outlook

We have developed and demonstrated an image super-resolution method—XR-UNLOC—for hard x-rays measured with fast-frame-rate detectors that is an adaptation of the principle of PALM/STORM, which pushed the resolution frontier of biological fluorescence imaging to sub-optical-wavelength scales. Using XR-UNLOC, we demonstrated that significant gains in sub-pixel resolution are achievable in the hard x-ray regime by analysis of individual photon locations in a series of “nearly-dark” instances of an image that each contain only a handful of photons. We show this by resolving the fine fringe detail of a high-energy x-ray Bragg coherent diffraction pattern up to an upsampling factor of $16$ of the native pixel pitch of 30 $\mu$m of a charge-integrating fastCCD detector. Central to our approach was the adaptation of the UNLOC photon fitting routine, originally developed for PALM/STORM with optical wavelengths, to the hard x-ray regime to handle much smaller point spread functions. We also developed a metric by which to estimate the limit of reliable upsampling with XR-UNLOC under a given set of experimental conditions in terms of the SNR of a photon detection event and the size of the PSF for guiding future experiments.

Several considerations should be noted when envisioning using XR-UNLOC to enable a full 3D BCDI imaging experiment in which a series 2D diffraction patterns are measured as a function of incident beam angle at the sample. Several key issues are listed here that can be accounted for via realistic hardware and software considerations:

Assuming that the issues raised above are indeed tractable, it is worthwhile to provide certain estimations to enable a comparison with BCDI as it is typically implemented at $\sim 9$ keV x-ray energies. We take the pattern in Fig. 4(a) as a basis for this comparison because the fringes become resolved at this upsampling rate of $2\times$, to a degree that would be suitable for BCDI image inversion. The peak pixel in this image contains 246 photons of 52 keV energy and was measured for an integrated time of 2000 s. This peak signal rate of 0.1 photons per second corresponds to the fully optimized coherent flux at 52 keV at the current APS Sector 1-ID beamline. It is anticipated that the coherent flux will improve by a factor of 700 at this energy at the APS after the source is upgraded to be diffraction limited [38]. Thus, the diffraction pattern in Fig. 4(a) would in principle be obtainable in 2.8 s as compared to 2000 s. This estimated signal rate becomes comparable to standard BCDI measurements of nanocrystals with reasonable scattering strength: 9 keV x-ray energy, $\sim 1-2$ s exposure time per 2D diffraction pattern, peak pixel count rates near the Bragg maximum of $\sim 250-1000$ photons.

A further question arises as to whether the XR-UNLOC strategy remains viable under experimental conditions of $\sim 700\times$ higher coherent signal rates, for which we can again provide estimations. Provided XR-UNLOC can eventually fit overlapping PSFs as we anticipate, a factor of 40 increase in signal rate could likely be handled. We base this estimate on the fact that 322 overlapping-PSF events were found (and discarded from consideration) in the data in this work, or 0.003 overlapping-PSF events per frame (see histogram in Figure 8). A factor of 40 increase in flux would result in an average of 5 overlapping PSF events per frame (presuming a quadratic relationship between flux and the increase in overlap likelihood in a 2D field), which is reasonable for UNLOC routines. Further, if real-time analysis can be imagined on very fast frame rate integrating detectors, overlapping-PSF XR-UNLOC fitting can conceivably run at 1000 Hz (as has been shown in biological imaging), allowing for another $20\times$ increase in signal rate. Together, these factors allow for an estimated $800\times$ increase in signal rate with XR-UNLOC. This corresponds well with the coherent flux increase expected after the APS upgrade and to the per-diffraction-pattern signal levels and few-second integrated exposure times of typical BCDI, providing a viable path forward.

The XR-UNLOC results we presented are made possible by the availability of fast-frame-rate charge-integrating detectors to take rapid snapshots of a pattern that contain only a few photon events, and we noted above a need for kHz-rate detectors for viable XR-UNLOC BCDI. Development of such detectors has recently been driven by the proliferation of latest-generation x-ray free electron lasers and synchrotrons. Further, beyond the UNLOC approach, fast software driven by widespread laboratory use of PALM/STORM also provides alternative robust methods of PSF estimation of individual photons that have the potential to keep up with photon-fitting in real time at >kHz frame rates [39]. Finally, the XR-UNLOC method for fast x-ray CCDs may be a powerful and potentially more flexible complement to hardware-based photon fitting detector chips that are being developed and that will further facilitate this approach [40,41].

Beyond coherent diffraction imaging, which originally motivated this work, XR-UNLOC can be utilized in a broader range of x-ray measurements in which detector pixelation is a limiting factor. Along this vein, it is worthwhile to point out that, in principle, XR-UNLOC is an approach amenable to a wide range of x-ray energies, not strictly limited to the high-energy regime explored in this work. Specific examples of experiments that stand to benefit including x-ray spectroscopy methods in which energy resolution is dictated by pixel size, direct-space imaging of sub-micron volumes of material as in near-field high-energy diffraction microscopy, and XPCS under conditions of very fine speckle granularity or low count rates.