1. INTRODUCTION

During the last decades, our technological capabilities to act at the micrometer, nanometer, or subnanometer scales has increased continuously. Nowadays, a wide range of apparatus are proposed for specimen machining or object characterization over an extended scale range. These instruments are equipped with sophisticated or servo-controlled positioning stages monitoring displacements with the required level of resolution. However, processes involved in the realization of ever more complex structures assume the succession of numerous steps and the specimens developed may have to be transferred several times from an instrument to another one. When such specimen transfers are necessary and despite the accuracy of the translation stages, the localization of the regions of interest remains a difficult task. This difficulty is due to the loss of the specimen position relative to the controlled stage because of the specimen transfer. A “blind search” has thus to be carried out that can be assisted by any kind of alignment marks inserted deliberately in the vicinity of the features of interest.

An interesting way to address this problem of the localization of the regions of interest after specimen transfers is to use position-referenced sample holders allowing the systematic determination of the relative position of the specimen with respect to the displacement stage. This approach was first introduced in the field of optical observation of biological samples with the proposal of microscope slides with different kinds of finder grids [1, 2, 3, 4, 5, 6]. In these cases, the operator remains in charge of the storing and of the visual recovering of the regions of interest and only a coarse positioning is achieved. In the field of robotics, this approach has been proposed by using microgrids in which the position is encoded within the width of the lines [7].

As proposed in a recent work, a pseudoperiodic encryption of an extended surface can be used for the unambiguous localization of any random zone of tiny dimensions [8]. This concept provides a high accuracy in both in-plane position and orientation through digital processing of the recorded image of the pseudoperiodic pattern. The method has been successfully applied to the imaging of biological materials, either on microscope slides [9] or within live cell culture dishes [10]. These applications of position referencing to the imaging of biological materials are straightforward since the microscope used for specimen inspection can also be used for the observation of the pseudoperiodic pattern. Additional equipments are thus unnecessary for the implementation of this localization approach and the only devices required are the suited position-referenced sample holders. In the general case of nonoptical instruments the problem is different since a vision system dedicated to the observation of the pseudoperiodic pattern (or of any localization mark) has to be inserted in the setup. On top of extra cost, the hosting of a vision system assumes that a sufficiently large volume is available. This spatial requirement can be problematic, especially when operating in a controlled environment as in a vacuum chamber for instance. In this perspective the compactness of the vision system is crucial.

In this paper, we demonstrate our capability to retrieve the in-plane position of a pseudoperiodically patterned plate observed with a lensless vision system with a high accuracy. The absence of an imaging lens allows the conception of a compact position probe for its easier insertion in already existing or future technological instruments. The imaging of the patterned plate is based on digital holography [11]. In this method, the solid-state camera records the interferogram formed by a known reference beam with the wavefront diffracted by the object of interest. Then the object image is reconstructed numerically through the coherent computation of the backward propagation of the diffracted beam recorded up to the object plane [12, 13]. Experimentally, we demonstrate a resolution of $100\mathrm{nm}$ in lateral positioning with an allowed travel range of more than $1.6\mathrm{cm}$. Numerical computations are adapted to the actual position of the patterned plate along the optical axis; the latter may vary of more than $5\mathrm{cm}$, constituting an extra large depth of focus.

2. PSEUDOPERIODIC PATTERN FOR LATERAL POSITION ENCRYPTION

In order to provide high accuracy in the positioning of small-sized areas within an extended two-dimensional (2D) work space, a kind of double-scale encryption principle has been introduced [8]. This principle might be compared to the Vernier principle as implemented in the Vernier calliper. In the latter, the final size (or position) is obtained by adding a fine but relative measurement given by the secondary Vernier scale to a coarse but absolute measurement given by the main millimetric scale. In the 2D encryption concept proposed for digital image processing, the two scales are nested with each other in the form of a pseudoperiodic pattern. The periodic frame provides the high resolution through its fine positioning with respect to the frame of image pixels of the vision system while the absolute position is encrypted within the alterations of the periodic frame introduced by the pseudoperiodic encoding. In our previous work [8] we addressed both in-plane position and orientation. In this paper, we only address in-plane position for cases in which orientation is already known, as, for instance, by mechanical guiding of a sample holder on an instrument stage. We are thus only concerned by translations and this allows a simplified position encoding as well as faster computations for digital position retrieval.

The pseudoperiodic encoding of the position is schematized in Fig. 1. Along one direction, position is encoded through a sequence of bright lines onto a dark background. The bright lines are distributed along a periodic frame but some of them are missing in order to encrypt the absolute position. In fact, the position along the sequence of lines is encrypted within a binary code based on linear feedback shift registers (LFSR) [14]. Bits of value 1 are made of three consecutive bright lines, whereas bits of value 0 are made of one absent line in the middle of two bright lines. Thanks to the LFSR principle, each set of N consecutive bits is representative of a position as represented in the figure for the case of $N=3$. Each word of N bits shares its $N-1$ least significant bits with the previous word as well as its $N-1$ most significant bits with the following word. This concept does not respect the natural binary sequence but it allows a drastic reduction of the number of bits necessary to encode $2^{N-1}$ words [8]. The transcription of the word read into the actual position along the sequence is obtained thanks to a look-up table given by construction of the LFSR sequence. By reproducing the same pseudoperiodic sequence in the perpendicular direction, we obtain a 2D encoding of position as represented by the pseudoperiodic distribution of bright spots in the right part of Fig. 1. With such a symmetrical encoding of position, there is a $\pi /2$ ambiguity in the pseudoperiodic pattern orientation. This is acceptable since we consider that in-plane orientation is a priori known. The unambiguous encoding scheme introduced previously is available for the general case in which both position and orientation have to be determined [8].

We may notice that LFSR sequences have already been used successfully for absolute position encoding and automatic detection [15]. In this case, a dual track scheme has been used and only linear displacements are addressed for applications to the field of computer numerically controlled machine tools. A high rate of detection has, however, been obtained thanks to a custom detector probe with integrated electronics.

3. DIGITAL HOLOGRAM RECORDING AND RECONSTRUCTION OF THE OBJECT IMAGE

Holography consists basically to record a propagating wavefront in both amplitude and phase onto a material support. Interferences between a reference beam and the object beam are formed in order to access the wavefront phase. At the reconstruction stage, the recorded wavefront is regenerated and propagates as in time of recording. Object images are thus obtained as if the object was still present [16].

Digital holography has benefited from the development of 2D solid-state image sensors to avoid the use of chemical photoresists. However, the limited size and resolution of image sensors restricts its field of applications. Nevertheless, digital holography is a convenient means to build automated and computer-controlled detection systems and various implementation designs have been proposed [11]. For our positioning application, we designed a digital holography probe, as presented in Fig. 2a. The beam-splitter cube divides the incident light beam for illuminating both the reference mirror and the pseudoperiodically patterned glass plate. After reflection and diffraction, the light beams are recombined by the beam-splitter cube and interfere on the camera sensor. For compactness and stability purposes, the image sensor chip is directly stuck on the output side of the beam- splitter cube. The reference mirror is directly made of the outside diopter of the beam splitter that can be previously coated for adjusting its reflection power. A tilt is introduced at this stage for generating a regular fringe carrier in the recorded holograms for allowing an efficient removal of the DC term at the reconstruction stage. Assuming a $1/2$” diagonal size of the sensor chip, a $10\mathrm{mm}$ beam-splitter cube would be sufficient and the thickness of the vision system could be maintained smaller than $20\mathrm{mm}$. Such a custom detection probe was not built and demonstration experiments were performed by means of separated parts from the shelf, as represented in Fig. 2b. Excepting extra size due to the packaging and supports of the elements, the setup used actually is representative of the concept of Fig. 2a. The camera used is a $USB2UI-1480SE-M$ from IDS Imaging with $2560\times 1920$ pixels. The beam-splitter cube has $20\mathrm{mm}$ sides, and a He–Ne laser beam is used as a light source. The pseudoperiodic pattern has been reproduced by photolithography on a microscope slide that is supported by a servo-controlled piezoelectric transducer ($\mathrm{PZT}\text{:}PI-\mathrm{P}615$). The latter allows the performance of calibrated displacements for method demonstration and characterization.

Figure 3 presents an example of a recorded hologram with a size of $2048\times 1920$ pixels as used for image reconstruction. Some regular features appear at the pattern periodicity because of the Talbot effect [17]. The insert zoom shows the fringe carrier frequency due to the mean tilt between the reference and object wavefronts. Such a frequency carrier is systematically used in order to produce well separated lobes in the spectral domain. This condition avoids crosstalk in the reconstructed image between the different diffraction orders; respectively, $-1$, DC, and $+1$.

Briefly, the reconstruction of the object image is calculated by the angular spectrum of plane wave approach in the scalar approximation [12]. The recorded hologram $H(x,y)$ is numerically propagated with a tilted reference plane wave in order to fit the experimental geometry. The complex amplitude distribution $A(x,y,z=0)$ in the hologram plane (distance $z=0$ before propagation) can be written as

The in-focus plane $z=z_{\text{obj}}$ is reached when the maximum of contrast is obtained in the reconstructed intensity image $(|A(x,y,z_{\text{obj}}){|}^2)$ and corresponds to the experimental hologram recording distance. This step is easily performed since the pattern exhibits a highly contrasted intensity variation along each period. Automatic focusing methods dedicated to digital holography could also be used (see, for example, [18]).

The spatial resolution in the reconstructed image plane is fixed by the speckle grain size s [19], which could be expressed for a square hologram by

Figure 4 presents a typical image of the patterned glass plate as reconstructed numerically with $z_{\text{obj}}=90\mathrm{mm}$. The variation observed in the mean pattern intensity is due to the Gaussian intensity profile of the illuminating light beam. The period observed of about 68 pixels is consistent with the actual pattern period of $150\mathrm{\mu m}$ and the pixel size of $2.2\mathrm{\mu m}$ knowing that the magnification of the digital holography setup used is unity. The lateral speckle size is about $14\mathrm{\mu m}$, which corresponds to around 6 times the value of the pixel and is enough to correctly resolve the pattern period. Whereas some noise is observed in the reconstructed image, the encoding pattern is clearly visible and allows the absolute determination of the lateral position of the zone observed with respect to the whole patterned plate as described below.

4. POSITION RECONSTRUCTION

The image processing described here is a particular case of a more general procedure described in full details elsewhere [8]. We take advantage of the known pattern orientation to improve the signal-to-noise ratio and to speed up the computations.

Since the pattern orientation is a priori determined in our case, we choose it parallel to the pixel frame of the image sensor for convenience. Then all lines, respectively columns, are independent reproductions of the same signal of the horizontal, respectively vertical, position of the patterned plate. To benefit from this large amount of information, the position decoding is based on the sum of a large set of lines, respectively columns. A convenient effect of this data averaging is to improve the signal-to-noise ratio by reducing the effects of noise; as, for instance, the oblique lines observed in Fig. 4 and caused by defaults in the patterned plate processing. Figure 5a presents the intensity profile obtained horizontally by summing 1000 lines in the central part of the image. Then a signal analysis is performed at the particular frequency of interest of the pseudoperiodic pattern. This is done by correlating this intensity profile with an analysis function made of a complex harmonic signal windowed by a Gaussian envelope. A narrow envelope providing a high spatial resolution is first used for identifying the location of the missing columns of spots representative of the bits equal to 0. This first step is illustrated in Fig. 5b representing the modulus of this space-frequency analysis. In this modulus profile, the missing columns of dots appear as local minima that are detected for the identification of the positions and values of the encrypted bits. From these data, a coarse but absolute position of the reconstructed view is obtained. A second space-frequency analysis is performed by using a wider envelope in order to get a high spectral resolution. In this second case, we are interested in the spectral phase, as represented in Fig. 5c. The latter is representative of the fine position of the pattern columns with respect to the horizontal frame of the camera pixels. Finally, the combination of these fine and coarse data leads to the absolute and high accurate position of the observed view along the horizontal direction. This processing is also applied vertically to the sum of 1000 columns and provides the complementary vertical position. The lateral position of the patterned plate is thus obtained with respect to the camera pixel frame.

5. EXPERIMENTS AND RESULTS

The method capabilities have been demonstrated through repeatability experiments as well as through the reconstruction of controlled displacements. To illustrate the extended depth of focus allowed by digital holography, experiments were carried out twice; first, with the PZT close to the beam- splitter cube and second, after shifting the PZT by a distance of $5\mathrm{cm}$ backwards from the beam-splitter cube. Figure 6 presents the deviation of the reconstructed positions of the patterned plate while it was maintained static for both positions of the PZT. For the closest position, the peak-to-peak level of noise is $0.6\mathrm{\mu m}$ in X and $0.18\mathrm{\mu m}$ in Z, while the standard deviations are $108\mathrm{nm}$ in X and $33\mathrm{nm}$ in Z. These rms levels of repeatability correspond to one twentieth of a pixel in X and one sixty-sixth of a pixel in Z. The difference between the X and Z directions is caused by an artefact due to the camera electronics that introduces a phase jitter in the line clock signal (jitter on PIXCLK of $1.03\mathrm{ns}$: Aptina Imaging datasheet) that is therefore independent of the intrinsic capabilities of the method. We may then evaluate the method resolution to be about $0.1\mathrm{\mu m}$ from the results observed along the Z direction by considering 3 times the standard deviation [20]. For the backward position of the PZT, the repeatability is slightly worse with standard deviations of $140\mathrm{nm}$ in X and $48\mathrm{nm}$ in Z. This reduction of the performance level is consistent with the lower signal-to-noise ratio of holograms recorded at this distance because of the larger size of the speckle grain as described previously. Even in this case, a resolution of $144\mathrm{nm}$ can be claimed in Z that corresponds to one fifteenth of a pixel.

Figure 7 (left-hand side) represents the reconstructed Z position while the patterned plate was shifted 250 times by steps of $50\mathrm{nm}$ by means of the PZT. The straight displacement is retrieved as expected and the deviation from the mean-square straight line is represented in Fig. 7 (right-hand side). Despite a curved shape due to the PZT nonlinearity, the standard deviation is of only $55\mathrm{nm}$. This value is consistent with the repeatability evaluation discussed above for the closest position of the PZT.

The PZT was also used to perform more significant dis placements, as represented in Fig. 8 for both PZT positions. In this case a spiral displacement was driven to the PZT by steps of $40\mathrm{\mu m}$ and reconstructed from the recorded holograms. We observe that displacements larger than the pattern period can be reconstructed without ambiguities thanks to the pseudoperiodic encryption and decoding that allow the identification of the actual index of lines and columns across the patterned plate. These results demonstrate again the extended focus capability.

We finally replaced the PZT translator by a set of three linear motors (PI-M111-DC with serial Mercury C-863 controller) with a nominal displacement range of $15\mathrm{mm}$. Again we applied an extended 2D displacement to the patterned plate and we reconstructed its absolute position from the holograms recorded. Figure 9 presents the results obtained for two axial positions of the plate, $10\mathrm{mm}$ apart from each other. The truncated shape of the spiral displacement observed is due to the driving signal applied. The latter had a full range of $18\mathrm{mm}$ that was distorted because of motor saturation at about $16.2\mathrm{mm}$ in both directions. These results demonstrate, however, the ability of the method to reconstruct the absolute position of the patterned plate over an extended range of displacement. The $16\mathrm{mm}$ excursion of these displacements have to be compared with the $5.63\times 4.22{\mathrm{mm}}^2$ chip size of the image sensor. During these displacements, nonoverlapping zones of the patterned plate were observed by the lensless vision system. The successive positions could still be registered with respect to each other with a subpixel resolution thanks to the decoding of the pseudoperiodic sequence and to the relative phase computations. In practice, the allowed lateral displacement is only limited by the extension of the encoded area on the patterned glass plate. In our case, it is of $28\times 28{\mathrm{mm}}^2$ with a pseudoperiodic encoding with words of $6\mathrm{bits}$, i.e., larger than the travel range of the actuators used.

6. DISCUSSION

We introduced digital holography as a lensless vision system for an application to in-plane position sensing of a patterned glass plate. Subpixel resolution is achieved allowing the combination of high accuracy positions with extended travel ranges. The choice of digital holography as imaging principle was aimed to conceive a compact vision system. This principle is clearly demonstrated and it can be very useful in confined application environments. However, this approach has other consequences on the final system specifications. On one hand, since images have to be reconstructed numerically, the processing time is longer, especially in cases of longitudinal motions that require the systematic determination of the best focus depth. On the other hand, digital holography allows an extra large depth of focus that makes a submicrometer position accuracy compatible with longitudinal displacements of several centimeters. Such a property could not be achieved with a conventional, refractive imaging system.

The expected depth of focus also determines the kind of light source to use. If the object is assumed to encounter axial displacements, a collimated laser beam would be preferred to minimize intensity variations due to the variable distance from the image sensor. In such cases, the light source can be made of a collimating lens illuminated by the output face of a single-mode optical fiber. Then the light source would be centimeter-sized. In other cases, i.e., if the object-image-sensor distance is known to remain almost constant, then a diverging light beam can be used and the light source can be reduced to the output face of an optical fiber, i.e., with a submillimeter size.

Thanks to the lensless vision system demonstrated here, the experimental capabilities of optical sensing have been enlarged. Among possible applications, we identified the systematic localization of sample holders in nanotechnological applications for the straightforward retrieval of tiny areas of interest on specimens that are transferred from one instrument to another one.

ACKNOWLEDGMENTS

The authors acknowledge C. Ecoffey and Q. Lacroix for their help in setting experiments up.

 

Fig. 1 Pseudoperiodic pattern for lateral position encryption with $3\mathrm{bits}$. Left: Bits are encoded through binary symbols of equal width. The value “0” is made of one black line in the middle of two white lines whereas the value “1” is made of three white lines. Bits are nested with each other in order to provide a continuous sequence of words. The position along the sequence is given by a lookup table. Right: 2D view of an encoded area of $10\times 10\mathrm{bits}$.

Download Full Size | PDF

 

Fig. 2 (a) Integrated concept of lensless optical head. CCD, image sensor; LB, laser beam; BS, beam splitter; M, mirrors; PPP, pseudoperiodically patterned plate. (b) Photograph of the setup used for experiments.

Download Full Size | PDF

 

Fig. 3 Example of digital hologram recorded (in gray levels). The insert shows the modulated fringe carrier due to the tilt of the reference mirror.

Download Full Size | PDF

 

Fig. 4 Image of the pseudoperiodic pattern as reconstructed numerically (pixel size: $2.2\mathrm{\mu m}$; pattern period: $150\mathrm{\mu m}$).

Download Full Size | PDF

 

Fig. 5 Image processing along the horizontal direction expressed in column pixels. a) Intensity profile obtained by summing 1000 image lines (reverse contrast); b) modulus of the space-frequency transform at the carrier frequency; c) wrapped phase.

Download Full Size | PDF

 

Fig. 6 Position deviation without driven displacement. Left: PZT next to the beam-splitter cube (st. dev. $108\mathrm{nm}$ in X; $33\mathrm{nm}$ in Z). Right: after shifting the PZT by $5\mathrm{cm}$ backward (st. dev. $140\mathrm{nm}$ in X; $48\mathrm{nm}$ in Z).

Download Full Size | PDF

 

Fig. 7 Left: reconstructed displacement while the PZT is shifted 250 times by steps of $50\mathrm{nm}$ in Z. Right: position deviation from the mean straight line ($\mathrm{rms}=54.5\mathrm{nm}$).

Download Full Size | PDF

 

Fig. 8 Example of reconstructed 2D displacements of the patterned plate. Left: PZT next to the beam-splitter cube. Right: after shifting the PZT by $5\mathrm{cm}$ backward.

Download Full Size | PDF

 

Fig. 9 Example of the reconstruction of extended 2D displacements. Left: patterned plate next to the beam-splitter cube. Right: after shifting the patterned plate by $1\mathrm{cm}$ backward.

Download Full Size | PDF

1. B. Feldman, “Microscope slide,” U.S. patent 4,183,614 (15 January 1980).

2. P. Lin and F. Ruddle, “Photoengraving of coverslips and slides to facilitate monitoring of micromanipulated cells or chromosome spreads,” Exp. Cell Res. 134, 485–488 (1981). [CrossRef]   [PubMed]  

3. K. M. Saleh, P. G. Toner, K. E. Carr, and H. E. Hughes, “An improved method for sequential light and scanning electron microscopy of the same cell using localising microcoverslips,” J. Clin. Pathol. 35, 576–580 (1982). [CrossRef]   [PubMed]  

4. F. Ruddle and P. Lin, “Method for engraving a grid pattern on microscope slides and slips,” U.S. patent 4,415,405 (15 November 1983).

5. G. Dimou and T. Pang, “Process for manufacturing a cover glass with a viewing field,” U.S. patent 5,766,677 (16 June 1998).

6. L. Hause and D. Jeutter, “Mapping method for a microscope slide,” U.S. patent 5,786,130 (28 July 1998).

7. D. St-Jacques, S. Martel, and T. B. FitzGerald, “Nanoscale grid based positioning system for miniature instrumented robots,” in Proceedings of IEEE Canadian Conference on Electrical and Computer Engineering (IEEE, 2003), pp. 1831–1834.

8. J. A. Galeano-Zea, P. Sandoz, E. Gaiffe, J. L. Prétet, and C. Mougin, “Pseudo-periodic encryption of extended 2-D surfaces for high accurate recovery of any random zone by vision,” Int. J. Optomechatron. 4, 65–82 (2010). [CrossRef]  

9. P. Sandoz, R. Zeggari, L. Froelhy, J. L. Prétet, and C. Mougin, “Position referencing in optical microscopy thanks to sample holders with out-of-focus encoded patterns,” J. Microsc. 225, 293–303 (2007). [CrossRef]   [PubMed]  

10. J. Galeano, P. Sandoz, E. Gaiffe, S. Launay, L. Robert, M. Jacquot, F. Hirchaud, J. L. Prétet, and C. Mougin, “Position- referenced microscopy for live cell culture monitoring,” Biomed. Opt. Express 2, 1307–1318 (2011). [CrossRef]  

11. U. Schnars and W. Jueptner, Digital Holography (Springer Verlag, 2005).

12. J. W. Goodman, Introduction to Fourier Optics (McGraw Hill, 1996).

13. K. Matsushima, H. Schimmel, and F. Wyrowski, “Fast calculation method for optical diffraction on tilted planes by use of the angular spectrum of plane waves,” J. Opt. Soc. Am. A 20, 1755–1762 (2003). [CrossRef]  

14. S. W. Golomb, Shift Register Sequences (Holden-Day, 1967).

15. K. Engelhardt and P. Seitz, “Absolute, high-resolution optical position encoder,” Appl. Opt. 35, 201–208 (1996). [CrossRef]   [PubMed]  

16. R. J. Collier, C. B. Burckhardt, and L. H. Lin, Optical Holography (Academic, 1971).

17. H. F. Talbot, “Facts relating to Optical Science No. IV,” Philos. Mag. 9, 401–407 (1836).

18. A. E. Mallahi and F. Dubois, “Dependency and precision of the refocusing criterion based on amplitude analysis in digital holographic microscopy,” Opt. Express 19, 6684–6698 (2011). [CrossRef]   [PubMed]  

19. J. C. Dainty, Laser Speckle and Related Phenomena, 2nd ed. (Springer-Verlag, 1984).

20. R. J. Hansman, “Characteristics of instrumentation,” in The Measurement, Instrumentation, and Sensors Handbook, J. G. Webster (ed.) (Springer-Verlag, 1999), Chap. 1.