1. Introduction

Integral photography is one of the 3D imaging technologies based on a pinhole array or a lenslet array to capture incoherent light rays from different directions [1]. In the traditional integral photography the rays emerging a 3D objects pass through a lenslet or pinhole array and are captured on a photographic film. Then the reconstruction is carried out optically by generating inverse propagating rays with system similar to the recording one. In [2] we used the term “integral imaging” to describe this technique since we are performing imaging. If the II is recorded digitally the 3D images can be reconstructed by computational methods [2,3,4,5]. A typical example of a digital II recording setup is shown in Fig. 1.

In this paper we present experimental results for CompSAII. In addition, we investigate the FOV and viewing angle limitation of integral imaging systems. We analyze those limitations for typical integral imaging system using a lenslet array. In general the FOV is limited by the aperture of the system. In order to increase the aperture we implement a synthetic aperture approach by which we produce a relative motion between the object and the II system. This process can be viewed as laterally scanning the object or alternatively moving the object across the viewing area. All optical 3D synthetic aperture technique for II was demonstrated in reference [6]. The reconstruction in reference [6] is performed optically which requires synchronizing the movement of the display with that of the II pickup. In this work we use computer reconstruction of 3D images from a SAII together with the synthetic aperture approach.

During the recording step multiple laterally displaced exposures of the 3D object are taken. The elemental IIs obtained this way are combined together to form a SAII. The SAII formed is equivalent to an II captured with an integral imaging system that has an enlarged aperture. The SAII is a compact representation of the information from multiple exposures. Since it has the same format as a regular II it can be further manipulated with tools developed for II. In this work we demonstrate perspective reconstruction of a 3D object from the SAII by applying digital computation methods developed for regular II’s [2–4].

 

Fig.1. A typical digital integral imaging recording setup

Download Full Size | PDF

2. FOV limitation in integral images

The analysis in the following is carried out in one dimension to simplify the mathematical expressions. We assume one-dimensional integral imaging system but the results can be easily expanded to two-dimensional system.

The FOV for any integral image (traditional or digital) is primarily limited by the maximum lenslet exit angle θ_l [7]:

given by:

where z ₁ is the distance from the object to the lenslet array, f_l is the lenslet focal length and p_l is the lenslet lattice pitch. This limitation rises from the need to avoid overlapping of the elemental images. Elemental images, that are the images obtained from each lenslet, should not overlap in order to avoid spatial information cross-talk. This angle also determines the maximal viewing area defined as the range in which the observer can move laterally [6]. The maximum size of an object L_0max that doesn’t cause image overlapping is:

An object obeying Eq. (3) can be placed in a lateral range (Fig. 2):

where L_A=(n-1)p_l is the lenslet array effective size and n is the number of lenses per dimension.

The object size restriction given by Eq. (3) can be remedied avoiding the overlapping of elemental images using gradient-index lens array or by barriers [8]. In this case, object covering the entire range of Eq. (4) can be imaged and reconstructed completely from at least one viewing direction.

 

Fig. 2. The FOV for is primarily limited by the maximum lenslet exit angle θ_l

Download Full Size | PDF

In general the FOV is further limited by the finite aperture of optical elements following the lenslet array such as that of pickup lenses or recording media. Figure 1 illustrates a typical case where a pickup lens with a finite aperture is placed behind the lenslet array. The amount of limitation depends on the viewing point location during the reconstruction. The maximal limitation is for a viewing point located at infinite distance from the lenslet array. In such a case the viewing rays are parallel, crossing the center of the lenslet. Computer reconstruction for a virtual viewing point located at infinity is obtained simply by sampling the II on a grid with lattice constant of the pitch size [2,4]. Other virtual viewing points, closer to the lenslet array, can be simulated by sampling a set of converging rays. In such a case larger viewing angles are permitted at the expense of angular resolution.

Figure 3 illustrates the FOV limitation for the parallel projective case (viewing point is distant from the lenslet array). The object is denoted by a stripped arrow having lateral size L₀. For a 3D object, L₀ can be approximated as the projection of the object in a plane perpendicular to the optical axis at distance z₁ from the lenslet array. Parallel rays are emerging from the object that go through the lenslet array to a pickup lens located at distance z₂ from the lenslet array. The pickup lens has an aperture diameter D. Let us define an observation angle as an angle for which finite size objects located somewhere in the FOV can be completely reconstructed. The observation angle range for this setup is:

where θ_l is the maximum lenslet array exit angle as defined in Eq. (2) and θ_m is an additional angle limitation given by:

The latitudinal location of the object is restricted in this case to the range:

 

Fig. 3. A finite pickup lens induces further limitation on the viewing angles and viewing area.

Download Full Size | PDF

The observation angles defining the FOV should be distinguished from the viewing (or the perspective) angle. We define a viewing angle as the angle from which an object at a given location can be completely seen. Fig 4 shows the viewing angle range for the optical setup of Fig. 3. The maximal viewing angle is $t{g}^{-1}\left[\frac{D-L_0}{2\left(z_1+z_2\right)}\right]$ for small pickup lens aperture and $t{g}^{-1}\left(\frac{L_A-L_0}{2z_1}\right)$ for large pickup lens aperture. Consequently the viewing angle range is given by:

 

Fig. 4 Minimal (solid line) and maximal (dashed) viewing angles of an object placed at optimum location for widest viewing angle range.

Download Full Size | PDF

3. Increasing the FOV and viewing angles by CompSAII

From Eqs. (3–7) it can be seen that the FOV and the viewing angles range of an integral image depend on the aperture of the system determined by the lenslet array aperture L_A and the pickup lens aperture D. Therefore the FOV and the viewing angles range can be increased by increasing the II system aperture. This can be done synthetically by taking multiple exposures while moving the II system relative to the object in a perpendicular plane to the optical axis. Figure 5 illustrates the principle of the synthetic aperture by scanning II the object in y direction. If the displacement between two consecutive exposures is Δx then the effective lenslet array aperture size is increased to L_A+Δx and the effective pickup lens aperture is increased to D+Δx. Consequently, the FOV is increased according to Eqs. (3–7) and the projective angle range is increased as demonstrated by the solid ray in Fig. 5 viewing the object by an angle β_m larger than the original angle α_m. The relative displacement Δx (either of the object or the II system) has to be smaller than the critical aperture of the system (the minimum of L_A and D).

 

Fig. 5. Synthetic aperture II principle. The object is scanned in the vertical direction. A displacement of Δx between two consecutive exposures increases the effective system aperture byΔx.

Download Full Size | PDF

The IIs captured by scanning the object can be combined together to form a SAII. The SAII is similar to the II that would have been obtained from an II system with an enlarged aperture. In order to construct the SAII the displaced IIs need to be registered properly, that is to realign them in order to compensate for the image motion in the detector plane. The exact displacement vector (DV) in the detector plane needs to be known. If the motion of II system during the scanning is mechanically controlled, the DV can be calculated from the precisely known motion steps. Alternatively, the DV can be estimated from the displaced IIs by using motion estimation methods [9].

4. Experimental results

In Figs. 6(a) and (b) the elemental images from two displaced II are shown. Only the central part consisting of 5X18 elementary image are shown. Note that elemental images distant from the center appear truncated due to the finite system aperture. The II were captured with the optical setup described in Fig. 1. The objects are two die with linear dimensions of L₀=15mm and 7mm and were placed z ₁=70mm from the lenslet array. The lenslet array has 53×53 lenslets. The lenslet pitch is approximately p_l≈1.1 mm and the lenslet array width is L_A=58.3mm. The focal length of the lenslets is f_l=5.2 mm. The pickup lens has an aperture of D=36mm and was placed z₂=110mm from the lenslet array. The two II images are captured by moving the II system Δx=25mm in the horizontal direction. The DV was estimated in the II plane by using a block matching algorithm [9]. The DV is found to be d=(d_x,d_y)=(743,1) pixels where (d_x, d_y) are the horizontal and vertical DV components. The II presented in Fig. 6(b) is shown after shifting it -d pixels in order to align it with respect the II shown in Fig. 6(a). Using the two IIs in Figs. 6(a) and (b) we formed the SAII shown in Fig. 6(c). It can be seen that the truncated elemental images in Figs. 6(a) and (b) appear completely in Fig. 6(c).

The viewing angle of the IIs shown in Figs. 6(a) or (b) is limited according to Eq. (7) to $2\mathit{tg}\left[\frac{D-L_0}{2\left(z_1+z_2\right)}\right]=6^0$ . Figure 7 demonstrates computer reconstructions from a single II and from the SAII. In the upper row are shown perspective images for different viewing angles reconstructed from the II shown in Fig. 6(a). In the lower row are shown reconstructions from the SAII of Fig. 6(c) for the same viewing angles. Reconstructions form the SAII can be viewed in an enlarged range of approximately 11 degrees which is consistent with Eq. (7). According to Eqs. (7) and (2), the viewing angle with the synthetically enlarged apertures L’_A=L_A+Δx=83.3mm and D’=D+Δx=61mm is limited by the maximum lenslet exit angle; 2θ_l=11.2⁰°.

 

Fig. 6 (a) and (b)- two displaced lls. (c) Sll computed from (a) and (b).

Download Full Size | PDF

 

Fig. 7. (Movie 21 KB) Computer perspective reconstruction from the II of Fig. 6(a) (upper row) and from the SAII shown in Fig. 6(c) (lower row). θ and ϕ are the horizontal and vertical components of the viewing angle respectively. It can be seen that images wider viewing area can be reconstructed from the SAII.

Download Full Size | PDF

5. Conclusion

In this paper we studied the FOV and viewing angle limitation of integral imaging technique. The main limiting sources are the finite aperture of elements of the imaging system. In order to relax those limitations we can use a computational synthetic aperture technique by which the integral imaging system or object are globally moved with respect to another. Integral images captured during the scanning process are combined together to create a synthetic integral image that has an enlarged effective FOV. The SAII is a compact representation of the captured information and have the same format as regular IIs, therefore can be manipulated in a similar way as IIs are. 3D computed images reconstructed from the SAII are demonstrated to have a wider viewing angle range then that from a single II.