Depth-driven variable-frequency sinusoidal fringe pattern for accuracy improvement in fringe projection profilometry

Gang Rao; Libin Song; Song Zhang; Xiangdong Yang; Ken Chen; Jing Xu

doi:10.1364/OE.26.019986

1. Introduction

3D surface measurement techniques have been applied in various fields [1–3]. Among these techniques, fringe projection profilometry has several advantages, such as being non-contact and having high accuracy and flexibility [4]. Phase shifting algorithms are extensively adopted in fringe projection profilometry because of its superior high resolution, high speed, and full-field inspection. A serial of encoded sinusoidal fringe patterns is emitted from a projector, and the deformed pattern is captured by a camera to decode the relative phase from −π to π via the arctangent function. To resolve 2π discontinuity for 3D surface measurement, a spatial or temporal phase unwrapping method is typically required [5]. The spatial phase unwrapping methods determine number of 2π to be added on each pixel by analyzing the phase map itself and usually via optimization. A number of reliability-guided spatial phase unwrapped methods have been reviewed in [6]. In general, it is difficult for a spatial phase wrapping method to be used for discontinuous 3D shape measurement. Temporal phase unwrapping methods relies on capturing additional patterns in a time sequence to resolve the 2π discontinuity problem. Such a method is usually pixel by pixel without optimization, and thus it is more robust and versatile. A number of temporal phase unwrapped methods have been developed over the years, including two- or multi-wavelength phase shifting [7, 8], gray coding, and spatial coding.

In a typical digital fringe projection system with a phase-shifting technique, a set of uniform-frequency sinusoidal fringe patterns are usually used; and the N-step phase shifting algorithm is extensively employed because of its ability to perform pixel-by-pixel measurement rapidly and accurately. In such a technique, generating high-quality sinusoidal fringe patterns is critical to achieving high measurement accuracy. However, commercial projectors such as those based on digital light processing (DLP) techniques typically large apertures to maximize the light efficiency (e.g., to increase output light intensity). Consequently, all these projectors have a narrow depth-of-field and display only non-distorted fringe images on a fronto-parallel screen at the focus depth [9]. By virtue of the defocus character of projectors, binary defocusing techniques have been proposed to generate high-quality sinusoidal fringe patterns [10, 11]. The intensity of fringe patterns in a projector is iteratively optimized by dithering technology to increase the captured fringe pattern quality [12–15]. Although the quality of a sinusoidal fringe pattern captured by a camera is improved by these methods, the pixel-by-pixel or subgroup-by-subgroup optimization based on object depth range to be measured has not been attempted. As a result, the measurement quality may vary across the object surface. This problem is more pronounced the object is moving when it is is difficult to generate optimized fringe patterns for each measurement.

Generally, the defocusing effect can be represented by the breadth of the point spread function (PSF), which is appropriated by a Gaussian filter [16], and the degree of blur is characterized by the defocus kernel. Different from camera defocus, the defocus degree of a projector is scene-independent [17], which is relative to the distance from the scene point to the projector lens center [18]. Several methods have been proposed for estimating the defocus kernel to recover the depth from the defocus-distance calibration relationship [9, 19, 20]. Ideally, if the captured images reflected from the measurement surface are noise-free and thus correspond to perfect sinusoidal waves, the defocus PSF is used as a standard Gaussian filter to attenuate only the amplitude of the sinusoidal wave. Consequently, the phase decoded by phase shifting will equal the designed phase regardless of the frequency and step number of phase shifting. However, commercial cameras and projectors suffer from different kinds of noise, such as noise caused by temperature, exposure time, environment, and digital quantization [21]. Thus, the noise is inherent to the captured images during the projection and capture processes, which affects the phase quality and thus pixel matching accuracy.

To increase the phase quality for more precise pixel matching, this paper analyzes the pixel matching error with respect to the frequency and amplitude of a sinusoidal fringe pattern captured by a camera. It is found that the higher the frequency and larger the amplitude of the captured sinusoidal fringe pattern, the lower the pixel matching error. On one hand, when the frequency becomes higher, the same intensity error in ordinate axis will be corresponding to a reduced pixels coordinate movement in abscissa axis. On another hand, when the amplitude of the sinusoidal wave is increased, the pixels coordinate movement in abscissa axis corresponding to the same intensity error in ordinate axis is reduced. However, the defocus PSF, generally described by a standard Gaussian filter, is a low-pass filter, that is, a higher frequency will result in a captured sinusoidal fringe pattern with a smaller amplitude. When the frequency becomes higher, the amplitude will be attenuated, so the decrease of the pixel matching error induced by the intensity error through increasing frequency is not monotonous. Therefore, there exist optimal frequencies of the sinusoidal fringe pattern for different defocus degrees in different measurement regions at various depths. Therefore, the sinusoidal fringe patterns to be projected should be designed with variable frequency to reduce the pixel matching error induced by image noise. In this case, according to the variable measurement depth, the corresponding variable defocus kernel of the projector and camera are calibrated using the methods presented in [22] and [23], respectively, and then the optimal variable-frequency sinusoidal fringe pattern along the abscissa and ordinate axes in the projector is designed to reduce the pixel matching error in the camera. Finally, the phase decoding algorithm for the variable-frequency sinusoidal fringe pattern is developed for a 3D measurement.

Section 2 details the entire principle of the optimization method for a depth-driven variable-frequency sinusoidal fringe pattern in fringe projection profilometry. To verify our proposed methods, measurement simulations and experiments are conducted in Section 3. We summarize this research in Section 4.

2. Principle

To improve the pixel matching accuracy for 3D measurement, the principle of the optimized depth-driven variable-frequency sinusoidal fringe pattern in fringe projection profilometry is proposed in this section. The 3D measurement accuracy for phase shifting mainly depends on the pixel matching accuracy, which determines the correspondence between the projector and camera. The pixel matching accuracy relies on the quality of the sinusoidal fringe pattern captured by the camera, which is affected by the inevitable image noise. Therefore, the pixel matching error induced by image noise in the captured defocused image is analyzed, and then the optimal frequency, according to the defocus analysis, is presented in the frequency domain; see section 2.1. Then, taking into account the defocus of the projector and camera, the online frequency optimization for the sinusoidal fringe patterns along abscissa and ordinate axes in the projector is presented in section 2.2 to improve the pixel matching accuracy. For phase shifting, the variable-frequency sinusoidal fringe patterns are designed in section 2.3, and the decoding algorithm is developed in section 2.4.

2.1. Pixel matching accuracy induced by intensity error

Without loss of generality, the sinusoidal fringe patterns along both abscissa and ordinate axes in the projector are used to improve the accuracy and robustness of correspondence matching in this paper.

The error of 3D measurement by phase shifting in fringe projection profilometry comes directly from the pixel matching error between the camera and projector, and the pixel matching error depends upon the decoded phase,

ϕ (U_{c}, U_{c}) = - \arctan {\frac{\sum_{i = 1}^{N} I_{i}^{c} (U_{c} V_{c}) \cdot \sin [(i - 1) \cdot 2 π / N]}{\sum_{i = 1}^{N} I_{i}^{c} (U_{c} V_{c}) \cdot \cos [(i - 1) \cdot 2 π / N]}},

where

I_{i}^{c} (U_{c} V_{c})

is the gray intensity of the images captured by the camera, the subscript and superscript letter c refers to the camera; N is the total number of steps of phase shifting; and i = 1, 2, ⋯, N is the sequence order of fringe patterns. As mentioned in the introduction, the sinusoidal fringe images captured by the camera are corrupted by different kinds of noise; thus, the captured sinusoidal fringes are not perfectly sinusoidal waves. We define the decoded phase ϕ (U_c, V_c) as a function of N gray intensities

I_{1}^{c} (U_{c}, V_{c}), I_{2}^{c} (U_{c}, V_{c}), \dots, I_{N}^{c} (U_{c}, V_{c})

of these images as follows:

ϕ (U_{c}, V_{c}) = f (I_{1}^{c} (U_{c}, V_{c}), I_{2}^{c} (U_{c}, V_{c}), \dots, I_{N}^{c} (U_{c}, V_{c})) .

Thus, the decoded phase error induced by image noise can be written as

Δ ϕ (U_{c}, V_{c}) = \frac{\partial f}{\partial I_{1}^{c}} Δ I_{1}^{c} + \frac{\partial f}{\partial I_{2}^{c}} Δ I_{2}^{c} + \dots + \frac{\partial f}{\partial I_{N}^{c}} Δ I_{N}^{c} .

Generally, the captured sinusoid fringe images with phase ϕ(x, y) can be expressed as

I_{i}^{c} (U_{c}, V_{c}) = I^{c}^{'} + I^{c}^{″} \cos [ϕ (U_{c}, V_{c}) + (i - 1) \cdot 2 π / N],

where I^c^′ is the average intensity and I^c^″ is the amplitude. Therefore, the derivative of f w.r.t

I_{i}^{c}

can be obtained as

\frac{\partial f}{\partial I_{i}^{c}} = - \frac{2 \sin [ϕ (U_{c}, V_{c}) + (i - 1) \cdot 2 π / N]}{N I^{c ″}} .

The intensities of all captured images are corrupted by noise due to various errors. The maximum error ΔI of all pixels is restricted because of limited energy ejecting. To present the maximum decoded phase error resulting from the intensity noise, without loss of generality, errors at the same pixel coordinates for the entire sequence of phase shifting images are set as ΔI under the same capture condition

Δ I_{1}^{c} = Δ I_{2}^{c} = \dots = Δ I_{N}^{c} = Δ I

. Then, Eq. (5) is substituted into Eq. (3). Hence, the maximum decoded phase error can be expressed as

| Δ ϕ |_{\max} = | \sum_{i = 1}^{N} \frac{\partial f}{\partial I^{c}^{″}} | \cdot | Δ I | = \frac{2}{N I^{c}^{″}} | \sum_{i = 1}^{N} \sin [ϕ (x, y) + (i - 1) / N \cdot 2 π] | \cdot | Δ I | = \frac{M_{\sin ϕ}}{N I^{c}^{″}} | Δ I | .

where the intermediate variable

M_{\sin ϕ} = 2 \cdot \sum_{i = 1}^{N} | \sin [ϕ (U_{c}, V_{c}) + (i - 1) / N \cdot 2 π] |

is constant for certain pixel coordinates (U_c, V_c). (U_p, V_p) is the matched pixel coordinates in projector corresponding to (U_c, V_c), and the absolute total phase along U_p axis in projector image plane at (U_p, V_p) can be generally obtained as

ϕ \int_{0}^{U_{p}} ω d U_{p}

, and the derivative of ϕ with respect to U_p is

\frac{d ϕ}{d U_{p}} = ω

. Thus, the maximum pixel matching error of U_p axis can be written as

| Δ U_{p} |_{\max} = \frac{| Δ ϕ |_{\max}}{ω} = \frac{M_{\sin ϕ}}{N I^{c}^{″}} \frac{1}{ω} | Δ I |,

where ω is the frequency of the sinusoidal fringes.

When the projector projects the image onto an out-of-focus plane, the displayed image is blurred due to defocus. The defocus can be modeled as a Gaussian filter as

G (U_{p}, V_{p}) = \frac{1}{2 π σ^{2}} e^{- \frac{U_{p}^{2} + V_{p}^{2}}{2 σ^{2}}} = (\frac{1}{\sqrt{2 π} σ} e^{- \frac{U_{p}^{2}}{2 σ^{2}}}) \cdot (\frac{1}{\sqrt{2 π} σ} e^{- \frac{V_{p}^{2}}{2 σ^{2}}}) = G (U_{p}) \cdot G (V_{p}),

where σ is the defocus kernel and G(U_p) and G(V_p) are the Gaussian filters for the U_p and V_p directions, respectively. Theoretically, the sinusoidal fringe patterns of N-step phase shifting for the U_p axis can be presented as

I_{i}^{p} {(U_{p}, V_{p})}_{u} = I^{p}^{'} + I^{p}^{″} \cos [ω_{u} U_{p} + (i - 1) \cdot 2 π / N],

where

I_{i}^{p} {(U_{p}, V_{p})}_{u}

is the gray intensity for the fringe pattern along the U_p direction in the projector, I^p^′ is the average intensity, I^p^″ is the amplitude, and ω_u is the frequency of the fringe pattern along U_p axis, the subscript and superscript letter p refers to the projector. The output image blurred by defocus can be modelled as the convolution of the defocus Gaussian filter on the designed sinusoidal fringe pattern as

I^{c} (U_{c}, V_{c}) = G (U_{p}, V_{p}) \otimes I^{p} (U_{p}, V_{p}) .

Equation (10) is transferred into the frequency domain by the Fourier transform as

ℱ (I^{c} (U_{c}, V_{c})) = ℱ (G (U_{p}, V_{p})) \cdot ℱ (I^{p} (U_{p}, V_{p})) = ℱ (G (U_{p})) \cdot ℱ (G (V_{p})) \cdot ℱ (I^{p} (U_{p}, V_{p})),

where

ℱ (G (U_{p})) = e^{- \frac{σ^{2} ω_{u}^{2}}{2}}

and

ℱ (G (V_{p})) = e^{- \frac{σ^{2} ω_{v}^{2}}{2}}

are the univariate Fourier transforms of G(Up) and G(Vp), respectively. Thus the displayed image can be retrieved as,

I^{c} {(U_{c}, V_{c})}_{u} = ℱ^{- 1} (ℱ (I^{c} {(U_{p}, V_{p})}_{u})) = I^{p}^{'} + ℱ (G (U_{p})) I^{p^{″}} \cdot \cos [ω_{u} U_{p} + (i - 1) \cdot 2 π / N] .

This equation indicates that the amplitude of displayed image I^c(U_c, V_c)_u blurred by defocus is attenuated by

ℱ (G (U_{p}))

namely

{\begin{cases} I^{c}^{'} = I^{p}^{'} \\ I^{c}^{″} = I^{p}^{″} \cdot e^{- \frac{σ^{2} ω_{u}^{2}}{2}} \end{cases}

which will decrease when the frequency ω_u is increased. Similarly, amplitude attenuation for fringe patterns along V_p axis can be achieved. Thus, the maximum pixel matching error in Eq. (7) can be rewritten as

| Δ U_{p} |_{\max} = (\frac{M_{\sin ϕ}}{N I^{p}^{″}} | Δ I |) \frac{1}{ω} e^{\frac{ω^{2} σ^{2}}{2}} (ω = ω_{u}) .

Equation (14) indicates that the maximum pixel matching error

{| Δ U_{p} |}_{\max}

depends upon fringe frequency. To improve the measurement accuracy,

{| Δ U_{p} |}_{\max}

must be minimized. Thus, we investigate

h_{σ} (ω) = \frac{1}{ω} e^{- σ^{2} ω^{2}} .

Figure 1 illustrates the intuitive illustration of the influence of the fringe frequency and image noise on pixel matching accuracy. From Eq. (14), Eq. (15), and Fig. 1, we can see that 1) when the frequency is higher, the attenuation item $ℱ (G (U_{p}))$ will reduce the amplitude of the fringe pattern on the display; thus, the phase decoding error induced by image noise will be increased; and 2) when the frequency of fringes is lower, the phase sensitivity with respect to intensity variation will be augmented. These two aspects will reduce the SNR of the sinusoidal fringe pattern; thereby, there exists an optimized frequency that minimizes the pixel matching error.

Fig. 1 Interactive influence of frequency of sinusoidal fringes and noise on pixel matching accuracy.

nT_h	10	15	20	25	30	35	Variable
${\bar{R M S}}_{1}$	0.0777	0.1166	0.2037	0.2256	0.9577	2.1546	0.0737
${\bar{R M S}}_{2}$	0.0706	0.0665	0.1825	0.2069	0.2196	0.4719	0.0615
${\bar{R M S}}_{3}$	0.0623	0.0620	0.1815	0.2185	0.2182	0.4464	0.0580

nT_h		10	15	20	25	30	35	Variable
${\bar{R M S}}_{1}$	70% SV	0.0692	0.084	0.1917	0.2253	0.4157	2.0205	0.0665
${\bar{R M S}}_{1}$	55% SV	0.0633	0.0619	0.1748	0.2293	0.2301	1.7242	0.0590
${\bar{R M S}}_{2}$	70% SV	0.0657	0.0578	0.1525	0.1900	0.2101	0.2474	0.0568
${\bar{R M S}}_{2}$	55% SV	0.0600	0.0519	0.1126	0.1671	0.2053	0.2168	0.0499
${\bar{R M S}}_{3}$	70% SV	0.0530	0.0496	0.127	0.1979	0.2091	0.2197	0.0476
${\bar{R M S}}_{3}$	55% SV	0.0472	0.0419	0.0651	0.1665	0.202	0.227	0.0408

nT_h	10	15	20	25	30	35	Variable
${\bar{R M S}}_{1}$	0.084	0.0792	0.1408	0.2164	0.2505	0.3762	0.0727
${\bar{R M S}}_{2}$	0.0819	0.0687	0.0785	0.1283	0.1556	0.1823	0.0672

period number nT_h		10	15	20	25	30	35	Variable
${\bar{R M S}}_{1}$	100 pxs Srk	0.0760	0.0675	0.0986	0.1861	0.2294	0.2645	0.0623
${\bar{R M S}}_{1}$	200 pxs Srk	0.0737	0.0646	0.0693	0.1599	0.2193	0.2551	0.0592
${\bar{R M S}}_{2}$	100 pxs Srk	0.0679	0.0539	0.0502	0.0982	0.125	0.1442	0.0512
${\bar{R M S}}_{2}$	200 pxs Srk	0.0645	0.0492	0.0436	0.0733	0.1068	0.1302	0.0459

period number nT_h		5	10	15	20	25	Variable
$\bar{R M S} (mm)$	PLANE 1	0.0916	0.0614	0.0531	0.0501	0.1230	0.0510
	PLANE 2	0.1154	0.0777	0.0740	0.3342	0.3957	0.0746
	PLANE 3	0.1408	0.0911	0.3554	0.5766	0.4658	0.0976
$\bar{A N G} (°)$	PLANE 1,2	0.2862	0.1393	0.2092	0.8561	1.0516	0.1569
$\bar{A N G} (°)$	PLANE 1,3	1.3635	0.6073	1.2503	2.4105	2.1901	0.2599

Abstract

1. Introduction

2. Principle

2.1. Pixel matching accuracy induced by intensity error

2.2. Optimized frequency based on depth

2.3. Variable-frequency shifting fringe pattern design

2.4. Variable-frequency sinusoidal fringe pattern decoding algorithm

3. Experiment

3.1. Projector and camera defocus kernel calibration

3.2. Simulations

3.3. Measurement of field objects with variable depth

4. Summary and discussion

Funding

References and links

Cited By

Figures (21)

Tables (5)

Equations (46)

Optics Express