1. Introduction

Fringe projection profilometry (FPP) is a 3D measurement technique that is known for its advantages of non-contact, high-precision and whole-field, and it is used for a variety of applications, such as mechanical measurements, industrial monitoring, and dental reconstruction [1,2]. However, balancing the accuracy and speed for 3D measurement is still challenging in many important applications for traditional FPP [3,4]. The reasons are: (1) The intensity range of 8-bit sinusoidal fringe patterns projected in traditional FPP is 0-255. Yet the maximum switching rate of 8-bit sinusoidal fringe patterns projected by a DMD-based projector is usually limited to 120Hz. It is not feasible to use an off-the-shelf digital projector to perform a high-speed task with sinusoidal fringe patterns projection. (2) If careful nonlinearity correction is not carried out in advance [5], the nonlinearity of DLP projectors may deteriorate the high-quality sinusoidal fringe patterns. For this purpose, some nonlinear correction techniques [6] can be used to compensate for phase errors caused by the nonlinearity and improve measurement accuracy.

Lately, numerous researchers have dedicated themselves to investigating binary encoding techniques to accomplish both high-speed and high-precision in 3D shape measurement. Due to the fact that binary encoding patterns only have two grayscale values (0 and 255), the nonlinearity can be avoided, and high-speed projection can be achieved by virtue of DLP technology with its inherit high binary pattern switching rate. Then, approximate sinusoidal fringe patterns with a reduced contrast can be created through defocused projection, allowing 3D measurement at a rapid speed while maintaining or even improving measurement accuracy. Typical binary encoding techniques can be classified into two categories: one-dimensional (1D) modulation encoding and two-dimensional (2D) modulation encoding.

The early binary encoding methods are mostly designed in 1D modulation. Lei et al. [7] proposed the squared binary modulation(SBM), but a large defocus was required to suppress high-order harmonics when the fringe period was larger. To generate high-quality sinusoidal fringes under a small projector defocus, Ayubi et al. [8] borrowed the idea of sinusoidal pulse width modulation(SPWM) from power electronics and applied it to the binary defocusing 3D measurement. By shifting non-fundamental frequency components to higher frequency components, high-order harmonics can be eliminated even with a small defocus, resulting in better sinusoidal characteristics of the fringe images. Later, Wang et al. [9] proposed the optimal pulse width modulation(OPWM), which could eliminate the harmonics of specific orders in the square wave, making it possible to obtain high-quality sinusoidal fringes after defocusing projection, and effectively improving the phase quality. Zuo et al. [10] studied the sensitivity of N-step phase-shifting to different components of high-order harmonics and found that if the four-step phase-shifting method was used, phase errors only affected odd-order harmonics. The authors further proposed three-level SPWM [11] and applied it to the reconstruction of dynamic scenes. Cai et al. [12] introduced a 3D shape measurement method based on 1D temporal-spatial binary encoding. This method utilized a spatial encoding mode and two temporal encoding modes to obtain a 6-bit code word, which could distinguish 64 fringe periods in sinusoidal phase-shifting images and achieve absolute phase unwrapping. However, in order to extract 6-bit code words from 1D mode, a complex decoding scheme was designed.

Researches have been conducted on 2D modulation techniques which include area modulation, ordered-Bayer dithering, and error diffusion. Su et al. [13] proposed an area modulation method using micro-machined gratings to create sinusoidal fringe patterns. Wang et al. [14] used the Bayer dithering algorithm to encode the sinusoidal pattern, generating good phase quality for wider fringe periods, but it has visible repetitive texture structures, leading to poor phase quality. Among these approaches, the spatial error diffusion (ED) method and its variants [15–20] have been most widely investigated and applied, which is generally a more reliable and efficient way to improve the encoding quality of binary patterns than 1D.

The spatial ED methods calculate the "quantization error" for each pixel when binarizing the grayscale pattern, and the quantization error is subsequently diffused to the adjacent pixels rather than simply discarding it, creating a more accurate representation of sinusoidal fringe pattern. Typical ED methods include Floyd Steinberg dithering, Sierra Lite dithering, Stucki dithering, etc. These methods have different error diffusion kernels, which are designed to spread the error as naturally and smoothly as possible across the pattern, but the same quantization threshold(0.5) is adopted, which is a predefined value to determine if a pixel should be replaced with black(0) or white(1). There will be large encoding errors in some areas, resulting in large phase jump errors in the final result and a loss of accuracy in the 3D reconstruction. Obviously, the quantization threshold can be further optimized to improve encoding quality.

Scholars have suggested optimizing the diffusion kernel in spatial ED methods. Zhou et al. [16] proposed a time-consuming kernel optimization algorithm with intensive search. While Zhu et al. [17] proposed using a genetic algorithm to speed up the optimization process.

The fixed threshold in spatial ED doesn’t fully utilize the characteristics of sinusoidal fringe, so non-fixed threshold dithering methods like Zheng’s sigmoid function and gradient descent algorithm [18] are emerged instead. However, careful parameter design is necessary to avoid local optima. Cai et al. [21] introduced threshold optimization, but its effective measurement depth range is limited because of defocused projection. The quality of binary coding patterns is further improved by the optimization algorithms. Common optimization objectives include intensity-based technique [22], phase-based technique [23], structural similarity error [20], frequency domain signal error, or their combination, which can be performed on the entire pattern, pattern block or diffusion kernel.

The projector needs to be defocused for the vast majority of spatial ED methods, in order to effectively suppress the higher-order harmonics contained in the binary encoding patterns themselves and to approach ideal sinusoidal fringes. However, defocusing projection inevitably leads to a reduction in the measurement depth and a decline in SNR of captured images, which can adversely affect the accuracy of phase extraction, particularly for high-frequency fringes [24] for final 3D reconstruction. To address this issue, temporal-spatial binary encoding methods have been proposed to ensure that the binary fringes in the form of focused projection still maintain good sinusoidal properties. Ayubi et al. [8] decomposed one sinusoidal fringe pattern into eight binary patterns, heavily increasing the number of projected patterns, which was also accompanied by a complex and time-consuming decoding process. Alternatively, Li et al. [25] proposed sampling on a sinusoidal curve to generate a series of square binary patterns with different widths. Zhu et al. [26] proposed a temporal-spatial binary(TSB) encoding method, which evenly divided the continuous grayscale intervals [0,1] into $K$ intervals during the error diffusion process and applied corresponding $K$ fixed quantization thresholds to binarize them. One sinusoidal fringe pattern is binarized into $K$ binary patterns, and after integral imaging, a corresponding approximate sinusoidal fringe is obtained.

The purpose of this study is to achieve fast and accurate 3D measurement through temporal-spatial encoding of sinusoidal fringes using the high-speed projection characteristics of DMD-based projector. In this paper, we propose a new dynamic threshold optimization model to generate temporal-spatial binary encoding patterns, which enables the encoded patterns to achieve smaller and uniformly distributed phase errors. We modify fixed thresholds by using the grayscale values of local regions and derive their optimal weight coefficient through the simulation experiment. We calculate the binary patterns for each interval by encoding the sinusoidal fringe pattern in the temporal axis and spatial coordinates. Finally, approximate sinusoidal fringes are obtained through integral imaging, and the phase analysis algorithm is used to acquire the absolute phase map. By combining it with the system calibration parameters, the 3D shape of the tested object is retrieved. The experimental results indicate that the temporal-spatial binary encoding method with dynamic threshold optimization proposed in this paper allows for the use of binary fringes in high-speed and high-precision measurement scenarios, even when the projector is nearly-focusing. This method greatly expands the measurement range of binary fringes and provides a depth measurement range that is equivalent to standard sinusoidal fringes.

2. Temporal-spatial binary fringe encoding based on dynamic threshold optimization

2.1 Encoding principle

Both the idea of temporal binary encoding and spatial binary encoding are carried over into the temporal-spatial binary fringe encoding. While spatial binary encoding refers to encoding within an image and the currently processed pixel is related to its surrounding pixels, temporal binary encoding refers to encoding along the time axis [26]. Temporal encoding often performs better than spatial encoding in terms of decoding and measurement accuracy.

Using the temporal-spatial binary encoding approach we provide, one computer-generated 8-bit standard gray sinusoidal fringe pattern yields more than two binary fringe patterns. One experimental sinusoidal fringe image can be produced with integral imaging (many temporal-spatial binary fringe patterns are simultaneously projected onto the target and imaged on the camera) by in-focus projecting the sequence; the process is illustrated in Figs. 1(a)-(c). The intensity-normalized sinusoidal fringe pattern $I_s^i$ shown in Fig. 1(a) is expressed by:

 

Fig. 1. The 2D and 1D schematic diagram of the temporal-spatial binary encoding process when $K$=4:(a)sinusoidal fringe pattern $I_s$; (b)temporal-spatial binary encoding patterns $\{B_k\}$; (c) the approximate sinusoidal fringe pattern $\tilde {I}_s$ after the low-pass filtering of system point spread function and integral imaging; (d)the detailed 1D curve of sinusoidal fringe pattern $I_s$; (e) 1D encoding patterns $\{B_k\}$ and the integral imaging pattern$\tilde {I}_s$.

Download Full Size | PDF

First, $K$ ($K \geq 2$ and $K$ are integers) intervals are evenly created from the normalized intensity $I_s(u,v)$(phase shift $i$ is omitted afterwards). For instance, the intensity intervals are [0, 0.25),[0.25,0.5),[0.5,0.75) and [0.5, 1] when $K=4$. As a result, one binary fringe pattern will correspond to $K=4$ intensity intervals, as shown in Fig. 1(b). In general, we may get $K$ binary fringe patterns ($B_k$) with $k$=1, 2,…, $K$, and the corresponding intensity intervals (($k$-1)/$K$, $k$/$K$) with $k$=1, 2,…, $K$ for $K \geq 2$. Equation (2) yields the average between the highest $k/K$ and the minimum intensity ($k$-1)/$K$ as the quantization threshold $\epsilon _k$ for each interval.

An symmetrical error diffusion kernel $H_{kernel}$ is used as the coefficients controlling the proportion of quantization error transferring to neighborhood pixels $\Omega$ in order to linearly and evenly distribute the quantization error to surrounding pixels thereby avoiding local accumulation:

Regarding the temporal encoding, when spatial encoding for a specific pixel is applied inside intensity interval $k$, the same pixel corresponding other intensity intervals will be encoded in accordance with the following specially created criteria.

Typically, Eq. (6) can be used to characterize the quantization error for the current pixel. As mentioned previously, $Q_{max}=1/(2K)$ should be the maximum quantization error.

To have a clearer understanding of the temporal-spatial binary encoding method, we illustrate the 1D encoding process with $K$=4 as an example, as shown in Figs. 1(d)-(e). Firstly, as the pixel intensity of the current pixel $p$ =(3,0.3087) (marked by the red dot in Fig. 1(d)) falls within the second intensity interval [0.25, 0.5), its intensity 0.3087 is compared with the quantization threshold $\epsilon _2 = 3/8$ (see Eq. (2)). Since 0.3087 is less than $\epsilon _2$, the corresponding pixel position in $B_2$ is assigned 0. Next, it is observed that intensity 0.3087 is greater than the first intensity interval and less than the third and fourth intervals. Therefore, the binarization results of the corresponding position $p$ in $B_1$, $B_3$, and $B_4$ are determined to be 1, 0, and 0 , respectively.

Generally, $K$=4 means that a sinusoidal pattern can be decomposed into four binary encoding patterns, denoted as $\{ {B_1},{B_2},{B_3},{B_4}\}$. The intensity range [0,1] of the sinusoidal pattern is divided into four intervals: [0-1/8, 1/4), [1/4, 1/2), [1/2, 3/4), and [3/4, 1+1/8)(the maximal quantization error $Q_{max}=1/8$). In fixed-threshold based temporal-spatial binary encoding methods [26], the thresholds for each interval are 1/8, 3/8, 5/8, and 7/8. For the Quantization error propagation and corrected image $I_s'$, its pixel values can be classified into four intervals based on its intensity values:

(1) Interval 1: For pixels $(u,v)$ with intensities within the first interval [0-1/8, 1/4): If $I_s'(u,v)$ is less than 0+1/40=1/40, ${B_1}(u,v)=0$. If $I_s'(u,v)$ is greater than 1/4-1/40=9/40, ${B_1}(u,v)=1$. If $I_s'(u,v)$ is within the range of [1/40, 9/40], the original threshold $\epsilon _k$ in Eq. (12) is set to 1/8. Subsequently, the dynamic quantization threshold $\hat {\epsilon }_k(u,v)$ can be calculated according to Sec 2.2. If $I_s'(u,v)$ is less than $\hat {\epsilon }_k(u,v)$, ${B_1}(u,v)=0$; if $I_s'(u,v)$ is greater than $\hat {\epsilon }_k(u,v)$, ${B_1}(u,v)=1$. The corresponding quantization error for pixels in this interval is:

(2) Interval 2: For pixels $(u,v)$ with intensities in the second interval [1/4, 1/2): If $I_s'(u,v)$ is less than 1/4+1/40=11/40, ${B_2}(u,v)=0$. If $I_s'(u,v)$ is greater than 1/2-1/40=19/40, ${B_2}(u,v)=1$. If $I_s'(u,v)$ is within the range of [11/40, 19/40], the original threshold $\epsilon _k$ in Eq. (12) is set to 3/8. Subsequently, the dynamic quantization threshold $\hat {\epsilon }_k(u,v)$ can be calculated according to Sec 2.2. If $I_s'(u,v)$ is less than $\hat {\epsilon }_k(u,v)$, ${B_2}(u,v)=0$; if $I_s'(u,v)$ is greater than $\hat {\epsilon }_k(u,v)$, ${B_2}(u,v)=1$. The corresponding quantization error for pixels in this interval is:

(3) Interval 3: For pixels $(u,v)$ with intensities in the third interval [1/2, 3/4): If $I_s'(u,v)$ is less than 1/2 + 1/40 = 21/40, ${B_3}(u,v)=0$. If $I_s'(u,v)$ is greater than 3/4 – 1/40 = 29/40, ${B_3}(u,v)=1$. If $I_s'(u,v)$ is within the range of [21/40, 29/40],the original threshold $\epsilon _k$ in Eq. (12) is set to 5/8. Subsequently, the dynamic quantization threshold $\hat {\epsilon }_k(u,v)$ can be calculated according to Sec 2.2. If $I_s'(u,v)$ is less than $\hat {\epsilon }_k(u,v)$, ${B_3}(u,v)=0$; if $I_s'(u,v)$ is greater than $\hat {\epsilon }_k(u,v)$, ${B_3}(u,v)=1$. The corresponding quantization error for pixels in this interval is:

(4) Interval 4: For pixels $(u,v)$ with intensities in the fourth interval [3/4, 1+1/8): If $I_s'(u,v)$ is less than 3/4 + 1/40 = 31/40, ${B_4}(u,v)=0$. If $I_s'(u,v)$ is greater than 1 – 1/40 = 39/40, ${B_4}(u,v)=1$. If $I_s'(u,v)$ is within the range of [31/40, 39/40], the original threshold $\epsilon _k$ in Eq. (12) is set to 7/8. Subsequently, the dynamic quantization threshold $\hat {\epsilon }_k(u,v)$ can be calculated according to Sec 2.2. If $I_s'(u,v)$ is less than $\hat {\epsilon }_k(u,v)$, ${B_4}(u,v)=0$; if $I_s'(u,v)$ is greater than $\hat {\epsilon }_k(u,v)$, ${B_4}(u,v)=1$. The corresponding quantization error for pixels in this interval is:

2.2 Dynamic threshold optimization

The binary encoding process has been introduced in section 2.1, and the principle of dynamic threshold optimization is described in the following equations.

The boundary values $A_{mmin}$, $A_{max}$, and intermediate values $A_{avg}$ for the $k$-th intensity interval $[(k-1)/K, k/K]$ are calculated by the following equation.

Comparing the pixel intensity value $I_s'(u,v)$ with the boundary values $A_{min}$ and $A_{max}$ of the $k$-th intensity interval $[(k-1)/K, k/K]$, and determining whether it is in the interval. For pixel intensity values that fall in the front 1/10 and back 1/10 of the current interval, the encoding errors are very small and binarization can be directly performed without considering the quantization threshold. That is, if $I_s'(u,v) \in [(k - 1)/K,A_{min}]$, the corresponding value is 0; if $I_s'(u,v) \in [A_{max},k/K]$, the corresponding value is 1. For pixel intensity values that fall in the middle 8/10 of the current interval [$A_{min},A_{max}$], dynamic threshold is used to perform the binarization, which can greatly reduce the errors generated by the encoding step, and then diffuse to the surroundings through error diffusion kernel along a path, and the overall error distribution is smaller and more uniform. The original quantization threshold of TSB method [26] is set to be the fixed value $\epsilon _k$, but here $\hat {\epsilon }_k$ is no longer a fixed value, but dynamically changing for the middle 8/10 subregion.

Equation (12) is used to calculate the dynamic threshold. It’s note that the original global threshold is the mid-point of each gray level interval. The weight $\gamma$ of the original global threshold $\epsilon _k$ in the Eq. (12) is determined by the simulation experiment.

The weight $\gamma$ takes values of 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, and 0.8 in the simulation experiments. Based on the different values of $\gamma$, the distribution of quantization thresholds is also different, which affects the phase result of temporal-spatial binary encoding. Using the root mean square error(RMSE) error of the phase of the temporal-spatial binary encoding image as the evaluation metric, then the best $\gamma$ value is selected for the dynamic threshold optimization algorithm.

In the simulation process, a size of $3 \times 3$ Gaussian filter with standard derivation 1 approximating the low-pass filtering of the nearly in-focus projector is used to blur the temporal-spatial binary encoding patterns, and the phase of the standard sinusoidal fringe is taken as the reference value. For $K$=2 and $K$=4, the simulation results are shown in Fig. 2. From the error distribution curves, it can be seen that the minimum phase error can be obtained for the majority of fringe periods when $\gamma$ = 0.4.

 

Fig. 2. The selection strategy of $\gamma$: (a) $K$ =2; (b) $K$ = 4.

Download Full Size | PDF

As shown in Fig. 3, when $K$=4, according to the dynamic threshold based temporal-spatial binary encoding process described above, a sinusoidal pattern is decomposed into four binary fringe patterns. Taking four-step phase shifting as an example, a total of 16 binary fringe patterns are generated, and these patterns are used for 3D shape measurements.

 

Fig. 3. When $K$=4, the phase calculation process of the four-step phase-shifting binary fringe patterns in the simulation experiment:(a)standard four-step sinusoidal fringe patterns; (b) the proposed temporal-spatial binary encoding patterns; (c) approximate sinusoidal fringe patterns by integral-imaging; (d)the wrapped phase and the absolute phase.

Download Full Size | PDF

In the simulation, $K$ binary fringe patterns are averaged to form a sinusoidal pattern by the integral imaging [26]. The four-step phase-shifting with phase shifts 0, $\pi$/2, $\pi$, and 3$\pi$/2 are used to obtained the wrapped phase, and the final absolute phase is calculated using the temporal phase unwrapping algorithm [4]. A $3 \times 3$ Gaussian filter with standard derivation 1 is used to simulate the point spread function(PSF) of a real measurement system.

3. Simulation experiment

To verify the effectiveness of the proposed method, four compared methods will be set up: (1) the spatial binary encoding method based on the Floyd-Steinberg error diffusion algorithm, referred to as the "FS" method; (2) the FS method with dynamic threshold strategy introduced in Sec 2.2, referred to as the "FSDT" method; (3) the traditional temporal-spatial binary encoding method, referred to as the "TSB" method [26]; (4) the proposed temporal-spatial binary encoding based on dynamic threshold, referred to as the "Proposed" method.

3.1 Dynamic threshold analysis and its impact on sinusoidal fringe patterns

The four binary encoding methods based on the Floyd-Steinberg error diffusion kernel were used in the comparative experiment. The experiment aimed to determine the impact of the quantization error correction on the quality of the resulting phase. It is noted that the feedback mechanism of the error diffusion algorithm maintains the energy of the original and encoded patterns, which contributes to the impact of quantization error correction. To further analyze the dynamic threshold change in relation to sinusoidal fringe patterns, a pattern with a resolution of 256$\times$256 pixels and a fringe period of P=64 was created. Fig. 4 displays the changes in original sinusoidal fringe pattern $I_s$ , the quantization error corrected pattern $I_s'$, and dynamic threshold $\epsilon _k$ for the four binary encoding methods used in the experiment.

 

Fig. 4. The 1D sinusoidal property of encoding patterns for different methods: (a)FS; (b)DT-FS; (c)TSB($K$=2); (d)TSB($K$=4); (e)Proposed($K$=2); (f)Proposed($K$=4).

Download Full Size | PDF

Figures 4(a),(c),and(e) represent the traditional encoding methods, while Figs. 4(b),(d),and (f) are the corresponding methods with dynamic threshold strategy. By calculating the gray-value average of each pixel position in the original image using Eq. (13), and then applying Eq. (12), the dynamic threshold of each pixel position can be obtained based on the local intensity average and weight $\gamma$. Taking $K$=1 as an example, the constant threshold of 0.5 is changed to a variable dynamic threshold as shown by the light blue curve in Fig. 4(a)(b). Similarly, when $K$=2,4, the threshold curves for TSB and Proposed methods are shown in Fig. 4(c)-(f), where constant thresholds are represented as a straight line and dynamic thresholds are represented as a changing curve. It can be seen that the FS method has the worst sinusoidal characteristic, while the TSB method improves the sinusoidal fringe pattern quality significantly by dividing the intensity range to multiple intervals. The improvement in $I_s'$ becomes more evident as $K$ increases.

3.2 Comparative results of phase accuracy

The spatial binary encoding methods FS and FSDT encode a sinusoidal fringe pattern that corresponds to a binary fringe pattern, while the temporal-spatial binary encoding methods TSB and Proposed decompose each sinusoidal fringe pattern into $K$ binary fringe. In this case, the number of projected patterns generated by four binary encoding methods(FS,FSDT,TSB,Proposed) are inconsistent. For fairy comparison, the strategy of generating binary fringe patterns by FS and FSDT methods with error matrix of Eq. (3) is shown in Fig. 5. The FS/FSDT encoding binary fringe pattern has an extended period, and the required phase-shifting patterns $B_k$ are cropped from it by a interval of $P$/4($P$ is the fringe period) as shown in Fig. 5. The spatially encoded fringe of FS/FSDT can be decomposed into $K$ binary patterns via shifting $\delta$ pixels vertical to the fringe arrangement direction (right arrow in Fig. 5), and next the resultant sinusoidal fringe pattern is formed by the integral imaging. To this point, for FS, FSDT, TSB and Proposed encoding methods, there are 16 binary fringe patterns when $K$ = 4 in four-step phase shifting algorithm.

 

Fig. 5. Strategy for generating $K$ binary patterns of two spatial encoding methods FS and FSDT.

Download Full Size | PDF

The procedure for comparative experiments can be described as follows:

The simulation results when $K$ = 2 and $K$ = 4 are shown in Fig. 6(a) and Fig. 6(b), respectively. Here, we could conclude the following:

 

Fig. 6. Comparison results of phase errors relative to the standard sinusoidal fringe pattern in simulation cases: phase error with fringe period $P \in [12,96]$pixels (a) when $K$=2;(b) when $K$=4.

Download Full Size | PDF

4. Real experiments

To provide further evidence supporting the superiority of our proposed encoding method for 3D measurement, we conducted experiments using the following experimental system shown in Fig. 7. The experimental setup consists of a MER2-131-210U3M camera (with a resolution of 1024$\times$1280 pixels and a maximum frame rate of 210Hz), a TI LightCrafter DLP4500 projector (with a resolution of 1140$\times$912 pixels and a maximum frame rate of 4225Hz for binary pattern projection), a PI M-414 precision linear electronic control transition stage (with a stroke of 300mm and 0.1$\mu$m minimum displacement), and a main control computer. The optimum work distance is about 750$mm$ away from the projector. The projector is used to in-focus project the binary encoding fringe pattern sequences, while the camera synchronously captures the deformed fringe images using integral imaging [26], that employs a circuit to ensure the synchronization.

 

Fig. 7. Experimental setup and the defocused state of the projector.(a) experimental setup; (b) the projector is fully focused; (c) the projector is nearly focused; (d) the difference in intensity change of the orange line indicated in Fig. 7(b) and Fig. 7(c).

Download Full Size | PDF

To retrieve the absolute phase map, the proposed method in Sec.2.1 has been used to encode three sets of computer-generated standard sinusoidal fringe patterns with fringe period $P=16$pixels. Each sinusoidal fringe pattern is decomposed into $K$ (2 and 4) binary fringe patterns. In order to ensure the fairness of comparison, FS, FSDT, TSB and proposed method, we use Floyd-Steinberg diffusion kernel(see Eq. (3)) for quantitative error allocation. Through simulation, plane and static object experiments, our proposed method demonstrates superiority in accuracy performance compared to other methods. To verify the advantages binary fringes in high-speed projection measurement, the method proposed in this paper is also applied to a dynamic object measurement.

4.1 Plane experiment

Plane experiment is conducted to verify the measurement advantages of the proposed method in nearly focused state in a real measurement scenario. The experiment involved comparing the phase error and reconstruction accuracy of a measured plane under different binary patterns projection. In our experimental system (Fig. 7), the camera is focused, while the projector is in a near-focused state. Figure 7(b) shows that when the projector is fully focused, the black and white boundary lines are jagged, while in a near-focused state, there is a slight gray-level transition at the black and white boundary (see Fig. 7(c)). Figure 7(d) shows the difference of intensity changes at the black and white edges between the fully focused and nearly focused states.

Six planes at different positions are calibrated for phase-height mapping [28] in order to reconstruct the plane and object in subsequent steps. The phase RMSE errors and reconstruction accuracy of different methods, including FS, FSDT, TSB, and the proposed binary fringe encoding methods, are calculated. The wrapped phase is obtained by the four-step phase-shifting algorithm, and multi-frequency method is used for absolute phase retrieval.

Figures 8(a)-(d) show the captured deformed fringe images of the plane, where the 600th row of the captured plane images under pattern illumination using four different encoding methods are drawn in red plots. And the reconstruction result of 16-step phase-shifting sinusoidal fringes in Fig. 8(e) is used for phase RMSE error comparison and reconstruction accuracy comparison.

 

Fig. 8. The fringes images captured by the camera for patterns projection of (a)FS, (b)FSDT, (c)TSB, (d)Proposed; (e)The reconstruction result of 16-step phase-shifting sinusoidal fringe patterns.

Download Full Size | PDF

 

Fig. 9. Experimental comparison results of phase errors of binary encoding fringe patterns in nearly-focus state for plane measurement: phase error with fringe period $P=12,\ldots,96$ (a) when K=2;(b) when K=4.

Download Full Size | PDF

The root mean square error ${\textrm{RMSE}} = \sqrt {\sum \nolimits _{j = 1}^M {\left ( {{h_j}\left ( {u,v} \right ) - {G_j}\left ( {u,v} \right )} \right )} } /M$ and maximum error ${\textrm{ME}} = {\textrm{max}}\left ( {\left | {{h_j}\left ( {u,v} \right ) - {G_j}\left ( {u,v} \right )} \right |} \right )$ are applied to evaluate the phase error and reconstruction accuracy, where $M$ represents the number of measurement points, $j = 1,\;2,\;3 \ldots M$ represents the index of measurement points, ${h_j}\left ( {u,v} \right )$ is the measured value, and the reconstruction result of 16-step phase-shifting sinusoidal fringes is used as the ground-truth ${G_j}\left ( {u,v} \right )$ .

Figure 10 displays the plane reconstruction results and regional reconstruction error statistics (RMSEs and MAEs) of the four binary encoding methods when $K$ = 2 and $K$ = 4, respectively. It is observed that increasing the number of projected images (i.e., from $K$ = 2 to $K$ = 4) leads to an improvement in the reconstruction accuracy of the four methods. From Fig. 10, regardless of whether $K$ is 2 or 4, the proposed method outperforms the other methods in terms of reconstruction accuracy. The MAE(when $K$=2) of the four methods (FS, FSDT, TSB, and proposed) are 0.4720mm, 0.1999mm, 0.2466mm, and 0.1726mm, respectively. Compared with the first three methods, the measurement accuracy of the proposed method has been improved by 63.43%, 13.66%, and 30.00%, respectively. The MAE(when $K$=4) of the four methods (FS, FSDT, TSB, and proposed) are 0.4248mm, 0.1604mm, 0.1445mm and 0.1185mm, respectively. Compared with the first three methods, the measurement accuracy of the proposed method has been improved by 72.22%, 26.12% and 17.99%, respectively. This plane experiment result verifies that the proposed method can significantly improve the reconstruction quality of the compared binary encoding methods even under nearly focused conditions.

 

Fig. 10. The results of different encoding methods for plane reconstruction when $K$=2 and $K$=4, as well as the difference map with the reconstruction result of sinusoidal fringe: (a) FS, (b) FSDT, (c) TSB, and (d) the proposed method.

Download Full Size | PDF

Figure 11 provides a detailed reconstruction plot of the four methods when $K$ is 2 and 4, which is marked as red lines in Fig. 10. As can be seen from the figure, the error of FS method is obviously higher than that of the other three methods. Nonetheless, the proposed method has the smallest reconstruction error, which is consistent with simulation experimental results. Accordingly, the plane experiment supports the effectiveness of the proposed method in improving reconstruction accuracy in the nearly focused state, especially for the small fringe period ($P$ = 16 pixels) utilized in the experiment.

 

Fig. 11. The reconstruction errors of the marked line for the four binary encoding methods.

Download Full Size | PDF

4.2 Static object experiment

The binary encoding fringes ($P$=16pixel) generated by the FS, FSDT, TSB and proposed methods are projected for reconstructing a complex object. The collected object images are fed into the four-step phase-shifting algorithm and the absolute phase maps are obtained. To quantitatively evaluate the accuracy performance of the above methods, the reconstruction result of 16-step phase-shifting sinusoidal fringes is used for comparison, as shown in Fig. 12(e), which has smooth reconstruction surface, complete and rich details, even the clearly visible eyelashes.

 

Fig. 12. The fringes images captured by the camera for patterns projection of (a)FS, (b)FSDT, (c)TSB, (d)Proposed; (e)The reconstruction result of 16-step phase-shifting sinusoidal fringe patterns.

Download Full Size | PDF

Figures 12(a)-(d) show the captured deformed fringe images of the mask (one of the three groups of phase-shifting fringe images), and their 3D reconstruction results are displayed in Fig. 13 ($K$=2 and $K$=4), respectively. Based on the smoothness and rich detail of the reconstructed surface, the proposed method performs better than the other methods in terms of reconstruction accuracy.

 

Fig. 13. The results of different encoding methods for static object reconstruction when $K$=2 and $K$=4, as well as the difference map with the reconstruction result of sinusoidal fringe: (a) FS, (b) FSDT, (c) TSB, and (d) the proposed method.

Download Full Size | PDF

When $K$=2, according to the reconstruction results inside the green rectangular box on the left side of Fig. 13, the reconstruction accuracy is FS < TSB < FSDT < proposed method. From the visualization results of the reconstruction error , the trend of the two error metrics (MAE and RMSE) are consistent with the simulation experiment and plane experiment mentioned above. Taking MAE as an example, the reconstruction errors of FS, FSDT, TSB, and the proposed binary coding method are 0.5119mm, 0.2180mm, 0.2776mm, and 0.1906mm, respectively. Compared with the first three methods, the measurement accuracy of the proposed method is increased by 62.77%, 12.57% and 31.34% respectively.

Similarly, the reconstruction results of $K$ = 4 in Fig. 13 demonstrate that the reconstruction accuracy of the four methods satisfies: FS < TSB < FSDT < Proposed. The reconstruction MAE errors of FS, FSDT, TSB and Proposed binary encoding methods are 0.5032mm, 0.1723mm, 0.1164mm and 0.1058mm, respectively, showing a decreasing trend. Compared with the first three methods, the measurement accuracy of Proposed method is 78.97%, 38.49% and 9.10%, respectively.

When $K$=2 and $K$=4, the error distributions of 390th column in the reconstructed results of the marked red are shown in Figs. 14(a)-(b), respectively. The results in Fig. 13 and Fig. 14 show that the proposed method has the highest reconstruction accuracy. The given 3D reconstruction results (point cloud) are rendered for better observation. It is worth noting that these 3D results have not undergone any post-processing, such as interpolation or smoothing.

 

Fig. 14. The reconstruction errors of the marked line for the four binary encoding methods.

Download Full Size | PDF

The next experiment is conducted to evaluate the 3D reconstruction accuracy of the proposed binary encoding patterns. The dumbbell gauge (center distance is 100mm, A and B sphere diameters are 50mm, respectively) are placed at the working distance($\sim$780mm) corresponding to a nearly-focused projection. The absolute phase maps are obtained by a three-frequency temporal phase unwrapping. Figure 15 shows the the phase map error curves, using the unwrapped phase map of the 16-step sinusoidal fringe as a reference to evaluate the phase quality of different kinds of binary encoding patterns. Figures 15(a) and 15(b) are one of the phase-shifting fringe images and the phase maps. Figure 15(c) is the crossed sections of the phase error distribution of different methods. The results show that the proposed method has the lowest phase error. Figure 15(d) shows the standard deviation statistics of the phase error curves.

 

Fig. 15. Phase distributions obtained from the standard spheres using different binary encoding patterns. (a) One of the phase-shifted fringes; (b) one of the absolute phase map; (c) phase error curves of the red dashed line in Fig. 15(b); (d) standard deviation statistics.

Download Full Size | PDF

Using the system calibration parameters, the phase maps of two spheres are further converted into 3-D surface shapes. Figure 16 shows the 3D reconstruction results of the spheres using different binary coding patterns. The reconstructed results shown in Figs. 16(a)-(b) suffer from serious errors, and the reconstructed result of TSB in Fig. 16(c) is relatively acceptable, while the reconstructed surface using the proposed binary encoding fringes retrieves the smoothest surface as shown in Fig. 16(d). For further evaluation of the reconstruction accuracy, four sets of reconstruction spheres are fitted, and the diameter deviation and surface errors are calculated. The RMSEs of A and B sphere diameters obtained by the proposed method are approximately 0.0238mm and 0.0251mm, respectively. Table 1 shows the RMSEs of sphere A, sphere B and center distance by using the different binary encoding patterns. The RMSE of the proposed method is the smallest among all the binary encoding algorithms.

 

Fig. 16. Reconstructed results of spheres by using four binary encoding methods. (a) FS; (b) FSDT; (c) TSB; (d) Proposed.

Download Full Size | PDF

Table 1. Reconstruction accuracy of dumbbell gauge(Units:mm)

View Table

4.3 Dynamic object experiment

Binary encoding pattern utilizing the high-speed pattern switching rate of DMD-based projector could be used as the candidate for fast even high-speed 3D measurement. In order to verify that the proposed method can be used for dynamic object measurement, a 3D measurement experiment of a moving palm is conducted. Three-step phase-shifting algorithm with two-frequency (37 and 38) heterodyne phase unwrapping is adopted to retrieve the absolute phase. Since the resolution of the projector is 1140$\times$912 pixels, the fringe periods are $P_1 = 912/37$ pixels and $P_2 = 912/38 = 24$ pixels, respectively. The binary fringe patterns with $P_1$ are generated using the FS method, while the higher frequency binary fringe patterns with $P_2$ are generated using the comparative four methods when $K$ = 2. By using the integral imaging strategy [26], 6 binary fringe images are required for one reconstruction result, and the object’s reconstruction frame rate is approximately 200/6 $\approx$ 22Hz. Figure 17 displays several key frames in the dynamic objects measurement process, with the first row representing the captured deformed fringe images and the second row as the reconstruction results.

 

Fig. 17. Several key frames of the moving palm(Visualization 1).

Download Full Size | PDF

5. Conclusion and future work

This paper proposes a temporal-spatial binary encoding method based on dynamic threshold optimization, which is not affected by the nonlinearity of the projector and can be used for fast and high-precision 3D shape measurement while maintaining the measurement depth range in a nearly focused projection. Computer simulation and objects experiments show that our proposed encoding method is far more accurate than the compared spatial binary encoding and temporal-spatial binary encoding methods. Experimental results using a mask with complex surface features verify that the proposed method enjoys an accurate 3D reconstruction capability. By synchronizing the camera and the projector to achieve fast and accurate 3D measurement, generating sinusoidal fringes through integral imaging, the captured image quantity and final 3D reconstruction almost approximates the traditional sinusoidal fringe patterns. The application prospect of this method in dynamic measurement is verified by a moving palm. In the future work, we will focus on improving the efficiency of phase unwrapping and studying binary encoding methods that incorporate both diffusion kernel and threshold optimization. This will enable the encoding method to achieve more accurate and faster 3D reconstruction.