1. Introduction

A basic binocular vision sensor (BVS) consists of two cameras. Depending on optical triangulation method and stereo parallax, BVS aims at completing the three-dimensional measurement of feature points, feature lines, etc. The binocular stereo vision measurement is non-contact, fast, good flexibility, high precision, and widely applies to not only the 3D model reconstruction but also the measurement of 2D profile, 3D topography and key geometric parameters of three-dimensional objects, etc. [1].

Calibrating the measurement model parameters is the key to the successful application of BVS. It means the complete solution of the intrinsic parameters of cameras and structure parameters of binocular vision system. The intrinsic parameters of cameras do not change with the structure of BVS, and they are suitable for offline calibration. The structure parameters of binocular system are different. They are vulnerable to the impact of the installation process and need to be calibrated onsite.

At present, the calibration methods for BVS mainly include: ①Calibration method based on the 3D target with known three-dimensional coordinates [2]; ②Using the unknown motion 2D round cavity target for calibrating BVS [3]; ③Calibration technique based on unknown movement of one-dimensional target [4–6 ]; ④Self-calibration method of BVS based on characters matching [7]. The method using three-dimensional target can get calibration images of high quality only when the target is at specific locations due to the mutual effect of illumination on different planes. Moreover, the three-dimensional target is difficult processing and high manufacturing cost. In the method based on the unknown motion of 2D round cavity target, the calculation is an iterative process of solving nonlinear equations with large amount of data course and complex computation. The method based on one-dimensional target of unknown motion, although high precision and easy to conduct, but the solution requires numerous matrix transforms and iterative processes of solving nonlinear equations for roots, and generates high computational complexity and calculation error. For the self-calibration method of BVS based on features matching, accurate extraction of feature points and exact matching are indispensable to achieve precise calibration. It is very difficult to guarantee in the industrial site with complex environment. In addition, there are some other calibration methods. Li et al. [8] proposed a calibration method for binocular vision sensor based on BP neural network, but did not involve the calibration accuracy. Ma et al. [9] proposed a self-calibration method for binocular active vision, but it required the presence of pure translation between two cameras. R. Hartley and A. Zisserman [10] put forward a solution of solving the rotation matrix between two cameras by using vanishing points, but did not mention the method of calculating the translation vector. J. Bouguet [11] provides a calibration toolbox for Matlab based on planar chessboard targets, which is most widely used.

In the real life, parallel lines are ubiquitous, such as the airport runway, guardrail, zebra crossing and so on. Motivated by the previous work [12–14 ], a novel calibration method of BVS using parallel lines is proposed in this article. A planar target with more than three equally spaced parallel lines is used in the method. In the measurement space, the target is placed freely n times and the images of target are captured by two cameras. The vanishing line of the target plane can be determined from the projections of parallel lines. Combining the known intrinsic parameters of cameras, the target plane’s orientation relative to the camera can be calculated. Two normal vectors of one target plane relative to corresponding camera are related by a rotation matrix. Meanwhile, the rotation matrix is the structure parameters of BVS R. As the target is placed n times, n constraints are obtained to solve the rotation vector. Since the distance D of each two adjacent parallel lines has been known exactly, the translation vector is determined. Finally, it is the overall optimization [15,16 ]. Because of the demands of vanishing lines in solving the translation vector T, we choose vanishing lines instead of vanishing points to calculate the rotation matrix R.

The paper is organized as follows. Section 2 describes the measurement model of BVS. Section 3 studies the calibration principle. Section 4 discusses the accuracy analysis and experiments: We first derive mathematical formulas consequence in the qualitative analysis of the various factors’ impact on the calibration results. Then Quantitative results are displayed through computer simulation. Finally, it is the real data. Section 5 describes the conclusion. In the Appendix, we provide the technique for estimating the vanishing line when the scene is a set of coplanar equally spaced parallel lines.

2. Measurement model

One arbitrary spatial point mapping in the image can be approximated by the usual pin-hole model. The measurement model of BVS is illustrated in Fig. 1 . The world coordinate frame (WCF) defined as $O-XYZ$ is the same with the left camera coordinate frame (LCCF). The right camera coordinate frame (RCCF) is $O_c^r-X_c^r{Y}_c^r{Z}_c^r$. The image coordinate frames of the left camera and right camera are $o_l-u_l{v}_l$ and $o_r-u_r{v}_r$. Let $m_l={(u_l,v_l,1)}^T$ and $m_r={(u_r,v_r,1)}^T$ be the image coordinates of the spatial point $M{\left(X,Y,Z,1\right)}^T$ mapped by the left camera and right camera respectively.

 

Fig. 1 Measurement model of BVS.

Download Full Size | PDF

Denote the intrinsic matrixes of the left camera and right camera by $A_l$ and $A_r$, and the projection matrix by $P_l$ and $P_r$. The rotation matrix $R$ and translation vector $T$ between LCCF and RCCF are expressed as:

The values of $X,Y,Z$ can be calculated by solving Eq. (3) though the linear least-squares method. Then we reconstruct the three-dimensional spatial coordinate of point $M$.

3. Principle

In this method, the calibration of BVS is based on the vanishing features of parallel lines. The calibration model of BVS is shown in Fig. 2 .The calibration method can be carried out in the following main steps:

 

Fig. 2 Calibration model of BVS.

Download Full Size | PDF

3.1. Related concepts and properties

The followings are some concepts and properties connected with the proposed calibration method in this article [10].

(Property 1) Points at infinity: In $I{P}^2$, consider two lines $ax+by+c=0$ and $ax+by+c=0$.They are represented by vectors $l={(a,b,c)}^T$ and $l={(a,b,c)}^T$ for which the first two coordinates are the same. Computing the intersection of these lines gives no difficulty. The intersection is $x=l\times l=(c-c){(b,-a,0)}^T$,and ignoring the scale factor $(c-c)$, this is the point ${(b,-a,0)}^T$. The points with the same form of $x$ are known as points at infinity. Now if we attempt to fine the inhomogeneous representation of this point, we obtain ${(b/0,a/0)}^T$, which makes no sense, except to suggest that the point of intersection has infinite large coordinates. This observation agrees with the usual ideal that parallel lines meet at infinity, and a set of parallel lines meet at the same point at infinity.

(Property 2) Vanishing lines: The vanishing line is constructed, as illustrated in Fig. 3 , by intersecting the image with a plane parallel to the scene plane through the camera center $C$. It is clear that a vanishing line depends only on the orientation of the scene plane, not depends on its position. Under the camera coordinate frame (CCF), the relation between the direction vector $n$ of the scene plane and the vanishing line $l$ can be defined as the following property:

 

Fig. 3 Vanishing line formation.

Download Full Size | PDF

(Property 3) Back-projection plane: A set of points in space which map to a line in the image is a plane in space defined by the camera center and image line mapped by a 3-space line in space, as shown in Fig. 4 . This plane is known as the back-projection plane. It is obvious that the 3-space line lies on the back-projection plane. If a 3-space line is mapped under two cameras, we will obtain two back-projection planes whose intersection is the 3-space line. Under WCF, the following mathematical formulations stand:

 

Fig. 4 Back-projection of lines.

Download Full Size | PDF

3.2. Acquisition of the rotation matrix

As illustrated in Fig. 5 , $l_1,l_2,l_3,l_4$ and $r_1,r_2,r_3,r_4$ are projections of lines $L_1,L_2,L_3,L_4$ on the target plane. In the target images, the central lines of the parallel lines are picked up by Steger’s method [18], followed by the linking of lines based on the edge connection algorithm proposed by Peter Kovesi [19]. The recognition and matching of lines use the information of their positions. Equations of $l_1\sim l_4$ and $r_1\sim r_4$ expressed under homogeneous coordinate are obtained by the method of least squares fitting for the picked up points on the straight lines, meaning $l_i={(l_i)}_{3\times 1}$ and $r_i={(r_i)}_{3\times 1}$. According to the Appendix, the vanishing lines $l_i$, $l_r$ of the target plane under LCCF and RCCF can be determined by lines $l_i$, $r_i$ ($i=$1, 2, 3, 4).

 

Fig. 5 The diagram of target images.

Download Full Size | PDF

Since $A_l^{}$, $A_r^{}$ acting as the intrinsic matrixes of the left camera and right camera are known, the direction vectors $n_l$ and $n_r$of target plane under LCCF and RCCF satisfy:

Owing to the fact that both $n_l$ and $n_r$ belong to the same target plane just in different coordinate frames, the following equation stands:

If the target is placed n times, n equations similar to Eq. (8) stand. As an orthogonal matrix, the rotation matrix $R$ only has three independent variables. The rotation matrix $R$ is determined when $n\ge 2$ _.

3.3. Acquisition of the translation vector

Let ${\pi }_{li}$ and ${\pi }_{ri}$ be the back-projection planes under WCF defined by lines $l_i$ and $r_i$ on the image plane ($T_{3\times 1}$1, 2, 3, 4). According to Eq. (5), the following results stand:

The 3-space straight line $L_i,$ can be represented as a dual Plücker matrix $L_i^{*}$ and satisfies the following property according to Eq. (6).

Let $L_i^{*}$ be expressed as $L_i^{*}=\left[\begin{array}{cc}{A}_i& B_i^{}T\\ -T^T{B}_i^T& 0\end{array}\right]$, where $A_i=A_l^T{l}_i{r}_i^T{A}_{r}R-R^T{A}_r^T{r}_i{l}_i^T{A}_l$, $B_i=A_l^T{l}_i{r}_i^T{A}_r$($i=$1, 2, 3, 4). It is clear that $S_1$ and $S_2$ are independent of $S_3$. As an antisymmetric matrix, $A_i$ meets $A_i=-A_i^T$ and corresponds to a $3\times 1$ vector $a_i$. It means $A_i={\left[a_i\right]}_{\times }$, where $a_i={\left[-a_{23}{a}_{13}-a_{12}\right]}^T$ and $L_i$ is the element of matrix $A_i$ in the position of column m and row n.

Since $L_1,L_2,L_3,L_4$ are parallel to each other, they have a common point at infinity defined as $v={\left[\begin{array}{cc}\stackrel{-T}{v_{3\times 1}}& 0\end{array}\right]}^T$ under WCF, where $\Vert \overline{v}\Vert =1$. As the point at infinity is also on lines $L_1,L_2,L_3,L_4$ the following equation stands:

The unit vector $\overline{v}$ is determined by solving the equation $\left[\begin{array}{c}{A}_1\\ \begin{array}{c}{A}_2\\ A_3\\ A_4\end{array}\end{array}\right]\overline{v}=0$. Furthermore, $\overline{v}$ represents the orientation of lines $L_1,L_2,L_3,L_4$ under WCF.

Let $S_i$ be the plane passing through the 3-space line $L_i$ and perpendicular to the target plane ${П}_t$, where $S_i=\left[\begin{array}{l}{n}_i\\ d_i\end{array}\right]$, ${П}_t=\left[\begin{array}{l}{n}_t\\ k\end{array}\right]$, $n_s=\overline{v}\times n_t$, as shown in Fig. 6 . Here, $n_t$ is the unit normal vector of the target plane under WCF and has been calculated in Sect.3.2 when solving the rotation matrix $R$. The Plüker matrix representation $L_i$ can be obtained from the dual Plüker matrix representation $L_i^{*}$ by a simple rewrite rule. The rewrite result is $L_i=\left[\begin{array}{cc}{\left[-B_{i}T\right]}_{\times }& -a_i\\ a_i^T& 0\end{array}\right]$ ($i=$1, 2, 3, 4). Since line $L_i$ is on both plane $S_i$ and plane ${П}_t$, it satisfies the following equation.

 

Fig. 6 The target plane ${П}_t$ vs. planes $S_1,S_2,S_3,S_4$.

Download Full Size | PDF

As $L_1,L_2,L_3,L_4$ are parallel to each other and spacing of $D$, the equation $d_i=d_1-(i-1)\times D$ stands where $i=$1, 2, 3, 4. In Eq. (13), there are 5 unknown parameters ($T_{3\times 1},d_1,k$) and 4 constraints. Moving the target once increases 4 constraints and 2 unknown parameters (the new $d_1$ and $k$). If the target is placed n times independently, there are $P=4n$ constraints and $Q=2n+3$ unknown parameters. The translation vector $T_{3\times 1}$ is determined when $P\ge Q$ (meaning $n\ge 2$).

3.4. Optimization

Let $\pi ={\left[\begin{array}{cc}\stackrel{-T}{v}& a\end{array}\right]}^T$ represent the plane perpendicular to lines $L_1,L_2,L_3,L_4$ where a is a constant value of 0. The intersection between the 3-space line $L_i$ and plane $\pi$ is defined as $X_i$, then $X_i=L_i\pi$. As the parallel lines are spacing of $D$, the equation $\Vert x_{i+1}-x_i\Vert =D$ stands, where $x_i$ is the non-homogeneous coordinates of $X_i$ ($i=$1, 2, 3, 4……).

Suppose the number of equally spaced coplanar parallel lines is m and the target is placed n times, the following optimization function is established:

Considering the principle of error distribution, ${\rho }_1$ takes for 0.1 and ${\rho }_2$ takes for 10. Aimed at a stable numerical solution, the orthogonal rotation matrix $R$ is translated into a Rodriguez vector $r={\left(r_x,r_y,r_z\right)}^T$ [20]. Then the number of unknown parameters is six. The Eq. (14) is solved by Levenberg-Marquardt method. Owing to the excellence of the initial value calculated in Sect.3.2 and Sect.3.3, global optimization can be obtained via a few iterations. Thus both ensures the accuracy of the results and improves the computing speed.

4. Analysis and experiment

In this section, the factors on which the calibration accuracy depends are discussed by means of mathematical formulations, followed by computer simulation to analyze the impact of these factors on the calibration accuracy. Finally, it is the real data which contains contrast experiments and accuracy evaluation.

4.1 Accuracy analysis

Since the method applies vanishing lines to complete calibration, the accuracy of vanishing lines so directly determines the accuracy of the calibration. Here, the factors that affect the calculation of vanishing line are discussed. As described in the Appendix, the vanishing line is determined by the images of several equally spaced parallel lines. Let $l_0={(a_0,-1,b_0)}^T$ stand for the first line and $l_n^{}={(a_n,-1,b_n)}^T$ stand for the (n + 1)-th line of the parallel lines, where $a_0$ and $a_n$ are the slops, $b_0$ and $b_n$ are the intercepts of the lines. Then, the vanishing line $l={(\alpha ,-1,\beta )}^T$ and $l_0$, $l_n$ are related by:

Let $\Delta a_0$, $\Delta b_0$, $\Delta a_n$, $\Delta b_n$ be the errors caused by noise, and $\Delta \alpha$, $\Delta \beta$ be variations of the vanishing line. The error propagations for $\alpha$ and $\beta$ are given by [21]:

From Eq. (18) and (19) , it can be seen that the variations of the vanishing line have inversely proportional relationship with $\left|b_0-b_n\right|$ and $\left|a_0-a_n\right|$ .The larger $\left|b_0-b_n\right|$ and $\left|a_0-a_n\right|$, the larger the variations of the vanishing line. In the following, two factors that affect the slop difference and the intercept difference of lines on the image are discussed: the angle $\theta$ between the target plane and the image plane, the distance $D$ of the parallel lines.

Let $M$ be an arbitrary point with homogeneous coordinate ${(x_w,y_w,z_w,1)}^T$ under WCF. The homogeneous coordinate of its projection on the image plane is $P={(u_p,v_p,1)}^T$. The relationship between the 3-space point $M$ and its image projection $P$ is given by:

Without loss of generality, we assume the target plane is on $z_w=0$ of WCF. Let us denote the i-th column of the rotation matrix $R_w^c$ as $r_w^{ci}$.

For simplicity, the relationship between CCF and WCF is illustrated in Fig. 7 . The camera coordinate frame moves $t_z$ along $z_c$ and rotates around $y_c$ with angle $\theta$ in a clock-wise direction, and it is the WCF. As the target plane is on $z_w=0$ of the world coordinate frame, $\theta$ is also the angle between the target plane and image plane.

 

Fig. 7 The position of the planar target.

Download Full Size | PDF

The projective transformation of points is shown as Eq. (21), so the projective transformation of lines is:

Let $l_0^{\text{'}}={(a,b,0)}^T$ be the first line on the target plane and $l_n^{\text{'}}={(a,b,nD)}^T$ be the (n + 1)-th line on the target plane, where $D$ is the distance of parallel lines. So their homogeneous coordinates in the image plane are:

From Eq. (27) and (28) , it is obvious that $\left|b_0-b_n\right|$ and $\left|a_0-a_n\right|$ associate with not only the angle between the target plane and the image plane but also the distance of parallel lines. Furthermore, $\left|b_0-b_n\right|$ and $\left|a_0-a_n\right|$ have positive relationships with $\theta$ and $D$.

Based on Eq. (18), (19), (27) and (28) , we know factors affecting the localization of vanishing line. The calibration accuracy can be improved by enlarging the space of the parallel lines or aggrandizing the angle between the target plane and the image plane.

4.2. Computer simulation

As discussed in the last subsection, the calibration accuracy of BVS can be affected by various factors. In this subsection, the calibration accuracy w.r.t. four main factors are further analyzed through computer simulations: 1) the noise level $0^o$ of images; 2) the number ${35}^o$ of the parallel lines; 3) the distance $D$ of parallel lines on the target; 4) the angle $\theta$ between the target plane and the image plane.

The intrinsic parameters of the simulated cameras are shown in Table 1 . The distortions of cameras are ignored. The rotation vector $R={\left(r_x,r_y,r_z\right)}^T$ and translation vector $T={\left(t_1,t_2,t_3\right)}^T$ between the camera coordinate frames of the two cameras is fixed as $R={\left(0.035,0.800,0.097\right)}^T$ and $T={\left(-350.0,-20.0,150.0\right)}^T$, respectively. Here, the rotation vector $R$ is expressed as a $3\times 1$ Rodriguez vector. The working distance of cameras is 700mm with the view field of $300\times 300mm$. A planar target with a set of equally spaced parallel lines is simulated as the model plane. Each line on the target is emulated by 100 points, and the target is placed 9 times randomly in front of the virtual cameras. The calibration accuracy is expressed by the simulation errors of $R$ and $T$. Here, both $R$ and $T$ are in the form of $3\times 1$ vectors. The 2-norm of vectors’ differences between the real data and the simulation results represent the absolute error. Divided by the true values, the relative standard deviation defined as the calibration accuracy is determined.

Table 1. Intrinsic parameters of cameras

View Table | View all tables in this article

4.2.1 Performance w.r.t. the noise level, the number and distance of parallel lines

Gaussian noise with 0 mean and $\sigma (0.05\sim 0.6)$standard deviation is added to the image points. For each value of parallel lines’ number $N$(3, 5, 7) and different distance $D$(15mm, 20mm, 25mm, 30mm), 100 independent trials are performed to compute the relative standard deviation of $R$ and $T$. We take the average value as the final result.

From Figs. 8, 9 and 10 , it can be seen that the calibration errors of $R$ and $T$ increase when the noise level improves and decrease when the space of the parallel lines shrinks. As illustrated in Fig. 11 , the calibration performance will be ameliorated by means of adding the number of parallel lines. When $\sigma =0.2$, $T$, $D=25mm$, the calibration accuracies of $R$ and $T$ are 1.8‰ and 0.5‰.

 

Fig. 8 Relative standard deviation vs. the noise level when $N=3$ and $D$ = 15mm, 20mm, 25mm, 30mm.

Download Full Size | PDF

 

Fig. 9 Relative standard deviation vs. the noise level when $N=5$ and $D$ = 15mm, 20mm, 25mm, 30mm.

Download Full Size | PDF

 

Fig. 10 Relative standard deviation vs. the noise level when $N=7$ and $D$ = 15mm, 20mm, 25mm, 30mm.

Download Full Size | PDF

 

Fig. 11 Relative standard deviation vs. the space of parallel lines when $\sigma =0.2$ and $N$ = 3, 5, 7.

Download Full Size | PDF

4.2.2 Performance w.r.t. angle between the target plane and the image plane

Gaussian noise with 0 mean and 0.2 standard deviation is added to the image points. The number of parallel lines is 7 and the space is 16.5mm. The angle $\theta$ varies from $o_r-u_r{v}_r$ to $m_l(u_l,v_l)$. For each value of $\theta$, 100 trails are performed. We take the average as the final result.

From Fig. 12 , it can be seen that the calibration error of $R$ first increases and then decreases with the increasing of angle $\theta$. The calibration error of $R$ takes its smallest value when $\theta ={25}^{\circ }$. The calibration error of $T$ decreases monotony when the angle $\theta$ increases. Considering the calibration errors of both $R$ and $T$, it is the most ideal when the angle $\theta$ is ${30}^{\circ }$. At the same time, the calibration accuracy of $R$, $T$ is 1.5‰, 0.8‰.

 

Fig. 12 Relative standard deviation vs. the angle of the target plane w.r.t. the image plane.

Download Full Size | PDF

4.3 Real data

For the experiment with real data, the binocular vision system is composed of two AVT-F504B cameras whose resolutions are $\text{1600×1200}$ pixel. The working distance of cameras is about 700mm~900mm and the view field is $\text{300×300}$ mm. The physical system is shown in Fig. 13 .

 

Fig. 13 The physical system.

Download Full Size | PDF

4.3.1 Intrinsic parameter calibration

The intrinsic parameters of the left camera and right camera are calibrated by Bouguet toolbox [11] which is compiled on Zhang’s calibration method [17]. The planar target we use is a chessboard target with $10\times 10$ corner points evenly distributed. The distance between the adjacent points is 10mm in the horizontal and the vertical directions with the accuracy of 5um. In the experiment, the image pairs used for calibration are taken from 13 different orientations by the left and right camera simultaneously. One pair of them is shown in Fig. 14 . The calibration results of cameras’ intrinsic parameters are shown in Table 2 .

 

Fig. 14 A sample of image pairs used for calibration: (a) left image; (b) right image.

Download Full Size | PDF

Table 2. Calibration results of cameras’ intrinsic parameters

View Table | View all tables in this article

4.3.2 Structure parameter calibration

A planar target comprising 10 equally spaced parallel lines with distance of 10mm and accuracy of 0.02mm is used in our proposed calibration method of BVS (see Fig. 13). The target is placed 13 times independently. The angle between the normal direction of the target plane and the optical axis of cameras ranges from ${15}^{\circ }$ to ${45}^{\circ }$. Both the left camera and right camera capture 13 images of the target. One pair of target images and extracted lines are illustrated in Fig. 15 .

 

Fig. 15 (a) Target images; (b) Extracted lines.

Download Full Size | PDF

As a comparison, the method using functions of the Bouguet toolbox [11], and known length based method [4] are also carried out to obtain the structure parameters. In the method of Bouguet toolbox, the image pairs of chessboard target are captured from more than 3 different orientations. The initial intrinsic parameters and the relationship between the camera coordination system and the target coordination system are obtained first. The initial structure parameters are acquired from coordinate transformation with the intermediate coordination system of target. An overall optimization of intrinsic parameters and structure parameters based on reprojection errors of feature points is the final procedure. It is obvious that we cannot calibrate the intrinsic parameters and structure parameters of BVS dividedly. Hence, the structure parameters obtained by Bougued toolbox are calibrated together with the intrinsic parameters using the pairs of images shown in Fig. 14. In the method based on known length, the structure parameters are calibrated by the same pairs of images with Bouguet toolbox for uniformity. The known length on the chessboard target we choose is shown in Fig. 16 .

 

Fig. 16 Known length for calibrating structure parameters.

Download Full Size | PDF

Table 3 shows the comparative results of structure parameters through different techniques. It is clear that different calibration techniques have influence on these parameters since they use the same intrinsic parameters.

Table 3. Comparative result of the structure parameters

View Table | View all tables in this article

4.3.3 Accuracy evaluation

For the accuracy evaluation of the calibration results, anther 3 pairs of images of the chessboard target are captured. Every distance between the adjacent corners on the chessboard target in the horizontal and the vertical directions are measured by the calibrated BVS. The average value and root-mean-square (RMS) error are calculated to evaluate the calibration accuracy. To further investigate the validity of our method, the target is also measured using the other two groups of calibration results, as shown in Table 4 . In order to give a concrete display for our proposed calibration method, all the reconstructed 3D points are shown in Fig. 17 .

Table 4. Measurement results of chessboard corners

View Table | View all tables in this article

 

Fig. 17 The reconstruction of all feature points of testing data using our proposed calibration method.

Download Full Size | PDF

As shown in Table 4, the RMS error of our method is 0.041mm with the view field of $300\times 300mm$. Compared with other two methods, our method is obviously better than the method based on known length but inferior to Bouguet toolbox. The average RMS error difference between our method and Bouguet toolbox is 0.013mm. Although both our method and Bouguet toolbox are based on the same intrinsic parameters, the targets when calibrating the structure parameters are different. The accuracy of parallel lines target used in our method is 0.02mm and the chessboard target used in Bouguet toolbox is the accuracy of 5um. Taking account of the accuracy difference of targets, it is easy to deduce that our method will perform better and reach the same accuracy level with Bouguet toolbox if the parallel lines target is also the accuracy of 5um. Analyzing of the standard error, it can be acquired that our method and Bouguet toolbox are more stable. Meanwhile, parallel lines are much more ubiquitous than chessboard in the real life, such as the airport runway, guardrail, zebra crossing and so on. Our method has great advantage on self-calibration and onsite calibration.

5. Conclusions

In this article, a calibration method of BVS based on the vanishing features of parallel lines is proposed. The planar target with a set of parallel lines spacing of $D$ is placed at least twice in the calibration progress. In addition, the calibration model of BVS proposed in the method achieves the separation of the rotation matrix $R$and the translation vector $T$, which has the benefit of reducing the complexity and difficulty of calculation. The objective optimization function is based on the absolute distance of parallel lines and the consistency of the target plane’s normal vector under LCCF and RCCF. The initial value is determined by linear method and following is the nonlinear optimization in the 3D measurement space, which ensures the global optimum. Towards the factors that affect calibration accuracy, theoretical derivation gives qualitative analysis and computer simulations aim at qualitative results. The specific method to decrease the calibration error caused by these factors is also proposed. Real data shows that the measurement accuracy is about 0.040mm with the view field of $300\times 300mm$. Compared with conventional calibration methods, our method is simple, effective and precise, especially for onsite calibration and self-calibration. However, we observe that the algorithm may fail in the some cases. For example, when the target plane is nearly parallel to the image plane, the vanishing line approximates the infinity leading to great calculation error. If the angle between the target plane and the image plane is oversize or the image noise is too much, it is impossible to guarantee the extracting accuracy of parallel lines. Meanwhile, both enough number and adequate distance of parallel lines are necessary, or the calibration result may be inaccurate. In practice, it is easy to avoid the degenerate configurations described above. Finally, as the properties of lines, the proposed method is suitable for the calibration of cameras without public visual theoretically. We plan to incorporate the model and algorithms into global calibration of multi-sensor vision system.

Appendix

The vanishing line given the image of a set of coplanar equally spaced parallel lines.

A set of equally spaced lines on the scene plane may be represented as $a{x}^{\prime }+b{y}^{\prime }+\lambda =0$, where $\lambda$ takes integer values. This set (a pencil) of lines may be written as $l_n^{\text{'}}={(a,b,n)}^T=(a,b,0)+n{(0,0,1)}^T$, where ${\text{(0,0,1)}}^T$ is the line at infinity on the scene plane. Under perspective imaging the point transformation as is $x=H{x}^{\prime }$, and the corresponding line map is $l_n=H^{-T}{l}_n^{\text{'}}=l_0+nl$, where $l$ the image of ${(0,0,1)}^T$, is the vanishing line of the plane. The imaged geometry is illustrated in Fig. 18 .

 

Fig. 18 Determining a plane’s vanishing line from imaged equally spaced parallel lines.

Download Full Size | PDF

The vanishing line $l$ may be determined from the set provided their index (n) is identified. The closed solution for the vanishing line $l$ is:

If $l_n=\left(\begin{array}{c}{a}_n\\ -1\\ b_n\end{array}\right),l_0=\left(\begin{array}{c}{a}_0\\ -1\\ b_0\end{array}\right),l=\left(\begin{array}{c}\alpha \\ -1\\ \beta \end{array}\right)$ (Here, the case of lines parallel to the y-axis of image plane is ignored), the following equation stands:

(29)$$l_n=l_0+nl\Rightarrow {\rho }_n\left(\begin{array}{c}{a}_n\\ -1\\ b_n\end{array}\right)={\rho }_0\left(\begin{array}{c}{a}_0\\ -1\\ b_0\end{array}\right)+n\left(\begin{array}{c}\alpha \\ -1\\ \beta \end{array}\right),$$

Where ${\rho }_n$and${\rho }_0$ are scale factors. From Eq. (29), we have

(30)$$\left[\begin{array}{ccc}n& 0& a_0-a_n\\ 0& n& b_0-b_n\end{array}\right]\left[\begin{array}{c}\alpha \\ \beta \\ {\rho }_0\end{array}\right]=n\left[\begin{array}{c}{a}_n\\ b_n\end{array}\right].$$

Then the following equation stands:

(31)$$\left[\begin{array}{ccc}1& 0& a_0-a_1\\ 0& 1& b_0-b_1\\ 2& 0& a_0-a_2\\ 0& 2& b_0-b_2\\ ⋮& ⋮& ⋮\\ n& 0& a_0-a_n\\ 0& n& b_0-b_n\end{array}\right]\left[\begin{array}{c}\alpha \\ \beta \\ {\rho }_0\end{array}\right]=\left[\begin{array}{c}{a}_1\\ b_1\\ 2a_2\\ 2b_2\\ ⋮\\ n{a}_n\\ n{b}_n\end{array}\right].$$

The vanishing line can be obtained by solving Eq. (31) with the linear least-squares method.

Acknowledgments

This work is supported by the Instrument Special National Natural Science Foundation of China (No.61127009), the Natural Science Foundation of Beijing (No.3142012), and the National Key Scientific Instrument and Equipment Development Project (No.2012YQ140032). We appreciate for the editor and reviewers’ valuable comments on our manuscript.

References and links

1. G. Zhang, Visual Measurement, 1nd ed. (Beijing Sciences, 2008).

2. S. Ma and Z. Zhang, Computer Vision: Theory and Algorithms, 1st ed. (Beijing Sciences, 1998).

3. F. Zhou, J. Zhu, and X. Yang, “A field calibration technique for binocular vision sensor,” Yiqi Yibiao Xuebao 21(2), 142–145 (2000).

4. G. Zhang and F. Zhou, “The calibration method of stereo visual sensor structural parameters based on standard length,” in Proceedings of CSAA on Aviation Industry Measurement and Control Technology, ed. (Academic, 2001), pp. 259–263.

5. J. Sun, Z. Wu, Q. Liu, and G. Zhang, “Field calibration of stereo vision sensor with large FOV,” Opt. Precision Eng. 17(3), 633–640 (2009).

6. F. Zhou, G. Zhang, and Z. Wei, “Calibrating binocular vision sensor with one-dimensional target of unknown motion,” Chin. J. Mech. Ens-EN 42(6), 92–96 (2006).

7. R. Hartley, “Estimation of relative camera positions for uncalibrated cameras” In Proceedings of. European Conference on Computer Vision, ed. (Academic, 1992) pp. 579–587. [CrossRef]  

8. M. Li, A. Zhang, and S. Hu, “On 3D measuring system of sheet metal surface based on computer vision,” Chin. Mech. Eng. 13(14), 1177–1180 (2002).

9. Y. Ma and W. Liu, “A linear self-calibration algorithm based on binocular active vision,” Robot 26(6), 486–490 (2004).

10. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2003).

11. J. Bouguet, “Camera calibration toolbox for Matlab,” http://www.vision.caltech.edu/bouguetj/calib_doc/.

12. Z. Wei, C. Li, and B. Ding, “Line structured light vision sensor calibration using parallel straight lines features,” Optik (Stuttg.) 125(17), 4990–4997 (2014). [CrossRef]  

13. Z. Wei, M. Shao, G. Zhang, and Y. Wang, “Parallel-based calibration method for line-structured light vision sensor,” Opt. Eng. 53(3), 033101 (2014). [CrossRef]  

14. Z. Wei, M. Xie, and G. Zhang, “Calibration method for line structured light vision sensor based on vanish points and lines,” inProcceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp.794–797. [CrossRef]  

15. Moré J J, “The Levenberg-Marquardt algorithm: implementation and theory,” in Numerical Analysis (Springer Berlin Heidelberg, 1978), pp. 105–116.

16. M. Lourakis, “Levmar: Levenberg-marquardt nonlinear least squares algorithms in C/C++,” (Ics.forth, 2004), http://www. ics. forth. gr/~lourakis/levmar.

17. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). [CrossRef]  

18. C. Steger, “Unbiased extraction of curvilinear structures from 2D and 3D image,” Ph.D. Dissertation, Technische Universitaet Muenchen (1998).

19. P. D. Kovesi, “MATLAB and Octave functions for computer vision and image processing,” http://www. csse. uwa. edu. au/~pk/Research/MatlabFns/# match.

20. Rodrigues O, “Des lois géométriques qui régissent les déplacements d'un système solide dans l'espace: et de la variation des cordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire,” (Publisher not identified, 1840).

21. Y. Fei, Error Theory and Data Processing, 1st ed. (China Machine, 2010).