Open Access
Add to BibSonomy
Abstract
Multi-line structured light three-dimensional (3D) scanning measurement system enables to obtain the richer 3D profile data of the object simultaneously during one frame, ensuring high accuracy while structured light is deformed for the modulation by the object. Nevertheless, current calibration methods cannot fully take advantage of its high precision. In this paper, a fast and high-accuracy 3D measurement system based on multi-line lasers with a spatially precise structure via integrating a diffraction grating was proposed. This helps achieve precise calibration results of the light planes by introducing spatial constraint relations of the diffractive light, thus improving measurement accuracy. The operating principle and the workflow of the proposed system were described in detail. The measurement accuracy of the developed prototype was verified through contrastive experiments. At a working distance of 400 mm, the results show that the root mean square error (RMSE) of the proposed system is 0.083 mm, which is improved by 37.6% compared to the traditional calibration method of light planes for the ranging system. The system utilizing a grating that facilitates the integration of the device has great application value.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
With the advantages of high accuracy, no contact, high robustness, and cost-effectiveness, Profilometry based on laser triangulation is widely used in industrial inspection and manufacturing [1], aerospace manufacturing [2], precision measurement [3], agricultural science [4], and other fields [5]. High efficiency has become a research priority when performing scanning measurements.
To achieve this purpose, the most commonly used solution is to project the multi-line laser, which can improve measurement efficiency while ensuring measurement accuracy. Li [6] presented a new non-contact measuring method using multi-structure linear lighting (or multi-light-knife) based on laser scanning measurement technology. A virtual net mapping and the least square theory are applied to calibrate the measuring device of the multi-light-knife in the whole measuring field respectively. Heo [7] used multiple line structured laser sources to realize profile measurement of large-scale hull pieces for fabrication and assembly. Wu [8] uses a semiconductor laser module which projects multiple light planes to the surface of the measured round steel simultaneously. The system was mounted at a random stance. So the calibration method is localized and has low accuracy. An optical sensor principle called “flying triangulation” is presented by Ettl [9] that enables a motion-robust acquisition of 3D surface topography. It combines a simple handheld sensor with sophisticated registration algorithms. For relating the line shift to the depth information through the entire measurement volume, a longitudinal z-calibration is performed by observing the projected line pattern for a planar object placed at known z-positions through the entire measurement depth range which is a very elaborate and time-consuming procedure. Gao [10] developed a novel 3D wide field-of-view (FOV) scanning measurement system which adopted two multiline structured-light sensors to solve the problem of low scanning efficiency. Since six light stripes are simultaneously projected on the object surface, the scanning efficiency is greatly improved. The calibration of system is calibrated on-site by a 2D pattern independently. A new 3D measurement method combining colored structured light projection with trigonometry computation was proposed by Luo [11]. Distinguishable colored structured lights improved 3D measuring speed and accuracy by less image acquirement and more concentrated structured lights. To reduce the relative error of the overall fitting result of the multi-line structured light planes Li [12] modified the existing calibration plate according to the requirements of the experimental design, fitting the light planes by a random sample consensus algorithm (RANSAC). Wang [13] proposed a mathematical model for a multi-parallel line laser 3D measurement system, introducing the concept of multiple base planes to derive the system measurement equation. The calibration method based on the system model can simplify the calibration process while sacrificing measurement accuracy.
As a precision optical element, the grating can generate various fine-structured light to illuminate objects and realize the 3D reconstruction process, which can achieve high measurement accuracy [14–16]. Ye [17] proved a laser triangulation sensor (LTS) with improved measurement accuracy via integrating a diffraction grating. The diffraction grating generates several diffraction light spots on the image sensor enabling it to obtain multiple results of the object displacement simultaneously during one sampling period. Thus, it is achievable to obtain a higher measurement accuracy by averaging operations due to the averaging-error effect. The grating-based scanning system attracts researchers’ interest due to its lightweight apparatus and precise structure. The proposed method in the past primarily focus on hardware design, specifically how to design optical modules to generate three-line spatial structured light with high precision.
In this paper, we developed a fast and precise multi-line laser 3D measurement system based on a diffraction grating. The high-accuracy calibration method of the proposed system utilizing the constraint of diffractive light to fit the light planes was performed. Compared to previous methods of fitting light planes separately [12], calibrating the system parameters based on the mathematical model instead of light planes [13], calibrating both the camera model and the laser planes [18] and other representative calibration methods [19], this diffraction model-based calibration method was independent on the mounting accuracy of the setup or the calibration accuracy of a particular light plane. The calibration process is simple and achieves higher calibration accuracy, thus improving the measurement accuracy of the system while ensuring measurement efficiency. The proposed method complements the aforementioned study, collectively improving the performance of grating-based multi-line laser scanning system.
The rest of the contents of this paper are organized as follows. In Section 2, the measurement principle of the proposed system is explained, and the working process is depicted. In Section 3, the calibration method of the light planes, is described. In Section 4, the calibration result of light planes and the experimental measurement results are presented In Section 5, the paper ends with conclusions.
2. Principle
2.1 Construction of the measurement system
The multi-line laser scanning measurement system shown in Fig. 1 is based on laser triangulation that determines the 3D coordinates of the target through the triangular relationship between the projection point and the imaging point. The configuration of the proposed system mainly consists of the light source, an adjustable aperture, a linear motion stage, a stepping motor and its controller, a camera with the lens and a computer.
The measured object is placed on the platform. A line laser emits the line-shaped laser, and a grating with precise grating constants is utilized to divide the line laser into multiple lines with certain spatial light distribution. The aperture blocks the high-order and low-energy laser lines. The camera captures the images of the distorted laser lines which contain the 3D characteristics information of the object surface since the lines are modulated by the depth. After calculating the 3D coordinates of points on the laser lines from the 2D images based on the parameters of the camera and light plane equations, the profiles of the light sections can be obtained. During the scanning process, a controlled displacement platform driven by the stepping motor moves with the object directionally. Thus, a series of profiles can be acquired.
For the sake of simplicity, the light source of proposed system is presented based on a grating.
2.2 Mathematical model of the measurement system
Similar to traditional multi-line laser measurement systems [13], the proposed system established in this paper is based on light plane calibration. The mathematical model of the proposed system shown in Fig. 2 is described below in detail. The definition of the coordinate systems in Fig. 2 is as follows: ${O_w}{x_w}{y_w}{z_w}$ is the world coordinate system (WCS). ${O_c}{x_c}{y_c}{z_c}$ is the camera coordinate system (CCS). The homogeneous camera coordinate system (HCCS) ${O_h}{x_h}{y_h}$ is located at the position of the camera coordinate system where ${z_c} = 1$. ${O_p}uv$ is the pixel coordinate system (PCS). Those coordinate systems are relatively static during the scanning process.
Based on the pinhole camera model [20], any point ${P_w}$ on the laser lines formed by the light plane ${\pi _i}$ and the object surface in the WCS can be transformed to a point ${P_h}$ in the HCCS [10].
The vector $\overrightarrow {{O_c}{P_h}} $ in the CCS is the direction vector of the reflected light. This gives the equation of a straight line for the reflected ray, namely
In the CCS, the 3D space equation of an arbitrary light plane ${\pi _i}$ can be described by
where $\textrm{i}$ represents the order of the diffractive laser lines. The 3D coordinates of the point ${P_c}$ can be obtained by solving simultaneous Eq. (1) and Eq. (2).2.3 Working principle
The working process of the proposed system shown in Fig. 3 is as follows.
Multiple frames of deformation pattern images where the multi-line laser incident on the object surface are collected as the motor moves at a constant speed with time. The undistorted pixel coordinates of the centerline of the multi-line laser are obtained by applying the centerline extraction algorithm [21] and the distortion coefficient of the camera. According to the internal matrix, the pixel coordinates of each line are converted into the homogeneous coordinates of the camera. Thus, the line equation of the reflected light in the CCS can be acquired. Determine the corresponding diffraction order of the centerline. The 3D coordinates in CCS at the position of the laser lines can be calculated by simultaneously combining the light plane equation of the corresponding laser line and the line equation of the reflected light. The final 3D point cloud data of the object at different positions can be registered by substituting the direction vector of the translation platform [10].
3. Calibration of multi-line light planes
The calibration accuracy of light planes is the crucial factor in multi-line structured light 3D scanning measurement system. However, the traditional calibration method calculates the light planes one by one based on Eq. (2). And the optimization algorithm for fitting the light planes is susceptible to noise, which can lead to overfitting. In this work, a high accuracy calibration method was proposed. The optimization algorithm was simplified by adding the constraint of the spatial distribution of the diffracted light to reduce the quantity of fitting parameters. And the proposed method based on diffraction equations of the grating is more resistant to noise which could achieve more accurate calibration result of the light planes.
3.1 Diffraction equations of the grating
The diffracted light of a grating has a precise spatial light distribution, which is the theoretical basis of the proposed method’s core advantage of high-precision measurement. The spatial brightness distribution of the laser lines diffracted by the grating [22] is described in the following equation
The position of the different orders of diffracted light, shown in Fig. 4(b), can be described using the grating equation, described by the following equation
where m is the order of diffracted light.
Fig. 4. Mathematical model of the diffraction grating. (a) The model of the diffraction by multiple slits. (b) The silhouette of the light planes proposed in the system.
3.2 Proposed calibration algorithm of light planes
The proposed algorithm focuses on how to describe and solve the optimization problem of light planes while considering the spatial distribution of the diffracted light. The spatial distribution of the diffracted light is represented from two aspects. First, the angles between the light planes are depicted by the grating equation shown in Eq. (5). Second, the light planes of diffracted light lines intersect in a single line.
In conventional calibration method, the parameters of each light planes described by Eq. (2) can be solved by SVD algorithm. Equation (6) is the mathematical form.
In the proposed algorithm, the normal vectors $({{a_i},{b_i},{c_i}} )$ of the light planes were optimized according to the first aspect. Take the square of the distance from the point $({{x_{im}},{y_{im}},{z_{im}}} )$ to the light plane ${\pi _i}$ as the objective function, shown in mathematical form as
Substituting the grating Eq. (5), the constraint of the normal vector of the light planes satisfies the following relationship
In the proposed algorithm, the positions ${d_i}$ of the light planes were optimized according to the second aspect. All light planes intersect in a single line and their equations can be described as a linear combination of two fundamental plane equations, namely
Applying the linear representation to express the light plane equations of each order of diffraction, we obtain a series of linear equations. Adopting the least squares optimization algorithm, we can solve the parameters of the base planes and the combination coefficients of each light plane. The fitting process of the parameter $({{\mathrm{\epsilon }_i},{\mu_i}} )$ and the two fundamental plane equations is performed, and finally the value ${d_i}$ of the light plane ${\pi _i}$ is calculated based on the fitting parameters.
To conclude, the diffraction angle constrains the normal vector of the light plane, and the condition that diffracted rays intersect in a line constrains the position of the light plane.
4. Experiments and results
4.1 Prototype development
A prototype of the proposed measurement system shown in Fig. 5 according to the schematic diagram shown in Fig. 1 was developed.
Since the energy of the diffracted light from the grating is concentrated on the first few orders, to simplify the processing, the width of the aperture is adjusted so that the lines of 0, ± 1 and ±2 orders are transmitted and projected onto the object. In the experiment, the working distance of the measurement system is 400 mm. The wavelength of the laser used is 660 nm, and the grating has 50 lines per millimeter.
4.2 Calibration result of the light planes
The traditional and proposed calibration algorithm of the light planes are implemented in the experiment. The calibration process and results are presented in this section.
Calibrate the image shown in Fig. 6(a) and get the camera's parameters by Zhang’s method [20]. Calculate the coordinates in CCS of the five sets of the centerlines in the image shown in Fig. 6(b) by Steger’s method [21]. In addition, a sub-pixel centerline extraction algorithm combining spatial filtering and adaptive thresholding is employed to ensure the accuracy of the centerline extraction algorithmically.
Fig. 6. The calibration images for the camera and the light planes at the same position. (a) The image of the chessboard for camera calibration at normal exposure. (b) The image of the laser lines on the chessboard for light plane calibration at low exposure.
After obtaining five sets of 3D points in CCS on the five light planes, the parameters of each light plane were then calculated by the traditional algorithm and the proposed algorithm separately. The results of the fitted light planes are shown in Table 1.
Table 1. The fitted results of light planes by the traditional and the proposed algorithm
4.3 Accuracy evaluation of the proposed system
To verify the accuracy of the system, the standard gauge with a real height of 8.9 mm and the accuracy 0.005 mm as shown in Fig. 7(a) was scanned. The deformation laser lines captured by the camera are shown in Fig. 7(b). Two calibration algorithms of the light planes, line-by-line calibration and the proposed calibration were adopted in the experiment, as shown in Table 1. To make a comparison, the measured results of the two methods are shown respectively in Fig. 8(a) and 8(b). The measured height deviation of the gauge from different orders denoted as different colors is reduced by applying the proposed calibration algorithm.
Fig. 7. The standard gauge to verify the measurement accuracy of the proposed system. (a) The photo of the gauge. (b) The image of deformed laser lines on the gauge snapped by the camera.
Fig. 8. Comparative 3D image results of the standard gauge for accuracy estimation with two different calibration methods. (a) The retrieval result of the standard gauge under the condition of calibrating the light plane line by line. (b) The retrieval result of the standard gauge under the condition of calibrating the light plane by the proposed method.
In this comparison experiment, 2000 points each of the measured height deviation of the gauge from different orders, as shown in Fig. 8, was selected randomly. The errors between the height deviation and real value of the standard gauge are shown in Table 2.
Table 2. Contrastive measurement accuracy evaluation for 5 laser lines respectively
The mean error (ME) and RMSE of the measurement results reflect the variation from the standard value and repeatability of the system. Compared to calibrating light planes separately, the overall RMSE of the proposed method was reduced from 0.133 mm to 0.083 mm, and the accuracy of the proposed system improves by 37.6%.
Furthermore, to demonstrate the capability of the proposed system in measuring curved surface profiles with accuracy, a standardized ball with a real radius 18.9 mm and the accuracy 0.005 mm shown in Fig. 9(a), specialized Ti-TimesCC-100-G-Ball, was measured. The measurement results including the point cloud and the model are shown in Fig. 9(b) and 9(c).
Fig. 9. The standard ball and the measurement result of the profile. (a) Photo of the ball. (b) Point cloud of the ball. (c) Model of the ball after the reconstruction.
To evaluate the precision of the proposed system, we conducted the comprehensive scan of the ball utilizing the diffraction light. Five sets of point cloud were acquired by five laser lines with different orders, Fitting process were operated to obtain the fitted center of the ball in the CCS and the measured radius respectively, as illustrated in the Table 3.
Table 3. The fitting results of the point cloud scanned by five laser lines respectively
The distance from the point cloud of the ball scanned by five laser lines to the respective fitted center were calculated. The error was the difference between the distance and the actual radius 18.9 mm of the standard ball. The error graph is presented in Fig. 10. The overall RMSE of the fitted ball is 0.089 mm, which aligns with the results obtained from the standard block, indicating that the proposed system is capable of measuring curved surfaces with precision.
Fig. 10. The error graph of the measured ball under different orders of the diffraction light, the error bar with the unit millimeter represents the difference between the distance from the point cloud to the center of the ball and the real radius. (a) Error graph of the point cloud scanned by the laser line with the order -2. (b) Error graph of the point cloud scanned by the laser line with the order -1. (c) Error graph of the point cloud scanned by the laser line with the order 0. (d) Error graph of the point cloud scanned by the laser line with the order 1. (e) Error graph of the point cloud scanned by the laser line with the order 2.
For the system developed in this work, the grating generates multi-line structured light with a precise spatial structure getting rid of the dependence on mechanical installation accuracy compared to the traditional multi-line 3D scanning measurement system. The fitted light planes are insensitive to noise utilizing the constraint described by the diffraction equation. All the plane data are fitted together, which plays a certain averaging error effect and reduces the measurement error.
4.4 Validation of scanning measurement
The scanning measurement process of the workpiece is performed using the multi-line laser 3D scanning measurement system based on grating diffraction proposed in this paper to demonstrate the applicability. Figures 11(a) and 11(b) show the captured workpiece image pair with laser lines projected. The 3D point cloud and model obtained are shown in Figs. 11(c) and 11(d). The point cloud data denoted by five colors calculated through the deformation of five diffraction laser lines are obtained simultaneously. After experimental verification, the proposed system and calibration method proposed in this paper can effectively improve measurement accuracy and reserve the advantage of measurement efficiency in multi-line measurement system.
Fig. 11. The photo and the reconstruction results of the measured workpiece in the experiment. (a) The photo of the workpiece. (b) The image of deformed laser lines on the workpiece snapped by the camera. (c) The point cloud of the workpiece. (d) The 3D model of the workpiece.
5. Conclusion
In this paper, the multi-line laser 3D scanning measurement system based on grating diffraction and the calibration method of the light planes are proposed. The calibration process of the light planes is optimized according to the constraints of the spatial distribution of the diffracted light. Compared to the traditional calibration method of multi-line laser measurement systems, the proposed method simplifies the optimization process and is more resist to noise, thus achieving accurate calibration result of the light planes. All the plane data are fitted together, which plays a certain averaging error effect and leads to an overall enhancement of the measurement accuracy of the system. The experiment verifies that this method shows higher accuracy compared to the traditional multi-line laser scanning measurement system. The proposed system based on the grating is easily capable of generating multi-line structured light, which inherits the high efficiency of multi-line structured light measurement system. The calibration method offers superior measurement accuracy. In addition, the selection of grating models offers the potential application of measurement volumes in the range from some millimeters up to meters. The system utilizing a grating which facilitates the integration of the device has great application value.
Funding
National Key Research and Development Program of China (2022YFB3603200).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but can be obtained from the authors upon reasonable request.
References
1. H.-L. Huang, W.-Y. Jywe, C.-H. Liu, L. Duan, and M.-S. Wang, “Development of a novel laser-based measuring system for the thread profile of ballscrew,” Opt Laser Eng 48(10), 1012–1018 (2010). [CrossRef]
2. B. Sun and B. Li, “Laser Displacement Sensor in the Application of Aero-Engine Blade Measurement,” IEEE Sensors J. 16(5), 1377–1384 (2016). [CrossRef]
3. J. P. Makai, “Simultaneous spatial and angular positioning of plane specular samples by a novel double beam triangulation probe with full auto-compensation,” Opt Laser Eng 77, 137–142 (2016). [CrossRef]
4. S. Paulus, T. Eichert, H. E. Goldbach, and H. Kuhlmann, “Limits of active laser triangulation as an instrument for high precision plant imaging,” Sensors 14(2), 2489–2509 (2014). [CrossRef]
5. W. Chen, Z. Ni, X. Hu, and X. Lu, “Research on pavement roughness based on the laser triangulation,” Photonic Sens 6(2), 177–180 (2016). [CrossRef]
6. B. Li, Z. Jiang, and Y. Luo, “Measurement of three-dimensional profiles with multi structure linear lighting,” Robot Cim-Int Manuf 19(6), 493–499 (2003). [CrossRef]
7. E. Heo, B. Kim, H. Lee, and J. Han, “3D Profile Measurement of Large-Scale Curvature Plates using Structured Light Source,” in Conference on Two- and Three-Dimensional Methods for Inspection and Metrology VI, (San Diego, CA, 2008), p. 70660O.
8. B. Wu, T. Xue, T. Zhang, and S. H. Ye, “A novel method for round steel measurement with a multi-line structured light vision sensor,” Meas. Sci. Technol. 21(2), 025204 (2010). [CrossRef]
9. S. Ettl, O. Arold, Z. Yang, and G. Hausler, “Flying triangulation-an optical 3D sensor for the motion-robust acquisition of complex objects,” Appl. Opt. 51(2), 281–289 (2012). [CrossRef]
10. H. Gao, F. Q. Zhou, B. Peng, Y. X. Wang, and H. S. Tan, “3D Wide FOV Scanning Measurement System Based on Multiline Structured-Light Sensors,” Adv. Mech. Eng. 6, 758679 (2014). [CrossRef]
11. B. Luo and G. P. Guo, “Fast 3D Measurement based on Colored Structured Light Projection,” in 12th IEEE Int Conf Ubiquitous Intelligence & Comp/12th IEEE Int Conf Autonom & Trusted Comp/15th IEEE Int Conf Scalable Comp & Commun & Associated Workshops/IEEE Int Conf Cloud & Big Data Comp/IEEE Int Conf Internet People, (Beijing, PEOPLES R CHINA, 2015), pp. 1537–1540.
12. W. G. Li, H. Li, and H. Zhang, “Light plane calibration and accuracy analysis for multi-line structured light vision measurement system,” Optik 207, 163882 (2020). [CrossRef]
13. Y. Wang and B. Lin, “A fast and precise three-dimensional measurement system based on multiple parallel line lasers,” Chinese Phys. B 30(2), 024201 (2021). [CrossRef]
14. K. Iwata, F. Kusunoki, K. Moriwaki, H. Fukuda, and T. Tomii, “Three-dimensional profiling using the Fourier transform method with a hexagonal grating projection,” Appl. Opt. 47(12), 2103–2108 (2008). [CrossRef]
15. J. Zhang, C. Zhou, and X. Wang, “Three-dimensional profilometry using a Dammann grating,” Appl. Opt. 48(19), 3709–3715 (2009). [CrossRef]
16. S. Wei, S. Wang, C. Zhou, K. Liu, and X. Fan, “Binocular vision measurement using Dammann grating,” Appl. Opt. 54(11), 3246–3251 (2015). [CrossRef]
17. G. Ye, Y. Zhang, W. Jiang, S. Liu, L. Qiu, X. Fan, H. Xing, P. Wei, B. Lu, and H. Liu, “Improving measurement accuracy of laser triangulation sensor via integrating a diffraction grating,” Opt Laser Eng 143, 106631 (2021). [CrossRef]
18. T.-T. Tran and C. Ha, “Extrinsic Calibration of a Camera and Structured Multi-Line Light using a Rectangle,” Int. J. Precis. Eng. Manuf. 19(2), 195–202 (2018). [CrossRef]
19. Y. Zou, L. Niu, B. Su, Y. Sun, and J. Liu, “Calibration of 3D imaging system based on multi-line structured-light,” in Three-Dimensional Image Acquisition and Display Technology and Applications, (2018), p. 108450C.
20. Z. Zhang, “A Flexible New Technique for Camera Calibration,” IEEE Trans. Pattern Anal. Machine Intell. 22(11), 1330–1334 (2000). [CrossRef]
21. C. Steger, “An Unbiased Detector of Curvilinear Structures,” IEEE Trans. Pattern Anal. Machine Intell. 20(2), 113–125 (1998). [CrossRef]
22. B. Max and E. Wolf, Principles of optics: electromagnetic theory of propagation, interference and diffraction of light, (Elsevier, 2013), Chap. 8.
