Open Access
Add to BibSonomy
Abstract
In the field of positioning measurement, a combination of complex components, a stringent environment, and time-consuming calibration are the main limitations. To address these issues, this paper presents a deep learning-based positioning methodology, which integrates image processing with nanomanufacturing technology. Non-periodic microstructure with nanoscale resolution is fabricated to provide the surface pattern. The main advantage of the proposed microstructure is its unlimited measurement range. A residual neural network is used for surface pattern recognition to reduce the search area, a survival probability mechanism is proposed to improve the transmission efficiency of the network layers, and template matching and sub-pixel interpolation algorithms are combined for pattern matching. The proposed methodology defines a comprehensive framework for the development of precision positioning measurement, the effectiveness of which was collectively validated by pattern recognition accuracy and positioning measurement performance. The trained network exhibits a recognition accuracy of 97.6%, and the measurement speed is close to real time. Experimental results also demonstrate the advantages and competitiveness of the proposed approach compared to the laser interferometer method.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Positioning measurement is used to obtain the displacement, or the distance, from an objective point relative to a reference point. Precision positioning can be described as a method which is able to provide a resolution better than 0.1 µm or a relative positioning error of less than 10−6 [1]. Among various positioning methods, optical sensors are the most widely applied because of their non-contact measurement, wide bandwidth, and high resolution. Optical sensors are currently based on one of three measurement principles: time-of-flight distance, laser interferometer, and optical encoders. All of them, however, have shortcomings. For example, since air refractive index error and wavelength error are the main sources of uncertainty, a stringent environment in terms of atmospheric pressure, temperature, humidity, and medium, is needed to control potential errors. Furthermore, the current methods involve numerous components that require a large space for installation and high assembly accuracy, both of which can cause systematic errors and incur extra costs. There is therefore a need for a positioning method based on a new measurement principle.
With the rapid development of image processing technology, more and more vision-based precision measurement methods have emerged [2–4]. Compared with the popular methods, vision-based measurement has high efficiency, simple structure, low cost, and non-damage detection features, which together meet the development requirements of modern measurement technology. Although vision-based measurement has been applied to precision positioning, it is currently based on the phase shift principle integrated with periodic surface patterns, and since the manufacturing accuracy of micro-patterns is extremely high, the cost of calibration is also high. A few methods using a non-periodic pattern have been investigated [5–7] by using template matching for position measurement. However, a low calculation efficiency is still a shortcoming for the application, which becomes more serious when the position search area is large. In recent years, neural network models involving deep learning areas have emerged [8]. Deep learning has been applied to many pattern-recognition issues such as face recognition, transportation, and medicine. Hence, high efficiency pattern recognition based on a neural network is the key to improving measurement speed and robustness.
This paper develops an artificial neural network-based nanoscale positioning measurement method. The measurement framework is illustrated in Fig. 1. The surface pattern is firstly captured by a charge-coupled device (CCD) camera, which is then recognized by the neural network that determines the sub-area it belongs to; this process greatly reduces the area for matching. The next step is matching the captured pattern and obtaining the position $({x_1},{y_1})$ before sub-pixel interpolation is applied to improve the positioning accuracy and produce the final measured position $({x_2},{y_2})$. In this paper, the designed surface pattern and the modified neural network are presented respectively. Experimental validation is conducted to demonstrate the performance of the proposed new approach.
2. Surface pattern
Before using the neural network to recognize patterns, the first step is to provide a high-resolution surface map. To ensure manufacturing accuracy and efficiency, this study used ultra-precision machining (UPM) technology to fabricate a micro-structured surface as shown in Fig. 2. The surface features of the machined micro-structured surface are composed of concentric circles and straight lines whose equal spacing is 50 µm, so that even under a microscope it is observed to be high resolution. Moreover, it should be noted that the designed surface structure ensures that the pattern in any area is unique, which is different from most related studies. Consequently, the intensity distribution of the images of the microstructure is unique in spite of machining errors. The microstructure surface not only guarantees pattern recognition but also eliminates time-consuming calibration.
3. Random residual neural network
This study used an artificial neural network (ANN) [9] to solve the pattern recognition problem. The development of ANN is rapid using a series of classic algorithms such as AlexNet [10], VGG [11], Inception [12]. In particular, the advent of residual neural network (ResNet) [13] overcomes the problem of performance degradation due to increased network depth, and makes deeper learning (network layers of more than 100) possible. Since ResNet usually requires long training and recognition time, this study randomly discarded some layers during training but used the full network for testing and named the generated network as random ResNet (R-ResNet); and hyper-parameter survival probability was added to the neural network transmission, as shown in Fig. 3. Hi denotes the output of the ith layer, fi refers to a typical convolutional transformation from layer i-1 to layer i. Pi denotes the survival probability of ith layer, obtained by Eq. (1):
where N is the number of total network layers, pN denotes the constant survival probability in the last layer, set to be 0.3. Pattern image is the input layer (the layer transmission principle follows ResNet), which is not expanded in this study. But Pi influences whether its layer has a contribution. The output can be obtained by Eq. (2): where ReLu is the network activation function, and id(.) denotes the identity transformation [14]. When $\alpha = 1$ the layer is chosen for network transmission, such as layer 1 and layer 3, and the layer is not chosen for training when $\alpha = 0$, such as layer 2 and layer 4. The random chosen layers improve the transmission efficiency and reduces the residual error, but still guarantee the training depth.After pattern recognition, the next step is the pattern matching. This study used template matching algorithm, with the normalized cross-correlation (NCC) value chosen to indicate the similarity [5]. Finally, sub-pixel interpolation needs to be conducted to improve the resolution. The pixel and its neighboring pixels are interpolated with the bilinear interpolation method [15].
4. Experimental results and discussions
Experiments were conducted to demonstrate the performance of the proposed approach from the view of pattern recognition and positioning measurement respectively.
4.1 Pattern recognition results
The experimental setup is shown in Fig. 4(a). The surface pattern was placed on the drive stage of the 3D optical surface profilers (The NewView 9000, Zygo). After auto-focusing by the microscope, the surface topography information was scanned by the profilometer. The whole pattern image was finally obtained as shown in Fig. 4(b). The actual size of the pattern was 2000 µm ${\times}$ 2000 µm, and the image pixels were 1045 ${\times}$ 1045. For pattern recognition training, the patterns was divided into 25 sub-areas from A1 to E5. To generate the training set, image segmentation was conducted in each sub-area. Figure 4(c) shows A3 sub-area as an example. A random 100 ${\times}$ 100 pixel area was cropped as the training set and repeated 50 times. Finally, the pattern database was established containing 25 categories and 50 data images in each category.
Fig. 4. (a) Layout of pattern recognition experiment using surface profiler (b) Area distribution of different patterns (c) Example of image database
The network was trained using 80% of the database with the remaining 20% used for validation. The learning rate was set to be 0.001 and the batch size was 32. The training cycle consisted of 6 epochs. For each iteration, the loss value and validation accuracy were recorded. The training results show that the validation loss value started from 2.28, decreased rapidly to 0.3 in the first epoch and reached 0.043 in the final iteration. Validation accuracy was calculated by the percentage of its correct pattern recognition and the final validation accuracy was 97.6%.
The performance of R-ResNet was compared with other reported recognition methods, and the following three algorithms were chosen: Absolute Template Matching (ATM) [5], Circle Hough Transform (CHT) [16], and ResNet [13]. In addition to pattern recognition accuracy as an important indicator, the whole recognition time was also recorded by the software Matlab 2019b. The comparison results are shown in Table 1, which demonstrate R-ResNet has a good performance for both pattern recognition accuracy and speed. Moreover, it should be illustrated that the surface map can be obtained with different instruments including both profilometer and microscope.
Table 1. Comparison with other recognition methods
4.2 Positioning measurement results
This experiment was done through measuring the feed distance of the computer numerical control (CNC) milling machine tool (Fadal VMC 4020) using two methods: the laser interferometer method (Renishaw ML10), and the proposed method. The experimental layout is shown as Figs. 5(a) and (b).
Fig. 5. Experimental layout of positioning measurement (a) layout of laser interferometer method (b) layout of the proposed method
In the experiment, the feed distance of the CNC tool ranged from 1mm to 10 mm with a 1-mm feed spacing. Each feed distance was repeated for 10 steps. The two methods measured the actual position of the tool synchronously (see Visualization 1). Therefore, the tool position error ${\varepsilon _{tool}}$ was obtained by Eq. (3):
where ${P_{measure}}$ refers to the measured position by the proposed method or laser interferometer, and ${P_{feed}}$ refers to the feed distance controlled by the CNC tool. Figure 6 shows the experimental results of ${\varepsilon _{tool}}$. The ${\varepsilon _{tool}}$ average values in X- and Y-direction using the laser interferometer method were measured to be 1.607 µm and 1.770 µm respectively, and were calculated to be 1.603 µm and 1.773 µm using the proposed method.Fig. 6. Comparison of the measurement results of the CNC tool positioning errors (a) X direction (b) Y direction
The standard deviation values of measure difference using the two methods were 36.6 nm and 53.0 nm in the X-direction and Y-direction respectively. The results demonstrate that the proposed method has the same level of measurement accuracy as the laser interferometer. However, the proposed method took up much less space and used less components than the laser interferometer. And, more importantly, the proposed method can measure the X and Y directions simultaneously. From the perspective of measurement speed, the total measurement time of the proposed method recorded by the software (Matlab 2019b) under the Intel i7-6600 CPU environment, was about 0.2 s. Although the speed is a little slower than that of the laser interferometer, which was 0.12 s, the measurement time can be improved greatly when it is integrated in a field-programmable gate array (FPGA).
5. Conclusion
This study firstly designed a non-periodic microstructure surface and then used the R-ResNet for pattern recognition of the designed microstructure so as to reduce the search area. Template matching and sub-pixel interpolation were then applied to improve measurement accuracy. Training results from the experiments show that R-ResNet has excellent performance for pattern recognition; the measurement speed is about 0.2 s, which is close to real-time measurement. The results indicate that the proposed method has the same measurement accuracy as the current precision positioning measurement methods. However, its advantages of taking up less space, using less components than the laser interferometer, and being capable of measuring the X and Y directions simultaneously. Moreover, the proposed microstructure surface enable this method not be limited by the measurement range. All of above advantages make it a superior nanoscale measurement method.
Funding
Science, Technology and Innovation Commission of Shenzhen Municipality (JCYJ20160608161156442).
Disclosures
The authors declare no conflicts of interest.
References
1. W. Gao, H. Haitjema, F. Fang, R. Leach, C. Cheung, E. Savio, and J.-M. Linares, “On-machine and in-process surface metrology for precision manufacturing,” CIRP Ann. 68(2), 843–866 (2019). [CrossRef]
2. H. Li, B. Zhu, Z. Chen, and X. Zhang, “Realtime in-plane displacements tracking of the precision positioning stage based on computer micro-vision,” Mech. Syst. Signal. Pr. 124, 111–123 (2019). [CrossRef]
3. H. Li, X. Zhang, B. Zhu, and S. Fatikow, “Online precise motion measurement of 3-DOF nanopositioners based on image correlation,” IEEE Trans. Instrum. Meas. 68(3), 782–790 (2019). [CrossRef]
4. V. Guelpa, P. Sandoz, M. A. Vergara, C. Clévy, N. Le Fort-Piat, and G. J. Laurent, “2D visual micro-position measurement based on intertwined twin-scale patterns,” Sens. Actuators, A 248, 272–280 (2016). [CrossRef]
5. C. Zhao, C. Cheung, and M. Liu, “Integrated polar microstructure and template-matching method for optical position measurement,” Opt. Express 26(4), 4330–4345 (2018). [CrossRef]
6. C. Zhao, C. F. Cheung, and M. Liu, “Nanoscale measurement with pattern recognition of an ultra-precision diamond machined polar microstructure,” Precis. Eng. 56, 156–163 (2019). [CrossRef]
7. C. Zhao, C. F. Cheung, and P. Xu, “High-efficiency sub-microscale uncertainty measurement method using pattern recognition,” ISA transactions (2020).
8. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]
9. D. C. Park, M. El-Sharkawi, R. Marks, L. Atlas, and M. Damborg, “Electric load forecasting using an artificial neural network,” IEEE Trans. Power Appar. Syst. 6(2), 442–449 (1991). [CrossRef]
10. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 1097–1105 (2012).
11. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).
12. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015), 1–9.
13. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016).
14. K. Hara, D. Saito, and H. Shouno, “Analysis of function of rectified linear unit used in deep learning,” in 2015 International Joint Conference on Neural Networks (IJCNN), (IEEE, 2015), 1–8.
15. K. T. Gribbon and D. G. Bailey, “A novel approach to real-time bilinear interpolation,” in Proceedings. DELTA 2004. Second IEEE International Workshop on Electronic Design, Test and Applications, (IEEE, 2004), 126–131.
16. H. Yuen, J. Princen, J. Illingworth, and J. Kittler, “Comparative study of Hough transform methods for circle finding,” Image Vision Comput. 8(1), 71–77 (1990). [CrossRef]
