Open Access
Add to BibSonomy
Abstract
High power laser ablation can lead to the creation of plasma and the emission of bright light, which can prevent the direct observation of the workpiece. Alternative techniques for enabling the visualization of the sample during laser machining are therefore of interest. Here, we show that the plasma created during laser ablation, when viewed perpendicular to the sample surface, contains information regarding the appearance of the sample. Specifically, we show that deep learning can predict the 2D appearance of the sample, directly from 2D projected images of the plasma produced during single pulse femtosecond laser ablation. In addition, this approach also enables the identification of the pulse energy of the most recent laser pulse used to machine the sample. This work could have applications across laser materials processing in research and industry, in cases where there is a requirement for real-time visualization of the sample surface during laser ablation.
Published by Optica Publishing Group under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.
1. Introduction
Laser ablation is a technique that involves using laser energy to remove material from a solid surface [1–3]. The process entails directing a laser beam onto a material, which then transfers energy to the material via the absorption of photons, resulting in the potential for melting and vaporization [4,5]. Laser ablation has a wide range of applications, including cutting [6–8], modifying surfaces [9,10], patterning [11,12], material removal [13,14], and depositing thin films [15,16]. However, one of the limitations is often the inability to image the surface in real-time, since laser ablation can lead to ionization of the laser material, which emits light that can obscure the workpiece and the machined area [17–19]. As such, a method for imaging the surface during machining could be invaluable.
Deep learning is a machine learning subfield that involves using artificial neural networks with multiple layers to recognize patterns in data [20]. It has found diverse applications across many fields, including image [21,22] and speech recognition [22], natural language processing [23], and autonomous driving [24]. In the field of photonics, deep learning has been applied to tasks such as predictive modeling [25], video processing [26], and control of laser systems [27]. Deep learning has the potential to significantly enhance the capabilities of laser physics and other scientific domains, enabling more efficient data analysis and system optimization [28]. For example, deep learning has been used for modelling of fibre laser cutting [29], motion control for laser machining [30], laser welding [31,32], imaged based monitoring of femtosecond laser machining [33], laser sintering [34] and machining optimization [35].
Previous work has shown the potential for using the plasma plume created during laser ablation for monitoring of laser welding [36], and for real-time composition monitoring for additive manufacturing [37]. These results confirm that the appearance of the plasma generated during laser materials processing is correlated with the experimental conditions. In this work, we show that deep learning can enable identification of machining conditions directly from images of the plasma generated during single pulse femtosecond laser ablation of glass. Firstly, we show the ability to predict the pulse energy of the previous laser pulse used for machining the sample directly from the plasma image. Secondly, we demonstrate the capability to generate a 2D visualisation of the sample directly from the plasma image.
2. Experimental methods
2.1 Setup
Figure 1 displays a diagram of the experimental setup used in this work, in which single 190 fs laser pulses from a Light Conversion Pharos SP laser system (central wavelength of 1030 nm, maximum pulse energy of 1 mJ, repetition rate of 6 kHz) were focused onto the surface of a 0.5 mm thick glass coverslip. The coverslip sample was attached to a motorised Thorlabs XYZ translation stage, which allowed for the positioning of the sample beneath a 20x objective to ensure that the surface of the coverslip was at the imaging camera focus and the laser focus. One camera (Basler, acA4112-20uc, 1914 × 1200 pixels, colour) was positioned above the sample and was used to image the surface of the coverslip, and a second camera (Basler, daA1920-160uc, 4096 × 3000 pixels, colour) was positioned perpendicular to the laser axis and the surface of the coverslip and was used to image the plasma plume created during laser ablation.
Fig. 1. Simplified diagram of the experimental setup used for recording surface images and plasma images before, during and after single pulse laser ablation. The plasma was imaged perpendicular to laser axis. The images in the figure are experimental data.
2.2 Data collection and processing
This work was formed of two separate experiments, as shown by the schematics in Fig. 2. For the first experiment (see Fig. 2(a) for concept), two laser pulses were used for each position on the sample, with “pulse 1” having a random pulse energy in the range 0.15 to 1 mJ in steps of 0.01 mJ (where a pulse energy of 1 mJ corresponded to a power density of approximately 1.7 × 1015 W/cm2 on the sample), and subsequently the “sensing pulse” having a pulse energy of 1 mJ. Note, the sensing pulse ablates the surface of the sample and creates a plasma plume. Both pulses were incident on the sample at the same position and on an unmachined region. A camera image of the sample surface after pulse 1 was recorded (1200 ms integration time) and a 2D projected plasma image during the sensing pulse (300 ms integration time) was recorded. This process was repeated for random pulse energies for pulse 1, with each trial taking approximately 8 seconds. After each trial, the sample was translated by 75 µm to present an unmachined region. The sample was translated in a spiral formation, starting from the centre of the spiral and translated in X and Y until a 10 × 10 grid of ablated holes was produced. After completion, the sample was moved to an unmachined region and adjusted in the Z-axis to refocus. This process of creating a 10 × 10 grid was repeated five times. One neural network was trained to predict the pulse energy of pulse 1 directly from the plasma image of the sensing pulse, and a second neural network was trained to predict the 2D appearance of the sample after pulse 1 directly from the plasma image of the sensing pulse.
Fig. 2. Concepts of (a) the first experiment where a random pulse energy ablated the surface then a sensing pulse was incident onto the surface to produce plasma and (b) the second experiment where random pulse positions ablated a surface then a sensing pulse was incident on the surface to produce plasma. All images in the figure are schematics.
In the second experiment (see Fig. 2(b) for concept), only 1 mJ pulses were used. Randomly positioned holes were ablated over the surface of the coverslip, with a density ensuring that approximately 50% of the sample was ablated, with cases where two or more pulses could potentially overlap. Again, a spiral formation of a grid of 10 × 10 single pulses were incident on the surface, where a camera image (1200 ms integration time) of the surface was recorded before the pulse, and a camera image (300 ms integration time) of the plasma during the pulse was recorded. The randomness ensured that the pulses in a grid formation would be incident on a surface with a random, yet identifiable morphology. Since only information from the region ablated by the sensing pulse would likely be captured by the sensing pulse plasma, the images of the random pulse ablation were masked so that only the sensing pulse region was used in the training of the neural network. A neural network was trained to predict the 2D appearance of the randomly machined sample directly from the plasma produced from a subsequent 1 mJ pulse. All of the images of the ablated surface and the images of plasma for both experiments were cropped and resized to 256 × 256 × 3 pixels prior to being used in training the neural networks. The light used to illuminate the surface of the coverslip for imaging was blocked via a shutter when the plasma images were recorded to enable higher signal-to-noise.
2.3 Neural networks
2.3.1 Pulse energy prediction
To predict the pulse energy of the first pulse, a convolutional neural network with a regression output and 18 layers was used. The first layer was the plasma image from the sensing pulse, and the final layer gave a numerical output that quantified the pulse energy of the first pulse. The initial learn rate was 0.0002, with a learn rate drop factor of 0.1 and period of 50, whilst the minibatch size was 2. The neural network was trained for 10 epochs. A total of 365 images were used in the training of the network, with a 90:10 split for training/validation, with the testing carried out on 25 images previously unseen by the neural network.
2.3.2 Ablated surface image generation
For each of set of experiments carried out, a separate convolutional generative adversarial neural network (cGAN) was trained to transform images of plasma into images of the ablated surface. In the first case, images of the plasma from the sensing pulse were transformed into images of the surface after the first pulse. In the second case, images of the plasma from the pulse were transformed into images of the surface that had randomly positioned ablated holes. The cGAN used in this work was based on the pix2pix architecture [38]. Both networks were trained for 100 epochs, using a minibatch size of 2, and a generator and discriminator learn rate of 2 × 10−4. The optimizer used was ADAM [39]. For the first case, 415 pairs of images were used for training, and for the second case, 475 pairs of images were used for training. All test images were not present in any of the training data.
2.3.3 Neural network training hardware
All neural networks were trained and tested on a computer that had an Intel Xeon Gold 5222 CPU @3.80 GHz and 3.79 GHz, 192 GB RAM and 3x NVIDIA A4050s (60 GB total).
3. Results and discussions
3.1 Pulse energy prediction
The capability of the neural network to predict the pulse energy of the first pulse from the plasma image from the subsequent pulse is shown in the Fig. 3(a). The R2 of the best fit is 0.9767, with a root-mean-square-error of 0.033 mJ, highlighting the accuracy of the neural network. Figure 3(b) shows the neural network activation map at the dropout layer (the higher the intensity, the greater the feature weighting in the neural network) and confirms that different channel features contribute to the predicted value of the pulse energy. In some channels the plasma near the surface has high activation, whilst in other cases the signal corresponding to the expansion of the plume further away from the surface plays a role.
Fig. 3. (a) The capability of the neural network to predict pulse energy directly from plasma images and (b) activation maps at the dropout layer when feeding a 0.3 mJ pulse through different channels.
3.2 Surface visualization for random pulse energy
Figure 4 shows the results of generating images of an ablated surface that was machined with a random pulse energy, directly from the plasma image from a subsequent pulse with energy of 1 mJ. The plasma images in the figure therefore correspond to a pulse energy of 1 mJ, and the label in each plasma image shows the pulse energy of the previous pulse. The actual and generated images correspond to the appearance of the surface prior to the 1 mJ pulse ablation. The comparison row shows the differences between the actual and generated images. The structural similarity index measurement (SSIM), which is a measurement of the similarity between two images (with a value of 1 meaning the images are identical), was calculated for each of the generated image and associated actual image pairs. The SSIM was calculated for 0.22 mJ, 0.42 mJ and 0.97 mJ laser pulse energies, and gave values of 0.67, 0.69 and 0.71, respectively. It is evident that the generated and actual images are very similar, with the greatest difference being the splatter and the rim of the ablation hole, as shown clearly in the comparison images.
Fig. 4. The capability of the neural network to generate images of the surface resulting from a pulse with a randomly chosen pulse energy, directly from an image of the plasma produced when a subsequent pulse was incident on the sample.
Figure 5 shows the result of passing an input image through a specific layer of the neural network, from the input image (layer 1) to the output image (layer 54). The images are grayscale as only the first channel is shown. The figure illustrates the transformation process, occurring predominantly due to the convolutional filters, in reducing the image down to a 2 × 2 pixel image and subsequently back to the original image size.
Fig. 5. Images of the first channel of each numbered layer as the result of feeding an input image (layer 1) into each numbered layer. The figure shows the transition from a plasma image to the generated surface image.
3.3 Surface visualization for random pulse positions
Figure 6 shows the process used for masking the experimentally recorded surface images, in preparation for training of the neural network. The figure shows how an example region of the randomly ablated surface is masked to only include the region of the sample that would be ablated by the subsequent 1 mJ pulse. As stated in the experimental section, a smaller masked circle was used for image generation to account for the potential variability in the size and shape of the 1 mJ pulse.
Fig. 6. The process for masking the experimentally recorded surface images, showing the position of the 1 mJ pulse (red dashed circle) and the masked region (blue dashed circle).
Figure 7 shows the capability of a neural network to generate surface images of a randomly ablated surface directly from an image of the plasma (column 1) from the subsequent 1 mJ pulse. The SSIM was calculated for the generated (column 2) and actual images (column 3) and had values of (a) 0.9029, (b) 0.9132, (c) 0.9077, (d) 0.8325 and (e) 0.8982. The fourth column in the figure shows the difference between each pair of generated and actual images, with darker pixels indicating a lower difference between the images. The generated images show similar features, namely, two lines in (a), the structure in the top right of (b), curved line and straight line at the centre of (c), line at the top left of (d), and two lines in (e). However, the locations of these structures are not exact, and some features are missing, as highlighted in the difference images.
Fig. 7. The capability for visualization of a randomly ablated sample directly from the image of the plasma created from a subsequent pulse with energy of 1 mJ.
Figure 8 shows examples of images of plasma (top row) resulting from the ionisation of (a) air, (b) cyanoacrylate and (c) non-porous carbon tape for 1 mJ pulses, along with associated images of the ablated surfaces (bottom row). The visible plasma emission for these examples provides strong evidence that the technique demonstrated in this work could be extended to many other materials.
Fig. 8. Images of plasma emission (top row) and associated images of the ablated surfaces (bottom row) for (a) air, (b) cyanoacrylate and (c) carbon tape.
4. Conclusion
In conclusion, we have shown the capability of deep learning for the identification of pulse energies, and for the visualization of the ablated sample, directly from images of plasma recorded during single pulse femtosecond ablation of glass. Specifically, we have shown that neural networks can enable the identification of the pulse energy of the most recent laser pulse used to machine the sample, and can enable predictive visualization of the surface morphology of a sample directly from 2D projected images of plasma. This work could have applications across laser materials processing in research and industry, particular in cases where there is a requirement for real-time visualization of the sample surface during laser ablation but direct observation of the sample is not possible due to it being obscured, for example by plasma.
Funding
Engineering and Physical Sciences Research Council (EP/P027644/1, EP/T026197/1, EP/W028786/1).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are available in Ref. [40].
References
1. E. G. Gamaly, A. V Rode, B. Luther-Davies, and V. T. Tikhonchuk, “Ablation of solids by femtosecond lasers: Ablation mechanism and ablation thresholds for metals and dielectrics,” Phys. Plasmas 9(3), 949–957 (2002). [CrossRef]
2. R. E. Russo, X. Mao, H. Liu, J. Gonzalez, and S. S. Mao, “Laser ablation in analytical chemistry-a review,” Talanta 57(3), 425–451 (2002). [CrossRef]
3. S. Nolte, C. Momma, H. Jacobs, A. Tünnermann, B. N. Chichkov, B. Wellegehausen, and H. Welling, “Ablation of metals by ultrashort laser pulses,” J. Opt. Soc. Am. B 14(10), 2716–2722 (1997). [CrossRef]
4. L. Cerami, E. Mazur, S. Nolte, and C. B. Schaffer, “Femtosecond Laser Micromachining,” in Ultrafast Nonlinear Optics, R. Thomson, C. Leburn, and D. Reid, eds. (Springer International Publishing, 2013), pp. 287–321.
5. K. Garasz, M. Tański, M. Kocik, E. E. Iordanova, G. P. Yankov, S. Karatodorov, and M. Grozeva, “The Effect of Process Parameters in Femtosecond Laser Micromachining,” in (2016).
6. A. Wetzig, P. Herwig, J. Hauptmann, R. Baumann, P. Rauscher, M. Schlosser, T. Pinder, and C. Leyens, “Fast laser cutting of thin metal,” Procedia Manuf 29, 369–374 (2019). [CrossRef]
7. A. N. Fuchs, M. Schoeberl, J. Tremmer, and M. F. Zaeh, “Laser cutting of carbon fiber fabrics,” Phys. Procedia 41, 372–380 (2013). [CrossRef]
8. J. S. Shin, S. Y. Oh, H. Park, C. M. Chung, S. Seon, T. S. Kim, L. Lee, B. S. Choi, and J. K. Moon, “High-speed fiber laser cutting of thick stainless steel for dismantling tasks,” Opt. Laser Technol. 94, 244–247 (2017). [CrossRef]
9. Y. S. Tian, C. Z. Chen, S. T. Li, and Q. H. Huo, “Research progress on laser surface modification of titanium alloys,” Appl. Surf. Sci. 242(1-2), 177–184 (2005). [CrossRef]
10. N. Jeyaprakash, C.-H. Yang, and D. R. Kumar, “Laser Surface Modification of Materials,” in Practical Applications of Laser Ablation (IntechOpen London, UK, 2020).
11. D. J. Heath, J. A. Grant-Jacob, M. Feinaeugle, B. Mills, and R. W. Eason, “Sub-diffraction limit laser ablation via multiple exposures using a digital micromirror device,” Appl. Opt. 56(22), 6398–6404 (2017). [CrossRef]
12. J. M. Solomon, S. I. Ahmad, A. Dave, L.-S. Lu, F. HadavandMirzaee, S.-C. Lin, S.-H. Chen, C.-W. Luo, W.-H. Chang, and T.-H. Her, “Ultrafast laser ablation, intrinsic threshold, and nanopatterning of monolayer molybdenum disulfide,” Sci. Rep. 12(1), 6910 (2022). [CrossRef]
13. C. Seo, D. Ahn, and D. Kim, “Removal of oxides from copper surface using femtosecond and nanosecond pulsed lasers,” Appl. Surf. Sci. 349, 361–367 (2015). [CrossRef]
14. G. Daurelio, G. Chita, and M. Cinquepalmi, “Laser surface cleaning, de-rusting, de-painting and de-oxidizing,” Appl. Phys. A 69(7), S543–S546 (1999). [CrossRef]
15. J. A. Grant-Jacob, S. J. Beecher, J. J. Prentice, D. P. Shepherd, J. I. Mackenzie, and R. W. Eason, “Pulsed laser deposition of crystalline garnet waveguides at a growth rate of 20 µm per hour,” Surf. Coat. Technol. 343, 7–10 (2018). [CrossRef]
16. M. N. R. Ashfold, F. Claeyssens, G. M. Fuge, and S. J. Henley, “Pulsed laser ablation and deposition of thin films,” Chem. Soc. Rev. 33(1), 23–31 (2004). [CrossRef]
17. G. K. Lewis and R. D. Dixon, “Plasma monitoring of laser beam welds,” Laser 13, 2 (1984).
18. Y. Kawahito, N. Matsumoto, M. Mizutani, and S. Katayama, “Characterisation of plasma induced during high power fibre laser welding of stainless steel,” Sci. Technol. Weld. Joining 13(8), 744–748 (2008). [CrossRef]
19. J. Greses, P. A. Hilton, C. Y. Barlow, and W. M. Steen, “Plume attenuation under high power Nd: YAG laser welding,” in International Congress on Applications of Lasers & Electro-Optics (Laser Institute of America, 2002), 2002(1), p. 47727.
20. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]
21. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in ICLR 2015 : International Conference on Learning Representations 2015 (2015).
22. K. Noda, Y. Yamaguchi, K. Nakadai, H. G. Okuno, and T. Ogata, “Audio-visual speech recognition using deep learning,” Appl. Intell. 42(4), 722–737 (2015). [CrossRef]
23. T. Young, D. Hazarika, S. Poria, and E. Cambria, “Recent trends in deep learning based natural language processing,” IEEE Comput. Intell. Mag. 13(3), 55–75 (2018). [CrossRef]
24. E. Talpes, D. D. Sarma, G. Venkataramanan, P. Bannon, B. McGee, B. Floering, A. Jalote, C. Hsiong, S. Arora, A. Gorti, and G. S. Sachdev, “Compute Solution for Tesla’s Full Self-Driving Computer,” IEEE Micro 40(2), 25–35 (2020). [CrossRef]
25. J. Francis and L. Bian, “Deep learning for distortion prediction in laser-based additive manufacturing using big data,” Manuf. Lett. 20, 10–14 (2019). [CrossRef]
26. D. Buongiorno, M. Prunella, S. Grossi, S. M. Hussain, A. Rennola, N. Longo, G. Di Stefano, V. Bevilacqua, and A. Brunetti, “Inline defective laser weld identification by processing thermal image sequences with machine and deep learning techniques,” Appl. Sci. 12(13), 6455 (2022). [CrossRef]
27. T. Baumeister, S. L. Brunton, and J. Nathan Kutz, “Deep learning and model predictive control for self-tuning mode-locked lasers,” J. Opt. Soc. Am. B 35(3), 617–626 (2018). [CrossRef]
28. B. Mills and J. Grant-Jacob, “Lasers that learn: the interface of laser machining and machine learning,” IET Optoelectron. 15(9), 207 (2021). [CrossRef]
29. A. F. Courtier, M. McDonnell, M. Praeger, J. A. Grant-Jacob, C. Codemard, P. Harrison, B. Mills, and M. Zervas, “Modelling of fibre laser cutting via deep learning,” Opt. Express 29(22), 36487–36502 (2021). [CrossRef]
30. Y. Xie, M. Praeger, J. A. Grant-Jacob, R. W. Eason, and B. Mills, “Motion control for laser machining via reinforcement learning,” Opt. Express 30(12), 20963–20979 (2022). [CrossRef]
31. M. Schmitz, F. Pinsker, A. Ruhri, B. Jiang, and G. Safronov, “Enabling Rewards for Reinforcement Learning in Laser Beam Welding processes through Deep Learning,” in 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA) (2020), pp. 1424–1431.
32. J. Günther, P. M. Pilarski, G. Helfrich, H. Shen, and K. Diepold, “First Steps Towards an Intelligent Laser Welding Architecture Using Deep Neural Networks and Reinforcement Learning,” Procedia Technology 15, 474–483 (2014). [CrossRef]
33. B. Mills, D. J. Heath, J. A. Grant-Jacob, Y. Xie, and R. W. Eason, “Image-based monitoring of femtosecond laser machining via a neural network,” JPhys Photonics 1(1), 015008 (2018). [CrossRef]
34. J. Tang, X. Geng, D. Li, Y. Shi, J. Tong, H. Xiao, and F. Peng, “Machine learning-based microstructure prediction during laser sintering of alumina,” Sci. Rep. 11(1), 10724 (2021). [CrossRef]
35. M. D. T. McDonnell, D. Arnaldo, E. Pelletier, J. A. Grant-Jacob, M. Praeger, D. Karnakis, R. W. Eason, and B. Mills, “Machine learning for multi-dimensional optimisation and predictive visualisation of laser machining,” J. Intell. Manuf. 32(5), 1471–1483 (2021). [CrossRef]
36. L. Li, D. J. Brookfield, and W. M. Steen, “Plasma charge sensor for in-process, non-contact monitoring of the laser welding process,” Meas. Sci. Technol. 7(4), 615–626 (1996). [CrossRef]
37. L. Song, W. Huang, X. Han, and J. Mazumder, “Real-time composition monitoring using support vector regression of laser-induced plasma for laser additive manufacturing,” IEEE Trans. Ind. Electron. 64(1), 633–642 (2016). [CrossRef]
38. P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2017), pp. 5967–5976.
39. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv, arXiv:1412.6980 (2014). [CrossRef]
40. J. A. Grant-Jacob, B. Mills, and M.N.. Zervas, Dataset to support the publication “Visualizing laser ablation using plasma imaging and deep learning,” University of Southampton, (2023). [CrossRef]
