1. Introduction

Femtosecond laser machining involves a series of complex processes that must be accounted for to produce an accurate model. One of the main steps is the absorption of energy by the machined material, specifically multi-photon absorption and the subsequent avalanche ionisation. This subject is normally simplified by the use of a two-temperature model, modelling the work-piece material as two systems, valence electrons and the lattice [–4]. The process of photon absorption happens extremely rapidly (during the laser pulse), however, the transfer of energy from excited electrons to the lattice via electron-phonon coupling is slower and is characterised by the energy relaxation time of the target material. Until the electrons and lattice return to thermal equilibrium, they can be considered as two connected systems with different temperatures (i.e. the two temperature model). While a simple version of this model is easy to implement, it does not contain many of the mechanisms seen when using femtosecond lasers. For a full model, incorporating features such as sample material phase-changes, the concomitant fluid and plasma dynamics, and eventual material redeposition as debris is non-trivial but essential [5–7]. Photon-atom interactions have formed the basis of approaches by others for a strict theoretical understanding [8–10]. Scaling these models to three-dimensions with multiple incident pulses, with all the complexity found in these interactions is, therefore, a large task to undertake.

In this study, multiple pulses with computer-controlled, spatial intensity profiles are used, and this would further complicate the challenge of creating a conventional model of this experiment. To expand the examined volume up to experimentally useful dimensions would require either an unrealistic level of computing power or simplifying assumptions to be made. The fact that the sample could be exposed to more than one pulse increases the complexity because the surface will only be uniform and smooth before the arrival of the first laser pulse. After the first pulse, several changes will have occurred, the simplest of these being that the surface will no longer be smooth and perpendicular to the beam, and depending upon the material, melting and potentially re-solidification into a different allotrope or chemical reactions (such as oxidisation) could take place. Taking the example of a typical Gaussian laser spot, the resultant feature after machining is expected to approximately resemble a crater. The sloping bottom of this crater would cause non-normal incidence for subsequent laser pulses, changing the detail of the light-matter interaction including introducing complex, possibly multiple, reflections. Other factors could include changing material properties, change in surface position compared to optimal focus, and the material temperature [11,12].

Neural networks (NNs) offer an alternative approach to modelling these complex interactions. They have been used in a wide range of laser machining tasks such as the real time monitoring of beam aberrations [13] and preventing over machining [14] to predicting hardness distribution for heat-treated steel when machined with a 2kW laser [15]. Outside of laser machining of metals, neural networks have been used across industries from predicting the appearance of dough when browned with a CO$_2$ laser [16] to controlling optical tweezers [17]. Neural networks can also be used to both optimise laser machining parameters and accurately simulate the result of machining blind holes [18]. Previously, we have demonstrated a deep learning approach for the simulation of a sample surface after a single ultrafast laser pulse with a spatially modulated intensity profile, via the use of a digital micro-mirror device (DMD), both in the form of SEM images [19] and 3D depth profiles [20]. Experimental examples of spatially shaped intensity profiles and their resulting interferometrically depth-mapped structures, in electroless nickel, were used with a deep learning approach to train a neural network. This NN was then used to predict the sample’s appearance after machining with binary intensity profiles that it had not seen. This approach required no coding of the physical principles involved in laser machining, a distinct advantage of the technique when attempting to model such a complex process. We have also shown an extension of this principle and demonstrated a neural network that could predict the machining result after multiple overlapping exposures, where the intensity profile was unique during each exposure [21]. For an in-depth overview of the use of machine learning across the laser machining industry, consult [22].

Relationships between dataset domains come in different forms including one-to-one, one-to-many, and many-to-one. In the example of neural networks, there will usually be two datasets in different domains, one domain containing the input data from the network, and the other containing the output data. In a one-to-one mapping, each input data item has a corresponding output data item. An example of this is multiplication of two prime numbers. Each pair of primes will produce a single number, and there is no other way to reach that number with any two other integers. Taking the product of any two numbers that includes a non-prime number is an example of a many-to-one mapping. Each pair of numbers will produce exactly one answer but there are multiple such pairs that will produce that same answer. The one-to-many case is the same as the many-to-one case but reversed, such as trying to find factor pairs of a number.

In the case of transforming from a sequence of DMD-patterned laser pulses to the resultant 3D surface profile, the relationship is approximately many-to-one, excepting minor variability due to instability of the beam profile and inhomogeneity of the target material. This transformation can be found using techniques such as optical filtering to account for diffraction. The same is not true in the reverse case; to produce a specific desired 3D surface profile, there are many DMD patterns, including their sequencing, that could achieve the same result, and hence identifying one of these sequences of patterns for a given 3D profile is a one-to-many problem. In addition to the one-to-many relationship, the network must learn features of the system such as the machining depth, diffractive effects, and beam inhomogeneities. This presents a particular problem to neural networks as paired data becomes less effective, and a correct solution is harder to distinguish from a wrong one.

Our primary motivation for this work was to apply our previously demonstrated network (i.e. DMD pattern to depth profile) [21] as a secondary network to assist in the training of the primary neural network (i.e. depth profile to DMD pattern). The technique of using a neural network that models an experiment, to assist in the training of a second neural network is an exciting step, which establishes the potential for bootstrapping the capability of neural networks for increasingly complex modelling.

2. Data collection

The first step in training a neural network is the collection of a suitable dataset. This work aimed to transform the desired depth profile into the pattern (i.e. spatial intensity profile), or sequence of patterns, required to produce it. Data collection consisted of two processes; the first was producing structured laser machined patterns, and the second was recording their depth. The machined features were produced using laser machining apparatus that had the capability to project computer-controlled binary intensity profiles, spatially modulated via a DMD, onto the work-piece. To measure, the resultant features a 3D surface profiler using white light interferometry was used. A schematic of the setup can be seen in Fig. 1 along with an example DMD pattern and machined surface profile pair. Depth values in the 3D depth profiles are in nanometers, with the surface of the work-piece set to a height of 0, meaning that negative values reference machined areas while positive ones denote debris or machining burr.

 

Fig. 1. Experimental schematic used to collect all training and testing data. Showing a) schematic of the laser machining setup, b) illustration of the sample characterisation method, c) the process for generating depth profiles using the network demonstrated in [21], and d) the process to generate DMD sequences using the novel network discussed in this work.

Download Full Size | PDF

To produce the shaped pulses a digital micro-mirror device with a spatial resolution of 840x480 pixels was used as a spatial light modulator. DMDs have been used as spatial light modifiers in both traditional laser machining and holography tasks [23,24], and also in the task of two photon microscopy [25]. Using the DMD as a spatial light modulator required a beam with a top-hat profile and this was achieved using a $\pi$Shaper 6_6. The DMD was used to spatially shape individual 150 fs, 800 nm, 1 mJ laser pulses from a Ti:sapphire amplifier. The energy of the pulses was reduced using paired neutral density filters down to 200 $\mathrm {\mu }$J. After these neutral density filters, further energy was lost, including losses through the $\pi$Shaper, ooverfilling of the DMD (to ensure a top hat profile), and uncaptured diffraction orders resulting in a fluence of $\sim$0.5 Jcm$^{-2}$ after being imaged onto the work-piece via a 50x microscope objective lens. In this work, the laser was operating in single pulse mode (i.e. pulses on demand) at approximately one pulse per second. The low repetition rate meant that thermal accumulation and other incubation effects were not present in the training data, and hence would not be learned by the neural network in this case.

This setup has a range of uses that can be found elsewhere [26–28]. An electroless nickel mirror (5 $\mathrm {\mu }$m electroless nickel layer deposited on copper, LBP Optics Ltd.), which had an amorphous structure and was chosen to reduce grain boundary effects, was used as the work piece. A Zygo Zescope white light interferometer was used to measure the depth profiles (accurate to <1 nm) after all desired structures had been laser machined.

While the DMD had a fixed size, the spatial resolution on the work-piece was determined by the magnification of the optical system. The mirrors of the DMD were approximately 7 $\mathrm {\mu }$m x 7 $\mathrm {\mu }$m in size but were projected down to a theoretical size of $\sim$90 nm (although the diffraction limit meant that individual pixels were not resolvable). In total each structure, formed by up to three uniquely shaped pulses, measured a maximum of $\sim$ 30 $\mathrm {\mu }$m across. The patterns used to shape the pulses were randomly generated, created by combining line segments, circles, and arcs of varying sizes. Examples of DMD patterns are shown in Fig. 2 where the binary mask for each pulse is shown individually, and finally all three are shown in a single image, with each pulse represented by a single colour channel. Rather than displaying each pulse in the sequence separately, this approach displays the sequence by overlaying them, alowing easy comparison between distribution and ordering. The colours of the RGB representation follow standard RGB image rules, so for example, an area that is only machined in the first pules, represented by the red channel, would be red. An area machined during both pulse 1 and pulse 2, represented by the green channel, would instead show as yellow. Figure 2 row 3 demonstrates the break down of colours and this convention is used in all subsequent figures where the sequence is displayed as a single image. In a third of cases, the images were inverted, meaning the area surrounding the shapes was machined rather than the shapes themselves.

 

Fig. 2. Schematic showing the principle used for visualisation of multiple spatial intensity profiles throughout this manuscript, showing columns a-c) spatial intensity profiles formed on the DMD, and d) an RGB visualisation of these three spatial intensity profiles.

Download Full Size | PDF

The minimum size of each shape was random, ranging from a projected size of 2.7 $\mathrm {\mu }$m down to $\sim$360 nm, which is below the diffraction limit of the system. The microscope objective used for focusing has an NA of 0.42 and the central wavelength of the laser is 800 nm giving an Abbé diffraction limit of $\sim$952 nm. As all experimental data was collected under these conditions and the neural network learns to incorporate this physical property into its predictions.

3. Application of the neural network

In 2014, I. Goodfellow released a paper [29] on a technique called generative adversarial networks (GANs), which is an effective method for generating random images with a particular style (e.g. images of human faces) or even transforming one style of image to another (e.g. turning semantic maps into photo-realistic images) [30,31]. GANs are composed of two different networks, where one is the generator and the other is the discriminator. This relationship can be thought of as similar to that between a counterfeiter (the generator) trying to create fake currency, and the police (the discriminator) trying to detect the fakes. Both start off not knowing anything about what real or fake currency is, but are trained simultaneously. The counterfeiter aims to make fake currency that will fool the police, while the police want to be able to detect fakes with 100% certainty. The end goal of the training process is to achieve Nash equilibrium where neither network can improve their ability to beat the other.

Often generative networks aim to produce realistic and visually pleasing images, such as producing faces that are hard to distinguish from real ones [30]. However, the network discussed here aimed to produce the DMD patterns needed to produce a certain machined surface profile. In this case, the overall appearance of the DMD patterns is less important than the accuracy of the surface profile produced when machining with the generated pattern. This means that conventional losses utilised in GANs (see a and b in Fig. 4) were not effective, and so, an alternative method was chosen.

When working with many-to-one or many-to-many domains, the dataset cannot be formatted as paired data. The type of network most commonly used when this occurs is called a cycle-consistent generative adversarial network (CycleGAN) [32]. The most common use for a CycleGAN is for style transfer [33,34], where there is a desire to link two different domains. The unique feature of CylcleGANs is that they use two separate generative networks, each transferring from one domain to the other. In this type of network, losses are calculated after applying both GANs, the input is transformed from one domain to another by the first GAN and then back to the original domain by the second GAN. The difference from the original input is then used to determine the performance of the network and is referred to as cycle consistency loss. In general, similarly to most generative networks, these networks are deemed to be successful when they create an output that is visually pleasing to a human observer as opposed to the accuracy required by classification or regression networks.

Another type of network used in one-to-many relationships is called a style GAN; these take several parameters as an input and produces easily manipulated output that is consistent is style with images in the training data. This method was first pioneered by Karras et al. in 2019 [35] and has since been used for many tasks including producing almost undetectable fakes of human faces [30]. Style GANs have also been extended to flexible style transfer [36] and creating realistic images from semantic maps combined with style adjustments [37]. Both CycleGANs and style GANs contain ideas that can be utilised in the network structure and loss functions required for the task undertaken in this manuscript.

The architecture chosen for the generator in this work is an up-scaling convolutional network, formed of a series of convolution blocks followed by x2 nearest neighbour up-scaling layers, starting at 4 pixels square, and ending with images of 512 pixels square. Each convolution block follows the same structure as the SPADE ResBlocks demonstrated in the SPADE neural network [37] with the resized depth profile, concatenated with the reshaped output of the mapping block, forming the image input. The mapping block was formed of 4 dense layers taking an input of the weighting parameters and noise concatenated. The inclusion of a noise input allowed for the network to be able to perform one-to-many transformations and the concatenation rather than addition allowed for this. The output from this layer was then reshaped to the 4-pixel square shape required. After the completion of all convolution blocks there was a single convolution layer to form the output of the network. This structure can be seen in Fig. 3(a). In the first SPADE ResBlock, the network used 1024 filters, reducing to 32 filters for the final layers that were operating on 512 square pixel tensors. An ADAM optimiser with a learning rate of 1e-5 was used to train the network over 50 epochs.

 

Fig. 3. The network structure for the generator (a) and discriminator (b) along with the flow of data through the networks.

Download Full Size | PDF

The network as designed only accepts the weighting vector as an input and to produce it as an output would require a different network structure. Our motivation here was to devise a method for generating sequences of DMD patterns, rather than a direct optimisation tool.

The network also takes noise an an input to the network. When training neural networks, there are two reasons to include noise in the model. The first is to prevent overfitting. In this case the noise is summed with the input so that the network is not repeatedly shown the same values. This noise should only be included during training.

The second reason to include noise is to allow for output variation. Given two identical inputs the network will produce two identical outputs. The noise allows for identical chosen weightings and depth profiles to produce different outputs as demonstrated in section 3.2 and is especially important with this network representing a one-to-many relationship. One common way to include noise as a variable is to concatenate it with other inputs.

While noise does serve a different purpose to the other inputs, this is not as extreme as it might appear at first. While there is a direct link between pixels in the DMD patterns and machined areas, the influence of the noise input is more subtle and might cover factors that are independent of the sequence of DMD patterns used, such as the surface profile of unmachined areas.

The discriminator has a simpler structure (Fig. 3(b)) taking inputs of both a sequence of DMD patterns and the weighting vector. The sequence of DMD patterns is stepped down in size through a series of convolution layers with a stride of two until the tensors have a size of 32x32. The final of these strided convolution layers has a filter size of one creating a tensor of dimensions 32x32x1, which is then flattened to form a 1024x1 vector. The weighing vector passes through a dense layer before being concatenated with the flattened convolution step. This combined vector then passes through two further dense layers, the final one having a single output. This single output corresponds to a prediction whether the input-output pair corresponds to experimental or a combination of experimental and generated data.

3.1 Training loss comparison

In general, networks trained on paired data perform better than those trained on unpaired data [38], and in the case of a traditional GAN vs a CycleGAN are quicker to train as they require fewer networks. We propose a combination of the two techniques, taking a partial CycleGAN where one network is trained as a traditional GAN, and another is trained with the aid of cycle consistent loss. The traditional GAN in this work is the multiple pulse, DMD to depth profile network demonstrated previously [21]. A new network was then trained to transform a depth profile into a DMD pattern, referred to as the primary network.

In this network, losses were assessed using three methods, as shown in in Fig. 4. In this figure, X refers to the depth profile, W refers to the pulse weighting, Y refers to the sequence of DMD patterns, and Z is the noise input into the network. Where two variables meet at a blank circle, the two tensors were passed as separate parameters, however, where they meet at a circle inscribed with a plus, the two inputs were concatenated. The curved, dashed arrow in Fig. 4(a) after the generator output represents a switching junction, where only one of the two paths, either the generator output or Y, was used. The initial generator in each example is the same network, duplicated in the figure for clarity.

 

Fig. 4. Network loss configurations. a) Discriminator (GAN) loss, b) Comparative (mean absolute error) loss, c) Cycle consistent loss. X and W are the network inputs (the depth profile and pulse weighting vector), Y is the experimental output target (the sequence of DMD patterns), and Z is noise.

Download Full Size | PDF

The first of the three losses was a discriminator (GAN) loss (Fig. 4(a)). In this loss, a second network, the discriminator, takes an input of X and one of either the output of the GAN or the experimental output data paired with the input and predicts whether they are a real data pair or not. The accuracy of this result is then used to train both the generator and discriminator. The second loss uses direct comparative losses, in this case the mean absolute error, between the generated and desired output (Fig. 4(b)) to provide a numerical error to the generator. The third loss used was cycle consistency loss (Fig. 4(c)). Cycle consistency loss is applied by passing the predicted DMD image back through the secondary (DMD-to-depth) GAN to produce a prediction of the depth profile which can then be compared with the input (the experimentally measured depth profile). As the secondary network has already been trained, only the generator of that GAN is needed and no updates (training steps) are performed on that network. For full details of this generator, its structure and how it was trained please see the paper by McDonnell et. al. [21]. These losses were combined by applying, and adjusting, a weighting to each, controlling the impact each loss has on the training of the networks, before summing them together to give a total loss.

At the start of training the comparative loss was given the most weight, compared to the discriminator and cycle losses, at a ratio of 10:1:1 respectively to encourage the pixel distribution of the generated DMD sequences to match that of the true values. By the end of training this was reduced to allow for the one-to-many nature of the data, with a final loss weighting between comparative, discriminator, and cycle losses at a ratio of 1:5:5. To calculate the loss at each stage the total weightings were normalised to 1 and then the resulting individual losses summed to provide a single loss value.

Once trained, the network can also be used to create novel DMD patterns by using the method shown in Fig. 5(b). Essentially, the neural network is now capable of answering the problem; what DMD pattern is required in order to achieve the desired 3D depth profile.

 

Fig. 5. The process for creating and testing the trained NN-generated DMD profiles, a) from experimental results, b) from a desired depth profile.

Download Full Size | PDF

A demonstration of the effectiveness of each network can be seen in Fig. 6 where the outputs from both a standard GAN and the novel combined approach are shown This figure used the same visualisation approach as Fig. 2, with each colour channel in the DMD patterns representing a separate pulse. As can be seen, there is a large amount of noise in the DMD pattern produced by the traditional GAN, specifically around the edges of each feature. Additionally, there are also patches of noise in spots across the image where no machining takes place. The DMD pattern produced by the hybrid GAN shows none of these features, and hence is the approach used in this work

 

Fig. 6. Comparison of a traditional GAN and the novel, hybrid, combined Style GAN and CycleGAN. The sequence of DMD patterns follow the colour convention shown in Fig. 2

Download Full Size | PDF

3.2 Exploring variance between network predictions

As stated, one of the difficulties faced when creating a network to tackle this problem is the one-to-many nature of the calculation, 3D depth profile to multiple DMD sequence, while also having measurable wrong answers. Figure 7 demonstrates how differing DMD patterns can all produce very similar depth profiles, with a mean absolute error (MAE) of 1.67 nm. In the three examples shown in Fig. 7 b), c) and d), the total number of white pixels across all three pulses remains almost constant. Despite this the distribution of white pixels (areas of laser machining) between pulses varies greatly. This can be seen especially in a comparison between Fig. 7 a) and b) were the amount of machined area in pulse 2 of b) is much higher than in a), with the trade off that less of the material is machined during pulses 1 and 3. Figure 7 c) is more similar to a) although there are differences, such as in pulse 3 where there are greater areas where the work piece is not machined. Each combination of pulses corresponds to a different random seed used to create the noise input for the network. This noise allows the network to produce different outputs from a single depth profile and weighting pair, in order to simulate the one-to-many relationship of the dataset.

 

Fig. 7. a) shows three pulses and a corresponding experimental depth profile. b-d) represent three predictions from the primary network from the experimental depth in a). Each of b-d) was generated using a different noise input.

Download Full Size | PDF

3.3 Generated data

One way to improve the results of a machine learning task is to increase the amount of data available for training [39]. Outside of simply collecting more data, the most common technique in classification problems is data augmentation [40,41] to simulate having collected more data. While this is a way to reduce the effects of overfitting it is not always suitable to use with generative networks. One reason for this is that the supplied images form the basis for the data distribution that the network tries to achieve. Augmenting the training data may mean that the network learns to produce images that also contain the augmented features that do not match experimental results. With laser machining in particular, some common augmentation techniques, scaling and rotation, would lose information from the final result such as the diffraction limit and beam properties.

While there have been other methods devised to improve network capability with limited data [42,43], most of them are situation-specific and/or require much fine-tuning, especially for generative networks. Even those that are robust still require images on the order of thousands [44], leading to unfeasibly large data collection periods. Another option is to produce additional data that matches the distribution of the experimental data via an alternative method.

As the use of augmented data was not feasible, for this work, a novel approach was taken, made possible by the use of two networks with one being easier to train than the other. This method involved using a pre-trained network able to convert DMD patterns into depth profiles. This was proven to be effective [21] and so can be used to simulate a large experimental dataset with little time investment. While the data could not be used to further train the original network, it could be used to train the neural network performing the reverse task while keeping the recorded experimental data as a validation set to measure its performance. This allowed for the creation of a dataset with 5000 samples in approximately 20 seconds, an improvement over the original 200 experimental data pairs.

4. Validation of the neural network

While the primary network was trained using generated examples, to truly test the accuracy of the network, it must be tested on real experimental data. The dataset used to test the network was the validation dataset from [21], meaning that the data had not been used to train any of the networks involved. Along with randomly generated patterns, user-defined structures such as grids and letters were included in this dataset.

Figure 8 shows three examples, each with two comparisons used to train the GAN, the DMD patterns produced by the network, and the depth profile that would be machined using them. When comparing the DMD patterns two factors can be judged, the shape of each layer, and the layer where machining takes place. While the shape of the DMD patterns is an easy comparison for humans to make, as discussed in the application of the neural network, it is ultimately not what this network is trying to optimise. A similar statement can be made for the position of machining, excepting cases where the ordering of pulses doesn’t matter. In this case, it is generally better to machine in the least number of pulses possible to reduce the total time taken. In contrast to this, a direct comparison between the experimentally measured depth profile and the one predicted from the generated DMD is useful. As the translation from DMD pattern to depth profile is not exactly one-to-one, due to factors such as laser pulse energy fluctuations and inhomogeneities in the sample, it is expected that there will be some difference between the predicted and experimental surface profiles. Despite this, it is still useful to make this comparison as the aim of the network is to be able to machine the desired depth profile.

 

Fig. 8. Three example DMD sequences generated from a single depth profile with input weightings defined as [pulse 1, pulse2, pulse 3]. Row a) was generated using a pulse weighting of [0, 0, 1], row b using a weighting of [1, 0, 0], and row c using a weighting of [1/6, 1/3, 1/2] In each case the experimental depth profile, as well as the depth profile produced as a result of the generated DMD patterns is shown. In addition the difference between the two depth profiles is included for each case in both a spatial and histogram format. The sequence of DMD patterns follow the colour convention shown in Fig. 2

Download Full Size | PDF

To better judge the effectiveness of the neural network, it is important to look in detail at the depth profiles resulting from the generated DMD patterns. As can be seen in Fig. 8 the predicted depth profiles show a high correlation to the measured ones, with all major features present and DMD patterns looking similar when using a single pulse. Most differences are observed when the features are below the diffraction limit, where the feature may not be present, or additional features can be seen, in the generated patterns. The edges of the straight sections also show some deviations.

To get a measurable error, first, the neural network predicted the DMD patterns required to produce the depth profiles that make up the validation dataset, and those DMD patterns were then converted to depth profiles again using the neural network presented in [21]. Across the full experimental validation dataset (the data that was not used to train the network), an MAE of 1.23 nm was observed on the generated depth profiles when compared to the true values. When including only a partial dataset including only the depth profiles from the validation set that were produced by a single pulse, the MAE was reduced to 1.01 nm.

5. Examination of the network

5.1 Controlling the weighting of pulses

One of the features of this work is that it can be used to predict the shapes of up to three pulses. As stated previously, one of the inputs into the network was the weighting vector formed of three values summing to 1. Each of these values corresponds to one of the three pulses used, and these numbers indicate the fraction of the total laser machined area that is to be machined by each pulse. For example, if a vector of [1, 0, 0] is provided, the network would try to ensure that only the first pulse contained areas to be machined. If, however, a vector of [0.5, 0.5, 0] is used the network would try to include an equal number of pixels in the DMD patterns used to machine the first and second pulses while having none in the third. One advantage of this is being able to control the overall pulses used, i.e. by telling the network that the majority of the laser machining should take place in a specific pulse or should be balanced across all three. The way this is implemented is by passing the network a three value vector, along with the depth profile, representing the proportion of total machining completed in each pulse.

Figure 9 demonstrates the effect of specifying pulse priority by asking the network to produce DMD sequences that would produce the "Experimental Depth" profile. As a reference, the original DMD pattern used to produce this profile is shown under "Experimental DMD". Each column in Fig. 9 represents a different input vector, Fig. 9(b) being [0, 0, 1], Fig. 9(c) [0, 1, 0], and Fig. 9(d) [1, 0, 0]. As can be seen in all columns, the generated DMD is almost entirely a single colour in the correct channel, meaning that the network operated within the constraints applied to it.

 

Fig. 9. Using the weighting input of the network to control the temporal position of white pixels in the sequence of DMD patterns. All rows were generated from the network using the experimental depth profile in a. The input weightings were defined as [pulse 1, pulse2, pulse 3] with b, c, and d using [1, 0, 0], [0, 1, 0], and [0, 0, 1] respectively. The sequence of DMD patterns follow the colour convention shown in Fig. 2

Download Full Size | PDF

Controlling pulse ordering can be beneficial, for example, when trying to minimise the amount of time taken by ensuring that all machining is done in as few pulses as possible, however, it is not always possible to machine in a single pulse (when the power is fixed). In such cases, the neural network is forced to work around the restrictions given and this can be seen in Fig. 10.

 

Fig. 10. Three example DMD sequences generated from a single depth profile with input weightings defined as [pulse 1, pulse2, pulse 3]. Row a was generated using a pulse weighting of [0, 0, 1], row b using a weighting of [1, 0, 0], and row c using a weighting of [1/6, 1/3, 1/2]. The sequence of DMD patterns follow the colour convention shown in Fig. 2

Download Full Size | PDF

When a weighting of [0, 0, 1] was given the network correctly established that the pattern still required three pulses to machine to the correct depth, Fig. 10(a). In addition to this, the network prioritised machining in the third pulse with every part of the machined area being illuminated in the final pulse. Beyond that, the network predicted an even mix of machining in the first and second pulses. The same is true when a condition of [1, 0, 0] is provided, Fig. 10(b), with the full area machined in the first pulse and then an even mix between the second and third. For Fig. 10(c), a condition of [1/6, 1/3, 1/2] was given, matching the ratios found in the initially created DMD pattern. This results in the pattern that is the most visually similar to the human-designed, experimental DMD pattern. It is important to note, however, that the resultant depth profiles produced using each of the generated DMD sequences remain similar to each other, and to the experimentally observed depth profile. Figure 10(a) and (b) produced depth profiles with a MAE of 1.86 nm and 1.88 nm respectively, almost identical, while the pattern machined with the true weighting vector gave a depth profile with a MAE of only 1.37 nm.

5.2 Comparing designed and generated DMD pattern sequences

As shown in Fig. 5(b), one of the applications of the demonstrated network is to generate a sequence of DMD patterns that would machine a desired depth profile. Due to the previously mentioned effects in the optical system, such as diffraction, the shape of the machined feature does not match the sequence of patterns used to produce it. In addition to this, even if computational methods were used to produce the DMD patterns, the actual properties of the laser pulses, such as a non-homogeneous intensity profile, would be difficult to model.

Figure 11 demonstrates the ability of the network to generate a DMD pattern that results in a more accurate depth profile than simply assuming that the ideal DMD pattern has the same spatial profile as the machined feature. First a desired depth profile is created, in this case using the approximate machined depth of a single pulse with a binary depth profile. After that, the primary neural network can be used to generate the sequence of DMD patterns that would best produce this desired depth profile. Alongside this, a DMD pattern was created by taking the depth profile and directly converting it by using the machined areas as white pixels, and unmachined areas black. The secondary network was then used to determine the depth profile that both sets of DMD patterns would result in and compared them to the original, desired, depth profile. The GAN-generated DMD patterns resulted in a depth profile with a MAE of 2.35 nm compared to the desired profile, while the directly converted pattern resulted in a higher MAE of 2.97 nm, indicating that the neural network is able to generate a sequence of DMD patterns that more accurately machine the desired depth profile.

 

Fig. 11. Generating sequences of DMD patterns to machine a specified depth profile. The neural network method is shown alongside a direct conversion method.

Download Full Size | PDF

There were two particularly interesting features of the GAN-generated DMD patterns. One was the areas of machining in a second pulse around the edges of the line features. This effect served to counteract the soft edges caused by diffractive effects in the initial laser pulse. Secondly the neural network attempted to offset the slope of the laser intensity profile by providing a larger area of a second pulse in the upper right corner of the machined feature. This is an effect that is particular to the experimental laser system used for collection of the training data, and hence would be different for every laser setup tested in this manner; showing the power of neural networks for bespoke modelling tasks.

6. Conclusion

In this work, we have demonstrated a neural network that is able to determine a sequence of DMD patterns that can be used to produce a desired depth profile. The network was also able to incorporate desired pulse ordering, with limitations, and still produce DMD patterns able to produce the desired depth profile. Different network losses were tested and a novel combination developed to best suit the requirements of the dataset, especially the one-to-many mapping between parameter spaces. This approach did not require the network to be explicitly taught the underlying physics, but rather it learned through empirical data. A second network was also used, alongside the one developed for this work, in order to increase the amount of training data available without having to rely on unsuitable augmentation techniques.