Rapid detection of rare events from in situ X-ray diffraction data using machine learning

High-energy X-ray diffraction methods can non-destructively map the 3D microstructure and associated attributes of metallic polycrystalline engineering materials in their bulk form. These methods are often combined with external stimuli such as thermo-mechanical loading to take snapshots of the evolving microstructure and attributes over time. However, the extreme data volumes and the high costs of traditional data acquisition and reduction approaches pose a barrier to quickly extracting actionable insights and improving the temporal resolution of these snapshots. This article presents a fully automated technique capable of rapidly detecting the onset of plasticity in high-energy X-ray microscopy data. The technique is computationally faster by at least 50 times than the traditional approaches and works for data sets that are up to nine times sparser than a full data set. This new technique leverages self-supervised image representation learning and clustering to transform massive data sets into compact, semantic-rich representations of visually salient characteristics (e.g. peak shapes). These characteristics can rapidly indicate anomalous events, such as changes in diffraction peak shapes. It is anticipated that this technique will provide just-in-time actionable information to drive smarter experiments that effectively deploy multi-modal X-ray diffraction methods spanning many decades of length scales.

1. Introduction

Metals play a significant role in modern society. They are used in various applications such as transportation, construction, energy generation, storage and delivery, and security. The performance requirements for these polycrystalline engineering materials have pushed the diversity and complexity of processes and elements employed in producing those materials to optimize structure and properties (Greenfield & Graedel, 2013; Graedel et al., 2015). Frameworks like integrated computational materials engineering (Olson, 2000) and materials informatics (Rajan, 2005) aim to accelerate material discovery and process optimization so that new materials that meet performance requirements can be deployed more quickly. The calibration and validation of these frameworks rely on microstructure and associated attributes acquired across multiple length scales under different conditions.

High-energy synchrotron X-ray (>50 keV) diffraction methods can non-destructively characterize metallic polycrystalline materials in their bulk form. In particular, high-energy diffraction microscopy (HEDM) can extract 3D microstructure information and grain-resolved attributes; it can also track their evolution when combined with in situ thermo-mechanical loading capabilities (Lienert et al., 2011; Schuren et al., 2015; Naragani et al., 2017). The far-field HEDM (FF-HEDM) variant (Bernier et al., 2020; Park et al., 2021) can provide the center of mass, crystallographic orientation and elastic strain tensor for each constituent grain in a polycrystalline aggregate. These FF-HEDM measurements are often combined with other measurement modalities such as near-field HEDM and tomography to fully understand the microstructure and state and their evolution in a polycrystalline material (Suter et al., 2008; Turner et al., 2017; Naragani et al., 2017; Sangid et al., 2020).

In a typical in situ HEDM experiment, the macroscopic stimulus to be applied on a sample is decided a priori, on the basis of a known response relationship. For instance, in an in situ HEDM experiment where a sample is subject to various levels of mechanical loading to study the material response heterogeneity at the mesoscale, the levels at which the loading is paused – to deploy a higher-resolution and more beamtime-consuming characterization method – are often decided according to macroscopic milestones such as yield strength in the macroscopic stress–strain curve of the material. Alternatively, such experimental decisions to be made on the fly may require heroic efforts during time-limited beam access to reduce the full HEDM data set and identify when the scientifically interesting phenomena occur (Naragani et al., 2017; Ravi et al., 2021; Suter et al., 2008; Li et al., 2023; Maddali et al., 2020; Simons et al., 2015). Hence, a high level of reliable automation is desirable and crucial for a successful study.

Here, we propose a robust, self-supervised, machine learning based framework (Fig. 1) that enables rapid identification of, and thus automated response to, the minute changes in diffraction spots measured by FF-HEDM and probable microstructural changes in polycrystalline metals. This framework is particularly timely with the improved brilliance and coherence of the next-generation diffraction-limited synchrotron X-ray sources and significant advances in detector technologies capable of rapid and accurate measurement of single-photon events – developments that allow many mesoscale measurement techniques like HEDM to be combined with nanoscale techniques like Bragg coherent diffraction imaging and dark-field X-ray microscopy in a single instrument (Maddali et al., 2020; APS, 2019). Our framework can provide quantitative actionable information about material state and guide experimenters as to when to deploy these higher-resolution techniques. We refer to this actionable information as the rare event indicator (REI). Using a variety of in situ FF-HEDM data, we demonstrate that the proposed framework and REI are capable of rapidly detecting mechanical yielding, an eventual but rare event that experimenters often seek to capture through in situ FF-HEDM experiments.

Figure 1 An illustration of the workflow for rapid inference of microstructural deformation. A polycrystalline sample is subject to mechanical loading while we acquire HEDM data. Bragg diffraction spots from the initial material state are used to train an image representation model (encoder) and a clustering model. These two models combined are sensitive to changes in the diffraction spots. Using these models, the REIs are computed as we continue to apply mechanical loading to the sample and acquire HEDM data. A significant increase in REI is quantitative actionable information that the experimenters can use to steer the course of the experiment.

2. Methods

2.1. Rare event detection workflow

For several years, event detection, also known as outlier or anomaly detection, has been a vibrant and dynamic area of study across diverse academic communities. Recently, applying deep learning has opened new avenues for event detection, emerging as a critical and promising direction in this field (Pang et al., 2021). Typical deep learning based event detection involves a three-phase process. First, in the data preprocessing phase, the data set is cleaned, normalized and prepared for modeling. Second, during model training, a deep learning model, such as an autoencoder, is trained on a `normal' or `baseline' data set to learn typical data patterns and accomplish the feature extraction task. Finally, in the event detection phase, this trained model evaluates new data, identifying `events' by measuring deviations from the learned patterns.

In this study, as shown in Fig. 2, we employ the BraggNN model (Liu et al., 2022) as our neural network model for feature extraction, specifically fine-tuned for image representation. To facilitate the unsupervised training of our novel image representation model, we adopt the BYOL (bootstrap your own latent) approach (Grill et al., 2020). Additionally, we utilize the K-means clustering algorithm to compute the actionable information as a rare event indicator. By separating the feature extraction tool and the event detector into two independent modules, our workflow can easily expand its support for event detection in various applications. Furthermore, our current design demonstrates promising results, showcasing the effectiveness of our event detection workflow on X-ray diffraction data.

Figure 2 Rare event detection workflow with three phases. The first phase (orange dashed rectangle) trains an image representation model (trained encoder) using a baseline data set for feature extraction. The trained encoder from the first phase is applied to a reference data set followed by the K-means clustering algorithm to obtain K centers to characterize the reference data set in the second phase (blue dashed rectangle). The output of the trained encoder from the first phase and the clustering model from the second phase, applied on the testing data set, is thresholded to determine REI for the testing data set in the third phase (green dashed rectangle). The different hyperparameters at each step are shown in red text.

We provide in the following a comprehensive overview of the different building blocks constituting our proposed workflow. Section 2.1.1 elucidates the preprocessing steps involved in handling the HEDM diffraction images. Section 2.1.2 describes the specifics of training the image representation model. We illustrate the modules associated with processing the reference data set and evaluating subsequent experimental data sets in Sections 2.1.3 and 2.1.4, respectively.

2.1.1. Data preprocessing

In the context of our workflow, a fundamental objective of data preprocessing is to transform each dark-field-subtracted diffraction pattern extracted from a stack of FF-HEDM diffraction patterns into a set of peak patches representing individual peaks. To accomplish this, we utilize the component analysis library provided by OpenCV (Bradski, 2000). A data set in our workflow is defined as the collection of peak patches (ranging in number from thousands to millions) extracted from diffraction images collected during a contiguous rotation segment (up to 360°). Depending on the material state during acquisition, data sets are labeled baseline, reference and testing data sets. The baseline and reference data sets are typically any data sets collected before applying a macroscopic stimulus to the material. The testing data set is fed to the workflow as the material undergoes microstructural change.

2.1.2. Image representation model training

During the training phase of the image representation model, we utilize the BYOL method (Grill et al., 2020) on the preprocessed baseline data set. BYOL has emerged as a prominent self-supervised learning technique in deep learning, addressing the challenge of training neural networks without explicit labels or annotations. In Fig. 2, BYOL, depicted within the orange dashed rectangle, leverages two sets of neural networks: online and target networks. Each network comprises an encoder and a projector. For the encoder, we refine the BraggNN model by removing its fully connected layers. The projector, a two-layer fully connected neural network, maps the encoder's output to a lower-dimensional space, enhancing the model's generalizability.

The encoder's role is to extract the features of each data patch, while the projector aims to reduce the dimensionality of the encoder's output, facilitating improved model performance. During training, the online network learns to represent each peak patch in a latent space, while the target network maintains a moving average of the online network's parameters. The trained encoder is saved for use in subsequent phases. The core idea of BYOL is to encourage the online network to generate latent representations similar to those produced by the target network. By comparing these representations, BYOL effectively learns powerful representations for the given task. This approach has demonstrated remarkable success across various domains, enabling models to achieve state-of-the-art performance without the need for costly annotated data.

Even though our workflow only uses the trained encoder of the online network from BYOL, due to the structure and training of BYOL, the dimensions of the encoder, projector and predictor networks and the number of epochs for training are important hyperparameters, which can affect REI when applied to the training data set later on. Section 3.1.1 describes the hyperparameters in more detail.

BYOL must be trained to differentiate between a peak's translation and rotation (transformation) and changes to salient features (such as the width) of a peak. A well trained BYOL is more sensitive to changing peak features and less sensitive to transformation. In each epoch of BYOL training, as illustrated in Fig. 2 within the orange dashed rectangle, we randomly select a peak patch from the preprocessed baseline data set. Fig. 3 illustrates the goal of BYOL training. In this figure, the first column shows a randomly selected diffraction peak. Fig. 3(a) shows the original diffraction peak (first panel) and ten transformed versions of it; Fig. 3(b) shows the original diffraction peak (first panel) and ten other unrelated randomly selected peaks. Each of these peaks is treated as a vector in the image representation space. If we use the cosine distance between vectors as the metric to indicate similarity or difference between two patches, the median cosine distance is 0.0058 for the patches in Fig. 3(a) while it is 0.051 for the patches in Fig. 3(b). [The cosine distance between vectors u and v is defined as

where u · v represents the dot product of u and v.] The training of BYOL aims to replicate this result by minimizing the distance between the output of the target network and the online network, ideally converging to zero.

Figure 3 For a randomly picked peak (left-most), the median cosine distance between the peak and ten transformed views (a) is 0.0058, while the median cosine distance with ten randomly picked peaks (b) is 0.051, nearly 10× further.

2.1.3. Reference data set processing

In the reference data set processing phase, our first step involves feeding all patches from the reference data set into the trained representation model (encoder) and converting them into a collection of representation vectors. These vectors capture each patch's essential features and characteristics.

Subsequently, we utilize the K-means clustering algorithm to group these representation vectors into K centers. This clustering process allows us to identify common patterns and clusters within the reference data set. The resulting K centers (another hyperparameter) serve as representative points that summarize the distribution and variations present in the reference data set.

The output of the reference data set processing phase, comprising the K centers obtained through K-means clustering, is saved for the subsequent testing data set evaluation phase. These centers provide a reference point for comparison and evaluation when analyzing the similarity and discrepancy between the testing data sets and the reference data set.

2.1.4. Testing data set evaluation

During the evaluation phase of a testing data set, we utilize the trained representation model (encoder) to generate a collection of representation vectors for the testing data set. Each vector captures the essential characteristics of a particular data instance (peak patch) within the testing data set.

Next, we compute the Euclidean distances between each representation vector and the K centers obtained from the previous phase. This distance calculation allows us to determine the closest center for each vector, enabling us to assign it to a specific cluster. Additionally, we update the vector's closest center to obtain an assignment to K centers and its corresponding confidence level.

We introduce REI to quantify the uncertainty in the assignments. This score is determined by calculating the percentage of representation vectors whose confidence level is below a predefined threshold value, denoted as t (another hyperparameter). A lower confidence level indicates higher uncertainty in the assignment to a specific center. Accordingly, the REI values will fall between 0 and 1.

By repeating this procedure for all testing data sets, we obtain REIs for each data set. These scores provide an indication of the level of uncertainty associated with the assignment of data instances to the K centers. Collectively, these REI values form a figure that summarizes the uncertainty across all data sets, allowing for a comprehensive assessment of the performance and reliability of the event detection workflow.

2.2. High-energy diffraction microscopy experiments

The FF-HEDM data sets presented here were acquired at the 1-ID beamline of the Advanced Photon Source (APS) at Argonne National Laboratory. Details of the FF-HEDM instrument geometry (Fig. 4) are given by Park et al. (2021). The main features are as follows:

Figure 4 A schematic of the FF-HEDM instrument geometry. The stage stack consists of translation and rotation capabilities to align the ω-rotation axis and the sample to the X-ray beam. The picture shows the setup and sample geometry used to acquire the in situ FF-HEDM data for the 304L-SS sample.

(i) Monochromatic, high-energy, synchrotron X-rays were used as the probe to interrogate the material.

(ii) Transmission geometry with an area detector (Lee et al., 2007) was used to capture the intensity of the diffraction spots in reciprocal space. Table 1 lists the X-ray energies and sample-to-detector distances used.

Table 1
FF-HEDM instrument parameters

	X-ray energy (keV)	Sample-to-detector distance (mm)
304L-SS (Section 3.1)	71.68	803
Ti-6-4 (Section 3.2)	71.68	1163
Sand (Section 3.3)	71.68	1601
Ti-7 (Section 3.6)	61.33	756

(iii) The incident X-ray beam had a rectangular shape. The beam size along x_L was sufficiently large to illuminate the cross section of the sample in its gauge section, and the beam size along y_L was varied depending on the experiment's main objectives and required resolution in between layers; it was varied only when investigating the effect of beam size on REI.

(iv) Fig. 4 shows an example sample geometry used for the uniaxial tension testing. When possible, a set of 30 µm cube gold markers (Shade et al., 2016) were attached on the sample gauge section surface so that an identical volume of material was interrogated throughout the in situ HEDM experiment.

(v) The sample was rotated about y_L at a constant angular speed over an ω range of 360° while a set of diffraction patterns were acquired. Typically, this set consisted of 1440 frames, each covering an angle of 0.25°. Acquiring a set took approximately 6 min.

(vi) In a typical in situ FF-HEDM experiment at the APS, Microstructural Imaging using Diffraction Analysis Software (MIDAS) (Sharma et al., 2012a,b; Sharma, 2023) is used to analyze the set of diffraction patterns and extract a 3D map. At moderate deformation levels, it takes approximately 8 min to acquire the 3D map from a data set on a high-performance computing cluster.

3. Results and discussion

We acquired four sets of in situ FF-HEDM diffraction patterns using the instrument described in Section 2.2. The first set of patterns was acquired with a 304L stainless steel (304L-SS) sample with a face-centered cubic crystal symmetry. The second set of patterns was acquired with a commercially pure Ti (CP-Ti) sample with a hexagonal close-packed crystal symmetry. These two data sets were acquired in uniaxial tension. The third data set used a pack of sand with a trigonal crystal symmetry under compression. The fourth data set was acquired on a Ti-7 Al alloy sample with a hexagonal close-packed crystal symmetry under continuous uniaxial tension.

We anticipated that the 304L-SS, CP-Ti and Ti-7 Al samples would accommodate mechanical yielding or plastic deformation by crystallographic slip (Taylor, 1938), resulting in diffraction spot smearing (Obstalecki et al., 2014), while the sand sample would exhibit brittle fracture with minimal diffraction spot smearing. For the sand sample, we expected the number of diffraction spots to increase with the applied load as the sand particles fracture into smaller but still coherent pieces; therefore, the shape of resulting diffraction spots was expected to exhibit minimal or no smearing, while their intensity would show a dramatic decrease. With these expectations, our rare event detection framework, which consists of an image representation model and a clustering model, was trained using various permutations of diffraction patterns acquired at zero load–zero strain state (reference state) from each sample. These framework permutations were deployed to assess whether yielding could be detected reliably. Furthermore, the sensitivity of our framework and REI to instrumental changes was assessed using the 304L-SS FF-HEDM patterns acquired using a variety of beam sizes and incident X-ray photon flux levels.

3.1. In situ FF-HEDM of stainless steel

The dotted yellow curve in Fig. 5 shows the typical stimulus (stress)–response (strain) relationship of a metallic material. At lower stresses, the relationship between macroscopic stress and strain is nominally linear and elastic, but at larger stresses, this is no longer the case. The level of stress where the linear relationship between stress and strain breaks from linearity is often referred to as the material's macroscopic yield point or yield stress (approximately 225 MPa in this case). Here, the stress–strain curve was acquired during our in situ FF-HEDM experiment under uniaxial tension. The geometry of the 304L-SS sample was identical to that described by Wang et al. (2022): 11 mm (length)  ×  1 mm (width)  ×  1 mm (thickness) in the gauge section. Gold markers were attached along the gauge section (Shade et al., 2016) so that an identical material volume could be measured throughout the experiment.

Figure 5 Applied stress, REI, average FWHM and average axial lattice strain versus strain for (a) the entire in situ FF-HEDM experiment, and (b) magnified view near the elastic–plastic regime. The error bars for REI, FWHM and lattice strain at a target stress level indicate the range of respective metrics observed in the four 0.4 mm-tall material volumes. The dotted yellow curve shows the stress–strain curve of the 304L-SS sample measured during the in situ FF-HEDM experiment.

In this in situ FF-HEDM experiment, there were four target stress levels in the elastic regime, five in the elastic–plastic transition regime (the `knee' of the stress–strain curve) and four in the plastic regime well beyond the knee. Each target stress level was reached through a constant displacement rate of 1.1 µm s⁻¹ to achieve a nominal strain rate of 0.0001 s⁻¹; once the target stress level was reached, the applied stress was intentionally relieved by approximately 10 before carrying out the FF-HEDM pattern acquisition to prevent further changes to the material by stress relaxation during the scans. (The sample was under displacement control during FF-HEDM pattern acquisition; the applied stress level fluctuation was minimal during an FF-HEDM scan even in the plastic regime.)

An X-ray beam size of 2 × 0.4 mm was used for FF-HEDM pattern acquisition. The width of the X-ray beam ensured that a 0.4 mm-tall material volume in the sample gauge section was always illuminated by X-rays during an FF-HEDM pattern acquisition. At each target load level, four sets of FF-HEDM pattern acquisition were executed, illuminating in total a 1.6 mm-tall contiguous volume (four layers) in the sample gauge section. The gold markers in the gauge section ensured that the identical material volume was interrogated across multiple target load levels reached during in situ loading.

At each target load level, FF-HEDM patterns were used to compute the REI,¹ average diffraction spot full width at half-maximum (FWHM) in the azimuthal direction and average axial lattice strain.² Fig. 5 shows the evolution of these metrics with applied stress, denoted by the green dots, the orange dots and the blue dots, respectively. Here, the image representation model and the clustering model to compute REI were trained using the 304L-SS sample's reference state diffraction pattern. The FWHM and axial lattice strain were computed using MIDAS (Sharma et al., 2012a,b). All three metrics, namely REI, average diffraction spot FWHM in the azimuthal direction and average lattice strain, require the extraction of diffraction peaks from 2D diffraction patterns, with the subsequent steps being inference (for REI, requiring ∼2 s computation time), peak-shape fitting (for FWHM, requiring ∼100 s computation time) and full microstructure reconstruction (for lattice strain, requiring ∼500 s computation time).

We demonstrate that REI can provide actionable information comparable to that from the traditional metrics for polycrystalline metallic alloys subjected to mechanical loading. Fig. 5(b) is a magnified view of Fig. 5(a) near the elastic and elastic–plastic transition regime. In this regime, researchers often need to make the difficult decision to pause the loading and conduct higher-resolution measurements and allocate precious beam time on the basis of limited a priori knowledge about the material (such as the stress–strain curve obtained ex situ) or relying on FWHM or lattice strain metrics that require FF-HEDM patterns acquired over a larger angular range, with substantial computing resources and time. In general, the three metrics follow the trends observed in the macroscopic stress–strain curve, with significant inflections occurring near the knee of the stress–strain curve. REI and FWHM remain fairly constant in the elastic regime. As the material approaches its macroscopic yield point, REI and FWHM both start to increase, most notably in the region highlighted by the purple ellipse, roughly coinciding with the macroscopic yield point. They continue to increase substantially as the sample transitions into the plastic regime. On the other hand, the average axial lattice strain increases linearly in the elastic regime. It breaks from linearity near the macroscopic yield point and then remains relatively constant in the plastic regime. [This is expected as lattice strain measures only the elastic strain (change in the lattice spacing).] This figure illustrates that REI is a metric sensitive to material yielding accommodated by crystallographic slip and delivers information comparable to that provided by the traditional metrics while being at least 50 times faster to compute.

3.1.1. Hyperparameter tuning

In contrast to model weights learned during training, hyperparameters are parameters that the model cannot optimize during training. Typically, multiple configurations of hyperparameters are used to train various models, and the best-performing values for hyperparameters are chosen accordingly. Hyperparameters play a pivotal role in the machine learning workflow, determining the configuration and behavior of a model. In our REI workflow, the hyperparameters during model training are as follows: the size of patches used for the workflow, the data set used for training the encoder, the data set used for training the clustering model, the encoder dimension, the projector dimension, the predictor dimension, the number of epochs for training the encoder, the confidence threshold, t, and the number of clusters, K.

The patches should be large enough to capture the shape of diffraction peaks and small enough not to negatively affect computation speed. We chose 15 × 15-pixel patches; this size is the same for the baseline, reference and training data sets.

As shown in Section 3.4, REI is more sensitive to changes in beam size, beam flux and rotation steps than to changes in position and starting rotation angles. Thus, the data sets used for training the encoder (baseline data sets) and for training the clustering model (reference data sets) only included data sets with fixed beam size, beam flux and rotation steps.

The encoder, projector and predictor dimensions determine the shape of BYOL. The choice of these numbers greatly affects the trained encoder that converts the patches into the latent representation. We chose the following values of the encoder, projector and predictor dimensions: 32, 64 and 64, respectively.

We employ a confidence summation approach to determine our model's optimal number of training epochs. The idea behind this method is straightforward: a well trained image representation model should be able to identify pairs of transformed patches within a set of random patches.

Our confidence summation process unfolds as follows. At the conclusion of each training epoch, we randomly select 100 patches from our data set, convert them into 100 representation vectors using our model, and calculate a confidence score for each one. This score is derived from the distances between a specific patch representation and ten other patches. To compute the confidence score, we use the formula

where D₁ and D₂ are the shortest and second shortest Euclidean distances, respectively, among all pairs of representation vectors. If the distance between two transformed patches is not the closest one among these ten patches, we assign a confidence score of −1. This process enables us to evaluate how patches differ from their nearest neighbors regarding distance. Finally, we sum all 100 confidence scores for all representations of patches.

In our experiments using the stainless steel data set described in Section 3.1, we observe that the confidence score stabilizes after approximately 100 epochs, as depicted in Fig. 6. Therefore, we opt to set the number of training epochs to 100 for our image representation model. The models were trained on a computer with an NVIDIA v100 GPU, and the training time was 111 min for 100 epochs.

Figure 6 Accuracy plot of the image representation model.

To determine the optimum number of clusters, K, and threshold, t, we will describe hyperparameter tuning as it was applied to the stainless steel data set described in Section 3.1. The performance of the model is evaluated by computing the change in REI when the material first undergoes plastic deformation (two data points in the purple ellipse in Fig. 5). REI sensitivity, REI_sensitivity (shown in Fig. 7), is maximized during hyperparameter tuning and defined as follows:

where is the smallest REI value for the data point after the onset of plastic deformation, is the largest REI value for the data point before the onset of plastic deformation in the elastic regime and ΔREI_plastic is the spread in REI (due to local microstructure variations) among different volumes scanned for the data point after the onset of plastic deformation. This criterion ensures that the goal of detecting the onset of plastic deformation is achieved: negative REI_sensitivity means the model would not distinguish between data sets collected under elastic and plastic regions. Furthermore, values greater than 1 ensure that the detected change between elastic and plastic deformation is greater than the uncertainty in REI during plastic deformation.

Figure 7 Contour plot of REI sensitivity as a function of number of clusters K and threshold t.

Fig. 7 shows a contour plot of REI_sensitivity as a function of number of clusters and confidence threshold. Only positive values of REI_sensitivity are shown. It can be seen that a higher number of clusters and higher confidence threshold result in higher REI_sensitivity. As depicted in Fig. 7, the selection of K as 40 and the threshold as 0.5 leads to the highest sensitivity of REI_sensitivity, reaching 5.6.

3.2. In situ FF-HEDM of titanium alloy

Fig. 8 shows the stress–strain curve and REI results for a CP-Ti alloy, magnified in the elastic and elastic–plastic transition regime. The dotted yellow curve shows the stress–strain curve of the CP-Ti material in uniaxial tension acquired during the in situ FF-HEDM experiment; it shows similar characteristics to the stress–strain curve for the 304L-SS sample (Fig. 5) with the elastic–plastic transition occurring approximately at 200 MPa. Here, the in situ FF-HEDM experiment was conducted using the rotation and linear axial motion system (RAMS) load frame (Schuren et al., 2015). The sample geometry was identical to the one presented by Menasche et al. (2021) with a 1 ×  1 mm cross section in the gauge section. Gold markers were attached in the gauge section (Shade et al., 2016) to keep the illuminated material volume consistent throughout the in situ experiment. The experimental procedure was similar to that used for the 304L-SS sample: the nominal strain rate was 0.0001 s⁻¹ and loading was temporarily paused to collect FF-HEDM patterns along multiple layers in the sample gauge section.

Figure 8 The in situ stress–strain curve of the CP-Ti alloy sample (yellow dots) and associated REI results (blue dots) in the elastic and elastic–plastic transition regimes. REI remains roughly constant in the elastic regime. As the material approaches its macroscopic yield point, REI also starts to increase significantly, highlighted by the purple ellipse, and it continues to increase as the plastic regime sets in. The abrupt change in REI near 0.1 strain is due to a beam size change between two target load levels; this is discussed further in Section 3.4.

The blue dots in Fig. 8 show the evolution of REI when the image representation model and clustering model were trained by using the 304L-SS FF-HEDM patterns acquired at zero load–zero strain (reference state). The overall evolution of REI is consistent with that observed in the 304L-SS sample: REI remains relatively constant in the elastic regime and then starts to increase near the knee of the stress–strain curve, demonstrating that the image representation model and the clustering model (and therefore REI) are sensitive to materials accommodating plastic deformation through crystallographic slip. Furthermore, Fig. 8 illustrates that those models are transferable between materials with similar deformation modes. The abrupt decrease in REI at approximately 0.1 strain (or 150 MPa applied stress) is due to a change made at this point in the beam size used for FF-HEDM experiments in order to alter the vertical spacing between layers. The robustness and sensitivity of REI are examined further in Section 3.4.

We computed several cases of REI in order to examine the performance of the rare event detection framework and the choice of baseline and reference data (Fig. 9). Table 2 summarizes the permutations of training reference state patterns to compute the REI cases. In all four cases, we used the optimized hyperparameter values determined from the 304L-SS experiment. Naturally, the absolute value associated with each REI case is different between the four cases as the training data sets are different; however, the four cases all share a similar trend in that they remain relatively constant in the elastic regime, start to increase near the elastic–plastic regime and continue to increase in the plastic regime. The inflection point where the REI values start to increase is consistently near the macroscopic yield point. This highlights that the image representation and clustering models trained on a data set with a similar deformation mode but collected on a different sample or material are transferable. While it is recommended that the patterns acquired before the beginning of in situ loading be used as the training set, such transferability can be useful when using REI to estimate the material state, for instance, in a high-throughput measurement setup.

Table 2
The reference state patterns employed to train the encoder and clustering models for the REI cases presented in Fig. 9

REI case	Training reference state patterns for encoder	Training reference state patterns for clustering model
1	CP-Ti sample	CP-Ti sample
2	CP-Ti sample	304L-SS sample
3	304L-SS sample	304L-SS sample
4	304L-SS sample	CP-Ti sample

Figure 9 The in situ stress–strain curve and REI evolution for the CP-Ti sample in the elastic and elastic–plastic transition regime. Four cases of REI using permutations of training reference state patterns (Table 1) are presented.

3.3. In situ FF-HEDM of sand

In Section 3.2, we established that the image representation and clustering models trained using the 304L-SS FF-HEDM patterns acquired at the reference state can be used to detect rare events and anomalies in other material systems that share a similar deformation modality. Here, we introduce a set of in situ diffraction patterns acquired from a material system that does not accommodate deformation by crystallographic slip.

Fig. 10 shows the stress–strain curve and associated REI for an aggregate of sand subject to compression. The dotted yellow curve shows the stress–strain curve of sand aggregate in uniaxial compression. The in situ mechanical loading procedure was similar to the one described by Hurley et al. (2017). The sample fractured catastrophically when compressed beyond 75 MPa. The sample also does not exhibit the elastic, elastic–plastic transition and plastic regimes that were observed in the 304L-SS (Fig. 5) and CP-Ti (Fig. 8) samples. Instead, the stress–strain curve shows a constant increase and an abrupt decrease in applied stress indicative of catastrophic failure. We consider two REI cases similar to Fig. 9. For REI case 1 (green dots), the image representation and clustering models are trained using the 304L-SS reference state patterns. For REI case 2 (orange dots), these models are trained using the sand reference state patterns. In both cases, the hyperparameters were optimized on the basis of the 304L-SS patterns. While the absolute magnitudes are different, the two REI cases show minimal changes with respect to loading. This implies that the underlying architecture in our rare event detection framework is tuned to detect anomalies in diffraction spots induced by crystallographic slip, but it is insensitive to deformation carried out by the fracturing of grains.

Figure 10 Stress–strain curve and REI evolution for the sand sample. Stress and strain are compressive, mimicking the compaction that sand is typically subjected to. The stress–strain curve does not exhibit the characteristics illustrated in Fig. 5 and Fig. 9. The orange dots show the REI case where the sand reference state patterns were used as the training data set. The green dots show the REI case where the 304L-SS reference state patterns were used as the training data set. Both REI cases show minimal changes as the sand sample is subject to compressive loads up to catastrophic failure of the material system.

3.4. Robustness to experiment configurations

We have demonstrated that REI is sensitive to changes in polycrystalline metals due to plasticity, but experiment configurations can also lead to changes in REI. Here, we investigate the sensitivity of REI to different configurations during the 304L-SS experiment in Section 3.1. Fig. 5 includes data collected with the following configuration: 0.4 mm X-ray beam height, rotation angle start at −180°, rotation angle step 0.25°. This was the reference state for computing the absolute change in REI (ΔREI). Table 3 shows ΔREI as a result of different changes in experiment configurations. For each ΔREI calculation, the material deformation state was constant, and only the corresponding experiment configuration was changed.

Table 3
Sensitivity to experiment parameters

Experiment change	ΔREI (×10^−2)	Average REI (−)
Different starting rotation angle	0.52	0.27
Different rotation angle step	1.7	0.28
Different position	1.1	0.31
Different position and X-ray beam size	2.1	0.28
Different position and X-ray beam flux	2.7	0.29
Different position and X-ray beam flux, large plastic deformation	5.2	0.52
Different position, X-ray beam size and beam flux	2.9	0.27

Changes in the starting rotation angle and rotation angle step change the position and slicing of diffraction spots (3D) on individual frames (2D). Since our model works with diffraction signals on individual frames instead of 3D shapes, these changes are expected to change REI as well. (Using 2D spots instead of 3D allows for improvements in computation speed and working with streaming data.) Table 3 shows that changing the starting rotation angle (offsets of ±0.125°, ±0.25°, ±0.314° and ±0.628° in starting rotation angle were used) has the smallest influence on REI. We will use this ΔREI as a baseline (denoted ΔREI_b) to compare against other experimental parameters. Changing the rotation angle step (0.1° versus 0.25°) has more than three times larger ΔREI than ΔREI_b.

Changing the position of data acquisition in the sample (four different positions) results in two times larger ΔREI than ΔREI_b. This is attributable to local changes in the microstructure at any given sample state. On the other hand, changes in other experiment configurations (with different positions in the sample) lead to more than four times higher ΔREI compared with ΔREI_b: four times for changing X-ray beam size (0.1, 0.2 and 0.4 mm), five to ten times for using sub-optimal incident X-ray flux for low and high plastic deformation,³⁴ and about six times higher for changing both X-ray beam size and flux.

Furthermore, for the titanium data set (Fig. 8) the change in REI near 0.1 strain by ∼1.3 × 10⁻² is because the vertical X-ray beam size was changed from 0.4 to 0.2 mm.

ΔREI for detecting the onset of plastic deformation in Fig. 5 is 1.38 × 10⁻², the difference between REI of two sample states in the purple ellipse. From Table 3, we can see that only changes in the starting rotation angle and position in the sample yield a ΔREI smaller than this value. This indicates that an experiment utilizing REI as an indicator can vary these two experiment configurations, namely, starting rotation angle and position in the sample, but must keep the other experiment configurations constant during the experiment: rotation angle step, X-ray beam size and X-ray beam flux. Conversely, REI can be a useful metric to detect unwanted changes to or drift from the experimental configuration.

3.5. REI using a partial data set

The ability to monitor the material state changes and provide actionable information in real time predicates having sufficient frames to extract reliable REI quickly. To estimate the minimum number of detector frames necessary to provide a reliable REI, we used the FF-HEDM patterns from the 304L-SS sample acquired over an ω range of 360°. For the FF-HEDM patterns acquired at a given target load level:

(i) A starting ω angle (or a frame in the stack of FF-HEDM patterns acquired over an ω range of 360°) is chosen randomly.

(ii) A contiguous segment of frames corresponding to a particular Δω range starting from the chosen ω angle is extracted from the FF-HEDM patterns.

(iii) REI for the contiguous segment is computed.

(iv) This procedure is repeated 20 times with random starting ω angles to estimate the range of REI values arising from using a partial FF-HEDM pattern set.

Fig. 11 presents a magnified view of the elastic and elastic–plastic transition regime for Δω of 5° (orange dots), 10° (blue dots), 20° (purple dots), 40° (black dots) and full 360° FF-HEDM patterns (green dots). The REI error bars show the range of REI values computed from 20 repeats. For clarity, the REI values are intentionally shifted by −0.1, 0.0, 0.1, 0.2 and 0.3 for Δω of 360, 5, 10, 20 and 40°, respectively. Table 4 shows the REI calculation time for each Δω case.

Table 4
REI calculation time for each Δω case

Patch extraction time is computed assuming 0.25° ω steps when acquiring FF-HEDM patterns and 20 frames per second processing time on the computing infrastructure used for this work.

Δω (°)	Time for patch extraction (s)	Time for REI calculation (s)
5	1	0.31
10	2	0.33
20	4	0.35
40	8	0.41
360 (full)	72	1.2

Figure 11 The in situ stress–strain curve and REI evolution computed using partial data sets for the 304L-SS sample in the elastic and elastic–plastic transition regime. For clarity, the REI values are intentionally shifted by −0.1, 0.0, 0.1, 0.2 and 0.3 for Δω of 360, 5, 10, 20 and 40°, respectively.

In all four Δω cases where a partial data set is used, the overall trends in REI are consistent with those observed when the full data set is used to compute REI. Smaller Δω results in larger REI error bars, most likely due to limited sampling and scatter in the FF-HEDM patterns. Nevertheless, the REI computed from Δω of 40° is very close to the benchmark REI. This implies that it is not necessary to acquire and analyze a full FF-HEDM data set acquired over an ω range of 360° to compute a reliable REI. Furthermore, when combined with the REI calculation time (Table 4), this observation shows that experimenters can consider a new mode of FF-HEDM data acquisition where the sample is continuously rotated and loading is not paused (the sample geometry and loading rate need to be moderated to a level that matches the diffraction pattern acquisition and REI determination rates) and diffraction patterns are streamed without pausing. (Such experimental modality will also require a stage stack that allows continuous ω rotation such as the RAMS load frame or one that employs a slip ring system.) Given the quick turnaround time for REI calculation, continuous quasi-static or cyclic loading can be possible. As diffraction patterns are streamed, REI can also be streamed concurrently to provide a scalar actionable quantity to the experimenters to steer the course of the in situ experiment. This loading modality cannot be realized easily with a conventional metric such as FWHM or lattice strain (Fig. 5) which requires a full reconstruction or, at the minimum, diffraction peak fitting.

3.6. REI with continuous loading

As alluded to in Section 3.5, the ability to compute reliable REI from a partial FF-HEDM data set allows a new loading modality where loading is not paused during an in situ FF-HEDM. We deploy our rare event detection framework on a previously published FF-HEDM data set that employed this loading modality (Pagan et al., 2017) to quantify the crystallographic slip strength and study microcrack initiation and propagation in a Ti-7 Al alloy. The experimental setup and sample geometry are similar to those used for the CP-Ti data presented in Section 3.2. Another key difference to note in this FF-HEDM data set is that the ω scan was broken up into six 60° segments so as to test both the continuous loading modality and partial data REI.

Fig. 12 shows the macroscopic stress–strain curve, REI values computed over the course of the in situ FF-HEDM and incident X-ray flux. As anticipated, the rare event detection framework can capture the elastic–plastic transition from a partial FF-HEDM data set, as indicated by the large increase in REI when the material transitions into the plastic regime. Furthermore, our rare event detection framework can detect changes in the incident X-ray flux. For instance, the REI values increase significantly with a large increase in the incident flux, measured by an ion chamber installed in the X-ray beam path immediately before the sample (purple ellipse in Fig. 12). We also observed an REI decay consistent with the incident beam flux decay associated with the 324 singlets (non top-up) APS storage ring operation modality when this data set was acquired. This trend is highlighted by the blue and green dotted arrow lines for REI and incident flux, respectively.

Figure 12 The Ti-7 alloy was continuously deformed in uniaxial tension while FF-HEDM patterns were acquired. REI shows significant changes in three different ways. It increases significantly when the material undergoes elastic–plastic transition. It also changes when the incident flux changes significantly; an example of such an instance is highlighted by the purple ellipses. Furthermore, REI is sensitive to a gradual decay in the incident X-ray flux; an example of such an instance is highlighted by the dotted green (incident flux) and blue lines (REI).

This application example shows that our rare event detection framework can track changes to the diffraction peaks induced by crystallographic slip and can detect small changes to the incident beam flux. This is particularly useful for newer FF-HEDM modes that require much longer scan time (on the order of several hours) such as stitching HEDM (Johnson et al., 2023) (where multiple field-of-view scans across a sample cross section are stitched to interrogate a sample cross section larger than the beam size available at the endstation) or point focus HEDM (Li et al., 2023) (where multiple scans across a sample cross section are conducted with a 2D focused beam to acquire intra-granular microstructure and micromechanical state information). REI can be an independent metric indicating changes to the FF-HEDM patterns due to incident beam characteristic changes or material state changes (such as creep or relaxation) during these scanning techniques.

4. Outlook

We have demonstrated that our rare event detection framework can detect changes to the FF-HEDM diffraction peaks due to material state change and instrument changes such as incident flux and beam size. This framework is based on the unsupervised image representation learning and clustering algorithms. The resulting REI can be a metric that experimenters can use to make informed decisions about the course of their in situ FF-HEDM experiment instead of relying solely on a conventional stress–strain curve. Continuous loading combined with in situ FF-HEDM can be more accessible as experimenters do not have to rely on full reconstructions to decide on when to stop the loading and deploy higher-resolution techniques.

The rare event detection framework presented in this work can inform the experimenter when a possible rare event is occurring. Yielding in a uniaxial tension experiment is an eventuality but a `rare' event with regard to when and where it occurs. If the rare event detection framework can detect where in the microstructure an anomaly is occurring, it will be an additional piece of information that experimenters can use to steer their experiment. There are several avenues to identifying the location. For instance, it can be accomplished by creating a subset of patches from a particular set of grains of interest and only monitoring the REI from those grains.

It is also noteworthy that several petabytes of FF-HEDM data from more than a decade of user operation at the APS 1-ID beamline cover many alloy systems and multi-axial loading paths. We intend to use these data sets to further evaluate our rare event detection framework and report on the findings in the near future.

5. Code and data availability

The data used here are archived on the APS Data Management System and available on request. The code is available on request.

Footnotes

‡These authors made equal contributions.

¹Here, the FF-HEDM patterns acquired at the zero load–zero strain point were used as the baseline data set to train the image representation model. For more details on the image representation model, readers are referred to Section 2.1.

²The FWHM and axial lattice strain are the more commonly used metrics to detect the rare event, namely yielding in this case.

³Here, optimal incident X-ray flux is the setting in which the full dynamic range of the area detector is utilized without acquiring saturated pixels during a FF-HEDM scan; sub-optimal incident X-ray flux means the dynamic range of the detector is under-utilized.

⁴High local microstructure changes from deformation also contribute to high ΔREI.