FullProfAPP: a graphical user interface for the streamlined automation of powder diffraction data analysis

FullProfAPP is a software tool for data processing, refinement and visualization of large collections of powder diffraction patterns. Featuring an intuitive graphical user interface, it seamlessly facilitates a variety of tasks. These include conducting full-profile phase searches, sequential and high-throughput Rietveld refinements, and managing background (and peak) detection. FullProfAPP also provides convenient interaction with crystallographic databases and supports the visualization and export of high-quality pixel and vector graphics depicting the refinement results, among other functionalities. FullProfAPP wraps around the refinement program FullProf [Rodríguez-Carvajal (1993), Physica B, 192, 55–69] and offers the flexibility of user-defined workflows by accessing and editing FullProf's input files and triggering its execution as necessary. FullProfAPP is distributed as open-source software and is presently available for Windows and Linux operating systems.

1. Introduction

Powder diffraction is a fundamental technique in materials science for the study of crystalline solids. It is essential in guiding experimental research by providing insights into the crystal structure, composition, purity, degree of disorder and microstructural information of materials, among other properties. Typically, experimental diffraction patterns are initially qualitatively analysed to identify peak positions, intensities and peak profiles. Subsequently, the phases present in the experimental samples are identified, peaks are indexed, and the refinement of the pattern's profile is conducted using methods such as those developed by Le Bail et al. (1988) or Rietveld (1969). This entails starting from an initial calculation of a powder diffraction pattern and iteratively optimizing the model's parameters by comparing the calculated patterns with experimental data. This optimization procedure enables the extraction of precise quantitative information about the structural, compositional and microstructural characteristics of the samples. The refinement process is assisted by refinement engines like FullProf (Rodríguez-Carvajal, 1993), which require significant user intervention and expert guidance. The quality of the refinement also relies on the data and refinement strategy employed, making it challenging to determine, a priori, the best parameter refinement order.

This relatively manual process contrasts with the increasing rate of powder diffraction data collection in recent years. This surge is primarily driven by the rise of materials discovery research through high-throughput screening (Curtarolo et al., 2013) and in situ and operando experiments (Liu et al., 2019; Saurel et al., 2021) which follow the evolution of the sample under study in dynamic conditions. The adoption of materials acceleration platforms (Flores-Leonar et al., 2020; Castelli et al., 2021; Stier et al., 2023; Lombardo et al., 2022) and the increased access to large-scale user facilities such as synchrotrons and neutron sources (Wang et al., 2018; Atkins et al., 2022; Saurel et al., 2021) are expected to fuel this trend. Consequently, there is a pressing need for software tools that can efficiently accelerate the analysis and processing of data, thereby facilitating an increased rate of scientific discovery. Another crucial consideration is the democratization of access to highly automated and large-scale experiments. This entails that the user base should have seamless on-site interaction with the software infrastructure. This accommodation is essential for users with diverse skill levels, including individuals who may not possess specialized skills in software development or scripting. By eliminating this potential obstacle, the efficient utilization of such platforms can be ensured.

One way to address this issue is to build graphical user interfaces (GUIs) that facilitate the use of the underlying logic and programs. In the context of powder diffraction pattern refinements, notable examples include commercial packages such as JADE (Jennings, 2021) and TOPAS (Coelho, 2018), or academic packages such as GSAS-II (Toby & Von Dreele, 2013), Profex (Doebelin & Kleeberg, 2015), MAUD (Lutterotti, 2010) and FullProf (Rodríguez-Carvajal, 1993). This last is included in the FullProf Suite, a set of crystallographic programs that provide GUIs, such as WinPLOTR (Roisnel & Rodríquez-Carvajal, 2001), for visualization, Rietveld analysis, and pre- and post-processing of neutron and X-ray diffraction data. Other programs, such as Sr.Rietveld (Tian et al., 2013) and AutoFP (Cui et al., 2015), offer GUIs with extended functionalities, which are specifically designed to facilitate refinement strategies for batches of experimental patterns. However, to the best of our knowledge, these programs are not compatible with modern versions of Python and have not been actively maintained for several years.

Considering all the above-mentioned points, the FullProfAPP software is presented here. This application, built in Python, acts as a wrapper for the FullProf software. It provides a user-friendly GUI and offers a high-level abstraction layer that enables the utilization of highly automated and flexible refinement workflows. With FullProfAPP, users can easily access and make use of all the capabilities of FullProf through a streamlined and intuitive interface. The software aims to simplify the refinement process by automating various tasks such as full-profile phase searches, sequential refinements using data derived from operando and/or in situ experiments, and refinement of large amounts of data resulting from high-throughput experiments, while providing a more user-friendly experience. Yet, FullProfAPP still allows users to access and manipulate the actual data and input files, enabling further processing with other utilities available in the FullProf Suite. FullProfAPP implements various functionalities by exploiting modern and efficient libraries to perform the data treatment and visualization of both experimental and calculated patterns. It incorporates full-profile phase search algorithms and highly automated refinements of batches of diffraction patterns with customizable workflows. These algorithms are distributed across multiple concurrent processes, harnessing the computational power of modern multicore computers and workstations to parallelize batch refinements and full-profile phase search tasks. Furthermore, it includes a client that interacts with materials databases, providing users with the ability to download crystallographic information files (CIFs) directly through the GUI. As of the present date, FullProfAPP is available in Windows and Linux operating systems.

2. The FullProfAPP program

2.1. Structure of the program

FullProfAPP is built around the refinement engine FullProf (Rodríguez-Carvajal, 1993). It efficiently manages input/output (I/O) data streams and executes the necessary commands to seamlessly operate a variety of Rietveld refinement workflows on powder diffraction data sets. The structure of the program is shown in Fig. 1, providing a clear overview of its components and their interconnections. In the following, a detailed description of the program's structure is given, outlining its key elements and their functionalities (as implemented in Version 1.2.3). The GUI is the component that controls the functionality of the program. It is implemented using PyQt5 (https://riverbankcomputing.com/software/pyqt/intro), a Python wrapper for Qt, which is a popular C++ library for GUI development. With the GUI, users can perform the following tasks:

Figure 1 The components of the FullProfAPP software. Right-angled rectangles indicate functionality, while rounded rectangles indicate the libraries, programs or APIs that the functionality depends on. A yellow background indicates that the software piece is written in (or aimed to be used in) Python. A green background indicates code written in Fortran. A pink background indicates a generic response format (JSON in this case).

(i) Visualize the powder diffraction data and refinement results. The visualization functionality is provided by PyQtGraph (https://www.pyqtgraph.org/), a scientific graphics library built for Python. PyQtGraph excels in terms of speed and performance, making it an ideal choice for rendering and displaying scientific data, especially when presented in large sets. It offers smooth real-time interactivity with 2D graphs and straightforward support for PyQt5-based applications. Therefore, PyQtGraph is the natural choice for a graphics library for FullProfAPP.

(ii) Process the diffraction data to extract the background signal and peak positions and intensities. The background signal is fitted using the asymmetric least-squares smoothing method (Eilers, 2003) as implemented in the PRISMA software (Flores et al., 2022). Peak positions and intensities are extracted using one of SciPy's (https://scipy.org/) peak-finder algorithms. The GUI enables easy variation of the method's input parameters, allowing the user to have immediate feedback on the resulting background shapes and peak positions.

(iii) Interact with the Crystallography Open Database (COD; http://www.crystallography.net/cod/) through OPTIMADE (Andersen et al., 2021), an application programming interface (API) with a common specification for several materials databases. FullProfAPP allows users to download CIFs containing the necessary information to perform subsequent Rietveld refinements using the CIFs_to_PCR program as implemented in the FullProf Suite.

(iv) Refine the loaded experimental patterns using the Rietveld method as implemented in FullProf. FullProfAPP offers several workflows that achieve a variety of tasks, such as full-profile phase searches on the downloaded CIFs; simulation of X-ray diffraction or neutron powder diffraction profiles for preliminary visual checks on the experimental data; manual and automated Rietveld refinements where users can select flexible workflows for the refinement order; and sequential refinements where those flexible workflows are run along the pattern series, largely preventing refinement divergences. While most operations related to the pattern's refinements are I/O bound, some internal crystallographic calculations are also done using the CrysFML library (Rodriguez-Carvajal & Gonzalez-Platas, 2002) and interfaced with FullProfAPP through Forpy (https://ylikx.github.io/forpy/index.html). The following sections offer more details and examples for the above-mentioned workflows.

The application is distributed under the GNU General Public Licence Version 3 (GPLv3). The program can be downloaded from either the BIG-MAP AppStore (https://big-map.github.io/big-map-registry/), the FullProf website (https://www.ill.eu/sites/fullprof/) or CIC energiGUNE's web­page (https://cicenergigune.com/en/fullprof-app). Within the distribution, installers for both Linux and Windows are provided, such that the user has no need to install a Python interpreter with all the dependencies and can readily use FullProfAPP. However, users will still need to keep the FullProf Suite updated to the latest version. The source code is found in the GitLab repository at https://gitlab.com/d7081/fullprofapp. Documentation showing the details of the GUI and working examples can be found on the website https://fullprofapp.readthedocs.io/en/stable.

2.2. The GUI

The application's main window is divided into four sections (Fig. S1 in the supporting information), the toolbar, the graphics area, the terminal and materials area, and the protocols and file system area. Each of these sections is resizable by dragging splitters that exist within the areas.

2.2.1. Toolbar

The toolbar contains four menus, `File', `Pattern', `Materials' and `View'. The `File' menu is used to select the working directory for the project, save the current state or load a previous state of the application. All the data generated by FullProfAPP are saved in the working directory and classified into folders that are named after the type of job, i.e. FPSEARCH, MANUAL, FPAUTO and FPSEQ for phase searches and manual, automatic and sequential refinements, respectively. Each job type is described in detail in the following sections. The loading/saving of the application state is done through an INI file with a .fpapp extension. This file can be loaded through the `File' menu or directly with a double click on the file in the Windows or Linux file manager.

The `Pattern' menu is used to load, edit and export experimental powder diffraction patterns, backgrounds and peak information. The `Import Patterns' action allows the user to load one or many diffraction patterns in different formats. The use of an instrumental resolution function (IRF, available in different formats, see Section S1 in the supporting information for a full description) to account for instrumental profile parameters is mandatory to work with the current version of FullProfAPP. The `Automatic Background' and `Peak Detection' actions are used to extract background points from the experimental patterns and detect peak positions and intensities, which can be edited by activating the `Edit Background' or `Edit Peaks' modes. Lastly, the `Export Background' and `Export Peaks' actions allow the user to export background and peak information to a text file for later use.

The `Materials' menu enables two actions. The `Search in Remote Database' action allows the user to filter crystallographic phases in the COD, download their corresponding CIFs into the working directory and load them into the application. For example, by searching for phases that are known to contain {Fe, O} while optionally considering the inclusion of {Li, P}, the application can automatically download all CIFs in the Fe–O phase diagram and retrieve those phases that also involve Li or P. However, recognizing that sometimes the querying criteria provided by the application may be too general, resulting in long waiting times, or that users may possess their own CIFs, FullProfAPP also offers the flexibility to search for a collection of CIFs within the user's drives. In that case, the `Search in Local Database' action allows users to select CIFs that are already stored on their system.

Lastly, the `View' menu contains actions that allow visualization of the results of the Rietveld refinement jobs. The actions `Current (.prf)' and `Current (WinPLOTR)' plot the profiles resulting from the most recent refinement process inside the graphics area or from executing WinPLOTR (Roisnel & Rodríquez-Carvajal, 2001), respectively. The actions `Select (.prf)' and `Select (WinPLOTR)', on the other hand, plot the refined profiles from any other folder in the system. The `Current Results (.mic/.sum)' and `Select Results (.mic/.sum)' actions plot a summary of the results obtained from the refinements, such as weight fractions, cell parameters and atomic positions, amongst others.

2.2.2. Graphics area

The graphics area contains all the functionalities for the visualization of the experimental data and the results from the refinement runs. It contains tabbed sections that are shown at different stages of the analysis. The first `Exp. Data' tab is always visible and contains the loaded experimental patterns, as well as the background points and peak positions and intensities (if requested). The `Prf Data' and `Summary' tabs are hidden at the start but become visible after the first successful run of a Rietveld refinement. In particular, the `Prf Data' tab shows the experimental profiles superimposed on the total calculated profiles, the individual phase profiles, the difference between experimental and calculated profiles, and the positions of the hkl reflections. The `Summary' tab, on the other hand, gathers and visualizes important parameters that result from the refinement runs. A tickable parameter tree is implemented that allows the user to select parameters of interest for further visualization and analysis.

This area also contains a small toolbar which is meant to help the user navigate between all the patterns and refined profiles that are loaded into the application. This toolbar contains the `Pattern Checks' menu, which acts as a selector for loaded patterns that can be chosen for further analysis. In short, the patterns that are marked as `ticked' will be used in subsequent refinement jobs, while those marked as `unticked' will be disregarded. However, there is one exception to this rule when it comes to sequential refinements. In sequential refinements, FullProfAPP searches for the first pattern file that is marked as `ticked' and utilizes its refinement data as the input for the next pattern's refinement. This process continues until there are no more patterns to refine or until the last `ticked' pattern is found. The following sections will delve into further details about this process, providing a more comprehensive understanding of its operation and implications.

Additionally, the `Contour Plot' button located in the toolbar allows the user to plot contours easily on the loaded data. It allows playing with the colour scale by moving the mouse wheel and adding `Ctrl' and `Shift' modifiers to select upper and lower bounds of the colour scale, respectively.

2.2.3. Terminal and materials area

The terminal and materials area contains the following tabs: `Terminal', `Queried Materials', `Selected Materials' and `Results'. The `Terminal' tab contains a display that allows the user to obtain information about the state of execution of the programs that are run in the background, such as FullProf and CIFs_to_PCR. It also contains the `Kill' and `Clear' buttons, which immediately stop the processes that run in the background and clear the display text, respectively.

The `Queried Materials' tab includes a table that contains crystallographic information from the CIFs that were loaded with the `Search in Remote Database' or `Search in Local Database' actions (see Section 2.2.1). The user can inspect the contents of each CIF that correspond to each row on the table by double clicking on the table items. The user can also filter the table entries by coincidence of the text that is written in the editable text section above. Additionally, this tab includes `Submit Selection' and `Simulate Selection' buttons. The former saves the selected phases from the `Queried Materials' tab into the `Selected Materials' tab, which are then going to be used in subsequent refinements. The latter, on the other hand, prepares FullProf simulations of the selected phases and compares them with the experimental pattern that is currently plotted as a reference.

Lastly, the `Results' tab includes an additional table containing some basic information on the selected phases after a successful Rietveld refinement run.

2.2.4. File system and Protocols area

This area contains two tabs, `File System' and `Protocols'.

The `File System' tab shows the folder and file structure of the working directory, where it is possible to inspect the contents of the files or delete them by interacting with the `Show File' and `Delete' actions that appear by right-clicking on the items.

The `Protocols' tab contains expandable buttons that contain parameter fields and utilities that are necessary to run the corresponding Rietveld refinement protocols. These protocols achieve a variety of tasks by making FullProfAPP automatically run Rietveld refinement jobs in a controlled way using FullProf. FullProfAPP implements three options, `Full-Profile Phase Search Protocol', `Automatic Refinement Protocol' and `Manual Refinement'. The next section provides detailed information for each of these protocols.

2.3. Protocols

2.3.1. Full-Profile Phase Search Protocol

As the name of the section indicates, FullProfAPP implements the full-profile search–match algorithm originally devised by Lutterotti et al. (2019). Unlike traditional search–match approaches, this protocol uses the Rietveld method to discern the constituent phases of the experimental samples. While a detailed description of the method can be found in the original paper, the main aspects of the algorithm and the small variations in the program's implementation are introduced in the following.

In this protocol, the set of candidate phases S₁ = {p₁,…, p_N} and an initially empty set of detected phases S₂ = { } are defined. Regarding the Rietveld refinement process, two distinct types are used. The first type focuses on fast refinements and is used in the scanning step. The second type, which employs more stringent fitting criteria, is used to quantify phase fractions. The order in which the model parameters are refined remains the same for both types of refinement and proceeds as follows. Initially, the scale factors are refined. Subsequently, if the weight fractions of the phases fall below a user-specified threshold (f_R0), those phases are not considered further in the refinement process. Next, if the weight fractions of the remaining phases surpass another weight fraction threshold (f_R1), the program proceeds to refine the cell parameters and constant 2θ shifts. Finally, the crystallite size and strain parameters are refined, as well as the overall isotropic displacement parameters, for weight fractions larger than a third threshold (f_R2).

This algorithm runs fast refinements for each phase in S₁. Each phase is then ranked according to the following custom figure of merit (FoM):

Here, the FoM has been adapted to work with FullProf. R_wp is the weighted profile factor, v_r (v₀) is the refined (initial) cell volume, f_r is the weight fraction, 〈D〉 is the average apparent crystallite size and 〈ɛ〉 is the average maximum strain. The superscript p indicates that the values correspond to a phase in S₁ that is being evaluated in the refinement cycle. The parameters a, b, c and d are weighting coefficients that alter the value of the FoM in favour of phases that fulfil certain conditions. The parameters c and d become zero if 〈D〉 > D_min and 〈ɛ〉 < ɛ_max, respectively, where D_min = 20 Å and (see the description of the microstructure in Section S1 of the supporting information).

After all phases of S₁ have been scanned, the phase with the highest FoM is removed from S₁ and appended to S₂. Additionally, if the weight fractions that resulted from the scan are smaller than a certain threshold (f_S1), those phases are also suppressed from S₁. At this point, only a single phase is present in S₂. The algorithm continues by scanning all the remaining phases in S₁ with the one that was newly added to S₂ and, once more, taking the highest-FoM phase from S₁ and including it in S₂. Before including the new phase, the program checks for duplicated phases that may already exist in S₂. Here, two phases are considered as duplicated if the similarity index (de Gelder et al., 2001) between their respective calculated diffraction profiles is above a predefined threshold. If this is the case, the phase is not included in S₂ but it is still removed from S₁. Then, the phase with the next highest FoM is considered and the same checks are performed. After a new phase is accepted, FullProfAPP initiates a quantification Rietveld refinement of the phases in S₂, aimed at achieving a more precise fit and obtaining accurate values for the phase weight fractions. If the quantification results in phases with weight fractions that fall below the threshold (f_S0), those phases are eliminated from S₂. At this point, FullProfAPP proceeds to start another cycle to select the next phase from S₁ to be added to the refinement process.

This whole process is repeated until one of the following conditions is met: (i) the last phase that was added to S₂ is removed because its weight fraction is below the threshold f_S0, (ii) there are no more phases in S₁ or (iii) the number of phases in S₂ is over a certain limit. Since the Rietveld refinements are performed one phase at a time, FullProfAPP is designed to capture refinement errors effectively from the FullProf standard output stream. In this case, if the errors originate from the scanning Rietveld step of S₁, the FoMs of the related phases are set strictly to zero, meaning that they will rank last in subsequent steps. Similarly, if the error originates during the quantification step, the cause of the error is assigned to the last phase that entered S₂. In such cases, the problematic phase is eliminated from the list and the search–match algorithm is terminated.

2.3.2. Automatic Refinement Protocol

This is the most general way of using FullProfAPP. This protocol window provides several functionalities for the user to perform Rietveld refinements in flexible user-defined refinement sequences. The protocol window is mainly divided into two sections, `Patterns' and `Workflow'.

The `Patterns' section is used to load the refinement parameters (instrumental parameters, background, profile parameters, crystal structure etc.), linking them to the experimental patterns that were selected using the `Pattern Checks' selector menu (see Section 2.2.2). These refinement parameters can be loaded from an existing PCR file (standard FullProf input file) or can be generated using the phases present in the `Selected Materials' tab (see Section 2.2.3) and the background from the graphics area. This is achieved by executing the program CIFs_to_PCR.

Once all the necessary input information is loaded and linked to the corresponding experimental diffraction patterns, the Rietveld refinement strategy can be selected in the `Workflow' section (Fig. S2). The selection of the refinement sequence is done using the tickable parameter tree. The items on the tree correspond to refinable parameters, and expanding the tree items will show the phases and crystallographic sites to which the refinements will be applied. Checking the items in the tree and selecting them as part of the workflow will update the panel to the right of the tree and show all the parameters that are set to be refined and the ones that are set fixed. The user can add further steps to the refinement by subsequently ticking items in the tree and selecting them as part of the workflow. FullProfAPP will then run this user-defined refinement process for each of the experimental patterns that were selected for processing.

FullProfAPP gives several options to tune the Rietveld analysis further. For instance, users can impose soft linear constraints between two or more parameters. In this case, the refinement procedure tries to fulfil the condition that the linear relation between the selected parameters, scaled by some user-defined coefficients, is equal to a given value within a tolerance. Users can also define hard constraints between the refinement parameters. In this case, the constrained parameters are forced to change according to the coefficients given by the user. For example, if two parameters are related by coefficients 1.0 and −1.0, any increase in the value of the former must be matched with an equal decrease in the value of the latter.

Users can select the `Profile Matching' mode in a particular phase if no information about the crystal structure is available. FullProfAPP offers two ways of performing the `Profile Matching' analysis. The first one uses constant scale factors, meaning that the LeBail fitting method (Le Bail et al., 1988) is applied. The second option uses constant relative intensities that are initially fed from a structure factor file (.hkl) and allows for the scale factor of the phase to be refined.

The `Automatic Refinement Protocol' window can also be used to treat sequential data. Here, when the `Sequential Mode' box is ticked, FullProfAPP assumes that the loaded experimental patterns constitute a series in which the refinement parameters of a particular pattern are related to those of the previous pattern within the sequence (e.g. temperature ramps, electrochemistry or similar scenarios). The user can interact with the protocol window exactly as explained above, but keeping in mind that there are some differences in the handling of the loaded patterns. For example, in contrast to the default mode, where only those patterns that were selected in the `Pattern Checks' selector menu (see Section 2.2.2) would be treated, if the sequential mode is activated all the patterns between the first and last selected ones will be considered. In this case, the selected patterns work as `checkpoints' that the user can employ to mark parts of the pattern series where qualitative differences appear with respect to the starting point, such as the appearance of a new phase. The user can first focus on these `checkpoints' to provide well refined initial parameters, and then FullProfAPP will check for all unique phases that may appear between `checkpoints' and prepare a single input file to be run in sequence. FullProfAPP connects the input parameters of the current pattern's refinement with the output parameters of the previous pattern, and it additionally monitors the weight fractions of the studied phases to automatically switch off the refinement of those that fall below a 2% threshold, which allows for sequential refinements that are robust and effective in avoiding divergences.

2.3.3. Manual Refinement

The `Manual Refinement' window allows users to interact with the input refinement parameters in the same way they would with a text editor for typical FullProf PCR files. In fact, FullProfAPP shows most of the contents of the loaded PCR files formatted in tables that contain editable cells. In order to minimize the common errors that occur when editing the values and flags of the PCR files, some cells are kept fixed while others react automatically to the user's changes (e.g. number of background points, number of excluded regions or number of crystallographic sites). Some of the tables contain tickable boxes that activate additional refinement parameters and modes. Among these, FullProfAPP supports the following options: (i) micro-absorption parameters; (ii) parameters necessary for bond-valence sum calculations; (iii) phase-dependent constant 2θ shifts, which are useful when the phases that contribute to the diffraction pattern are at different optical centres of the diffractometer, such as with samples formed of several polycrystalline thin films; (iv) anisotropic displacement parameters for the crystallographic sites; (v) anisotropic strain broadening and Lorentzian strain broadening; and (vi) anisotropic size broadening with the spherical harmonic expansion. The last two depend on the specific Laue class of the phase, which is inferred from the space group, and the corresponding parameters are automatically updated in the `Manual Refinement' window.

Before delving further into the use cases and examples for FullProfAPP, it is important to acknowledge that FullProfAPP has certain limitations and may not be suitable for handling more advanced analyses that require specialized knowledge of FullProf, such as symmetry modes, magnetic structure refinements, single-crystal diffraction and others. In such cases the user must default to interacting directly with FullProf through the FullProf Suite. Thus, FullProfAPP should be viewed as a user-friendly interface that automates and extends functionality for high-throughput powder data treatment, and helps ease the entrance barrier for those users who need a fast and easy way of handling their data with FullProf.

3. Examples

In the following, we present the different functionalities of FullProfAPP. As already stated, FullProfAPP always requires the user to work with an IRF file. We recommend reading the corresponding text in the supporting information (Section S1) to obtain more information about the defaults that FullProfAPP uses to run Rietveld refinements with FullProf.

3.1. Egyptian makeup pigments

This example demonstrates a full-profile phase search conducted on an Egyptian makeup pigment sample using the methodologies presented in Section 2.3.1. The pigments discussed in the original work (Walter et al., 1999) dated from the period between 2000 and 1200 BC. Rietveld analysis of the X-ray powder diffraction patterns revealed the presence of naturally occurring minerals like galena (PbS), cerussite (PbCO₃) and gypsum (CaSO₄·2H₂O), alongside synthetic compounds such as laurionite (PbOHCl) and phosgenite (Pb₂Cl₂CO₃). The diffraction patterns were obtained on the DW22 beamline of the LURE laboratory (Orsay, France; λ = 0.9627 Å). To build the list of candidate phases, a series of CIFs were downloaded using the `Search in Remote Database' action in the `Materials' menu (see Section 2.2.1). The CIFs were downloaded from the COD, restricting the composition to phases that contained either Pb or Ca, and that optionally contained C, O, S or Cl. After eliminating bad entries, such as phases with low symmetry (e.g. P1 or ) and unspecified space groups, a total of 366 phases were selected as candidates for the phase search process.

The FullProfAPP implementation of the search–match algorithm of Lutterotti et al. (2019) correctly detects the phases that were originally reported (Walter et al., 1999). Table 1 and Fig. 2 show the results of the phase search analysis. Overall, the analysis conducted on a Hewlett–Packard 250G8 Notebook personal computer (Intel Core i5-1135 G7 at 2.4 GHz with 16 GB RAM) took approximately 6 min to complete, ensuring accurate operation of the search–match algorithm with a complex phase mixture within a reasonable time frame. This is, however, considerably longer than the timings reported in the original work for candidate lists of similar size. The reasons for this slower time are multifold: (i) the workstation used for the analysis is considerably less powerful than the setup used in the original work, resulting in slower processing times; (ii) there is a significant overhead associated with the execution of FullProf and the updating of the GUI state, which can make the overall process I/O bound, again leading to slower execution times; and (iii) each FullProf execution requires preparing the corresponding input PCR files, thus further slowing the analysis. We also note that the results of the refinements can be improved by taking a more careful and manual approach (Fig. S3). Nevertheless, the resulting weight fractions are very close to those obtained with the search–match algorithm.

Table 1
Egyptian makeup samples: results of the search–match algorithm on the 366 candidate phases downloaded from the COD, including phases with a composition containing Pb or Ca, and optionally containing C, O, S or Cl

Phase	COD ID	Formula	Space group	Weight fraction (%)	Weight fraction (%) (manual)
Laurionite	9008250	PbOHCl	Pcmn	38.68	38.89
Cerussite	5000208	PbCO3	Pmcn	21.95	21.32
Phosgenite	9009573	Pb2Cl2CO3	P4/mbm	16.97	17.41
Galena	9008694	PbS		4.6	4.73
Gypsum	2300258	CaSO4·2H2O	I12/c1	17.80	17.65

Figure 2 Results of the phase search algorithm on the Egyptian makeup samples. χ^2 = 7.86 and Rwp = 14.5. The experimental X-ray powder diffraction pattern (blue curve) is compared with the calculated profile (red line). The black line denotes the difference curve between them and green vertical lines indicate the Bragg reflections corresponding to each of the identified phases.

We argue that this `brute force' approach will hardly ever be the best solution to a sensible phase search analysis on powder patterns containing unknown phases. Instead, first pre-screening the candidate phases with the traditional search–match approach [as implemented in programs such as Match! (https://www.crystalimpact.com/match/) or Bruker's Diffrac.EVA] and then performing a full profile phase search is probably more efficient, especially when dealing with well crystallized laboratory-synthesized samples, where the set of possible phases is much more limited. However, we believe that FullProfAPP can become a powerful assistant when the user aims to screen automatically a series of dozens of patterns belonging to a given chemical system (e.g. high-throughput exploration of compositions or synthesis conditions within a defined chemical space), where the expected phases and possible secondary phases and impurities can be predicted by the user.

3.2. Operando studies

3.2.1. Two-phase reaction between LiFePO₄ and FePO₄ under battery operation

This section describes experiments carried out on the hard X-ray absorption and powder diffraction beamline BL16-NOTOS of the ALBA Synchrotron with modified 2032 coin cells (Herklotz et al., 2016) (experimental details given in Section S2.1) to follow the evolution of the active material (LiFePO₄) upon oxidation for a certain galvanostatic current applied between given potential limits. Due to its phase-separating thermodynamics, the oxidation (lithium deintercalation) of LiFePO₄ results in the formation of a new phase, heterosite FePO₄, with a very similar ortho­rhombic crystal structure (space group Pnma), which starts to nucleate, increasing its phase fraction at the expense of LiFePO₄. At the end of oxidation only FePO₄ is expected to be present. Indeed, the evolving presence of the pair of phases LiFePO₄ and FePO₄ is visible in the continuous fading of some diffraction peaks and the strengthening of others [Fig. 3(a)]. Additionally, a nearly constant signal of the Al current collector is present. Setting up FullProfAPP to perform a sequential Rietveld refinement on these data is very straight­forward. After downloading and selecting the entries from the COD for the previously mentioned phases LiFePO₄ (COD ID 1101111), FePO₄ (COD ID 1525576) and Al (COD ID 4313206), the phases are loaded in the `Automatic Refinement Protocol' window only for the first diffraction patterns in the sequence, while the rest of the patterns are left `unticked' (Section 2.2.2). The refinement strategy is then selected with the tickable tree items. The refinement parameters are subsequently added to the refinement process in the following six-step workflow: (i) scale factors, (ii) constant 2θ shifts (zero shifts), (iii) background, (iv) cell parameters, (v) strain parameters and (vi) crystallite size parameters. The Al current collector is highly textured and is thus treated with the LeBail method (Fig. S2). Lastly, the `Sequential Mode' is activated, asking FullProfAPP to run the full sequence of diffraction patterns while monitoring the weight fractions of the selected phases in order to control the refinement of the parameters that are likely to produce divergences.

Figure 3 (a) A contour plot of the LiFePO4–FePO4 synchrotron operando data in the 11.5–30.0° 2θ range. (b) Evolution of the weight fractions (in wt%) of LiFePO4 (orange) and FePO4 (blue) phases resulting from the automatic sequential Rietveld refinements. (c) Results of the calculated profiles with the individual contributions of the phases of the tenth pattern in the sequence. The experimental X-ray powder diffraction pattern (blue curve) is compared with the calculated profile (red line). The black line denotes the difference curve between them and green vertical lines indicate the Bragg reflections corresponding to LiFePO4 and FePO4. Green, violet and purple patterns correspond to the individual calculated profiles of LiFePO4, FePO4 and Al, respectively. Note that the few unindexed peaks in the 12–16° 2θ range are due to other cell components and were not included in the refinement.

The refinement of the entire sequence using the above-mentioned six-step workflow is completed in under 5 min, with a sequence χ² average of 2.8 and with maximum and minimum values of 1.4 and 4.6, respectively. Fig. 3(b) shows the evolution of the weight fractions of LiFePO₄ and FePO₄, along with a series of patterns corresponding to an oxidation process. Note that these plots are a direct output of FullProfAPP, which required minimum editing to be included in Fig. 3(b). Note also how, even when refining the cell parameters and the strain (and size) parameters for both LiFePO₄ and FePO₄, the calculation did not diverge in the regions where the weight fractions were close to zero. Fig. 3(c) shows an example of a refined profile for the tenth diffraction pattern in the sequence. Visual inspection shows a good fit of the experimental pattern.

3.2.2. Delithia­tion and lithia­tion of layered LiNi_1/3Mn_1/3Co_1/3O₂ under battery operation following a solid solution reaction

In this experiment we used the operando LeRiChe'S Cell Version 2 (Choudhary et al., 2023) on the BL16-NOTOS beamline at the ALBA Synchrotron (experimental details in Section S2.2) to track the temporal evolution of the NMC111 material upon oxidation (delithia­tion) and reduction (lithia­tion) processes. In contrast to the LiFePO₄ example detailed in Section 3.2.1, the NMC class of materials exhibits a succession of steps involving either the formation of solid solutions or phase transitions, depending on composition (relative amounts of Ni, Mn and Co), temperature and applied currents (Zhou et al., 2016; Wang et al., 2022). Within this example, we followed a full charge/discharge cycle of the operando cell, during which we observed a smooth evolution of the diffraction peaks of a single NMC111 phase consistent with a solid solution behaviour [Fig. 4(a)]. Constant intensity peaks corresponding to the Al current collector and two Be windows are also present.

Figure 4 (a) A contour plot of the NMC111 synchrotron operando data in the 8.7–51.6° 2θ range. (b) and (c) Evolution of (b) the c and (c) the a cell parameters (in ångströms). (d) Results of the calculated profiles of the sixth pattern in the sequence. The experimental X-ray powder diffraction pattern (blue curve) is compared with the calculated profile (red line). The black line denotes the difference curve between them and green vertical lines indicate the Bragg reflections corresponding to NMC111 and Al. Note that, as marked in the legend of graph (d), the individual contributions of the phases are hidden to improve visibility. FullProfAPP allows this to be done by clicking on the legend item itself. All the plots were directly extracted from the GUI and required minimal editing.

The setup for the sequential refinement of the series of patterns is similar to the one described in Section 3.2.1. First, we downloaded the relevant phase data from the COD for NMC111 (COD ID 4002443) and Al (COD ID 4313206) and excluded from the refinement the regions where the peaks corresponding to the Be windows appear. The CIFs are loaded in the `Automatic Refinement Protocol' only for the first pattern in the sequence (see Section 2.2.2). The refinement strategy is then selected by adding steps to the workflow using the tickable parameter tree (Fig. S2). It consists of eight steps: (i) scale factors, (ii) constant 2θ shifts (zero shifts), (iii) background, (iv) cell parameters, (v) crystallite size parameters and (vi)–(viii) three anisotropic strain broadening parameters of the quartic form related to the Laue class as implemented in FullProf. These last turned out to be extremely important for the refinement due to the anisotropic volume fluctuations of NMC111 during oxidation and reduction. The Al current collector is highly textured and it is refined with the LeBail method. The `Sequential Mode' is activated, instructing FullProfAPP to copy the refined parameters of the previous pattern to the next one.

The refinement of 20 patterns using the eight-step workflow took approximately 2 min to complete, with a sequence χ² average of 13.6 and with maximum and minimum values of 9.7 and 20.2, respectively. Figs. 4(b) and 4(c) show the evolution of the c and a cell parameters of NMC111, where the characteristic behaviour of this class of materials is well captured. For example, the c cell parameter initially increases from 14.31 to 14.52 Å upon lithium extraction due to the increased electrostatic repulsion between the O anions of adjacent transition metal layers (Wang et al., 2022). Upon further oxidation the c cell parameter starts to decrease, from 14.52 to 14.39 Å, indicating a diminishing interlayer distance. In contrast, the a cell parameter decreases all along the oxidation process, with a small increase at the end. This behaviour is shown to be fully reversible during the reduction process. Fig. 4(d) shows the refinement profile of the sixth pattern in the sequence. A visual inspection reveals a good agreement with the experimental pattern, capturing even the anisotropic broadening of the peaks at high angles, which was attributed to anisotropic strains.

3.2.3. Lithia­tion of charged spinel Li_1−xMn₂O₄ (0 ≤ x ≤ 1) under battery operation following a mixed mechanism

This section describes an experiment carried out on the powder diffraction beamline BL04-MSPD at the ALBA Synchrotron (Fauth et al., 2013, 2015) (experimental details in Section S2.3), which follows the evolution of LiMn₂O₄ (LMO) upon the second cell cycle during reduction. The redox behaviour of LMO involves a combination of phase separation and solid solution mechanisms, as were outlined in Section 3.2.1 for LiFePO₄ and in Section 3.2.2 for NMC. Notably, a visual check of the contour plot illustrated in Fig. 5(a) reveals that three distinct phases emerge throughout the reduction process. The reduction of the starting delithia­ted material (λ-MnO₂ with space group ) initiates with the formation of a new spinel phase with the same space group . Upon lithia­tion this phase displays smoothly evolving diffraction peaks, indicating a solid solution reaction. When the reduction of LMO proceeds beyond a certain threshold a third phase appears, which corresponds to the fully reduced spinel LiMn₂O₄ phase (). Note that the oxidation and reduction processes of LMO spinels may also involve forming other secondary phases with space groups P6₃mc (Palacín et al., 2000; Dupont et al., 2000) and P2₁3 (Bianchini et al., 2015). However, it is difficult to extract concrete conclusions about the presence or absence of these phases with the data presented herein. Therefore, the data were fitted by considering spinel LMO phases () only.

Figure 5 (a) A contour plot of the LMO synchrotron operando data in the 17–26° 2θ range. (b) Evolution of the refined weight fractions of the four spinel LMO phases. (c) Evolution of the cell parameters (in ångströms) of the four spinel LMO phases. Regions of low weight fraction have been edited out for the sake of visibility and clarity. (d) Results of the calculated profiles in the second phase transition region (pattern number 35). The experimental X-ray powder diffraction pattern (blue curve) is compared with the calculated profile (red line). The black line denotes the difference curve between them and green vertical lines indicate the Bragg reflections corresponding to the four spinel LMO phases. Brown, yellow, violet and dun patterns correspond to the calculated profiles of these phases. Note that the few unindexed peaks in the 7–16° 2θ range are due to other cell components and were not included in the refinement.

After loading the pattern series and retrieving the phase data from the COD for LMO (COD ID 1514009) and Al (COD ID 4313206), we accommodated the distinct phases that appear along the pattern series using the `Simulate Selection' button to generate duplicates of the selected CIFs with appropriate starting values for the cell parameters. The setup for the sequential refinement of the diffraction patterns presented more difficulties than the examples shown in Sections 3.2.1 and 3.2.2 due to the confluence of highly overlapped diffraction peaks with evolving peak positions. In such a scenario, employing a single refinement strategy for the entire pattern series proved unsatisfactory. However, despite the limitations, a stable refinement without divergences was achieved, owing to the systematic addition of refinement parameters and the continuous monitoring of the phase weight fractions. This contrasts with the normal sequential mode available in FullProf. Instead, to refine the pattern series effectively, we did the sequential refinement in sections. Initially, focus was directed to the second phase transition region, selecting the pattern where the diffraction peaks appeared clearly distinguished. Using the `Automatic Refinement Protocol' window, we configured the refinement strategy by subsequently incorporating steps in the workflow using the tickable parameter tree (Fig. S2). The refinement strategy consisted of six steps: (i) scale factors, (ii) constant 2θ shifts (zero shifts), (iii) background, (iv) cell parameters, (v) Lorentzian strain broadening and (vi) Lorentzian size broadening. Fig. 5(d) shows the results achieved. To continue analysing the remaining data further, backward and forward sequential refinements were performed starting from this initially refined pattern using the same workflow. In all cases the Al current collector was refined in LeBail mode. Further sectioning of the pattern series was necessary to refine highly overlapped regions, where the peak broadening parameters were kept constant.

The refinement of the 48 patterns, including the required manual steps, took around 15 min to complete. The sequence χ² average was 10.2, with minimum and maximum values of 3.8 and 40.7, respectively. We note that the high χ² values were more prominent at the beginning of the reduction process, characterized by intense sharp peaks that could not be assigned to any cell component, and which abruptly appeared and suddenly vanished. In Fig. 5(b), the weight fraction evolution for the refined phases across the pattern series is depicted. It illustrates the formation of the three main spinel LMO phases (). However, to describe the second phase transition accurately, an additional spinel phase had to be considered as a short-lived intermediate phase. Its precise origin is unclear and it could simply be due to a delayed solid solution of LMO or inhomogeneities in the active material, but investigating this further is out of the scope of this work. We note that the combined weight fraction of the shown phases does not add up to 100% due to a fixed phase with a 10–15% weight fraction that was considered to represent inactive particles within the cell. Fig. 5(c) shows the evolution of the cell parameters for the four cubic phases observed throughout the pattern series. The reduction of LMO results in lithium intercalation, which leads to an increase in cell parameters for LMO phases. Our results agree well with the results reported in other operando studies (Sun et al., 2002).

3.2.4. Maricite NaMnPO₄ solid-state synthesis

In this example (experimental details in Section S2.4), we quantitatively follow the temporal evolution of the reactants, intermediates and final products of the maricite NaMnPO₄ (m-NMP) synthesis process during a temperature ramp. Following the previous examples, we first loaded the pattern series, extracted the information about the background and downloaded the corresponding CIFs for all the constituent phases. Guided by visual checks on the low-angle characteristic diffraction peaks [Fig. 6(a)], we used Diffrac.EVA and the COD and PDF-2 2022 (International Centre for Diffraction Data, Pennsylvania, USA; https://www.icdd.com) databases to index a total of seven candidate phases that appear along the pattern series. The diffraction peaks of the patterns at 30°C were described by the phases MnCO₃ (; ICSD refcode 8433; Inorganic Crystal Structure Database, FIZ-Karlsruhe, Germany; https://icsd.fiz-karlsruhe.de/index.xhtml), Na(NH₄)H(PO₄)·4H₂O () (ICSD refcode 2036) and (NH₄)₂(HPO₄) (P2₁/c) (ICSD refcode 2799), formed after an initial reaction of Na₂CO₃ and NH₄H₂PO₄ during the ball-milling step. Similarly, the diffraction peaks at 600°C were also well described with the maricite NaMnPO₄ phase (Pmnb) (ICSD refcode 31331). To index the three remaining intermediates we used the low-angle characteristic peaks as a reference [Fig. 6(a)] and assigned them to Na₃HMnP_1.8O₈ () (ICSD refcode 159096), a hypothetical phase Na₃HMnP₂O₈·½H₂O isostructural to Na₃HZrSi₂O₈·(H₂O)_0.525 (C2/m) (ICSD refcode 186147) and an intermediate phase (NH₄)Mn(PO₄)·H₂O isotructural to (NH₄)Mg(PO₄)·H₂O (Pmn2₁) (ICSD refcode 5148).

Figure 6 (a) A contour plot of the evolving diffraction peaks for maricite NaMnPO4 (m-NMP) synthesis with temperature. (b) Evolution of the refined weight fractions of the seven indexed phases. (c) Evolution of the average apparent size of m-NMP. (d) Results of the calculated profiles for a pattern where highly overlapped diffraction peaks exist and where broadening parameters are kept fixed (pattern number 45). The experimental X-ray powder diffraction pattern (blue curve) is compared with the calculated profile (red line). The black line denotes the difference curve between them and green vertical lines indicate the Bragg reflections corresponding to the five identified phases. Light-violet, light-green, red, dark-violet and purple patterns correspond to the calculated profiles of these phases.

The setup of the sequential refinement of the 119 diffraction patterns again required some degree of manual intervention, like the case presented in Section 3.2.3. Again, we divided the pattern series into sections and manually refined specific patterns where the contributions of the individual phases were clearly distinguishable. Backward and forward sequential refinements were then performed starting from these well defined points. Following the previous sections, we set up a six-step refinement workflow: (i) scale factors, (ii) constant 2θ shifts (zero shifts), (iii) background, (iv) cell parameters, (v) Lorentzian strain broadening and (vi) Lorentzian size broadening. To keep the refinement results consistent, the size and strain broadening parameters of low-crystallinity intermediates were kept constant when their contributions became masked by significantly broadened peaks.

The refinement of the entire series took approximately 1 h to complete. The sequence χ² average was 8.6, with minimum and maximum values of 3.1 and 18.3, respectively. Fig. 6(b) shows the evolution of the weight fractions of the seven phases that we indexed and refined. At 30°C the three reactant phases appear in approximately equal proportions. Immediately after, at ∼70°C, the first intermediate starts forming, corresponding to the hydrated phosphate Na₃HMnP₂O₈·½H₂O (C2/m). We note that this intermediate phase is already present in small amounts at 30°C. We also note the formation of a minority phase (NH₄)Mn(PO₄)·H₂O (Pmn2₁). Then, when the temperature increases to ∼170°C, the Na₃HMnP₂O₈·½H₂O (C2/m) phase transforms into the Na₃HMnP_1.8O₈ () dehydrated intermediate, which becomes the majority phase until it reacts with the remaining MnCO₃ () at ∼300°C to form the final maricite NaMnPO₄ (Pmnb). Finally, upon temperature increase to 600°C, m-NMP continues to crystallize, showing a steady increase in the average apparent size with a minimum at 130 Å and a maximum at 2160 Å [Fig. 6(c)]. Note that Fig. 6(c) features a long tail at low temperature where the value of the average apparent size remains unchanged. This is one of several such examples where the contribution of the m-NMP phase to the diffracted intensity is masked under broad peaks [Fig. 6(d)], making the refinement of the broadening parameters difficult. It also signals that the m-NMP crystals do not grow until a threshold temperature of ∼350°C is achieved.

4. Conclusions

This article provides an overview of FullProfAPP, a GUI that utilizes FullProf as the refinement engine. FullProfAPP provides a high-level abstraction layer, enabling users to execute different flexible refinement strategies. These strategies facilitate a variety of tasks such as full-profile phase searches and automatic refinements. FullProfAPP enables users to perform highly automated data treatment and Rietveld refinements on large batches of data in a simple and intuitive way, while still allowing interaction with the files generated by FullProf.

To illustrate the capabilities of FullProfAPP, we have introduced several examples, different in nature and covering some, but not all, potential use cases. For example, we show that FullProfAPP correctly distinguishes the five constituent phases in an Egyptian makeup sample out of a pool of 366 candidates in a few minutes. It is also able to perform sequential refinements of powder diffraction data from electro­chemical operando experiments with flexible workflows and parameter monitoring, largely preventing divergences. We tested three common cases where the electrode materials evolve through very different redox mechanisms involving the formation of a secondary phase (LiFePO₄), of a solid solution (NMC111) or a combination of the two (LMO). We additionally examined a complex case, where we tracked the evolution of the different phases observed during the in situ monitoring of the solid-state synthesis of m-NMP. In all cases, the setup of the sequential refinements using FullProfAPP was very straightforward and the complete pattern series were efficiently refined in minutes. The last two cases proved to be more difficult due to the confluence of overlapped diffraction peaks and evolving peak positions. Nonetheless, satisfactory results were obtained with minimal manual intervention.

All in all, FullProfAPP could be useful to those users who routinely deal with large volumes of data, helping them reduce the delay between data collection and analysis time, and thus assisting, for example, on-site scientific and technical decisions on synchrotron experiments. Additionally, it can serve as an entry point for novice users, by reducing the entrance barrier to the full functionality of the FullProf Suite.

The program is distributed under an open-source GPLv3 licence with installers for Windows and Linux operating systems.

5. Related literature

For further literature related to the supporting information, see Järvinen (1993), Rodriguez-Carvajal et al. (1991), Stephens (1999) and Thompson et al. (1987).