A step toward standardization: development of accurate measurements of X-ray absorption and fluorescence

This paper explains how to take the counting precision available for XAFS (X-ray absorption fine structure) and attenuation measurements, of perhaps one part in 10⁶ in special cases, to produce a local variance below 0.01% and an accuracy of attenuation of the order 0.01%, with an XAFS accuracy at a similar level leading to the determination of dynamical bond lengths to an accuracy similar to that obtained by standard and experienced crystallographic measurements. This includes the necessary corrections for the detector response to be linear, including a correction for dark current and air-path energy dependencies; a proper interpretation of the range of sample thicknesses for absorption experiments; developments of methods to measure and correct for harmonic contamination, especially at lower energies without mirrors; the significance of correcting for the actual bandwidth of the beam on target after monochromation, especially for the portability of results and edge structure from one beamline to another; definitions of precision, accuracy and XAFS accuracy suitable for theoretical model analysis; the role of additional and alternative high-accuracy procedures; and discusses some principles regarding data formats for XAFS and for the deposition of data sets with manuscripts or to a database. Increasingly, the insight of X-ray absorption and the standard of accuracy needed requires data with high intrinsic precision and therefore with allowance for a range of small but significant systematic effects. This is always crucial for absolute measurements of absorption, and is of equal importance but traditionally difficult for (usually relative) measurements of fluorescence XAFS or even absorption XAFS. Robust error analysis is crucial so that the significance of conclusions can be tested within the uncertainties of the measurements. Errors should not just include precision uncertainty but should attempt to include estimation of the most significant systematic error contributions to the results. This is essential if the results are to be subject to deposition in a central accessible reference database; it is also crucial for specifying a standard data format for portability and ease of use by depositors and users. In particular this will allow development of theoretical formulations to better serve the world-wide XAFS community, and a higher and more easily comparable standard of manuscripts.

1. Introduction

Many XAFS (X-ray absorption fine structure) beamlines around the world have high flux, high counts per second and hence high statistical precision per data set. This depends upon experimental methodology. Variance is often due to factors other than the purely statistical. In an X-ray absorption setting the International Union of Crystallography (IUCr) decades ago proposed a round-robin to assess the reproducibility of the measurements of attenuation coefficients of several key standards by a wide range of laboratories (Creagh & Hubbell, 1987, 1990; Barnea & Mohyla, 1974; Creagh, 1999). The consequences of the conclusions of this round-robin remain highly relevant in the pursuit of accurate XAFS, for investigating several key systematics which would render the data non-portable, non-transferable or simply with significant and unassessed systematic errors. Considered, and yet uncritical, compilations of experimental absorption measurements have been made particularly by the extensive work of Hubbell (Hubbell, 1982; McMaster et al., 1970), but also as represented in work such as Henke et al. (1993). Theoretical studies have been particularly useful in empirically revealing dramatic inconsistencies in these experimental results (claimed accuracies often some 10 or 20 times smaller than the point precision, and up to 30% away from a probable reference value) (Chantler, 1995; Chantler et al., 2009; Saloman et al., 1988; Berger et al., 1998), but have particular limitations near edges or in particular regimes (Chantler, 2000).

Similarly, the rapid development of XAFS for local or dynamical structural investigations, entwined with the rapid development of synchrotron facilities, has cried out for the development of routines for determining precision, key systematics and accuracy so that a data deposition format and representation might be useful and portable from one beamline or experimental configuration to another; or for comparison with or fitting by theoretical methods. International meetings of the XAS community have been held over decades to address this issue, perhaps notably including Kobolov et al. (2002), Ascone et al. (2012) and Oyanagi (1988), but with little success until now. The only readily accessible sources for calibration of XAFS have been the Lytle database (https://ixs.iit.edu/database/data/Farrel_Lytle_data), with highly variable content, the spectral profiles of Wong (1999) used as standards by numerous beamlines, but without absolute calibration, and edge definitions based on Deslattes et al. (2003), of variable provenance given broadening and instrumental effects unaccounted for. However, a critical mass of experts are now calling to address and improve this situation experimentally and internationally. Further, the recent activities of the IUCr International Commission on XAFS have been working with the wider international XAS community towards a set of common definitions and approaches, so that results from different authors may be compared on a uniform footing (Ascone, 2011a,b). This augurs well for an increasingly self-critical appraisal of approaches, and the consequential improvement of data to the point where a deposition format for good quality data sets can naturally be defined, for use by all researchers.

The intrinsic variance due to X-ray counting statistics may be illustrated by considering typical ion chambers, with fluxes from the beamline of 10⁹ to 10¹¹ and with output from the analog-to-digital convertor (ADC) after a gain of, for example, a maximum reading of 10⁶ counts per second (c.p.s.), with a typical dwell time of 0.1–10 s, and hence integrated counts (for the monitor or upstream ion chamber) of the order of 10⁵ to 10⁷. When this is a true representation of the X-ray counting statistics and is normally or binomially distributed, this represents an intrinsic precision corresponding to a standard deviation of 0.1% or better. Yet almost no experimental data sets have presented results with this level of accuracy or even precision, whether applied to interpreting features, structural determinations, bond lengths or thermal parameters. Why is this? What can be done to improve our XAFS, absorption and fluorescence experiments? This is the key topic of this manuscript.

The X-ray extended range technique (XERT) is a method for measuring absorption and scattering to high accuracy. Hence it can measure absorption coefficients, fluorescence signals and structures near absorption edges, like a detailed extended XAFS (Chantler, 2009, 2010; Chantle, Islamr et al., 2012). While XERT has been applied to high-accuracy diffraction measurements and scattering measurements, as well as fundamental absorption measurements, it has found the greatest application with XAFS. It uses a few key principles, which are also of course principles for high-accuracy XAFS. Some of these concepts are used by many other groups, and some of these are detailed in this discussion.

2. Modes of XAFS operation and intrinsic counting statistic

While the monitor (upstream) count can easily lead to a counting precision of 0.1% or better, an absorption mode experiment will be limited by the detector (downstream) signal and counting statistic. With ADC gains adjusted so that count readings are up to 400000 counts per second, a modest attenuation through the air path, ion chambers and windows, and a sample absorption following standard beamline criteria (a log ratio of 1–2), the intrinsic counting precision can still correspond to a standard deviation of 0.1–1% or better. We assume that the number of X-rays absorbed exceeds the recorded count (i.e. that the true statistic is in excess of that based on the recorded count), which is often the case.

However, for dilute systems in absorption mode, the log ratio for the (active species) signal can easily be less than 0.1, even if the attenuation follows standard criteria, and integrated counts over a similar period may typically be 10³–10⁵. Then the intrinsic (counting) precision for the near-edge region may correspond to a standard deviation of ∼10% or better.

When XAFS is collected in fluorescence stepping mode, with a typical fluorescence yield f of 0.2–0.9 (K-edge, Z 20) and a solid angle (for the whole detector) of 1–0.01 sr, corresponding to an efficiency factor of ≃ ≃ 0.1–0.001, with a sample absorption of ∼0.1, absorption/self-absorption of ∼0.1–0.4, and a similar dwell time as suggested earlier (0.1–10 s), then a plausible intrinsic counting precision standard deviation can be 1–10%. These estimates depend critically upon the geometry, energy range and sample. Conversely in fluorescence mode collection for dilute systems, where the background scattering and self-absorption might follow conventional criteria (log ratio 1–2) but the signal might be much weaker (log ratio < 0.1), the intrinsic (counting) precision for a near-edge region might correspond to a standard deviation of ∼5–30%. Clearly the information content which can be extracted from one mode or geometry will be significantly different from that of a different set-up.

Many experiments use multiple scans, typically three to six repeated scans and so one can use an integrated count time per step, ideally decreasing the intrinsic counting standard error by or . Many standard implementations use fast, or slower, scanning modes. Depending upon the intrinsic instrumental resolution, an estimate of the intrinsic precision can use an integrated counting time per eV or equivalent step-size. Often the total time might be represented by some 500–1500 steps (data points in energy steps) in stepping modes or the equivalent in scanning modes, with effective step-sizes below 0.5 eV in the near-edge region, with a range for XAFS of ∼1 keV or 20.

With third-generation synchrotrons, wigglers and undulators, numerous advanced XAFS beamlines can provide very high fluxes across central energy ranges. If samples are stable under this flux density and this is then complemented by very high detector linearity and very fast stable detectors (Stötzel et al., 2012; Lutzenkirchen-Hecht & Frahm, 2001), then (intrinsic) standard errors can be reduced by a factor of ten or so, or alternatively the counting time can be reduced by 100 or 1000, whether for dynamic investigations or simple throughput.

A conclusion is that intrinsic standard errors can certainly reach below 0.1%; but are often more commonly 1%. Even 10% standard errors can still be valuable for XANES of dilute systems. Key limitations naturally include the X-ray count absorbed and detected in the monitor and detector, and the linearity of the detector chain, together with the possible sample damage and particularly for fluorescence modes the active fluorescence signal compared with the background. However, in many data sets the observed variance (purely precision) is usually much larger than this intrinsic counting estimate due to beam instability and drift, monochromator settling time and other noise contributions. In fluorescence modes this may be increased by an additional factor of ten compared with absorption modes.

3. Absorption equation

Rather than the Beer–Lambert ideal equation, we must take account of detector dark current and noise, and air and window paths, and of course the monitor signal to derive an attenuation coefficient. While the detector signal is always normalized to the monitor signal in XAFS experiments, it is not uncommon that air and window paths and (ion-chamber) dark currents are ignored. Often XAFS experimenters do not measure the spectrum without the sample, leaving no opportunity to obtain the real attenuation curve or the homogeneity of it. Normalizing the edge jump to 1 as is common in software packages further obscures problems with samples. As an example, Figs. 1 and 2 show a data set with and without correction for the dark current signal. This is a relatively strong example in that the sample was thicker than a conventional XAFS experiment, but typical in that these same qualitative changes in relative attenuation and transform are common (Glover & Chantler, 2007).

Figure 1 Effect of dark-current correction on the raw attenuation signal for metallic Mo. The thickness chosen corresponds to a log ratio of 4, emphasizing the qualitative distortions which remain present down to and below log ratios of 2. Many users feel that they have an accurate undampened structure, but should explicitly include the influence of dark current and avoid the structural modification. Dark-current levels of up to 1000–4000 counts per second have a very large impact even on an almost perfectly thin sample with perhaps 10^6 counts s^−1, especially on relative amplitudes, peak locations and structural solutions at the 0.1–0.4% level, although less visible to the eye than this example.

Figure 2 Effect of dark-current correction on the versus k signal following Fig. 1.

The basic equation should analyse upstream and downstream repeated measurements,

where I is the attenuated intensity, I₀ is the unattenuated intensity and D is the recorded dark current. The subscripts s and b represent the measurements with a sample in the path of the X-ray beam and without a sample in the path, respectively. The first part of the equation links up with past usage in the literature, although we might recommend using u for the monitor detector upstream and d for the downstream detector as given in the second form of the same equation. This allows for measurement and correction for electronic noise, air path and detector efficiency (Chantler, 2009; Chantle, Islamr et al., 2012). In particular, if the amplification of the monitor is A_u and of the detector is A_d, and if the corresponding electron yields per X-ray are Y_u and Y_d with (ion chamber) efficiencies and , s refers to the sample coefficients and a refers to the air path, then

This equation automatically makes allowance for two important and energy-dependent non-linearities in the response function, reducing amplitude distortions. This is one of the key principles of XERT (Chantler et al., 1999; Tran, Chantler et al., 2003) and is reflected in its potential to obtain higher accuracies and a lower variance (higher precision). Similarly, the equation emphasizes that the result is an uncertainty or standard error in , the linear attenuation coefficient multiplied by the sample thickness, or in , the mass attenuation coefficient multiplied by the integrated column density , and not directly either or . Since XAFS is ideally based on the absorption fine structure or (pe = photoelectric coefficient, i.e. absorption), this can be significant where scattering is significant and energy dependent.

The equation also emphasizes that intrinsic (counting) precisions on the monitor and detector yield uncertainties more than their sum in quadrature; and that a standard error in the logarithm corresponds to a percentage uncertainty in the detector or monitor counts. Nonetheless, assuming that the monitor is set or tuned to almost 10⁶ counts s⁻¹ (photons absorbed) and that the detector ion chamber is matched, equation (3) still indicates that a counting statistical precision can yield uncertainty of the log ratio with a standard error of <0.1–1%.

The conventional criterion expected at many beamlines is to use the original or modified Nordfors criterion () (Nordfors, 1960; Creagh, 1999), but in fact a much wider range is both possible and necessary either across an absorption edge or to interrogate key systematics. Probably all XAFS researchers have found that if their sample obeys the Nordfors criterion above a K-edge then it is often `impossible' for it to obey it below the edge for important XANES structure or pre-edge features, and often these data are considered unusable or of low quality. However, the range suffers a loss of less than a factor of two in standard error at the extremes (Fig. 3), and so remains ideal for all absorption experiments and can be maintained across absorption edges (de Jonge et al., 2005, 2007; Glover et al., 2008). Using a daisy-wheel approach, the full range of this graph and more can be sampled using reference foils.

Figure 3 Usually a sample is chosen to fulfil the traditional Nordfors criterion; however, the extended range criterion plotted provides a very similar statistical information content (Chantle, Tran, Barnea et al., 2001). If the experiment crosses the edge, the traditional definition is often unrealistic for real samples; and if experiments use more than a single sample thickness or concentration, as is argued herein, then the extended range is critical as it permits investigation of key systematics.

4. Systematics affecting observed precision: harmonic components

The equation above assumes explicitly that the beam is ideally monochromatic and the sample thickness and density are perfectly uniform, which in turn also assumes that the beam is parallel. Of course these assumptions are usually not obeyed. The most discussed source of failure of the above equation relates to the presence of additional harmonics. Assuming a single contaminating harmonic with a fraction x of harmonic photons in the incident beam, this can be represented by

where F refers to the main energy of the beam (usually the fundamental) and H refers to the dominant harmonic energy (often the third harmonic in silicon monochromation). Ideally the energy dependence of the fundamental energy of interest and the relevant harmonic for both yields, ion-chamber efficiencies and absorption, and air path should be included in this functional. This issue is addressed by Barnea & Mohyla (1974), Nomura et al. (2007) and Nomura (2012), but also particularly by Tran, Chantler et al. (2003), Tran, Barnea et al. (2003), de Jonge et al. (2006) and Glover & Chantler (2009).

This issue of harmonic contamination has been addressed by many authors since the work of Barnea & Mohyla (1974), and there are discussions for example in Lee et al. (1981) and Goulon et al. (1982), as well as more recently in on-line tools (Newville, 2001, 2004) and recent texts (Bunker, 2010). Lee et al. (1981) particularly look at signal-to-noise but neglecting dark-current and air-path contributions. This report discusses the problems of background removal and E₀ definition, and correlation effects. Nonetheless, it is primarily a handbook of the methodology used at that time for interpreting Fourier transform XAFS. In this manuscript we are primarily concerned with attaining high-accuracy and high-information-content data for absorption XAFS, rather than discussing processing ambiguities or problems [especially E₀, bond accuracies and correlation errors, for which see particularly Glover et al. (2010) and Glover & Chantler (2007)].

Goulon et al. (1982) particularly discuss `leakage' of X-rays from straight-through paths, which in the modern era with synchrotron radiation should be carefully eliminated, and harmonic contamination. Despite the absence of a description of variables, this is a useful summary and provides expressions for higher-order harmonics, limited bandwidth or monochromator resolution (see §5), scattered radiation, detector ion recombination and sample inhomogeneity. However, it should be noted that equations therein do not partition the total flux and are therefore unnormalized. This work also treats fluorescence radiation and hence fluorescence XAFS, as does recent work by Ravel et al. (2012) and Chantler, Rae et al. (2012). While it may be attractive to collect many of these ideas and detailed investigations into some future review, it is not the purpose of the current discussion.

The key resulting observation is that detuning of a double-bounce monochromator can be effective at medium-to-high energies, but that typically somewhere between 5 and 7 keV detuning is inadequate (for most synchrotron beamlines), especially if a beam purity of 99.99% or better is required. In the lower-energy X-ray regime, obtaining a high purity requires a mirror operation to exclude higher harmonics. However, there are at least two diagnostic techniques which are valuable for directly and experimentally diagnosing and measuring the harmonic content of a tuned, detuned or other incident beam. These are the multiple-foil measurement technique (Tran, Barnea et al., 2003; de Jonge et al., 2007) and the daisy-wheel harmonic measurement (Tran, Chantler et al., 2003; Tran, Barnea et al., 2003; de Jonge et al., 2005; Glover et al., 2008).

Some beamlines approximate these using a weakly calibrated filter bank, which is still useful as an indication of harmonic content but usually is not configured to provide an accurate quantification. A third approach, based on a continuous wedge measurement, is under development. All of these methods require a set of samples of common density and composition, preferably elemental, which are either independently or internally calibrated. Any deviation from a linear relation with increasing thickness (or integrated column density), especially for lower energies, leads then to a characterization and observation of the harmonic content. Other non-linear effects with thickness or attenuation, including a poorly determined dark current, have different and orthogonal signatures and so can be separated by their dependence upon log ratio and energy.

The advantage of the multiple-foil technique is that it is able to simultaneously address numerous other systematics and relates directly to the experimental samples of interest. The advantage of the daisy-wheel harmonic technique is that it samples across a huge range of attenuation and thereby defines both the dark-current value accurately and can be based on direct experimental measurements of standard and in-beam calibrated reference foils. Both techniques are remarkably consistent and can explicitly measure harmonic contamination down to or one part in 10000. In particular, we have used these techniques to measure harmonic contamination in different modes to, for example, 18.70 0.07, 1.09 0.02 or 0.18 0.01% (Fig. 4). In these cases, despite a large harmonic on different beamlines which would quite invalidate absorption or XAFS accuracies and interpretation, the quantification of these components allows an accurate determination of the correct , irrespective of monochromator detuning and especially in an energy range from 10–11 keV down to 5 keV. Such tests also directly investigate the linearity of detectors, relating to such issues as detector saturation or intrinsic non-linearities (Barnea et al., 2011).

Figure 4 Three trial runs using the daisy-wheel method to demonstrate the high accuracy of the diagnostic for harmonics. Even when 19% of the incident monochromated beam is a higher harmonic, the measurements can characterize this to 0.07% and permit accurate determination of attenuation and XAFS (Tran, Barnea et al., 2003). This routine can detect 0.18% contributions of higher harmonics with an accuracy of 0.01% to yield a highly accurate determination of XAFS and attenuation.

5. Systematics affecting observed precision: bandwidth

A similar issue for a non-ideal monochromated beam relates to the incident bandwidth or distribution in energy (within the single fundamental peak). For a typical double-bounce monochromator, the incident energy is both spatially dependent, and hence dependent upon the beam height or aperture size, and also dependent upon any detuning or differential heat load on the primary versus secondary crystals. The profile of the beam is not normally distributed in energy, but can still be approximated by a FWHM. While theoretical modelling and beamline programs can estimate this broadening, the actual value has great significance in the portability of results, especially wherever the XAFS signal shows sharp structure, i.e. below, at the edge and in the XANES region. While there have been numerous studies of the stability of basic edge location with bandwidth, it has also been shown clearly that attenuation coefficients in the neighbourhood of an edge can vary by up to 1.4% in a strongly energy-dependent manner, changing both the apparent structure and shape (de Jonge et al., 2004). In particular, the experimental definition of the edge energy is often given as the extremum of the derivative of with E, and this value can vary by several eV as a consequence of different bandwidths. For example, molybdenum has a typical edge width of almost 20 eV, with two extrema of the derivative separated by 10 eV of magnitude affected by even as little as a 1.6 eV bandwidth.

In an experiment at the APS in Chicago on molybdenum, the bandwidth in a monochromated beam was measured to be 1.57 0.03 eV at an energy of 20 keV. There are several methods by which this beam width can be measured. The most direct but difficult method is to use a (calibrated) high-resolution monochromator on the otherwise incident energy profile. While we have put a double-crystal monochromator after a primary double-crystal monochromator (de Jonge et al., 2007), this has complex alignment difficulties. Easier is a six-circle goniometer following the beam (de Jonge et al., 2004), which can characterize the energy of the beam but then it is difficult to characterize the width with a single-bounce reference crystal given other experimental broadening. This is similarly true for a powder diffraction spectrometer, which can determine a width but which tends to be dominated by scanning or plate resolution and methodology (Rae et al., 2006). If the core-hole lifetime is known then this can be removed from the overall edge-width, possibly determined from the derivative signature. Unfortunately, it is normal that hole-widths and the overall edge-widths are not well determined, so may have 20% uncertainties in the derivation. Additionally, bound–bound transitions and pre-edge features will complicate this kind of determination.

Perhaps the easiest is a direct modelling of the observed shift and broadening of an edge in an XAFS measurement, as a function of log ratio (thickness). This is a clean method in that the hole-width, satellite and pre-edge features, and even any shake-up features, are uniform for all sample thicknesses, but that the influence of bandwidth is quite distinct. Perhaps surprisingly, this minor broadening leads to corrections of 0.35–1.4% of the values of attenuation around the edge (Fig. 5), which is quite significant for both XANES and low-k studies. As a consequence, or proof if you will, the variance after correcting for this systematic changed from a standard error of 1% down to a standard error of 0.1% (de Jonge et al., 2005).

Figure 5 Percentage correction to the measured [μ/ρ]. The bandwidth of the incident beam. Here a small finite bandwidth (1.57 eV) measured with an accuracy of 0.03 eV (model-dependent) permits the effect of 1.4% on the absorption measurement to be corrected, and thereby a platform-independent absorption coefficient to be obtained (de Jonge et al., 2004). Cross symbols, plus-sign symbols and diamond symbols refer to corrections relating to three foils of different thicknesses: 10, 50 and 25 µm, respectively.

6. Definition of terms

There are three independent but critical estimates of standard error of absorption or attenuation for an XAFS or XANES measurement. Any one of these can permit a comparison of quality of particular XAFS data sets; without any of these we can neither responsibly improve quality or even define it. The intrinsic (counting) precision discussed earlier indicates only what might be possible – not in any way what has in fact been achieved.

The (absolute) accuracy is the [authors'] best determination of all statistical and systematic uncertainties and their effects upon the data [that is, the estimate (standard error) of the mean sample discrepancy from the true parameter on whichever axis], energy (eV), wavelength (m) or wavenumber (cm⁻¹), mass attenuation coefficient (cm² g⁻¹), mass absorption coefficient (cm² g⁻¹), X-ray cross-section (barns atom⁻¹), form factor f (electrons atom⁻¹), XAFS or effective wavenumber k (Å^-1 or cm^-1), , , , spatial transform r (Å), , etc.

The precision is the reproducibility of a result under re­peated measurements. The measurement may be completely incorrect with a good precision; but the reproducibility must be assessed by making repeated measurements. This includes the statistical precision due to the photon and electron hole-pair counting statistics, but also includes any source of noise or systematic error with a distribution function, which will add to the variance and standard error.

For XAFS, when collecting data on an axis of versus E, or similar variations, we must define a third measure: the XAFS accuracy or relative accuracy. While precision reflects the point reproducibility, but makes no allowance for systematic errors, the XAFS accuracy includes all those systematics which have an energy dependence or structural dependence across and above the edge and therefore which affect the point-to-point correlation and scaling. Such errors will affect the bond distances, Fourier transforms, broadening, ionization state, coordination, etc. This includes all statistical uncertainty and variance, including variance from distribution functions of unknown systematics, and hence includes everything included in the observed measurement of precision. Hence, it includes most components relating to the accuracy but, by contrast, the (absolute) accuracy will include explicit uncertainty for a constant offset of the x and y axes {for energy and , or for and k}. This is the best answer to the question `what uncertainty of information content best represents the XAFS signature for structural fitting?'.

This last measure is often the most important uncertainty for XAFS measurement, especially when fitting on a k or transformed r axis. This is the authors' best determination of all point-to-point variation including energy or k-dependent systematics. It is always larger than the estimate of precision and less than the estimate of (absolute) accuracy. Both XAFS accuracy and accuracy should include the uncertainty of energy drift or scaling. Ideally the relative accuracy in the versus E data is the best approximation to the accuracy of the transformed versus k plot for fitting or comparison to theory.

High XAFS accuracy is needed for high-k sensitivity, for multiple paths in XAFS, for accurate displacement parameters, non-nearest neighbour bond lengths and many other areas of physics or bonding. High (absolute) accuracy is needed for absolute absorption measurements, form factor determinations, measurements of nanoroughness, electron-mean-free paths, etc. High frequency sampling is needed for fine XAFS, avoiding aliasing artefacts, especially for transforms and other applications. Medium XAFS accuracy and good precision is needed for decent XANES. Often good energy calibration is needed. Any comparison with theory needs an estimate of the accuracy or relative accuracy, allowing an interpretation of the significance of the result from a fit.

7. Limitations to precision

Any limitations to precision will likewise be limitations to accuracy and relative accuracy and are hence limitations to the significance of any result or interpretation of XAFS. We have emphasized the effects of harmonics and bandwidth on observed variance (precision). Most direct measures of precision (reproducibility) can be 10–100 times worse than the ideal statistical estimate, owing to noise produced by systematic errors with some asymmetric distribution function, uncorrected or unknown.

Using the full equations above, and characterizing detector linearity and statistics, allows optimization of the precision obtainable and is a necessary precursor for accuracy; this precision should be observed (measured), and propagated in analysis. The measurement is simple: repeat a scan or point measurement preferably three to ten times, use different sample thicknesses if possible and hence derive the variance of any given point of the spectrum.

Apart from those discussed above, limitations to the precision and observed variance especially include flux (limitations), non-linearities, sample damage, monochromator drift, sample scattering, position-dependent self-absorption, roughness, position-dependent detector efficiency, recombination and others. Each of these adds to noise, and hence to measured precision, but often also adds an energy-dependent systematic shift to the experimental results.

Two key topics are raised in detail by Nomura (2012) and Srihari et al. (2012), namely the calibration of energy and the characterization of detector non-linearities. We will attempt not to duplicate that material here, but both issues are obviously critically important in determining and improving the quality of XAFS (Barnea et al., 2011). Common systematic errors in the energy determination of an XAFS or XANES measurement, in part beautifully discussed by Diaz-Moreno (2012), include the drifts and offsets of calibration and monochromation. As well as discussions of energy calibration and stability, there have been major discussions about defects of normalization owing to monochromator or sample reflections (Tran, Chantler et al., 2003; Chantler et al., 2010; Diaz-Moreno, 2012), but there is insufficient space to detail these and many other interesting and important systematics. Rather it is important to investigate each of these but in any data set to be deposited as a reference data set to indicate what is and is not addressed, and what technique is used to address the key issues.

If the functional form of the error versus energy is a constant offset, a single calibration edge to a reference sample might make a good relative correction, though note the effect of bandwidth on this (Glover & Chantler, 2007). If the functional form is a straight line, two edges might allow correction, subject to the accuracy of their calibration. More generally, an accurate measurement of energy is needed across the extended range of energy investigated. For XANES measurements, small relative offsets are often crucial; for XAFS analysis drifts and curvature with energy are often considered more important, because an offset is incorporated within the E₀ edge energy offset parameter in most XAFS fitting routines.

This discussion has mainly concentrated on issues relating to absorption XAFS, that is, measurements of attenuation used to derive XAFS signals rather than the very popular fluorescence XAFS, i.e. measurements of fluorescence used to derive XAFS signals. All of these issues are equally important for fluorescence XAFS, but there are numerous additional issues involved in extracting quality XAFS curves from fluorescence data. Self-absorption, differential counter efficiencies, solid angles, scattering and the fluorescence signal versus background structure all contribute significantly. Rather than extend this discussion further, we refer to a recent publication (Chantler, Rae et al., 2012) which discusses some of the background and key equations.

8. The role of XERT: the X-ray extended range technique

The X-ray extended range technique aims to directly and independently calibrate the monochromated energy, harmonics and bandwidth where feasible, avoiding 3–10 eV or 30–100 eV errors or offsets seen at several beamlines. Historically, XERT has focused on absorption measurements and absorption XAFS. In this context it has used a stepping methodology rather than a continuous scanning methodology; the former, in principle, allows direct statistical sampling and calibration of each energy point (although this is sometimes unrealistic with a fine grid). Energy is stepped commensurate with structure, obtaining a finer grid near edges. The continuous scanning methodology does provide an advantage in sampling all energies at some constant rate in angle or energy, but a disadvantage in that the statistical accuracy in each region is less readily optimized. XERT usually investigates a wide range of energy, well below the edge and far above the limit of much XAFS. It does this in order to: (i) increase the accuracy of the baseline determinations; (ii) investigate the approach of the condensed matter system to the atomic and independent atom limits; (iii) investigate and separate the interplay of scattering and fluorescence contributions; (iv) gain high accuracy in the difficult high-k region. How useful these questions are depends upon the nature of the samples.

Each dimension of experimental inquiry is generally extended, subject to experimental optimization of beam time of course. Key requirements are that multiple samples of solids are prepared, over a much wider range than the Nordfors criterion, as discussed earlier. For solids, these would generally be different sample thicknesses of the same bulk structure. Obviously, the concern is to measure and represent accurately the bulk structure and attenuation of a system. Materials preparation difficulties do lead to limitations in the useful ranges for particular systems; in principle the active should preferably be unchanged, but for t to be increased over a good range. In general, the intent is that multiple thickness foils would be measured at each energy. The easiest way to provide this range for a solution cell is to increase and decrease the active concentration, subject to the condition that phase changes or significant structural changes do not ensue.

Questions of scattering, fluorescence and back-scattering are addressed by measurements of multiple apertures for each sample, placed on two daisy wheels downstream of the monitor and upstream of the detector. In the fluorescence XAFS modality, a downstream detector, similarly equipped, is still an important correction for the back-scattering into the upstream monitor.

As discussed earlier, for each sample–aperture–energy combination, measurements are made of the sample, of the system with the sample removed (blank or background) and of the dark current or detector noise. Each measurement is repeated five to ten times. Before or after the experiment, efforts are made to provide detailed characterization and profiling of the materials to hopefully determine an absolute accuracy of the measurement.

Recent work is investigating the application and development of XERT and high-accuracy measurement to fluorescence XAFS, to low-temperature XAFS and high-temperature XAFS, to phase changes, and to dilute non-crystalline systems: glasses, polymers, composites and solutions. A key question here is what is the maximum information content compared with a noise level or even a degradation time, and is this information content sufficient to answer key structural or other issues?

So the key role of XERT is as a guide towards the improved quality of XAFS, or at least the quality and diagnostic side of any XAFS measurement. Some issues will be more important on some beamlines or in particular experiments compared with others, but all can be improved with a better characterization of precision and significance. Fig. 6 illustrates a typical XERT set-up although this changes significantly with synchrotron and beamline. Some typical comparisons of the ideas presented so far may now be given, by way of illustrating the potential of XAFS and XANES.

Figure 6 Typical set-up for an absorption XAFS XERT experiment at ANBF, Tsukuba. Energy is independently calibrated with a powder diffractometer; harmonics and scattering are independently calibrated using daisy wheels; multiple samples assess a range of other potential systematics; dark current and air path effects are independently measured; and precision is sampled at each point in parameter space.

So what can this achieve? Is the improvement in quality worth the effort? In early implementations of this technique (Chantler et al., 2000; Chantler, Tran, Paterson et al., 2001; Tran, Chantler et al., 2003) we proved that a precision of 0.02% could be achieved, over energy ranges from 5 to 20 keV, even though a series of systematic effects limited that data set to an absolute accuracy of 0.27–0.5%, a record at the time, and also extracted the form factor and its uncertainty; and a precision as low as 0.012% with an accuracy down to 0.07%. The focus here was in understanding the methodology and whether the unknown error sources could be identified and characterized, in order to compare with theory.

In a later stage we developed methods for producing XAFS spectra from 13.5 to 41.5 keV, with (absolute) accuracies down to 0.02% and precision of course superior to that (de Jonge et al., 2005), and indeed up to 60 keV, with (absolute) accuracies down to 0.04% despite the high energies and low fluxes; at the higher-energies flux limitations certainly limited accuracies to 1–3% (de Jonge et al., 2007). We also used this absorption data to apply to key questions of bonding and XAFS structure, by propagating the uncertainties to versus k space, determining the bond length of molybdenum in an absolute sense, without using a relative change with temperature etc., which was accurate to 0.001 Å (standard deviation), in agreement with that from the best X-ray crystallographic analyses within 0.006 Å or 0.2% (Smale et al., 2006; Glover & Chantler, 2007).

An output on a bond length from, for example, IFEFFIT is not necessarily a standard deviation, especially since the input uncertainties are not propagated. In our work these input uncertainties are propagated, so that at least the result represents a propagated uncertainty. However, of course there are uncertainties in other fitted parameters such as backscattering amplitudes, phase shifts, mean-free paths which correlate with, for example, uncertainties of the bond length. The most obvious of these are the determinations of E₀ and the relative energy. Normally a fit should provide the correlated uncertainties in any final result by the off-diagonal elements of the derivative matrix; however, it is well known that in many XAFS analyses the apparent uncertainty underestimates or neglects this correlation. One particular advantage of our studies is that Mo and Au have high symmetry and a single length scale, deliberately avoiding any such correlation; and that our results thereby directly measure the technique with minimal (negligible?) additional correlation uncertainty.

A third concern is that the XAFS data measure the average distance between atoms, whereas Bragg crystal diffraction determines the distance between average positions. These are most decidedly not the same, and in our papers we discuss this in detail. For asymmetric or complex molecules, these two measures can vary quite significantly. However, we must remember that any accuracy defined by XAFS is by definition an accuracy on dynamical bond distance. A comparison of this with an X-ray diffraction difference of mean positions is exactly that. However, by choosing suitable systems [Mo, Au] where these definitions should coincide, we generate a direct test of alternate methodologies.

Similarly at lower energies, we have obtained measurements with (absolute) accuracies of 0.09% from 5 to 20 keV in developments of the technique (Glover et al., 2008), and confirmed the independent results and accuracy claimed by the earlier experiment; have discovered a systematic effect in absorption measurements owing to nanoroughness (Glover et al., 2009), which leads to a technique for measuring nano­roughness non-destructively; and have applied these principles directly to XAFS investigations, obtaining high-accuracy data even in the presence of strong single-crystal diffraction (Chantler et al., 2010) and explained how to obtain such data.

This development has shown that a modest increase in data collection time, by some factor of five to ten, can serve to provide critical diagnostics which can reduce uncertainties and systematics by one or two orders of magnitude; that is, by much more than the plausible improvement in counting statistics. This is of course true because many of these experiments are not now limited by pure photon-counting statistics but by a range of systematics of the order of 1–6%. Addressing these for key diagnostics for key reference samples and for key systems in a sub-field is obviously of high importance. This extra accuracy is essential in assessing limitations and developments of theory, but may not be warranted for samples subject to beam damage which only last for, for example, 10 s. Interestingly, even in such cases there are major advantages to the sorts of diagnostic and error propagation we are discussing.

In an investigation of zinc metal we have raised the issue of precision versus accuracy in the interpretation of data, and introduced the discussion of `relative accuracy' as an important consideration for the analysis of XAFS structure (Rae et al., 2010). While the (absolute) accuracy in that study reached as low as 0.044%, the relative accuracy reached 0.006%, yielding a very highly accurate investigation for XAFS studies. Two studies of gold (Glover et al., 2010; Islam et al., 2010) led to the dynamical bond length of f.c.c. gold being determined to an accuracy of 0.004 Å or 0.1%, allowing discussion of the differences between the dynamical bond length of XAFS and the mean separation of lattice positions in crystallography.

We have applied these methods to XAFS investigations directly in the pursuit of improvements of theory including FEFF and FDM approaches (Kas et al., 2010; Bourke & Chantler, 2010) and discovered effects of nanoroughness, inelastic mean-free path, and structural bonding questions including paths, bond lengths, displacement parameters and subtle conformations in fluorescence (Chantler, Rae et al., 2012). Our current investigations are pursuing complex structures, dilute systems and fluorescence XAFS in addition to the more traditional reference areas. It seems that the opportunities for high accuracy to yield major new areas of science and new information for applied fields is rich and opportune.

9. Data format for XAFS

One major topic of recent discussions is the feasibility and timeliness of a database for XAFS, with some flexible but well defined data format. Clearly if these data are to be available to many users, it should represent good examples of a standard including considerations already presented.

A key question in working towards a data format for XAFS is: what would be required for the deposition of a data set, line-by-line, with a manuscript? If the data set and explanation are not good enough for a journal, then they are not likely to be good enough for data deposition. In some sense we must be guided by past successful efforts in this regard. There are numerous historical examples of XAFS and attenuation data deposited in Physical Review, Journal of Physical Chemistry Reference Data and elsewhere. There have been collegial and long-term efforts by both IXAS and the IUCr Commission perhaps noting particularly the activities of Oyanagi on both organizations over decades (Oyanagi, 1988). Definitions of what is deposited or presented are necessary precursors to a deposition format which everyone can consider, debate or respond to.

For example, the number of alternative presentations of the edge energy or edge offset leads to an infinitude of different possible plots or columns of versus k, and hence is not a useful representation of a data set. For a common set of definitions relating to these, please see https://www.iucr.org/resources/commissions/xafs/xafs-related-definitions-for-the-iucr-dictionary. Note that such definitions are open to revision and that additional clarifications and definitions are and will be needed. In some cases this is just confirming a common mind of the world-wide community, in others there is real disagreement over what can or should be used, and in some cases this disagreement can have sound theoretical grounds, possibly based on different software or theoretical formulations. Wherever the last is true, the purpose of definitions is to clarify and expose these issues to further scientific inquiry. There has been much discussion of these issues at recent conferences, including IUCr and XAFS discussion at the Congress in Madrid and the XAFS tutorial and subsequent scientific sessions: https://www.iucr.org/resources/commissions/xafs/iucr-2011-xafs-tutorial; and including bulletins of the IXAS Society and website. Additionally, we have extensive on-line material and documentation by, for example, Newville, Bunker, Ravel, Rehr to name a few (Ravel et al., 2012).

So some consequences for a suitable deposition format include: the deposited format should not just be a truncated plot of versus k with or without windows. This can create or destroy artefacts from the same raw data set, so is neither reliable nor reproducible. Similarly, no plot of the Fourier transform versus r is viable; this further massages the data idiosyncratically. While many software packages use either of these as active tools to identify key structural features, which remains the key objective of much experimentation, the deposited framework should make each step clear and unambiguous, including the representation of the pre-processed attenuation data.

A minimum set with some uncertainty estimation such as k, , is also not adequate. As a step in the right direction, this would allow estimates of to indicate the significance of a conclusion, but would be highly prone to beamline-dependent systematic errors and hence non-portable. These last three ideas could be included as extra material but must be supplementary to the primary deposition. There will be many questions raised as to the of derived plots. While the dominant features leading to this have been addressed in some works (Smale et al., 2006; Glover & Chantler, 2007; Glover et al., 2010; Chantler, 2010; Chantler, Rae et al., 2012) there remains few implemented methods for achieving realistic estimates of uncertainty in the extracted scale, and it is likely that significant further discussion and testing will be useful in this context.

For whatever format is worked towards, it must be accompanied or presented with a description of the experimental conditions and effects measured and corrected for, so that one data set which incorporates corrections for, for example, three systematics on one beamline can be compared with another data set which incorporates corrections for five (different) systematics on a different synchrotron. In this manner the assumptions and comparisons can be usefully and constructively made. It is likely that this information needs to be embedded as meta-data to be uniquely and safely tied to the data set it represents. As perhaps is clear from the discussion, the table and plot of versus E or perhaps versus E is foundational for all current theoretical methods of analysis, with an estimated and described uncertainty, even if the measurement was relative.

An estimate of uncertainties as columns in the data set is crucial for the assessment of any significance or consistency. The best estimate of uncertainty is the standard error of the relative accuracy, which might be called the XAFS accuracy, but must be distinguished from the (absolute) accuracy.

If any theory is to fit the data on the mass attenuation coefficient scale or linear attenuation coefficient scale , then the most appropriate and most complete uncertainty is the standard error of (absolute) accuracy, which will assess edge jumps, edge energies and offset systematics or unknowns. As a minimum there must be repeated measurements (e.g. at least six to ten) either in scanning mode or in step mode to indicate an estimate of precision.

As a single illustration, we present a few details of the deposited data set for molybdenum, deposited as part of de Jonge et al. (2005) as EPAPS E-PLRAAN-71-012502. This was a table of versus E, including and . For clarity, the percentage accuracy of was also presented as a separate column, and because of the focus of the paper the extracted atomic form factor was presented in the final column, with its uncertainty. Here the data per se were deposited without explanation, but with a header file which has reference to all the details in the primary manuscript, and with a brief summary, although not sufficient for the metadata purpose of cross-comparison: this contains an estimation of uncertainty, which is explained in detail in the manuscript, and summarized in the table. The electronic tabulation includes 425 measurements between 19.56 and 21.454 keV, made at energy intervals down to 0.5 eV. These further measurements include detailed XANES and EXAFS. A direct link to this document may be found in the online article's HTML reference section. The document may also be reached via the EPAPS homepage https://www.aip.org/pubservs/epaps.html or from ftp://ftp.aip.org in the directory/epaps/. See the EPAPS homepage for more information.

The Header and Readme should contain a reference number, the journal citation, authors, title, file description and description of data columns and uncertainties, with units. Some brief comments could explain the type of data (e.g. fluorescence XAFS) and systematics detailed, estimated or corrected for. In later publications and depositions (Rae et al., 2010; Glover et al., 2010) we have emphasized and clarified columns for and separately, and separated and further clarified the estimates of precision and XAFS accuracy versus accuracy. These ideas will hopefully lead to a useful and common deposition format for the future and perhaps thereby a quantifiable accuracy and standard of XAFS and XANES, which can then be improved further. Any feedback from the whole community is very welcome!

10. Summary and conclusions

This paper discusses requirements for XAFS accuracies to approach one part in 10⁴ and similarly to accuracies of absorption to one part in 10⁴ or 0.01%, including the need for dark current and air/window path measurement and normalization, a correct choice of multiple thicknesses for absorption measurements across an extended range, the need to assess and optimize the linearity of every detection system especially including non-linearities like dark current and back-scattering, which are often significantly energy-dependent around and above the absorption edge, and including the effects of bandwidth on the determined edge location and profile especially for XANES comparisons for beamline-independent analysis. Harmonic contamination especially at lower energies without mirrors are strongly non-linear effects and will likewise distort the XAFS response function. We explain definitions of precision, accuracy and XAFS accuracy suitable for theoretical model analysis; and discuss some principles regarding data formats for XAFS and for the deposition of data sets with manuscripts or to a database.