research papers\(\def\hfill{\hskip 5em}\def\hfil{\hskip 3em}\def\eqno#1{\hfil {#1}}\)

Journal logoBIOLOGICAL
CRYSTALLOGRAPHY
ISSN: 1399-0047
Volume 71| Part 9| September 2015| Pages 1799-1811

Accounting for partiality in serial crystallography using ray-tracing principles

aCrystal and Structural Chemistry, Bijvoet Center for Biomolecular Research, Utrecht University, Padualaan 8, 3584 CH Utrecht, The Netherlands, and bM4I Division of Nanoscopy, Maastricht University, PO Box 616, 6200 MD Maastricht, The Netherlands
*Correspondence e-mail: l.m.j.kroon-batenburg@uu.nl

Edited by Z. Dauter, Argonne National Laboratory, USA (Received 7 May 2015; accepted 19 June 2015; online 25 August 2015)

Serial crystallography generates `still' diffraction data sets that are composed of single diffraction images obtained from a large number of crystals arbitrarily oriented in the X-ray beam. Estimation of the reflection partialities, which accounts for the expected observed fractions of diffraction intensities, has so far been problematic. In this paper, a method is derived for modelling the partialities by making use of the ray-tracing diffraction-integration method EVAL. The method estimates partialities based on crystal mosaicity, beam divergence, wavelength dispersion, crystal size and the interference function, accounting for crystallite size. It is shown that modelling of each reflection by a distribution of interference-function weighted rays yields a `still' Lorentz factor. Still data are compared with a conventional rotation data set collected from a single lysozyme crystal. Overall, the presented still integration method improves the data quality markedly. The R factor of the still data compared with the rotation data decreases from 26% using a Monte Carlo approach to 12% after applying the Lorentz correction, to 5.3% when estimating partialities by EVAL and finally to 4.7% after post-refinement. The merging Rint factor of the still data improves from 105 to 56% but remains high. This suggests that the accuracy of the model parameters could be further improved. However, with a multiplicity of around 40 and an Rint of ∼50% the merged still data approximate the quality of the rotation data. The presented integration method suitably accounts for the partiality of the observed intensities in still diffraction data, which is a critical step to improve data quality in serial crystallography.

1. Introduction

X-ray free-electron lasers and high-brilliance undulator beamlines at synchrotrons have been used to perform serial (femtosecond) crystallography, collecting diffraction data from a large number (thousands up to millions) of micrometre-sized or nanometre-sized crystals (Chapman et al., 2011[Chapman, H. N. et al. (2011). Nature (London), 470, 73-77.]; Boutet et al., 2012[Boutet, S. et al. (2012). Science, 337, 362-364.]; Redecke et al., 2013[Redecke, L. et al. (2013). Science, 339, 227-230.]; Gati et al., 2014[Gati, C., Bourenkov, G., Klinge, M., Rehders, D., Stellato, F., Oberthür, D., Yefanov, O., Sommer, B. P., Mogk, S., Duszenko, M., Betzel, C., Schneider, T. R., Chapman, H. N. & Redecke, L. (2014). IUCrJ, 1, 87-94.]; Demirci et al., 2013[Demirci, H. et al. (2013). Acta Cryst. F69, 1066-1009.]). Individual crystals may be hit by an X-ray pulse, thereby producing a diffraction pattern within the 10–50 fs pulse duration, before being vaporized by the transferred energy. This principle of `diffraction before destruction' has been demonstrated by experiments on the Linac Coherent Light Source (LCLS) hard X-ray free-electron laser (Chapman et al., 2011[Chapman, H. N. et al. (2011). Nature (London), 470, 73-77.]). Since the X-ray pulses are shorter than it takes for radiation-induced structural changes to occur, this approach of serial crystallography overcomes radiation damage, which has become a major problem with highly brilliant synchrotron sources (Weik et al., 2000[Weik, M., Ravelli, R. B. G., Kryger, G., McSweeney, S., Raves, M. L., Harel, M., Gros, P., Silman, I., Kroon, J. & Sussman, J. L. (2000). Proc. Natl Acad. Sci. USA, 97, 623-628.]; Ravelli & McSweeney, 2000[Ravelli, R. B. G. & McSweeney, S. M. (2000). Structure, 8, 315-328.]; Burmeister, 2000[Burmeister, W. P. (2000). Acta Cryst. D56, 328-341.]) using conventional rotation methods of collecting data from one or very few larger crystals. The diffraction images in serial crystallography are single snapshots of nonrotating crystals: so-called still images. As opposed to the conventional rotation data, the reflections are not fully integrated but are partials, except possibly when using future pink XFEL beams (Dejoie et al., 2015[Dejoie, C., Smeets, S., Baerlocher, C., Tamura, N., Pattison, P., Abela, R. & McCusker, L. B. (2015). IUCrJ, 2, 361-370.]). The particular orientation of the crystal lattice determines the extent of this partiality, which is a great unknown in the data-reduction process.

The specific challenges in data processing are the indexing of the stills, the reconstruction of full intensities and the merging of data obtained from different crystals, in addition to the handling of huge amounts of data. Three software packages are available to process serial X-ray diffraction patterns: CrystFEL (White et al., 2012[White, T. A., Kirian, R. A., Martin, A. V., Aquila, A., Nass, K., Barty, A. & Chapman, H. N. (2012). J. Appl. Cryst. 45, 335-341.], 2013[White, T. A., Barty, A., Stellato, F., Holton, J. M., Kirian, R. A., Zatsepin, N. A. & Chapman, H. N. (2013). Acta Cryst. D69, 1231-1240.]; White, 2014[White, T. A. (2014). Philos. Trans. R. Soc. B Biol. Sci. 369, 20130330.]), cctbx.xfel from the Computational Crystallographic Toolbox (Sauter et al., 2013[Sauter, N. K., Hattne, J., Grosse-Kunstleve, R. W. & Echols, N. (2013). Acta Cryst. D69, 1274-1282.]; Hattne et al., 2014[Hattne, J. et al. (2014). Nature Methods, 11, 545-548.]) and nXDS (Kabsch, 2014[Kabsch, W. (2014). Acta Cryst. D70, 2204-2216.]). For indexing, rotation-method indexing packages such as MOSFLM (Leslie & Powell, 2007[Leslie, A. G. W. & Powell, H. R. (2007). Evolving Methods for Macromolecular Crystallography, edited by R. J. Read & J. L. Sussman, pp. 41-51. Dordrecht: Springer.]), DirAx (Duisenberg, 1992[Duisenberg, A. J. M. (1992). J. Appl. Cryst. 25, 92-96.]) and LABELIT (Sauter et al., 2004[Sauter, N. K., Grosse-Kunstleve, R. W. & Adams, P. D. (2004). J. Appl. Cryst. 37, 399-409.]) are being used. In 2010, Kirian and coworkers proposed a Monte Carlo integration method that, by averaging large numbers of diffraction spots, averages out the unknown partialities as well as differences in crystal size, beam flux and the incident spectrum (Kirian et al., 2010[Kirian, R. A., Wang, X., Weierstall, U., Schmidt, K. E., Spence, J. C. H., Hunter, M., Fromme, P., White, T., Chapman, H. N. & Holton, J. (2010). Opt. Express, 18, 5713-5723.]). Thousands of diffraction images are needed for this method to converge (Boutet et al., 2012[Boutet, S. et al. (2012). Science, 337, 362-364.]). It is generally believed (White, 2014[White, T. A. (2014). Philos. Trans. R. Soc. B Biol. Sci. 369, 20130330.]) that estimation of partialities could reduce the number of images needed for the Monte Carlo integration method and could improve the data quality. Three approaches have been proposed to estimate partialities. All three use post-refinement to improve the partiality correction factors and scale factors for each image. Kabsch (2014[Kabsch, W. (2014). Acta Cryst. D70, 2204-2216.]) derived an analytical expression for partiality from a Gaussian mosaic spread function. Comparison of ultrafine-sliced rotation images treated as stills or as normal rotation images gave satisfactory results. Kabsch includes a Lorentz factor for still data explicitly. The still data processing is not as good as one would expect, according to Kabsch. He concludes that this may be caused by two-dimensional rather than three-dimensional profile fits and the lack of other unimplemented corrections. White (2014[White, T. A. (2014). Philos. Trans. R. Soc. B Biol. Sci. 369, 20130330.]) considers the overlap of reciprocal reflection volumes with a nest of Ewald spheres and calculates partialities from the distance of reciprocal-lattice points to the two limiting Ewald spheres. Using modelled data, White shows that the partiality estimation improve the data, with significant improvement of the statistics upon post-refinement. Most recently, Sauter (2015[Sauter, N. K. (2015). J. Synchrotron Rad. 22, 239-248.]) and Uervirojnangkoorn et al. (2015[Uervirojnangkoorn, M., Zeldin, O. B., Lyubimov, A. Y., Hattne, J., Brewster, A. S., Sauter, N. K., Brunger, A. T. & Weis, W. I. (2015). eLife, 4, e05421.]) presented a partiality model that is implemented in cctbx.xfel. They calculated the intersection with the Ewald sphere of a spherical reciprocal-lattice point, where the radii of the lattice points are determined by mosaic spread and (asymmetric) beam divergence. Sauter (2015[Sauter, N. K. (2015). J. Synchrotron Rad. 22, 239-248.]) also includes a parameter for the coherently scattering volume of mosaic blocks. Using this approach on XFEL data with post-refinement of crystal orientations, scale factors and beam parameters, the data are improved in quality as judged from molecular-replacement scores, structural refinement and anomalous difference maps (Uervirojnangkoorn et al., 2015[Uervirojnangkoorn, M., Zeldin, O. B., Lyubimov, A. Y., Hattne, J., Brewster, A. S., Sauter, N. K., Brunger, A. T. & Weis, W. I. (2015). eLife, 4, e05421.]). Moreover, they show that reliable structures can be obtained with a lower number of images. Unfortunately, these authors do not mention merging R factors. Another correction that potentially improves Monte Carlo integration convergence for nanocrystals is explored by estimation of the crystal sizes and their corresponding diffraction power, as described by Qu et al. (2014[Qu, K., Zhou, L. & Dong, Y.-H. (2014). Acta Cryst. D70, 1202-1211.]). They show that the geometric correction factor, solely based on the maximum of the Laue interference function for each crystal with size Nx × Ny × Nz, is superior to Monte Carlo integration for simulated data. Although the above efforts were made to improve processed serial crystallography still data, many questions still need to be addressed. Why do the data not improve rigorously with the current partiality-correction models? What factors exactly determine the partiality? Which errors dominate the partiality-estimation schemes?

Here, we describe an extension of the EVAL profile-prediction algorithm to process still images. EVAL is a data-reduction method designed for integrating reflection intensities through profile fitting using ray-tracing simulations (Duisenberg et al., 2003[Duisenberg, A. J. M., Kroon-Batenburg, L. M. J. & Schreurs, A. M. M. (2003). J. Appl. Cryst. 36, 220-229.]; Schreurs et al., 2010[Schreurs, A. M. M., Xian, X. & Kroon-Batenburg, L. M. J. (2010). J. Appl. Cryst. 43, 70-82.]). We derived a general interference function that is valid for crystals of any size and effectively includes the shape transform. The diffraction process is simulated by typically 10 000 rays, which are diffracted by an equal amount of reciprocal-lattice vectors. In the rotation method, we bring reciprocal-lattice vectors onto the Ewald sphere by rotation around the spindle axis. However, in the still diffraction method we calculate the deviation from the exact Bragg condition for each ray and estimate its contribution to the total diffracted intensity using the interference function. By summation, the partiality of a reflection is obtained and, as we will show, also the still Lorentz factor. To test the approach, we used two still data sets collected on our in-house diffractometer using a single lysozyme crystal: one consisting of consecutive, stepwise stills and one consisting of stills from arbitrary orientations. Both were compared with conventional rotation data collected under the same conditions. We show that for these data sets the reflection partialities can be estimated by the ray-tracing simulation method and that the presented approach significantly improves the mean intensities of the observed reflections.

2. Diffraction theory

Reflection profiles from a crystal in EVAL are simulated by generating ray traces. We consider a crystal to be built up from small crystallites by dividing the crystal on a three-dimensional grid (sampled from a distribution K) that can have random orientations taken from a mosaic distribution (M). Incident X-rays are emitted from a virtual focus (e.g. a square area F) in direction k0 with respect to the crystallite and with wavelength λ (sampled from a spectral distribution L). A crystallite with an orientation of the reciprocal-lattice vector [{\bf d}_{hkl}^*] gives rise to a diffracted ray in direction k1 as determined by the Ewald construction. For F, L, K and M several statistical distributions are available (Schreurs et al., 2010[Schreurs, A. M. M., Xian, X. & Kroon-Batenburg, L. M. J. (2010). J. Appl. Cryst. 43, 70-82.]). In the simulation of rotation data the vectors [{\bf d}_{hkl}^*] are rotated around the spindle axis so as to match the Bragg condition and then touch the Ewald sphere (Fig. 1[link]).

[Figure 1]
Figure 1
Ewald construction for a rotation experiment. An incident X-ray beam k0 with length 1/λ is reflected by a series of net planes in the direction k1 if the corresponding reciprocal-lattice vector [{\bf d}_{hkl}^*] coincides with the Ewald sphere. In an arbitrary orientation of the crystal lattice [{\bf d}_{hkl}^*] is not in a reflecting position but can be rotated around the spindle axis so that the Bragg condition is met and [{\bf d}_{hkl}^*] makes an angle of 90° − θ with the X-ray beam k0.

In case of still diffraction experiments with one particular orientation of the crystal, none of the crystallite lattice vectors [{\bf d}_{hkl}^*] are exactly on the Ewald sphere. However, some vectors are within a certain tolerance in angular deviation () of the Bragg angle θ and may give rise to diffracted intensity that is a function of (see below). The integrated intensity of all vectors [{\bf d}_{hkl}^*] from the various crystallites depends on the mosaic spread, the wavelength dispersion, the beam size and divergence, the crystal size and the crystallite size itself. The latter corresponds to the coherently diffracting volume of the mosaic blocks, and the total reflected intensity of the crystal is the incoherent sum of all diffracted rays.

2.1. Still diffraction images

The scattered intensity of a crystal at Bragg angle θ can be thought of as the coherent sum of scattering by s layers of thickness d, according to what we call the James–Buerger theory (James, 1958[James, R. W. (1958). The Optical Principles of the Diffraction of X-rays. London: G. Bell & Sons.]; Buerger, 1960[Buerger, M. J. (1960). Crystal Structure Analysis. New York: John Wiley & Sons.]). The scattered intensity of a single layer is

[I = {I_0}{F^2}p\left({{e^2} \over {mc^2}}\right)^2\lambda^2\left({{nsd} \over {\sin\theta}} \right)^2, \eqno(1)]

where F2 is the squared structure factor, I0 is the incident photon flux, p is the polarization owing to the reflection, e2/mc2 is the Thomson scattering length of one electron and n is the number of unit cells per unit volume. A mosaic crystal is made up of tiny crystallites with associated reciprocal-lattice vectors [{\bf d}_{hkl}^*] that are spread over an angular range μ, and each of them may not be perfectly oriented to be in Bragg condition. The s layers within such a crystallite then scatter slightly out of phase and their scattered intensity is given by the interference function

[I(\varepsilon) = {I_0}{F^2}p\left({{e^2}\over{mc^2}} \right)^2n^2\lambda ^3{1\over{2\sin^2\theta}} V_{\rm crystallite}{1 \over s}{{\sin^2(sB\varepsilon)} \over {(B\varepsilon)^2}}, \eqno(2)]

where is the deviation of the Bragg angle θ, B = 2πdcosθ/λ and Vcrystallite is the reflecting volume of the crystallite.

The James–Buerger theory can be extended by writing the total diffracted intensity as an integral over all possible orientations of the crystallite vectors [{\bf d}_{hkl}^*] that make angles of 90° − η with the incident X-ray beam (90° − θ in the Bragg condition) and replacing Vcrystallite by the volume of the crystal V and by θη (see Fig. 2[link]). This results in

[\eqalignno {I &= {I_0}{F^2}p\left({{e^2}\over{mc^2}}\right)^2V_{\rm crystal}\,n^2\lambda ^3{1 \over {2\sin^2\theta}}\cr & \quad {\times}\ {1 \over s}{\textstyle\int\limits_{\eta_{\min}}^{\eta_{\max}}} {{\sin^2[sB(\theta - \eta)]} \over {[B(\theta - \eta)]^2}} P(\eta | {\eta_0})\,{\rm d}\eta, &(3)}]

where P(η|η0) is the probability distribution of η angles given the angle η0 of the central reciprocal-lattice vector [{\bf d}_{hkl}^*](0) of the crystal as obtained from the unit-cell matrix.

[Figure 2]
Figure 2
Angular spread of mosaic vectors. η0 denotes the angle of the central mosaic vector (i.e. that which makes an angle of 90° − η0 with the X-ray beam), which may be deviating from the ideal Bragg angle θ. The full mosaic spread is defined as six times the standard deviation of a Gaussian distribution.

The mosaic spread, the divergence of incident rays, the wavelength variations and the crystallite positions being slightly off-centre in a larger crystal are the cause of deviations for the individual rays. For the discussion here, we will concentrate on the mosaic spread, but the other parameters are accounted for as well in our ray-tracing simulations.

The distribution function can take several forms. Suppose that P(η|η0) is uniform, while [\int P(\eta|\eta_0){\rm d}\eta=1], then the integral over dη in (3)[link] reduces to sπ/B and

[I = C{1 \over {2\sin^2\theta}}\,{\pi \over B} = C{1 \over {2\sin \theta \cos \theta }}, \eqno (4)]

where C = I0F2p(e2/mc2)2λ3n2. The last term in (4)[link] is familiar: it is the Lorentz factor for rotation in the equatorial plane and is equal to the powder Lorentz factor. Thus, for still images the powder Lorentz factor applies. When P(η|η0) is a normal or an otherwise monotonous distribution, we should explicitly include it in the calculation of (3)[link]. However, it cannot be reduced to a simple trigonometric function nor to an erf (see Kabsch, 2014[Kabsch, W. (2014). Acta Cryst. D70, 2204-2216.]) because of the presence of the sinc function in the integral. Instead, it can be evaluated numerically.

2.2. Rotation diffraction images

Rotation images can be regarded as a superposition of many stills separated by an infinitesimal rotation angle ω. The integrated intensity for these is

[I = C{1\over{2\sin^2\theta}}\,{1\over s}{\textstyle \int\limits_{\omega _{\min}}^{\omega_{\max}}} {\textstyle\int\limits_{\eta_{\min}}^{\eta_{\max}}} {{\sin ^2[sB(\theta - \eta_\omega)]} \over {[B(\theta - \eta_\omega)]^2}} P(\eta_\omega| \eta _{0,\omega})\,{\rm d}\eta_\omega\,{\rm d}\omega. \eqno(5)]

In a sufficiently large ω scan each [{\bf d}_{hkl}^*] vector makes a complete pass through the Ewald sphere, so that we can write

[\eqalignno{I &= C{1\over {2\sin^2\theta}}\,{1 \over s} {\textstyle \int\limits_{\eta_{\min}}^{\eta_{\max}}} {\textstyle\int\limits_{\omega_{\min}}^{\omega_{\max}}} {{\sin^2[sB(\theta - \eta_\omega]} \over {[B(\theta - \eta_\omega)]^2}}\,{\rm d}\omega \,P(\eta|\eta_0)\, {\rm d}\eta \cr & = C{1\over {2\sin^2\theta}}\,\,{1 \over s}{\textstyle \int\limits_{\eta_{\min}}^{\eta_{\max}}} P(\eta|\eta_0){\textstyle \int\limits_{\eta'_{\min}}^{\eta'_{\max}}} {{\sin^2[sB(\theta - \eta')]} \over {[B(\theta - \eta ')]^2}} {{{\rm d}\omega } \over {{\rm d}\eta'}}\,{\rm d}\eta'\,{\rm d}\eta \cr & = C{1\over {2\sin^2\theta}}\,\,{1 \over s}{\textstyle \int\limits_{\eta_{\min}}^{\eta_{\max}}} P(\eta|\eta_0){\textstyle \int\limits_{\eta'_{\min}}^{\eta'_{\max}}} {{\sin^2[sB(\theta - \eta')]} \over {[B(\theta - \eta')]^2}} L'\,{\rm d}\eta'\,{\rm d}\eta \cr & = C{1\over {2\sin^2\theta}}\,\,{1 \over s}{\textstyle \int\limits_{\eta_{\min}}^{\eta_{\max}}} P(\eta|\eta_0)\,s\pi/BL'\,{\rm d}\eta = CL{\textstyle \int\limits_{\eta_{\min}}^{\eta_{\max}}} P(\eta|\eta_0)\, {\rm d}\eta = CL, \cr && (6)}]

where L′ is the duration component of the Lorentz factor L for the rotation experiment (equal to the reflection range expansion factor in Kabsch, 2014[Kabsch, W. (2014). Acta Cryst. D70, 2204-2216.]). The rotation Lorentz factor may alternatively be written as 1/[d*(k0 × ω)] (Milch & Minor, 1974[Milch, J. R. & Minor, T. C. (1974). J. Appl. Cryst. 7, 502-505.]). In case of rotation in the equator, (6)[link] reduces to (4)[link]. In rotation data, therefore, the specific distribution function P(η) is irrelevant to the integrated intensity and complete reflections are obtained.

2.3. Implementation of the interference function in EVAL

In EVAL a large number of vectors [{\bf d}_{hkl}^*](i) are generated from a Gaussian or Lorentzian two-dimensional mosaic distributions (M) and combined with vectors k0 from F, L, K distributions. The contribution of each of these to the scattered intensity is calculated with

[I_i = {1 \over {2\sin^2\theta}}\,{1 \over s}\,{{\sin^2(sB\varepsilon_i)} \over {(B\varepsilon_i)^2}}. \eqno(7)]

Summing all contributions gives the total scattered normalized intensity (i.e. C = 1.0; see text below equation 4[link]), which is effectively an integral over dη, dk0 and, to a minor extent, dλ, because our beam is almost monochromatic. This normalized intensity is stored in the parameter `partiality' after correction for the still Lorentz factor, i.e. the partiality is [\textstyle \sum_{i} I_{i}/L_{\rm still}]. The only new parameter introduced is the number of unit cells in the crystallite Ncell, where s = Ncell(|h| + |k| + |l|), the number of reflecting planes, while the crystallite size equals sdhkl.

Every [{\bf d}_{hkl}^*](i) produces its own impact on detector pixel coordinates (x, y) and is weighted by contribution Ii. All impacts together build the two-dimensional reflection profile that is used as the model profile in the EVAL least-squares fit to obtain the observed integrated intensity for each diffraction spot on an image. Both the observed intensity and the summed interference function (7)[link] contain the still Lorentz factor, and by dividing one by the other we extract F2. We also correct for the polarization and apply possible incidence corrections.

2.4. Laue interference function

In this paper, we follow the James and Buerger approach, as explained in §[link]2.1, for deriving diffracted intensities by crystals. The resulting interference function only depends on the deviation from the Bragg angle θ and the number of unit cells contained in the crystal. An alternative is to use the three Laue conditions, and the squared sinc function in (3)[link] is replaced by

[I(\Delta {\bf k}) = {\left ({{\sin N_1\pi\Delta {\bf k}\cdot{\bf a}} \over {\sin \pi \Delta {\bf k}\cdot{\bf a}}} {{\sin N_2\pi \Delta {\bf k}\cdot {\bf b}} \over {\sin \pi \Delta {\bf k}\cdot{\bf b}}} {{\sin {N_3}\pi \Delta {\bf k}\cdot{\bf c}} \over {\sin \pi \Delta {\bf k}\cdot {\bf c}}} \right)^2}. \eqno(8)]

Here,Δk = k1k0 and N1, N2 and N3 are the number of unit cells in the three periodic axis directions. In Appendix A[link], we show that the two approaches are exactly the same.

(8) is often referred to as the shape transform of the crystal (Kirian et al., 2010[Kirian, R. A., Wang, X., Weierstall, U., Schmidt, K. E., Spence, J. C. H., Hunter, M., Fromme, P., White, T., Chapman, H. N. & Holton, J. (2010). Opt. Express, 18, 5713-5723.]; Spence et al., 2011[Spence, J. C. H., Kirian, R. A., Wang, X., Weierstall, U., Schmidt, K. E., White, T., Barty, A., Chapman, H. N., Marchesini, S. & Holton, J. (2011). Opt. Express, 19, 2866-2873.]).

2.5. Impact positions and refinement

Peak-position refinement in PEAKREF (Schreurs, 1999[Schreurs, A. M. M. (1999). PEAKREF. Utrecht University, The Netherlands.]) minimizes the peak-position residuals and the Bragg angle deviation 0 of the central reciprocal-lattice vector either using peak maxima from the peak search or using optimized profile centroids from the EVAL profile fit. Inclusion of 0 in the unit-cell matrix refinement avoids divergence of unit-cell orientations through rotations perpendicular to the incident beam, as discussed by Sauter et al. (2014[Sauter, N. K., Hattne, J., Brewster, A. S., Echols, N., Zwart, P. H. & Adams, P. D. (2014). Acta Cryst. D70, 3299-3309.]). Similarly, Kabsch (2014[Kabsch, W. (2014). Acta Cryst. D70, 2204-2216.]) uses the angular deviation τ divided by the mosaic spread σM in the target function for peak refinement. All three approaches use the β axis, defined for each reflection as the axis perpendicular to the incident and diffracted beams, to calculate the deviation from the Bragg angle θ (Schutt & Winkler, 1977[Schutt, N. K. & Winkler, F. K. (1977). The Rotation Method in Crystallography, edited by U. W. Arndt & A. J. Wonacott, pp. 173-186. Amsterdam: North Holland.]). For each still image, the following target function was minimized to refine the unit-cell matrix,

[\Psi = {{\textstyle \sum\limits_i^{N_{\rm peak}}[(\Delta x_i)^2 + (\Delta y_i)^2]^{1/2}} \over {N_{\rm peak}}} + {{\textstyle\sum\limits_i^{N_{\rm peak}} | \theta_i - \eta_{0,i}|} \over {N_{\rm peak}}}, \eqno(9)]

where Δxi and Δyi are the differences of observed and calculated peak positions. PEAKREF can optimize many instrumental parameters such as detector-offset positions, primary beam direction and crystal position, which in the current analysis were fixed in the still data refinement and based on the rotation data (see below).

We found that the peak-position residuals from the post-refinement were much smaller than from the peak maxima found on a single still image, despite the much larger number of peaks. This was caused by a shift in the observed θ value for the partial reflections with large 0. For large mosaic crystals such as our lysozyme crystal measured with a divergent beam, these shifts in θ occur because only distinct directions of the primary rays or distinct points on the crystal are active dependent on the deviation 0 (Fig. 3[link]). A negative value for 0 results in an apparent larger θ and a positive 0 in an apparent smaller θ. This θ-divergence effect has to be taken into account when the cell matrix is determined and refined from peak positions. We introduced a parameter `flex' in PEAKREF that is jointly refined and takes account of this shift. The `flex' parameter turned out to have a constant value for all still images and it appears to be a property typical for the crystal and the beam divergence of the particular experiment.

[Figure 3]
Figure 3
Schematic illustration of conditions causing an apparent shift in 2θ. A mosaic vector [{\bf d}_{hkl}^*](0) is slightly off the Bragg condition for an X-ray beam from the centre of the focus F (blue line) and makes an angle of 90° − η with the X-ray beam. However, the Bragg condition tends to be fulfilled with rays emerging from the lower part of the focus (red line; the arc corresponds to an angle of 90° − θ) or for the upper part of the crystal (green line). Therefore, the reflected beam appears at a larger 2θ angle. Owing to averaging of many rays this effect is not, or is hardly, visible for rotation images, but it is for still images or ultrafine-sliced images. As the shift in apparent 2θ is always in the direction of 2η, we can account for this by the `flex' parameter in PEAKREF.

2.6. Post-refinement

We implemented a post-refinement procedure in which both the peak positions from the EVAL integration and the partialities could be refined. For this purpose, we calculate the mean intensity of all equivalent reflections h as

[\langle I^h \rangle = {{\textstyle \sum\limits_e^{N^h} w_e s_f/p_e I_e^h} \over {\textstyle \sum\limits_e^{N^h} w_e s_f/p_e}}. \eqno(10)]

Weights are obtained from the standard deviations from the EVAL profile fit (Schreurs et al., 2010[Schreurs, A. M. M., Xian, X. & Kroon-Batenburg, L. M. J. (2010). J. Appl. Cryst. 43, 70-82.]) and are given by we = 1/σe2. The partialities pe arise from the EVAL ray-tracing simulation, and the image scale factors sf are determined in ANY (Schreurs, 2007[Schreurs, A. M. M. (2007). ANY. Utrecht University, The Netherlands.]), assuming a constant sum of Bragg intensities in each frame. The summation in (10)[link] runs over all equivalents of reflection h (Nh) in the data set. In PEAKREF image scale factors sf′ and unit-cell parameters and crystal orientation angles are refined using the target function

[\Psi = {{\textstyle \sum\limits_i^{N_{\rm peak}} [(\Delta x_i)^2 + (\Delta y_i)^2]^{1/2}} \over {N_{\rm peak}}} + {{\textstyle \sum\limits_i^{N_{\rm peak}} w_i| s'_f I_i - p'_i \langle I^h\rangle|} \over {\textstyle \sum\limits_i^{N_{\rm peak}} w_is'_fI_i}}. \eqno(11)]

We specifically include peak positions in this refinement step in order to avoid unwanted divergence from peak position-derived unit-cell parameters and orientations. In this post-refinement step the EVAL ray-tracing procedure is not repeated to obtain partialities; instead, we use a fitted partiality versus curve with a single Gaussian. The parameters in the Gaussian were kept fixed in the refinement; changes with the unit-cell parameters from which we recalculate the partiality (pi).

3. Materials and methods

3.1. Crystal preparation

Hen egg-white lysozyme (Sigma–Aldrich, Schnelldorf, Germany) was crystallized using the hanging-drop vapour-diffusion method with a protein concentration of 75 mg ml−1 in 0.1 M sodium acetate buffer pH 4.8. The precipitant consisted of 0.1 M sodium acetate buffer pH 4.8, 10–15%(w/v) sodium chloride, 30%(v/v) ethylene glycol (Sutton et al., 2013[Sutton, K. A., Black, P. J., Mercer, K. R., Garman, E. F., Owen, R. L., Snell, E. H. & Bernhard, W. A. (2013). Acta Cryst. D69, 2381-2394.]). Drops of 4 µl were set up with a 1:1 protein:precipitant ratio.

3.2. Data collection

A crystal of dimensions 250 × 250 × 150 µm was vitrified in a cold N2-gas stream from an Oxford Instruments 700 series jet operated at 100 K. Data were collected on a Bruker–AXS X8 Proteum in-house source with Cu Kα radiation. The rotating anode was operated at 45 kV and 60 mA. The reference rotation data set was collected by rotating over 190° in φ in 0.5° steps per frame. Data were recorded on a PLATINUM135 CCD detector with a sample-to-crystal distance of 52 mm. 380 still images were collected with identical angular settings as the starting angles for each of the rotation frames; thus, 380 still images were collected at 0.5° intervals. An additional 394 stills were recorded by random selections from ω scans 0–7° in ω apart at 15 different ω, κ and φ goniometer settings. The exposure time for all images was 5 s.

3.3. Data processing and analysis

VIEW was used for image display and peak search (Schreurs, 1998[Schreurs, A. M. M. (1998). VIEW. Utrecht University, The Netherlands.]). Both the rotation images and stills were indexed using DirAx (Duisenberg, 1992[Duisenberg, A. J. M. (1992). J. Appl. Cryst. 25, 92-96.]). Almost all stills could be indexed without manual intervention. Bravais lattice constraints were applied and the unit cells were made congruent (using the goniometer positions), ensuring a consistent choice of unit-cell axes. The unit-cell matrix and detector positions were refined from 649 peak positions in the rotation data. For the still peak positions we used different refinement options. In the first approach we made use of our knowledge of the relative positions of the goniometer axes, so that a global single unit-cell matrix could be refined against 10 728 peak positions. In the second approach, we determined and refined a unit-cell matrix for each image from 300 peak positions, as would be the normal procedure in serial crystallography. The detector-offset positions were taken from the peak-position refinement of the rotation data. The unit-cell matrix was refined against the observed peak positions, using the `flex' parameter to account for apparent shifts in θ, simultaneously minimizing the off-Bragg angle 0 (9)[link]. Using the unit-cell matrix, we extracted three-dimensional and two-dimensional reflection boxes for rotation and still images, respectively, and processed these with EVAL. For every reflection, 10 000 rays were simulated and the impacts were collected in pixels contained in the box. In case of still data every individual ray is associated with a reciprocal-lattice vector [{\bf d}_{hkl}^*](i) with a small angular deviation from the Ewald sphere i and is weighted by the interference function (7)[link]. The impact position on the detector is given by the direction of the shortest distance of the reciprocal-lattice vector to the Ewald sphere. The divergence effects are accounted for in the ray tracing and thus the profiles are generated correctly at deviating positions in θ (i.e. without the need for a `flex' parameter as used at the peak-refinement stage).

The parameters for crystal size, mosaic spread and beam divergence were optimized automatically in the reflection profile fitted to ∼50 reflections with I/σ(I) > 20 using a simplex method (see Schreurs et al., 2010[Schreurs, A. M. M., Xian, X. & Kroon-Batenburg, L. M. J. (2010). J. Appl. Cryst. 43, 70-82.]). For comparison reasons, identical values of parameters in the ray-tracing simulations were used for both types of data sets, although a similar optimization can be performed for still images. In addition, for the still images we used Ncell = 25 in the interference function (the number of unit cells in a crystallite as described in §[link]2.3). Sampling of the interference function converges much faster with low values of Ncell, typical for nano-sized crystals. The current data imply a larger value of Ncell, which in the current implementation would require many more rays (up to 106 instead of 10 000) to sample reflection profiles smoothly. The integrated intensities are obtained by a least-squares fit of the three-dimensional and two-dimensional model profiles to the observed pixel intensities for the rotation and still reflections, respectively. EVAL then delivers the profile-fitted, Lorentz- and polarization-corrected intensity values in an XML-type datafile that is further processed in ANY (Schreurs, 2007[Schreurs, A. M. M. (2007). ANY. Utrecht University, The Netherlands.]). In this program, we determine image scale factors, correct for the partiality factor and output the intensities and standard deviations to an hkl- or mtz-type file. Many of the graphical plots and statistical analyses are made using ANY.

All still images were also processed with the CrystFEL software suite v.0.5.1 (White et al., 2012[White, T. A., Kirian, R. A., Martin, A. V., Aquila, A., Nass, K., Barty, A. & Chapman, H. N. (2012). J. Appl. Cryst. 45, 335-341.]). Structural refinements were carried out with REFMAC5 (Murshudov et al., 2011[Murshudov, G. N., Skubák, P., Lebedev, A. A., Pannu, N. S., Steiner, R. A., Nicholls, R. A., Winn, M. D., Long, F. & Vagin, A. A. (2011). Acta Cryst. D67, 355-367.]) and scaling between data sets with SCALEIT (Howell & Smith, 1992[Howell, P. L. & Smith, G. D. (1992). J. Appl. Cryst. 25, 81-86.]), both from the CCP4 suite (Winn et al., 2011[Winn, M. D. et al. (2011). Acta Cryst. D67, 235-242.]). ANODE (Thorn & Sheldrick, 2011[Thorn, A. & Sheldrick, G. M. (2011). J. Appl. Cryst. 44, 1285-1287.]) was used to calculate anomalous difference densities.

4. Results

We collected rotation and still diffraction data from one lysozyme crystal and formed three data sets for analysis: a 190° rotation data set collected in 380 images in ranges of 0.5° for reference, a consecutive still data set of 380 images collected in steps of 0.5°, and this consecutive data set combined with 15 wedges of separate arbitrary orientations totalling 774 images (Table 1[link]). Indexing and peak refinement by PEAKREF/EVAL and CrystFEL yielded unit-cell dimensions that varied by ∼0.3–0.7% between the separate stills. Initially, the average residuals in peak positions for the still data were significantly larger than for the rotation data. Introduction of the `flex' parameter, which takes into account the apparent shift in θ owing to divergence effects (see §[link]2.5), reduced the positional residuals of peak maxima on still images significantly: from 0.13 to 0.07–0.08 mm on average. This deviation is only slightly larger than that observed for the rotation data, which was 0.06 mm. The residual in rotation angle for the rotation data was 0.039° (for 0.5° scan width). For the still data the deviation from the Bragg angle 0 was 0.18°, which is consistent with a mosaic spread of 0.5°. Relaxing the unit cells by a separate matrix for each image lowered the 0 residuals to 0.14° (compare `single unit cell' versus `unit cell per image' in Table 1[link]). In our setup the orientation of each matrix was known because the crystal orientation was set using a gonio­meter. The r.m.s. deviations between the set and refined orientations of the reciprocal-lattice vectors a*, b* and c* were 0.03° for the consecutive still data and increased to ∼0.10° using all still data (consecutive and random orientations). Overall, the number of observations taken into account by EVAL were 106 × 103 for the rotation data set, 325 × 103 for the consecutive still data and 657 × 103 for all still data, whereas CrystFEL took 733 × 103 into account for all still data (Table 2[link]). All processed sets resulted in ∼8300 unique reflections. The multiplicity of the consecutive still data was roughly three times that of the rotation data, indicating that reflections were, on average, sliced through three times in our still data-collection experiment.

Table 1
Indexing and peak-refinement statistics

      EVAL, consecutive stills EVAL, all stills
  EVAL, rotation CrystFEL, all stills Single unit cell Unit cell per image   Post-refined
No. of images 380 774 380 380 774  
No. of peaks used for indexing 649 300–490 10728 259–300 289–300  
No. of peaks used in refinement 101246   217272 113418 230759 396201
Unit-cell parameters
a, b (Å) 78.666 78.735–79.453 78.834 78.765–79.041 78.679–79.119 78.719–79.049
c (Å) 36.819 36.819–37.481 36.842 36.706–36.906 36.706–36.989 36.810–36.980
Orientation r.m.s. (°)
a*       0.028 0.102 0.103
b*       0.038 0.108 0.095
c*       0.035 0.107 0.111
Average residuals
 Horizontal, vertical (mm) 0.0622   0.0884 0.0785 0.0766 0.099
0 (°) 0.0390   0.1850 0.1474 0.1496 0.173
†Per image.
‡Average angular deviation of central reciprocal-lattice vector d* with Ewald sphere (see text for explanation).

Table 2
Statistics for the integration and merging of the rotation and still data

      EVAL, Monte Carlo EVAL, Lorentz corrected EVAL, partiality corrected
  EVAL, rotation CrystFEL, all stills Consecutive stills All stills Consecutive stills All stills Consecutive stills All stills
Unit cell     Image Image Image Image Single Image Image
No. of reflections 106021 733504         320139 325753 657782
No. unique 8291           8301 8328 8352
Multiplicity 12.8 88.4         38.6 39.1 78.8
Completeness (%) 100 100         100 99.0 99.2
CC1/2 (%) 100 87.4 (80.3) 94.1 (88.6) 88.6 (79.9) 94.3 (88.5) 97.3 (90.9) 95.3 (88.1) 97.5 (90.9)
I/σ(I)〉 13.9 (1.7)  
I/σ(I)〉, merged 47.7 (20.3) 10.1 (5.7) 8.0 (5.1) 11.5 (7.4) 8.1 (5.4) 11.8 (7.7) 12.0 (7.4) 11.0 (6.6) 15.1 (9.0)
Rint§ (%) 3.8 (7.1) 106.2 (94.5) 104.9 (91.2) 108.0 (97.1) 106.5 (94.4) 57.5 (58.1) 62.6 (62.6) 63.8 (63.4)
Post-refinement
 〈I/σ(I)〉, merged             13.2 (7.9) 13.8 (8.2) 18.4 (10.8)
Rint§ (%)             55.7 (56.1) 54.4 (54.5) 55.7 (55.1)
†Point group 4/mmm.
‡Still data are not scaled by SADABS like the rotation data and no error model is determined for σ. In the merging step σ is determined from the internal standard deviation [\textstyle \sum |I - \langle I \rangle |/ N(N - 1)^{1/2}].
§Rint = [\textstyle \sum_h^N \sum_e^{N^h} (|I_e - \langle I^h\rangle|)/][\textstyle \sum_h^N \sum_e^{N^h} I_e}], where the summations runs over all N unique reflections h and equivalents.

The statistics for the integration and merging of data for the rotation and still data are shown in Table 2[link]. Processing of the reference rotation data yielded an internal merging Rint of 3.8% with an 〈I/σ(I)〉 after merging of 47.7. Processing of the still diffraction data without correction, referred to as Monte Carlo averaging, produced Rint values exceeding 100% and 〈I/σ(I)〉 values that were about fourfold lower than that for the rotation data using the same number of images. Application of the still Lorentz correction (4)[link] slightly increased the Rint (Table 2[link]).

To estimate partialities, we determined the parameters for mosaic spread, divergence of the incident beam, crystal size and Ncell by optimizing two-dimensional profile fits using figures of merit (Schreurs et al., 2010[Schreurs, A. M. M., Xian, X. & Kroon-Batenburg, L. M. J. (2010). J. Appl. Cryst. 43, 70-82.]) on a subset of reflections in EVAL. Mosaic spread was set to 0.5°, beam divergence to 8.6 mrad, crystal size to 130 × 130 × 130 µm (although we estimated a slightly larger size when selecting the crystal under the microscope) and Ncell to 25. The ray-tracing procedure yielded partialities which showed a Gaussian-like distribution with 0 (Fig. 4[link]a). Notably, the computed still partialities are not normalized and exceed a value of 1, and hence are used as relative scale factors. In rotation data the partiality is defined up to 1 for a fully observed reflection (Rossmann & Beek, 1999[Rossmann, M. G. & van Beek, C. G. (1999). Acta Cryst. D55, 1631-1640.]); in contrast, the partiality in still diffraction is determined by the angular width of the intersection with the Ewald sphere, which depends on various instrumental and crystal parameters such as those given by (3)[link]. Lorentz-corrected still and (Lorentz-corrected) rotation reflections on average give the same absolute intensities. Fig. 5[link] shows that some still partialities are larger than 1.0 and the still intensities scatter around the rotation intensity. Further, to illustrate that the partialities depend strongly on the precise ray-tracing model parameters, Fig. 4[link](b) shows the partialities as a function of 0 in the case of a long focus for the incident beam, which results in two Gaussian-like curves superimposed. This implies that a simple Gaussian model for the partiality is not always correct. When divided into 0 bins, the observed average intensities correlate well with the estimated partialities (Fig. 6[link]). Application of the partiality model resulted in average I/〈I〉 values that varied around the ideal value of 1.0. Subsequent merging of these data, i.e. with both Lorentz and partiality corrections applied, reduced the Rint values to 57 and 63% for the data sets with consecutive and all stills, respectively.

[Figure 4]
Figure 4
Partialities of reflections plotted against their 0. (a) From the ray-tracing model used in this paper, (b) from ray tracing using a long focus.
[Figure 5]
Figure 5
Lorentz-corrected still (not partiality-corrected) versus Lorentz-corrected rotation intensities. All occurrences of reflection 1, −6, 5 are marked by yellow dots. No scaling was applied so that intensities are on an absolute scale. Dots above the red line with slope = 1.0 indicate that the observed still intensity is larger than its corresponding rotation intensity, resulting in a partiality larger than 1.0. On average, the equivalent observations of a unique still reflection are equal to the rotation intensity (see text).
[Figure 6]
Figure 6
Histogram of reflection data versus 0. (a) Partialities, (b) I/〈I〉 for equivalent reflections and (c) I/〈I〉 after partiality correction.

Next, the effects of Lorentz and partiality correction were evaluated by comparing the data with the reference rotation data set. The uncorrected and the Lorentz-corrected intensities have high internal Rint values of 104.9 and 106.5%, respectively, consistent with the scattering in Fig. 5[link]. The Lorentz- and partiality-corrected intensities have an Rint of 63.8%. Upon merging the data to unique reflections the agreement with the rotation data improved dramatically; the scatter diagrams in Fig. 7[link](a) and 7[link](b) reflect the improvement corresponding to the uncorrected (Monte Carlo) and corrected (Lorentz and partiality) data. The effects from the still data corrections are more clearly demonstrated by the R factors with respect to the reference rotation data, which we refer to as Rcomp (Table 3[link]). Rcomp (on intensities) was 26% using Monte Carlo averaging. Application of the Lorentz correction alone decreased the Rcomp to 12%. Application of both Lorentz and partiality corrections yielded an Rcomp of 5.3%.

Table 3
Comparison of stills with EVAL rotation data

  CrystFEL, Monte Carlo EVAL, Monte Carlo EVAL, Lorentz correction only EVAL, partiality corrected EVAL, partiality corrected, post-refined
  All stills Consecutive stills All stills Consecutive stills All stills Consecutive stills All stills All stills
Rcomp (%) 32.0 26.7 26.4 12.9 12.0 6.5 5.3 4.7
R on F (%) 18.9 14.5 15.2 10.0 9.0 4.9 4.1 3.1
†Data merged in point group 4/mmm.
‡From SCALEIT: overall scale.
[Figure 7]
Figure 7
Scatter plots of still versus rotation reflection intensities. (a) Uncorrected Monte Carlo and (b) corrected reflection intensities after merging in point group 4/mmm.

Although the Lorentz and partiality corrections significantly improved the quality of the merged data, the merging Rint value remained high (i.e. 63.8% for all still data). To improve the partialities, we performed post-refinement of the image scale factor, unit-cell parameters and orientations, minimizing the target function of (11)[link]. Post-refinement of the `all stills' data gave scale factors of 0.84–1.35 (additional to the scale factor sf used in equation 10[link]) and sharpened the distribution of unit-cell dimensions, with virtually no effect on the variation of crystal orientations (Table 1[link]). These adjustments resulted in a significant, but modest, reduction of Rint from 63.8 to 55.7% (Table 2[link]). The progress in the precision of processing the data is reflected by the distributions I(hkl)/〈I(hkl)〉 shown in Fig. 8[link]. Ideally, I(hkl)/〈I(hkl)〉 values form a sharp distribution around 1 (as a reference, we depict the distribution resulting from the rotation data in Fig. 8[link]e). Figs. 8[link](b) and 8(c) reflect the striking improvement obtained by modelling the partiality in EVAL and subsequent post-refinement. Fig. 8[link](d) shows that mainly the weak data do not profit from the post-refinement. Comparison of the merged data sets shows that the improvement in precision is matched by an improvement in accuracy. Post-refinement reduced the Rcomp from 5.3 to 4.7% (Table 3[link]).

[Figure 8]
Figure 8
Histogram of I(hkl)/〈I(hkl)〉. (a) Uncorrected, (b) partiality-corrected, (c) partiality and post-refined still data, (d) the same as (c) but omitting weak reflections and (e) rotation data

To illustrate the data quality, we refined the lysozyme crystal structure and computed anomalous difference densities. The structure was refined starting from PDB entry 193l (Vaney et al., 1996[Vaney, M. C., Maignan, S., Riès-Kautt, M. & Ducruix, A. (1996). Acta Cryst. D52, 505-517.]) against the reflection data using REFMAC, and we observed similar Rwork and Rfree values for the differently processed data (Table 4[link]). Significant differences between the methods were observed for the resulting average isotropic B factors. Monte Carlo averaging of the data in CrystFEL and EVAL yielded increased B factors (21–25 Å2) compared with the reference defined by the rotation data set (〈B〉 = 13.8 Å2). The Lorentz correction had a large effect on the B factors and produced an average B factor of 11.8 Å2; this large effect on the B factors is explained by a comparable fall-off in θ of the Lorentz factor and the temperature factor. When the Lorentz and partiality corrections were both applied, the B factors became more similar to those obtained when using the rotation data (13.2 versus 13.8 Å2). Anomalous differences are much more sensitive to the accuracy of the data than structure refinement. We generated anomalous difference densities based on the processed data sets using phases from the refined structure by ANODE. For the methionine sulfur positions the anomalous density from the rotation data gave a peak height of 13.3σ. The uncorrected, Monte Carlo averaged still data yielded a weak anomalous signal: a 4.2σ peak for methionine S, corresponding to 32% of the peak height using the rotation data. Lorentz correction improved the methionine S signal to 35%, whereas including partiality corrections resulted in 47% of the signal. Finally, this signal improved to 54% after post-refinement. This shows that both Lorentz and partiality correction improved the intensities deduced from the still data.

Table 4
Comparison of data quality

  EVAL CrystFEL, Monte Carlo EVAL, Monte Carlo EVAL, Lorentz correction only EVAL, partiality corrected EVAL, partiality corrected, post-refined
  Rotation All stills Consecutive stills All stills Consecutive stills All stills Consecutive stills All stills All stils
Refinement
Rwork (%) 15.9 17.4 16.8 16.7 16.7 16.6 15.9 16.0 16.0
Rfree (%) 19.8 21.4 20.3 20.2 20.2 20.2 20.0 20.0 20.2
 〈B〉 (Å2) 13.8 25.0 20.7 20.6 11.8 11.8 13.0 13.2 13.4
Average anomalous densities (σ)
 No. of reflections§ 6420 6614 6293 6595 6266 6571 6091 6403 6406
 Met SD 13.3 3.2 3.2 4.2 3.6 4.6 5.2 6.3 7.2
 Cys SG 10.9 2.1 2.9 3.3 2.8 3.3 3.8 5.2 6.0
 Cl 5.0 1.0 1.0 1.0 1.1 1.2 1.7 2.1 2.3
 Na+ 1.8 1.0 0.8 2.0 0.8 2.0 1.5 1.5
†Restraint refinement, individual isotropic B factors, input structure PDB entry 193l .
ANODE with data merged in point group 422. Averaged densities over similar atom types (two for Met SD, eight for Cys SG, 14 for Cl and one for Na+).
§Selected by SHELXC.

We tested the effect of data-set size by limiting the still data to 60 images (Table 5[link]). For the reduced `consecutive still' data we used images 250–310. For the `random still' data 60 images from three different wedges were used. For these limited data sets (91.7 and 97.3% completeness, respectively), the Rfree factors show that the structure quality deteriorated. Furthermore, the anomalous signal is largely lost. For both structure refinement and anomalous density analyses the Lorentz and partiality-corrected data outperform the noncorrected Monte Carlo processed data.

Table 5
Comparison of data with reduced statistics (60 frames)

  EVAL, Monte Carlo EVAL, partiality corrected
  Consecutive stills Random stills Consecutive stills Random stills
Refinement
Rwork (%) 20.7 25.6 17.3 20.7
Rfree (%) 25.1 32.3 21.8 26.3
 〈B〉 (Å2) 20.9 21.6 14.3 13.8
Average anomalous densities (σ)
 No. of reflections§ 862 1265 2212 2166
 Met SD 0.6 0.8 1.6
 Cys SG 0.8 1.1
 Cl
 Na+
†Restraint refinement, individual isotropic B factors, input structure PDB entry 193l .
ANODE with data merged in point group 422. Averaged densities over similar atom types (two for Met SD, eight for Cys SG, 14 for Cl and one for Na+).
§Selected by SHELXC.

5. Discussion and conclusions

We used our ray-tracing profile-prediction methods to model partialities of the observed reflections in still diffraction data and adapted the programs PEAKREF and EVAL to process still diffraction images. By taking experimental conditions into account, we compute 10 000 rays generated from focus, crystal grid points, wavelength spectrum and mosaic distributions, and calculate the interference-function weighted contribution to an observed reflection and hence derive its partiality. Our formalism implicitly models for the Lorentz factor, mimicking the contribution of the Lorentz factor to the observed intensities. Our approach differs fundamentally from other still data-processing methods. Kabsch (2014[Kabsch, W. (2014). Acta Cryst. D70, 2204-2216.]) defined an analytical erf function for the partiality, which is the integral over a Gaussian mosaic function. It is equivalent to our integral in (3)[link] for an infinitely sharp sinc function (implying that integration over this function is complete within a solid angle smaller than the pixel size of the detector), while ignoring broadening effects other than the mosaic spread. Kabsch explicitly corrects for the still Lorentz factor. White (2014[White, T. A. (2014). Philos. Trans. R. Soc. B Biol. Sci. 369, 20130330.]) and Sauter (2015[Sauter, N. K. (2015). J. Synchrotron Rad. 22, 239-248.]) use reciprocal-lattice point volumes for calculating the partiality. White (2014[White, T. A. (2014). Philos. Trans. R. Soc. B Biol. Sci. 369, 20130330.]) accounts for spectral width and beam divergence by calculating the overlap of a reciprocal-lattice volume with a nest of Ewald spheres. Sauter (2015[Sauter, N. K. (2015). J. Synchrotron Rad. 22, 239-248.]) and Uervirojnangkoorn et al. (2015[Uervirojnangkoorn, M., Zeldin, O. B., Lyubimov, A. Y., Hattne, J., Brewster, A. S., Sauter, N. K., Brunger, A. T. & Weis, W. I. (2015). eLife, 4, e05421.]) use a single Ewald sphere and calculate the intersection with a spherical reciprocal-lattice volume, the size of which is determined by beam divergence, mosaic spread and spectral dispersion. Both approaches account for increase of reciprocal diffracting volume with resolution, and in this way for the wider range of acceptable off-Bragg angles (d; see Appendix A[link]). However, both approaches lack the reflectivity part of the Lorentz factor (dΩ; see Appendix A[link]). If the spectral width of the beam becomes large, an additional Lorentz factor needs to be accounted for, as used in the Laue method (Zachariasen, 1945[Zachariasen, W. H. (1945). Theory of X-ray Diffraction in Crystals. New York: Dover.]). Uerviroj­nangkoorn et al. (2015[Uervirojnangkoorn, M., Zeldin, O. B., Lyubimov, A. Y., Hattne, J., Brewster, A. S., Sauter, N. K., Brunger, A. T. & Weis, W. I. (2015). eLife, 4, e05421.]) very recently presented their results on XFEL data. They showed that the Rwork and Rfree of refined structures improved and part of the anomalous signal was retrieved. Unfortunately, they do not provide merging Rint or a comparison to a rotation data set, i.e. Rcomp, to evaluate the resulting quality of the data more directly. In our approach, the integration of (3)[link] is achieved by simulation of the rays that contribute to an observed reflection spot. Because of the simulation, the derivation of analytical functions for the various effects is not needed and the Lorentz effect is implicitly taken into account. Moreover, the interference function can be taken into account in our approach.

For the initial development of the method, we used an experimental setup that allowed a direct comparison to the conventional rotation method. Our analysis showed a dramatic improvement in data quality after partiality and Lorentz correction. Both data processing and structure refinement showed that Lorentz correction is important and that omission of the Lorentz correction strongly affects the temperature factor. The anomalous sulfur densities increased 1.7-fold upon Lorentz and partiality correction of the still data. Overall, our approach markedly improved the Rcomp factor between the intensities derived from rotation and still data from 26% to a final value of 4.7% after Lorentz and partiality correction and post-refinement.

Concurrent with the improvement in the final data quality upon Lorentz and partiality correction in EVAL, the internal merging Rint decreased from 105 to 64%. When we were developing the method, we hoped that post-refinement of the parameters would improve the final unique intensity data as well as further reduce the internal merging Rint factor. Post-refinement improved the precision of the modelled unit-cell dimensions and scale factor per image, although the error in modelled crystal orientations remained ∼0.1°. These more precise parameters indeed improved the resulting intensities (Rcomp decreased from 5.3 to 4.7%). The internal statistics improved as well (Rint decreased from 64 to 56%); however, the final Rint factor remained high. This high Rint could be owing to features that were not included in our ray-tracing model, such as possible asymmetry in the focus, (anisotropic) mosaic spread or crystal form, or absorption by the crystal. Notably, crystal absorption may have a significant effect on the presented data because a relatively large crystal was used in this experiment. Crystal absorption is likely to be negligible when data are collected from microcrystals or nanocrystals, as is the case in serial crystallography. Obviously, further development of our approach is needed to account for the experimental conditions of serial (femtosecond) crystallography using XFEL or synchrotron sources. Automated schemes will be needed to model, for example, the large number of single-crystal diffraction images and fluctuations in beam spectra. In general, comprehensive modelling of the relevant experimental conditions should improve both the internal merging statistics and the resulting intensities. Not modelling significant effects that are present in the data can only be overcome by collecting more data to allow the averaging out of these effects by the Monte Carlo approach. In a real-case scenario the rotation data will not be available to evaluate the data quality, and an Rint of ∼50% may possibly be a practical metric to judge the resulting data quality.

Overall, we have shown that ray tracing can produce reliable partialities that improve the resulting data quality originating from still diffraction images. Moreover, our method is versatile and allows the modelling of a wide variety of effects, including those that yield non-Gaussian, asymmetric effects on the diffraction spot. In particular, the approach can take the interference function into account, which will be critical for processing data obtained from nanocrystals. Thus, in this paper we have presented the theoretical framework and demonstrated the potential of the ray-tracing methodology for processing still diffraction data.

The rotation and still diffraction images are available at http://rawdata.chem.uu.nl/c003 .

APPENDIX A

Comparison with Laue interference function

The diffracted intensity reflected by a small crystal bathed in an incident monochromatic beam is proportional to the shape transform of the crystal. The reflected intensity received by the detector in a small cone of solid angle dΩ, while the reciprocal-lattice vector [{\bf d}_{hkl}^*] has a small deviation of the Bragg angle θ, is given by the Laue interference function (Laue, 1936[Laue, M. von (1936). Ann. Phys. 41, 971-988.]; James, 1958[James, R. W. (1958). The Optical Principles of the Diffraction of X-rays. London: G. Bell & Sons.]) and is used in papers by Kirian et al. (2010[Kirian, R. A., Wang, X., Weierstall, U., Schmidt, K. E., Spence, J. C. H., Hunter, M., Fromme, P., White, T., Chapman, H. N. & Holton, J. (2010). Opt. Express, 18, 5713-5723.]) and White et al. (2012[White, T. A., Kirian, R. A., Martin, A. V., Aquila, A., Nass, K., Barty, A. & Chapman, H. N. (2012). J. Appl. Cryst. 45, 335-341.]),

[I_0|F|^2p\left({{e^2}\over {mc^2}}\right)^2 {{\sin^2\pi \xi N_1} \over {\sin^2\pi\xi}} {{\sin^2\pi \eta N_2} \over {\sin ^2\pi \eta }} {{\sin^2\pi \zeta N_3}\over {\sin^2\pi \zeta }}\, {\rm d}\Omega\, {\rm d}\varepsilon. \eqno (12)]

N1, N2 and N3 are the number of unit cells in the three dimensions of the parallepiped crystal. ξ is the scalar product Δk · a and in near-Bragg condition it is equal to h + Δh, and likewise for the other directions. As we are only interested in the diffracted intensity close to the Bragg condition, we introduce a local reciprocal axis system and replace ξ by the nonperiodic Δh. The terms in the denominator of (12)[link] can then be written as (πΔh)2 because they concern only small numbers,

[I_0|F|^2p\left({{e^2}\over {mc^2}}\right)^2 {{\sin^2\pi \Delta hN_1} \over {(\pi \Delta h)^2}} {{\sin^2\pi \Delta k{N_2}} \over {(\pi \Delta k)^2}} {{\sin^2\pi \Delta l N_3} \over {(\pi \Delta l)^2}}\, {\rm d}\Omega\,{\rm d}\varepsilon. \eqno (13)]

It is more convenient to choose the reciprocal axes system such that Δl is along the reciprocal-lattice vector [{\bf d}_{hkl}^*] and Δh and Δk are parallel to the diffracting Bragg plane hkl (Authier, 2001[Authier, A. (2001). Dynamical Theory of X-ray Diffraction. Oxford University Press.]). Such a transformation can be carried out because the normal to the Bragg plane is always a reciprocal-lattice vector. (Note that Δh, Δk and Δl are dimensionless.) The Jacobians of the transformation of the integration variables are dΩ = V*cell(dhkl/sinθ)λ2d(Δh)d(Δk) and d = λ/(2dhklcosθ)d(Δl) (Authier, 2001[Authier, A. (2001). Dynamical Theory of X-ray Diffraction. Oxford University Press.]), leading to

[\eqalignno{I_0&|F|^2p\left({{e^2}\over {mc^2}}\right)^2 {{\lambda ^3} \over {V_{\rm cell} \sin 2\theta }} {{\sin^2\pi\Delta hN_1} \over {(\pi\Delta h)^2}} \cr & \times {{\sin^2\pi\Delta kN_2} \over {(\pi\Delta k)^2}} {{\sin^2\pi \Delta lN_3} \over {(\pi \Delta l)^2}}\, {\rm d}\Delta h\,{\rm d} \Delta k \, {\rm d}\Delta l. & (14)}]

By integration over Δh and Δk the diffracted intensity for a given value of is obtained,

[I_0|F|^2p\left({{e^2}\over {mc^2}}\right)^2 \lambda^3 {{N_1 N_2} \over {V_{\rm cell}\sin 2\theta}} {{\sin ^2 \pi \Delta l N_3} \over {(\pi \Delta l)^2}}\, {\rm d}\Delta l. \eqno (15)]

The equivalence of this expression to that of James (1958[James, R. W. (1958). The Optical Principles of the Diffraction of X-rays. London: G. Bell & Sons.]) and Buerger (1960[Buerger, M. J. (1960). Crystal Structure Analysis. New York: John Wiley & Sons.]), as we use in EVAL, is shown by the following. How does Δl depend on a small deviation Δθ = ? We write Δl = d(l)/dθ = d(c/dhkl)/dθ = d(cd*hkl)/dθ = c2cosθ/λ. cN3 can be written as ldhklN3, and further we use the property that the volume of the crystal Vcrystal= N1N2N3Vcell and B = 2πdhklcosθ/λ. Writing (14)[link] in terms of d gives

[\eqalignno{I_0&|F|^2p\left({{e^2}\over {mc^2}}\right)^2 \lambda^2 {{N_1 N_2} \over {V_{\rm cell}\sin 2\theta}} {{2ld_{hkl}\cos \theta} \over \lambda} \cr &\times {{\sin^2 2\pi(N_3l d_{hkl}\cos \theta /\lambda\varepsilon)} \over {(2\pi \,ld_{hkl}\cos \theta /\lambda \,\,\varepsilon)^2}}\,{\rm d}\varepsilon. & (16)}]

Using N1N2/Vcell = Vcrystal/V2cell(1/N3) (James, 1958[James, R. W. (1958). The Optical Principles of the Diffraction of X-rays. London: G. Bell & Sons.], p. 43) and dhkl = λ/2sinθ, we can write

[I_0|F|^2p\left({{e^2}\over {mc^2}}\right)^2 \lambda^3 {{V_{\rm crystal}} \over {V_{\rm cell}^2}} {1 \over {2\sin^2\theta}} {1 \over {N_3l}} {{\sin^2\pi N_3 l B\varepsilon} \over {(\pi B\varepsilon)^2}} \, {\rm d}\varepsilon. \eqno (17)]

(17)[link] is exactly the equation used in EVAL (2)[link], as the number of layers s = N3l, and using n = N1N2N3/Vcrystal we can write Vcrystal/V2cell = n2Vcrystal.

Acknowledgements

We gratefully acknowledge Thomas White and Henry Chapman (CFEL, Hamburg) for discussions. This work was supported by the Council for Chemical Sciences of the Netherlands Organization for Scientific Research (NWO-CW) grant No. 175.010.2007.013.

References

First citationAuthier, A. (2001). Dynamical Theory of X-ray Diffraction. Oxford University Press.  Google Scholar
First citationBoutet, S. et al. (2012). Science, 337, 362–364.  CrossRef CAS PubMed Google Scholar
First citationBuerger, M. J. (1960). Crystal Structure Analysis. New York: John Wiley & Sons.  Google Scholar
First citationBurmeister, W. P. (2000). Acta Cryst. D56, 328–341.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationChapman, H. N. et al. (2011). Nature (London), 470, 73–77.  Web of Science CrossRef CAS PubMed Google Scholar
First citationDejoie, C., Smeets, S., Baerlocher, C., Tamura, N., Pattison, P., Abela, R. & McCusker, L. B. (2015). IUCrJ, 2, 361–370.  Web of Science CrossRef CAS PubMed IUCr Journals Google Scholar
First citationDemirci, H. et al. (2013). Acta Cryst. F69, 1066–1009.  CrossRef IUCr Journals Google Scholar
First citationDuisenberg, A. J. M. (1992). J. Appl. Cryst. 25, 92–96.  CrossRef CAS Web of Science IUCr Journals Google Scholar
First citationDuisenberg, A. J. M., Kroon-Batenburg, L. M. J. & Schreurs, A. M. M. (2003). J. Appl. Cryst. 36, 220–229.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationGati, C., Bourenkov, G., Klinge, M., Rehders, D., Stellato, F., Oberthür, D., Yefanov, O., Sommer, B. P., Mogk, S., Duszenko, M., Betzel, C., Schneider, T. R., Chapman, H. N. & Redecke, L. (2014). IUCrJ, 1, 87–94.  Web of Science CrossRef CAS PubMed IUCr Journals Google Scholar
First citationHattne, J. et al. (2014). Nature Methods, 11, 545–548.  Web of Science CrossRef CAS PubMed Google Scholar
First citationHowell, P. L. & Smith, G. D. (1992). J. Appl. Cryst. 25, 81–86.  CrossRef Web of Science IUCr Journals Google Scholar
First citationJames, R. W. (1958). The Optical Principles of the Diffraction of X-rays. London: G. Bell & Sons.  Google Scholar
First citationKabsch, W. (2014). Acta Cryst. D70, 2204–2216.  Web of Science CrossRef IUCr Journals Google Scholar
First citationKirian, R. A., Wang, X., Weierstall, U., Schmidt, K. E., Spence, J. C. H., Hunter, M., Fromme, P., White, T., Chapman, H. N. & Holton, J. (2010). Opt. Express, 18, 5713–5723.  Web of Science CrossRef PubMed Google Scholar
First citationLaue, M. von (1936). Ann. Phys. 41, 971–988.  Google Scholar
First citationLeslie, A. G. W. & Powell, H. R. (2007). Evolving Methods for Macromolecular Crystallography, edited by R. J. Read & J. L. Sussman, pp. 41–51. Dordrecht: Springer.  Google Scholar
First citationMilch, J. R. & Minor, T. C. (1974). J. Appl. Cryst. 7, 502–505.  CrossRef IUCr Journals Web of Science Google Scholar
First citationMurshudov, G. N., Skubák, P., Lebedev, A. A., Pannu, N. S., Steiner, R. A., Nicholls, R. A., Winn, M. D., Long, F. & Vagin, A. A. (2011). Acta Cryst. D67, 355–367.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationQu, K., Zhou, L. & Dong, Y.-H. (2014). Acta Cryst. D70, 1202–1211.  CrossRef IUCr Journals Google Scholar
First citationRavelli, R. B. G. & McSweeney, S. M. (2000). Structure, 8, 315–328.  Web of Science CrossRef PubMed CAS Google Scholar
First citationRedecke, L. et al. (2013). Science, 339, 227–230.  Web of Science CrossRef CAS PubMed Google Scholar
First citationRossmann, M. G. & van Beek, C. G. (1999). Acta Cryst. D55, 1631–1640.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationSauter, N. K. (2015). J. Synchrotron Rad. 22, 239–248.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationSauter, N. K., Grosse-Kunstleve, R. W. & Adams, P. D. (2004). J. Appl. Cryst. 37, 399–409.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationSauter, N. K., Hattne, J., Brewster, A. S., Echols, N., Zwart, P. H. & Adams, P. D. (2014). Acta Cryst. D70, 3299–3309.  Web of Science CrossRef IUCr Journals Google Scholar
First citationSauter, N. K., Hattne, J., Grosse-Kunstleve, R. W. & Echols, N. (2013). Acta Cryst. D69, 1274–1282.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationSchreurs, A. M. M. (1998). VIEW. Utrecht University, The Netherlands.  Google Scholar
First citationSchreurs, A. M. M. (1999). PEAKREF. Utrecht University, The Netherlands.  Google Scholar
First citationSchreurs, A. M. M. (2007). ANY. Utrecht University, The Netherlands.  Google Scholar
First citationSchreurs, A. M. M., Xian, X. & Kroon-Batenburg, L. M. J. (2010). J. Appl. Cryst. 43, 70–82.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationSchutt, N. K. & Winkler, F. K. (1977). The Rotation Method in Crystallography, edited by U. W. Arndt & A. J. Wonacott, pp. 173–186. Amsterdam: North Holland.  Google Scholar
First citationSpence, J. C. H., Kirian, R. A., Wang, X., Weierstall, U., Schmidt, K. E., White, T., Barty, A., Chapman, H. N., Marchesini, S. & Holton, J. (2011). Opt. Express, 19, 2866–2873.  Web of Science CrossRef CAS PubMed Google Scholar
First citationSutton, K. A., Black, P. J., Mercer, K. R., Garman, E. F., Owen, R. L., Snell, E. H. & Bernhard, W. A. (2013). Acta Cryst. D69, 2381–2394.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationThorn, A. & Sheldrick, G. M. (2011). J. Appl. Cryst. 44, 1285–1287.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationUervirojnangkoorn, M., Zeldin, O. B., Lyubimov, A. Y., Hattne, J., Brewster, A. S., Sauter, N. K., Brunger, A. T. & Weis, W. I. (2015). eLife, 4, e05421.  CrossRef Google Scholar
First citationVaney, M. C., Maignan, S., Riès-Kautt, M. & Ducruix, A. (1996). Acta Cryst. D52, 505–517.  CrossRef CAS Web of Science IUCr Journals Google Scholar
First citationWeik, M., Ravelli, R. B. G., Kryger, G., McSweeney, S., Raves, M. L., Harel, M., Gros, P., Silman, I., Kroon, J. & Sussman, J. L. (2000). Proc. Natl Acad. Sci. USA, 97, 623–628.  Web of Science CrossRef PubMed CAS Google Scholar
First citationWhite, T. A. (2014). Philos. Trans. R. Soc. B Biol. Sci. 369, 20130330.  CrossRef Google Scholar
First citationWhite, T. A., Barty, A., Stellato, F., Holton, J. M., Kirian, R. A., Zatsepin, N. A. & Chapman, H. N. (2013). Acta Cryst. D69, 1231–1240.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationWhite, T. A., Kirian, R. A., Martin, A. V., Aquila, A., Nass, K., Barty, A. & Chapman, H. N. (2012). J. Appl. Cryst. 45, 335–341.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationWinn, M. D. et al. (2011). Acta Cryst. D67, 235–242.  Web of Science CrossRef CAS IUCr Journals Google Scholar
First citationZachariasen, W. H. (1945). Theory of X-ray Diffraction in Crystals. New York: Dover.  Google Scholar

This is an open-access article distributed under the terms of the Creative Commons Attribution (CC-BY) Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are cited.

Journal logoBIOLOGICAL
CRYSTALLOGRAPHY
ISSN: 1399-0047
Volume 71| Part 9| September 2015| Pages 1799-1811
Follow Acta Cryst. D
Sign up for e-alerts
Follow Acta Cryst. on Twitter
Follow us on facebook
Sign up for RSS feeds