Evaluation of models determined by neutron diffraction and proposed improvements to their validation and deposition

Models of crystal structures determined by neutron diffraction and deposited in the Protein Data Bank to date were analysed. The lessons learned from this data-mining effort are summarized and suggestions for improvements to the deposition and validation of neutron models are outlined.


Introduction
The predominant method to determine the three-dimensional structure of macromolecules is X-ray crystallography (Fig. 1), which is based on the interaction between X-rays and the electrons of the atoms constituting the crystal. Neutron diffraction is a complementary technique that relies on the interaction of neutrons with atomic nuclei. The neutron scattering cross-section, which determines the probability of a neutron being scattered by a nucleus, varies by element (or isotope) in a nonlinear fashion, as opposed to X-rays, where the scattering increases with the number of electrons. This is why neutron diffraction complements X-ray diffraction by enabling the location of very light atoms or ions such as hydrogen or protons in protein structures. As the knowledge of H-atom positions is important for determining the protonation states and reaction pathways of proteins (Engler et al., 2003;Weber et al., 2013;Haupt et al., 2014;Casadei et al., 2014;Howard et al., 2016), neutron diffraction is able to provide valuable information for the understanding of catalytic mechanisms and ligand binding (Yamaguchi et al., 2009;Bryan et al., 2013;Knihtila et al., 2015).
However, neutron diffraction may be challenging in practice for the following reasons.
(i) Experimental. The beam flux at neutron sources is relatively weak compared with X-ray sources, necessitating the use of larger crystals (typically at least 0.1 mm 3 ) and longer data-collection times for the experiment (Howard et al., 2011;Weber et al., 2013;Ng et al., 2015). To date, the smallest crystal used for neutron data collection had a volume of 0.05 mm 3 (Howard et al., 2016). It is difficult to grow crystals of most proteins to such large sizes, and the number of proteins that can be explored by neutron crystallography is therefore relatively small. Furthermore, data-collection times are typically several days to a month on contemporary neutron sources (Blakeley et al., 2008;Coates et al., 2015;Chen & Unkefer, 2017). As hydrogen has an incoherent scattering cross-section that contributes to a high neutron scattering background level, it is preferable to replace hydrogen by deuterium, which has a much smaller incoherent scattering cross-section. Further, deuterium has a larger coherent scattering cross-section than hydrogen, and replacing hydrogen by deuterium therefore increases the signal-to-background ratio of diffraction peaks. Accessible H atoms in polar bonds (such as N-H, S-H and O-H, known as exchangeable H atoms or labile sites) can be either fully or partially exchanged for deuterium by soaking crystals in a deuterated buffer for several days prior to the diffraction experiment. However, in order to replace hydrogen by deuterium in nonpolar covalent bonds (such as C-H, known as non-exchangeable H atoms or nonlabile sites) protein expression must take place using fully deuterated reagents to produce what are referred to as perdeuterated samples. Obtaining perdeuterated samples can be a costly and time-consuming process (Price & Fernandez-Alonso, 2017) and can be an experimental obstacle. Apart from lowering the background scattering, perdeuterated samples offer several other benefits, such as the ability to use smaller crystal volumes, higher resolution data and faster data collection. Many studies are therefore performed with perdeuterated crystals (Meilleur et al., 2013;Coates et al., 2014;Cuypers et al., 2016;Li et al., 2017;Shu et al., 2000;Fisher et al., 2014;Blakeley et al., 2015).
(ii) Quality of the diffraction data. Neutron data typically have a lower completeness compared with X-ray data. It is desirable that the completeness of a typical X-ray data set is greater than 95% (Dauter, 2017), but only a few neutron data sets satisfy this criterion (Fig. 2). The majority of data sets are less complete, averaging about 80%, owing to several factors including the relatively low flux of available neutron beams, reduction in signal to noise owing to incoherent scattering if any hydrogen is present, and the limited data-collection time available on highly oversubscribed macromolecular neutron crystallography instruments (for example, the oversubscription rate on the MaNDi and IMAGINE beamlines at the Spallation Neutron Source and High Flux Reactor neutron sources at Oak Ridge National Laboratory is typically greater than 300%). We observe that the completeness of neutron data has not improved notably during the past 25 years.
(iii) Model building and refinement. Using neutron diffraction data, H (or D) atoms can be refined individually along with the non-H atoms. If this strategy is applied, the number of parameters to be refined increases substantially, as about half of the atoms in a protein are H atoms. Furthermore, the neutron scattering length of hydrogen is negative, which can lead to scattering cancellation in medium-to low-resolution nuclear scattering length density maps when hydrogen is bound to atoms with a positive scattering length, such as in CH 2 groups. 1 To avoid negative scattering of H atoms, hydrogen can be partially or fully exchanged by using soaked or perdeuterated crystals (see above). However, the presence of different levels of H/D exchange makes model building more complicated, as there can be both H and D atoms, or either of them, at one location. We note that if the occupancy ratio of the H and D atoms at exchanged sites is about 0.6:0.4 the scattering is canceled [for illustrations, see Afonine et al. (2010) and references therein]. To tackle challenges in the model refinement process owing to low data completeness, low signal to noise and the increased number of parameters, the concept of joint X-ray and neutron refinement (hereafter referred to as joint XN refinement) was introduced (Coppens, 1967;Orpen et al., 1978;Wlodawer, 1980;Wlodawer & Hendrickson, 1982;Adams et al., 2009;Afonine et al., 2010). In joint XN refinement a single model is simultaneously refined against X-ray and neutron data. Both data sets should be collected at the same temperature and should ideally be from the same or a highly isomorphous crystal, although this cannot always be realized. Patched versions of programs originally designed for the refinement of X-ray structures were made Experimental methods used to determine models in the PDB. The predominant method is X-ray diffraction, followed by NMR and cryo-EM. Other methods are shown in the bar chart. available to perform refinement using neutron data (Ostermann et al., 2002;Engler et al., 2003;. Also, as the number of neutron structures is still rather small, there are as yet no community-wide conventions for dealing with models obtained from joint refinement and/or that contain both H and D atoms. When computational tools are developed, it is desirable to exercise the new algorithms using all available data and models (see, for example, Afonine et al., 2009;Weichenberger et al., 2015). This ensures that the new developments work not only on the developer's favourite examples but are also robust enough to work generally, which is the key for automated software development. New tools for joint XN refinement are being developed in the framework of the PHENIX software suite . To test the algorithms, all neutron models and diffraction data available as of 8 September 2017 in the Protein Data Bank (PDB; Berman et al., 1977Berman et al., , 2000 were analyzed. An approach for the early detection of issues that could cause problems is to use the deposited data and model to calculate R work and R free , and compare the obtained values with the published values. A mismatch may be indicative of various issues, ranging from trivial typos to incomplete or incorrect annotations in the deposited data. We find a surprising number of models that show large differences (reaching up to 30%) between the reported and recomputed R factors. These models and data were inspected in order to determine the origin of the differences. This study summarizes the lessons learned from the data-mining effort.

Materials and methods
2.1. Collecting the data from the PDB All computations were performed with PHENIX tools . Models determined by neutron diffraction were identified using the 'experimental method' search option on the PDB website. The model PDB and data files were obtained with the phenix.fetch_pdb tool. Information relevant to recomputing R factors using the same conditions as were used for refinement of the final structure by the depositors were automatically extracted from the PDB file header: minimum and maximum resolution limits and cutoff as well as the twin law, if present. Furthermore, crystallographic R factors (R work and R free ), the deposition year and the program used for refinement were obtained from the PDB file header.

Diffraction data labels for joint XN data sets
In the case of models determined by joint XN refinement, the corresponding data file should contain at least two data arrays: one for the neutron data and one for the X-ray data. It is therefore important to know which data array corresponds to which experiment. In the data CIF file the item _diffrn.details can be used to describe the details of the diffraction measurement, such as 'first data set reflections X-ray diffraction' and 'second data set reflections neutron diffraction'. We note that annotations could not be parsed automatically. The keyword or sentence was not consistently the same and in several instances only one data array had an annotation while the other did not. However, a practical way to determine which data array corresponds to which experiment is to compute R work using X-ray and neutron scattering factors for both data arrays; the wrong set of scattering factors leads to higher R factors.
2.3. Model files 2.3.1. Assessment of hydrogenation state. We define the hydrogenation state as a model feature describing how the experimentalists chose to model H-atom sites (using H, D or H and D). The presence of H and D atoms in the PDB file was used to sort models into four different categories.
(i) Predominantly D atoms are present. This case occurs for crystals of perdeuterated protein containing deuterated solvent.
(ii) Predominantly H atoms are present. This case occurs for crystals of hydrogenous protein containing hydrogenous solvent.
(iii) Significant amounts of both H and D atoms are present, with more H atoms than D atoms. This case occurs for crystals of hydrogenous proteins containing a relatively small amount of deuterated solvent, or for crystals of perdeuterated protein containing relatively large amounts of hydrogenous solvent.
(iv) Significant amounts of both H and D atoms are present, with more D atoms than H atoms. This case occurs for crystals of perdeuterated protein containing deuterated solvent, if metabolites were used during protein expression that were not fully deuterated, if some D atoms have been back-exchanged by H atoms during sample preparation or handling, or for crystals of hydrogenous proteins that contain relatively large amounts of deuterated solvent.
Case (i) is worthy of further consideration. Even if a protein is expressed from organisms cultured in deuterated reagents and crystallization is performed in deuterated solutions, there is a chance that the sample will have been exposed to ambient hydrogenated moisture at some stage. It is therefore unlikely that all H atoms (100%) are replaced by D atoms (an all-D refinement protocol might nevertheless be chosen, for example to increase the data-to-parameter ratio). Also, it may happen that some D atoms back-exchange to hydrogen if hydrogenated reagents are used in one of the protein crystal-production steps (such as purification; Haupt et al., 2014;Yee et al., 2017). Some models therefore contain a majority of D atoms and very few H atoms. To prevent the misinterpretation of such a model as containing both H and D, which means that H atoms are at all exchangeable sites, a cutoff was applied. If more than 90% of atoms are of one type (H or D) this type is assigned. We chose 90% because it represents a compromise between a strict separation of perdeuterated versus hydrogenated and the experimental reality that even perdeuterated crystals can contain some H atoms.
Furthermore, for each model we determined the total number of H or D atoms and the number of H atoms, D atoms and exchanged sites in protein (or RNA/DNA) residues. Here, an exchanged site is not counted twice as belonging to the H and D atoms as well; for example, an H atom is either H, D or exchanged. A site was identified as being exchanged if both H and D were used to model it. The number of H or D atoms in other molecular species was also determined, including water molecules and ligands. Finally, the percentage of H/D-exchange sites per protein H and D atom was analysed.

Properties of H and/or D
atoms. In addition to counting H and D atoms (x2.3.1), we also looked at (i) models containing H or D atoms with occupancies smaller than zero; (ii)  Table 1 H and D atoms in protein residues, waters and other entities.
Water molecules in alternative conformations were not counted among the categories with zero, one or two D atoms. This result is not shown, so the sum is not always equal to the total number of water molecules.

Generation of H and/or D
atoms. If the deposited model did not contain H or D atoms according to the published information, they were generated using phenix.ready_set. If both H and D atoms were added at exchangeable sites, the occupancy ratio was set to 50:50. These curated models were used to test hypotheses about particular issues. The reported values in Tables 1, 2 and 3 are based on original models (unless curation was necessary to be able to process the file; for example, a few models contained corrupt atom names).

Model-to-data fit: computation of R factors
To assess the model-to-data fit, R work and R free were computed using resolutions and cutoffs as reported in the PDB header or the literature. X-ray (Maslen et al., 1992;Waasmaier & Kirfel, 1995;Grosse-Kunstleve et al., 2004) and neutron (Sears, 1992) scattering tables were used as appropriate.

Results and discussion
3.1. Overview of neutron models deposited to date As of 8 September 2017, the number of neutron diffraction models deposited in the PDB was 122. Fig. 3 shows the cumulative number of neutron models per year. The first model in the database was determined in 1984 and corresponds to the structure of a bovine pancreatic trypsin inhibitor determined by joint XN refinement (PDB entry 5PTi; Wlodawer et al., 1984). However, several structural reports predate the establishment of the PDB, such as a model of myoglobin (Schoenborn, 1969), or were not deposited in the PDB, such as a model of crambin (Teeter & Kossiakoff, 1984).
It can be noted that no models were deposited between 1990 and 1998 owing to the unavailability of macromolecular neutron crystallography facilities in the early 1990s.   (Cipriani et al., 1997) in France; PCS (Langan et al., 2004), MaNDi  and IMAGINE (Meilleur et al., 2013) in the USA; BioDiff (Ostermann & Schrader, 2015) in Germany; and iBIX (Kurihara, Tanaka, Muslih et al., 2004) in Japan. New methods and technologies have been developed, such as the development of the neutron image-plate detector (Niimura et al., 1994) and the development of new types of macromolecular neutron crystallography beamlines based on the use of powerful time-of-flight techniques at spallation sources (Langan et al., 2004). The rate of structure deposition will increase further with several next-generation advanced neutron sources that are under construction or commissioning, including the ESS in Sweden (https:// europeanspallationsource.se) and the CSNS in China (http://english.ihep.cas.cn/ csns/).
The total number of deposited structures has grown since the 1980s, but the number of depositions per year is low compared with X-ray crystallography and has varied between three and 22 during the past decade. Among the 122 deposited structures, 55 were determined using neutron data alone (coral in Fig. 3) and 67 were obtained from joint XN refinement (blue in Fig. 3). Most of the recently deposited structures were refined using the joint XN refinement method. The development of robust refinement algorithms for joint XN refinement has enabled the increased use of macromolecular neutron crystallography and has provided more complete (including all atoms) and more accurate structures. Fig. 4 shows the resolution of the neutron diffraction data sets as a function of deposition year. Interestingly, the average resolution has not improved in a period of more than 35 years, with the majority of data sets having resolutions of between 1.5 and 2.5 Å . The mean data resolution for all 122 deposited models  Table 2 Summary for models determined using neutron data alone.
The models are sorted according to their deposition year, except for the six models without data, which are at the end.  (Cuypers et al., 2013), which has neutron data extending to 1.05 Å resolution. This is related to the primary reason that researchers conduct neutron crystallography studies of biological macromolecules. Neutron crystallography is not used to determine the structures of biological macromolecules; that is best performed using X-ray crystallography. Rather, neutron crystallography addresses critical science questions that require the direct location and visualization of functionally important H atoms or protons. Using neutron crystallography, H atoms can be located at resolutions of 2.5 Å or less, i.e. the resolution of almost all deposited neutron structures. An exception is PDB entry 3VXF, which was determined with neutron data collected to 2.75 Å resolution (Yamada et al., 2013).
The earlier models were refined with PROLSQ (Hendrickson & Konnert, 1979) and some models determined with neutron data alone were refined using X-PLOR (Brunger, 1992) or SHELX (Gruene et al., 2014;Sheldrick, 2015). We note that the neutron community is increasingly using programs tailored to handle neutron data, such as PHENIX (Afonine et al., 2010) and nCNS , which can be used for joint XN refinement (  Table 3 Summary for models determined using joint XN refinement.  2011 all_d PHENIX 1.85 n/a n/a n/a 16.6 21.4 n/a n/a n/a n/a n/a n/a 3QZA 2011  are available (PDB entries 3QF6, 4Q49, 3KKX, 5DPN, 3iNS and 5A93). 2 In these cases it is possible to refine models against the neutron data alone, but the joint refinement cannot be reproduced. The absence of the X-ray data is largely a result of limitations in earlier PDB deposition processes. It is important that experimental data should be deposited and made available. Of the 122 models determined via neutron or joint XN refinement, nine do not have neutron data, which is more than 7%. 3.2.2. Type of diffraction data. When multiple data arrays associated with a PDB entry are available, it is important to be able to identify whether an array corresponds to X-ray or neutron data. Only 27 of the 67 joint data sets had an annotation in the CIF file, whereas a majority of 40 models did not have any specification. These annotations cannot be processed automatically as they are inconsistent or incomplete in many cases. For example, in some instances there was an annotation for only one array while the other array had none. By comparing R factors using X-ray and neutron scattering factors for both data arrays, their type could be identified. However, this may be complicated if this is convoluted with the issue of incorrect H/D assignment (see x3.3.2).

Incomplete or missing cross-validation (R free ) sets.
The R free flags in 24 data sets do not match the available data. This means that at least one reflection in the data file did not have an R free flag assigning it to the test set or the working set. If R free flags are present, PHENIX tools require a data file to have these flags for every reflection.
3.2.4. Wrong data annotations. It is important to know whether diffraction data are intensities or amplitudes. For example, the neutron data array for PDB entry 2iNQ is indicated as structure-factor amplitudes in the CIF file. The recomputed R work and R free are 26.6 and 30.6%, respectively. If the data array is treated as intensities, R work and R free are 20.9 and 25.0%, respectively, which are much closer to the published values of 18.2 and 23.3%. This is likely to be owing to incorrect annotation during deposition or conversion.

Figure 3
Cumulative number of neutron models in the PDB. Coral, models determined using neutron data alone. Blue, models determined using joint XN refinement.
tions, such as the resolution limit or the cutoff, have not been included. Furthermore, there are cases where the information is different in the header and in the concomitant paper. For example, the header of PDB entry 1WQ2 reports 22.9 and 28.9% for R work and R free , respectively, while the paper indicates values of 28.2 and 30.1% (Chatake, Mizuno et al., 2003). The latter are similar to the recomputed R factors (28.5 and 31.1% for R work and R free , respectively). The H (or D) atoms and the presence of exchanged sites with both H and D are most likely to be the largest source of confusion in model files (discussed below).

Availability of H/D atoms.
(i) No H or D atoms are deposited. One model (PDB entry 5KSC) was deposited without any H or D atoms on the protein residues (in contrast, all water molecules have two D atoms). The primary purpose of a neutron diffraction experiment is to obtain information about H atoms. If the deposited model is lacking H atoms, an important interpretation of the experimental result is not accessible.
(ii) Wrong atom type is deposited. PDB entry 1CQ2 contains H atoms in the protein chain, while all water molecules are D 2 O molecules. The PDB header suggests that the protein is fully deuterated (OTHER_DETAILS: PROTEIN IS FULLY DEUTERATED). Switching from the original to a fully deuterated model decreases R work and R free from 47.1 and 47.7% to 21.7 and 26.8%, respectively.
(iii) Only one atom type at exchanged sites. PDB entry 1C57 only contains H atoms, while the literature (Habash et al., 2000) describes that the model was refined with D atoms at the backbone amide groups. Another model, an earlier version of which was in the PDB at the time this manuscript was prepared, contained H atoms with full occupancy, while no D atoms were present (the model was meanwhile curated and contains now both H and D). The issue of missing atoms at exchanged sites is difficult to detect, but is mainly associated with early structures that were determined while robust refinement methods were still being developed.
(iv) Missing H or D atoms. We found that several models with some lysine side-chain terminal NH 3 groups or CH 3 methyl groups did not contain all H or D atoms, i.e. one or two of the three H (or D) atoms were missing. For example, eight models contained at least ten residues where exactly one H atom was missing in propeller groups. When two (or one) of the atoms are present in the XH 3 group, the location of other atoms is automatically determined. In some cases, the omission may reflect a different protonation or charge state, which may be functionally important. In fact, one of the goals of a neutron diffraction study may be to determine the charge state of a catalytically important lysine residue. In other cases, it is possible that the software unintentionally omitted the H atoms. To be able to distinguish these two scenarios, it should be explicitly marked when residues are in a charged state, such as for neutral lysine (for example, using a PRB remark). As H and D atoms are non-negligible scatterers in neutron models, their unjustified systematic omission deteriorates the model quality.
3.3.3. Modeling of partially exchanged sites. Atomic models of partially deuterated crystals contain sites with both H and D atoms sharing the same location. This situation arises if only a fraction of a particular H atom of all of the molecules in the crystal was replaced by a D atom. At least three approaches for the simultaneous modeling of an H and D atom at the same location were found in models deposited in the PDB. Fig. 5 shows the PDB format for an amide H atom for the three modeling options. The PDB format lines describe the same information, i.e. an H atom with occupancy 0.77 and a D atom with occupancy 0.23 at the same location. The lines look rather different for the different methods and they are explained below.
(i) Modeling the H and D atoms as a double conformation. This is the most common approach, which aims to prevent the application of nonbonded repulsion restraints during refinement between the H and the D atoms. An important difference when compared with alternative conformations for an entire residue is that the atom names of the H and D atoms are different. For example, in the serine residue hydroxyl group, the H atom will be modeled in conformation A with the name 'HG', while the D atoms is in conformation B with the name 'DG'. This method has both the H atom and the D atom in the model file. However, it is unclear how residues in alternate conformations and simultaneously containing exchanged H/D atoms will be defined. PDB entry 3VXF, for example, has alternative conformations of a leucine residue in conformations A and C, while the exchanged amide D atom has conformation B. This is the major limitation of this method. We note that 13 models had at least one H/D pair with erroneous atom names and had to be curated. For example, the amide H/D atoms H and D both had the alternative conformer identifier (ID) A in Ile10 of PDB entry 2VS2. The correction included changing the alternative conformer ID of one of the atoms to B. Resolution of neutron models in Å as a function of the year of deposition (black circles). The black line represents a linear fit. The average resolution did not improve over a period of more than 35 years, as shown by the linear fit, which is almost parallel to the y axis at a resolution of (ii) Only one atom type (H or D) is present in the PDB file with occupancy q. The other atom type is not present in the file, but it is situated at the same position as its exchanged partner atom and has a complementary occupancy of 1 À q. This scenario occurs in PDB entry 3BYC, which was determined from a soaked crystal, and should therefore contain both H and D atoms. The deposited model file only contains protein H atoms with occupancy q. A remark in the PDB header states that the D-atom occupancies are 1 À q. Other models have similar configurations but do not contain a remark (such as PDB entry 4G0C). An obvious disadvantage of this approach is that the deposited model is incomplete because it does not contain all H/D atoms and therefore such a model needs manipulation (adding missing D atoms) before it can be used. Although this is straightforward to interpret individually, it can lead to confusion during automatic data mining.
(iii) Altering the definition of occupancy. The occupancy of an atom reflects the fraction of molecules in the crystal in which this atom occupies a certain position. Therefore, to be meaningful the occupancy value is expected to be between zero and one. However, some PDB entries contain D atoms with negative occupancies (PDB entries 3CWH, 3KCJ, 3KCL, 3KCo, 3KKX, 3KMF and 3oTJ) in order to represent the H/D-exchange ratio. The value of the occupancy ranges from À0.56 (H fully occupied) to 1 (D fully occupied) (Kawamura et al., 2011). In this approach the definition of the occupancy value is misused, as it does not reflect the occupancy of the atom in question. We note that of the seven models that use the apparent occupancy only one contains a remark explaining the modified meaning of the occupancy in the PDB file header (3oTJ). Furthermore, similarly to the second method, this approach yields an incomplete model, as not all H or D atoms are present in the file. Clearly, all three of the above approaches have their particular advantages and limitations. Method (iii) correctly reflects the scattering factors during refinement but it creates occupancy definition issues for automatic PDB mining. Methods (ii) and (iii) lead to atom-incomplete models that require curation to be usable.
3.3.4. Hydrogenation state. Fig. 6 shows a histogram of the hydrogenation state, as determined by the procedure described in x2.3.1. Most models (88) contain significant amounts of both H and D atoms, with a majority of H atoms. It is likely that these models correspond to crystals of hydrogenated protein soaked in D 2 O. 19 models contain predominantly D atoms (among H and D) and are likely to originate from crystals of perdeuterated protein containing deuterated solvent. Ten models contain significant amounts of both H and D atoms, with a majority of D atoms. In most of these cases the proteins were expressed in a deuterated medium that contained D 2 O but with hydrogenated glycerol, which leads to mixed H/D occupancy at every nonexchangeable C-H site (PDB entries 4JEC, 5E5J, 5E5K and 5T8H), hydrogen labeling (PDB entry 3KYY; Gardberg et al., 2010) or selective protonation or deuteration PDB entry 4NY6). Five models are in the fourth category and contain mainly H atoms (among H and D), such as PDB entries 5D97 (a hydrogenated crystal) and 1NTP (contains a small number of exchanged H atoms).
As the hydrogenation state is difficult to assess algorithmically, we suggest that the PDB or mmCIF file should contain a specific keyword identifying the protonation state. For example : 'protonation: H' (or 'D' or 'H and D' in the other cases). Table 1 shows a more detailed breakdown of the H-and D-atom count, sorted according to the hydrogenation state of the model, for protein residues, water molecules and other entities (such as ligands). The percentage of H/D sites represents how many of the total H sites in a protein are modeled with both an H and a D atom. A large number of models containing both H and D atoms do not have shared sites, i.e. a site is either occupied by an H atom or a D atom. Most Illustration of PDB format for an exchanged amide H atom in an arginine residue (residue 38 of chain A). All three possibilities describe the same configuration: one H atom with occupancy q H = 0.77 and a D atom with complementary occupancy q D = 1 À q H = 0.23. Top, method (i), using double conformation and different atom names. Middle, method (ii), implying the D atom with q D = 1 À q H . Bottom, method (iii), using an apparent occupancy. The occupancy mimics the total scattering contributions of the H and D atoms, which are of different signs, and varies from À0.56 (H fully occupied) to 1 (D fully occupied).

Figure 6
Distribution of hydrogenation states. The majority of models contain both H and D atoms.
notably, of the 88 models that contain more H than D atoms, 23 do not have any shared sites. It is not possible to determine algorithmically whether this choice was made on purpose (for example to decrease the number of refinable parameters by avoiding H/D-occupancy refinement) or whether the complementary atom is assumed to be accounted for but is not physically present in the file [such as for method (ii) described in x3.3.3].
For models containing shared sites, the ratio of exchanged sites and all modeled protein H atoms in the model in question varies between 4 and 23%. Notable exceptions are models 4JEC, 5E5J, 5E5K and 5T8H, where the ratio of exchanged H/D is 83, 78, 75 and 69%, respectively. As mentioned above, the samples for these models were prepared in a special manner and are expected to contain H and D atoms at the majority of sites (exchangeable and not exchangeable).
The table also lists the number of water molecules modeled with no, one or two D atoms. In 52 of the 122 models all water molecules were modeled as D 2 O molecules. However, it was reported that only a fraction of water molecules show a distinguished triangular shape in nuclear scattering length density maps that allows the location of both D atoms (Chatake, Ostermann et al., 2003). Fig. 7 shows the percentage of water molecules modeled as D 2 O as a function of resolution. Only with the higher resolution data sets is it possible to accurately differentiate between different water species (OD À , D 2 O and D 3 O + ).
3.3.5. Properties of exchanged sites. As D has a larger mass than H, it is expected that D has a lower ADP. However, the resolution of most macromolecular neutron diffraction data sets is not sufficient to detect this difference. Imposing the same ADPs and coordinates for H and D atoms is therefore a reasonable approximation. The sum of occupancies at H/D sites is constrained to 1. We analysed whether exchanged sites in all models fulfil these criteria.
Out of 81 models with at least one exchanged H/D, 20 have different coordinates (25%), ten have sites with different ADPs and eight and six have the sum of occupancies smaller and larger than one, respectively. The number of mismatches per model can range from one (one coordinate mismatch, PDB entry 3U2J) to 542 (coordinate mismatch and occupancy sum < 1; PDB entry 2DXM).
In some cases, the mismatch comes from model errors, such as in PDB entry 4JEC, where the HG3 atom of proline 1 (chain A) has the wrong atom name, which should be correctly indicated as HG2. It has the same coordinates and ADP as DG2 and the sum of occupancies q DG2 + q HG3(HG2) = 1. DG3, on the other hand, is modeled as being fully occupied. It therefore cannot have an exchanged partner. 364 atoms suffer from mislabeled atom names in this model.
In other models, such as 3FHP, the H and D atoms of the amide N atom are modeled systematically with different coordinates. The distance between the atoms ranges from 0.01 Å (Gly20, chain D) to 0.5 Å (Leu6, chain B).
Model 3HGN has 232 sites with different ADPs (but the same coordinates) for the H and the D atom. The difference can reach up to 11 Å 2 (Asn148, chain A). It is possible that the ADPs were refined individually for both atoms (as opposed to being constrained to be equal to each other, as is desirable).
3.3.6. The covalent X-H bond lengths are set to standard X-ray distances. The X-H bond length is different in models derived from X-ray and neutron diffraction data. X-rays interact with electrons, and in the case of the H atom (which has only one valence electron and therefore no core electrons) the electron distribution is shifted along the covalent X-H bond towards atom X. Neutrons interact with the nuclei, which are not affected by deformations of the valence electron density owing to chemical bonds. H-atom nuclear scattering length density peaks are therefore at a different location to electron-density peaks (Fig. 8), and X-H bond lengths thus appear to be shorter in X-ray models than in neutron models. The difference in bond length is 10-20% (Allen, 1986;Allen & Bruno, 2010), requiring that standard neutron distances be used for the refinement of H and D atoms in neutron models. It was mentioned by Gruene et al. (2014) that several neutron models were refined with X-ray X-H bond lengths. Of the 122 neutron models deposited in the PDB, the H (or D) atoms are located at X-ray distances in more than 40 models. Percentage of water molecules with two D atoms as a function of neutron data resolution (black circles).

Figure 8
Schematic figure illustrating the X-ray and neutron N-H bond lengths for an amide H atom. The nuclei are represented by black spheres and the electron cloud of the H atom is represented by the blue gradient-coloured oval. The centre of the electron distribution is shifted towards the N atom along the N-H covalent bond.
Using shorter (X-ray) instead of longer (neutron) bond lengths may not affect R-factor values greatly and the effect may largely depend on the data resolution (lower impact at lower resolution, greater impact at higher resolution). For example, in PDB entry 2GVE, which contains 4195 H atoms, most of them are placed at standard X-ray distances; the recomputed R work and R free are 24.7 and 30.0%, respectively. Using standard neutron distances, R free decreases to 29.8% while R work remains the same. However, to obtain a model that reflects the experimental data correctly, the X-H distances should be according to commonly accepted targets for neutron distances. Table 2 provides a summary of the following parameters for all neutron models: PDB code, deposition year, H/D state, refinement program, high-resolution and cutoff, published and recomputed R work and R free . Table 3 lists the same information for models from joint XN refinement, along with relevant cutoffs and R factors for the X-ray data sets.

Summary of the lessons learned from the survey
To address the differences between models described in x3, we suggest that the following guidelines are adopted during the deposition and validation of neutron models.
(i) All H (D) atoms used in refinement should be deposited.
(ii) Information describing the experiment or the results should be correct and be consistent with the concomitant publication, such as resolution limits and cutoffs.
(iii) For joint models, all data should be made available, i.e. X-ray and neutron diffraction data, and the data arrays should be unambiguously marked.
(iv) A community-wide accepted description of H/Dexchanged locations did not exist during the early days of macromolecular neutron crystallography, and three different approaches have been used. Moving forward, there is an opportunity to adopt a new description that is compatible between different software packages and does not change the usual definition of existing parameters (such as the occupancy). As the community transitions to the mmCIF format, 3 this is a good opportunity to address this issue. A solution could be more-than-one-letter alternative conformation IDs.
(v) We also note that when models determined via neutron diffraction are being deposited a validation report is generated but it is not made available in the PDB. 4 (vi) There is a need for validation tools specifically designed for neutron crystallography. Current validation software either ignores H atoms or only uses them to validate heavyatom positions and geometry. It should be also noted that the PDB allows authors to correct a structure at any point, i.e. deposit a revised version, which could be an opportunity to curate some of the issues that are described in this report.

Development of a validation tool for H atoms
The work described in this report led to the development of a new tool in PHENIX that can comprehensively validate neutron models and data. It is available in PHENIX release 1.13 and later. The following validation tasks are performed.
(i) Identification of missing H (or D) atoms.
(ii) An accounting of the number of H, D and exchanged H/D sites.
(iii) Identification of H/D sites with an occupancy ratio that leads to nearly full cancellation of their density (approximately 0.35/0.65). If such a site has a degree of freedom, it should be checked.
(iv) Identification of H/D sites with different coordinates, ADPs and unlikely occupancy values.
(v) A count of water molecules with zero, one or two D atoms.
(vi) A warning message if X-ray X-H distances are used. Broader use of this tool will help address some of the issues that are raised in our analysis.

Conclusions
Neutron models constitute a small fraction of the models deposited in the PDB; however, the information that they provide is unique and of great importance for understanding biological function. At present, X-ray crystallography is the method of choice for determining the structure of biological macromolecules. Neutron crystallography is used only in cases where a critical science question requires the direct localization and visualization of H atoms or protons. The initial goal of surveying neutron models was to verify the suitability of their use in the development and benchmarking of new robust computational tools for neutron crystallography. However, a preliminary assessment of model-to-data fit quality has revealed opportunities to improve the PDB annotation and validation methods and the deposition process itself. Implementation of the suggested improvements will minimize inconsistencies between the deposited neutron models available in the PDB and therefore the possibility of misinterpretation. Most of the issues identified concerned the handling of H and D atoms. The survey led to the development of a new tool in PHENIX that can comprehensively validate H and D atoms in protein models. Since the primary use of macromolecular crystallography is to locate and directly visualize H atoms, it is important to address these issues, so that deposited neutron models allow the retrieval of the maximum amount of information with the smallest effort of manual intervention.