In defence of our science – validation now!

As 2009 ended, a scientific scandal of epic proportions shook macromolecular crystallography to its core. On 9 December last year, the University of Alabama at Birmingham (UAB), USA, announced that, after a thorough examination of the available data, the structures associated with the PDB codes 1bef, 1cmw, 1df9, 2qid, 1g40, 1g44, 1l6l, 2ou1, 1rid, 1y8e, 2a01 and 2hr0 were based on fabricated data and recommended that they be removed from the public record (see https://main.uab.edu/Sites/reporter/articles/71570/). Even our journals fell victim to this deception: 1cmw, the deposition code for a structure of DNA polymerase I from T. aquaticus (TaqP), was published in Acta Cryst. D about a decade ago (Urs et al., 1999). As serious as this is, there are reasons to suspect that these 12 structures may not be the only problematic ones.

Macromolecular crystallography has faced challenges to the quality of its science in the past. In the mid 1980s a few structures were published that turned out to be incorrect. Space groups were determined incorrectly, the wrong hand of the heavy-atom structure was chosen or parts of the protein chain were traced incorrectly. As serious as these errors were, they were honest errors. They were caused by the limitations imposed by the experimental data, by the lack of proper validation tools and probably also by the mounting pressure on the scientists to be first, to publish first and to publish well. These incidents justifiably tarnished the unrealistic reputation of macromolecular crystallography for scientific infallibility, but they also precipitated very positive developments. A great strength of crystallography is the ability to rigorously check structural models against the experimental data. Stimulated by those few errors, many new validation and cross validation tools were developed by the community. These developments still continue and today it is virtually impossible for crystallographers to arrive at an incorrect structure, except when they are determined to ignore all the warning signs.

The current crisis is a significant step to the dark side. From the UAB report, it appears that the scientist responsible did not base the structures on experimental diffraction data, but rather made a more-or-less educated guess on what the structures might be, fabricated the underlying data and, we must assume, duped collaborators. Since science is a social exercise based on mutual trust, this deception is an affront to honest scientists everywhere.

Our own journals have always strived to provide as much information as possible to our editors and referees in order to make sure that each structure is properly assessed and evaluated before it is published. We ask the authors of structural papers to provide a large amount of experimental evidence in the experimental tables, and, for some time now, we have encouraged them to provide the validation report that they obtain from the Protein Data Bank upon deposition of coordinates. We now go one step further: we will make the submission of a validation report mandatory for every structure that is submitted to our journals. This will force some changes that may cause discomfort to authors, by requiring deposition before submission, but this discomfort is the price for assuring quality. We also follow closely the work of the worldwide PDB's validation task force and will do our best to offer authors the benefits of any new developments. We hope that by requiring validation reports at the time of submission we send an important signal to our authors that it is in their own interest to be as transparent as possible, to other journals to follow us in that direction, and to the scientific community at large that macromolecular crystallography will do its very best to make it harder and harder for such deceptions to slip through the net in the future.