Sixth blind test of organic crystal-structure prediction methods

Over the past 15 years progress in predicting crystal structures of small organic molecules has been charted by a series of blind tests hosted by the Cambridge Crystallographic Data Centre. This letter announces a sixth blind test to take place between September 2014 and August 2015, giving details of the target systems and the revised procedure. We hope that as many methods as possible will be assessed and benchmarked in this new blind test.

Organic molecules can have a remarkable array of solid forms, including different polymorphs and various multi-component systems such as salts and co-crystals. The potential diversity of this solid-form landscape presents both opportunities and headaches for the practical use of molecules in solid forms. One classic example is the serious consequences that unforeseen polymorphism in an active pharmaceutical ingredient can have if it emerges once the drug has reached the market.

Experimental screening of the solid-form landscape can be a time-consuming and expensive process. It is therefore not surprising that over the past 25 years numerous computational methods have been developed to predict crystal structures, providing an alternative or supplement to experimental screening of solid forms and allowing us to explore the solid state of molecules that have yet to be synthesized.

In the case of organic crystal-structure prediction (CSP), progress over the last 15 years has been charted by a series of blind tests of CSP methods that have been hosted by the Cambridge Crystallographic Data Centre (CCDC). Five blind tests have been held to date in 1999 (Lommerse et al., 2000), 2001 (Motherwell et al., 2002), 2004 (Day et al., 2005), 2007 (Day et al., 2009) and 2010 (Bardwell et al., 2011). The participants are given only the two-dimensional structure of the target molecule and its crystallization conditions. The tests have grown from involving three `simple' target systems and 11 participants (Lommerse et al., 2000) to 15 research groups targeting large `drug-like' molecules, co-crystals and known polymorphic systems (Bardwell et al., 2011).

The blind tests of methods have shown great advances in the ability to generate and rank putative crystal structures (Day et al., 2009; Neumann et al., 2008) and in our understanding of cohesion in the solid state. In the most recent blind test all of the target molecules were predicted by at least two different methods (Bardwell et al., 2011). However, many challenges remain in making CSP a reliable and efficient tool. The computational cost of some methods limits their high-throughput use, and reliability and applicability for the full diversity of organic molecules and solid forms remains an open question.

Following dialogue with the CSP community, the CCDC has decided to host a sixth blind test of organic CSP methods. This test will provide the community with a fair benchmark of the state-of-the-art in CSP methodology. We hope that it will, once again, act as a platform for communicating ongoing progress and challenges in CSP and will spur the continued development of these methods. The invitation to participate is an open one.

This test will run from 1 September 2014 until 31 August 2015. We envisage five target categories, which may be any space group:

• (i) Rigid molecules, with functional groups restricted to CHNO, halogens, S, P and B; one molecule in the asymmetric unit; up to ∼ 30 atoms.

• (ii) Partially flexible molecules with two to four internal degrees of freedom; one molecule in the asymmetric unit; up to ∼ 40 atoms.

• (iii) Partially flexible molecules with one or two internal degrees of freedom as a salt; two charged components in the asymmetric unit, in any space group; up to ∼ 40 atoms.

• (iv) Multiple, partially flexible (one or two degrees of freedom) independent molecules as a co-crystal or solvate in any space group; up to ∼ 40 atoms.

• (v) Molecules with 4–8 internal degrees of freedom; no more than two molecules in the asymmetric unit, in any space group; 50–60 atoms.

We hope that these target systems will allow as many methods as possible to be applied, while also representing a significant and useful challenge. Participants are not required to attempt all five target systems. Obtaining suitable structures for the blind test is challenging and we would welcome any donations of structures that fit these categories from experimentalists.

In recent years it has been increasingly clear that CSP is capable of yielding a solid-form landscape of possible structures rather than solely `the' crystal structure, with various factors controlling what structures can be isolated and characterized experimentally (Price, 2013). Recognizing this, participants are invited to submit a list of up 100 structures, ranking these with their underlying method (e.g. lattice energy or some scoring function). Participants may also submit an additional list that combines the underlying approach with additional information, such as structural informatics and consideration of kinetic effects. We hope this will yield more insight into how the solid-form landscape of CSP can be related to experiment.

The outcomes of the blind test will appear in Acta Crystallographica Section B, in a special issue on Crystal Structure Prediction. A website will also host the results, predicted structures and a summary of each methodology. The predicted structures and other data will be directly citable with individual digital object identifiers (DOIs). A workshop will take place shortly after the results of the blind test are announced to allow discussion of the results and different methodologies.

The blind tests of organic CSP methods have been successful and useful to the scientific community due to the hard work and scientific endeavour of the numerous researchers that have participated in them and the support of experimentalists who have donated structures as target systems. We hope that the sixth blind test will be equally successful and useful. For further details, or to express interest in taking part in the sixth blind test, please contact Colin Groom ([e-mail]) or Anthony Reilly ([e-mail]). The blind test website can be found at https://www.ccdc.cam.ac.uk/Community/Initiatives/Pages/CSPBlindTests.aspx , and will be updated over time with details, progress and the results of the blind test.