Comments on A new theory for X-ray diffraction

Fewster [(2014), Acta Cryst. A70, 257–282] claimed that a new theory of X-ray diffraction is required, and that small crystallites will give rise to scattering at angles of exactly twice the Bragg angle, whatever their orientation. This article demonstrates that this theory is in error.


Introduction
Despite the field of X-ray diffraction being more than a century old, in an article entitled A new theory for X-ray diffraction , hereafter referred to as NTXRD, it is claimed that a new theory of diffraction is required to explain the intensities observed in powder diffraction and other diffraction geometries. Within NTXRD a theory of X-ray diffraction is proposed which predicts that 'the scattering from a crystal or crystallite is distributed throughout space [which] leads to the effect that enhanced scatter can be observed at the 'Bragg position' even if the 'Bragg condition' is not satisfied' and that 'the scatter from a single crystal or crystallite, in any fixed orientation, has the fascinating property of contributing simultaneously to many 'Bragg positions''. If this new approach were correct it would certainly have significant implications for the whole field of X-ray diffraction, and given the prominence afforded to this new theory (it featured on the front cover of the published volume), its veracity or otherwise deserves appropriate scrutiny. However, we show here that the analysis presented within NTXRD is incorrect, and that the underlying concepts upon which the theory is based are not new but were known to the earliest pioneers of X-ray diffraction.
At the outset we emphasize that in this article we will not ourselves be undertaking the task of proposing an explanation for the several interesting pieces of experimental data presented by Fewster, which are certainly worthy of further study and attention. Rather, our more restricted aim is to demonstrate that the new theory that he puts forward is incorrect, and we identify the sources of error in the arguments put forward in NTXRD. Secondly, whilst the interested reader would no doubt benefit from reading the NTXRD article in full, we present in the section below the key result of the theory within NTXRD which we deem to be erroneous. Thirdly, it is important to note that the new theory of X-ray diffraction that Fewster puts forward is based on a set of highly simplifying assumptions. These assumptions are the very same approximations made over a century ago by the doyens of the field. We adopt the same approach here: following Fewster we will be assuming that the crystal of interest is irradiated by a monochromatic plane wave with a transverse coherence length larger than the crystal, and that the diffraction observed in the far field is in the Fraunhofer limit: that is to say that the size of the illuminated crystal is w ( ðRÞ 1=2 , where R is the distance to the detector and the wavelength of the X-rays, such that the condition should be reasonably well obeyed for diffraction from crystals of the order of 1 mm in size when the detector is several tens of cm distant. Further, the kinematic approximation with zero absorption is also assumed, we treat the atoms as point scatterers, and neglect the effects of polarization and of finite temperature. Whilst it is well known that the assumptions made above can break down even for diffraction from small crystallites (Shabalin et al., 2017), for the sake of direct comparison we use the same assumptions as those made in NTXRD.

Fewster's theory
Consider the diffraction geometry shown in Fig. 1, adapted from Fig. 4(a) of NTXRD. Fewster derives the following formula [the square of the amplitude, A 2 , calculated in equation (5) of NTXRD] for the scattered intensity from a set of atoms, recorded by a detector placed at an angle 2 to a beam of monochromatic radiation of wavelength which is incident at an angle to the crystal plane: where L x is the length of the crystal, d is the plane spacing, n denotes the 'order' of planes from which the X-rays are diffracting and N is the number of planes in the stack contributing to the reflection. This formula predicts maxima in the scattered intensity whenever ¼ (i.e. a specular peak) and when ¼ arcsinðn=2dÞ (the Bragg peak), no matter what angle the crystal is placed at relative to the incident beam, and this prediction forms the fundamental basis of the new theory of diffraction described in NTXRD. However, equation (1) is incorrect and, as we shall show, the actual formula for the angle-dependent scattering, known since the earliest days of X-ray diffraction, leads to substantially different conclusions. We discuss the error in Fewster's analysis in x3, after first outlining the specific predictions of NTXRD.
In Figs. 2(a) and 3(a) we plot the intensity observed at 2, calculated from jA 2 ð1Þj 2 and j P 2 n¼0 A 2 ðnÞj 2 , respectively, where A 2 is defined as in equation (1), for a range of angles of incidence, ¼ f Â B . For this particular case we have set =d ¼ 0:5 and N ¼ 1000 [this ratio of wavelength to spacing is within 2% of that used by Fewster, who uses a value of 0.491 corresponding to the diffraction of Cu K radiation from the (111) planes of silicon, although specific lattices are not mentioned within NTXRD]. It can be seen in both figures that for all values of there is some enhanced scattering at a position corresponding to exactly that of the Bragg condition, along with a peak that corresponds to specular scattering (the two being identical for f ¼ 1).
Note that the inclusion of the planes n ¼ 0 and n ¼ 2 makes Fig. 2(a) identical to Fig. 5 of NTXRD. On the basis of this plot it is claimed within NTXRD that for a set of crystallites with random orientations the specular scattering associated with each crystal will occur at different scattering angles, thus producing a background intensity, whereas because each crystallite produces some scattering at exactly the Bragg condition, the intensities at the scattering angle 2 B from all of the crystals add, giving rise to a sharp peak. This result forms the basis of the work within NTXRD. However, we show below this analysis to be in error.

The error in Fewster's analysis
Fewster's analysis contains three errors -one minor and two major. Firstly, he states that the amplitude, A 1 , of X-rays diffracted from a single (the first) plane shown in Fig. 1 is given by A schematic diagram of the diffraction setup. Radiation is incident on a crystal plane at a variable angle , and the detector is placed at an angle of 2 with respect to the incident X-rays. denotes the rotation axis used in x7.

Figure 2
A comparison of the predictions of (a) equation (1), the NTXRD result for n ¼ 1, and (b) equation (21), our result for a cubic shaped crystallite with faces aligned to the cube axes, for radiation incident at an angle ¼ f Â B for a variety of f values. Both distributions exhibit specular reflections. Whilst the NTXRD result predicts a further peak at exactly 2 B , we find this second peak to vary in angle as described in the text.

Figure 3
A comparison of the predictions of the theory presented in (a) NTXRD and in (b) the theory represented by equation (20), including contributions from higher-order planes. In addition to the deviations previously noted in Fig. 2, the higher-order terms of equation (20) produce subsidiary maxima in (b) which are not present in (a).
atoms (assumed here to be point-like, i.e. ignoring the atomic form factor) separated by a distance a, the scattered amplitude from a single plane of atoms is This is only a minor error since, in the small-angle limit, equations (2) and (3) are in very close agreement, but diverge for larger angles (we discuss further the relationship between the use of sinc functions to describe the diffraction and the ratio of two sine functions in x4). The first of the major errors in Fewster's analysis is as follows. He states correctly that the phase difference for the scattering from successive planes, Á', is ð2=Þd½sin þ sin ð2 À Þ. However, he erroneously assumes that this phase difference can be approximated as ð2=Þð2dsin Þ. This is incorrect, and it is this approximation that leads to NTXRD always giving a peak in the scattered intensity at the Bragg condition. We discuss the origin of this approximation below. If one instead uses the correct phase difference, then summing the complex amplitudes over the N y planes yields which, on inserting the correct value of Á', yields where we have set d ¼ a (a simple cubic lattice) and which can be recognized as the two-dimensional form of the result obtained by Scherrer (1918), as outlined in the classic text by Warren (1969). A further error asserted within NTXRD is that the analysis presented holds true for all crystal shapes. This is incorrect. The results presented in this section only hold true for an orthorhombic crystal with the sides cut parallel to the unit-cell axes. We discuss diffraction from more general crystal shapes in x5.
At this juncture we discuss in more depth the origin of the specific error leading Fewster to assert that some peak in the scattered intensity always occurs at the Bragg condition, independent of . Whilst we find it somewhat difficult to follow the line of reasoning taken in NTXRD (as it appears to rely on only taking into account specific scattering points, rather than correctly summing all of the complex amplitudes from all scatterers), during the preparation of this article Paul Fewster drew our attention to one of his later articles within which he puts forward additional arguments as to why he maintains there is always enhanced scattering at the Bragg condition . However, whilst the additional argument within Fewster (2016) is also deeply flawed, it does give some further insight into the origin of the error. Consider the diagram shown in Fig. 4 [adapted from ], which shows the path length l ¼ a þ b between two points (where here a denotes the distance shown in Fig. 4, rather than the lattice spacing): the first, P, in the upper plane, and a point Q in the lower plane. As pointed out by Fewster, the difference in path length between the waves scattering from P and Q is given by The nub of the claim in Fewster (2016) is that it can be shown that for a fixed scattering point, P, the relevant number of scattering points, Q, in the next plane that scatter with a path length that differs from by Á (where Á is some fixed difference in path length that we choose such that Á ( ) maximizes at the Bragg condition ¼ B ¼ sin À1 ð=2dÞ independent of , and hence enhanced scattering will always be seen at the Bragg angle. This is illustrated in Fig. 3 of , which we shall in due course replicate below. We assume that this is why, in , he makes the small-angle approximation detailed above. However, we demonstrate below that the above claim is also false and identify the origin of the error.
Let us consider how we should calculate the effective length, Áx, along the lower plane that contains points that scatter in such a way so as to have a path length with respect to P that differs from by Á (as the relevant scattering amplitude will be proportional to this length). Let the coordinate of Q along its plane be x (such that the fixed x coordinate of P is 0). Then, the length along the lower plane that contains points that scatter with path lengths within Á of will be proportional to Áx, We state here the error that Fewster makes. He does not calculate the number density of points in the second plane as a function of the deviation in the path difference. Instead, he calculates the number density of path lengths that are within Á of as a function of , and then evaluates how many scattering points in the second plane are associated with each of the path lengths that fulfil this condition. To put it in simple mathematical terms, he effectively only considers the second The geometry for calculating the positions Q, in terms of incident angles and detector capture angles 2, used to construct the path lengths described by equation (6): l = a + b = ðd= cos Þ½sinð þ Þ + sinð2 À À Þ [adapted from ].
term appearing in the chain rule on the right-hand side of equation (7), i.e. he erroneously assumes That this is being assumed can be confirmed by examining the short Python code in the supporting information to , from which Fig. 3 in that article is produced, and from the statement within Fewster (2016) that 'we can decide on an acceptable path difference, Á ¼ ja þ b À nj and sum the number of values, for specific and 2 values, that have a path difference < Á'. Whilst this term does indeed peak close to the Bragg angle for all , it does not represent the required physical quantity which is correctly described by equation (7). When multiplied by the second term in the chain rule [i.e. equation (7) is evaluated], this effect vanishes, as would be expected. Let us now show this. From equation (6) differentiation of l with respect to yields However, to evaluate equation (8) we seek solutions where the Á is the deviation in path length from . Now, by rearrangement of equation (6) Substituting this solution for tan (with l ¼ ) from equation (10) into equation (9) we find d dl l¼ ¼ ½cosð2 À Þ À cos ½cosð2 À Þ À cos 2 þ ½sin þ sinð2 À Þ À =d 2 : We plot Áx 0 ð; Þ calculated from equations (8) and (11) in Fig. 5, using the same ratio of and d (=d ¼ 0:5) as used in . It can be seen that whilst equation (11) does not appear in , this plot is indeed identical in form to Fig. 3 in . We stress again that this does not represent the relevant number of points along the scattering plane that scatter with path lengths within Á of . That density is represented by equation (7), which we now evaluate.
Consider the first term in the chain rule of equation (7). As x ¼ d tan , then, without setting the constraint l ¼ , and substituting equations (9) and (12) into equation (7) we find We note that this is a function with no dependence on . Since in the derivation above we have not yet set the constraint l ¼ , a large value of Áx represents a turning point in the path difference of any value, l 0 . From equation (13), we see that Áx always maximizes upon the specular condition being met ( ¼ ) (or conversely we can say that the path length as a function of the position of Q minimizes at this condition), and thus for Áx to maximize under the constraint that l ¼ , then ¼ ¼ B , consistent with traditional diffraction theory. As expected, no scattering peak at B is seen at any other value of .

Calculation of the scattered intensity
In order to elucidate further errors described in NTXRD, in this section we note the well known result that equation (5) can, via the method of Poisson sums, be written in terms of the Fourier transform of the shape function of an orthorhombic shaped crystal (sinc functions) centred on the infinite reciprocal lattice [see equation (20) below]. By use of such shape functions we will, in x5, show results for diffraction from spherical crystals, which are also discussed erroneously in NTXRD. Furthermore, we will demonstrate that some enhancement close to, but not exactly at, the Bragg condition can arise from the conventional analysis of diffraction from certain planes of a restricted set of shapes of crystals, without the need to appeal to a new theory. By working in reciprocal space we illustrate the origin of these 'Bragg-like' peaks, as well as of the specularly diffracted radiation, and show that, contrary to the claims within NTXRD, these types of effects are well known.
Under the simplifying assumptions made in x1, the intensity of radiation scattering from a crystal of N atoms is given by where Ák is the difference between the wavevectors of the incident and scattered radiation, r j is the position of atom j and f j is the usual atomic form factor. In order to calculate the diffraction pattern from a finite crystal of a particular shape, we use the method of Poisson sums in three dimensions (Stein & Weiss, 1971), which gives, for a well behaved function g, P where Ã andÃ Ã are the direct and reciprocal lattices, respectively, G is a reciprocal-lattice vector andg g is the threedimensional Fourier transform of g. We consider first a crystal infinite in extent. By writing r j ¼ r Ã þ r j , where r j is the relative coordinate of the atom in the basis and r Ã is the position of the associated lattice point, equation (14) can be rewritten as sums over the lattice vectors r Ã and the basis B: At this stage, assuming that the left-hand side of the above equation extends over an infinite crystal, we can apply equation (15), giving the result that diffraction only occurs when the Bragg condition is satisfied: where FðÁkÞ is the geometric structure factor FðkÞ ¼ P j2B f j expðiÁk Á r j Þ. For a crystal that is finite in extent, the sum can be extended over the infinite lattice Ã by introducing a function, g S ðrÞ, to describe the shape of the crystal, such that g S ðrÞ ¼ 1 within the volume S enclosed by the surface of the crystallite, and 0 elsewhere. Equation (14) can then be written Thus, using equation (15) and the convolution theorem, Hereg g S is the three-dimensional Fourier transform of the shape function ('the shape transform'). In reciprocal space, equation (19) has a simple geometric interpretation: it is the convolution of the shape transformg g S with the reciprocal lattice.
For the purposes of this article, we will be dealing with a single-atom basis and we shall also assume point-like scattering, such that we may assume throughout that f j and F are independent of Ák. As with the initial analysis of NTXRD, we have also ignored the effects of absorption and extinction.
Consider a crystallite with a primitive cubic lattice of lattice spacing a. We assume that the shape of the crystallite is orthorhombic and that the normals to the faces of the cube lie along the principal axes of the cubic unit cell such that the size of the crystallite L x;y;z ¼ N x;y;z a. The reciprocal lattice is cubic, with reciprocal-lattice spacing 2=a, and this is convolved with the shape transform of the crystal such that equation (19) yields The equation above shows the link between the form used by Scherrer [equation (5), ratios of sine functions] and a set of sinc functions which are functions of ðÁk i À G i Þ, but then summed over all reciprocal-lattice vectors. The two forms produce identical results, but using the approach of equation (19) is more convenient for the present discussion, as it allows us readily to calculate the diffracted intensity for crystallites of arbitrary shape. A schematic plot of the distribution of intensity in the Ák x ; Ák y plane of reciprocal space given by equation (20) is shown in Fig. 6. We note that this figure is identical in form to Fig. 6-3(1) in the book edited by Ewald (1962).
For the sake of simplicity, consider now a crystal that is cubic in shape, such that L x ¼ L y ¼ L z . Fig. 6 shows the position Ák in reciprocal space corresponding to the scattering geometry of Fig. 1, where the cubic shaped crystal is set up for diffraction from the (010) planes, which also form a planar surface of the crystallite. Assuming that the number of planes is large, we can assume that for regions close to the Bragg condition G ¼ ð0; 2=a; 0Þ dominates in the sum in equation (20). From the geometrical construction in Fig. 6 we see that Ák x = ð2=Þ½cosð2 À Þ À cosðÞ, Ák y = ð2=Þ½sinð2 À Þ + sinðÞ, Ák z ¼ 0 so that equation (20)  A plot of the intensity in reciprocal space predicted by equation (20) for a crystal with a cubic lattice, and of cubic shape, with the orientations of the crystal faces along the principal axes as described in the text. The incident and scattered X-rays and Ewald sphere corresponding to the setup in Fig. 1 are also shown.
The intensity predicted by equation (21) is plotted in Fig.  2 We note that the first term in equation (21) is identical to that in equation (5) of NTXRD [our equation (1)] for the case n ¼ 1, and hence still gives rise to a specular peak when ¼ . However, we no longer find a peak at exactly the Bragg condition as the angle deviates from B . Nonetheless, we do find a peak at an angle which for small deviations from the Bragg angle, ¼ B þ , gives a peak in the intensity distribution when the detector is at an angle This 'pseudo-Bragg' peak is thus, for this particular case, a weak function of , but we note that in contrast to NTXRD we do not find a diffraction peak exactly at the Bragg angle as the crystal is rotated away from the Bragg condition.
Recalling the error in the derivation of equation (1) [equation (5) of NTXRD] as outlined in x3, we note that, comparing our result with that of NTXRD, it can be seen that if the approximation that ¼ is made in the second term (but not the first) of equation (21), then the NTXRD formula, equation (1), is recovered.
We can also see from Fig. 6 that, for values of significantly larger than B , the Ewald sphere would intersect arms of the shape function that are associated with being centred on reciprocal-lattice vectors with ðhklÞ different from (010). We thus plot in Fig. 3(b) the intensity predicted by the full formula of equation (20), accounting for all reciprocal-lattice vectors with h; k or l less than or equal to 2. As predicted, additional peaks seen around the detector positions of 2 ' 60 are visible, due to the Ewald sphere crossing the 'arms' of the shape transform lying between (110) and (120).

Geometrical interpretation and the general case
A consideration of the geometry of the shape transform shown in Fig. 6 enables us to see why we observe a specular peak in intensity for this particular cubic shaped crystal, why this crystal also provides a peak in scattered intensity at an angle close to (but not exactly at) the Bragg angle when it is oriented away from the Bragg condition, and why in the general case NTXRD is incorrect.
The specular peak can be explained as follows. For an orthorhombic shaped crystal, with the facets cut as described thus far, the sinc functions associated with the shape transform form 'arms' of intensity parallel to the k x ; k y ; k z axes in reciprocal space. If the reflection in which we are interested has a reciprocal-lattice vector which lies along one of these arms then the arms of the shape function form a chord on the Ewald sphere (for the crystal cut as described here, any reciprocallattice vector in the family {m00} will meet this criterion). As can be seen from Fig. 6 the length of this chord will change as we vary , but at a scattering angle 2 ¼ 2 such that the reflection is always specular -a point to which we will return later.
The constructions in reciprocal space also allow us to see why we obtain some, albeit weak, scattering at close to (but not exactly at) the Bragg angle as this particular crystal is rotated for scattering associated with this particular reciprocal-lattice vector. Consider Fig. 7(a), where we show the shape transform for our cubic crystal at the Bragg condition and slightly rotated away from it. As an arm of the shape transform at the Bragg condition is perpendicular to the reciprocal-lattice vector, the position where the Ewald sphere crosses the arm of the shape transform is such that the angle of 2 at such a point is initially a slowly varying function of the angle of rotation. A schematic diagram of the shape transform in reciprocal space for a cubic crystal rotated such that radiation reflects from (110). Unlike for the (010) reflection, this does not exhibit a persistent pseudo-Bragg or specular peak in the intensity distribution.

Figure 7
The maxima of the shape transforms for a cubic crystal (a) and a spherical crystal (b) are shown in schematic form for crystals rotated 6 from the Bragg condition for the (010) reflection. The k 0 vectors indicate the intersection of the Ewald sphere and the shape transform maxima. The spherical shape transform has been truncated for clarity to only show the first three maxima.
We thus predict that diffraction from the same crystallites as considered to date (i.e. comprising a primitive cubic lattice, and cubic in shape with facets along the principal axes), but now diffracting from the (110) planes, will not exhibit peaks at the specular condition, or close to the Bragg condition when deviates from B . This can be seen from a sketch of the geometry in Fig. 8, where we can see that the arms of the shape transform are rotated =4 with respect to the reciprocal-lattice vector.
This is indeed the case, and in Fig. 9(b) we show the results of the intensity predicted by equation (20) when diffracting from the (110) plane as a function of f as the crystal is rotated about the (001) axis. Once more we take =a ¼ 0:5, so that =d 110 ¼ 0:709 and N x ¼ N y ¼ 1000. There is no peak in the diffracted intensity at the specular position, and diffraction associated with the original Bragg peak falls rapidly in intensity as f differs from 1.
This lies in stark contrast to the NTXRD result for the same reflection, seen in Fig. 9(a), which shows no qualitative difference to Fig. 2(a) besides the shifting of the Bragg angle.
Finally in this section we consider diffraction from a spherical crystal. We do so as within NTXRD it is claimed that 'the introduction of various shapes creates a different distribution of fringing, but the enhancement at [the Bragg angle] is still present ' -i.e. there is always an enhancement exactly at the Bragg angle, and spherical crystals are explicitly considered within NTXRD. As the Fourier transform of a solid uniform sphere can be written in terms of the half-integer Bessel functions of the first kind, for a spherical crystal of radius R (where we assume R is large compared with the lattice spacing), equation (19) can be written Thus, as can be seen from the reciprocal-space plot shown in Fig. 7, the spherical crystal shows a completely different pattern to that present in the cubic crystal previously discussed. Unlike the cubic shape transform's distinct 'arms' which gave rise to the specular and slow-moving peaks,f f sphere exhibits 'ripples', which cross the Ewald sphere a large number of times, giving rise to an extremely large number of residual peaks around a central maximum, the exact number of which changes rapidly as a function of crystal rotation.
The intensity as a function of f for a spherical crystal is shown in Fig. 10 for =a ¼ 0:5 and R ¼ 500a. This figure is otherwise an exact replica of Fig. 2. Therefore, by the claims of NTXRD, we should see the same enhancement at the Bragg peak even when the Bragg condition is not satisfied, as well as specular reflections as observed in Fig. 2. However, no such features are observed, with only significant diffraction occurring at the Bragg condition, as expected.
We note that the fact that equation (24)  Intensity distribution contribution from the (110) reflection for the cubic crystal, using (a) the NTXRD method and (b) the method of equation (19). Note that while the two methods produced broadly similar results for the (010) reflection in Fig. 2, for this reflection they produce very different results.

Figure 10
Intensity distribution for radiation incident at an angle ¼ f Â B to the (010) plane of a spherical crystal, for a variety of values of f, as calculated from equation (24).

Size broadening
Thus, contrary to the claims made within NTXRD, crystals with different shapes do not have a persistent peak at the Bragg condition when differs from B . Indeed, the effects discussed thus far were already well understood in the earliest days of X-ray diffraction, and the widths of the Bragg peaks have been (within the approximations of this simple model) understood for of the order of a century. The Scherrer equation (Scherrer, 1918;Patterson, 1939) relates the peak width (full width at half-maximum, FWHM), Áð2Þ, to the crystallite dimension L for nano-scale particles (L < $ 0:2 mm): where K is the Scherrer constant, a function of crystal shape and which typically has a value of the order Oð1Þ. Fig. 11 shows a simulation of the variation of the FWHM of the central peak with the crystallite dimension L of a variety of crystal shapes [cuboid (N x 6 ¼ N y ), cubic (N x ¼ N y ) and spherical] calculated using equation (19). As a comparison, the region described by the Scherrer equation for 0:75 < K < 1:4 is also plotted, and it can be seen that all three of the simulations fall within this region.
More detailed analysis shows that each of these lines is accurately fitted by the Scherrer equation (within the nanocrystallite regime) with K values of 0.854, 0.898 and 1.156, respectively. Furthermore, that the finite size of crystals would give rise to diffraction away from the Bragg condition has also long been recognized (Bragg & Lipson, 1938).

Rotations about two axes
As well as calculating the diffracted intensity for crystallites rotated about an axis perpendicular to the plane containing the source and detector, results are also given within NTXRD for simultaneous rotations of the crystallites through angles about a second axis, perpendicular to the first -being parallel to the x axis and passing through the crystal, as shown in Fig. 1. We consider once more the cubic shaped crystal, initially set up for Bragg diffraction from (010). We calculate the intensity at any given scattering angle as a function of and from equation (20).
We consider diffraction in the Bragg-Brentano geometry (Bragg, 1921;Brentano, 1946), in which the detector is rotated along with the sample, such that ¼ . Once more we consider a cubic shaped crystal with a primitive cubic lattice and set =a ¼ 0:5. We show in Fig. 12 the intensity prediction as a function of and .
The (010) peak in the scattering occurs, as expected, at the Bragg condition, ð; Þ ¼ ð0:253; 0Þ, but we note that we can also observe a number of other Bragg peaks, with the (020) peak occurring at ð; Þ ¼ ð0:525; 0Þ, and finally the ð011Þ and ð011Þ at ð; Þ ¼ ð0:362; AE=4Þ, respectively. The scattered intensity that can be seen along ¼ 0 corresponds in form to the intensity as a function of 2 shown in Fig. 3 for n ¼ 1.
In addition to these features, we also see an arc in the intensity distribution, passing through the Bragg condition, such that for values of greater than that at the Bragg peak, for fixed two further peaks are seen at finite . These peaks are easily understood in terms of the shape transform in reciprocal space. As the shape transform associated with the reciprocal-lattice point is rotated about the x axis, the arms of the shape transform lying along k z intersect the Ewald sphere for ! B ; similar arcs can be observed elsewhere in the pattern, associated with the (020), (011) and (011) reflections, as expected from this model.
At this juncture it would be useful for the reader to refer to Fig. 7 of NTXRD, which we have reproduced in Fig. 13(a). Our Fig. 12 resembles the NTXRD figure in a remarkable fashion, with the only major difference being that the NTXRD graph contains only contributions from (010), and hence does not display the other Bragg peaks. Some care should be taken in comparing the two plots, as we believe that Fewster may be assuming diffraction from the (111)  The FWHM of the diffracted intensity for a variety of crystal sizes and shapes calculated using equation (19). Each of these is accurately fitted by the Scherrer equation with K in the expected region.

Figure 12
The intensity distribution for a two-axis rotation in the Bragg-Brentano geometry is shown for a cubic crystal of dimension 0.8 mm. Four Bragg peaks can be observed, as well as a series of arcs connecting them. point is that in NTXRD it is stated that Fig. 7 of that article is calculated for a fixed detector as and are varied. We disagree that the sort of behaviour we observe in our Fig.  13(a), and in Fig. 7 of NTXRD, can correspond to the fixed detector geometry; it should only arise for the Bragg-Brentano geometry used to produce Fig. 12. Indeed, we plot in Fig. 13(b) the prediction of equation (20) for a fixed detector (a= ¼ 0:5, N x ¼ N y ¼ 1000). As expected from any conventional diffraction theory, under such conditions we then find only significant diffraction at the Bragg position itself.
It should be noted that the reverse does not hold true: the NTXRD prediction for a Bragg-Brentano detector does not resemble either Fig. 12 or Fig. 13(b).

Conclusion
The effects that the finite size of crystals has on X-ray diffraction have been discussed and considered since soon after the foundation of the field. Within NTXRD mistakes are made in summing the phases of scattered X-rays from a crystal with an orthorhombic shape, which lead to the incorrect conclusion that such crystals always have some peak in scat-tering at the Bragg condition. It is also claimed that this result holds for crystals of a general shape. As we have shown, these conclusions are in error, and the effects that the shape and finite size of crystals have on the diffraction pattern are well described by conventional diffraction theory.
Whilst a study of finite crystallite size effects will no doubt continue to be of importance in relating experimental and computed diffraction profiles, and the experimental data presented in  and  are no doubt worthy of further study, the specific claim made within NTXRD that simple theory predicts a peak in the scattered intensity to occur exactly at the Bragg condition when small crystallites are rotated away from that condition is false. The intensity distribution for a two-axis rotation for a fixed detector is shown for a cubic crystal of dimension 0.8 mm using (a) the NTXRD formula and (b) equation (20).

Response to Fraser & Wark's comments on A new theory for X-ray diffraction
Paul F. Fewster* Brighton, UK. *Correspondence e-mail: paul.f.fewster@gmail.com The criticisms of my theory, as given by Fraser & Wark [(2018), Acta Cryst. A74, 447-456], are built on a misunderstanding of the concept and the methodology I have used. The assumption they have made rules out my description from which they conclude that my theory is proved to be wrong. They assume that I have misunderstood the diffraction associated with the shape of a crystal and my calculation is only relevant to a parallelepiped and even that I have got wrong. It only appears wrong to Fraser & Wark because the effect I predict has nothing to do with the crystal shape. The effect though can be measured as well as the crystal shape effects. This response describes my reasoning behind the theory, how it can be related to the Ewald sphere construction, and the build-up of the full diffraction pattern from all the scatterers in a stack of planes. It is the latter point that makes the Fraser & Wark analysis incomplete. The description given in this article describes my approach much more precisely with reference to the Ewald sphere construction. Several experiments are described that directly measure the predictions of the new theory, which are explained with reference to the Ewald sphere description. In its simplest terms the new theory can be considered as giving a thickness to the Ewald sphere surface, whereas in the conventional theory it has no thickness. Any thickness immediately informs us that the scattering from a peak at the Bragg angle does not have to be in the Bragg condition to be observed. I believe the conventional theory is a very good approximation, but as soon as it is tested with careful experiments it is shown to be incomplete. The new theory puts forward the idea that there is persistent intensity at the Bragg scattering angle outside the Bragg condition. This intensity is weak ($10 À5 ) but can be observed in careful laboratory experiments, despite being on the limit of observation, yet it has a profound impact on how we should interpret diffraction patterns.

Introduction
The new theory of X-ray diffraction arose from trying to account for inexplicable experimental observations. Neither the conventional dynamical nor kinematical theories could explain the measurements. The microstructure would have to be fantastical to account for some of these observations. Several experimental examples are included in this article that support the theoretical interpretation. My questioning of conventional theory started in the 1990s when using the nearperfect diffraction space probe (Fewster, 1989) to study polycrystalline materials and perfect semiconductors, with work on a different description beginning in the mid-2000s. It was clear that the observed features could no longer be dismissed as artefacts of the instrument, requiring an alternative explanation of experimental data.
This article is in five sections. The first relates the new theory to the Ewald sphere construction to give a better visual description, which is achieved by simply translating equation (5) of Fewster (2014) into graphical form. The second part describes the build-up of the scattering and where the inten-sity is concentrated, including the simple error/misunderstanding/assumption made by Fraser & Wark (2018). The third section gives some experimental evidence of the persistent intensity at the Bragg scattering angle when not in the Bragg condition. The fourth section considers the impact of crystal shape. The fifth section lists some of the examples that are difficult to explain using the conventional theory that are easily explained with the new theory.

The relationship of the new theory to the Ewald sphere
The whole basis of the new theory is that a strong scattering feature, e.g. a Bragg peak, can still be observed as the crystal is rotated away from its position on the Ewald sphere. This applies to all the diffraction features, e.g. thickness fringes and crystal truncation rods, but will be weak. The distance of a diffraction feature from this 'conventional' Ewald sphere surface is given by the length of the arc of a vector (for the feature of interest) rotated about 000 (Fig. 1). The length of the vectors in the figure corresponds to 1/d hkl . The arcs touch the Ewald sphere at 2 hkl with a residual amplitude given by equation (4) of . The next section explains why there is intensity at this position. Thus, a considerable proportion of the full diffraction pattern should be observed if there is sufficient intensity. This is exactly what we would expect from optical diffraction. Rotating the crystal just increases or decreases the intensity of the features in the diffraction pattern, e.g. Bragg peaks, thickness fringes, crystal truncation rods, fringes from spherical crystals etc., and when they coincide with the surface of the 'conventional' Ewald sphere the intensity for that feature reaches its maximum value. The 'conventional' Ewald sphere just represents the specular condition and has no width. The new theory just expresses that there is a residual specular contribution that does not go to zero as soon as the feature giving rise to it is rotated away from the optimum position on the sphere surface.
There is also a philosophical question here: if the Ewald sphere has no width then how can a reciprocal-lattice point interact with it? If the crystal is stationary, the source is monochromatic and there is no beam divergence, what would the intensity be? This was a serious problem for Wojtas et al. (2017) in their interpretation of XFEL (X-ray free-electron laser) data, requiring the partial capture of a reciprocal-lattice point and invoking angular tolerances to obtain some explanation of the data. If there were too many 'Bragg peaks' then they assumed that they were capturing data from more than one crystal and rejected the data. The new theory defines a width for the sphere surface and this dilemma does not exist. Because it has a width then intensity will be captured away from the Bragg condition. The new theory describes the thickness profile and the associated residual amplitude that is captured.
So, what evidence is there for this? Well there is plenty of evidence, from calculating the diffraction pattern from first principles, results from XFEL sources and even data collected from standard laboratory sources. Let us start with the calculated evidence from my colleague John Anderson and presented by Fewster (2017). This considers a singlewavelength plane wave impinging on a three-dimensional array of point scatterers, which will form a spherical wave from each point. When the scattering is brought together in the far field, i.e. the waves travelling in a parallel scattered direction are brought together, a diffraction pattern is formed. The phases of the contributions depend on the difference in path lengths of all the contributions at each 2 value. The first thing to notice is that the full diffraction pattern exists (Fig. 2a). That is not predicted in conventional theory where intensity from a feature only occurs when it touches the surface of the Ewald sphere. This figure is plotted on a logarithmic scale to reveal the detail. For a real experiment the data will have a finite dynamic range and only the strong features are likely to be observed (Fig. 2b). These simulations reveal the fringing due to the crystal surface boundary conditions (the shape transform) and if a fringe is close to the Ewald sphere then it could be more intense than the associated Bragg peak that is more remote, e.g. Fewster (2016)  The new theory in terms of the Ewald sphere construction. All the reciprocal-lattice points coloured blue can form intensity at this incident angle at their respective 2 B values (e.g. green dots) if 0 < < 2 B . The distance of the reciprocal-lattice point to the surface of the Ewald sphere along an arc in defines its amplitude, which decreases as the distance increases. For example, 0 1 13 is in the Bragg condition and the amplitude is at its maximum value, whereas 002 is weaker and 0 2 23 is very weak etc. The arcs drawn for some of the reflections give a guide to the strength of the scattering. The Ewald sphere surface can be considered to have a thickness with a profile given by equation (4)  observe intensity enhancement at these angles for this single incident angle. It must be recognized though that there will be peak movements resulting from the interference of the amplitude oscillations related to shape effects and those related to the enhancement effect as is varied. This will also be influenced by how close their contributions are to the surface of the Ewald sphere. The overlap of fringes from reflections of different order will also influence the observed diffraction pattern, which is particularly relevant for small, perfect crystals (Holý & Fewster, 2008;Fewster, 2015Fewster, , 2018. We can separate out the shape effects by extending the familiar description of Bragg's law.

The explanation of the persistent peak at 2h B and response to the Fraser & Wark analysis
A series of diagrams (Fig. 3) is given that explains the thinking behind the new theory and the reasoning of Fraser & Wark to make it clear where their misunderstanding has occurred.
A point P 0 on the upper plane will be in phase with any point in any position on the lower plane Q when in the Bragg condition, which in turn will also be in phase with all other points on the upper plane (Fig. 3a). When the planes are rotated away from the Bragg condition, the point P 0 will have a close phase relationship with several points on the lower plane, Q 01 , Q 02 , Q 03 , Q 04 etc., and we would expect to see some residual intensity at the specular angle (Fig. 3b). The point P 0 can never be exactly in phase with a Q 0 point for this combination of and 2 outside the Bragg condition (i.e. = B ). The Fraser & Wark analysis to this point would be the same; then they consider this angular spread of acceptable phases combined with the density of scattering points on the lower plane to give rise to an intensity. I have no dispute with this.
If we now include another point on the top plane, which we call P 1 (Fig. 3c), then there will be another set of points on the lower plane that have the same relationship as for P 0 . We shall call these points Q 11 , Q 12 , Q 13 , Q 14 etc. These scattering points on the lower plane Q 1n will have some overlap with the points Q 0n . Since there are as many scattering points on the P and Q planes we should pair every P point with a Q point, and the conclusion would be the same as before if all the P points are in phase (Fig. 3c). This arrangement of scattering points produces a peak of intensity at the specular scattering angle that we can call 2 s . This scattering angle is defined by the crystal surface where the scattered wave exits the crystal and is a result of the boundary condition, which requires the component of the electric field parallel to the surface of the crystal to be continuous. This explains the fringing associated with the crystal shape, often termed the shape transform. If the incident angle is not equal to the Bragg angle, then 2 s can never equal 2 B . This is the conclusion in Fraser & Wark that I agree with; it is purely a conclusion of the conventional theory.
What happens if the detector is moved to a different 2 angle, whilst maintaining the same incident angle? The description of Fraser & Wark or the conventional theory does not consider this. The scattering does not correspond to the specular condition (Fig. 3d) and P 0 is no longer in phase with P 1 and similarly the phase relationship between the scattering from the points P and Q has changed. Conventional theory and that of Fraser & Wark simply assume that intensity only exists when the points P are perfectly in phase. But what happens if the points P scatter slightly out of phase? Is it realistic to assume that there is no intensity in this case? This is a major anomaly in the conventional theory and can be interpreted as the Ewald sphere surface having no thickness.
If we postulate that the points P 0 and P 1 can scatter in a less than perfect phase alignment, then we must conclude that there is intensity outside the specular condition. This has nothing to do with crystal shape. If the detector is moved further the phase relationships between all the P points and all the Q points will change again. Because the phase relationship between all P points can be determined and every P to every Q can be determined, the PQ pair can be paired in an arbitrary way. It is convenient to find the PQ pair that forms a path length difference closest to one wavelength. The phase The simulation of the diffraction pattern from a three-dimensional array of point scatterers with dimensions 40 Â 39 Â 40 nm with point separations of 2 Â 3 Â 4 nm using a wavelength of 1.54 nm. The whole pattern is revealed in a logarithmic plot (a). When plotted on a linear scale (b) there are six 'peaks' observed. This is very characteristic of data from XFELs. Diffraction based on the conventional theory would reveal nothing in this arbitrary orientation (these are not in the Bragg condition). The central peak in (a) is the direct beam and is removed from the linear plot in (b), to reveal the other peaks with linear scaling. The plots are displayed on a radius of 2 out to a maximum of 90 . The peaks can be indexed based on their 2 B values and the restriction 0 < < 2 B , yet their intensities vary significantly indicating that the reciprocal-lattice points cannot all be close to their Bragg conditions. It can be seen in (b) on a linear scale that peak intensities <$1% of the most intense peak are not observed. difference between P 0 , P 1 , P 2 etc. is determined purely by the incident angle to their plane and the detection point 2 [equation (4), ], which defines the maximum amplitude possible from the P plane for this at 2, i.e. A . A applies to the second and all subsequent planes and the maximum amplitude that can exist for this incident angle occurs when all planes scatter in phase with each other, i.e. NA where N is the number of planes. This will only occur if there are PQ pairings that have a path length of one wavelength. By taking a point P on the upper plane and an incident angle , we search for a pairing with a Q position that will give a path length difference of one wavelength by allowing 2 to take on any value. Fig. 4 is a plot of the angle combinations and 2 where a one-wavelength path difference can exist between a P position and a Q position. For any given incident angle there is a one-wavelength path difference possible at 2 B . We can consider that an incident angle below the Bragg angle will form a specular peak at 2 s with a maximum path length difference < and by increasing 2 the path length difference can be increased. Similarly, for an incident angle above the Bragg angle a specular peak will form at 2 s with a minimum path length difference > and by reducing 2 the  When the scattering planes are rotated away from the Bragg angle a point P 0 cannot scatter in phase with any point Q at the specular scattering angle 2 s . (c) For the same incident angle and the same specular scattering angle the near-phase relationship holds across the plane for P 0 , P 1 etc. (d) However, if we move the detector to a different 2, P 0 and P 1 no longer scatter perfectly in phase and similarly the phase relationship associated with P and Q points will change. The phase relationship between the scattering from P and Q points can therefore be varied by moving the detector. If there is a detector position where the path length difference is then all the planes will scatter in phase, with a maximum value defined by the phase sum of the amplitudes of points P 0 , P 1 etc.

Figure 4
The distribution of path lengths equal to one wavelength (to within a very small tolerance) from scattering points on adjacent planes. As the tolerance is reduced it concentrates on a single value at 2 B and the other coincidences become sparser. path length difference can be decreased. In both cases we can achieve a path length of to form an amplitude of NA .
This same analysis can be performed for any part of the truncation rod; however, the path length difference never reaches one wavelength but would be associated with a path length above or below this value. The conclusion is that the diffraction pattern is rich with information as in Fig. 2(a). This approach ensures that all scattering centres across these planes and by extension all planes in the stack are included.
The new theory therefore predicts that a scan in 2 over a large range at a fixed incident angle would encounter a peak at 2 s corresponding to the specular condition (e.g. crystal truncation rod) and at 2 B (the enhancement or persistent peak). This is exactly what was observed by  and further clearer examples are given in the following section, including the measurement of the predicted arc in Fig. 1 [example (iv) in x4].

Experimental evidence from laboratory sources
(i) The first example was an early test of my theory. The sample is a large, perfect crystal wafer of 111-oriented silicon. The incident beam (Cu K) is collimated to give an angular divergence of 0.03 and the crystal is set to several incident angles, , either side of the 111 Bragg angle ( B ). The scattering is captured by scanning in 2 (Fig. 5a). Peaks are observed that correspond to the intersection of the crystal truncation rod at 2 = 2 and further peaks at 2 = 2 B for both the Cu K and Cu K wavelengths for the d 111 crystal planes. The 2 = 2 B peaks are observed for incident angles up to 6 away from the Bragg condition.
(a) How can a crystal set at an incident angle remote from the Bragg condition produce a peak at 2 B ? (b) How can two 2 B peaks associated with different wavelengths that require different incident angles be observed simultaneously? (a) Several 2 scans for fixed settings with the interpretation in (b) based on the modified Ewald sphere construction. The spheres have different radii: 1/ and 1/ , centred on p and o, respectively. Consider the 2 scan for = 12.5 in (a) (the crystal is orientated 1.7 from the Bragg angle B for the Cu K wavelength). There is a single specular peak (the intersection of the 2 scan and the truncation rod) that is described in (b), where the specular contributions occur at the same 2 but capture different positions on the truncation rod at a and b, which is the same for both conventional and new theories. The two peaks, c and d, correspond to the d 111 plane spacing for both the Cu K and Cu K wavelengths, i.e. 2 and 2 ; in the conventional description these should not exist. The peaks at c and d can only be described with the new theory, i.e. the persistent intensity at 2 and 2 . The 2 peak can be observed up to | À B | $ 6 . The specular peaks are sharp (they are dominated by the proportion of the incident-beam divergence that satisfies this condition, i.e. a small region on the sample), and the enhancement peaks are broad (because all the incident-beam divergence directions will form intensity at 2 B and these exist over the full footprint of the beam on the sample. As the Bragg condition is approached the peak will sharpen because the strongest contributions come from a smaller range of divergence and smaller regions on the sample and dominate). The features at the base of the specular peaks are tube focus artefacts.   (2014)]: (a) the scattering pattern from $120 crystals (or if perfectly packed 300 crystals) isolated with a 3.5 mm incident beam that perpendicularly intersects a 1 mm-wide single layer of crystals of LaB 6 with sizes of 2 to 5 mm. (b) gives the profile with $30 crystallites or if perfectly packed 75 crystallites (3.5 mm Â 0.25 mm sample size), where only three reflections are clearly resolved compared with all ten in the larger sample size. The data were collected with a 0.01 divergent Cu K 1 beam from a 1.8 kW X-ray laboratory source in 35 min. The samples were stationary throughout, so the incident beam only explored one orientation from each crystal. The peaks are narrow and occur at the correct 2 B positions and correspond to the interpretation where each crystal contributes intensity as in Fig. 1.
The explanation based on the new theory is given in Fig.  5(b), and because of the large dimension parallel to the surface the shape function is dominated by the crystal truncation rod. The residual peaks at 2 B follow the prediction of equation (4), .
(ii) A very highly collimated monochromatic beam 3.5 mm wide (horizontal with a divergence of 0.01 ) 2 is incident on a 1 mm-wide (vertical) polycrystalline sample to form a cross section of 0.0035 mm 2 that is one crystal thick. The average crystal is 3.5 mm in diameter; this illuminated area and absorption measurements (to estimate the packing density) suggest there are $120 crystals being illuminated. The sample is kept stationary and the scattering is captured on a positionsensitive detector (the angular spread normal to the scattering plane is limited to 2.3 with a Soller slit). All ten possible peaks at their correct 2 B are observed and are sharp (Fig. 6a). The probability of capturing one crystal in the Bragg condition is 1 in 23 000, and therefore to capture all ten is 1 in 4 Â 10 43 .
(a) How, when the probability of observing a peak at the Bragg condition is 1 in 100 000, can a repeat experiment with $30 crystals form three clear peaks (Fig. 6b)?
(b) Is it reasonable to expect each crystal to be composed of $100 000 mosaic blocks?
(c) If there are 100 000 mosaic blocks in each crystal, they would have an average diameter of $0.075 mm. How can the average intrinsic width for these mosaic blocks ($0.11 ) be reconciled with the measured width of 0.026 ? 3 The new theory has a simple explanation by building all the weak contributions from each crystal as in Fig. 1. (iii) This is an example of the data from the beam selection diffractometer (Fewster, 2004). This instrument creates very high intensity, near-'zero' wavelength dispersion and well understood instrumental artefacts. The scattering from the sample 004 reflection is captured with a single reflection 004 analyser crystal (Fig. 7a). The sample is a perfect crystal. The combination of the analyser crystal and a slit to control the wavelength dispersion still shows the remnants of the Cu (a) A diffraction space map close to the 004 reflection (logarithmic scale) from an InGaAs structure grown epitaxially on a GaAs substrate. The data were collected with the beam selection diffractometer (Fewster, 2004), with a single reflection 004 analyser crystal (stepping in followed by a scan with movements in and 2 maintaining a 1:2 ratio). The strong fringing is associated with the layer structure (the shape transform) and occurs along the crystal surface normal. The streak where 2 = 2 corresponds to the incident-beam divergence and the streak along 2 for a constant value corresponds to the detector acceptance range (in this case the diffraction profile of the analyser crystal). The remaining streak at constant 2 B for varying values is the 'enhancement' peak for the substrate (as in Fig. 5b). (b) This is the extracted profile along the 2 B enhancement that is smoothly decreasing from the peak as expected, apart from interference of the Cu K 2 streak on the high-angle side. If all the artefacts could be removed and the alignment improved, this could be considered as the thickness profile of the Ewald sphere surface for this reflection and crystal.

Figure 8
The complex scattering (logarithmic scale) close to the 113 reflection from a Si (001) wafer, with a 46 nm epitaxial layer of Si 0.21 Ge 0.79 on top, obtained with a high-resolution diffractometer, courtesy of A. Kharchenko and J. Woitok. The fringing relates to the thickness of the SiGe layer, which can all be explained by conventional (dynamical) theory. The various features determined by the instrument and diffraction geometry are given in the figure and can be related to those in Fig. 7(a). The streak of intensity at constant 2 B cannot be explained with conventional theory but is predicted by the new theory and corresponds to an arc in Fig. 1. are the influences of the incident-beam divergence and the detector acceptance, which are clearly revealed as streaks emanating from the intense substrate peak. In addition, there is a prominent streak at constant 2 B . The crystal plane rotation is not accurately normal to the reciprocal-lattice mesh, so this streak is inclined to the plane of the diffractometer.
(a) What is the explanation for the streak of intensity at constant 2 B as the crystal is rotated in ?
The new theory predicts this 2 B streak, its shape and how it changes with crystal alignment. Fig. 7(b) gives an indication of the intensity along the 2 B streak for this sample, i.e. 10 À5 to 10 À6 of the Bragg peak at an angle of 0.15 from the Bragg condition.
(iv) This example uses a high-resolution monochromator and a position-sensitive detector to study a (001)-oriented Si wafer that has a single Si 0.21 Ge 0.79 46 nm layer grown epitaxially on top. The data were collected close to the 113 reflection by stepping in and scanning in 2, and plotted in reciprocal-space coordinates forming an arc of captured data (Fig. 8). The SiGe layer is tilted with respect to the substrate, 4 giving a tilted truncation rod (their individual crystal truncation rods are not coincident but still interfere with each other). The substrate gives rise to the most intense peak and the layer gives a broad peak with fringes. The influence of the incidentbeam divergence and the 2 capture line for a fixed incident angle can all be explained within the description of conventional theory. The substrate is perfect device-grade Si and is not mosaic. There is a very prominent arc of intensity at constant 2 B which corresponds exactly to the substrate d 113 plane spacing. This is the persistent intensity or 'enhancement' predicted by the new theory.
(a) Is there any explanation within the confines of conventional theory that can explain this arc of intensity at constant 2 B from a perfect crystal as it is rotated in ?
This arc of intensity follows the description in Fig. 1 (and discussed later in Fig. 10). It cannot be described by any shape function.
(v) This example is taken from a careful experiment on a structure composed of two epitaxial layers of GaAs/InGaAs on a GaAs substrate. The structure appears to be perfect until it is studied in greater detail with a very high resolution diffractometer (Fewster, 1989) (Fig. 9). There are two significant features that are observed: a crystal truncation rod that 'wiggles' 5 and an intensity streak along 2 B associated with the substrate. These features are a common observation in well aligned, good quality crystals, for layer structures and blank crystal wafers. The fringes associated with the layers indicate that the interfaces are flat and parallel. There is interference between the crystal truncation rod for the substrate and the layers, which is only possible if there is significant overlap. The intensity spreading at constant 2 B for each part of the structure would account for this overlap and the wiggles.
(a) How can the truncation rods of the substrate and layers interfere without some overlap to create these 'wiggles'?
(b) What is the reason for the 2 B streak that also gives rise to a broadened base of the substrate peak in an open detector rocking curve?
The new theory predicts the existence of the streak in 2 B , which in turn will account for the interference of the crystal truncation rods to explain the 'wiggles'. It also indicates how a full two-dimensional diffraction space map can be simulated.

The impact of crystal shape
The crystal shape will modify the intensity close to the Bragg peak, which was recognized by Fewster (2014) p. 262: 'Hence a powder sample that has a distribution of orientations will create fringes associated with its size and surface shape and an enhancement at 2 B for each crystallite plane'. The main thrust of this theory is to concentrate on the persistent intensity at 2 B , whereas all shape effects will modify the intensity around the Bragg condition peak and will not form intensity at 2 B unless by chance. Equation (5) in  can be considered as the formula for a crystal wafer with crystal planes parallel to the surface. For other crystal shapes, the full shape transform can be included, but the position of the Bragg condition is unchanged. To include the shape transform for a parallelepiped, as in the work of James (1962) and Authier (2001), for a small crystal, would involve extra terms in equation (5), i.e. of the form sinðNxÞ= sin x and sinðNyÞ= sin y. Since so few crystals conform to this shape I refer to my original statement above, i.e. any shape can be included but the persistent intensity at 2 B still exists. The 004 diffraction space map (logarithmic scale) expanded normal to the crystal truncation rod to emphasize the wavy streak of the 80 Å In 0.15 Ga 0.85 As quantum well, buried in a complex AlGaAs/GaAs structure. The other dominant feature is the streak along 2 B . When the data were projected along 2 B , the resultant profile fitted precisely with the simulation based on dynamical theory. Collecting data with a high-resolution diffractometer without an analyser (a rocking curve) gave small fringe displacements with a broadened base to the substrate peak (a commonly observed feature, which can be associated with the 2 B enhancement for varying ), whereas a single scan along the crystal truncation rod gave regions of missing intensity.
To explain the diffraction in the new theory compared with the conventional theory for a parallelepiped, consider Fig. 10 (shape function A), where its shape transform has been simplified to a cross with the tails diminishing in magnitude further from the reciprocal-lattice point. The conventional theory will reveal intensity where the shape transform intersects the Ewald sphere surface, resulting in two peaks. In the new theory the Ewald sphere surface has a thickness given by equation (4) of . This results in intensity associated with all parts of the shape function and much of it will be very weak. The two peaks as in the conventional theory may well be the most dominant features; however, a strong feature like the maximum in the shape transform will also produce a peak, which may or may not be observed depending on the measurement conditions as in the examples above.
The example given in Fig. 8 has a shape transform like B in Fig. 10 and interacts with a different position on the Ewald sphere surface. The arc of intensity measured corresponds precisely to the prediction in the new theory. More details are given in the caption.
In the new theory, a very small crystal will have a very broad thickness profile for the surface of the Ewald sphere. This increases the observed intensity of features remote from the optimum position on the Ewald sphere surface; so, although the fringing could be touching the optimum position, the main peak in the shape transform can still dominate. This is exactly what is observed in the simulation from a perfect parallelepiped crystal in Fig. 2.

The difficulties with 'conventional theory'
Requiring crystals to be mosaic to suppress dynamical effects (Darwin, 1922) for the kinematical approximation to be applied in structure determination puts a big onus on all crystals. Is that reasonable? The number of crystals required to form a reliable polycrystalline diffraction pattern is greater than in a typical sample, in which case microdiffraction will not work; but it does, so what is going on? This did not go unnoticed by Alexander et al. (1948) who suggested crystals in a powder diffraction sample must be mosaic; but how small are they? De Wolff (1958) suggested that slack gearing in diffractometers may be the cause, but high-quality diffractometers of today would rule that out. Smith (1999) concluded that the data cannot be reliable even with the numbers of crystals used in Bragg-Brentano geometry. More recently, the data from XFELs show that there are reflections simultaneously observed in a snapshot from a single crystal, which should be a very rare event but is very common. This has led to a plethora of complex explanations to account for the data, e.g. Wojtas et al. (2017).
Each explanation is specific to the method by stretching the limits of conventional theory, which is in danger of becoming inconsistent with itself. The descriptions presented by the early workers in this field were valid explanations for their time, but perhaps they cannot be universally applied today. Suppose the fundamental theory is not the complete answer, then the results could be unreliable. Kuhn (2012) would view the conventional theory as a powerful paradigm needing a crisis, e.g. inexplicable results, to change it. Have we reached that stage yet? Or can the conventional theory still reveal reliable results? Popper (2002) suggested all theories are waiting to be disproved and therefore should be falsifiable. The assumptions in conventional theory have increased to accommodate these diverse experiments to prevent falsification. This situation is not favoured by the law of parsimony (Occam's razor), which would prefer the theory with the fewest assumptions, because it is easier to falsify.

Conclusions
The new theory explains the experimental results. There is, as far as I know, no alternative explanation within the confines of conventional theory. Those who can understand my description as well as the conventional theory should be able to compare these two approaches and make a judgement on which best describes their data. The new theory could be considered as defining a thickness profile for the Ewald sphere surface. In conventional theory this surface has no thickness, placing all the experimental interpretation on changing the shape of the reciprocal-lattice point, e.g. mosaic crystals. Shape effects cannot explain the results described above and therefore the conventional theory can only be an approximation. I consider my theory to be a better description of The interaction of different shape functions with the Ewald sphere. A gives rise to peaks a c1 and a c2 where the tails touch the Ewald sphere; this is the interpretation based on the conventional theory. In the new theory there is another term [equation (4), ], so that three peaks appear a n1 , a n2 and a n3 (a n3 is the enhancement peak) and there is also residual intensity associated with the whole of the shape function. The shape function given at B corresponds to the sample used in Fig. 8, i.e. for a crystal wafer with a truncation rod normal to the surface with a very short arm parallel to the surface. At this orientation the conventional theory predicts no peaks since no part of the shape function touches the Ewald sphere. The new theory predicts a peak at 2 B (b n1 ) for all orientations in . The reciprocal-space map B can be compared with the measured data from Fig. 8 (inset) to show how a single extracted 2 scan away from the Bragg condition forms enhanced intensity at 2 B . X-ray diffraction. The criticisms of my theory by Fraser & Wark are therefore based on an invalid argument.