Concepts and Tools for

NMR Restraint Analysis

and Validation
SANDER B. NABUURS,1 CHRIS A.E.M. SPRONK,1 GERT VRIEND,1 GEERTEN W. VUISTER2
1
Center for Molecular and Biomolecular Informatics, University of Nijmegen, Toernooiveld 1,
6525 ED Nijmegen, The Netherlands
2
Department of Biophysical Chemistry, University of Nijmegen, Toernooiveld 1, 6525 ED Nijmegen, The Netherlands

ABSTRACT: The quality of NMR-derived biomolecular structure models can be assessed

by validation on the level of structural characteristics as well as the NMR data used to derive
the structure models. Here, an overview is given of the common methods to validate
experimental NMR data. These methods provide measures of quality and goodness of t of
the structure to the data. A detailed discussion is given of newly developed methods to
assess the information contained in experimental NMR restraints, which provide powerful
tools for validation and error analysis in NMR structure determination. 2004 Wiley
Periodicals, Inc. Concepts Magn Reson Part A 22A: 90 105, 2004

KEY WORDS: structure validation; experimental restraints; restraint validation; structure

renement

INTRODUCTION the spectroscopic data directly, geometric confor-

The result of a biomolecular structure determina-
tion by solution nuclear magnetic resonance (NMR)
spectroscopy is typically a family of structural
models describing the accessible molecular confor-
mations. This family, or ensemble, of structure
models should agree as a whole with the experi-
mental NMR data used in the procedure, as well as
other additional data. Typically, rather than using
Received 2 March 2004; revised 13 April 2004; ac-
cepted 13 April 2004
Correspondence to: Geerten W. Vuister; E-mail:
vuister@nmr.kun.nl
Concepts in Magnetic Resonance Part A, Vol. 22A(2) 90 105 (2004)
Published online in Wiley InterScience (www.interscience.wiley.
com). DOI 10.1002/cmr.a.20016
2004 Wiley Periodicals, Inc.
90
mational restraints are derived from these data,
which are subsequently used to calculate the struc-
tures (1). Derivation of such structural restraints
from NMR spectra is complicated because spectral
overlap, spin diffusion, local dynamics, and inter-
converting conformations have to be taken into
account. The traditional manual assignment of
NMR resonances and conversion of NMR peaks
into structural restraints is an extremely time-con-
suming process, even for experienced spectrosco-
pists. Further, manual interpretation of NMR data is
prone to human error and, possibly, manipulation.
These problems are being alleviated by the recent
development of several automated methods (for
detailed discussions see [2, 3]), which have in-
creased the speed, reliability, and reproducibility of
the interpretation and analysis of NMR data and the
subsequent structure calculation process.

It is of great importance that the structural re-
straints are subjected to thorough validation. First, in
the process of optimization and interpretation of the
structure model, a good assessment of the reliability
of the data and structures has to be made before
publication and deposition in the BioMagResBank
(BMRB) (4) and the Protein Data Bank (PDB) (5, 6).
Second, users of structures from the PDB have at
present access only to the derived structural restraints
and not the original spectroscopic data. Thus, a good
assessment of the reliability of the complete structure
determination procedure is difcult after deposition of
the structure models and data and therefore relies on
proper procedures by the submitting authors.
In this review, we discuss various common and
new methods for validation of NMR-derived struc-
tural restraints. An overview will be given of the
different types of restraints that can be derived from
NMR data, their application in structure calculations,
and the concepts and tools that are available for their
validation.
DATASETS
Throughout this review we use experimental datasets
as deposited in the PDB to demonstrate applications
of the different restraint analysis methods discussed.
To calculate trends or averages for multiple datasets,
we use the experimental data of a set of 100 proteins,
ranging in size between 20 and 370 amino acids,
which we analyzed and validated previously (7). As
an individual example, we use the NMR dataset of a
recently determined high-resolution structure of the
B1 immunoglobulin binding domain of protein G (8)
(GB1), as shown in Fig. 1.
RESTRAINT TYPES
In NMR structure determination, three major classes
of structural restraints can be distinguished: distance
restraints, torsion angle restraints, and orientational
restraints. Additionally, other sources of information
have been used in the renement of the derived struc-
tures, such as information from chemical shifts (9)
and high-resolution structure databases (10).
Distance Restraints
NOE-Derived Distance Restraints. Traditionally,
nuclear Overhauser effect (NOE)-derived distance re-
straints are the most important source of structural
information in the structure determination of biomol-
NMR RESTRAINT VALIDATION 91
ecules using NMR spectroscopy (11). NOEs provide
essential information to dene secondary and tertiary
structure as they connect pairs of atoms that are in
close proximity in Cartesian space but potentially far
apart in sequence space. Peaks observed in NOESY
spectra (1214) are translated into distance restraints
using the volume of the cross-peak (V), which is
related to the distance d ij between the two interacting
atoms i and j by
V d 6
ij f c [1]
The transfer of magnetization depends on both the
variation in d ij , as expressed in the averaging of this
term, and effects of internal- and global motions, as
accounted for in the function f( c ). Although average
distances are sometimes calculated directly from the
NOE peak volumes, it is customary to classify peak
volumes into weak, medium, and strong signals (15),
where each class corresponds to a set of approximate
distance bounds that should reect the uncertainties in
the derived distances (1). These uncertainties may
arise from effects such as spin diffusion and local
dynamics and from problems such as errors in peak
integration or spectral overlap.
Apart from taking the various origins of uncertain-
ties in NOE peak volumes into account, possible
multiple contributions to the peak volumes have to be
incorporated in the treatment of the distance re-
straints. These multiple contributions may arise from
true ambiguities due to spectral overlap of resonances
from different (groups of) atom(s) or from degeneracy
of resonances within a group of covalently con-
strained atoms. An example of the rst would be the
two methyl groups of two different alanines that over-
lap in the spectra and of which the resonances show
an NOE with a third (group of) atom(s). An example
of the latter type of degeneracy would be the three
protons within a methyl group that show a single NOE
cross peak, with three contributions, to another pro-
ton.
Various different ways have been used in the past
to take into account spectral degeneracy of, e.g., the
protons in methyl groups or aromatic rings within a
residue. Among these is the use of pseudoatoms in
combination with corrections on the distance bounds
(denoted in XPLOR/CNS terminology as center av-
eraging) or methods that sum up the individual con-
tributions (denoted in XPLOR/CNS terminology as
r6 sum averaging) (16).
True ambiguities in the assignment of NOE cross-
peaks can often only be resolved on the basis of a
structural model, which is usually not available at this
92 NABUURS ET AL.

stage of spectral assignment. To address this problem,

the concept of ambiguous distance data was intro-
duced (17). An ambiguous distance restraint (ADR) is
described in terms of the distances between all pairs
of protons that may be involved, as shown in Eq. [2].

1/6
NF 1 ,F 2

dF 1,F 2 d6
k [2]
k1

where k runs through all N(F1, F2) contributions to

a crosspeak at frequencies F1 and F2, and dk is the
distance between the two protons corresponding to
the contribution k calculated from a model struc-
using an appropriate target
ture. By restraining d,
function, to the distance bounds derived from the
volume of the NOE crosspeak, ambiguous restraints
are now commonly included in NMR structure cal-
culations (18 ).
Hydrogen Bond Restraints. Hydrogen bond re-
straints can be very useful during structure calcula-
tions, especially in dening secondary structure ele-
ments such as -helices and -sheets. A problem with
hydrogen bond restraints is that they are commonly
assigned based on indirect experimental information,
such as nonexchanging amide protons (19), chemical
shifts (20), and one-bond coupling constants (21),
which only identify the donor atom involved in the
hydrogen bond. However, 3h J and 2h J spin-spin cou-
plings can now be measured across hydrogen bonds,
providing for direct evidence of the acceptor atom
(22, 23). This procedure works well for oligonucleo-
tides and in favorable cases for proteins. In the latter
case, the heteronuclear 2h J(CH) and 3h J(CN) cou-
plings provide the experimental evidence for the pres-
ence of a hydrogen bond (23, 24).
Even though the acceptor atom can now be iden-
tied experimentally, it is not uncommon to infer the
acceptor from structural models or assumptions about
regular secondary structure. In these cases, ambiguous
distance restraints are useful to allow for some ambi-
guity or multiple possibilities in the assignment of
potential hydrogen bond acceptor(s). An example of
how the hydrogen bonding network can be dened in
an -helix using ambiguous distance restraints is
shown in Fig. 2. The ambiguous distance restraint
illustrates the formation of either the i, i 3, i, i
4, or the i, i 5 hydrogen bond, thus allowing for
the various types of helices dened by these different
hydrogen bonding patterns.
Figure 1 (A) Ensemble of 30 structural models of GB1
(8 ). The -helix is shown as a blue ribbon, the -sheets are
indicated with red ribbons. Hydrogen atoms have been
omitted for clarity. (B) Restrained minimized average struc-
ture of GB1, with the 659 experimental distance restraints in
the experimental dataset shown in yellow. Restraints involv-
ing groups of hydrogen atoms are, for clarity reasons, only
shown for one of the protons involved. Figure made using
YASARA (http://www.yasara.org).
Torsion Angle Restraints
Torsion angle restraints are derived from the vicinal
coupling constant, 3 J, by means of the well known
Karplus relations (25):
3
J A cos 2 B cos C [3]
where describes the torsion angle between the four
involved atoms. The three parameters A, B, and C
have been empirically parameterized using molecules
for which X-ray models are known and depend on the
coupling constant under investigation. The 3 J-cou-
pling constant information can be translated into al-
lowed ranges for the relevant torsion angles and as
such can be incorporated in the structure calculation.
In case of 13C and 15N isotope labeling coupling
constants involving these nuclei can provide many
useful additional parameters that can provide struc-
tural information on backbone and side chain torsion
angles (26).
Whereas the atoms involved in an NOE restraint
can be far apart in sequence space, torsion angle
restraints provide only information on local confor-
mations. As a result, the contribution of torsion angle
restraints to the global fold is limited (27), though
they do provide important information to restrain the

Figure 2 An ambiguously assigned hydrogen bond re-
straint indicated in an -helix. The three contributing dis-
tances are shown in yellow, allowing for the formation of
either the i, i 3, the i, i 4, or the i, i 5 hydrogen
bond. In this case, the distance would be restrained to 2 .
For any given structure model, the effective distance is
calculated using Eq. [2]. Figure made using YASARA
(http://www.yasara.org). [Color gure can be viewed in the
online issue, which is available at www.interscience.wiley.
com.]
backbone torsion angles and , and the side chain
torsion angles.
Orientational Restraints
A more recent addition to the arsenal of experimental
restraints available to the NMR spectroscopist are the
orientational restraints derived from residual dipolar
couplings (RDCs). To obtain structural information
from dipolar couplings, the internuclear dipolar inter-
actions must be observable. In solution, the distribu-
tion of the molecular orientations is normally random
in the absence of a magnetic eld, and as a result the
internuclear dipolar interactions average to zero and
thus cannot be observed. In the presence of a mag-
netic eld, most biomolecules will still be oriented
randomly, with the exception of molecules that have a
large magnetic dipole moment, such as nucleic acids,
which may show a small degree of alignment with the
magnetic eld (28).
Alignment of biomolecules can be achieved by
immersing them into a slightly anisotropic environ-
ment, such as solutions containing phages, bicelles, or
NMR RESTRAINT VALIDATION 93
polyacrylamide gels with anisotropic cavities. Phages
and bicelles readily align with the magnetic eld, and
steric or electrostatic interactions with these anistropic
molecules introduce a small net alignment of the
biomolecule of interest. As a result, dipolar couplings
no longer completely average out and give rise to a
observable residual dipolar coupling, typically scaled
down by a factor of 103 relative to its static value.
In this case, the overall appearance of the NMR
spectrum is not altered, allowing for the direct mea-
surement of these couplings as small contributions to
the J-couplings of the involved nuclei (29, 30). The
observed residual dipolar coupling (D) between two
nuclei, p and q, is given by
3
D pq, D a 3 cos 2 1 Dr sin2 cos2
2
[4]
where D a is the magnitude of the dipolar coupling
tensor and D r the rhombicity. For a given value of
D pq , the cylindrical coordinates and describe a
cone of solutions for the orientation of the vector pq
in the principal axis system of the molecular align-
ment tensor. Provided that the molecular alignment
tensor is known, residual dipolar couplings present
information on the orientation of internuclear vectors
relative to an external reference frame. This external
frame can be introduced in the structure calculation
process as a tetra-atomic pseudomolecule, which rep-
resents the alignment tensor as an orthogonal axis
system (28). In addition to providing constraints on
local geometry, the residual dipolar couplings also
restrain the orientation of all the involved bond vec-
tors relative to this common reference frame. This
provides long-range order information, which is not
directly accessible from any of the other commonly
used NMR parameters. During structure calculations,
the orientational restraints derived from residual di-
polar couplings can be used together with distance
restraints and torsion angle restraints to yield more
accurate NMR structures (28, 31).
ANALYZING SETS OF RESTRAINTS
Sets of experimentally derived restraints, as discussed
in the previous sections, can be used for different
types of analyses, pertaining to four different aspects
of validation discussed throughout this section. The
rst and most widely used method is a statistical
analysis of the restraints, e.g., the total number of
restraints that were obtained from the different exper-
94 NABUURS ET AL.

Table 1 Restraint Validation Scores for the GB1 Dataset (PDB Entry 3GB1)

Torsion Angle Dipolar Coupling

Distance Restraints Restraints Restraints

Number of restraints 735 145 300

(37/17/11/35%)e
Restraints per residue 13.1 2.6 5.4
NOE completenessa 49%
Average number of violationsb,c 1.0 1.0
Cross-validated average number
of violationsb,c 4.5 1.5
RMS violationsb 0.02 0.67
Cross-validated RMS violationsb 0.16 19.0
Independent RDC R factord 26%/18%d
RDC R factord 8%/7%d
QUEEN set information 98.4% 1.6%
(0.4/2.4/11.8/85.4%)e
Nonredundant distance restraints 480
(9/18/19/53%)e
a
Determined at a 4 cutoff with all restraints included.
b
Determined in structures calculated without the inclusion of residual dipolar coupling data.
c
A violation threshold of 0.2 and 2 was used for the violation analysis.
d
The independent dipolar coupling R factor was calculated with only distance and dihedral restraints included in the structure calculation.
The rst and second values relate to NH dipolar coupling measured using tobacco mosaic virus and bicelles, respectively (31).
e
The numbers in brackets relate to the distribution (expressed in percentages) over the intraresidual, sequential, medium-range, and
long-range restraint classes, respectively.

iments. Second, once a structure model has been
calculated, it can be compared to the restraints and a
measure of how well the structure ts the data can be
obtained. This normally includes a restraint violation
analysis and calculations of the root mean square of
the restraint violations. Third, a measure of accuracy
is desirable, providing an estimate of how accurately
the structure model represents the truth. Like in X-ray
crystallography, this may be done by calculation of
the completeness of a dataset, R-factors, and by cross-
validation. The last type of analysis discussed relates
to the information contained in experimental re-
straints, i.e., how important a particular (type of)
restraint is for the nal structure that is derived. One
of the classical measures of information is the classi-
cation of distance restraints in short-, medium-, and
long-range restraints. This classication is based on
the distance a restraints spans in sequence space, a
measure independent of its Cartesian distance. More
recently, we proposed measures based on information
theory (27).
Here, we explain the classical and new methods for
validation and analysis of experimental restraints. An
overview of the various validation tools applied to the
GB1 dataset is given in Table 1 and will be discussed
throughout the text. Possible drawbacks and advan-
tages of different methods are discussed and we at-
tempt to assess their value for judging the quality of a
structure model.
Number of Restraints
One of the most straightforward indicators of the
quality of an experimental restraint set is the number
of restraints, which is sometimes reported as the num-
ber of restraints per residue [cf. Table 1 and Fig. 3B].
This indicator can be heavily biased, however, if not
derived carefully. Fewer than average, or no NOEs at
all, are typically observed for the regions of molecules
with conformational disorder, e.g., exible termini or
large external loops. When disordered residues are
included in the restraint count, this can result in an
articially low number of restraints per residue. To
prevent this, the analysis is generally restricted to the
well-dened regions of the molecules under investi-
gation, e.g., those areas with a low ensemble RMSD
or circular variance, or those residues involved in
secondary structure elements. For our test set of 100
NMR-derived structural ensembles, the number of
distance restraints per residue increases from 12 5,
calculated using all residues, to 16 7, if only those
residues involved in secondary structure elements are
included in the analysis. Adding to the difculty of
interpreting the total number of distance restraints is
 NMR RESTRAINT VALIDATION 95

Figure 3 NOE completeness values and number of NOEs for the GB1 dataset (8 ). (A) The
cumulative and per-shell completeness (left y-axis) versus the radius of the distance shell and the
number of observed and expected NOEs (right y-axis) versus the distance shell radius. (B) NOE
completeness at 4 cutoff radius (left y-axis) and number of NOE restraints per residue (right
y-axis) for the 56 residues of GB1.

their dependence on the size and shape of the biomol-
ecule, and to some extent on the residue-type compo-
sition. Plotting the number of distance restraints per
residue corrects to some extent for the size and shape
dependence; however, for very small molecules the
maximum number of restraints per residue that are
expected will be lower than for large molecules. Fur-
ther, the residue-type dependence will be expressed
clearly in such plots, e.g., glycines will generally be
involved in fewer NOEs than the larger leucines (32).
Yet another pitfall in deriving the number of re-
straints per residues is redundancy in the experimental
input data. It has been show that a signicant fraction
of the distance restraints used in a structure determi-
nation is typically redundant (27, 33). In order to use
the number of restraints as a measure for the amount
of information in the data set, redundant distance
restraints have to be removed prior to the analysis.
Methods to do so are discussed in some of the fol-
lowing sections.
For restraints derived from chemical shifts and
through-bond interactions, such as J-couplings and
residual dipolar couplings, the number of restraints is
more indicative of the quality of the experimental
data, since the total number of measurable interac-
tions is known from the primary structure of the
molecule. Restraints that are derived from J-cou-
plings across hydrogen bonds present an exception to
this, as these cannot be predicted unambiguously from
sequence information alone.
NOE Completeness
The NOE completeness was developed in the late
nineties to provide a more informative measure of the
information contained in data sets then the number of
restraints per residue. The completeness is dened as
the ratio, expressed as a percentage, of the number of
matched experimentally observed NOEs (N observed )
and the number of expected NOEs (N expected ) (32):
N observed
completeness 100% [5]
N expected
and thus normalizes the number of observed re-
straints. Unfortunately, there is no way to determine
the number of expected restraints a priori, and there-
fore a structural model has to be known to determine
the NOE completeness. Given this structural model,
the expected NOE restraints are derived by selecting
all typically observable proton-proton distances below
a given cutoff value. Protons that are rarely observed
in NMR experiments, such as those in amino (Lys,
96 NABUURS ET AL.
N-terminus), carboxyl (Asp, Glu, C-terminus),
sulphhydryl (Cys), and hydroxyl (Ser, Thr, Tyr)
groups, and those connected to the imidazole group of
histidine and the guadinium group in arginine, are
excluded from the analysis. The set of expected NOE
restraints is then compared to those actually observed
within the limits imposed by the cutoff value. The
number of matched restraints is then used to calculate
the NOE completeness for that particular cutoff value
using Eq. [5]. By increasing the cutoff value in small
steps, gradually all observed NOEs are included in the
analysis. In case of structural variability, the expected
NOE distance is dened as the average distance in the
different members of the ensemble. If this average
distance is above the cutoff value, it will be discarded
from the set of observable contacts. This way, a
exible region of a protein can have a similar com-
pleteness as a rigid region although the latter is de-
ned by more NOEs per residue.
Figure 3A and Table 1 show the NOE complete-
ness values for the GB1 dataset. The cutoff value was
increased from 2.5 to 8.0 with a step size of 0.5 .
The completeness drops from 55% in the 2.0 2.5
shell to 0% in the 7.5 8.0 shell. In addition to the
completeness per shell, the cumulative completeness
is also shown. The cumulative completeness for the
GB1 dataset is 53%, 49%, and 33% up to 3, 4, and 5
cutoff distances, respectively. A comparison be-
tween the NOE completeness per residue, calculated
with a 4 cutoff, and the number of restraints per
residue is shown in Fig. 3B. Although the overall
trend between the two graphs is similar, the two
quantities are only weakly correlated as expressed by
a correlation coefcient of 0.4, illustrating the differ-
ent nature of the two information measures.
Restraint Violations
The structural models resulting from an NMR struc-
ture calculation are rarely in exact agreement with the
experimental input data used to calculate them. Pos-
sible reasons include inconsistencies in the input data,
which can originate from assignment errors, calibra-
tion problems, or the presence of several different
conformers in solution. The best models generated in
a structure calculation can be selected using different,
often subjective, criteria, e.g., force eld energies or
the number and size of the experimental restraint
violations. In the latter case, there is general consen-
sus in the NMR community that structures without
violations 0.5 can be considered acceptable,
although over the past years lower cutoffs of 0.3
have also been reported. When selecting structures
based on a violation size criterion, it can be informa-
tive to also study the size and number of smaller
violations below the cutoff value, as they can also
pinpoint problematic regions in the structure or erro-
neous restraints in the dataset.
The agreement of an ensemble of structural models
with the experimentally derived distance restraints
can be judged by the root mean square (RMS) NOE
deviation:
Nr Nm
1
RMS NOE 2
Nr Nm k1 l1
kl
dkl r upper
k kl dkl r kupper
with r k dkl r k kl 0
lower upper
[6]
dkl r lower
k kl r lower
k dkl
with d kl the actual distance for restraint k in model l,
r upper
k the upper bound of restraint k and r lower
k the
lower bound of restraint k. The sum is calculated over
all N r distance restraints and N m structural models.
An identical expression can be used to calculate the
RMS torsion angle restraints deviations (accounting
for the circular nature of this property), with d kl the
actual torsion angle measured in model l, and r upperk ,
and r lower
k , the upper and lower limit of the torsion
angle range allowed for by restraint k, respectively.
Fig. 4 shows the distribution of the size and num-
ber of distance restraint violations in the reference set
of 100 NMR structural ensembles before and after
renement of these structures in explicit solvent (7).
In the original structures a signicant number of vio-
lations between 0.5 and 1.0 are found, most of
which are absent in the rerened structural models.
This improvement in the t of the models to the
experimental data is also reected in the RMS NOE
violation for the original and rerened models, which
is 0.089 and 0.029 , respectively (7).
NMR R Factor
A more direct indicator for the quality of structural
models calculated on the basis of experimental NMR
data is the agreement between the original experimen-
tal data and the data back-calculated from the pro-
posed structures. This agreement can be expressed
using the R factor, the normalized mean deviation
between the measured and back-calculated data, a
quality measure often used in X-ray crystallography.
The lower the R factor, the better the model represents
the experimental data. As the principal data in NMR
structure determinations are the NOE signals with
their corresponding intensities, the NMR R factor is
dened as the measure of agreement between the
 NMR RESTRAINT VALIDATION 97

Figure 4 Occurrence of NOE-derived distance restraint violations in 100 NMR-derived structural

ensembles as function of the violation size shown before (red) and after (blue) renement in explicit
solvent. The 100 ensembles consist of 1567 structural models in total. [Color gure can be viewed
in the online issue, which is available at www.interscience.wiley.com.]

observed and back-calculated intensities observed in
the NOESY spectra (34). Theoretical NOE signal
intensities can be calculated using relaxation matrix
theory, which solves the Bloch equations for the com-
plete spin system for a given mixing time mix (35,
36). In this approach, all interacting spins are treated
as a network, and the volumes of the NOE cross peaks
are calculated by exponentiating the matrix of cross-
relaxation rates: the relaxation matrix.
An R factor denition to determine the agreement
between the measured and calculated intensities, anal-
ogous to that used in X-ray crystallography, is
i, j W ij mixA calc
ij mix A ij mix
p exp
R [7]
i, j Wij mix Aexp
ij mix
p
tions and uncertainties involved in the computational
procedures, these problems make the R factor a rarely
used validation criterion in structure determinations
by NMR spectroscopy.
Complete Cross Validation
As mentioned, in X-ray crystallography the R factor is
a much more commonly used criterion for judging the
quality of structural models. But the crystallographic
R factor is not without problems, as it can be arti-
cially reduced by introducing more parameters to
p describe the model. Kleywegt and Jones (37) have
shown that it is possible to overt diffraction data to

the extent that a structural model with every amino

with A calc and A exp the calculated and experimental
ij ij
cross-peak intensities resulting from the interacting
spins i and j, respectively, W ij ( mix ) a mixing time-
dependent weighting factor, and the power p can
simply be 1 or, e.g., 1/6 to reect the asymptotic
behavior of the NOE. The weighting factor should
account for the uncertainties arising from the experi-
mental and computational procedures. It is also here
that the problems with an NMR R factor lie. Estimat-
ing the error of the signals observed in the NOESY
spectrum is complicated. Together with the assump-
acid at an incorrect position can still be rened to a
reasonable R factor. The R free factor was introduced
as a quality indicator less prone to overtting (38). Its
denition is identical to that of the traditional R fac-
tor, except that the R free value is calculated for a small
subset of the data which was not used in the rene-
ment of the model. Therefore, the R free measures how
well the model predicts experimental data not used to
derive the model, and hence can detect overtting of
the data. This method is also commonly referred to as
cross-validation.
98 NABUURS ET AL.
In the application of cross-validation in crystallog-
raphy, typically 10% of the reections are omitted
(the test set) and the remainder of the data (the work-
ing set) is used for model renement. In contrast to
X-ray crystallography, where each single reection
contains information about the whole structure, each
NOE signal in NMR spectroscopy provides only local
information. Hence, where deletion of 10% of the data
in a crystallographic dataset results in a 10% loss in
information content, this will not be case for NMR-
derived datasets. Additionally, the information con-
tent of the separate restraints is not identical through-
out an NMR dataset, e.g., intermolecular restraints
versus intraresidual restraints (27, 33). As a result of
this, cross-validation with a single test set is not
appropriate for NMR structure determinations.
To circumvent these problems, the concept of com-
plete cross-validation was introduced (39). In this
approach the NMR restraints are randomly partitioned
into test sets of roughly equal size, and cross-valida-
tion is performed with each of the test sets. Statistical
quantities are then averaged over the different test
sets. By doing so, the differences in information con-
tent between the test sets is expected to average out,
resulting in more meaningful cross-validated mea-
sures of t for NMR datasets. Despite this, the fact
remains that NOEs are not independent observations.
NOEs can often only be assigned based on informa-
tion provided by earlier assigned NOEs, e.g., from
structures calculated using these NOEs. If one of
these early assigned NOEs is then moved to the test
set, it will still inuence the structure if assignments
made based on this restraint are still present in the
working set. Thus, if not treated carefully, the calcu-
lated free R values are not as free as one might
assume.
Using the deposited GB1 restraint set (8), we cal-
culated and cross-validated an ensemble of 30 struc-
tures using the default CNS (40) protocols. Cross-
validated values typically are signicantly higher
when compared to their full dataset counterparts (39)
[see Table 1], and no direct conclusion can be derived
from their comparison. They are, however, useful
when comparing the effects of variations in the struc-
ture calculation protocol, such as the usage of multi-
ple models (41).
Quality Factors
In a so-called independent validation of structures,
some of the experimental data is completely kept out
of the structure calculation and renement process
and used only to validate the resulting structural mod-
els. For example, using anisotropic carbonyl chemical
shifts, different X-ray and NMR-derived structural
models were validated in this way (42). Changes in
the chemical shifts of carbonyl carbons were mea-
sured (meas) in a dilute liquid crystalline phase and
back-calculated from structures (pred) that were de-
termined without the inclusion of the chemical shifts
as structural restraints. Structures calculated with re-
sidual dipolar couplings as orientational restraints
showed an improved t to the carbonyl chemical shift
data as compared to structures calculated without the
inclusion of residual dipolar couplings. The measure
for the agreement between the structures and the
observed property, in this case the changes in car-
bonyl chemical shift, was obtained by using the qual-
ity or Q factor (42), dened as:
rms meas pred
Q [8]
rmsmeas
Like the R factor, the closer the Q factor is to zero, the
better the agreement between the model and the ob-
served independent data.
A similarly dened Q factor has been used as a
measure of t for residual dipolar couplings (43, 44),
and is equivalent to 2 times the dipolar coupling R
factor (45). When not directly used in the calculation
protocol, the RDC R factors for GB1 have values of
26% and 18% for data recorded in tobacco mosaic
virus and bicelles, respectively. In contrast, these val-
ues drop to 8% and 7% upon their inclusion in the
calculation, indicating that they convey unique infor-
mation about the biomolecular structure (31).
As a second example of independent validation, we
mention the use of the residual dipolar coupling Q
factors to check improvements in renement schemes
and force elds (46, 47). Structures of the protein
ubiquitin were calculated and rened using distance
and torsion angle restraints, and independently vali-
dated against different sets of residual dipolar cou-
plings. Structures rened in explicit solvent showed
signicant decreases in the residual dipolar coupling
Q factors as compared to the original structures,
which was accompanied by a concomitant decrease in
NOE RMS values, indicating a better overall agree-
ment with the experimental data.
Information Content
Recently, we proposed a method for the quantitative
evaluation of experimental NMR restraints (QUEEN)
(27). QUEEN is based on a representation of the
structure in distance space and concepts derived from
information theory. As many of the commonly used
 NMR RESTRAINT VALIDATION 99

Figure 5 The structural uncertainty, H structure , of GB1 as a function of the number of included
distance restraints. The distance restraints are grouped into four sets: intraresidual restraints (i j
0, IR), sequential restraints (i j 1, SQ), medium-range restraints (i j 5, MR), and
long-range restraints (i j 5, LR). Two different orders of addition of the experimental data
are shown: (A) IR-SQ-MR-LR and (B) LR-MR-SQ-IR. (C) Comparison of the average relative
dataset size and average set information content for 100 NMR-derived experimental datasets. The
fraction of restraints in these datasets that belongs to each of the four mentioned restraint classes is
shown in the left bar. The right bar shows the average distribution of the structural information over
the different restraint classes a determined by the QUEEN (27 ) method. [Color gure can be viewed
in the online issue, which is available at www.interscience.wiley.com.]

experimental input data can be readily represented in
distance space, it is possible to construct a distance
matrix representing all available distance information.
In this matrix, a structure with N atoms can be
uniquely dened by N(N 1)/ 2 interatomic dis-
tances. As structures are typically not exactly dened
by NMR-derived data, distances are described by an
upper and lower bound between which the true value
for that distance must lie. QUEEN uses an N N
matrix for storing the upper and lower bounds. By
means of a bound-smoothing algorithm (48, 49), the
distance information contained in the experimental
restraints can be introduced in this distance matrix.
We express the amount of structural knowledge
about a specic system in terms of a uncertainty, H.
Using concepts derived from information theory (50)
we derived (27) the following denition for the struc-
tural uncertainty, H structure :
tainty, the information I r contained in an experimental
restraint (or a set of restraints) is dened as the differ-
ence in uncertainty of the system before (Hstructure) and
after (H structurer ) addition of the experimental data r:
I r H structure H structurer [10]
A bound-smoothed distance matrix containing only
the covalent constraints is taken as the initial state,
H structure0 , prior to the addition of any experimental
restraints. After addition of a restraint, or a set of
restraints, the upper and lower bounds on all distance
limits are adjusted and the uncertainty of the structure
can then be calculated using Eq. [9]. Fig. 5A and 5B
show the decrease in the structural uncertainty for
GB1 as the 659 distance restraints of its dataset are
incorporated into the distance matrix. The two graphs
clearly illustrate the context dependency of restraint
information: e.g., the information contained in the

logd
N N medium range restraints is strongly reduced if the
1
H structure upper
d lower [9] long-range restraints are already incorporated in the
NN 1 s1 ts
st st
distance matrix. The context dependency can be re-
moved by averaging the information content of the
where N is the number of atoms in the structure, different restraint sets over all their possible permu-
upper
d st is the upper bound of the distance between tations. Fig. 5C shows the average set information
lower
atoms s and t, and d st is the lower bound of that content for the 100 experimental datasets, together
same distance, with all bounds given in ngstrom with the distribution of the restraints in these datasets
units. With this denition for the structural uncer- over the different restraint classes. Traditionally, the
100 NABUURS ET AL.

classication of restraints in intraresidual, sequential,

medium range, and long range is used to provide a
qualitative measure of the information content in
NMR-derived distance restraints (15), as it is gener-
ally recognized that the more long range the interac-
tions are observed, the larger their impact on the
structure determination and thus the more information
this category contains. Fig. 5C shows that this as-
sumption is indeed true; calculating the information
content of the different classes using the QUEEN
procedure shows that the long-range restraints are the
most informative class of restraints. However, it also
shows that the distribution of the information over the
four restraint classes is completely different from the
distribution of the actual number of restraints over
those classes, rendering the latter a rather poor indi-
cator for the information content in experimental re-
straint sets.

ANALYZING INDIVIDUAL RESTRAINTS

In addition to analyzing restraints sets as a whole, it is

also informative to analyze experimental restraints
individually. During the structure determination pro-
cess, this approach can be especially helpful in the
interpretation and validation of the derived restraints
and their resulting structures. At this stage it is still
possible to return to the experimental data and verify
the assignment and volume integrations of those re-
straints that are identied as problematic. Once the
structures have been deposited, and the original spec-
troscopic data often is no longer available, the differ-
ent restraint analyses can only be used to identify
problematic regions in structural models, without the
possibility of reevaluation against the original spec-
troscopic data.
Consistent Violations
Violation analyses and statistics for complete datasets
have been discussed in the previous sections. Here,
we briey mention the approach of analyzing individ-
ual restraint violations throughout the different mod-
els in an NMR ensemble. When analyzing individual
restraints, it is important to distinguish between con-
sistent violations, occurring throughout the majority
of the generated models, and those occurring ran-
domly. Consistent violations are a powerful indicator
of inconsistencies in the experimental data, where
those violations occurring less frequently might just
indicate that the structure calculation algorithm has
not reached convergence yet. The consistently violat-
ing restraints are not necessarily also the incorrect
Figure 6 (A) The relative information content, I ave /I total ,
and (B) the relative unique information content, I ave /I total ,
both plotted as a function of the NOE restraint index of the
GB1 dataset (8 ). The different restraint classes are labeled
as in Fig. 5. Note that the similar gures in (27 ) are
generated using the similar but not identical original GB1
dataset (57 ).
restraints. Often, incorrect restraints induce violations
in nearby correct restraints. Therefore, if a consis-
tently violating restraint is found, one should reexam-
ine all restraints in that region of the structure.
Restraint Information Content
Next to analyzing the agreement of the applied exper-
imental restraints with the generated structures, the
experimental restraints can also be analyzed indepen-
dently of the structural models. Using QUEEN, the
overall importance and the unique contribution of
each restraint can be determined (27). Similar to
calculating the information content for a set of re-
straints, the average information content (I ave,r ) of a
single restraint r can be calculated as:
I ave,r H structure H structurer [11]
The average information content provides a measure
of the overall importance of a restraint within the
complete dataset. Due to the context dependency of

the information content a meaningful comparison of
information measures between different datasets is
not possible, though we are currently in the process of
developing methods to allow such analyses. Fig. 6A
shows the relative importance of each of the 659
distance restraints in the GB1 dataset. As expected
from the results shown in Fig. 5C, the medium- and
long-range restraints contain the most experimental
information. However, a large variation in informa-
tion content is observed, even between restraints
within one restraint class.
In addition to the average contribution of a re-
straint, the unique information a restraint adds to the
dataset can also provide useful insights. The unique
information content (I uni,r ) of a restraint r is dened
as the information it adds given knowledge of all
other (R r) restraints in the dataset (R):
I uni,r H structureRr H structureR [12]
The unique information content of the distance
restraints in the GB1 dataset is shown in Fig. 6B and
shows a very different pattern compared to that ob-
served for the average information. Restraints with a
high unique information content are important be-
cause they are less well supported by the remainder of
the dataset, indicating that they either provide crucial
knowledge about this particular structure or alterna-
tively suggest a potential error. In either case, these
restraints are interesting and denitely warrant further
investigation.
We have shown previously that combining the
unique and average information content can be useful
in the identication of problematic restraints in an
experimental dataset (27). Furthermore, when analyz-
ing large sets of restraints, QUEEN can be used to sort
the restraints based on their information content. The
concept of placing a lower condence on restraints
that are not well supported by other experimental data
has been applied previously in spectral assignment
methods (51, 52). By verifying those restraints with a
high average and unique information content rst,
inconsistencies and errors are likely to be found
faster, as the lesser-supported data is found at the
beginning of such a sorted restraint list. Approaches
like this can potentially shorten the time needed to
manually verify and validate experimental restraint
datasets and can be used in intelligent automated
procedures.
Redundant Restraints
In a study assessing the quality of NMR structures,
Doreleijers and colleagues (33) observed that for a
NMR RESTRAINT VALIDATION 101
group of 15 entries (those with the highest fraction of
intra-residual restraints), 58% of these restraints were
redundant. To detect these redundant restraints, they
implemented a check in the AQUA program, which is
aimed at the analysis of biomolecular structures de-
termined by NMR (53). AQUA identies redundant
intraresidual restraints by comparing the upper and
lower bounds of the experimental restraints to the
distance limits observed for that particular intrare-
sidual distance in a reference molecular dynamics
simulation. Restraints are considered redundant when
both the upper and lower bound are within the dis-
tance limits imposed by the molecular geometry. Be-
cause of the limited sampling possible in a molecular
dynamics simulation, AQUA only considers redun-
dancy in the intraresidual restraints (33).
With the information measures calculated by
QUEEN, it is straightforward to identify redundant
restraints in any of the restraint classes. If a restraint
has a unique information content of zero, it contains
no information that is not already present in the data-
set, and is therefore redundant (cf. Eq. 12). However,
because of the context dependency, not all restraints
with a unique information content of zero can be
immediately removed from the dataset. To ensure
consistency, the unique information content has to be
reevaluated after each restraint deletion. By doing so,
a minimal set of restraints can constructed, which no
longer contains structural redundancy in any of the
restraint classes.
It is important to note here that we do not advise
the use of redundancy checks to lter restraint data-
sets. Removal of the redundant restraints results in
smaller datasets without any loss of structural infor-
mation, which are therefore useful in retrieving more
sensible restraint counts, e.g., for the number of in-
formative restraints per residue. For the GB1 dataset,
Table 1 shows that about 35% the experimental dis-
tance restraints are redundant. However, nonredun-
dant datasets are deprived of valuable conrmative
information. All restraints in the dataset will now
have high values for the amount of unique informa-
tion they carry, making validation by, e.g., the
QUEEN software, uninformative.
CONCLUSIONS
In this review we have discussed different validation
tools available to NMR spectroscopists to assess the
quality of experimentally derived restraint datasets. In
practice, the approaches discussed are often only ap-
plied at the end of the structure determination process.
We feel, however, that the quality of both the struc-
102 NABUURS ET AL.
Figure 7 Fraction of NMR structures in the PDB with associated deposited experimental re-
straints. The best linear t is shown as a dashed line.
tures and the generated datasets would benet from
restraint analyses as an integral part throughout the
iterative assignment and structure calculation process.
Fortunately, this is becoming more feasible as ever
more sophisticated automated assignment and struc-
ture calculation approaches are becoming available
(54 56). Additionally, more frequent usage of robust
quality indicators, such as complete cross-validation,
would provide a better quality assessment of the de-
rived structural models. Deposition of the experimen-
tal restraints together with the structures ensures that
other researches can reproduce the structures and use
the restraints for development and testing of new
structure calculation, renement, and validation pro-
tocols. Fortunately, the number of structures that is
deposited together with the experimental restraints
used to calculate them gradually increases over the
years [as shown in Fig. 7] and provides the ground for
development and testing of more elaborate validation
procedures.
Selected Hyperlinks
The QUEEN software package can be downloaded from
http://www.cmbi.kun.nl/software/queen/. The DRESS
database of rened structures and validation reports is
available at http://www.cmbi.kun.nl/dress/. The Collab-
orative Computing Collaborative Computing Project for
the NMR Community (CCPN) is accessible at http://
www.ccpn.ac.uk/. The PROCHECK_NMR program
suite can be downloaded from http://www.biochem.
ucl.ac.uk/roman/procheck_nmr/procheck_nmr.html.
Finally, the AQUA program suite can be downloaded
from http://urchin.bmrb.wisc.edu/jurgen/aqua/.
ACKNOWLEDGMENTS
We thank all NMR spectroscopists who deposited the
experimental restraints together with their structures,
especially G.M. Clore and coworkers for making
available the high-quality datasets of GB1. Financial
support from the Netherlands Foundation for Chemi-
cal Research (NWO/CW) to C.S. and from the Euro-
pean community (5th Framework program NMR-
QUAL contract number QLG2-CT-2000-01313) to
S.N. is gratefully acknowledged.
REFERENCES
1. Wuthrich K. 1986. NMR of proteins and nucleic acids.
New York: Wiley.
2. Moseley HNB, Montelione GT. 1999. Automated anal-
ysis of NMR assignments and structures for proteins.
Curr Opin Struct Biol 9(5):635 642.
3. Guntert P. 2003. Automated NMR protein structure
calculation. Prog Nucl Magn Reson Spectrosc 43(3 4):
105125.
4. Seavey BR, Farr EA, Westler WM, Markley JL. 1991.
A relational database for sequence-specic protein
NMR data. J Biomol NMR 1(3):217236.
5. Bernstein FC, Koetzle TF, Williams GJ, Meyer EF, Jr.,
Brice MD, Rodgers JR, Kennard O, Shimanouchi T,
Tasumi M. 1977. The protein data bank: a computer-

based archival le for macromolecular structures. J Mol
Biol 112(3):535542.
6. Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat
TN, Weissig H, Shindyalov IN, Bourne PE. 2000. The
protein data bank. Nucleic Acids Res 28(1):235242.
7. Nabuurs SB, Nederveen AJ, Vranken W, Doreleijers
JF, Bonvin AMJJ, Vuister GW, Vriend G, Spronk
CAEM. 2004. DRESS: a database of rened solution
NMR structures. Proteins 55:483 486.
8. Kuszewski J, Gronenborn AM, Clore GM. 1999. Im-
proving the packing and accuracy of NMR structures
with a pseudopotential for the radius of gyration. J Am
Chem Soc 121(10):23372338.
9. Case DA. 1998. The use of chemical shifts and their
anisotropies in biomolecular structure determination.
Curr Opin Struct Biol 8(5):624 630.
10. Kuszewski J, Gronenborn AM, Clore GM. 1996. Im-
proving the quality of NMR and crystallographic pro-
tein structures by means of a conformational database
potential derived from structure databases. Protein Sci
5(6):10671080.
11. Clore GM, Robien MA, Gronenborn AM. 1993. Ex-
ploring the limits of precision and accuracy of protein
structures determined by nuclear magnetic resonance
spectroscopy. J Mol Biol 231(1):82102.
12. Jeener J, Meier BH, Bachmann P, Ernst RR. 1979.
Investigation of exchange processes by two-dimen-
sional NMR-spectroscopy. J Chem Phys 71(11):4546
4553.
13. Macura S, Ernst RR. 1980. Elucidation of cross relax-
ation in liquids by two-dimensional NMR-spectros-
copy. Mol Phys 41(1):95117.
14. Kumar A, Ernst RR, Wuthrich K. 1980. A two-dimen-
sional nuclear Overhauser enhancement (2d Noe) ex-
periment for the elucidation of complete proton-proton
cross-relaxation networks in biological macromole-
cules. Biochem Biophys Res Comm 95(1):1 6.
15. Markley JL, Bax A, Arata Y, Hilbers CW, Kaptein R,
Sykes BD, Wright PE, Wuthrich K. 1998. Recommen-
dations for the presentation of NMR structures of pro-
teins and nucleic acids. J Mol Biol 280(5):933952.
16. Fletcher CM, Jones DNM, Diamond R, Neuhaus D.
1996. Treatment of NOE constraints involving equiva-
lent or nonstereoassigned protons in calculations of
biomacromolecular structures. J Biomol NMR 8(3):
292310.
17. Nilges M. 1995. Calculation of protein structures with
ambiguous distance restraints. Automated assignment
of ambiguous NOE crosspeaks and disulphide connec-
tivities. J Mol Biol 245(5):645 660.
18. Nilges M. 1997. Ambiguous distance data in the cal-
culation of NMR structures. Fold Des 2(4):S5357.
19. Wagner G, Wuthrich K. 1982. Amide protein exchange
and surface conformation of the basic pancreatic trypsin
inhibitor in solution. Studies with two-dimensional nu-
clear magnetic resonance. J Mol Biol 160(2):343361.
20. Wishart DS, Sykes BD, Richards FM. 1992. The chem-
ical shift index: a fast and simple method for the as-
NMR RESTRAINT VALIDATION 103
signment of protein secondary structure through NMR
spectroscopy. Biochemistry 31(6):16471651.
21. Juranic N, Ilich PK, Macura S. 1995. Hydrogen bond-
ing networks in proteins as revealed by the amide
1JNC coupling constant. J Am Chem Soc 117(1):405
410.
22. Dingley AJ, Grzesiek S. 1998. Direct observation of
hydrogen bonds in nucleic acid base pairs by inter-
nucleotide (2)J(NN) couplings. J Am Chem Soc
120(33):8293 8297.
23. Cordier F, Grzesiek S. 1999. Direct observation of
hydrogen bonds in proteins by interresidue (3h)J(NC)
scalar couplings. J Am Chem Soc 121(7):16011602.
24. Cordier F, Rogowski M, Grzesiek S, Bax A. 1999.
Observation of through-hydrogen-bond (2h)J(HC) in a
perdeuterated protein. J Magn Reson 140(2):510 512.
25. Karplus M. 1959. Contact electron-spin coupling of
nuclear magnetic moments. J Chem Phys 30(1):1115.
26. Vuister GW, Tessari M, Karimi-Nejad Y, Whitehead B.
1998. Pulse sequences for measuring coupling con-
stants. In: Berliner LJ, Krishna NR, editors. Modern
techniques in protein NMR. Vol. 16. New York: Ple-
num p 195257.
27. Nabuurs SB, Spronk CA, Krieger E, Maassen H,
Vriend G, Vuister GW. 2003. Quantitative evaluation
of experimental NMR restraints. J Am Chem Soc
125(39):12026 12034.
28. Tjandra N, Omichinski JG, Gronenborn AM, Clore
GM, Bax A. 1997. Use of dipolar 1H15N and 1H13C
couplings in the structure determination of magneti-
cally oriented macromolecules in solution. Nat Struct
Biol 4(9):732738.
29. Tolman JR, Flanagan JM, Kennedy MA, Prestegard JH.
1995. Nuclear magnetic dipole interactions in eld-
oriented proteins: information for structure determina-
tion in solution. Proc Natl Acad Sci USA 92(20):9279
9283.
30. Tjandra N, Bax A. 1997. Direct measurement of dis-
tances and angles in biomolecules by NMR in a dilute
liquid crystalline medium. Science 278(5340):1111
1114.
31. Clore GM, Starich MR, Bewley CA, Cai ML, Kuszew-
ski J. 1999. Impact of residual dipolar couplings on the
accuracy of NMR structures determined from a mini-
mal number of NOE restraints. J Am Chem Soc
121(27):6513 6514.
32. Doreleijers JF, Raves ML, Rullmann T, Kaptein R.
1999. Completeness of NOEs in protein structure: a
statistical analysis of NMR. J Biomol NMR 14(2):123
132.
33. Doreleijers JF, Rullmann JA, Kaptein R. 1998. Quality
assessment of NMR structures: a statistical survey. J
Mol Biol 281(1):149 164.
34. Gonzalez C, Rullmann JAC, Bonvin AMJJ, Boelens R,
Kaptein R. 1991. Toward an NMR R factor. J Magn
Reson 91:659 664.
35. Bonvin AM, Boelens R, Kaptein R. 1991. Direct NOE
104 NABUURS ET AL.
renement of biomolecular structures using 2D NMR
data. J Biomol NMR 1(3):305309.
36. James TL. 1991. Relaxation matrix analysis of two-
dimensional nuclear Overhauser effect spectra. Curr
Opin Struct Biol 1(6):10421053.
37. Kleywegt GJ, Jones TA. 1995. Where freedom is given,
liberties are taken. Structure 3(6):535540.
38. Brunger AT. 1992. Free R value: a novel statistical
quantity for assessing the accuracy of crystal structures.
Nature 355:472 475.
39. Brunger AT, Clore GM, Gronenborn AM, Saffrich R,
Nilges M. 1993. Assessing the quality of solution nu-
clear magnetic resonance structures by complete cross-
validation. Science 261(5119):328 331.
40. Brunger AT, Adams PD, Clore GM, DeLano WL, Gros
P, Grosse-Kunstleve RW, Jiang JS, Kuszewski J, Nil-
ges M, Pannu NS, et al. 1998. Crystallography and
NMR system: a new software suite for macromolecular
structure determination. Acta Crystallogr D Biol Crys-
tallogr 54(Pt 5):905921.
41. Bonvin AM, Brunger AT. 1996. Do NOE distances
contain enough information to assess the relative pop-
ulations of multi-conformer structures? J Biomol NMR
7(1):7276.
42. Cornilescu G, Marquardt JL, Ottiger M, Bax A. 1998.
Validation of protein structure from anisotropic car-
bonyl chemical shifts in a dilute liquid crystalline
phase. J Am Chem Soc 120(27):6836 6837.
43. Meiler J, Peti W, Griesinger C. 2000. DipoCoup: a
versatile program for 3D-structure homology compari-
son based on residual dipolar couplings and pseudocon-
tact shifts. J Biomol NMR 17(4):283294.
44. Zweckstetter M, Bax A. 2000. Prediction of sterically
induced alignment in a dilute liquid crystalline phase:
aid to protein structure determination by NMR. J Am
Chem Soc 122(15):37913792.
45. Clore GM, Garrett DS. 1999. R-factor, free R, and
complete cross-validation for dipolar coupling rene-
ment of NMR structures. J Am Chem Soc 121(39):
9008 9012.
46. Spronk CA, Linge JP, Hilbers CW, Vuister GW. 2002.
Improving the quality of protein structures derived by
NMR spectroscopy. J Biomol NMR 22(3):281289.
47. Linge JP, Williams MA, Spronk CA, Bonvin AM,
Nilges M. 2003. Renement of protein structures in
explicit solvent. Proteins 50(3):496 506.
48. Crippen GM. 1977. A novel approach to the calculation
of conformation: distance geometry. J Comp Phys 26:
449 452.
49. Havel TF, Kuntz ID, Crippen GM. 1983. The theory
and practice of distance geometry. Bull Math Biol
45:665720.
50. Shannon CE, Weaver W. 1949. The mathematical the-
ory of communication. Urbana, IL: University of Illi-
nois Press.
51. Englander SW, Wand AJ. 1987. Main-chain-directed
strategy for the assignment of 1H NMR spectra of
proteins. Biochemistry 26(19):59535958.
52. Herrmann T, Guntert P, Wuthrich K. 2002. Protein
NMR structure determination with automated NOE as-
signment using the new software CANDID and the
torsion angle dynamics algorithm DYANA. J Mol Biol
319(1):209 227.
53. Laskowski RA, Rullmann JA, MacArthur MW,
Kaptein R, Thornton JM. 1996. AQUA and PRO-
CHECK-NMR: programs for checking the quality of
protein structures solved by NMR. J Biomol NMR
8(4):477 486.
54. Linge JP, Habeck M, Rieping W, Nilges M. 2003.
ARIA: automated NOE assignment and NMR structure
calculation. Bioinformatics 19(2):315316.
55. Herrmann T, Guntert P, Wuthrich K. 2002. Protein
NMR structure determination with automated NOE as-
signment using the new software CANDID and the
torsion angle dynamics algorithm DYANA. J Mol Biol
319(1):209 227.
56. Gronwald W, Moussa S, Elsner R, Jung A, Ganslmeier
B, Trenner J, Kremer W, Neidig KP, Kalbitzer HR.
2002. Automated assignment of NOESY NMR spectra
using a knowledge based method (KNOWNOE). J Bi-
omol NMR 23(4):271287.
57. Gronenborn AM, Filpula DR, Essig NZ, Achari A,
Whitlow M, Wingeld PT, Clore GM. 1991. A novel,
highly stable fold of the immunoglobulin binding do-
main of streptococcal protein G. Science 253(5020):
657 661.
BIOGRAPHIES
Sander Nabuurs (second from left) studied chemistry at the Uni-
versity of Nijmegen. During his studies he participated in research
projects at the Department of Biophysical Chemistry under the
supervision of Geerten Vuister and at the Center for Molecular and
Biomolecular Informatics (CMBI) with Gert Vriend. They now
both act as his Ph.D. supervisors on the topic of NMR structure
validation. In addition to his graduate research, he very much
enjoys building homology models in collaboration with several
experimental groups.
Chris Spronk (second from right) studied chemistry at the Uni-
versity of Nijmegen and received his Ph.D. in 1999 at Utrecht
University. Starting with the application of solution NMR to solve
and study protein and DNA structures, his main eld of interest
shifted toward the computational aspects of NMR structure deter-

mination, such as improved NMR renement protocols and devel-
opment of new NMR validation tools. His other goals are to educate
scientists in the basic principles of structure validation, provide
automated tools and improved structure models to the NMR com-
munity, and stress the importance of good validation methods for
NMR structures.
Gert Vriend (left) studied biochemistry at Utrecht University and
received his Ph.D. in 1983 at the University of Wageningen. He
held postdoctoral positions in Purdue, Indiana, USA, and in Gro-
ningen, The Netherlands, working on several structures while start-
ing the WHAT IF project. In 1989 he moved to the EMBL in
Heidelberg, Germany, where he continued working on (and with)
the WHAT IF program. In the summer 1999 he was appointed a full
NMR RESTRAINT VALIDATION 105
professor at the University of Nijmegen, establishing the Center for
Biomolecular Informatics. His main research topics are model
building by homology, structure verication, specialized databases,
and the application of computers in wet biology.
Geerten Vuister (right) studied chemistry at the University of
Groningen and received his Ph.D. at Utrecht University. Between
1991 and 1994 he held a postdoctoral position at the NIH, Be-
thesda, Maryland, USA. Subsequently, he was appointed assistant
professor in the Department of NMR Spectroscopy at Utrecht
University. In 1997 he became associate professor at the Depart-
ment of Biophysical Chemistry, NSRIM at the University of Ni-
jmegen, where his research focuses on high-resolution NMR pro-
tein structure determination and validation.