Interactive Consistency With Multiple Failure Modes
Philip Thambidurai and You-Keun Park
Aerospace Technology Center
Allied-Signal Aerospace Company
Abstract
The theoretical requirement for Interactive Consis-
tency, N > 3t, where t is the maximum number of
faulty processors and N the total number of proces-
sors, is based on worst case models of failure. A de-
sign based on this model, while making no assump-
tions about failure modes, imposes a severe penalty
in terms of hardware and performance. We address
the problem of reaching Byzantine Agreement in a
distributed system in the presence of different types
of faults, and show that significant improvements in
reliability and performance are possible if faults can
be partitioned into disjoint classes. We show that in
a distributed system, to guarantee Byzantine Agree-
ment requires N > 2a+2s+b+r where N is the total
number of processors, a is the number of malicious
asymmetric faults ( a 5 T ) , s the number of malicious
symmetric faults, b the number of non-malicious or
intercepted faults, and r is an algorithm dependent
term. The practical value of this unified model in
designing ultra-reliable systems is demonstrated by
examples.
1 Introduction and Motiva-
t ion
Agreement [1,2] is one of the fundamental problems
in the design of reliable distributed systems. In
these systems, a single processor might be required
to transmit the result of some computation to all
the other processors in the system. If the sending
processor (called the Transmitter) is faulty, it may
send conflicting values to different processors, caus-
ing disagreement among the recipients of the message
(called the Receivers). This situation could arise even
if the Transmitter is non-faulty because the commu-
nication network may be faulty. Such a fault could
result in some of the Receivers taking a certain action
while other Receivers take a different action, leading
to inconsistency or disagreement among non-faulty
processors. This problem was called the Byzantine
93
CH2612-0/88/0000/0093/$01.000 1988 IEEE
Generals Problem by Lamport et al [2]. Any solu-
tion to this problem must satisfy two conditions:
Agreement: All non-faulty Receivers will agree on
the value of the h n s m i t t e r .
Validity: If the h n s m i t t e r is non-faulty, then the
value selected by each non-faulty Receiver will
correspond to the Ifammitters own value.
Together these conditions are referred to as the In-
teractive consistency conditions [l]or the Byzan-
tine Agreement conditions [2]. To guarantee Byzan-
tine Agreement, the total number of processors must
be more than three times the maximum number of
faults [1,2]. The behaviour of faulty nodes (faults)
is not restricted in any way, i.e., no assumptions are
necessary about the failure modes (we refer to this as
the Byzantine Generals model [Z]). It is also known
+
that at least t 1 rounds of messages are required by
any algorithm, where t is the maximum number of
faults [3], unless it is assumed that a faulty processor
cannot forge the signature of a non-faulty processor
(authentication assumption).
While Byzantine Agreement is a desirable feature
to have in ultra-reliable distributed systems, it does
not come without a price. The algorithms are ex-
pensive in terms of the number of messages, in terms
of time, and in the number of processors required.
A t-round algorithm could require O ( N ) messages,
+
and message paths of length up to t 1, where N is
the number of processors. It is this performance and
cost overhead which precludes the use of these algo-
rithms in many real systems. However, by making as-
sumptions, it is possible to devise relatively efficient
algorithms. For instance, the number of processors
required and/or the number of rounds of message
exchange can be reduced by placing appropriate re-
strictions on the failure modes. These restrictions are
usually justified by the architecture and the applica-
tion environment. Given that Byzantine Agreement
is extremely expensive to implement, this paper in-
troduces realistic failure models which still allow ar-
bitrary failures.
 A significant advantage of the Byzantine Gener-
als Model in designing critical fault-tolerant systems
is that it greatly simplifies a Failure Modes and Ef-
fects Analysis (FMEA). This type of fault analysis
is required for certification of systems used in crit-
ical applications where loss of the system may be
too costly. The FMEA procedures are complex and
error-prone because typically a human must system-
atically categorize all the failure modes of the sys-
tem. In general, for complex systems it is impossible
to consider all failure modes ( a combinatorial prob-
lem). Therefore, this approach can result in some
subtle failure modes being excluded from considera-
tion. The Byzantine Generals Model however, makes
absolutely no assumptions about failure modes; thus,
the FMEA process is greatly simplified.
A real design problem is usually constrained in
some way or other (eg. number of processors, pro-
cessor speed, cost, reliability of processors, inter-
processor communications, etc.), regardless of the
specifications or requirements. With this in mind, we
develop a unified theory to allow the designer to build
a system which can take advantage of constrained
fault behaviour, if such constraints are present; but,
unlike other approaches, our models still allow arbi-
trary failures.
In the quest for realistic models (i.e., practical),
several algorithms have been proposed that guar-
antee Byzantine Agreement under limited classes of
faults. However, these algorithms assume that faults
belong to a single class. No attempt has been made
to combine these different types of faults under one
model, except for [4]. Such models are limited in
their application because the system designer must
show that his system cannot exhibit failure modes
not covered by the restricted model: this means that
-all faults are restricted. These models essentially re-
quire that the probability of a failure mode not cov-
ered by the model be zero. A significantly more use-
ful approach is to allow different types of restricted
faults to occur, while limiting the maximum number
of unrestricted faults, in the same model; the prob-
ability of an arbitrary fault may now be non-zero.
This approach can then provide optimal reliability
in the presence of different types of faults.
Since the Byzantine Generals model [2] is expen-
sive to implement, the majority of work in this area
has dealt with restricted scenarios: truly arbitrary
failure modes are not permitted. One such assump-
tion is authentication [2,5] which assumes that a non-
faulty processors signature cannot be forged. Other
approaches limit fault behaviour, for example, omis-
sion faults [5,6] and timing faults [5,4]. To ensure
that failure modes are restricted, it is either neces-
sary to restrict the architecture so that a different
kind of failure mode does not occur, or, to assume
that the probability of other failure modes is negligi-
ble relative to the system specification.
In [5], these restrictions take the form of an
atomic broadcast protocol. Efficient but separate
algorithms are presented to tolerate timing faults,
omission faults, and Byzantine faults (using authen-
tication), respectively. Faults are partitioned into
processor faults and link faults; the communication
network need not be fully connected. A relatively
significant restriction of the atomic broadcast is
the requirement that every message that is broad-
cast be delivered to all non-faulty receivers or to none
of them; this restriction can be enforced with high
probability by constraining the architecture. The al-
gorithms for omission and timing faults cannot cope
with arbitrary faults.
Assuming the existence of a redundant broadcast
network, Babaoglu et al[6] present interactive consis-
tency algorithms to cope with processor faults, chan-
nel faults, and link faults. The redundant broadcast
network and the authentication assumption permit
these algorithms to require only two rounds of mes-
+
sage exchange, rather than the t 1 rounds of most
algorithms. Their results are useful in practice be-
cause of the fault partitioning and because of the
very low message overhead. However, they do not
address the case where a processor may have multi-
ple failure modes.
Meyer and Pradhan [4] have developed a model for
interactive consistency in the presence of two types of
failure modes: arbitrary faults and benign faults
(strictly omission or delay of messages). Their algo-
rithm, while being complex and inefficient, does not
require a fully connected system. We note that our
definition of non-malicious faults which follows, in-
cludes both omission and timing faults, but is not
restricted to these.
In the following section we present our classifi-
cation of faults for distributed systems. Section 3
presents our central result: a model for interactive
consistency in the presence of different types of fail-
ure modes. Section 4 is a brief discussion of the ap-
plicability of the model in designing fault-tolerant
systems.
2 Fault Taxonomy
Our fault classification (Figure 1) is based on the
model of a fully connected distributed system in
which processors communicate by messages. A pro-
cessor can infer the state of the other processors in
 ALL FAULTS
I
I I
NON-MALICIOUS MALICIOUS
I
SYMMETRIC ASYMMETRIC
Figure 1: Fault Partitioning
the distributed system only by receiving messages
(or by the lack of expected messages relative to some
protocol). The effect of a fault within a processor is
completely invisible to the other processors except for
the messages that the faulty processor sends. This
means that from the system perspective, it is mean-
ingful to define faults only in the context and content
of messages transmitted by the faulty processors.
An arbitrary fault or Byzantine fault includes
every type of fault in Figure 1, because no restric-
tions are imposed on faults (i.e., any fault classi-
fication is meaningless). We partition the set of
all possible faults, 7 , into two disjoint subsets:
Non-malicious faults E?, and Malicious faults M. A
Non-malicious or intercepted fault is defined to be a
fault which is detected by every non-faulty receiver
of the message. This detection must, of course, oc-
cur prior to the use of the message in the interactive
consistency algorithm. Examples of Non-malicious
faults include Timing faults [5,4], Omission faults
[5,7] and Crash faults [6]. The case where all non-
faulty processors can detect a fault in a message by
virtue of its contents (for example, out of bounds
data) is also a Non-malicious fault. A Malicious
fault refers to the case when not every non-faulty
receiver can detect that a fault has occurred. It is
these faults that require the use of Byzantine Agree-
ment algorithms with several rounds of message ex-
change. Unlike the current literature, we partition
Non-malicious and Malicious faults into two disjoint
subsets, Symmetric faults S , and Asymmetric faults
A.
Asymmetric faults refer to the case when a mes-
sage is not received identically by all the non-faulty
receivers of that message. If a processor sends a mes-
sage to other processors via different communication
channels, the received messages could differ either
due to the processor lying or due to faults in the
communication network. In either case, if not all
non-faulty receivers receive identical messages (erro-
neous or not) we say that an asymmetric fault has oc-
curred. Some receivers might be able to detect errors
in the message while others receive a valid message:
95
Figure 2: Example of dedicated broadcast: Single
Transmitter and many Receivers
this is also an example of an asymmetric fault. The
distinguishing feature of an asymmetric fault is that
at least one non-faulty recipient receives a different
message.
A Symmetric fault refers to the case when all re-
ceivers obtain exactly the same message. An example
is the case where processor PA received a 1 from
PB, but informs all other processors that it received a
0. The receiving processors do not know what PB
actually transmitted to PA. Symmetric faults may
occur as a result of faulty processors or as a result of
faulty communication channels. The case for defining
a symmetric fault is best expressed by a dedicated
broadcast channel (see Figure 2). In this method
of communication, the channel is dedicated in the
sense that only one processor may transmit on that
channel; all other processors are receivers. Note that
data transfer is strictly unidirectional. This is the
approach taken by ultra-reliable fault-tolerant sys-
tems such as SIFT [8] and MAFT [9,10].The major
advantage of this approach is that the sending pro-
cessor (the Transmitter) is physically limited to send-
ing the same message to all the receivers. Therefore,
the sending processor may commit only a symmetric
fault. Symmetric faults are of course not restricted
to dedicated broadcast networks. In the worst case,
even the communication link (or any communication
device in general) could be a source of asymmetric
faults. However, a communication device (transmit-
ter, receiver, channel) is extremely simple relative to
a processor. Hence the probability of an asymmetric
fault could be much less than the probability of a
symmetric fault; our model permits the system de-
signer to take advantage of this (if it is true in a
particular architecture).
Our fault partitioning is useful because the prob-
abilites of the different types of faults are different
in practice. Non-malicious faults are far more likely
to occur than malicious faults. If only non-malicious
faults occur, the algorithms for Agreement are very
simple and efficient. When a processor is faulty, the
probability that it will generate valid messages is
quite low; the probability of that faulty processor
sending different but valid messages to different re-
ceivers is even smaller (by a valid message we mean a
message that is correct relative to the message proto-
col such as timing, ECC, framing). Thus, in practice,
Pr{Non-malicious fault} > Pr{Malicious Symmetric
fault} > Pr{Malicious Asymmetric fault}.
Also observe that it is possible for a fault to be
symmetric or asymmetric and still be non-malicious.
This partitioning of a non-malicious fault into sym-
metric and asymmetric cases is not shown in Fig-
ure 1, because our unified model is capable of treat-
ing both cases uniformly. A non-malicious asymmet-
ric fault is no worse than a non-malicious symmetric
fault.
3 The Unified Model
We are interested in providing Byzantine Agreement
in the presence of different types of faults. The orig-
inal Byzantine Generals Model [1,2] does not take
advantage of the case where some faults may be ar-
bitrary while others are restricted. By bounding the
number of arbitrary faults, we develop a Unified
model, which takes advantage of the fact that re-
stricted faults do not require N > st, while also al-
lowing for the possibility of arbitrary faults.
Theorem 1 below shows that it is possible to guar-
antee Byzantine Agreement with much less processor
and message overhead if faults can be partitioned
into symmetric faults, asymmetric faults, and non-
malicious faults. In the following discussion, a is
the number of malicious asymmetric faults, s the
number of malicious symmetric faults, b the number
of non-malicious faults (includes both non-malicious
symmetric and non-malicious asymmetric), P is the
number of rounds of message exchange excluding the
initial transmission, and N the total number of pro-
cessors in the system.
We make the following assumptions:
A1 A direct path exists from every processor to
other processors.
all
A2 The absence of a message can be detected by
the intended recipient of that message. We use
a synchronous agreement protocol.
A3 If both the sender and recipient of a message are
non-faulty, then the message is delivered cor-
rectly. A fault in the communication link be-
tween any two processors is treated as a fault of
one of these processors.
The algorithm Z(P), defined inductively for T >_ 0,
solves the Byzantine Agreement problem when there
+ + +
are more than 2a 2s b T processors, in the pres-
+ + <
ence of a s b faults, for a P. Note that P must
96
be fixed prior to execution of the algorithm. In al-
gorithm Z(T), a processor selects the value E if it
detects an error in the message, or if it receives no
message; this value is used in subsequent transmis-
sions (if any). The algorithm uses a majority func-
tion to vote on a vector of values. This majority
function excludes all elements having the value E be-
fore voting; the majority function operates only on
the non-E values. If a majority does not exist among
the non-E values (i.e. there is a tie), a default value
may be chosen, including the value E ; of course this
default must be defined prior to execution of the al-
gorithm. The algorithm presented below is adapted
from Lamport et al 121, except for its use of the E
values.
Algorithm Z(P)
Step 1: The Transmitter sends its value to every
Receiver.
Step 2: For each i, let vi denote the value that Re-
ceiver i gets from the Transmitter; if no message
is received or if an errbneous message is received,
the value E is chosen for vi. If r = 0, then ev-
ery Receiver uses the value vi. Otherwise, every
Receiver acts as the Transmitter in Algorithm
Z(P- 1) to send the value vi to each of the N - 2
other Receivers.
Step 9: For each i, and each j # i, let v j be the
value that Receiver i received from Sender j in
step 2 by using Algorithm Z(P-1); if no message
is received or if an erroneous message is received,
the value E is used for v j . Receiver i uses the
majority value from the set ( v 1 , q ,...,v ~ - ~ ) ,
excluding elements having the value E prior to
the vote.
Lemma 1 FOPany T, any a, any s, andany b, Al-
gorithm Z(T)satisfies the Validity condition if N >
+ + +
2a 2s b P.
Proof: Note that the Validity condition applies
only when the Transmitter is non-faulty. The proof
is by induction on T.
1. Basis step: If P = 0, since the Transmitter is
non-faulty, all non-faulty Receivers must receive
the Transmitters value v by assumption A3.
Thus, the Lemma is true when P = 0.
2 . Induction step: Assuming that the Lemma is
true for T - 1, (P > 0), we show that it is true
for r.
In step 1, the Transmitter of Algorithm Z(P)
sends a value v to each of the N- 1 Receivers and
each of the N - 1 Receivers executes Algorithm
Z(T - 1) in step 2.
+ + +
From the hypothesis N > 2a 2s b T , we
+ + +
have (N - 1) > 2a 2s b (T - 1). Thus,
by the induction hypothesis, we conclude that
every non-faulty Receiver gets v j = v for each
non-faulty Receiver j .
Since each of the N - 1 Receivers acts as the
Transmitter in Z(T - I), each Receiver gets a
vector of N - 1elements after the completion of
Z(T - 1). Each element of this vector can take
on one of three possible values (correct value,
incorrect value, and E). Let k denote the total
number of error values (E) and m the total num-
ber of non-error values ( v j # E) in this vector.
Then, m = N - 1- le.
By hypothesis,
so that
m > 2a + 2s + b + T - 1- k.
Since k 2 b, we can write k = b + a, where
a 2 0. Then,
m > 2a +28 +b + T - 1- ( b + a)
which reduces to
m >2U$2S +T - 1 - a.
Note that the set of a elements (each element
is E) cannot result from malicious symmetric
faults. By definition, the E values can be gen-
erated only by non-malicious faults and ma-
licious asymmetric faults. All non-malicious
faults must generate E values, while all the ma-
licious asymmetric faults need not generate E
values. Therefore, we can write a = a' a and, +
substituting into equation 2 yields
The total number of faulty values among the set
+
of m elements, t , is then equal to a' S. Then,
equation 3 becomesm > Z t + a + r - l . But since
a 2 0 and T > 0, we can write m 2 2t. Thus
a majority of the m values are correct (equal to
v ) . Therefore each non-faulty Receiver selects v
for the Transmitter's value by the majority vote
in step 3 of the algorithm, which satisfies the
Validity condition. 0
L e m m a 2 FOT any T , any s, any b, and a <T,
Algorithm Z ( T ) satisfies the Agreement condition if
+ + +
N > 2a 2s b T .
Proof: If the Transmitter is non-faulty, we have
shown that Algorithm Z ( T )satisfies the Validity con-
dition; in this case Validity implies Agreement. We
only have to prove that the Lemma is true when the
Transmitter is faulty. The proof is by induction.
Basis Step: When T = 0 there cannot be any ma-
<
licious asymmetric faults because a T . Hence
all non-faulty Receivers will receive an identical
message, so that Z(0) satisfies the Agreement
condition.
Induction step: Assuming that the Lemma is
true for T - 1, we show that it is true for T ,
where T > 0.
In step 1, the Transmitter of Algorithm Z ( T )
sends a value v to each of the N - 1 Receivers
and each of the N - 1 Receivers executes Z(T- 1)
in step 2.
If the Transmitter commits a malicious symmet-
ric or a non-malicious fault, all non-faulty Re-
ceivers receive the same value, and by Lemma 1
every non-faulty Receiver gets the same val-
ues for each non-faulty Receiver from Algorithm
Z(T - 1). Hence the algorithm Z ( T )satisfies the
Agreement condition if the Transmitter commits
a malicious symmetric or a non-malicious fault.
When the Transmitter commits a malicious
(2) asymmetric fault, there is one less malicious
asymmetric fault and one less processor in Algo-
rithm Z(T - 1) than in Algorithm Z ( T ) ,but the
total number of malicious symmetric and non-
malicious faults is the same in both Algorithms.
+ + +
Hence, from the hypothesis N > 2a 2s b T ,
+
a 5 T , we have ( N - 1) > 2 a + 2 s b + (T - 1) >
2(a-1)+2s+b+(r-1) and (a-1) 5 ( T - 1 ) . w e
can therefore apply the induction hypothesis to
conclude that every non-faulty Receiver gets the
same value in step (3). Thus, every non-faulty
Receiver selects the same value which is the ma-
jority value of m non-error values (m= N - le,
where m and k are defined in Lemma 1). Thus,
Algorithm Z ( T )also satisfies the Agreement con-
dition. 0.
The central result of this paper is expressed in the
following Theorem.
T h e o r e m 1 FOTany T , any s, any b, and a <
T,
Algorithm Z ( T ) satisfies Interactive Consistency if
+ +
N > 2a -t 2s b T .
91
 Proof: By Lemma 1 amd Lemma 2 , both the con-
ditions necessary for Interactive Consistency are sat-
isfied. Therefore, the Theorem is true. 0
We now consider some special cases that follow
from Theorem 1. The corollaries can easily be proved
and are essentially special cases of Theorem 1;we do
not supply the proofs of these corollaries here.
Corollary 1 For any T , any s, and any a 5 r , al-
gorithm Z ( T ) guarantees Interactive Consistency if
+ +
N > 2a 2s r .
The above corollary is useful if all faults are con-
sidered to be malicious; there are no non-malicious
faults. Although Theorem 1 above may be used
(b = 0), we differentiate this case because now there
is no need for a special E value when there are no
non-malicious faults. Algorithm Z ( T ) is modified so
that there is no E value corresponding to a non-
malicious fault, and the majority vote does not ex-
clude any elements; let this modified Z ( r ) be called
algorithm Z(r). Note that Z(r) is identical to the
well known algorithm O M ( r ) of [2].
We now consider the case when there are no ma-
licious symmetric faults (s = 0). Then, the Unified
model reduces to a form which is essentially equiva-
lent to that of [4], whose benign faults are a subset
of our non-malicious faults. However, our algorithm
Z ( T ) is far more efficient than that provided by [4],
but we do not consider arbitrary connectivity as they
do. To obtain their result, we set T = a and s = 0 in
Theorem 1, which then leads to the following corol-
lary.
Corollary 2 For any a and any b, Algorithm Z ( a )
satisfies the Interactive Consistency conditions, if
N > 3a+b.
The following corollary shows that the Unified
model reduces to the result N > 3r of Lamport et
al [2] if b = 0, s = 0, and a = T , where T is the
maximum number of faulty processors.
Corollary 3 If N > 3r for any r , where T is the
mazimum number of faulty processors, then algo-
rithm Z(T)satisfies the Interactive Consistency con-
ditions.
4 Applications
In this section we briefly examine the practical ad-
vantages of the unified model in estimating system
reliability and in the design of fault-tolerant systems.
In the discussion below, the expression BG model
98
Model NI [ P(Fai1ure) I Faults I
BG I 4 I 6.0 x lo- I 1 arbitrary
BG I 6 I 1.5 x lo- I 1 arbitrary
UM I 4 I 6.0 x lo- I . . b = 0, s = 0
1 arbitrary,
UM 5 1.0 x lo- 1 arbitary, b = 1, s = 0
UM 6 2.0 x lo- 1 arbitary, b = 0, a = 1
UM 6 1.1x 1 arbitary, b = 2, s = 0
Table 1: Reliability data for Example 1
refers to the model in which no assumptions are made
about fault behaviour, i.e., all faults are arbitrary.
This example illustrates that the BG model actu-
ally imposes constraints on the system designer that
may be unreasonable in some cases. Consider the
case where all faults are arbitrary. Assume that a
1-round algorithm is used, so that at most one fault
can be tolerated. At least four processors are nec-
essary, and the occurrence of a second fault is as-
sumed to cause system failure. Now suppose that
the designer wishes to increase the system reliability
by using more processors. If the BG model is used,
then, increasing the number of processors from four
to either five or six, will only decrease the system re-
liability because there are more components to fail.
Increasing the number of rounds to T = 2 will not
help if the number of processors is either five or six.
If N is the total number of processors, and X the fail-
ure rate of a processor, the system failure probability
at time t , for T = 1, is approximately (for At << 1)
Pr(SystemFai1ure) x ( N ( N - 1)Xat)/2
which increases as N. The probability of failure at 1
hour is shown in Table 1,for systems with four, five,
and six processors, respectively (UM in the table
refers to the Unified Model). The processor failure
rate was assumed to be 1.0 x lo- per hour.
If the BG model is used, the system designer can-
not increase reliability simply by using additional
processors. It is necessary to change the algorithm
and also to increase the number of processors in steps
(as the above example demonstrates, increasing from
four to five or six processors is of no benefit). In-
creasing T results in an exponential increase in the
number of messages, resulting in severe performance
penalties.
Even if 7 processors are used with an r = 2 algo-
rithm, the problem does not entirely go away. The
7-processor system can tolerate at most two faults.
Now suppose that a fault has occurred, so that there
are six non-faulty processors. From the standpoint
of system reliability it would be best to degrade the
system to four nodes and use an T = 1 algorithm.
However, under the BG model, it is not possible in
general to tell which processor is faulty. But we have
shown above that it is preferable to use 4 processors
and not 6 processors if only one fault can be toler-
ated. Thus, the BG model cannot provide optimal
reliability, in the sense that an increase in the number
of processors need not always result in an increase in
reliability.
An interesting question arises: since the five and
six-processor systems cannot tolerate any more arbi-
trary faults than the four-processor system, is there
any way of increasing the system reliability by in-
creasing the system size to five or six processors?
The Unified model allows the designer to linearly
increase the number of processors to obtain increased
reliability, without increasing the number of rounds.
Since r = 1, the model states that N > 2a 2s b + ++
1, so that a 5-processor system can indeed tolerate
two faults (one malicious asymmetric and one non-
malicious). A 6-processor system can tolerate two
faults (a = 1, 8 = 1, b = 0) or three faults (a = 1,
b = 2, s = 0). Table 1 shows that a significant
increase in reliability is possible using the new model.
We now consider a hypothetical design problem
having the following requirements and constraints:
1. System Failure Probability must be less than
1.0 x lo-' for a 10-hour mission (reliability re-
quirement).
2. Use the minimum number of processors (cost
and space constraint).
3. Interactive Consistency is required (reliability
requirement).
4. Individual processor failure rate is X = 1.0 x lo-'
per hour.
5. The system must tolerate at least one arbitrary
fault.
Item 5 may first appear to be unreasonable; how-
ever this requirement is extremely useful from the
perspective of "certification" and FMEA.
If the BG model were used to solve this problem, it
can easily be shown that at least 10 processors and
three rounds of message exchange (T = 3) are re-
quired. The performance overhead in this case is pro-
hibitively high due to the message overhead ( O ( N ' ) ) .
The failure probability is 2.0 x 10-l'. There are many
models in the literature which can be used if it is as-
sumed that none of the faults is arbitrary: these do
not satisfy the specifications because of item 5 above.
Ideally we would like a model which takes advantage
99
I I
Table 2: Resiliency of a System based on the Unified
Model (minimum number of processors required)
of different types of faults, and does not treat all
faults as arbitrary. Such an approach can result in
less cost and greater performance for a given level of
reliability than the BG model.
The problem can be neatly solved by using our
model, which, for this specific example, states that
+ + +
N > 2a 28 b 1, where a 5 1. The model
offers a choice of configurations to the designer. Us-
ing Markov models [ll]and the SHARPE modelling
package [12], it was found that the design specifica-
tions can be satisfied with six, seven, or eight pro-
cessors. The 6-processor system is based on the oc-
curence of at most one arbitrary fault and two non-
malicious faults; the failure probability in this case
is 1.5 x lo-" at 10 hours. A 7-processor system
is necessary if the designer wishes to consider the
occurence of at most one arbitrary fault, one mali-
cious symmetric fault, and one non-malicious fault;
the failure probability is 3.5 x lo-" at 10 hours. An
8-processor system is required if one arbitrary fault
and two malicious symmetric faults must be toler-
ated, the failure probability being 7.0 x lo-" for the
10-hour mission. Under our assumptions, it is inter-
esting to observe that the BG model with 10 proces-
sors and three rounds of message exchange provides
less reliability than the Unified model using 6,7, and
8 processors.
In Table 2 we show the minimum number of pro-
cessors required to guarantee Interactive Consistency
using the Unified Model, for various combinations of
faults, when T = 1.
5 Summary
A new model has been presented for interactive con-
sistency in the presence of different types of faults;
this model is much more useful in the design of real
systems than the Byzantine Generals model. Our
fault partitioning is unique and covers every possible
type of fault. We believe that our bounds are tight,
and think this can be proven using the results of [3].
We have shown that, if an interactive consistency
algorithm is used, then the ability to tolerate even
non-malicious faults is diminished. If the probabil-
ity of malicious asymmetric faults can be bounded,
then the system reliability is vastly superior to that
predicted by the Byzantine Generals model; the en-
hanced reliability is obtained at a performance and
cost penalty that is much less than the Byzantine
Generals model. Essentially, we have established a
link between the theory and system design under re-
alistic constraints.
6 Acknowledgements
The concept of symmetric and asymmetric faults and
the suggestion to integrate them under the Byzantine
Generals framework is due to A.M. Finn, R.M. Kieck-
hafer, and C.J. Walter. We acknowledge M.C. McE1-
vany and John Wensley for critically reviewing the
document.
References
[l] M. Pease, R. Shostak, L. Lamport, Reaching
Agreement in the Presence of Faults, JACM,
Vol. 27, No. 2, pp. 228-234,April 1980.
[2]L. Lamport, R. Shostak, M. Pease, The Byzan-
tine Generals Problem, ACM hnsactiona on
Programming Languages and Systems, Vol. 4,
No. 3, pp. 382-401,July 1982.
[7] T.K. Srikanth, S. Toueg, Simulating authenti-
cated broadcasts to derive simple fault-tolerant
algorithms, Distributed Computing (Springer-
Verlag), V01.2, pp. 80-94, 1987.
[8] J.H. Wensley et all SIFT: Design and Analysis
of a Fault-Tolerant Computer for Aircraft Con-
trol, Proceedings of the ZEEE, Vol. 66, No.10,
October 1978.
[9] R.M. Kieckhafer, C.J. Walter, A.M. Finn,, P.M.
Thambidurai, The MAFT Architecture for Dis-
tributed Fault Tolerance, ZEEE lfnturactiona
on Computers, Special Issue on Fault-Tolerant
Computing, Vol. 37, No. 4, pp.398-405, April
1988.
[lo] C.J. Walter, R.M. Kieckhafer, A.M. Finn,
MAFT: A Multicomputer Architecture for
Fault-Tolerance in Real-Time Control Sys-
tems,, Proc. ZEEE Real-Time Systems Sympo-
sium, pp. 133-140,December 1985.
[I11 K.S. Trivedi, Probability d Statistice with Reli-
ability, Queuing, and Computer Science Appli-
cations, Prentice-Hall, 1982.
[12]R. Sahner, K. Trivedi, A Hierarchical, Combi-
natorial - Markov Method of Solving Complex
Reliability Models, Pwxeedings A CM/IEEE
Fall Joint Computer Conference, November
1986, Dallas, Texas.

[3] M.J. Fischer, N.A. Lynch, LALower Bound for

the time to assure Interactive Consistency, Zn-
formation Processing Letters, Vol. 14, No. 4, pp.
183-186,June 1982.

[4] F.J. Meyer, D.K. Pradhan, Consensus with

Dual Failure Modes, Seventeenth Znt. Sympo-
sium on Fault Tolerant Computing, July 1987.

[5] F. Christian, H. Aghili, R. Strong, Atomic

Broadcast: From Simple Message Diffusion to
Byzantine Agreement, Fifteenth Znt. Sympo-
sium on Fault Tolerant Computing, pp. 200-206,
June 1985.

[6] 0.Babaoglu, R.Drummond, Streets of Byzan-

tium: Network Architectures for Fast Reliable
Broadcasts, ZEEE Tmnsactions on Software
Engineering, Vol. SE-11,No.6, pp. 546-554,
June 1985.