WDS'06 Proceedings of Contributed Papers, Part I, 124129, 2006.

ISBN 80-86732-84-3 MATFYZPRESS

On Rejewskis Solution of Enigma Cipher

J. V abek
Charles University, Faculty of Mathematics and Physics, Prague, Czech Republic.

Abstract. In this contribution I want to present the extract of the article about Rejewskis solution of Enigma cipher, which is being prepared. The story of Enigma is a part of modern cryptology. However, there still remain some unclear facts about Enigma. The reason is that the information were for a long time classied, now there is a lack of reliable sources. Moreover, some of the reports are even conicting. Main subject is an analysis of the method developed by Rejewski to reveal the internal connections in the machine. His description is not complete, it is impossible to calculate the internal wirings of all three rotors by the method he developed. I present here one of the two new methods how to calculate the connections in the third rotor together with the probability of having suitable data for this method. In original article there is also a detailed analysis of the complexity of Rejewskis method.

Introduction
Rejewskis calculation of the internal wiring of the military Enigma machine in December 1932 is one of the most remarkable events in the history of cryptology. The importance of Enigma and its solution before and during the WWII was described e.g. in [Bauer, 2000; Kozaczuk, 1984]. These sources emphasize the signicance of contribution by Rejewski and also by his colleagues from the Polish military cryptanalytical service during the period between 1932 and 1939. Subsequent history of breaking the Enigma cipher in Bletchley Park during the WWII can be found e.g. in [Singh, 1999] Rejewski himself described his solution in two dierent manuscripts [Rejewski, 1940; Rejewski, 1967] (available only in Polish). Only long after the end of the WWII, in 1980, two papers prepared by Rejewski were published (only posthumously) in English. Probably due to historical circumstances (the rst monograph was prepared in France in 1940, the second one in Poland in 1967), Rejewskis description of his work is rather brief in some aspects. Moreover, long breaks between writing dierent papers (after the WWII Rejewski lived in the then Communist Poland and had in fact to hide his work for the prewar Polish Government as well as for the Polish Government in exile in London in the years 1940-1945) probably contributed to the fact that some moments in the history of solving the Enigma cipher were described in dierent manuscripts in a dierent way. In this paper we summarize the basic steps of Rejewskis reconstruction of the Enigma machine and attempt to ll in some missing points. Some of the information described in Rejewskis Polish manuscripts appear here for the rst time in English. We shall assume familiarity with the operation of the military Enigma machine and its basic operations. Details can be found e.g.in [Kruh, 1985].

Summary of Rejewskis Method

The Begining of The Work The rst Rejewskis task was to uncover the internal wirings in the three rotors and in the reector. He had at his disposal several dozens enciphered messages from each day, later the daily keys from September and October 1932 (obtained through espionage) and also the operation manual (also through espionage) describing the structure of the daily and message keys. Probably already before obtaining the operation manual he discovered that the rst six letters of each enciphered message was the message key enciphered twice as one can conclude from [Rejewski, 1967], which is probably the source of corresponding information also in [Bauer, 2000]. Let us rst denote by the letters A, . . . , F , six permutations dened by the rst six successive positions of the rotors set up according to the daily key. These six permutations were used to double

124

 VABEK: ON REJEWSKIS SOLUTION OF ENIGMA CIPHER

enciphering the three letters of the message key. If we denote by L, M , N the three rotors (from left to right), by R the reector, by H the connections from plugboard to the entry wheel, by S the connections in the plugboard and by P the permutation cyclically rotating the letters of the alphabet, we can express these six permutations through the internal settings of the machine, see [Rejewski, 1980a]: A B C D E F = = = = = = SHP N P 1 M LRL1M 1 P N 1 P 1 H 1 S 1 SHP 2 N P 2 M LRL1M 1 P 2 N 1 P 2 H 1 S 1 SHP 3 N P 3 M LRL1M 1 P 3 N 1 P 3 H 1 S 1 SHP 4 N P 4 M LRL1M 1 P 4 N 1 P 4 H 1 S 1 SHP 5 N P 5 M LRL1M 1 P 5 N 1 P 5 H 1 S 1 SHP 6 N P 6 M LRL1M 1 P 6 N 1 P 6 H 1 S 1 .

(1)

Unfortunately, the permutations A,. . ., F were unknown as well as the permutations H , M , N , L, R, S . However, Rejewski noticed, that he could extract some information from the encrypted message keys: suppose we have intercepted the following indicator dmq vbf Rejewski assumed that this was an encrypted version of a doubled message key xyzxyz . It means that xA = d and also xD = v . But because also dA = x (self-reciprocity), we get by substitution for x from the last expression that dAD = v It means that the image of the letter d by the permutation AD is the letter v . If we have enough enciphered messages from one day (according to Rejewski about 80 messages [Rejewski, 1980b; Rejewski, 1940] or even 60 messages [Rejewski, 1980a], we can reconstruct the whole permutation AD. Permutation Theory Now it is necessary to introduce some denitions and state some known facts from the theory of permutations. Denition 1. The alphabet is the permutation on the set of 26 letters {a, b, c, . . . , x, y, z }, which comprises of 13 disjunctive cycles of length 2 (transpositions). Remark 2. The term alphabet uses Turing [Turing, 1940] mostly for the whole permutation, which denes, how each of the letters will be enciphered in one particular setup of the machine. Denition 3. The characteristic is the permutation on the set of 26 letters {a, b, c, . . . , x, y, z }, which is composed from two alphabets. Remark 4. Rejewski uses in his texts the expressions characteristic of a given day only for three permutations AD, BE and CF . Lemma 5. Two permutations X and Y are conjugated (i.e. there exists Z , such that X = ZY Z 1 ) if and only if they are of the same type (i.e. they have the same cyclic type). From the proof of this well known lemma we get also an easy recipe, how to construct the permutation Z , which conjugates permutations X and Y provided they have the same cyclic type. It is enough to subscribe the cycles of the permutation Y beneath the cycles of the same length of the permutation X and we can directly see the mapping Z - image of each letter is right beneath. This also shows us, how many such permutations Z we can construct and that the number depends on the cyclic type of the permutations X and Y . We denote by ki the number of cycles of length i in permutation X (or Y ). Then the number of possible permutations Z is:
n

iki ki ! ,
i=1

where n is maximum length of cycles

(2)

This is because there are ki ! ways of ordering cycles of length i and each such cycle can be written in i dierent ways. The important theorem on product of two alphabets follows (this theorem and its proof can be found in [Rejewski, 1980b]):

125

 VABEK: ON REJEWSKIS SOLUTION OF ENIGMA CIPHER

Theorem 6. For a permutation A there exist two alphabets X and Y such that A = XY , if and only if A contains an even number of cycles of each length. From this theorem and denition 3 we can see following straightforward corollary. Corollary 7. Each characteristic contains an even number of cycles of each length. Rejewski was then able to decompose the characteristics AD, BE and CF into the alphabets . This decomposition is not unique, but thanks to stereotypical selection of message keys by operators Rejewski was able to choose the right one (details can be found e.g. in [Rejewski, 1980a; Rejewski, 1967]). Simplifying The Equations There were still too many unknowns in the equations. At this point Rejewski got help from espionage as it was mentioned before. From the tables of daily keys he obtained the permutation S and the starting positions of the rotors for each day. Another problem was the permutation H . Rejewski found out that if he knew the connections to the entry wheel, the system of equation would be solvable. First he supposed, these connections are the same as in a commercial model (the letters were wired to the entrance wheel in the order of keys on the Enigma keyboard), but after many computations it gave no results. Rejewski then tried the lucky guess, that maybe the letters were wired to the entrance wheel in alphabetical order (thus H was identical permutation) and this guess showed up as the right one. Then we can continue as follows. Let us rst dene the permutation Q to shorten the equations. Q = M LRL1M 1
def

(3)

Now we can transform the set of equations 1, using the substitution 3, in such way that we put as much as possible known permutations on the left sides of the equations. Let us denote these left sides by the letters U ,. . .,Z : U V W X Y Z
def

P 1 H 1 S 1 ASHP P 2 H 1 S 1 BSHP 2 P 3 H 1 S 1 CSHP 3 P

4

= N P 1 QP N 1 = N P 2 QP 2 N 1 = N P 3 QP 3 N 1 = N P 4 QP 4 N 1 = N P 5 QP 5 N 1 = N P 6 QP 6 N 1 (4)

def

def

def

DSHP

def

P 5 H 1 S 1 ESHP 5 P 6 H 1 S 1 F SHP 6

def

Now we multiply each two successive equations: UV VW WX XY YZ = = = = = N P 1 QP 1 QP P N 1 N P 2 QP 1 QP P 2 N 1 N P 3 QP 1 QP P 3 N 1 N P 4 QP 1 QP P 4 N 1 N P 5 QP 1 QP P 5 N 1

(5)

From these equations we can eliminate system: UV = VW = WX = XY =

the common expressions QP 1 QP and obtain the following N P N 1 V W N P 1 N 1 N P N 1 W XN P 1 N 1 N P N 1 XY N P 1 N 1 N P N 1 Y ZN P 1 N 1

(6)

Now we can see that the permutations U V , V W , etc. are conjugated by N P N 1 . From each of the equations 6 we can get typically several dozens solutions for N P N 1 and there is almost always only one common solution for all these equations. Thus we have N P N 1 , which is conjugated with P by N . Therefore we have 26 solutions for N . These solutions are in fact close to each other, they dier only in the torsion of two sides of the rotor.

126

 VABEK: ON REJEWSKIS SOLUTION OF ENIGMA CIPHER

The third rotor

Rejewski and third rotor As it was said, Rejewski had only the daily keys from two months, September and October 1932. It was the time when the order of the rotors was changed quarterly. Since September and October belong to two dierent quarters and the right rotors were dierent in these two months, Rejewski was able to nd the connections in two dierent rotors by the same method described above. No further details are known about reconstructing the connections in the third rotor and the reector, or about the precise order of the rotors in these two months. How did Rejewski nd out the connections in the third rotor? We state here Rejewskis comments about these calculations from dierent reports. These reports dier from each other or in some cases even slightly contradict each other. In [Rejewski, 1940] Rejewski wrote that the revealing of the connections in the third rotor brought nothing new and an authentical example of enciphered text helped him. In [Rejewski, 1967] Rejewski speaks about having enough enciphered material. In [Rejewski, 1980a] Rejewski refers again to the authentical example of encryption. Finally, in [Rejewski, 1980b] Rejewski actually denies the fact that he had only two sets of daily keys from two months in which only two rotors occurred as the fast right rotor, but this variant seems to be less probable. Because of this information and the fact that the method of recovering the connections in the third rotor and the reector is not explicitly described in Rejewskis memoirs, we proposed in original article two dierent methods (here only one) how to compute them. These methods are based on the same ideas from the theory of permutations that Rejewski used to calculate the connections in the fast right rotor. Whether Rejewski in fact used some of our methods will probably remain unknown for ever. Computation of the third rotor Let us denote again by letters L, M , N the permutations dened by the three rotors from the left to right in the order used in one of the two months. Then we can list in Table 1 all possible orders of the rotors in both months, from which we have the sets of daily keys (we take into account the fact that two dierent rotors appeared as the fast right rotor in these two months). We can apply the rst method in the situation, when the unknown rotor is on the left in at least one of the two months. Thus we can use it in the situations a), b) and c). In what follows we use the method for the situation a) and the order of rotors (from the left to the right) L, M , N . From the month when the unknown third rotor appears as the left one we need data from two days, for which the dierence of the positions of the left rotor is odd and dierent from 13. Let us denote the dierence between these two positions by v , where 0 v 25. Suppose we have alphabets from two such days. We denote the alphabets from the rst day by Ai and the alphabets from the second day by Bi , for i = 0, 1, . . . , 25. The permutation dened by the left rotor is L in the rst of these two days and P v LP v in the other. From the rst day we have (for our calculations only three alphabets are enough) A0 A1 A2 and from the second day: B0 B1 B2 = = = P s N P s P r M P r P v LP v RP v L1 P v P r M 1 P r P s N 1 P s P s+1 N P s1 P r M P r P v LP v RP v L1 P v P r M 1 P r P s+1 N 1 P s1 P s+2 N P s2 P r M P r P v LP v RP v L1 P v P r M 1 P r P s+2 N 1 P s2 . (8) = = = N M LRL1M 1 N 1 P N P M LRL1M 1 P N 1 P 1 P 2 N P 2 M LRL1M 1 P 2 N 1 P 2 (7)

Table 1. Possible orders of the rotors. a) b) c) d) 1st month LM N LM N LM N LM N 2nd month LN M N LM MNL NML unknown rotor L L M M

127

 VABEK: ON REJEWSKIS SOLUTION OF ENIGMA CIPHER

For i = 0, 1, 2 we calculate R from the equation 8 for Bi and substitute it to the corresponding equation for Ai in 7. We get A0 A1 A2 = = = N M LP v L1 P v P r M 1 P r P s N 1 P s B0 P s N P s P r M P r P v LP v L1 M 1 N 1 P N P 1 M LP v L1 P v P r M 1 P r P s+1 N 1 P s1 B1 P s+1 N P s1 P r M P r P v LP v L1 M 1 P N 1 P 1 P 2 N P 2 M LP v L1 P v P r M 1 P r P s+2 N 1 P s2 B2 P s+2 N P s2 P r M P r P v LP v L1 M 1 P 2 N 1 P 2 .

(9)

To simplify the equations 9 we denote by Qi for i = 0, 1, 2 the following expressions Q0 Q1 Q2 = = = P v P r M 1 P r P s N 1 P s B0 P s N P s P r M P r P v P v P r M 1 P r P s+1 N 1 P s1 B1 P s+1 N P s1 P r M P r P v P v P r M 1 P r P s+2 N 1 P s2 B2 P s+2 N P s2 P r M P r P v . (10)

We get the equations A0 A1 A2 = = = N M LP v L1 Q0 LP v L1 M 1 N 1 P N P 1 M LP v L1 Q1 LP v L1 M 1 P N 1 P 1 P 2 N P 2 M LP v L1 Q2 LP v L1 M 1 P 2 N 1 P 2 . (11)

Now we move as many known permutations as we can to the left side: M 1 N 1 A0 N M M 1 P N 1 P 1 A1 P N P 1 M M 1 P 2 N 1 P 2 A2 P 2 N P 2 M = = = LP v L1 Q0 LP v L1 LP v L1 Q1 LP v L1 LP v L1 Q2 LP v L1 . (12)

and multiply the tow pairs of subsequent equations. We get M 1 N 1 A0 N P N 1 P 1 A1 P N P 1 M M 1 P N 1 P 1 A1 P N P N 1 P 2 A2 P 2 N P 2 M = LP v L1 Q0 Q1 LP v L1 = LP v L1 Q1 Q2 LP v L1 . (13)

From these two equations we can compute LP v L1 and then L in the same way as Rejewski calculated the permutation N . The number of solutions for L depends on v . Because of our assumption v odd and dierent from 13 the permutation P v contains only one cycle of length 26 which gives us only 26 possible solutions for L. We excluded the case v = 13 since then he permutation P v would have 13 cycles of length 2 which would gave us 213 13! dierent solutions for L and it would be far more dicult to choose the right one. In calculations of L we have never used the assumption that v is odd. In fact we could proceed in the same way even in if v was even. In such a case we would obtain 2 132 = 338 solutions for L, since P v has 2 cycles of length 13 if v = 0. Computation of The Probability Suppose the daily keys for each day from the month we used were selected independently and randomly. Thus the probability of each of the 26 possible positions of the left rotor is 1/26 in each of the days of the month. What is the probability of nding two days from the month such that the dierence of the positions of the left rotor is odd and dierent from 13? It depends of course on from how many days of the month we have enough encrypted messages. Let us denote the number of suitable days by n. We assume n 2. Let us denote by pn the probability that we nd among these n days two days when the dierence between the positions of the left rotor is odd and dierent from 13. In what follows we compute the number qn = 1 pn , which is the probability that in arbitrarily chosen two days out of our set of n days the dierence of the positions of the left rotor is always even or 13. We will proceed by induction on n. In the next part we denote by a natural number x {0, 1, . . . , 25} the position of the left rotor. We introduce the following denition: Denition 8. A bad sequence of length n is a sequence x1 , x2 , . . . , xn of positions of the left rotor in n days such that there is no pair of two days among these n days where the dierence between the positions of the left rotor in these two days is odd and dierent from 13. If we have a bad sequence x1 , x2 , . . . , xn1 of length n 1, then we call a position xn for the n-th day bad if the sequence x1 , x2 , . . . , xn is also bad. By sn we denote the number of bad sequences of length n. Then we have qn = sn /26n .

128

 VABEK: ON REJEWSKIS SOLUTION OF ENIGMA CIPHER

If n = 2 then for any position x1 there are 14 bad positions x2 for the second day. Thus s2 = 26 14 and q2 = 7/13. In general for n 3 we have the following situation. We denote the position of the left rotor for the i-th day by xi . Then the probability qn is the ratio of the number of the bad sequences x1 , x2 , . . . , xn to the number of all possible sequences. We will compute qn by an induction on n. Suppose we already know qn1 . We divide all possible bad sequences of length n into three groups: 1. If |x1 x2 | = 0, then we can identify these two days since the positions of the left rotor in these two days are the same. Then we have a bad sequence x2 , . . . xn of length n 1. Therefore the number of those bad sequences of length n where x1 = x2 is sn1 . 2. If |x1 x2 | = 13 (there are 26 possibilities for the pair x1 , x2 in this case), then there are only two bad positions out of all 26 possible positions for every i, i = 3, . . . , n, namely those that are the same as x1 or x2 . In all other cases the dierence between the position xi , i = 3, . . . , n and one of the positions x1 , x2 is odd. Thus the number of bad sequences of this type and of length n is 26 2n2 . 3. If |x1 x2 | is even and nonzero (there are 26 12 possibilities for the pair x1 , x2 in this case), then there are 13 bad positions out of all 26 possible positions for every i, i = 3, . . . , n. It follows from the obvious fact that if for example the dierence |xi x1 | = 13, then the dierence |xi x2 | is again odd and dierent from 13. Thus the number of bad sequences of this type and of length n is 26 12 13n2 . Therefore we get sn1 + 26 2n2 + 26 12 13n2 qn1 6 1 = + (14) + n n 2 26 26 26 13 13 2n2 The values of pn for 2 n 15 can be found in Table 2, for n 16 the value of pn 99, 995. qn = Table 2. The probability of having suitable data with respect to the number of days, from which we have enough enciphered material. n pn n pn 2 46.15 9 99.61 3 74.56 10 99.80 4 87.46 11 99.90 5 93.75 12 99.95 6 96.87 13 99.98 7 98.44 14 99.99 8 99.22 15 99.99

References
Bauer F. L., Decrypted secrets: Methods and Maxims of Cryptology, 2nd edition. Springer-Verlag, Berlin, 2000. Kozaczuk W., Enigma: How the German Machine Cipher Was Broken, and How It Was Read by the Allies in World War Two, Edited and Translated by Christopher Kasparek. Frederick, Maryland: University Publications of America, Inc, 1984. Deavours, Cipher A. and Louis Kruh, Machine Cryptography and Modern Cryptanalysis, Norwood, MA:Artech House. See Chapter III, 1985. Rejewski M., How the Polish Mathematicians Broke Enigma. Reprinted in [Kozaczuk, Appendix D, 1984], 1980a. Rejewski M., An Application of The Theory of Permutations in Breaking The Enigma Cipher, Applicationes Mathematicae 16, No. 4, Warsaw. Available at http://mad.home.cern.ch/frode/crypto/rew80.pdf, 1980b Rejewski M., Enigma (1930-40), Method and History of Solving The German Machine Cipher. Unpublished manuscript (in Polish). Available at http://www.spybooks.pl/en/enigma.html, about 1940. Rejewski M., Memories from My Work in Cipher Bureau Section II of General Sta in the years 1930-1945. Unpublished manuscript (in Polish). Available at http://www.spybooks.pl/en/enigma.html, 1967. Turing Alan M., Turings Treatise on Enigma. NARA College Park, Maryland, Record Group 457, Historic Cryptographic Collection, Box 201, NR 964. This document is being edited by Frode Weirud, Ralph Erskine and Philip Marks for publication on Frode Weiruds website. Several chapters have been published already at: http://frode.home.cern.ch/frode/crypto/Turing, 1940. S. Singh, The Codebook, The Evolution of Secrecy from Mary Queen of Scots to Quantum Cryptography, Doubleday, New York, 1999.

129