In the course of WWII both the Allies and the Axis powers were able to gain information of great value from reading their enemies secret communications. In Britain the codebreakers of Bletchley Park solved several enemy systems with the most important ones being the German Enigma and Tunny cipher machines and the Italian C-38m. Codebreaking played a role in the Battle of the Atlantic, the North Africa Campaign and the Normandy invasion.
In the United States the Army and Navy codebreakers solved many Japanese cryptosystems and used this advantage in battle. The great victory at Midway would probably not have been possible if the Americans had not solved the Japanese Navy’s JN25 code.
Britain, the Soviet Union and the United States did not have impenetrable codes. In the course of WWII all three suffered setbacks from their compromised communications. One of the worst failures of US crypto security was the extensive compromise of State Department communications in the period 1940-44.
The Cryptographic Section of the Division of Communications and Records
Since its creation in 1789, the Department of State has been responsible for promoting U.S. interests in the world and implementing US foreign policy.
During WWII communications between the State Department and the US embassies and consulates around the world were encrypted using various systems. Up to late 1944 the unit responsible for the production and distribution of codes and ciphers was the Cryptographic Section of the Division of Communications and Records, headed by David A. Salmon(1).
This unit prepared codebooks and cipher keys but did not have the means to properly evaluate the security of the various cryptosystems or ensure that they were used properly.
State Department codes and ciphers 1939-1944
The cryptosystems used by the Department up to late 1944 were letter codebooks (both enciphered and unenciphered), the M-138-A strip cipher and the cipher machines SIGABA (Converter M-134-A) and HCM (Hebern 5-rotor type) (2).
The codebooks Gray and Brown were used for messages classified ‘Restricted’.
Gray was a 5-letter one-part codebook of 68.000 values, introduced in 1918. Brown was a 5-letter two-part codebook with 124.000 values, introduced in 1938. Most of the time these codebooks were used without additional encipherment.
When the Gray and Brown codes were used with enciphering tables they were called ‘Special Gray’ and ‘Special Brown’.
The codebooks A1, B1, C1, D1 were used for messages classified ‘Confidential’. These were 5-letter two-part codebooks, introduced in 1919, 1922, 1927 and 1934 respectively. A1 and B-1 had 114.000 values, while C-1 and D-1 had 60.000.
The codebooks were always used with enciphering tables, since they transmitted highly confidential information.
In the 1930’s the US military introduced the M-138-A cipher as a new high level system. The M-138-A was based on the same cryptographic principles as the older M-94 cylinder/disk cipher but it was much easier to produce and use since it was made up of paper alphabet strips instead of the metal disks used on the M-94. The M-138-A was used extensively by the US Army and Navy in the late 1930’s and in WWII.
Example of the M-138-A board and alphabet strips (3):
In the late 1930’s the State Department also adopted the M-138-A as a high level system. The cipher was used for messages classified ‘Secret’.
Each embassy had 50 ‘special’ alphabet strips and 50 ‘circulars’. The ‘specials’ were used for direct communications between that embassy and Washington. The ‘circulars’ were used for communications between embassies and for messages sent from Washington to more than one embassy.
The way the system worked was that each day 30 alphabet strips were chosen out of the available 50 (both for the ‘circulars’ and the ‘specials’). The strips used and the order that they were inserted in the metal frame were specified by the ‘daily key’.
In the period 1939-1944 the State Department relied almost entirely on hand ciphers. Cipher machines were only used by the US embassy in London and later on by the embassies in Latin American countries (Mexico City, Panama, Bogota, Lima, Santiago, Buenos Aires, Montevideo, Rio) and in Moscow (4).
The embassy in London had the early version of the SIGABAcipher machine, the Converter M-134-A model.
The embassies in Latin America and in Moscow had the 5-rotor Hebern cipher machine, loaned to the State Department by the US Navy.
Axis exploitation of State Department codes and ciphers
Germany, Italy, Japan and Finland were able to read classified State Department messages during the war both through cryptanalysis and by physical compromise. Unfortunately many aspects of the Axis effort are vague; however the available information points to a serious compromise of most of the State Department cryptosystems.
In 1941 members of a special unit called the Extraction Section (Sezione Prelevamento) were able to enter the US embassy in Rome and they copied the Military Intelligence Code No11used by military attaches. By having the MI Code No 11 they could read the communications of US military attaches from important embassies such as Cairo, Egypt and Moscow, Soviet Union.
It is reasonable to assume that they also copied diplomatic codebooks, however details are lacking. According to postwar reports the Italian codebreakers had a copy of the Brown codebook (5).
Japanese effort in 1940-1941
Since the 1920’s the codebreaking departments of the Japanese Army, Navy and Foreign Ministry intercepted and solved US diplomatic traffic.
In 1939 or 1940 agents of the military police Kenpeitai gained access to the US consulate in Kobe and they copied cipher material (including alphabet strips for the M-138-A system) (6).
The Japanese were able to get copies of the codebooks A1, Gray, Brown and the M-138-A strips 9-1, 10-1, 18-1, 0-1 (together with their daily key table and numerical key) (7).
This material gave them access to the communications of the US ambassador Joseph Grew and especially important were the messages concerning US policy regarding Japan (8). Messages from other US embassies were also read by using the 0-1 circular strips (9).
German effort in 1940-1942
Foreign diplomatic codes and ciphers were worked on by three different German agencies, the German High Command’s deciphering department - OKW/Chi, the Foreign Ministry’s deciphering department Pers Z and the Air Ministry’s Research Department - Reichsluftfahrtministerium Forschungsamt.
The German codebreaking agencies successfully solved the diplomatic cryptosystems of many countries. In the late 1930’s and early part of WWII their emphasis was on the systems used by France, Poland, Britain and the minor European nations.
US diplomatic traffic was monitored and solved but it was not a main target due to limited role of the United States in international affairs. From 1941 more resources were put on the solution of US codes since the US government followed a more interventionist policy.
At the High Command’s deciphering department - OKW/Chi, US diplomatic systems were worked on by a subsection of main Department V. This unit was headed by Senior Councilor (Oberregierungsrat) Nikolai Rohen. Other important members of the US section were dr Franz Weisser and Councilor Schulz (10).
The unit read the Military Intelligence Code, the War Department Confidential Code, the War Department Telegraph Code and the diplomatic systems Gray, Brown, A1, C1 and M-138-A (11).
Work on difficult cipher procedures (like the M-138-A) was also carried out at the mathematical research department (department IV of OKW/Chi). The person in charge of strip cipher research was the mathematician Wolfgang Franz.
In the summer of 1941 a Japanese mission headed by Colonel Tahei Hayashi, former head of the Japanese Army’s cryptologic agency, visited Germany and exchanged US and British codes with systems solved by the Germans (12). The Germans got a copy of the Brown and A1 codebooks and also the M-138-A strips (9-1, 10-1, 18-1, 0-1 with their daily key table and numerical keys) and the instructions for use of the M-138-A system. This material allowed the Germans to immediately start exploiting strip cipher traffic and of course to read fully all Brown messages upon interception.
At the Foreign Ministry’s deciphering department Pers Z US codes and ciphers were worked on by the ‘American and Scandinavian language group’, headed by Technical Assistant Karl Zastrow.
Additional research on complex ciphers was carried out at the ‘Mathematical Cryptanalytic Subsection’, headed by dr Werner Kunze. Their M-138-A expert was Professor Hans Rohrbach.
According to German reports the main systems used were the Gray and Brown codes, followed by the strip cipher and the enciphered codebooks. Gray had been copied in the past and the Brown code was worked on successfully so that by 1941 about 80% of the messages could be read. The enciphered codebooks A1 and C1 were also worked on with success and code values recovered through cryptanalysis. The M-138-A traffic was investigated in early 1940 but it was judged to be too difficult to deal with by Pers Z (13).
Pers Z reports for 1940 and 1941 show that most of the US traffic was sent on the Gray and Brown codes, followed by the strip cipher (14):
Note that B3=Gray, B8=Brown, SV (Streifenverfahren)=M-138-A, B6a=A1, B7=C1.
Pers Z also received the Japanese material given to OKW/Chi:
In the period 1941-42 the Germans were in a favorable position versus US systems since they had copies of the widely used Gray and Brown codes and work on the enciphered system A1 was made much easier since they no longer had to recover the book’s code values.
Regarding the strip cipher it is clear that they could read current traffic, as the circular strips 0-1 were valid till August 1942. Moreover by having examples of the strips, daily tables, numerical keys as well as the instructions they could bypass the difficult initial phase of a cryptanalytic attack. Work on the M-138-A cipher was split between OKW/Chi and Pers Z, as OKW/Chi attacked the ‘special’ traffic while Pers Z worked on the ‘circular’ strips (15).
Details of their success with the strip systems in 1941-42 are hazy.
The circular strips 0-1 were replaced in August 1942 with the 0-2 set. The new set was solved by Pers Z but it is not clear of this was achieved in late 1942/early 1943 or late 1943 (16).
At OKW/Chi the mathematician Wolfgang Franz developed theoretical methods of solution which however did not have to be used as simpler attacks were possible (reuse of alphabet strips, reencodements between strip systems, reencodements between codebooks and strip systems, stereotypical beginnings etc). It seems that by early 1943 he was able to solve the ‘special’ strips used by the US embassy in Bern, Switzerland (17).
Finland’s codebreaking agency was established in 1927 and its main effort was directed against the communications of the Soviet Union. However foreign diplomatic systems were also worked on, as far as limited resources could allow.
In WWII the department dealing with diplomatic traffic was based in Mikkeli. Head of the diplomatic department was Mary Grashorn. Other important people were Pentti Aalto (effective head of the US section) and the experts on the M-138 strip cipher Karl Erik Henriksson and Kalevi Loimaranta. The diplomatic section had 38 cryptanalysts in 1944, with the majority working on US codes (18).
In early 1942 the Finns received from the Germans the Japanese material and especially the M-138-A strips and keys (19). During the second half of the year they were able to use this material in order to break into the traffic of several US embassies by taking advantage of mistakes in the use of the strip cipher by the State Department.
When examining the cipher messages of various US embassies the Finnish codebreakers discovered that the same alphabet strips were reused. When one embassy stopped using them they were not discarded but instead they were sent to another embassy to be used there. Also the same daily tables and numerical keys were used by all embassies for their ‘special’ strip traffic. The Finns also used reencodements of the same message on the 0-1 circular strips in order to break into ‘special’ traffic. Sometimes the same message would be enciphered on the 0-1 circular strips and sent again on an embassy’s ‘special’ strips. Since the Finns had the 0-1 strips and could read the message this was a clear case of cleartext-ciphertext compromise (20).
In 1943 the State Department continued to use the codebooks Gray, Brown, A1 and C1 that could be exploited by the Axis powers.
Gray and Brown could be read fully. The substitution tables used with the codes A1 and C1 were solved by the Germans, with dr Erich Huettenhein(chief cryptanalyst of OKW/Chi) stating that ‘it took about two months to build up tables, so that we could only read the traffic currently during the last month of the period’ (21). Prior to July 1942 the codebooks were enciphered using digraphic substitution tables. After July ’42 monographic tables were used. These were composed of 10 pages with 20 alphabets per page. The D1 codebook seems to have been withdrawn from use (22).
The M-138-A system was also exploited to a considerable degree by the Germans. Unfortunately there are few details on the strips that they solved or the traffic that they could read. However the available information from various sources points towards considerable success both with the ‘special’ strips and the ‘circulars’.
Dr Huettenhein stated that ‘most of the American strip cipher was read’ and dr Franz claimed that his unit solved 28 strip sets plus 6 numerical keys (23). Considering the improvement of State Department cryptosecurity that took place in late 1943 and throughout 1944 it seems that these statements referred to their successes in the year 1943.
In early 1943 the Germans were certainly reading the ‘special’ strips of the Bern embassy and it is reasonable to assume that during the year they also had access to the messages of the other US embassies in European and Middle Eastern countries. At the time the M-138-A cipher mostly used either a straight board system (meaning 30 cipher letters chosen from one column) or the split board system (meaning 15 cipher letters chosen from one column and the other 15 chosen from a different column) (24). Both these procedures were insecure and could be solved by the German codebreakers.
Their success with the strip cipher led them to design and build special cryptanalytic equipment that could speed up the solution.
At OKW/Chi they built a device called ‘Tower Clock’ (Turmuhr). The device was used when ‘cribbing’ was not possible and worked according to the following principle (25):
‘The machine found message parts with the same generatrices by the following method - having established a depth by finding repeats, the letter frequencies for each column were weighted. The correct generatix for additional message parts was then found by totaling these weights. The machine gave the result as a needle graph. It ticked after each set of calculations; hence the name Turmuhr. OKH also used the machine. It could not be used after the system was modified by the withdrawal of strips.’
At Pers Z a special device was built for rapid decoding of messages, after the alphabet strips had been recovered. It was called the ‘Automaton’ and it allowed a small number of workers to decode a large volume of past traffic. A description is available from TICOM report I-89 (26):
‘The decipherment of a double line consists of two operations: (1) arranging the strips so that the cipher text letters are made to lie in a row, (2) selecting the line containing the true reading out of 25 parallel lines. The adjustment of the strips that move up and down, so that the true reading can be read horizontally, is accomplished by the machine quite automatically. The cipher text may be touched by hand on the keyboard of a typewriter, or be taken by means of a sounding device from the Hollerith cards that had already been punched. Finding the true reading is simplified by the fact that the letters on the strips are printed in two different weights, the most frequent letters in the English language (about 80% of true reading) are printed in a heavy tone, the others in a light tone. A line consisting of 15 letters chosen arbitrarily would contain 6 bold ones on an average, while the true reading line of 15 letters with 12 bold ones on an average stands out distinctly; moreover this line is indicated by a bright spot of light on the edge of the set of strips. The 30 strips necessary for the decipherment of a double line, are arranged side by side in two groups of 15 each for the line; if the left-hand group is in the first movement, the right-hand one is in the second movement and vice versa. During the time when the clerk copies the true reading from the indicated line on the typewriter, the machine prepares automatically the adjustment of strips for the next line and performs it at the touch of a key. In this way the decipherment of a double line, requires barely half a minute on an average. By means of this machine the total material could be deciphered within a month.’
It seems that the ‘Automaton’ was built by early 1943, as it was displayed to the Army cryptanalyst dr Luzius in March 1943 (27):
In March the circular strips 0-2 were replaced by the new edition 0-3. From then on a new set would be used for each month (0-4 for April, 0-8 for August etc). The exception to this rule was the embassy in Bern, which did not have access to the new strips so they continued to use the 0-2 set (28).
It seems that this change hindered the operations of the German codebreakers as in July ’43 there was a meeting between Kunze (Pers Z), Weisser (OKW/Chi) and Voegele (Luftwaffe Chi Stelle) in order to discuss cooperation between their departments on the strip cipher (29).
Voegele was chief cryptanalyst of the Chi Stelle and in late 1942 and early 1943 he had solved a USAAF ferry traffic strip system which used 30 strips selected from 100. The system was solved in 1942 but in 1943 channel elimination was introduced (meaning 5 channels in the M-138-A panel were left empty). Voegele was also able to solve this procedure by using IBM/Hollerith equipment (30). Although the report says that there was disagreement between Voegele and Kunze in other reports Voegele stated that he demonstrated his cryptanalytic attack to representatives of OKW/Chi and Pers Z and they adopted his method of solution (31).
Axis collaboration in 1943
During 1943 there was exchange of information on State Department codes and ciphers between the German, Finnish and Japanese signal intelligence agencies.
The Finns had managed to solve several special strips in 1942 and in early 1943 they gave copies to the Japanese military attaché so that he could transmit this information back to Tokyo (32). These messages were in turn decoded by the Allied codebreakers and they clearly revealed the compromised M-138-A strips 10-3, 10-1, 18-1, 4-1, 7-1.
More messages were exchanged regarding US codes and telegram No 101 of March 1943 contained the 33-1 strips while No 102 had solved messages on the 0-1 and 0-2 strips. The exchange of information was not entirely one-sided since the Japanese shared the strips used in 1942 by the US embassy in Vichy France (33).
Relations between the Finnish and German codebreakers were closer, with visits of personnel to each country and regular exchange of solved material. As has been mentioned earlier the Finnish success with US diplomatic systems was possible thanks to the material they received from the Germans in 1942.
In January 1943 Karl Erik Henriksson visited OKW/Chi and was shown reconstructed M-138-A alphabet strips (34). In November ’43 it was dr Franz’s turn to visit the Finns (35).
State Department security measures in 1943
In 1943 the Americans learned from decoded Japanese telegrams and from German sources (Fritz Kolbe and Hans Bernd Gisevius) that their high level diplomatic communications were being read by the Axis powers. Although they officially downplayed the compromise (36) it is clear that their cipher procedures were upgraded in terms of security. Regarding the strip cipher several changes were made in the use of the system (37):
1). From March 1943 a new set of circular strips was used for each month (0-4 for April, 0-8 for August etc, the strips used in 1944 were numbered 0-13 to 0-24).
2). The embassy in Bern, Switzerland received 6 new cipher systems in June 1943. In July they started using the strips 60-3 for intercommunication between Bern, London, Lisbon, Algiers and Washington.
3). A set of strips titled 00-1 (and key table C) was introduced in late 1943 for enciphering the confidential traffic of other US government agencies such as the Office of Strategic Services, Office of War Information and Military Intelligence Service, Foreign Economic Administration, War Shipping Administration, Office of Lend-Lease Administration and the War Refugee Board. Previously messages of other US organizations were simply sent using State Department codes and ciphers, with the result that they could be easily read by the Axis powers.
4). A security survey of State Department codes, ciphers and security procedures was undertaken in November and December 1943. The survey evaluated the State Department cryptosystems and found many problems both in their theoretical security and also mistakes in their use by State Department cipher clerks. Regarding the strip cipher it was noted that the same strips were used by 2-4 stations but an effort was underway to introduce new ones, used by only 2 holders at the same time and to expand the use of channel elimination (38):
‘The old systems did not use channel elimination and the same set of strips was sometimes held by 2 to 4 stations, however new systems using channel elimination and limited to 2 holders are being distributed as fast as possible’
In 1944 the Axis codebreakers continued to read fully the Gray and Brown codebooks till late 1944 when Gray was withdrawn from service and Brown was to be used for condensation purposes only (39).
Regarding the enciphered codebooks A1 and C1 it seems that the Germans solved the C1 cipher tables valid till 31 December 1943 and these continued to be used by the Bern embassy in 1944. It is unclear if the A1 code was read in 1944 (40).
An activity report of OKW/Chi, covering the first half of 1944, says that ‘Government codes and ciphers of 33 European and extra-European States and agents lines were worked on and deciphered. 17.792 VN were produced including 6.000 agents messages. From point of view of numbers the list was headed by Government reports of the USA, Poland and Turkey’ and ‘A number of complicated recipherings, principally American (USA) and Polish, have been broken. (41)’
As has been mentioned previously in 1944 the M-138-A cipher was used in a more secure manner by the State Department, with frequent changes of alphabet strips and use of channel elimination.
According to German accounts they could not solve strips using channel elimination (42) but they continued to read some US strip cipher traffic since alphabet strips that had been solved in previous years continued to be used in 1944 (43).
It seems that the embassies in Bern, Switzerland, Stockholm, Sweden and Madrid, Spain continued to use old strips for some of their traffic (44).
Apart from these messages the Germans and the Finns also solved back traffic from 1942 and 1943 (45).
It seems that their success with the M-138-A cipher ended in September 1944. Dr Huettenhein (chief cryptanalyst of OKW/Chi) said in his unpublished manuscript ‘Einzeldarstellungen aus dem Gebiet der Kryptologie’ that the strip cipher was read from 1942 till September 1944 (46).
Professor Rohrbach received the War Service Cross 2nd Class in September ’44 for his work on the strip cipher (47).
The Finns also revealed their success to the Americans in September. On 29 September 1944 colonel Hallamaa(head of the Finnish signal intelligence agency) met with L. Randolph Higgs of the US embassy in Stockholm and told him about their successes with US diplomatic systems (48).
Even though the German success probably ended in September ’44 that does not mean that they did not continue to research this system. In January 1945 a lecture was held at OKW/Chi on the US strip cipher procedure. According to a summary found in the War Diary of Inspectorate 7/VI:
‘On 26 January [1945] the chief of department 1 and the chiefs of sections 1a and 2a participated in a talk at OKW/WNV/Chi IV on the American strip system. (Increase in the importance of this procedure in diplomatic traffic, where it has replaced other methods. Concerning the processing no fundamentally new ideas proposed compared to the methods known here. Processing recently complicated by improved cipher techniques, in particular by a not yet clarified method of strip selection that varies from message to message.)’ (49).
Axis collaboration in 1944
In 1944 both the Finns and the Germans sent strip cipher material to Japan (50). In summer 44 the Japanese transmitted to the Finns information on messages from the US embassy in Chungking, China (wartime capital of Nationalist China). These were telegrams of the years 1942 and 1943. It seems that these messages were transported by courier to Finland, solved by the Finns and then sent back to Japan.
In July ’44 the Germans sent to the Japanese copies of the M-138-A ‘special’ strips 38-1, 22-1 and the ‘circular’ set 0-5.
In late ’44 the ‘circular’ strips 0-2, 0-3, 0-4 and tables for the A-1 and C-1 codes were transmitted by the Japanese attaché in Budapest.
According to the Finns they received the circular strips 0-2, 0-3, 0-4, 0-5 from OKW/Chi, so this material originated from Germany (51).
Introduction of cipher machines and creation of the Division of Cryptography
In 1944 the State Department took measures to secure its communications and in the second half of the year the M-325 - SIGFOY cipher machine was introduced into service.
The M-325 was a 3-rotor Enigma type device, designed by William F. Friedman and according to the available reports 1.000 devices were built for State Department use. Delivery of the machines began in July ’44 and by March ’45 the M-325 was in use at all foreign posts (52).
By 1945 the State Department was also given access to the Army’s SIGTOT (one time tape) cipher teleprinter network in London, Paris and Moscow (53). During 1945 the introduction of more secure systems continued and by early 1946 all posts were supplied with a version of the Combined Cipher Machine (non reciprocal 5-rotor cipher machine) and one time pads for use with the codebooks (54).
These efforts to secure State Department communications were the result of several security studies, undertaken in the period 1941-44. Investigations were carried out in June 1941, November and December 1943 and June 1944 (55). The findings of these committees showed that the codes and ciphers in use had many vulnerabilities and the cipher clerks made mistakes that compromised their security.
There were also serious problems with the handling of classified material, unclear classification procedures, insecure safes in embassies and locks in courier pouches etc
The most important recommendations of the surveys concerned the introduction of a sufficient number of secure cipher machines and the creation of a dedicated cryptology department.
As we have seen cipher machines and one time pads were widely distributed in the period 1944-45. The other major accomplishment in the field of communications security was the establishment of a separate Division of Cryptography, tasked with creating and evaluating State Department codes and ciphers.
In May 1944 two experienced cryptologists joined the State Department. These were Commander Lee W. Parke, US Navy and Major James G. Moak, US Army. Both were assigned to the office of Assistant Secretary Gardiner Howland Shawbecause Shaw had overall responsibility for the State Department’s cipher unit (56).
In June ’44 Commander Parke was designated Assistant Security Officer in the Office of the Assistant Secretary and Security Officer. David Salmon, former Assistant Security Officer, was designated Consultant on Cryptography in the Office of the Assistant Secretary Shaw.
Finally in September 1944 the new Division of Cryptography was established. The operations section of the Division was responsible for developing the State Department’s cryptographic plan, providing suitable cryptosystems, distributing and keeping account of the crypto material. The security section of the Division was responsible for reviewing radio communications and detecting security violations as well as rendering technical assistance on the preparation of crypto systems and instructions for their implementation.
Initially the unit functioned as part of Assistant Secretary Shaw’s office but in November ’44 a new Departmental designation was issued and Commander Parke became chief of the Division. Major Moak was made assistant chief for Operations and mr Salmon an adviser. In December 1944 a Departmental reorganization order placed the Division under the office of Departmental Administration.
Downplaying the compromise of US diplomatic cryptosystems
During the war the State Department received information on the compromise of its radio communications from several sources.
In early 1943 the German officials Fritz Kolbe and Hans Bernd Giseviustold US representatives about the compromise of their codes and during the year Japanese messages containing US crypto material were decrypted by the Allies.
In late 1944 more Japanese messages were decrypted and they revealed that the Germans had given to the Japanese the M-138-A strips 38-1, 22-1, 0-2, 0-3, 0-4, 0-5 as well as A1 and C1 substitution tables (57).
Also in late 1944 members of the Finnish signal intelligence service met with US officials in Sweden and gave them a detailed account of their solution of State Department codes and ciphers (58).
Just from the decoded Japanese messages the US officials knew that the circular strips 0-1, 0-2, 0-3, 0-4, 0-5 and the specials 10-3, 10-1, 18-1, 4-1, 7-1, 33-1, Vichy, 38-1, 22-1 had been solved by the Axis powers and these were just the strips mentioned in the Japanese traffic and not necessarily the only strips solved by the Axis.
Yet the postwar ‘European Axis Signal Intelligence in World War II’ volumes only mention strips 0-1 and 0-2, not the rest of them. Nor do they mention the specific systems solved by the Finnish codebreakers even though they had a detailed report on the subject.
Volume 1 ‘Synopsis’, p6 says:
‘The U. S. Army Converter M-134A lSIGMYC) and the U. S. Navy Cipher Machine (HCM), furnished by the Navy to the State Department, were not read by the Germans. The State Department Strip systems 0-1 and 0-2 were solved, the former probably through a compromise and the latter through cryptanalysis. Several State Department codes including the Brown code (unenciphered) and Code A-1 (enciphered) were compromised and read, probably from 1938 and 1939, respectively.’
………………………………………………………………. The value of the intelligence which the Germans got from State Department codes and strip ciphers is not accurately known. The strip systems were probably read too late to be of any great value.
There is also no mention of specific embassies such as Moscow and Bern, whose messages were known to have been read by the Germans through the material found in the OKW/Chi archives and the OSS reports (59).
Failures of cryptosecurity
The fact that the Axis codebreakers could exploit the main US diplomatic codes and ciphers for such a long period of time was a consequence of the theoretical and practical insecurity of these systems (60).
1). Gray Code: The Gray codebook had been used since 1918 and it was considered to have been physically compromised. The fact that it continued to be used during WWII was a serious mistake by the Americans.
2). Brown Code: The Brown codebook was a new system and it was used extensively in the period 1939-44. The fact that it had been introduced in 1938 led the State Department cipher clerks to overestimate its security and thus use it to transmit sensitive information.
This can be seen in numerous WWII telegrams that contain important reports even though they are only classified ‘Restricted’ (61). The State Department security survey of 1943 pointed out that:
‘As regards the low-grade basic systems, the BROWN and GRAY codes are two-part codes; BROWN has been in effect since 1938 and GRAY since 1918. There is positive evidence to indicate that both of these codes have been compromised and that the Axis powers have been deriving useful intelligence from the reading of messages in these codes. The committee considers that these codes are not adequate for use in war-time even for restricted traffic’.
However it was not until late 1944 that the Brown code was downgraded to unclassified traffic.
3). Enciphered codebooks: The codebooks A1, B1, C1, D1 were insecure since they were used for a long time and it was reasonable to assume that a foreign power would have been able to acquire copies. Moreover their enciphering tables could be solved if a lot of traffic was sent on these systems.
The security survey of 1943 said that:
‘The committee feels that in view of their long usage the basic codes (A, B, and C) must be considered compromised and that the cryptographic system for superenciphering messages in these codes does not yield adequate security for a voluminous number of confidential messages’.
The 1944 survey suggested the introduction of new codebooks and the use of one time pads for encipherment. The introduction of one time pads took place in 1945-46.
The M-138-A system was introduced in the late 1930’s and the Department used it for its most important messages. The strip system was a powerful encryption method but it had serious vulnerabilities that could be exploited by enemy cryptanalysts. The main problem was that it was vulnerable to a plaintext-ciphertext compromise.
In the strip system a letter could not be enciphered as itself so it was possible to place ‘cribs’ with a high certainty of success. Thus messages whose content could be guessed or was known from other sources (reencodements or press releases) could be easily solved and the alphabet strips recovered (62).
The State Department’s cipher unit also used the M-138-A system in an insecure manner (63). The biggest mistakes were the use of only 40 different keys (daily arrangements for the strips) for an entire year, the use of the same 'special' alphabet strips by several embassies and the coupling of the ‘special’ strips with only one key list.
5). Cipher machines: Clearly the main failure of the State Department was the delay in the introduction of a cipher machine till late 1944-early 1945. The US Armed Forces had managed to introduce the SIGABA (Converter M-134-C model) in 1941 and that device gave them the ability to send radio messages quickly and securely to all posts.
In 1941 the State Department was advised to acquire cipher machines but apart from the small number of Converters M-134-A and HCM’s no large scale effort was undertaken till the introduction of the SIGFOY in 1944 (64).
To add insult to injury the new cipher machine was not a successful design and it was quickly replaced by the Combined Cipher Machine in 1945-46 (65).
Limitations of codebooks and of the M-138-A
Apart from their limited security the main problem with hand systems (such as the codebooks and the strip ciphers) was that it took cipher clerks too long to process the cipher messages. Considering the wartime growth of the Department and the major increase in radio messages from its embassies it is clear that hand systems greatly hindered the rapid transmission of information.
The delay in sending and receiving cipher messages must have been the main reason for the continued use (and misuse) of the Brown codebook.
Notable cases of compromise
US-Japanese negotiations in 1940-41
Thanks to the material copied in 1940 from the US consulate in Kobe the Japanese leadership had access to ambassador Joseph Grew’s messages (66).
Messages of the US embassy in Chungking, China from 1942-1943
During the Second Sino-Japanese War the city of Chungking became the provisional capital of Nationalist China. Apart from housing the Chinese government it was also the base of Joseph Stilwell, deputy commander for the Allied South East Asia Command.
Messages of the Chungking embassy from 1942 and 1943 were read by the Finns and the Japanese in 1944 (67).
Messages of the US embassy in Moscow, Soviet Union
German and Finnish accounts mention the solution of messages from the US embassy in Moscow and these statements can be confirmed from the surviving archives of OKW/Chi and of the Finnish codebreaking organization (68). The compromise of traffic between the United States and a major wartime ally must be rated as a significant failure for the Allied side.
Messages of the US consulate in Algiers, Algeria
According to German accounts they could read the messages of the US consul in Algiers Robert Daniel Murphy (69). In 1942-43 Murphy was the personal representative of President Roosevelt in North Africa and he was engaged in difficult negotiations with the various French political groups (De Gaulle, Giraud, Vichy).
It seems that his use of stereotypical beginnings was exploited by the German codebreakers (his messages began with ‘From Murphy’ and those addressed to him ‘For Murphy’).
Messages of the US embassy in Bern, Switzerland
The Bern embassy was one of the most important foreign posts of the State Department, since Switzerland had close economic relations with Germany, was the home of international organizations like the Red Cross and the Bank of International Settlements and due to its geographical position it offered a base from which the US representatives could acquire economic, political and military intelligence about current European events.
The US ambassador Leland B. Harrisonsent detailed reports back to Washington and many were read by the Axis codebreakers (70).
Messages of other US government organizations
1). Messages of the OSS Bern Station were read by the Germans (71). During the war Bern was a hotspot of agent activity both Allied and Axis. The goal of the Office of Strategic Services was to recruit informants and gather intelligence on European affairs. Especially important was the need to recruit German agents to report on that country’s internal condition and policies. The Bern station also had contact with members of the German resistance.
2). General Barnwell R. Legge was the US military attache to Switzerland and during the war he worked to promote US interests and he also cooperated in intelligence gathering activities with Allen Dulles, head of the local station of the Office of Strategic Services. Legge developed his own intelligence networks and he sent reports dealing with military developments and Axis war potential to the Military Intelligence Service in Washington. Some of these reports were read by the Germans and the Finns (72).
4). The War Refugee Board was established by President Franklin D. Roosevelt in January 1944 with the goal of providing aid to civilian victims of the Axis powers. In 1944-45 representatives of the WRB in Europe collected information about the persecution of the Jewish population in the occupied territories and they tried to save as many people as possible by engaging in negotiations with German officials. Some of their communications were read by OKW/Chi (74).
In WWII the Axis and the Allies fought not only in the fields of battle, using troops, tanks, planes and ships but also in the field of signals intelligence and codebreaking.
The United States military forces were well equipped with secure cipher machines (SIGABA, SIGTOT, Converter M-228) so that they could quickly and securely transmit messages to their units. These cipher machines were designed according to sound principles and high level US military communications were secure during the war (75).
However US diplomatic communications were protected by insecure systems and the limited security of these systems was further compromised by the mistakes made in their use by the State Department’s cipher unit and by the department’s cipher clerks.
From the available information it is clear that in the period 1940-1944 the codebreakers of Japan, Germany and Finland could exploit low, mid and high level codes and ciphers of the State Department.
The compromise of State Department communications was one of the biggest failures of US cryptosecurity during WWII however this case has not received the attention it deserves because there is limited information available. The fact that several countries and organizations were involved in the solution of the US diplomatic systems also means that the relevant information is fragmented.
Even though the full story is not known (and will probably never be known) the remaining files of the Axis codebreaking organizations show that they had regular access to messages containing sensitive information regarding US foreign policy and goals. The solution of State Department systems also gave them access to the communications of other US government agencies (since they occasionally used the State Department’s cryptosystems and radio facilities) such as the Office of Strategic Services, Military Intelligence Service, Office of War Information, Foreign Economic Administration, War Shipping Administration, Office of Lend-Lease Administration and the War Refugee Board.
This was a significant defeat for the Allied side in the field of communications security.
(1). NARA - collection RG 59 - War History Branch Studies - CY ‘Division of Cryptography’.
(16). Dr Schultz, a member of the Mathematical Cryptanalytic Subsection of Pers Z said in TICOM report I-22 ‘Interrogation of German Cryptographers of Pers Z S Department of the Auswaertiges Amt’, p16 that the solution of the 0-2 strip was a major achievement of the unit since it was solved by hand, (without using special statistical/cryptanalytic equipment). Professor Hans Rohrbach, said in TICOM I-89 ‘Report by Prof Dr. H Rohrbach of Pers Z S on American strip cipher’ that the strips were recovered in late 1943 by using the IBM/Hollerith statistical equipment. However in the first page of the report he also mentioned that there may be mistakes in the dates because he was working from memory.
(19). 'Finland's Codebreaking in World War II' (chapter 20 of ‘In the Name of Intelligence: Essays in Honor of Walter Pforzheimer’)
(21). TICOM report I-2 ‘Interrogation of Dr. Huettenhain and Dr. Fricke at Flensburg, 21 May 1945’, p3, NARA - RG 457 - Entry 9032 - box 1.018 - ‘JAT write up - selections from JMA traffic', NARA - RG 457 - Entry 9032 - Box 214 - ‘M-138-A numerical keys/daily key table/alphabet strips’ and UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence' (A telegram from the Japanese military attaché in Hungary (dated June 1st 1944) said that the Germans would share the A-1 and C-1 enciphering tables used till 31 December 1943).
(22). 'JCS Ad hoc committee report on cryptographic security of government communications' - The report of November 1943 only mentions the codebooks A1, B1, C1. Note that telegram statistics from June 1941, found in the NSA’s Friedman collection file ‘Distribution of telegrams according to codes, June 3 to 7, 1941’ (A67352), show that the D1 code was barely used by the State Department.
(23). TICOM report I-2 ‘Interrogation of Dr. Huettenhain and Dr. Fricke at Flensburg, 21 May 1945’, p2 and TICOM report DF-176 ‘Answers written by professor doctor Wolfgang Franz to questions of ASA Europe’, p9
(24). 'JCS Ad hoc committee report on cryptographic security of government communications' – report of November 1943.
(26). TICOM I-89 ‘Report by Prof Dr. H Rohrbach of Pers Z S on American strip cipher’, p13-14
(27). War Diary of Inspectorate 7/VI - Report of Referat 1 for March 1943
(28). NARA - RG 59 - Purport Lists for the Department of State Decimal File 1910-1944 - microfilms 444 and 611 - 119.25/Strip Cipher
(29). TICOM D-60 ‘Miscellaneous Papers from a file of RR Dr. Huettenhain of OKW/Chi’, p5-6
(32). UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence' and NARA - RG 457 - Entry 9032 - box 1.018 - ‘JAT write up - selections from JMA traffic'
(33). NSA Friedman collection - telegram Tokyo-Helsinki No 719 and UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence'
(34). 'Finland's Codebreaking in World War II' (chapter 20 of ‘In the Name of Intelligence: Essays in Honor of Walter Pforzheimer’)
(35). TICOM report DF-176 ‘Answers written by professor doctor Wolfgang Franz to questions of ASA Europe’, p10
(36). UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence'. Friedman’sresponse to the Japanese telegrams containing solved M-138-A information was: ‘Friedman states facts do not correspond with USA use of strip nor does indication of keys apply’.
(37). NARA - RG 59 - Purport Lists for the Department of State Decimal File 1910-1944 - microfilms 444 and 611 - 119.25/Strip Cipher and NARA - collection RG 457- Entry 9032- box 1384 - 'JCS Ad hoc committee report on cryptographic security of government communications'.
(40). TICOM I-201 ‘Interrogation of Franz Weisser , Dr Phil Studienassessor of Anglo-American section of OKW/Chi’, p2-3, NARA - RG 457 - Entry 9032 - box 1.018 - ‘JAT write up - selections from JMA traffic', NARA - RG 457 - Entry 9032 - Box 214 - ‘M-138-A numerical keys/daily key table/alphabet strips’ and UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence'– (A telegram from the Japanese military attaché in Hungary (dated June 1st 1944) said that the Germans would share the C-1 enciphering tables used till 31 December 1943 (still used in 1944 in the Bern traffic) and the A1 tables valid from January to April ‘will probably have been readable in May’).
(44). NARA - RG 226 - Entry 210 - box 348 - Director’s Office records relating to developments in Sweden, ca. May 1944 – January 1945 and NARA - RG 59 - Purport Lists for the Department of State Decimal File 1910-1944 - microfilms 444 and 611 - 119.25/Strip Cipher (the 0-2 strips were used for communications between Madrid and Bern till summer 1944)
(45). In the surviving archives of OKW/Chi and of the Finnish department there are US messages from 1942-43 decoded in 1944 - NARA - RG 457 ‘Records of the National Security Agency’ - Entry 9032 - boxes 205-213 ‘German decrypts of US diplomatic messages 1944’ and Finnish national archives - folders T-21810/4 and T-21810/5.
(48). NSA study: ‘History of Venona’, p51 (Ft. George G. Meade: Center for Cryptologic History, 1995)
(49). War Diary of Inspectorate 7/VI - Report of January 1945 (translation by Frode Weierud).
(50). UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence'
(51). NARA - RG 457 - Entry 9032 - box 1018 - ‘JAT write up - selections from JMA traffic' and 'Finland's Codebreaking in World War II' (chapter 20 of ‘In the Name of Intelligence: Essays in Honor of Walter Pforzheimer’)
(53). US Army Center of Military History: ‘Signal Corps: The outcome’, p586
(55). NARA - collection RG 457- Entry 9032- box 1384 - 'JCS Ad hoc committee report on cryptographic security of government communications'
(56). NARA - collection RG 59 - War History Branch Studies - CY ‘Division of Cryptography’
(57). NARA - RG 457 - Entry 9032 - box 1018 - ‘JAT write up - selections from JMA traffic', p16
(58). NSA study: ‘History of Venona’, p51-53 (Ft. George G. Meade: Center for Cryptologic History, 1995) and NARA - RG 226 - Entry 210 - box 348 - Director’s Office records relating to developments in Sweden, ca. May 1944 - January 1945
(60). NARA - collection RG 457- Entry 9032- box 1384 - 'JCS Ad hoc committee report on cryptographic security of government communications'
(61). There are decoded US messages in NARA - RG 457 ‘Records of the National Security Agency’ - Entry 9032 - boxes 205-213 ‘German decrypts of US diplomatic messages 1944’ and Finnish national archives - folders T-21810/4 and T-21810/5. Some of them contain valuable information such as economic and intelligence reports and the original US telegrams have the classification ‘Restricted’ so the Brown code must have been used.
(62). NSA oral history: 1974 interview of Frank B. Rowlett, p236-237
(64). NARA - collection RG 59 - War History Branch Studies - CY ‘Division of Cryptography’, p3 and NARA - collection RG 457- Entry 9032- box 1384 - 'JCS Ad hoc committee report on cryptographic security of government communications' [It is interesting to note that the Hebern cipher machine was used in Latin American countries and not in the important European embassies (with the exception of Moscow)]
(67). UK national archives - HW 40/132 'Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence'
(68). NARA - RG 457 ‘Records of the National Security Agency’ - Entry 9032 - boxes 205-213 ‘German decrypts of US diplomatic messages 1944’ and Finnish national archives - folders T-21810/4 and T-21810/5. Note that the 38-1 strips were used by the Moscow embassy (NARA - RG 59 - Purport Lists for the Department of State Decimal File 1910-1944 - microfilms 444 and 611 - 119.25/Strip Cipher)
(69). ‘Decrypted Secrets: Methods and Maxims of Cryptology’, p69 (Fräulein Asta Friedrichs, who took part in this activity, said after the war, as she was detained in Marburg and saw him drive by one day: “Ich wollte ihn anhalten und ihm die Hand schütteln,—so viel hatte er für uns getan.” [I wanted to stop him and shake his hand—he’d done so much for us.]), according to NARA - RG 59 - Purport Lists for the Department of State Decimal File 1910-1944 - microfilms 444 and 611 - 119.25/Strip Cipher the Algiers consulate used the alphabet strips 33-1 and 22-1 that were solved by the Germans and the Finns.
(70). NARA - RG 457 ‘Records of the National Security Agency’ - Entry 9032 - boxes 205-213 ‘German decrypts of US diplomatic messages 1944’ and Finnish national archives - folders T-21810/4 and T-21810/5, various TICOM reports, UK national archives file KV 2/95, NARA - RG 226 ‘Records of the Office of Strategic Services’ - Entry 123.
