Quantcast
Channel: Christos military and intelligence corner
Viewing all 538 articles
Browse latest View live

Decoded US diplomatic messages from 1944

0
0
In the US National Archives, in collection RG 457 ‘Records of the National Security Agency’ - Entry 9032 - boxes 205-213 ‘German decrypts of US diplomatic messages 1944’ one can find many decoded messages from US embassies and consulates around the world.



These were messages decoded in WWII by the Germans. Specifically the codebreaking department of the Supreme Command of the Armed Forces – OKW/Chi (Oberkommando der Wehrmacht/Chiffrier Abteilung).

The codebreakers of OKW/Chi could solve most State Department cryptosystems, not only the low level ‘Gray’, ‘Brown’ but also the high level ‘A1’, ‘C1’ and the M-138-A strip cipher.



Unfortunately we don’t know the full story of their success with State Department systems.
Their main targets were the US embassies in European countries like Spain, Portugal, Sweden, Switzerland and the Soviet Union. However it seems they did not neglect to intercept and decode messages from around the globe.


Here are some interesting samples.

Mrs Morgenthau’s trip to Moscow



 


 
Kabul government recognizes Syria and Lebanon as independent republics




 



Information on South African gold mines




Speech by Gandhi and information on Purshotamdas Thakurdas and Ardeshir Dalal


 


Saudi gold


 

Railway policy in Iran



 



Arab union resolution on Palestine


 


US policy in Greece


 

Avoidance of Soviet shipping routes in the Far East for US officials



 



Nickel purchases in Australia



Mishaps at the front

0
0
During WWII visits to the front by important officials needed to be planned in advance so there would be no mishaps. However things didn’t always work out as planned…



Close to the front it was difficult to find food, so one would expect important officials to bring their own rations. Reichsführer-SS Heinrich Himmler forgot this rule, with hilarious consequences.

From ‘Walter Schellenberg: The Memoirs of Hitler's Spymaster, p75-76



On these tours we usually started out for the front at nine or ten in the morning, and would return to the train towards nightfall. We had to supply our own provisions—sandwiches, thermos flasks of hot tea, and cognac to fortify us against the increasingly cold weather. As the SS adjutants were already overburdened with other duties, it was my job to secure these provisions. One day we returned so early that a lot of our food and drink had hardly been touched. The next day we were called out early and the thermos flasks were not ready. I only had time to take what was left over from the previous day—a bottle of cognac, half-full, and two packets of sandwiches, which I had placed near a window, hoping they would remain fresh overnight. After driving for about two hours in the open car, Himmler asked for something to eat, so Gruppenfuehrer Wolff took a packet of sandwiches from me and they both began to eat. They had already got through the first packet when they happened to look at the second. The rest of the sandwiches were all covered with green mould. Himmler's face grew even greener as he tried desperately not to be sick. I quickly offered him some cognac—usually he did not drink; at the most two or three glasses of table wine—but he took a deep gulp and then, as he recovered, fixed me with a steely glance. I was prepared for the worst. 'I notice you ate none of the sandwiches yourself.' I hastened to explain, but there was a terrible look in his eyes as he thanked me for restoring his life with the cognac after having tried to poison him.

Another rule should be to visit the bathroom before going on a long trip. Even the mighty Stalin forgot this rule and paid the price!



Stalin cut off the briefing, contenting himself with giving some orders, then dismissed the generals who had to slog back to the real fray. Stalin asked if he could go further towards the fighting but Beria forbade him. He visited the hospital at Yukono, according to his bodyguards, and was depressed by so many amputees. Afterwards, he felt ill and his arthritis played up. Stalin returned by road in his armoured Packard and a convoy of security cars. Suddenly the cars stopped. 'He needed to defecate,' wrote Mikoyan, who heard the story from someone who was there. Stalin got out of the car and asked 'whether the bushes along the roadside were mined. Of course no one could give such a guarantee ... Then the Supreme Commander-in-Chief pulled down his trousers in everyone's presence.' In a metaphorical commentary on his treatment of the Soviet people, and his performance as military commander, he 'shamed himself in front of his generals and officers ... and did his business right there on the road.'

NSA in the news - yet again!

Update

0
0
I made the following changes:



1). Added pics from Foreign Military Studies B-644 in The RAF Cypher

 


2). Added the following paragraph in Polish Stencil codes and secret agent ‘’Knopf’’

The person who was responsible for this success at OKW/Chi was the mathematician Ernst Witt. According to TICOM report DF-176 ‘Answers written by professor doctor Wolfgang Franz to questions of ASA Europe’, p11


The most successful work along with that on the Am10 was that of professor Witt, who very skillfully solved a cipher of the Polish Government in Exile in London. This was a large complicated grille which was laid over a large number sheet. Several such grilles were constructed and messages were read currently. Photographic aids were used in the process.



3). Added the following paragraph in Soviet naval codes and the Arctic convoys

The compromise of the routes of PQ17 and PQ18 from reading Soviet naval aviation codes is confirmed from another source. The Swedish codebreakers were able to decode German messages travelling through their telephone network, even if they were enciphered with the Siemens T-52 cipher teleprinter. Some of these reports mentioned the decoded Soviet messages dealing with the northern convoys. The Swedish codebreaker Sven Wasstrom, who examined these messages, became distraught at this drama.

General Onodera’s intelligence bazaar

0
0
In the course of WWII all the participants tried to gather secret intelligence using spies.



For example the Brits had their Secret Intelligence Service and the Special Operations Executive plus they worked together with the Polish, French and Czech intelligence services. The Germans had their Abwehr and Sicherheitsdienst. The Americans created the Office of Strategic Services etc etc

Spying was a hard business. Recruiting trustworthy individuals, training them, providing them with false identities, necessary paperwork and foreign currency was not easy. Inserting them into an enemy country was difficult with the majority being caught in a relatively short time. Even those that survived could usually only gather information of limited value.



The Germans built up large spy networks in neutral countries like Spain, Turkey, Sweden and Switzerland but they did not have similar successes in the US and the UK.

Prior to WWII they had compromised the USAAF’s most advanced bombsightbut during the war their attempts to insert agents all (?) failed.



In the UK their attempts were so clumsy that the Brits not only captured the spies but in many cases convinced them to change sides and send false information to the Germans.

Does that mean that the Germans failed to get any useful information from these countries during the war? Not quite. Although the Germans didn’t have spy networks in the US and UK they were able to acquire some accurate information on Allied war production (and possibly other areas).



How could they have done so? Although they didn’t have spy nets that doesn’t mean that there weren’t other countries that did. Diplomats and businessmen of neutral countries learned a great deal by talking to Allied officials and some of this information was leaked or sold to the Germans. At the same time there was an exchange of information between Germans, Italians, Japanese and Hungarians.

For example in Sweden Karl-Heinz Kraemer, secretary of the German legation in Stockholm, was able to gather valuable information on US and UK war production mainly through his contacts in business and government circles. In 1944 the Allies considered Kraemer to be one of the most dangerous German agents and they were worried that he might compromise the security of operation ‘Overlord’.



One of Kraemer’s best sources was the Japanese military attaché in Sweden, general Makoto Onodera. In 1944-45 they regularly met and exchanged information.

General Onodera



In Europe one of the top officials of the Japanese intelligence service was the military attaché in Sweden, general Makoto Onodera.
 
 
 
 
 
According to US reports (available from the CIA’s FOIA website) Onodera was born in September 1897 in Iwata, Japan. He came from a prominent family and pursued a military career. In the period 1912-1920 he studied first at a local cadet school, then at the central cadet school in Tokyo and finally at the Military Academy (infantry course). During his time at the academy he learned Russian.

 
In the period 1926-1928 he attended the War College. In the 1930’s he served in the intelligence department of the Army General Staff and then lectured at the Army General Staff College. During this time Onodera was acknowledged as an expert on Soviet affairs.


In order to continue his intelligence activities on the Soviet Union he was appointed military attaché to Latvia in 1936, where he established close ties with the military and intelligence authorities of that country and the other Baltic nations of Estonia and Lithuania. These countries were fearful of the Soviet Union and they were willing to exchange secret intelligence with Japan.
 
 
 
 
After a brief stint back to the General Staff in Tokyo in 1938 and then an assignment in the Expeditionary Force in China he was given the important position of military attaché in the Japanese embassy in Stockholm, Sweden. He officially occupied this position from February 1941 till the end of the war.


This post used to have limited value in the field of intelligence but during the war the Soviet occupation of the Baltic countries and the German occupation of Poland meant that many intelligence officials of these countries found refuge in neutral Sweden. Both the Baltics and Poland had ties to the Japanese intelligence service, especially in their anti-Soviet activities. Finding themselves without a country many of Onodera’s former acquaintances had no alternative but to make a living by selling secret intelligence.



Japanese cooperation with foreign countries in the interwar period

The Japanese approach to secret intelligence was to cooperate with other countries by offering mutually beneficial deals.



In the interwar period the Baltic countries had reasons to fear the Soviet Union and they built up spy networks to monitor that country. However their efforts were hindered because they had limited resources to invest in intelligence. The Japanese were able to take advantage of this issue by providing funds to the Baltic countries that were used for intelligence purposes. In exchange they got copies of the reports.

The Japanese also had a strong connection with Poland. They cooperated with that country not only against the Soviet Union but also in the field of cryptology. In 1923 Captain Kowalewski of the Polish Army was invited to Tokyo to teach cryptology and the Japanese sent some of their officers to Poland to train in cryptography and cryptanalysis.



 
 
Another country with anti Soviet policy was Finland. In the 1930’s relations with Finland were strengthened in the intelligence field. The Japanese were impressed by the performance of the Finnish codebreakers, especially during the Winter War.

 
Wartime activities



In the period 1939-1940 Japan’s Eastern European allies were occupied by Nazi Germany (Poland) and the Soviet Union (Estonia, Latvia, Lithuania). Some government officials of these countries fled abroad to neutral states like Sweden. Meanwhile the Polish intelligence service continued its operations against Germany and the Soviet Union with support from Britain. Surprisingly the anti German activities of Polish intelligence did not hinder the exchange of information between them and the Japanese. On the contrary the Japanese tried to protect some of the Polish spies by giving them Japanese or Manchurian passports.

From Sweden Onodera relied mainly on his old contacts from the intelligence services of the Baltic countries, on the Polish intelligence service, the Finnish intelligence department, his German counterparts and Swedish diplomatic and military sources.



Let’s have a quick look at each case:

Polish intelligence service



During WWII the Polish IS operated throughout Europe and scored many successes against Germany. They even had a spy in the German High Command (probably the Oberkommando des Heeres-OKH). The Japanese supported the Polish IS and in turn benefited from the information obtained but this definitely strained their relations with the German security services.

 
In time the Germans uncovered cases of espionage perpetrated by persons holding Manchukuo passports and they obviously realized that the Japanese were protecting the Poles. Their protests led to the Japanese scaling back their activities but the cooperation between Japan and the Polish IS was continued.



Onodera’s contact with the Polish IS was his close associate Michal Rybikowski, a former intelligence officer, whose cover name was ‘Piotr Ivanov’. Onodera protected him from the Germans and the effort paid off as he received valuable information on the German military, the Soviet Union and the fighting in the Eastern front.



 
 

 
 Was Onodera’s support of Rybikowski completely justified? A US report says that though him the Poles and the British were able to monitor the Japanese activities.  



 
 
 
Considering the close relationship between the Japanese and the Polish IS it is possible that Rybikowski betrayed only parts of Onodera’s activities to the British.



Finland

During the war Onodera received information from the Finnish intelligence service and especially their codebreaking department. The Finnish codebreakers were able to solve several important foreign cryptosystems, including Soviet military codes and the State Departments strip cipher. The Allies learned of this arrangement when they decoded Japanese messages mentioning the solved codes.



In September 1944 Finland signed an armistice with the Soviet Union. The intelligence officials anticipated this move and fearing a Soviet take-over of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris. In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. Onodera did not forget his Finnish friends and he supported them with all the funds at his disposal (roughly 250.000-300.000 Swedish Kroner).

 
From the Finns he got the foreign codes that they had solved during the war.



 
 
 
 Balts



In Sweden Onodera was able to benefit from the acquaintances he had made during his tour as military attaché to Latvia in 1936-38.

His most valued associate was colonel Maasing, chief of the intelligence service of the Estonian General Staff. Maasing directed espionage activities against the Soviet Union and in 1940, when the country was occupied by Soviet troops, he fled to Sweden. Up to 1942 he worked for the German intelligence service but in April of that year he returned to Stockholm to work for Onodera.



 
 
  Maasing knew many officers in the Swedish armed forces and the national police.




 
Swedish sources



In Sweden Onodera kept in contact with diplomats and military officials.  According to his interrogation report most of the time he wasn’t able to get information directly from the Swedes but had to rely on the connections of his associates Maasing and Karl-Heinz Kraemer. Maasing knew several important Swedish officials, especially in the police force from his time as director of Estonian intelligence. Kraemer on the other hand had contacts with Swedish businessmen.

 
From the interrogation report it is obvious that some interesting information was acquired through Swedish officials such as major Petersen of the intelligence service.


 

Germans


Onodera’s relations with his German counterparts were poor, as they did not share valuable intelligence and in any case he did not consider them to have an efficient espionage service.



The exception to this rule was Karl-Heinz Kraemer, secretary of the German legation in Stockholm, whom Onodera considered to be a valuable agent. In the period 1944-45 Onodera and Kraemer exchanged information.
 
 

Hungarians

Onodera exchanged information with the Hungarian military attaché Frigyes Kobor and his assistants Vagy and Voeczkoendy, however he did not have a lot of respect for them.

 
More details on the exchange of information are available from Voeczkoendy’s interrogation report CI-FIR/117 (available from fold3.com)
 

General Onodera and the market of secret intelligence – A Soviet connection?

Onodera collected information from several sources and then exchanged some of the material with his various contacts.

According to general Schellenberg of the SD foreign intelligence department in ‘Walter Schellenberg: The Memoirs of Hitler's Spymaster’, p153-154

The Japanese Ambassador in Stockholm, Onodera, was one of the key figures of the Japanese Secret Service in Europe. He received secret information from Vichy, Rome, Belgrade and Berlin for transmission to Tokyo and also gathered other material himself. Much of this information—though only what was absolutely reliable—he used for a sort of barter trade, and he expected the same standard of integrity from those with whom he did business. If one gave him bad material once only, he would from then on refuse to engage in any further dealings with that informant. Piotr was in a sense his chief liaison with the Russians ……………. I found out that Onodera made the material collected by K—available to both the British and the Soviet Secret Services. Later I succeeded in insinuating one of my agents into Onodera's barter trade. He posed as a representative of the Italian Secret Service, a disguise which he was able to maintain until the end of 1944- It was a really fascinating game requiring the highest intelligence and skill. The material which we traded in Stockholm was prepared by me personally, usually late at night. It was a very careful blend of false, even misleading, material and valid information, the latter mostly of a less important nature. In this trade the British showed themselves somewhat slow and clumsy, while the Russians were extremely quick and active. I must confess that the material they collected and offered for trade was excellent. That on Great Britain, for instance, showed that they must have had agents in the highest circles of the government. Through them we even got material that came directly from the British War Office. The Russians were already working with the Secret Service of the Chinese Communists and used the Chinese very skillfully, especially in diplomatic circles in London.

The German assistant air attaché in Sweden Friedrich Busch also believed that Onodera was in contact with the Soviet intelligence service, as can be seen from report CI-FIR/67 (available from fold3.com)
 
The Soviets certainly had ample material to trade with. Their agents in the US and UK had infiltrated the most secret government departments. Could they be exchanging material with Onodera? There is no evidence of such an arrangement in Onodera’s interrogation reports or in the book that his wife wrote postwar.

However that doesn’t mean that we know the full story. It’s up to historians to find out more!

Sources: reports titled ‘Onodera, Makoto’ from the CIA’s FOIA website, CI-FIR reports 67 and 117 (from fold3.com) and the chapter ‘Military attaché Onodera’ from authors Simon Olsson &Tommy Jonason.
Acknowledgements: I have to thank Simon Olsson &Tommy Jonason for sharing the information on Onodera contained in ‘Karl Heinz Kraemer’.(In manuscript 2014)
 
 
 
 
 
 
 
 
 

State Department’s strip cipher – reuse of alphabet strips and key lists

0
0
During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1,B1,C1,D1) enciphered with substitution tables were available.



Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until late 1944) and by reading these messages the Axis powers gained insights into global US policy.

The strip cipher was not a weak system cryptologically, even though it could not offer the security of cipher machines. The success of German and Finnish codebreakers was facilitated in many cases by the poor way that the system was used by the State Department.



Use of the M-138 strip cipher by the State department

Each embassy had 50 ‘circular’ alphabet strips and 50 ‘specials’. The ‘circulars’ were used for communications between embassies and for messages from Washington to all embassies. The ‘specials’ were used for direct communications between Washington and a specific embassy.



The way the system worked was that each day 30 alphabet strips were chosen out of the available 50 (both for the ‘circulars’ and the ‘specials’). The strips used and the order that they were inserted in the metal frame was the ‘daily key’. 

The strip system did not have a separate ‘key’ for each day. Instead there were only 40 different rearrangements.



The daily key table indicated which of the 40 keys was valid for the specific day. For example in the following table assuming that the date is April 10 then the numerical key to be used is 30.





Using the 30thnumerical key we would insert in the panel first the alphabet strip with the number 19, then in the second position no43, in the third position no24……… and finally in the 30th position no21.



Obviously having only 40 keys was not the best choice. Enemy codebreakers would only need to solve the traffic of 2-3 months and then they would have recovered almost all the numerical keys. Although I’ve accused the State Department for misusing the M-138-A in this case they were simply following the same instructions as the US Armed Forces.
According to Special Research History SRH-366 ‘History of Army Strip Cipher devices’ the US Army also used a daily key table (but with 50 keys) till mid 1942. From then on they introduced a separate key for each day.
This change however was not implemented by the State Department. Was this the only mistake of the State Department’s cipher unit? Apparently not.

Reuse of alphabet strips and key lists

According to the information presented so far it is clear that the strip system was based on the daily key table, the numerical key and the alphabet strips. Enemy codebreakers would need to recover all three parts in order to solve large numbers of messages. The only other way was to attack messages individually and that was too costly in terms of time and manpower.



The work of German and Finnish codebreakers was made much easier due to the fact that all three parts of the strip system were reused. Let’s have a look at the available information from postwar reports.

From TICOM I-201 ‘Interrogation of Franz Weisser , Dr Phil Studienassessor of Anglo-American section of OKW/Chi’, p3



The small Finnish staff under Colonel HALLAMAA also collaborated quite effectively giving hints. Thus it found out that the systems used in different countries after some time emerged again in other countries.

From TICOM I-145 ‘Report on the US strip system by Reg Rat Dr Huettenhain’  



Only a little of the material received could be read at once. Generally it was back traffic that was read. As, however, the different sets of strips were used at different times by other stationsit was possible, in isolated cases, to read one or the other of the special traffics currently.

The use of the same alphabet strips by different embassies is confirmed by a Japanese message listing the different strips and the posts that used them. This was information they received from the Finnish codebreakers.



Thanks to the continued use of alphabet strips the Axis codebreakers only needed to recover the daily key tables and numerical keys.

Or did they?



According to David Kahn in 'Finland's Codebreaking in World War II':                                                        

Each post had its own set of strips; the key changed daily but was the same for all posts. This cryptographic weakness was probably permitted for logistical reasons.’



If the daily key for the ‘special’ strips was the same for all posts that would mean they had the same daily key table and numerical key. Although this could be true i think it’s more likely that this is a reference to the use of a limited number of daily key tables and numerical keys.
In the US National Archives collection RG 457 there is a file with several sets of daily key tables, numerical keys and alphabet strips (both ‘circular’ and ‘special’). It seems that these were intercepted Japanese messages on the link Budapest-Tokyo, probably from late 1944.

At that time the Germans and the Finns had given the Japanese officials a lot of material on the strip cipher for transmission to Tokyo.





According to this file there seem to have been a limited number of strips and key lists and these were coupled for use by specific embassies. They also used the same daily key table for all the circular strips in use since 1943.


 

 


For example the daily key table and numerical key No19 was used with strips 25-4 and 113-1 by the Calcutta and Vladivostok consulates.  No 3 daily key table and numerical key was used with strips 33-1 and 34-1 etc etc


 
 
 
 


Conclusion


If this interpretation is correct then it seems that the State Department used for most of the war a limited set of alphabet strips, daily key tables and numerical keys. This obviously made the work of the Axis codebreakers much easier than it should have been.



The State department implemented security changes in January 1944 and these limited the Axis success but did not defeat it. It is possible that the main change was the introduction of new sets of strips, daily key tables and numerical keys for direct communications with Washington.

GCHQ versus Anonymous?

British cryptologic security failures in WWII

0
0
Signals intelligence and codebreaking played an important role in WWII. British and American codebreakers solved many important Axis crypto systems, such as the German Enigma machine and the Japanese Navy’s code JN25.



Historians have not only acknowledged these Allied successes but they’ve probably exaggerated their importance in the actual campaigns of the war.

Unfortunately the work of the Axis codebreakers hasn’t received similar attention. As I’ve mentioned in my piece Acknowledging failures of crypto security all the participants suffered setbacks from weak/compromised codes and they all had some successes with enemy systems.



Britain, the Soviet Union and the United States did not have impenetrable codes. In the course of WWII all three suffered setbacks from their compromised communications.

Time to take a look at the British side and their worst failures.



Book cyphers

The basic British cryptosystem for important radio-traffic was the enciphered codebook. These 4-figure codebooks were enciphered with subtractor tables, using the non-carrying system. The military services had their own series of cyphers such as the War Office Cypher for the Army and the RAF cypherfor the airforce plus there were diplomatic editions for the Foreign Office and the Interdepartmental Cypher that was used both by the services and the civilian organizations.



The codebook was basically a dictionary that assigned a 4-figure group to each word. For example the word ‘division’ would have the code 5538, ‘attack’ 2090, ‘artillery’ 0231 etc etc

So the cipher clerk would first use the codebook in order to find the code groups corresponding to the words of the message and then he would have to use the subtractor tables in order to encipher them. This means that each codegroup would be subtracted from the key groups (of the subtractor table) without carrying over the numbers.



For example let’s say that the following message is handed to the cipher clerk:

Enemy frigate sighted South-West of Malta.



Let’s assume that using the codebook this becomes: enemy=2591 , frigate=7482 , sighted=5556 , SW=3309 , Malta= 4610

So the message becomes 25917482555633094610



This will be enciphered using the subtractor table valid for this time period. This was a book containing (usually) 15.000 numerical groups. The cipher clerk had to choose a random page and random starting point and then use that numerical sequence for enciphering the codegroups. Let’s say that the clerk chooses the 9th page and 2nd line from the subtractor table. He needs five key groups and these are 5668 8301 3496 3540 7778.

Now he would subtract the code groups of the message from these key groups, without carrying.



Enemy frigate sighted South-West of Malta   


25917482555633094610 - code groups

5668 8301 3496 3540 7778 - key groups



-------------------------------------

3177 1929 8940 0241 3168 – cipher groups



The receiving party would identify from the indicator of the message the page and line that was used from the subtractor book and then subtract the received cipher sequence from the numerical key sequence. This would reveal the code groups whose meaning would be deduced from the Cypher book.

Obviously this was a time consuming operation and prone to errors due to mistakes in encipherment.



However it was thought at the time that the use of both a codebook and enciphering tables provided a very high level of security against enemy codebreakers.

Mistakes in the use of enciphered codebooks



The War Office Cypher and the Interdepartmental Cypher were read by the German codebreakers mainly thanks to physical compromise.

The Germans captured two copies of the WOC in 1940, one during the Norway campaign and the other near Dunkirk. By having the codebook and by taking advantage of frequent ‘depths’ (messages enciphered with the same numeric sequence) they were able to read traffic in the Middle East Theatre in the period March (or August) 1941 till April 1942.



The official history ‘British intelligence in the Second World War’ vol2, p298 says: 
If under-estimation of the quality of Rommel's equipment was one reason why British confidence was high when the Crusader offensive began, another was the failure to allow for the efficiency of his field intelligence. By August 1941 the Germans were regularly reading the War Office high-grade hand cypher which carried a good deal of Eighth Army's W/T traffic down to division level, and they continued to do so until January 1942. Until then, when their success was progressively reduced by British improvements to the recyphering system, whereas GC and CS's success against the German Army Enigma continued to expand, this cypher provided them with at least as much intelligence about Eighth Army's strengths and order of battle as Eighth Army was obtaining about those of Rommel's forces.

The Interdepartmental Cypher was used by the Foreign Office, Colonial, Dominions and India offices and the Services. Also used by the Admiralty for Naval Attaches, Consular Officers and Reporting Officers. The basic book was captured from the British consulate in Bergen in May 1940, allowing the Germans to solve ‘depths’ during the period 1940-43. In their efforts they were assisted by poor British cipher practices. A security investigation in 1942 showed that the tables were overloaded, leading to heavy ‘depths’ and the indicators were not selected correctly.
 


The German exploitation of the ID Cypher ended on 15 June 1943 when the codebook was changed.

In both cases it is highly doubtful that the Germans could have solved this traffic on their own, without having the basic codebooks. It was a big mistake to continue using codebooks that were almost certainly expected to have fallen in enemy hands. Had the Brits introduced new versions in late 1940 or early 1941 the German success would have been nipped in the bud.



This was not however true in the case of the RAF Cypher. This was a 4-figure codebook enciphered with subtractor tables. The codebreakers of the Luftwaffe’s Chi Stelle did not have the basic book but they were able to ‘break’ into the system in March 1940 and traffic between the Air Ministry and Gibraltar, Malta, Habbaniya, Ismailia was read. From early 1941 to November 1942 traffic in the Med/Middle East was also compromised. According to Dr Voegele, chief cryptanalyst of the Luftwaffe: ‘From Sept. '41 to Nov. '42 the majority of the 200 - 400 daily intercepted 4 fig. messages could be decyphered with an average delay of 5 - 10 days, in single cases messages were decyphered the day of intercept.’

The German success was once again dependent on ‘depths’ and they didn’t have to search hard to find them. According to the report AIR 20/1531 ’R.A.F. signal communications: security’ in 1940-41 there were two series of enciphering tables in use with the RAF Cypher, the ‘Special’ for higher formations and the ‘General’ for all units. Traffic was not split evenly between the two tables because the units that had the ‘Special’ sets were usually equipped with the Typex cipher machine so they relied on that. Considering that the ‘General’ tables were the ones used all the time and that they were valid for three months it was only natural that there would be heavy ‘depths’, although the number of 150 given by the report is definitely impressive!



 
Thanks to the heavy ‘depths’ and the heavy traffic (up to 600 messages daily) the Luftwaffe cryptanalysts were able to solve new editions of the RAF Cypher in 1942. Their luck run out in November ’42 when the British security measures finally defeated them. Had the Brits introduced enciphering tables faster and thus avoided ‘depths’ it is difficult to see how the Germans could have succeeded in solving the RAF Cypher.



Naval codebooks

The Royal Navy’s main cryptologic systems were the Code and Cypher. The former was used for mid-level traffic and the latter for high level messages. Naval Cypher No1 (a 4-figure book) and the Administrative Code (a 5-figure book) were used from 1934 till August 1940. There was also the Auxiliary Code No3, (a 4-letter book) used from 1937 till August 1940 by small units. This was discontinued in August 1940 and instead small units used the Naval Code enciphered with Auxiliary Vessels tables.



It is immediately obvious that these systems were used for too long, thus compromising their security. If that wasn’t enough, instead of introducing new editions at the start of WWII they continued to be used up till August 1940!

These mistakes did not pass undetected by the codebreakers of Germany and Italy. The German naval codebreakers attacked these cryptosystems with success. In 1935-6 the Administrative Code was solved and in 1938 the Naval Cypher followed. At the start of the war the Auxiliary Code was also read with little difficulty. They were greatly aided in their efforts by the poor British decision to use the Administrative and Auxiliary codes unreciphered for non-confidential traffic. This allowed them to recover the true values of the codebooks and then focus only on breaking the enciphering tables.



Additionally there were mistakes in the construction of the codebooks and their use. According to the article ‘The Cryptographic Services of the Royal (British) and Italian Navies’, written by Admiral Luigi Donini, one of the top Italian codebreakers of WWII, some of the main British mistakes were:

1). Reuse of the same key sequence over and over, leading to ‘depths’.



2). The period of validity of the enciphering tables was badly commensurate with the level of radio traffic.

3). Numerical sequences from old table were sometimes added in the new ones. His explanation for this was ‘perhaps for quicker compilation’.



4). For many years the indicators were not enciphered thus making it very easy to locate ‘depths’. When enciphered indicators were first used the method was so clumsy that ‘it only caused us a two week crisis’.

5). The Cypher book did not contain homophones, ie did not assign two or more codegroups for a certain word of very high frequency.



6). Signals almost always began with the complete address to …from…

7). According to the article: ‘Geographical names were coupled, as an alternate interpretation, to vocabulary entries beginning with the same two or three letters. e.g.: 7184 = Give/Gibraltar, 0921 = Last/La Spezia, 4650 = Make/Malta, 2935 = All Concerned/Alexandria, 7714 = Left/Leghorn. This was of great help in identifying the geographical term when we had already identified the vocabulary term coupled thereto and vice versa;



Donini also points out that the failure to introduce new Code and Cypher books at the start of the war was the biggest mistake:

‘In my opinion, however, the most serious British carelessness or inadvertence was to keep in use for the whole first year of war with Germany the same principal naval cipher which was in force since the time of the Spanish civil war, broken by us (and by the Germans) in 1938.’



These failures meant that during most of WWII the Royal Navy’s most important cryptosystems were systematically exploited by the enemy with disastrous consequences for the Allies.

Disaster in the Atlantic – The case of Naval Cypher No3



Probably the biggest failure of British crypto security was the compromise of Naval Cypher No3, also known as the Convoy Cypher, since it was used in the Atlantic by the British, American and Canadian Navies.

In peacetime Britain was dependent on imports of raw materials and agricultural products from around the world. During WWII these shipments were vital for the smooth functioning of the war economy. The Germans knew that if they managed to sink the majority of supplies crossing the Atlantic then Britain would be economically strangled and would have no choice but to sue for peace. Even if this did not happen the lack of military supplies would make it impossible to launch heavy attacks on continental Europe.



The head of the U-boat service, Admiral Doenitz had calculated that with a large numbers of submarines he could achieve victory. His operational strategy was to overwhelm convoys with a large number of U-boats. These wolfpacks could evade the few escort vessels and sink the majority of merchant ships. The rest would disperse and could be picked off at a later time. In order for the wolfpack strategy to work the Germans needed to know the route and speed of the convoys in advance.

During the war the Germans had limited success in fielding a naval airforce and although they used traffic analysis and direction finding they had to rely mostly on codebreaking in order to track the Allied convoys.



According to TICOM report I-143 'Report on the Interrogation of Five Leading Germans at Nuremburg on 27th September 1945', p6

Doenitz stated emphatically that Sigint had been very valuable to him. It had been the best source of Naval Intelligence, and indeed, when air recce, etc., were not available, had often been the only source of operational information.



Since the convoys were vital for the Allied cause one would expect that every measure would be taken to ensure the impenetrability of their communications. Unfortunately the blunders of the responsible departments defy belief.

The need for a special codebook solely for Allied convoy duties had not been foreseen prior to WWII and thus the Brits had to give the Americans and Canadians copies of their Naval Cypher No3. The first problem was that their own codebook had to be used for longer than anticipated since Cypher No3 could not replace it as planned.
 


The security of the enciphered codebook system depended on the frequent changes of enciphering tables and the introduction of a new codebook after a reasonable period of time. Obviously the meaning of ‘reasonable’ changed during the war. The first Naval Cypher was used from 1934 till August 1940. The next edition Naval Cypher No2 was valid from August 1940 till January 1942. Its replacement Naval Cypher No4 (as No3 was used in the Atlantic) was valid from January 1942 till June 1943. So in the period 1940-1943 Cyphers were changed roughly every 1.5 years. One would expect that the Convoy Cypher would be valid for a similar timeframe or perhaps for security reasons it would have been changed sooner, maybe after 1 year.

Unfortunately Naval Cypher No3 wasn’t changed after 1 year. It wasn’t changed after 1.5 year. It was actually used from June 1941 till June 1943.



The continued use of the same basic codebook for two years meant that the codebreakers of the B-Dienst had an easy time recovering the true values and then they only needed ‘depths’ in order to read current traffic. The heavy traffic in the Atlantic combined with the small size of the enciphering tables (at 15.000 groups) led to heavy ‘depths’. For example M table-General: 218,000 groups in August ’42, S table-Atlantic: 148,000 groups in October 1942 and 220,000 in November.

The Germans would have to be incompetent in order to be defeated by a system that was used for two years and had huge ‘depths’ on a daily basis. They were able to ‘break’ into the traffic in December ’41, by February ’42 they had reconstructed parts of the book and till 15 December ’42 they were reading a large proportion of the traffic (at times up to 80% of intercepted messages). In December an indicator change set them back but from February ’43 they were again able to read the messages. Their greatest success with the Convoy Cypher was achieved in 1943. From February till June they often read signals 10-20 hours in advance of the actions mentioned in them. Also from February ‘42 to June ‘43 they could decode the daily Admiralty U-boat disposition signal nearly every day.



The German naval codebreakers were finally defeated in June ’43 when Naval Cypher No5 replaced No3 in the Atlantic. It didn’t take a miracle to end their success, just a new basic codebook that was long overdue….

Merchant Ships code



Since I’ve covered Naval Cypher No3 it’s important to also have a quick look at another very important system connected with the convoy battles. This was the cryptosystem used by merchant ships. Merchant ships had their own codebooks. During 1939 this was the International Code and Naval Appendix, from January ’40 to April ’42 it was the Merchant Navy Code and for the rest of the war the Merchant Ships Code. The Merchant Navy code and the Merchant ships code were captured from merchant ships and their enciphering tables were solved throughout the war. 

‘British intelligence in the Second World War’ vol2, p639 says that these two systems ‘were a prolific source of information to the B-Dienst second only to the Naval Cypher No3 in their importance to the battle of the Atlantic’.



Since these codebooks were in circulation around the world it’s not strange that they were physically compromised.  Once they had the basic book it was obviously easy to break into the traffic by taking advantage of ‘depths’. Could the British authorities have taken measures to secure this system? Yes and no. On the one hand a codebook used in such numbers, by ships around the world could not have been made secure. On the other hand a simple British countermeasure greatly limited the value of these decoded messages for the B-Dienst.

What was this measure? Simply enciphering the coordinates sent in the messages. This security measure was implemented in December 1943. Better late than never…



Cipher machines

The extensive use of enciphered codebooks by the British authorities does not imply that they were not aware of the benefits of a modern cipher machine. On the contrary had things gone according to plan they would have been supplied with one in large numbers and thus have been spared the losses that resulted from the fact that their communications were compromised.



In 1926, the British Government set up an Inter-Departmental Cypher Committee to investigate the possibility of replacing the book systems then used by the armed forces, the Foreign Office, the Colonial Office and the India Office with a cipher machine. It was understood that a cipher machine would be inherently more secure than the codebook system and much faster to use in encoding and decoding messages. Despite spending a considerable amount of money and evaluating various models by 1933 the committee had failed to find a suitable machine. Yet the need for such a device continued to exist and the Royal Air Force decided to independently fund such a project. The person in charge of their programme was Wing Commander Lywood, a member of their Signals Division. Lywood decided to focus on modifying an existing cipher machine and the one chosen was the commercially successful Enigma. Two more rotor positions were added in the scrambler unit and the machine was modified so that it could automatically print the enciphered text. This was done so these machines could be used in the DTN-Defence Teleprinter Network.

The new machine was called Typex (originally RAF Enigma with TypeX attachments). In terms of security it was similar to a commercial Enigma but had the additional security measure of multiple notches per rotor. This meant that during encipherment the rotors moved more often than in the standard Enigma machines.



The problems with Typex were:

1). Due to the failure of the Inter-Departmental Cypher Committee to select a cipher machine for mass production and the solitary efforts of the RAF in the mid 1930’s there were only a small number of Typex machines available at the start of WWII. The first contract in 1938 was for 350 machines and it’s doubtful that all would have been delivered by September 1939. Note that at that time the Germans had about 10.000 Enigmas in use.



2). The ability to print the enciphered text came at a heavy (literally!) price. While the German Enigma machine was relatively small and compact, the Typex version built in large numbers Typex Mk II was bulky and weighed 54kg. Thus it could only be used at prepared sites.


 
3). For the same reason Typex was too complex to mass produce during the war. According to ‘The Development of Typex’:


Some of the reasons for the low production rate are clear. Any rotor-based machine tends to be very complex mechanically. Figures 2 and 3 illustrate just how many different parts a Typex machine included. Typex Mk. VI contained about 700 parts, few of which were common to other models. Typex must have been a quartermaster's nightmare - much more so than Enigma, because of Typex's printer. Typex's relative complexity proved too much for the British machine tool industry. Overloaded as the industry was with the demands of the war economy generally, it took almost two years to obtain the machine tools required to manufacture Typex, despite the priority that would have been accorded to it. Only 2,300 Typex machines had been made by the end of 1942, 4,078 by December 1943 and 5,016 by May 1944.



For comparison’s purposes at least 40.000 Enigma machines were built by the Germans.

4). Because they were complex the machines often malfunctioned





For these reasons the Brits did not have, during WWII, a cipher machine in widespread use like the Germans did.

Low level codes



Does the compromise of low level codes count as a failure of cryptologic security? By their very nature low level codes are expected to secure information only for a limited amount of time. Still during WWII important information was passed on these systems and by reading these messages the Axis powers got order of battle data and even information on upcoming operations.

Some of the British low level codes extensively exploited by the Germans were the Syllabic cipher, Slidex, RAF Syko/Rekoh cards, Bomber code and the RN’s small ships codes.



The war diary of the German Army’s Inspectorate 7/VI has references to the Syllabic cipher, used at division level by the British Army. Although details on this system are lacking it is possible that it was similar to the well know Slidex.

The Slidex card system was used extensively by the British armed forces in the period 1943-45 but had very limited security. The decoded traffic provided the Germans with valuable intelligence, especially since it was used by ALO’s (Army Liaison Officers) in requesting air strikes.



Sykoh/Rekoh cards were used by the RAF as a low level system and they were extensively read by the codebreakers of the Luftwaffe. Messages from the planes of Coastal Command had important information.

 

 
The RAF also used the Bomber code, a daily changing 2-letter code table. During the Combined Bomber Offensive the RAF and the USAAF used the same tables each day with unfortunate consequences for at least one of them.





 
Small naval units used several systems (LOXO, FOXO, COFOX, MEDOX, TRAXO). These were extensively exploited by the B-Dienst.




The information was sometimes very important as in the run-up to the Normandy invasion.

 

 
Regarding low level codes it should be pointed out that the successes of General Rommel in N.Africa were definitely linked to the performance of his tactical signals intelligence unit NFAK 621,headed by Captain Seebohm. This unit relied on British low level codes, traffic analysis and direction finding and radio-telephone communications in order to monitor and anticipate British moves. Especially in conditions of mobile warfare, when messages were exchanged quickly with little regard for security, Seebohm’s men gave Rommel the edge.



Special Operations Executive codes

At the start of WWII the British foreign intelligence service SIS did not perform well. This led to the establishment of a similar organization in 1940, called SOE - Special Operations Executive. SOE was created to ‘set Europe ablaze’, so the main task was to organize resistance groups in the occupied countries and provide them with weapons and explosives. Still the fact that SOE had agents, spy groups and informers meant that it was trespassing on the activities of the SIS. As would be expected countless power struggles ensued between these two agencies.



The wartime performance of SOE was mixed at best. Although they certainly had their successes, countless SOE networks were compromised and their members arrested and executed. In Holland their entire network fell under German control in the famous Englandspiel operation.  In France they lost countless agents and networks. Just the fall of their Prosper network in 1943 led to the arrest of hundreds of resistance members.

SOE was disbanded in 1946 and most of its archives were destroyed postwar with some lost in a fire. Unfortunately the loss of the archives means that many questions about SOE wartime operations can never be answered.



Were some of the failures of SOE in Western Europe connected with their insecure cryptosystems? Leo Marks, head of the SOE cipher section, was constantly worried about the insecurity of their poem code but it took him till late 1943 to introduce the unbreakable letter one time pad. The change was gradual and even in 1944 many insecure systems continued to be used.

The main crypto system used by SOE for most of the war was double transposition, using a poem as a ‘key’ generator.



Why did SOE use this system for so long? In the beginning SOE communications were handled by SIS and it seems that the double transposition system was imposed by SIS. According to ‘Between Silk and Cyanide: The Story of SOE's Code War’, p40

Ozanne was advised by his friend and mentor Brigadier Gambier-Parry, C’s director of Signals, that their agents were going to continue using the poem-code (or some minor variation of it) as he had no doubt whatever that agents codes should be carried in their heads. This was all Ozanne needed to hear. What was good enough for the agents of the British Secret Service must be good enough for SOE.



and in page 57 when Marks wanted to change it he was told ‘Furthermore, I had greatly exaggerated the poem-code’s insecurity. Properly used it was perfectly suitable for SOE’s purposes.

Was the double transposition a secure and reliable system? Let’s see what a report titled ‘S.O.E. FIELD CIPHERS’ has to say:



II. CONCLUSION. 1942

Transposition systems based on poems or emergency phrases carried in the agent's head are a complete failure as the main system for S.O.E. type clandestine traffic.



 IV. WHY ARE THEY A FAILURE

1). Because if an agent is caught and tortured, he will almost certainly reveal the details of his poems – thus enabling the enemy to decipher all of his traffic which they have intercepted. This consideration is of paramount importance.



2). Low grade security is afforded when stereotyped messages are sent in transposition.

3). Under emotional stress the agent cannot remember his poems without difficulty, and when he can remember them he has not the time to use the code with the accuracy it requires.



4). After about 15-20 messages have been passed on a poem, the agent has a tendency to repeat the indicators he has previously used. If he is instructed to retain the list of indicators he will also retain his list of en clair messages; if he is instructed to destroy his messages, then it is a psychological certainty that he will revert to using indicators which have been tried and proven.

THE UNCONSCIOUS MIND HAS A TENDENCY TO REVERT TO WHAT IT BELIEVES TO BE SAFE.



"If Indicator 123 has been cleared by the Home Station once, let us use it again and take no chances."

Leo Marks upgraded the security of SOE communications by introducing prepared keys for the double transposition system (called WOK’s-Worked-Out Keys) and with the Letter One time Pad- LOP. Even so it took a long time to replace the vulnerable systems with new ones and the report says:



‘S.O.E.'s field cipher security abroad was a mess from 1942 to the middle of 1943; from 1943 to 1945 it was put on a sound basis, but the rot was so embedded that some agents were using in 1945 conventions with which they had been issued in 1942.‘

Could SIS have helped out with SOE’s cryptologic problems?



According to ‘Between Silk and Cyanide: The Story of SOE's Code War’, p250 in early 1943 Leo Marks met Commander Dudley-Smith of Bletchley Park and demonstrated the Letter One time Pad- LOP system that he had devised. Dudley-Smith was impressed by the fact that Marks had thought of it by himself but also said: ‘As a matter of fact letter one time pads have been working very successfully for quite a long time’.

So SIS obviously knew how weak the double transposition system and probably did not use it themselves but they insisted that SOE should use it. For protecting their own communications they had LOP’s but they did not share this information with SOE. The strange behavior of the SIS leadership could be explained by their willingness to sabotage SOE operations, or at least to keep a close eye on them by forcing SOE to use a crypto system that could not resist a serious cryptanalytic attack.



If this interpretation is correct then the underhanded behavior of SIS definitely hurt the Allied war effort as some SOE communications were in fact decoded by Referat 12 (Referat Vauck) of the German Army’s signal intelligence agency Inspectorate 7/VI!

Securing British codes – 1943-45



Considering the information presented so far it is obvious that British cryptologic security was (more or less) a mess in the first half of WWII and continued to have serious vulnerabilities even in 1943-45.

However it should be acknowledged that the Brits systematically upgraded their systems and were finally able to secure their mid and high level codes in the period 1943-45. Did they succeed by investing huge resources on cipher security? Or by developing codes and ciphers so elaborate that no human mind could solve them? No, of course not. Simple security measures coupled with the new Stencil Subtractor system were enough to defeat the limited resources of the German codebreakers.



Book Cyphers were secured by introducing new versions and by enciphering them with the Stencil Subtractor Frame, a stencil that was used together with a daily changing numerical table. The SS Frame defeated ‘depths’ as the user could select different starting points for the enciphering sequence and that point was further enciphered.

The Royal Navy introduced new editions of the Code and Cypher every 6 months and it was expected that even this was not enough.




 
Without the basic codebooks and faced with the difficulty of locating depths, due to new stencil system, the German codebreakers were defeated.



In the case of cipher machines the Typex was upgraded with several sets of ‘split’ rotors, indicator books and a rewirable reflector. As Ralph Erskine puts it in ‘The Development of Typex’:

For the Germans to have been on an equal footing with Typex, as used by the British Army and the RAF, they would have had to find the wirings of from 120 to 252 rotors. Even Marian Rejewski or Alan Turing might have blanched at that Herculean task



Finally I’ve mentioned that SOE codes were upgraded with the Letter One time Pad, a system that if used correctly is unbreakable.

These improvements impressed the Germans and the report FMS P-038 'German Radio Intelligence’ says: ‘British radio communication was the most effective and secure of all those with which German communication intelligence had to contend’.



There was a downside however to all this. British cryptologic security was effective in this period but not efficient, in the sense that significant resources had to be invested in printing huge numbers of codebooks, enciphering tables, stencils, indicator books etc and transporting them to units across the globe. In the case of Typex the new procedures meant that cipher work fell to a fraction of the output achieved at the beginning of the war.


 
Conclusion



Allied successes in signals intelligence and codebreaking during WWII have received a lot of attention from Anglo-American historians. In Britain the successes of Bletchley Park are a source of national pride. The Germans had the Tiger tanks, their Me262 jet fighters and the V-2 rockets, the Americans were the ‘arsenal of democracy’ but the Brits had Bletchley Park.

However the many successes of Bletchley Park against Axis codes should not be used to distract from its failure to secure British codes. According to the article ‘Tunny Reveals B-Dienst Successes Against the ‘Convoy Code’:



GC&CS excelled at breaking the codes and ciphers of the Axis powers, and devoted huge resources to doing so. In March 1942, GC&CS employed about 1600 people on codebreaking operations, but only Travis (in theory) and Dudley-Smith were then assigned to investigating cipher security, even though Comsec was one of GC&CS’s two main functions. It was clearly too few, especially since Travis had no time to devote to Comsec, and Dudley-Smith was not a cryptanalyst. Even in October 1943, when GC&CS’s staff had more than trebled to over 4800, only Dudley-Smith (in a ‘part-time’ capacity!) and ‘two or three girls’ worked in the ‘Security of Allied Communications’ section, which investigated the security of the Army’s and Royal Air Force’s signals (and even those of some allies), in addition to the Royal Navy’s signals. Comsec is not as glamorous as codebreaking, but is probably more important.

Especially at sea the British mistakes prolonged the war. The Battle of the Atlantic was the longest continuous military campaign of World War II but it didn’t have to be. The only thing needed to stop the U-boat command was to secure the Convoy Cypher and here Bletchley Park undeniably failed.



It is said that ‘victory has a thousand fathers, but defeat is an orphan’. Perhaps that is the reason why the Allied failures of cryptologic security during WWII have not received the same attention as the ULTRA story. It is up to historians to correct this mistake.

Sources: various TICOM reports, ‘British Intelligence in the Second World War vol2’, ADM 1/27186Review of security of naval codes and cyphers 1939-1945’, AIR 20/1531’R.A.F. signal communications: security’, ‘European Axis Signal Intelligence in World War II’,FMS P-038 'German Radio Intelligence’, ‘Between Silk and Cyanide: The Story of SOE's Code War’, HS 7/41 ‘Section II: field cyphers; appendices L, M, N and O’ (from website arcre.com), Intelligence and National Security article: Tunny Reveals B-Dienst Successes Against the ‘Convoy Code’, The Journal of Intelligence History article: ‘The Admiralty And Cipher Machines During The Second World War: Not So Stupid After All’, Cryptologia article: ‘The Cryptographic Services of the Royal (British) and Italian Navies’ , The Enigma Bulletin article: ‘The Development of Typex’, ‘Intelligence and Strategy: Selected Essays’, cryptomuseum

Acknowledgements: I have to thank Ralph Erskine for sharing information on British codes and ciphers, especially his article on Typex, the report AIR 20/1531 and the Naval Cypher No3’s S and M table statistics.

RAF Strength Far East Command – January 1942

0
0
In December 1941 Japan entered WWII on the side of the Axis by attacking the forces of the USA and UK stationed in the Pacific.



The most audacious attack was against the US fleet in Pearl Harbor but in the same period the Japanese invaded the Philippines, Thailandand British controlled Hong Kong and Malaya.

Both the Americans and the British had underestimated Japan’s military and they paid the price. The British also suffered from their ongoing war against Germany and Italy. They had limited military resources and the choice had been made to concentrate these in Europe. Thus their forces in the Far East were equipped with outdated weapons. This is obvious in the case of the RAF, as shown by the following strength reports:



 


Source: AIR 22 ‘Air Ministry: Periodical Returns, Intelligence Summaries and Bulletins’

The types available were second line aircraft like the Hawker Audax, Westland Wapiti, Westland Lysander, Vickers Vildebeest and a handful of relatively modern Curtiss P-36, Bristol Blenheim, Bristol Beaufort, Lockheed Hudson and Brewster F2A Buffalo.



These forces were not capable of standing up to the modern Japanese planes, especially the Mitsubishi A6M Zero.

Update

James Bamford NSA video

0
0
Interesting presentation on the history of the NSA by author James Bamford




In the news

Recommended reading on the Eastern Front

0
0
Every year countless books dealing with WWII are published worldwide. Many cover the fighting in the East between Nazi Germany and the Soviet Union. This conflict was the largest land campaign of the war and claimed the lives of millions of people. There is no shortage of great battles for authors to write about such as the battles of Moscow, Stalingrad, Kursk etc



However with so many books available there is the problem of quantity vs quality. How many books have accurate information from both sides? How many present new information?

Unfortunately many books that I’ve read have serious mistakes because they rely mostly on other books for their information and not on the relevant archives. This is understandable since researching the archives is very costly both in terms of time and money.



What i look for in a book is clearheaded analysis and lots of statistical data from both sides. Based on this what are the books that I can recommend?

First let’s have a look at the ones that I found lacking.



Two of the most popular Eastern Front histories are ‘When Titans Clashed: How the Red Army Stopped Hitler’ by David Glantz and Jonathan House and ‘Russia's War: A History of the Soviet Effort: 1941-1945’ by Richard Overy. Having read them I can say that they have serious mistakes since they rely on Soviet sources for information on German strength and loss statistics. This leads to exaggerations that could have been avoided had the authors used the reports available from the German archives.

In addition there are other mistakes scattered throughout both books. For example Overy writes in chapter ‘The Citadel: Kursk 1943’ that the T-34 got a 3-man turret in 1943. He’s off by a year. This might seem like a small mistake but it undermines his argument that the Soviet forces won the battle not through numerical superiority but because they upgraded their equipment and tactics.



‘When Titans clashed’ is guilty of perpetuating several WWII myths. For example the chapter ‘An army in disarray, 1937-1939’ exaggerates the effects of the purges on the Red Army.  Despite its flaws ‘Russia’s war’ points out in chapter ‘The darkness descends’ that during the period 1936-38 41.218 officers were dismissed, not executed (or even all arrested). By May 1940 11.596 officers had been reinstated. The book says ‘Of the 179.000 officers employed in 1938 only 3.7 per cent were still formally discharged by 1940’. It is true that the higher ranks suffered disproportionately but many of these officers (such as Marshall Tukhachevsky)were lackeys of the regime and had not gained their position by merit. In chapter ‘The Red Army’ the T-26 tank is called ‘aging’ and ‘obsolete’ even though its operational characteristics were similar or superior to the German tank types Pz I, Pz II, Pz 35, Pz 38 and Pz III. Why was this tank ‘obsolete’? Its main problem was the 2-man turret and by that standard the mythic T-34 was also ‘obsolete’ since it only got a 3-man turret in 1944.

Having said that let’s take a look at some very interesting books:



 
 
 
This is the best single volume history of the Eastern Front bar none. The author is a professor at the University of Glasgow and he has written several books on WWII.




‘Thunder in the East’ covers the entire conflict from 1941 to 1945 and there is a detailed examination of all the important aspects such as German-Soviet relations prior to operation Barbarossa, the Soviet military plans in early 1941 and their intelligence assessments, the actual military operations plus war production and diplomacy.


Mawdsley takes an objective look at Soviet prewar policy, their massive armaments program and their support of Nazi Germany in 1939.  Unlike other authors he is not afraid to look into the state of Soviet planning for offensive operations in the period 1940-41. The rest of the book has solid analysis of the military operations and detailed statistics for the Soviet side.



The only negative remarks I have are that the author has presented a lot of detailed data for the Soviet side but not for the Germans and he also hasn’t covered the signals intelligence side (a common problem for EF histories).

2). Earl F. Ziemke’s ‘Moscow to Stalingrad: Decision in the East’ and Stalingrad to Berlin: the German defeat in the East’ (both available online from the US Army Center of Military History).



 
 


Ziemke was a historian for the US Army’s Office of the Chief of Military History and later became a professor at the University of Georgia. He specialized on the Eastern Front. His two books are well written, filled with interesting information plus he’s one of the few Eastern Front historians to include information from the intelligence assessments of the Fremde Heere Ost/Foreign Armies East organization. To his credit Ziemke pointed out several problems with Soviet era WWII historiography such as the coverage of operation Mars.

The only negative thing that can be said about Ziemke’s books is that he is too reliant on the German archives for information on the Soviet forces. Then again at the time he wrote his books ‘Soviet Casualties and Combat Losses in the Twentieth Century’ hadn’t been published.



3). I have already covered ‘Kursk 1943: A Statistical analysis’ and ‘Stopped at Stalingrad’. Both are excellent.

4). The air war histories of Christer Bergström are considered to be among the best sources for the operations of the Luftwaffe and the Red Air force in the East. They have analysis of the operations, strength and loss statistics for both sides and lots of very interesting pictures:



 






5). Krivosheev’s ‘Soviet Casualties and Combat Losses in the Twentieth Century’ is the ‘official’ Russian source for Red Army strength and loss statistics during WWII.


 
The book doesn’t have analysis of the battles, it just consists of countless tables with information on casualties, production etc. Data is broken down by year, quarter and by operation. If you want to ‘scientifically’ analyze the Eastern Front battles, then you need this book.



6). ‘Europe at War 1939-1945: No Simple Victory’ by Norman Davies is not a history of the Eastern Front but a book critical of standard WWII historiography.
 


The author points out several inconvenient facts such as going to war to protect Poland from Germany but then doing nothing while it’s taken over by the Soviets, fighting for ‘freedom’ and ‘democracy’ while one of the Allies was the undemocratic Soviet regime etc. This book will make you think, which is why I added it in the list.


7). ‘The Nazi War Against Soviet Partisans, 1941-1944’ by Matthew Cooper is a short history of the Soviet partisan movement, its evolution in the period 1941-44 and its impact on the actual war.


 
Soviet historians claimed that the partisans played a big role in the actual fighting by blowing up countless supply trains and killing scores of German troops. Cooper however is critical of the partisan groups and points out that their contribution to the war was limited. There were several important problems that limited their effectiveness, such as the lack of trained men, lack of supplies, Moscow’s efforts to control all the partisan operations and the fact that partisans preferred to stay in their remote areas instead of attacking enemy troops.



The Germans were able to keep the partisans under control through passive(strongpoints along major routes, routine patrols, watch towers, clearing the sides of the roads in forested regions for up to 9 miles, arming supply trains) and active (SS and police occupation units, airpower, anti-partisan offensives by military units) measures. The author considers the former to have been successful but not the latter.

8). ‘The Chief Culprit: Stalin's Grand Design to Start World War II’ by Victor Suvorov (Vladimir Rezun) is a controversial book.
 


Rezun was a Soviet military intelligence officer who defected to the West in the 1970’s. Since then he has written several books on WWII. The most famous was ‘Icebreaker: Who Started the Second World War?’ and it caused quite a stir in the 1980’s since it claimed that in 1941 Stalin was preparing to attack Germany. The controversy was so great that two books were written with the expressed purpose of debunking his claims, ‘Stumbling Colossus: The Red Army on the Eve of World War’ by David Glantz and ‘Grand Delusion: Stalin and the German Invasion of Russia’ by Gabriel Gorodetsky. ‘The Chief Culprit’ is an expanded version of ‘Icebreaker’ based on new information. Despite the controversy i think it deserves a read.

9

). The book ‘Accounting for War: Soviet Production, Employment, and the Defence Burden, 1940-1945’ by Mark Harrison (professor at Warwick) has some valuable information on Soviet war production, not only armaments but all kinds of products.



 


Unfortunately there are only a few tables with interesting information as most of the book is taken up by the author’s effort to create a mathematical input-output model of the Soviet economy.

Articles:



Apart from books there are some articles that I would like to recommend:

From the ‘Journal of Slavic Military Studies’:



Analyzing World War II eastern front battles



Summer 1941



The role of lendlease in Soviet military efforts, 1941–1945



Did Stalin intend to attack Hitler?



From the ‘Journal of Contemporary History’:

Foreign Armies East and German Military Intelligence in Russia 1941-45



From the ‘International History Review’:



Update

Update


The secrets of room 6527

0
0
Interesting files are available from site room6527.com



The project is described as:

The FBI has been hiding sensitive records of American eavesdropping operations from parliamentary scrutiny for decades. FBI director J. Edgar Hoover (right) gave orders in 1948 for tricky political papers to be stored away in Room 6527 – known as the Confidential File Room - at its Washington headquarters. The records did not show up in any index so that the FBI would be able to deny any knowledge of the relevant documents should a parliamentary control commission ever start to ask questions.
Along with records of US eavesdropping on friendly states, Hoover also stashed away documents about Eastern Block spies or reports about the unusual sexual practices of senior Communist officials and politicians. There were so many documents that they began to threaten the vast official building’s structural mechanics. An internal FBI memo from September 1961 notes that secret papers had to be immediately transferred to other rooms due to the weight of Room 6527’s 26 filing cabinets. Thanks to a Freedom of Information Act request, the SonntagsZeitung and Le Matin Dimanche have gained access to these historic and previously unpublished intercept records.



Regarding the available files I had a quick look and the pdf ‘diplomatic code Greek’, pages 5-23 has a summary of AFSA (predecessor of the NSA) efforts versus the codes of Eastern European countries in the 1950’s. Very interesting stuff!

United States cryptologic security failures in WWII

0
0
Signals intelligence and codebreaking played an important role in WWII. British and American codebreakers solved many important Axis crypto systems, such as the German Enigma machine and the Japanese Navy’s code JN25. 



Historians have not only acknowledged these Allied successes but they’ve probably exaggerated their importance in the actual campaigns of the war.

Unfortunately the work of the Axis codebreakers hasn’t received similar attention. As I’ve mentioned in my piece Acknowledging failures of crypto security all the participants suffered setbacks from weak/compromised codes and they all had some successes with enemy systems. 



Britain, the Soviet Union and the United States did not have impenetrable codes. In the course of WWII all three suffered setbacks from their compromised communications.

After having dealt with the British side let’s have a look at the Americans and their worst failures. 



First it’s time for a short history lesson on the organizations responsible for making and breaking codes.

Herbert Yardley, MI-8 and the American Black chamber



The first dedicated codebreaking unit of the US military was organized during WWI by Herbert Osborne Yardley. Yardley had worked as a telegrapher and then as a State Department code clerk. During WWI he demonstrated the insecurity of US diplomatic codes by solving, on his own, a message sent from Colonel House to President Woodrow Wilson. This attracted the attention of his superiors and Colonel Ralph H. Van Deman, head of the Military Intelligence Division, made him a first lieutenant and assigned him head of the MI-8 department, responsible for codes and ciphers.

The MI-8 unit solved several foreign codes and their success led the War Department and the State Department to jointly fund Yardley’s activities in the postwar era. The codebreaking department was moved to New York and called the ‘Black chamber’. Their main effort in the 1920’s was against Japanese diplomatic codes and in this area they were able to prove their worth. Yardley’s group not only solved regular Japanese diplomatic traffic but scored a great victory during the Washington Naval Conference by discovering Japan’s minimum acceptable battleship requirements. This allowed the US diplomats to get the Japanese representatives to agree to a battleship ratio of 5-5-3 for USA-UK-Japan.




Gentlemen do not read each other's mail’and the Great Depression

Yardley’s unit had performed well in the 1920’s but the new Secretary of State Henry L. Stimson was shocked when he learned of the existence of the Black chamber’ and famously stated ‘Gentlemen do not read each other's mail’. Without government funds Yardley’s group disbanded in October 1929. It was at this time that the world experienced an economic downturn, called the Great Depression.



Finding himself without a job and with his investments having lost their worth Yardley was forced to write about his secret activities in order to get money. He wrote the history of the ‘Black chamber’ for the‘Saturday Evening Post’ and later published the book ‘The American Black Chamber’ which became a best-seller not only in the US but around the world and especially in Japan.

By publishing a summary of his codebreaking activities Yardley compromised the solution of foreign codes and in that sense hurt US national interests. Many foreign governments changed their codes after the publication of the book.  The US government wanted to prosecute Yardley but since he hadn’t broken an existing law the Espionage Act was amended to prohibit the disclosure of foreign codes or anything sent in code.



US Army and Navy agencies

The decision by the US government to stop codebreaking activities and the limited funds available for military spending had a negative impact on the performance of US signals intelligence in the 1930’s.



Despite these problems two small groups under the Army and the Navy continued to work on the solution of foreign codes, with considerable success in the case of Japanese systems.

In the field of crypto security the new systems introduced from the 1930’s up to 1941 were the M-138-A strip cipher and the SIGABA cipher machine.




 
 SIS- Signal Intelligence Service




The US Army concentrated its cryptologic functions in the newly established Signal Intelligence Service, whose cipher research department was headed by William F. Friedman. The SIS was part of the Signal Corps and officially they were responsible only for preparing and testing the security of US Army codes. However in practice the military authorities were not willing to stop all codebreaking work so the department continued to attack foreign codes.

The work of the SIS was made easier by the appointment of Major General Joseph O. Mauborgne as Chief Signal Officer in the period 1937-41. Mauborgne was no stranger to cryptology and while in office he expanded the SIS and made the unit report directly to him.



In the 1930’s Friedman hired a small group of young mathematicians (Rowlett, Sinkov, Kullback) and the department concentrated on mathematical research in order to solve foreign codes. Their main success up to the Pearl Harbor attack was the solution of the Purple cipher machine, used by Japan’s Foreign Ministry. This success enabled them to read the diplomatic messages sent from Tokyo to its embassies abroad.

Their other great success was the development of the SIGABA cipher machine, a device that was vastly more secure than any other cipher machine of that time period.



Navy’s OP-20-G

The US Navy had its own cryptologic unit, the OP-20-G - Office of Chief Of Naval Operations (OPNAV), 20th Division of the Office of Naval Communications, G Section / Communications Security.



This unit was headed by Commander Laurence F. Safford and was responsible not only for codebreaking but also direction finding, traffic analysis and the production of codes for USN use.

The naval codebreakers worked mostly on Japanese naval codes and they were able to solve these from the 1920’s up to 1940.



In the field of crypto security they adopted the Army’s idea for SIGABA and named their modified version ECM MARK II. In 1941 Army and Navy agreed to jointly produce the modified SIGABA/Converter M-134-C/ECM MARK II as their high level crypto system.

Overview of US crypto systems in 1941



In the 1930’s the US cryptologic agencies were hampered in their operations by the lack of adequate resources. Still in that timeframe they were able to introduce the M-138-A strip cipher and the SIGABA. The M-138-A was quite hard to solve, provided it was used properly, while the SIGABA was the most secure cipher machine of WWII.

The main problem for US crypto security was the continued use of a large number of outdated systems. Without the necessary funding both military and civilian authorities were forced to use old codes and ciphers, that were not only insecure from a security standpoint but were also suspected to have been physically compromised during their long time in service!



Military systems

US Army and Navy used the M-94 cylinder even though it offered very limited security and had been introduced in 1921. The Army and USAAF would continue to rely on it even as late as 1943.



 



For administrative traffic the services used the War Department Telegraph Code 1919 edition - SIGRIM. This was a 5-letter codebook used without additional encipherment.

Systems considered to be quite secure were the Military Intelligence Code and the War Department Confidential Code (both letter codes). They were enciphered with substitution tables.



Diplomatic systems

The State Department had a small unit responsible for the production of codes in the Division of Communications and Records. This unit was controlled by David Aden Salmon.



The basic cryptosystems were codebooks. These were the enciphered codes A1 (introduced in 1919), B1 (introduced in 1920), C1 (introduced in 1927) and D1 (introduced in 1928). The unenciphered Gray code (introduced in 1918) was used for low level traffic.

In the late 1930’s two new systems were introduced, the Brown codebook and the M-138-A strip cipher. Although there was a survey proposing the introduction of cipher machines this idea was rejected for financial reasons.





 
Voice communications



The Bell Labs A-3 speech scrambler was used by the military authorities and on the civilian link Washington-London. Even though it wasn’t considered 100% secure it was the only speech privacy system in widespread use.

Failures of crypto security during WWII



During WWII military high level communications were secure, thanks to the advanced SIGABA machine, however the other cryptologic systems used by the US military and civilian authorities had vulnerabilities and the Axis powers were able to compromise almost all of them.

Notable cases



Fellers code

Colonel Bonner Fellers, US military attaché in Cairo during 1940-2, sent back to Washington detailed reports concerning the conflict in North Africa. In his reports he mentioned morale, the transfer of British forces, evaluation of equipment and tactics, location of specific units and often gave accurate statistical data on the number of British tanks and planes by type and working order. In some cases his messages betrayed upcoming operations.



Fellers used the Military Intelligence Code No11, together with substitution tables. The Italian codebreakers had a unit called Sezione Prelevamento (Extraction Section). This unit entered embassies and consulates and copied cipher material. In 1941 they were able to enter the US embassy in Rome and they copied the MI Code No11. A copy was sent to their German Allies, specifically the German High Command's deciphering department – OKW/Chi. The Germans got a copy of the substitution tables from their Hungarian allies and from December 1941 they were able to solve messages. Once the substitution tables changed they could solve the new ones since they had the codebook and they could take advantage of the standardized form of the reports. Messages were solved till 29 June 1942 and they provided Rommel with so much valuable information that he referred to Fellers as his ‘good source’.


A-3 speech scrambler

The Bell Labs A-3 speech scrambler was used by the military and on the civilian link Washington-London. Two different German teams solved this system and they were able to decode the conversations in real-time.  Traffic was successfully recorded from late 1941 up to late 1944. Through this operation they got political, economic and military intelligence. Their greatest success was the interception, on 29 July 1943, of a conversation between Roosevelt and Churchill which revealed negotiations with the newly established Badoglio government in Italy. This convinced the Germans that the Italians were trying to exit the war and thus they stepped up their plans to occupy the whole country.



M-94 strip cipher

The strip cipher M-94 was the basic US military cryptosystem in the interwar period and continued to be used widely by the Army and USAAF even as late as mid 1943. As a system it offered limited security and in early 1942 the German Army’s codebreakers solved it. According to the war diary of Inspectorate 7/VI it was solved in May ’42 and traffic from several networks read till July 1943. In German reports the M-94 was called ACr2. Some of the networks read had the indicators CDAF, URSAL, USABU, SENOB as can be seen from the following report taken from the war diary of Inspectorate 7/VI, month of September 1942:



 



The M-94 was used for administrative traffic and by military units at the division level. It was replaced in 1943 by the M-209 cipher machine.

M-209 cipher machine



The US authorities had the SIGABA for high level traffic but lacked a secure device for mid-level traffic. The Swedish inventor Boris Hagelin had developed a small enciphering device called the C-38, which was an improvement of his earlier design C-36 (sold in the 1930’s to the French Army). When he offered this device to the US government it was tested by the SIS and after minor modifications it was produced from 1942 till the end of the war as the M-209. Roughly 140.000 devices were built. The M-209 was used by the US Army at division level and by the USAAF for administrative traffic.

Its first use in the field was during the Tunisia Campaign of 1942-43. According to the war diary of Inspectorate 7/VI the German Army’s codebreakers investigated this traffic in the first half of 1943, ascertained that it was a Hagelin type device and found ways of solving it by using two messages ‘in depth’ (enciphered with the same internal and external settings). By retrieving the internal settings they were able to decode the entire day’s traffic. Their designation for the M-209 was AM-1 (Amerikanische Maschine 1).



Reports based on decoded M-209 messages were first issued in July 1943, as can be seen from the following paragraph taken from the war diary of Inspectorate 7/VI, month of July ‘43:
 



The M-209 continued to be solved till March ’45, with the following list showing date/indicator/frequency/department that solved relative settings/ department that solved absolute settings/date solved.




It’s interesting to note that in 1944 the Germans even built a cryptanalytic device for speeding up the solution of M-209 messages.



Slidex

The Slidex code was a simple bigram substitution table. It was used extensively by the US forces in 1944 during the liberation of Western Europe. Slidex offered very limited security but was well liked by troops because it was easy to use.



The Germans had no problem in solving the messages and reconstructing the Slidex tables. In late 1944 their solution of Slidex traffic from military police units gave them an advantage during the battle of the Bulge. This episode shows that even the compromise of low level codes can sometimes have strategic consequences.

However the limitations of Slidex had not gone undetected and in January ’45 the Signal Division recommended that Slidex be replaced within the U.S. forces.



War Department Telegraph Code

The War Department Telegraph Code was used for administrative traffic. The 1919 edition - SIGRIM was used at the start of the war till 1943-44 when the new version WDTC 1942 edition - SIGARM was introduced.



According to TICOM reports both versions were solved by the Germans. The war diary of Inspectorate 7/VI shows that a codebook A1 (later called AC1) was solved since 1942. It’s not clear to me whether this A1 code was the WDTC 1919 edition or the A1 also used by the State Department.

The next version WDTC 1942 edition was called TELWAby the Germans due to its indicator. It was solved cryptanalytically, in part by taking advantage of a parity check in its values.



It doesn’t seem like TELWA survived the war. A US report from May ’45 said that it would be replaced by a new system and existing copies had to be destroyed.



 
ACAN -Army Command and Administrative Network


The radio network of the US military was called ACAN -Army Command and Administrative Network. During the war it was greatly expanded in order to cover the worldwide deployment of US forces and satisfy their needs for secure and reliable communications.



German radio intelligence agencies intercepted traffic of the ACAN network and were even able to follow the activation and deployment of military units from the US interior to the theatres of war.



 
In this case they took advantage of the limitations of commercial equipment used by ACAN. In the period 1941-43 the network had to be built up fast so some types of equipment that did not fully meet specifications were accepted. The Signal Corps wanted modern radio-teletype equipment of the Baudot type with automatic enciphering and deciphering capability but up to 1943 no device satisfied these requirements.


While new equipment was in development the Signal Corps had to make use of a commercial design. The IBM corporation sold the Radiotypea 6-unit teletype machine that was a commercial success. In 1942 the Signal Corps introduced the Radiotype in military networks and the official history ‘Signal Corps - The Test’ says:



The International Business Machines Corporation had worked out an imperfect solution involving equipment that the firm called radiotype, using, unfortunately, not the standard five-unit teletypewriter code but a special six-unit code. Like a narrow gauge railroad adjoining a standard line, this special code necessitated much hand labor at conversion points where standard teletypewriter texts had to be shifted onto radiotype circuits, and vice versa. Moreover, the standard automatic cipher machines could not function with the six-unit system. Notwithstanding these inconveniences, the Signal Corps early in the war began making use of radiotype, leased from IBM. It was another step in the right direction, toward automatic, high-speed, heavy-duty communications for the Army.’

The same source says that the use of radiotype was extensive during the war: ‘Radiotype would continue to be used considerably. Not till September 1943 would the Signal Corps stop its procurement in favor of radioteletype and not until May 1945 would the Army take its last radiotype out of service (on the WAR-Accra circuit). Then the triumph of radioteletype would be complete.



The main problem with Radiotype was that due to its 6-unit code it could not be connected to a standard cipher machine since these used the Baudot 5-code system. While it was used Radiotype allowed the Germans to intercept valuable traffic up to mid-late 1943 when the new SIGCUM and SIGTOT teletype devices were introduced.



Diplomatic and OSS systems

The State Department’s A1 and C1 codebooks were read by German codebreakers through cryptanalysis. The low level Gray code had been solved in the 1930s and the new Brown code was received from the Japanese in 1941 together with the A1.



The M-138-A strip cipher was the State Department’s high level system and it was used extensively in the period 1941-44. Although we still don’t know the full story the information available points to a serious compromise both of the circular traffic (Washington to all embassies) and special traffic (Washington to specific embassy) in the period 1941-44. In this area there was cooperation between Germany, Japan and Finland. The German success was made possible thanks to alphabet strips and key lists they received from the Japanese in 1941 and these were passed on by the Germans to their Finnish allies in 1942. The Finnish codebreakers solved several diplomatic links in that year and in 1943 started sharing their findings with the Japanese.

German and Finnish codebreakers cooperated in the solution of the strips during the war, with visits of personnel to each country. The Axis codebreakers took advantage of mistakes in the use of the strip cipher by the State Department’s cipher unit.



Apart from diplomatic messages their success against the strip cipher also allowed them to read some OSS -Office of Strategic Services messages from the Bern station.


Naval Cypher No3

Naval Cypher No3 was a British 4-figure code enciphered with subtractor tables. It was used in the Atlantic by the US, UK and Canadian Navies in their convoy operations and its compromise by the codebreakers of the B-Dienst led to heavy losses for the Allied merchant ships.



Since it was a Royal Navy system its failure should be attributed to the British side, however the Americans contributed by withholding their secure cipher machine. In theory the ECM MARK II could have been given to the Royal Navy to equip the units operating in the North Atlantic but this was not done for reasons of national security, since only US personnel were allowed to use this advanced machine and the US authorities did not want to divulge its operating principle even to their closest ally!


Other systems: 

The Germans easily solved several editions of the Division Field Code used by frontline troops, the Bomber code used by USAAF units during the combined bomber offensive, the Combined Assault Codeused during naval landings and the Aircraft movement code used by the USAAF ferry service.



A more interesting case is the AN/GSQ-1 speech scrambler. One of these devices was retrieved from a crashed US fighter plane and the codebreakers of OKW/Chi were able to find a solution, although the chief cryptanalyst Huettenhain was doubtful of whether the key could have been solved in the few hours that the missions took place.   



 

Responding to failures of crypto security



In the course of WWII US Army and Navy cryptologists worked hard to secure existing codes and design new ones. Special teams of codebreakers tried to solve US systems by every way possible. Based on their findings changes were made in operational procedures.

The Army created dedicated radio units called SIAM - Signal Information and Monitoring, whose sole mission was to monitor the radio traffic of US units for violations of signal procedures and cipher security.



A case that shows how seriously cipher security was taken by the US was the Colmar incident. A truck carrying the SIGABA machine of the 28thInfantry division was lost in Colmar, France in February 1945. The vehicle had been stolen by someone while the crew where sleeping indoors. Immediately a huge search was organized to retrieve the vehicle and the cipher material and it was eventually found on 9 March ’45 in a wooded area but with the safe missing. During the same day a French unit found the safe submerged in the Gressen River. After the safe was examined it did not show any signs of being opened nor was material missing. In addition the rotors of the machine were set up in the correct arrangement for 5 February. Even though the examination showed that the material had not been compromised a decision was made to rewire the rotors for all the SIGABA machines in use!


Denying cases of compromise

Based on the information presented so far one would expect that any sign of compromised codes would have led to an exhaustive search for the truth. As with all things in life the reality was more complicated.



Although the Americans placed a high value on crypto security there were some embarrassing cases during the war where they obstinately refused to admit (and in some cases still do) that their systems were compromised.




Fellers code


In 1942 when the codebreakers of Bletchley Park decoded German Enigma messages from the ME theatre, they were surprised to find information that could only have come from the US diplomatic mission in Egypt. This convinced them that a cipher used between Washington-Cairo was being read by the Germans but they found it very, very hard to convince the Americans.



When the US authorities were informed of the affair they refused to believe that they were the source of the leak and instead suspected the Brits of having solved US codes. In his NSA interviews the legendary codebreaker Frank Rowlett says about this affair ‘G2’s reaction was as follows: This could not happen in G2. It could not be Fellers.


 
The code was finally changed on 29 June 1942. According to Rowlett a SIGABA machine was sent to Egypt to replace the MI code.





State Department strip cipher



During WWII there was exchange of information on State Department codes between Germans, Finns and Japanese. The Finnish codebreakers had solved several links in 1942 and in early 1943 they shared their results with the Japanese. These messages were in turn decoded by the British codebreakers and clearly revealed the compromise of State Department systems.



What was the US response when they were told of this? Did they quickly institute several security changes? Let’s have a look:



 

 

The belief of the US officials that their systems were essentially secure meant the Germans and Finns would continue to solve the strip cipher till late 1944!

Office of Strategic Services - Bern station



Several sources state that the messages of the OSS station in Berne were read during the war. It is possible that this only happened when they used State Department codes (for convenience) or when some of their reports of general political nature were given to the US embassy for transmission to the Secretary of State.

It is definitely amusing that the Americans were warned twice to change their codes by none other than Admiral Canaris, head of the military intelligence service Abwehr (through the agent Halina Szymańska) and by General Walter Schellenberg of the Sicherheitsdienst.



The American response was: ‘Dulles made little secret of what he was doing, but he was good at separating valuable informants from Nazi plants, and his codes were never broken. He recognized Schellenberg’s ploy.’

The Brits were smarter than that and their report ZIP/D-S/G.9 of 10th April 1943 says:



 
Conclusion



The USA entered WWII in 1941 with the secure cipher machine SIGABA, the M-138-A strip cipher that offered adequate security but was burdensome to use and a large number of outdated and insecure crypto systems. In the course of the war modern cipher machines were designed and built to replace the old systems and securely cover all types of traffic.

In 1942 the M-209 device was used in the field and in 1943 the cipher teleprinters Converter M-228 - SIGCUM and SIGTOT were introduced in communications networks. In the summer of ’43 a new speech privacy device called SIGSALY became operational and the first system was used on the link Pentagon-London.  At the end of the year the CCM - Combined Cipher Machine was used in the Atlantic and in 1944-45 the British relied on the CCM as much as they did on their own Typex.



By the end of war the US had several types of cipher machines in use, all offering a very high degree of security. Even older systems like the M-138-A and the codebooks were used in such a way that solution was very difficult if not impossible (daily change of key settings, use of one time pads).

However the success of the US in securing its codes should not hide the failures of crypto security that took place during the war. Especially the Fellers case, the A-3 speech scrambler and the State Department’s strip cipher revealed to the enemy valuable intelligence. At the same time several cases of compromise took too long to resolve due to the belief of US officials that their codes were impregnable.



It’s up to historians to cover these historical events in more detail.  

Sources: ‘The Codebreakers’, ‘Hitler’s spies’, NSA interviews of Frank Rowlett (NSA oral histories 1974 and 1976), various TICOM reports, ‘European Axis Signal Intelligence in World War II’, Cryptologia article: ‘The Sigcum story: cryptographic failure, cryptologic success’, Cryptologia article: ‘The ECM MARK II: design, history, and cryptology’, SRH-366 ‘The history of Army strip cipher devices’, Cryptologia article: ‘The Slidex RT Code ‘, ‘Achievements of the Signal Security Agency in World War II’, ‘United States Diplomatic Codes and Ciphers: 1775-1938’, State Department history: ‘Division of cryptography’, War Diary of Inspectorate 7/VI, FMS P-038 ‘German radio intelligence’, ‘Delusions of intelligence’  USS PAMPANITO, British archives- HW 40/132 ,Wikipedia



Pics: M-94 and M-138-A pics from SRH-366, paragraph on State department security survey from ‘Division of cryptography’, solution list for M-209 taken from TICOM DF-120, report on TELWA found in NARA-RG 457- box 797, ACAN pics from FMS P-038, paragraph on AN/GSQ-1 taken from TICOM I-31, strip cipher compromise messages from British archives HW 40/132.

Acknowledgments: I have to thank Rene Stein of the National Cryptologic Museumfor the Frank Rowlett files and TICOM report I-31.

The US AN/GSQ-1 (SIGJIP) speech scrambler

0
0
During WWII all the major participants used a number of speech privacy systems in order to protect their confidential voice communications sent over landline or radio-telephone links.



The US authorities used up to mid 1943 the Bell Labs A-3 speech scrambler, a device that utilized speech inversion. The A-3 was not secure and in the period 1941-44 the Germans were able to decode the conversations in real-time. American cryptologists knew that the A-3 was insecure and they developed the SIGSALY, a device that was a quantum leap in terms of security. However SIGSALY weighed over 50 tons and thus could only be used at prepared sites.

For field communications a speech privacy system was urgently needed but at the time it was not possible to combine a high level of crypto-security with a small and compact design. According to the postwar history of the US Army’s SSA-Signal Security Agency, in page 45, a portable speech scrambler called AN/GSQ-1 - SIGJIP was developed by Bell Telephone Laboratories in collaboration with the Signal Corps and in 1943 the responsibility for its further development was transferred to the SSA. The device was tested and its security was found wanting, however the need for such a device was so great that in 1944 several units were sent to the European, Mediterranean and South Pacific theaters. 


According to TICOM reports the Germans were able to retrieve such a device from a downed Mustang fighter and although they solved the cryptologic system, in practice it was thought that finding the ‘key’ used for each mission would be difficult due to the time element.


From TICOM I-31 ‘Detailed interrogations of Dr. Hüttenhain, formerly head of research section of OKW/Chi, at Flensburg on 18-21 June 1945’




From TICOM I-58 ‘Interrogation of Dr. Otto Buggisch of OKW/Chi’



 

More information on the AN/GSQ-1 is available from the book ‘Information security: An elusive goal’ by George F. Jelen, pages II-17 and II-18. According to the author the AN/GSQ-1 divided the speech signal into 37 -1/2 millisecond segments and then rearranged them based on the ‘key’. In 1944-45 they were installed on P-51 reconnaissance planes operating in Europe and one of the first in service was lost over Germany.The AN/GSQ-1 was not a success and was retired after a brief period of time not only because it offered limited security but also because P-51 pilots preferred to remove the SIGJIP and in its place install a tail-warning radar.




For those who want more info on the AN/GSQ-1- SIGJIP in the US National Archives and Records Administration - RG 457 - Entry 9032 - box 792- there is the file NR 2228 CBLL24 6144A 19450927 ‘PERFORMANCE SPEECH EQUIPMENT AN/GSQ-1 AN/GSQ-1A SIGJIP-SIGMAR’.

New books on Soviet cryptology in WWII

0
0
The role of signals intelligence and codebreaking in WWII has received a lot of attention from historians, especially after the release of new information in the 1970’s (ULTRA story). Information on the successes of the Axis codebreakers is not as easy to find but at least we know the main cases (Fellers code, Naval Cypher No3, etc) fairly well. However the work and successes of the Soviet codebreakers are still shrouded in secrecy.



During Soviet times there was no direct acknowledgment of cryptanalysis of Axis codes.  Soviet histories either glossed over that part of the war or referred to ‘radio-electronic combat’ which was limited to D/F, traffic analysis and jamming. After the breakup of the SU some new books and articles have been published that have a lot of information on the prewar organization and successes of the Soviet codebreaking agencies but not as much regarding their wartime efforts. Recently two new books have been published with more information on Soviet WWII codebreaking: ‘History of cryptology’ by Grebennkov Vadim Viktorovich and ‘The cryptographic front’ by Butirsky, Larin and Shankin.

I have read a chapter from the first book (with the help of google translate) that the author was kind enough to send me. There is certainly new information presented such as the solution of the codes of Germany’s allies in the Eastern Front. As for the second book, according to the table of contents it has the following chapters:



Preface
Chapter 1. History of manual encryption devices
Chapter 2. The twentieth century - the era of rotor cipher machines
Chapter 3. History of telephone speech coders
Chapter 4. Soviet cryptographic service
Chapter 5. Cryptographic WWII. Soviet decryption service
Chapter 6. Exploration and guerrillas
Chapter 7. Worked as a counterintelligence
Chapter 8. The agent radio
Chapter 9. Steganographic transmission means covert messages
Chapter 10. Postwar period
Conclusion
References and Resources


Apparently there is information on the decryption of foreign codes but i don’t have the book and I can’t read Russian. Perhaps a Russian reader of this site can read this book and give an overview.



Some information from the same authors is presented in a series of essays published in site journal.ib-bank.ru :

Pioneers of domestic machine cryptography



In the service of the motherland, mathematics and cryptography



These essays have very interesting information on Soviet cryptology and some of the early cipher machines in use. In ‘Transformation in the fundamental science of cryptology’ i noted these very interesting statements (through google translate) regarding codebreaking:



Meanwhile, this statement is fundamentally wrong. Many results have been achieved "clean" methods, using mathematics and computer science.

…………………………………………….



Meanwhile, many American ciphers were then opened by the combined use of cryptanalytic and engineering methods and powerful computing. However, for the opening of these ciphers such a way - the only possibility.

These statements refer to the Cryptologia article ‘Soviet comint in the Cold war’ by David Kahn, specifically the part in page 7: ‘First, the Soviet Union seems to have gained most of its communications intelligence, not from cryptanalysis, but from bugs and traitors’ and ‘High-level American cryptosystems appear not to have been solved by analysis, but some were read because traitors had sold them to the Soviet Union’.



Perhaps one day when the story of codebreaking during the Cold War is fully revealed it will be interesting to compare the NSA’s efforts with their Soviet counterparts. 

In the news

Viewing all 538 articles
Browse latest View live




Latest Images