Spanish crypto site
↧
↧
Excellent TED talk
↧
VICE interview with Edward Snowden
↧
Interesting articles
Here are some recent articles that I found interesting:
↧
22 June 1941 - Operation Barbarossa
On June 22 1941 the forces of Nazi Germany attacked the Soviet Union. This was the largest land campaign in history and casualties on both sides number in the tens of millions.
Some interesting aspects of the German invasion:
1). Was Stalin a fool who refused to believe that Hitler would attack him or was the situation more complex?
2). Were Soviet codes impregnable?
3). Were the German shocked by the performance of the new Soviet tanks like the T-34?
4). Some statistics on German and Soviet losses:
↧
↧
Victory!
I recently presented new information on the use of the M-138-A strip cipher by the State Department in the period 1940-44.
However there was a small problem! The entries for the second half of 1944 were not in the microfilm that contains the material for 1940-44.
This meant that I had to find the microfilm with the relevant entries and this was not an easy task. After examining the finding aid for the Department of State Decimal File it was clear that the 119.25/Strip cipher entries would be in film 611.
Thankfully this turned out to be the case and I finally have all the 1944 entries. It will take me a few days to read them and then I’ll add the information in New developments in the strip cipher case.
↧
Missing paragraphs from TICOM report I-22 ‘Interrogation of German Cryptographers of Pers Z S Department of the Auswaertiges Amt’
I uploaded TICOM report I-22 back in 2012, however my copy from the British national archives had several paragraphs deleted in pages 16-18.
I don’t like loose ends, so i was able to copy the same pages from NARA and they don’t have any deletions:
I’ve re-uploaded TICOM I-22 with the new pages in my Google drive and Scribd accounts.
↧
Finding aid for the National Cryptologic Museum Library
A list of the books, reports and other files available at the National Cryptologic Museum Library has been uploaded at the NSA website.
I’ve added the file to my Google Drive and Scribd folders.
I’ve added the file to my Google Drive and Scribd folders.
↧
July 2011 to July 2016 - 5 years of Christos military and intelligence corner
This July marks 5 years since I created the Christos military and intelligence corner blogsite. During this time I’ve written many essays on WWII military, economic and intelligence history, I have attracted a small but dedicated audience and I think that I’ve made valuable contributions to WWII cryptologic history.
Did I start with that goal in mind? No.
Prior to 2011 I was simply a person who had read a lot of books on WWII and occasionally took part in conversations at various internet forums. Back then social networking sites hadn’t taken off so lots of interesting and knowledgeable people frequented internet forums. Some were WWII buffs (like me), others hobbyists, wargamers or aspiring historians. Although internet forums weren’t perfect it was possible to have great debates about battles, weapon systems, strategies, personalities etc.
Some of these individuals had researched these cases thoroughly and they had documents from the archives that contradicted the arguments made in ‘popular’ history books. Through these forums I learned that many of the things I thought to be true because I read them in ‘best selling’ books were in fact completely wrong.
In order for someone to take part in these debates and not look like a fool it was necessary not only to have read a few books on the subject but also to have specialized information from academic journals and from government archives. That’s why my next step was to download several articles from academic journals. I also ordered files from the British national archives through their website.
I not only read this material but I also wrote down the main points and created excel tables with interesting statistics (strength and loss reports for men, tanks, planes etc). Thus I was able to debate some of the ‘old timers’ on an equal footing.
At that time I had read a lot on WWII but there was an aspect of the conflict that I had neglected. That was the role of intelligence(not only spies but also signals intelligence and codebreaking). In 2010 the NSA published on their website the ‘European Axis Signal Intelligence in World War II’ volumes and I happened to find them through a google search. I was impressed with this material and especially the fact that after all these years the operations of the German codebreakers were unknown to the public. I knew of the German Navy’s B-Dienst due to the role that it played in the Battle of the Atlantic but I had never heard of Inspectorate 7/VI or the Luftwaffe’s Chi Stelle.
After reading these reports I knew that it would be interesting to research some of these cases further, so I emailed several people (academics and authors) that were known in the field and asked for their assistance plus i told them about my own findings regarding the Russian radioteletype equipment mentioned in the books ‘Body of secrets: anatomy of the ultra-secret National Security Agency’ and ‘The ultra Americans:the U.S. role in breaking the Nazi codes’ (see Bamford, the Russian ‘FISH’ and Unteroffizier Karrenberg - part 1, part 2, part 3).
Unfortunately I learned that most of them either do not respond to emails or if they do they will just say that they cannot help (or worse).
People in this ‘field’ are weird!
I decided to persevere on my own, so had a quick look online on how to start a blog, how to set it up, how to upload pics etc and I decided to create Christos military and intelligence corner. In the beginning I posted the information i had on WWII statistics and of course my research on the German exploitation of Soviet multichannel radio-teletype networks 1936-1945.
In order to find more material on the work of the Axis codebreakers I contacted a researcher at the British national archives and one at the U.S. National Archives and Records Administration. Thus I was able to copy a lot of the original TICOM reports, i started posting them online at my Google Docs account and I also created a Scribdaccount in 2012.
One thing that this experience taught me is that proper research requires a lot of time and money. That’s why most history books are shit. Authors simply do not have the resources to check the archives thoroughly.
Especially in the field of intelligence things are further complicated by the fact that government agencies hold important files classified for too long and when they release them they do so in a haphazard manner.
Even so I pushed on and I think that I’ve been able to cover almost all the cases that interested me. In order to achieve this I had to spend my own money, I copied material from government archives in the US, UK, Germany and Finland and I’ve also been lucky enough to receive help from like minded individuals.
A lot of people have helped me by giving me information and/or files and I’ve tried to repay them by giving them some of my own material.
I hope that I’ve helped you as much as you’ve helped me!
Toughest cases
You can find my best essays here.
Some of them required a lot of work either in locating the files or in reading and comprehending them. Here are some memorable cases:
1). The Russian FISH case
In the book ‘Body of secrets: anatomy of the ultra-secret National Security Agency’, p15-16 it said:
Within a few days the team struck gold. They came upon an entire convoy of four German signal trucks, complete with four Fish machines, a signals technician, German drivers, and a lieutenant in charge. Arthur Levenson and Major Ralph Tester, a British expert on the Fish, escorted the whole lot, including the Germans, back to England. Once at Bletchley Park the machines were reverse-engineered to determine exactly how they were built and how they operated. (Levenson would later return to Washington and go on to become chief of the Russian codebreaking section at NSA.)
With enough Fish and other equipment to keep the engineers busy for a long time at Bletchley, the team began a manhunt for key German codebreakers. On May 21, 1945, Lieutenant Commander Howard Campaigne and several other TICOM officers interviewed a small group of Sigint personnel being held in Rosenheim. They had all worked for a unit of the Signals Intelligence Agency of the German Abwehr High Command, a major target of TICOM. What the prisoners told Campaigne would lead to one of the most important, and most secret, discoveries in the history of Cold War codebreaking. Their command, they said, had built a machine that broke the highest-level Russian cipher system. The machine, now buried beneath the cobblestones in front of a building nearby, had been designed to attack the advanced Russian teleprinter cipher-the Soviet equivalent of the Fish.
If this was true, it was breathtaking. For over six years US. and British codebreakers had placed Japan and Germany under a microscope, to the near exclusion of Russia and almost all other areas. Now with the war over and with Communist Russia as their new major adversary, the codebreakers would have to start all over from scratch. But if a working machine capable of breaking high-level Russian ciphers was indeed buried nearby, years of mind-numbing effort would be saved.
The Germans, eager to be released from prison, quickly agreed to lead TICOM to the machine. Campaigne wasted no time and the next day the twenty-eight prisoners, dressed in their German Army uniforms, began pulling up the cobblestones and opening the ground with picks and shovels. Slowly the heavy wooden boxes began to appear. One after another they were pulled from the earth, until the crates nearly filled the grounds. In all there were a dozen huge chests weighing more than 600 pounds each; 53 chests weighing nearly 100 pounds each; and about 53 more weighing 50 pounds each. It was a massive haul of some 7-1/2 tons.
Over the next several days the dark gray equipment was carefully lifted from its crates and set up in the basement of the building. Then, like magic, high-level encrypted Russian communications, pulled from the ether, began spewing forth in readable plaintext. Whitaker, who pulled into the camp a short time later, was amazed. "They were working like beavers before we ever arrived," he scribbled in his notebook. "They had one of the machines all set up and receiving traffic when we got there."
The Russian system involved dividing the transmissions into nine separate parts and then transmitting them on nine different channels. The German machines were able to take the intercepted signals and stitch them back together again in the proper order. For Campaigne and the rest of the TICOM team, it was a once-in-a-lifetime discovery. Back in Washington, Campaigne would eventually go on to become chief of research at NSA. Once the demonstration was over, Campaigne had the German soldiers repack the equipment and the next day it was loaded on a convoy, completely filling four heavy trucks. Two TICOM members, including I First Lieutenant Sehner Norland, who would also go on to a long career at NSA, accompanied the equipment and soldiers back to England. There it was set up near Bletchley Park and quickly put into operation. It, or a working model, was later shipped back to Washington. The discovery of the Russian codebreaking machine was a principal reason why both the US. and British governments still have an absolute ban on all details surrounding the TICOM operations.
Initially I wrote about this case in ‘Bamford, the Russian ‘FISH’ and Unteroffizier Karrenberg’ - part 1, part 2, part 3.
I continued to research this case and after locating the reports SI-32 - Special Intelligence and CSDIC SIR 1717 i presented all the available information in German exploitation of Soviet multichannel radio-teletype networks 1936-1945. (note that Randy Rezabek has covered aspects of this case in Case Studies: Russian FISH)
2). Compromise of the State Department’s strip cipher
In the period 1940-1945 the US State Department used the M-138-A strip cipher for encrypting messages classified SECRET. Each embassy had 50 alphabet sets for decrypting circular messages and 50 alphabet sets for direct communications with Washington. The codebreakers of Germany, Finland and Japan were very interested in these messages and during the period 1940-1944 they were able to exploit this traffic.
The German success was made possible thanks to alphabet strips and key lists they received from the Japanese in 1941 and these were passed on by the Germans to their Finnish allies in 1942. The Finnish codebreakers solved several diplomatic links in that year and in 1943 started sharing their findings with the Japanese. German and Finnish codebreakers cooperated in the solution of the strips during the war, with visits of personnel to each country. The Axis codebreakers took advantage of mistakes in the use of the strip cipher by the State Department’s cipher unit.
This has been the hardest case I’ve had to research because the information is scattered in various files, in various collections and in the archives of several countries!
For example I’ve had to copy relevant reports from the US National Archives and Records Administration (OSS, NSA and State Department collections), from the British national archives, from the German foreign ministry’s political archive, from the Finnish national archives, from the Bavarian State Library, from the US National Cryptologic Museum, from books written by Erkki Pale and Aladár Paasonen, from the Japan Center for Asian Historical Records and I’m still not done researching this case!
So far I’ve covered important aspects of this case:
3). Engineering analysis of the Russian T-34/85 tank
The US report Engineering analysis of the Russian T-34/85 tank has a detailed examination of a Soviet T-34/85 tank captured in Korea. I’ve added information from that report in my essay WWII Myths - T-34 Best Tank of the war but locating the report proved to be really, really hard!
The report is mentioned in Osprey books but the author didn’t have a specific reference. I emailed his publisher and they forwarded my request but I never got a response from the author. Instead I tried to find the file at NARA but I was not successful. After emailing the US Army Center of Military History I was told to check with the National Armor and Cavalry Archives and they did have the file but it was in an unpacked box and since they were in the process of unpacking their files they could not copy it for me right away.
By pure luck I saw in a google search that the CIA’s FOIA office listed this file and my researcher went to NARA and copied a few pages from the CIA collection. Thus I was able to confirm that this was the file I was looking for and I requested it from the CIA’s FOIA office.
Then I waited. And waited. And waited…
Then, more than a month later, I got a phone call from the post office asking if i had ordered stuff from the US. The CIA’s FOIA office had gotten my address wrong so the post office people were trying to find where to send the report. After clearing things up I finally got the report, scanned and uploaded it. I also had to pay the CIA’s FOIA office for the copying cost.
4). Inspectorate 7/VI war diary
While looking at the finding aid to the NSA collection RG 457 – entry P11 I saw several files titled Journal/Activity Report, Wehrmacht/Army High Command. For some reason I thought this was the OKW/Chi (Signal Intelligence Agency of the Supreme Command, Armed Forces) war diary and I told my researcher to copy some of the months.
When I got the reports I saw that they were the war diary of the German Army’s signal intelligence service Inspectorate 7/VI and I was not happy. First of all I didn’t think these reports would be very interesting and they were written in German! (Scheiße)
It turns out that they are interesting, very much so, and even though I can’t read German google translate does an adequate job (plus I convinced Frode Weierud to translate some of it).
A friend from the Balkans copied several more of the monthly reports and we exchanged material plus I also located other Inspectorate 7/VI reports (in entry 9032) listed in the sources of the book Delusions of Intelligence: Enigma, Ultra, and the End of Secure Ciphers.
Unfortunately NARA does not have copies of all the monthly reports and many of the ones it does have are of such poor quality that they are practically unreadable. The solution to this problem was clear. I would have to copy the remaining reports from the TICOM collection of the German Foreign Ministry’s Political Archive (Auswärtiges Amt Politisches Archiv).
Together with some friends we formed a team and we copied the material. This was undoubtedly a great success.
5). Books written by former Finnish codebreakers
The Finnish codebreakers solved several foreign cryptosystems during WWII. Their greatest successes were the solution of Soviet military codes and of the State Department’s strip cipher. I was interested in what they had to say about the strip cipher so I tried to find copies of 'Marsalkan tiedustelupäällikkönä' by Aladár Paasonen and ‘Suomen radiotiedustelu 1927-1944’ by Erkki Pale.
It turns out that getting copies of these books is not easy but in the end I got the pages that dealt with the Finnish work on the strip cipher.
My friend Frode Weierud had 'Marsalkan tiedustelupäällikkönä' so I got that part from him. Two friends in Finland had a copy of ‘Suomen radiotiedustelu 1927-1944’ and they sent me chapter ‘DIPLOMAAT TISANOMIAKIN AVATTIIN’. Then it was easy for me to OCR and translate the text.
6). Did the German codebreakers solve the Japanese Purple cipher machine? – Conversation with Otto Leiberich
In the late 1930’s the Japanese Foreign Ministry distributed the Purple cipher machine to its most important embassies and it was used to encipher high level messages to and from Tokyo. Unfortunately for the Japanese the introduction of this new cipher machine wasn’t able to secure their diplomatic communications.
The codebreakers of the US Signal Intelligence Service were able to solve this device in 1940 and according to Russian historians the codebreakers of the Soviet Union, led by Sergei Tolstoy, also solved it. The British codebreakers were not able to solve this system on their own but they received information and a copy of the device from their American allies in 1941.
US reports based on the interrogation of German cryptanalysts claim that the Germans made an effort to solve the Purple cipher machine but were not successful.
There is information pointing to the compromise of this device by the Germans and I’ve presented a summary in German success with Purple?
In 2013 I tried to contact mr Otto Leiberich, chief cryptologist of the German cipher department in the period 1972-1990, because he had written about the Purple machine in his article Vom diplomatischen Code zur Falltürfunktion. Hundert Jahre Kryptographie in Deutschland:
‘Zwei Erfolge verdienen eine besondere Würdigung: die Entzifferung des Purple-Verfahrens der Japaner und die Entzifferung der amerikanischen Chiffriermaschine M 209.
Während des Krieges hatten die Japaner eine Chiffriermaschine entwickelt und zum Einsatz gebracht, die der amerikanischen Aufklärung größte Probleme bereitete. Da gelang es einer amerikanischen Gruppe um den Kryptologen William Friedman, diese Maschine, die als purple machine bezeichnet wurde, zu rekonstruieren und zu entziffern.
Dies gilt seither in Amerika als der größte Erfolg in der Kryptologie-Geschichte. Angeregt durch eine kürzlich ausgestrahlte Fernsehsendung fragte ich bei einem ehemaligen Kollegen nach, der während des Krieges auf diesem Gebiet tätig gewesen war, und erhielt bestätigt, woran ich bis dahin nur eine ungefähre Erinnerung hatte: Auch die Deutschen hatten die Sendungen der verbündeten Japaner bearbeitet, insbesondere die Meldungen, die der japanische Botschafter Oshima aus Berlin nach Tokio sandte. Einer Gruppe von Kryptologen und Technikern der Chiffrierabteilung des Oberkommandos der Wehrmacht (OKW) unter der Leitung des Mathematikers Erich Hüttenhain war die Entzifferung ebenso gelungen wie den Amerikanern. Hin und wieder war ein Bericht schon entziffert und weitergeleitet, wenn Tokio wegen Übermittlungsfehlern um nochmalige Übersendung bitten mußte. Wenn also die Entzifferung der purple machine der größte Entzifferungserfolg während des Zweiten Weltkrieges gewesen wäre (er war es nicht!), so hätten ihn Hüttenhain und sein Team ebenfalls errungen. Leider existieren in Deutschland hierzu keine Unterlagen mehr.’
Während des Krieges hatten die Japaner eine Chiffriermaschine entwickelt und zum Einsatz gebracht, die der amerikanischen Aufklärung größte Probleme bereitete. Da gelang es einer amerikanischen Gruppe um den Kryptologen William Friedman, diese Maschine, die als purple machine bezeichnet wurde, zu rekonstruieren und zu entziffern.
Dies gilt seither in Amerika als der größte Erfolg in der Kryptologie-Geschichte. Angeregt durch eine kürzlich ausgestrahlte Fernsehsendung fragte ich bei einem ehemaligen Kollegen nach, der während des Krieges auf diesem Gebiet tätig gewesen war, und erhielt bestätigt, woran ich bis dahin nur eine ungefähre Erinnerung hatte: Auch die Deutschen hatten die Sendungen der verbündeten Japaner bearbeitet, insbesondere die Meldungen, die der japanische Botschafter Oshima aus Berlin nach Tokio sandte. Einer Gruppe von Kryptologen und Technikern der Chiffrierabteilung des Oberkommandos der Wehrmacht (OKW) unter der Leitung des Mathematikers Erich Hüttenhain war die Entzifferung ebenso gelungen wie den Amerikanern. Hin und wieder war ein Bericht schon entziffert und weitergeleitet, wenn Tokio wegen Übermittlungsfehlern um nochmalige Übersendung bitten mußte. Wenn also die Entzifferung der purple machine der größte Entzifferungserfolg während des Zweiten Weltkrieges gewesen wäre (er war es nicht!), so hätten ihn Hüttenhain und sein Team ebenfalls errungen. Leider existieren in Deutschland hierzu keine Unterlagen mehr.’
Initially I contacted the editorial board members of a journal that dealt with intelligence. Leiberich was also a member of this board but there were no contact details for him. The people I spoke with told me that they could not give me his contact details (which probably makes sense considering his previous government position…).
However his name was listed in the German yellow pages and I decided that I might as well call him and see if I can find out more on the Purple case.
I called twice and he picked up the phone the second time. For some reason I did not really believe that he would be Leiberich the cryptologist and I hadn’t prepared my questions in advance. It had also been a while since I had spoken in English and to make things worse he couldn’t hear me very well!
After asking him if he was Otto Leiberich, the mathematician, and explaining who I was he said that he had written that article a long time ago and he could not remember all the details. The information in the article came from conversations with his coworkersduring their lunch break, especially since some of them had worked in this field during WWII.
I apologized several times for calling him at home and he was interested in the fact that I was calling from Greece (Griechenland).
Moral of the story, it’s probably not a good idea to call government officials at their residence, although in this case I’m glad I did!
7). Carlson-Goldsberry report
As I said previously the Finnish codebreakers solved several foreign cryptosystems during WWII and one of their greatest successes was the solution of the State Department’s strip cipher.
In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).
According to the NSA study History of Venona (Ft. George G. Meade: Center for Cryptologic History, 1995) by Robert Louis Benson and Cecil J. Phillips, it was at that time that the Finns revealed to the US authorities that they had solved their diplomatic codes. On 29 September 1944 colonel Hallamaa met with L.Randolph Higgs of the US embassy in Stockholm and told him about their success.
In response two cryptanalysts were sent from the US to evaluate the compromise of US codes in more detail. They were Paavo Carlson of the Army’s Signal Security Agency-SSA and Paul E. Goldsberry of the State Department’s cipher unit. Their report dated 23 November 1944 had details on the solution of US systems.
Unfortunately I haven’t been able to locate this report at NARA. The NSA’s foia office however has located the file and it has been placed in the review queue. The problem is that it takes a long time for reports to be reviewed and declassified.
We’ll see….
8). Compromise of Polish military intelligence codes and Major Szczesny Choynacki, Polish deputy consul in Bern, Switzerland.
One day, while thinking about the compromise of Polish communications in WWII, I remembered that several sources mentioned a person named Choynacki.
According to Wilhelm Flicke’s ‘War Secrets in the Ether’ a captain Choynacki who collaborated with the office of the Polish military attaché in Bern had agents whose information showed that they were in ‘Hitler’s immediate vicinity’.
Keith Jeffery also mentioned Choynacki in ‘MI6: The History of the Secret Intelligence Service 1909-1949’.
So it was easy for me to put two and two together and I wrote about this case in Polish Stencil codes and secret agent ‘’Knopf’’.
Still important information was missing and I had to wait till Craig McKay covered this case in Major Choynacki’s Ace: the Solution to an Old Puzzle of Wartime Intelligence in order to get the whole story.
9). Referat 12 reports
In 1942 the German Army’s signal intelligence agency Inspectorate 7/VI created a new department to deal exclusively with the solution of enemy agents codes. This was Referat 12 and it was headed by 1st Lieutenant Dr Wilhelm Vauck, a talented mathematician.
I was very interested in locating the reports of Referat 12 and in fact I thought that it would be unlikely that they survived the war.
My first move in tracking them down was to file a FOIA request with the British national archives. Unfortunately that was rejected, so I thought that I would never find them.
Previously I said that I was lucky to find the war diary of Inspectorate 7/VI. Since Referat 12 was a part of Inspectorate 7/VI its reports were included in the war diary, thus I killed two birds with one stone!
That wasn’t the end of this story. Since the reports were in German I used OCR software and google translate plus some parts had to be typed by hand. After translating and studying this material I wrote the essay Allied agents codes and Referat 12.
10). Czechoslovak report ‘Dopady lúštenia šifrovacieho systému čs. londýnskeho MNO z rokov 1940-1945 na domáci odboj’
After writing the essay Svetova Revoluce and the codes of the Czech resistance i’ve tried to find out more on the compromise of Czechoslovak ciphers in WWII. Recently i saw online a reference to the report ‘Dopady lúštenia šifrovacieho systému čs. londýnskeho MNO z rokov 1940-1945 na domáci odboj’ and I tried to locate it.
I emailed a well known Czech academic who is an expert on the Czechoslovak resistance but he did not respond.
I requested this report from the Czech Defense Ministry’s history department but they could not locate it.
I even called the editor of their military history magazine in case he knew how to proceed but he wasn’t interested in this case.
How did I solve this problem? I simply asked Jozef Krajcovic. It turns out that the report is held at the archive of the Museum of the Slovak National Uprising in Banská Bystrica.
11). TICOM report DF-112 ‘Survey of Russian military systems’
The report DF-112 ‘Survey of Russian military systems’ was written in 1947 by Alexis Dettmann (an important member of the German Army’s signal intelligence agency) and it contains lots of information on the solution of Soviet military, NKVD and partisan codes.
I saw this report mentioned in the Cryptologia article ‘Cryptology in the early Bundesrepublik’ and after failing to locate it at NARA I tried to contact the author of the article mr Michael van der Muelen. This proved to be harder than expected but in the end a friend of a friend was able to give me his email.
Mr Muelen sent me a copy of the report and I scanned and uploaded it. Thus I was able to learn a lot about German work on Soviet ciphers.
12). Rommel’s supply convoys
One of the most important questions regarding the war in North Africa, during WWII, is what effect did the sinking of Axis convoys have on the overall campaign. Can Rommel’s defeat be attributed to his lost supplies? Or were the losses tolerable?
In order to answer this question I wanted to find the detailed statistics on what was transported from Europe to N.Africa by the Axis powers.
This wasn’t as easy as you’d think. Books on the subject do not have the actual tables. Instead authors give figures or percentages for some of the months. I wanted all of the data.
I first emailed the owner of a website on the Italian Navy but his response was that ‘I’m limiting my assistance only to academic research’.
I guess the rest of us are the unwashed masses and we don’t need these files…
Anyway, I tried to find another source and I asked Andreas Biermann for this information. He scanned the relevant pages for 1941-42 and I typed the data into an excel file.
Then I was able to write ULTRA intelligence and Rommel’s convoys.
These are just a few of the cases that proved hard to crack. In fact even easy cases had parts that required a lot of work to get right.
Remaining cases
At this time I’m only actively researching the case of the strip cipher. I’m also waiting for several of my cases to be processed by the NSA’s FOIA office.
Hopefully these will be released soon and they will add to our knowledge of WWII history.
Be patient and let’s keep our fingers crossed!
For now enjoy these interesting files:
↧
↧
Book review – ‘Code Warriors: NSA's Codebreakers and the Secret Intelligence War Against the Soviet Union’
Stephen Budiansky, author of ‘Battle of Wits: The Complete Story of Codebreaking in World War II’ and ‘Blackett's War: The Men Who Defeated the Nazi U-Boats and Brought Science to the Art of Warfare’ has published a new book, this time dealing with the Cold War operations of the NSA and the efforts to solve Soviet high level cryptosystems.
‘Code Warriors: NSA's Codebreakers and the Secret Intelligence War Against the Soviet Union’ is a history of the National Security Agency with an emphasis on the work done on Soviet cryptosystems.
The book starts in 1943, when US codebreakers were solving several important Axis cryptosystems (such as the German Enigma and the Japanese PURPLE cipher machines). At the time Soviet diplomatic traffic was being collected but it was only sorted not actively attacked. During the year a small group was formed to study this material and make an attempt at solution.
In theory the Soviet authorities used codebooks enciphered with one time pad which meant that their messages should have been unbreakable. However the ‘Russian problem’ group was able to make a stunning discovery. It turned out that there were cases of additive pad reuse, which meant that some messages could be decoded.
This was the start of the famous VENONAproject and thanks to the decoded messages of the Soviet intelligence agencies it was possible to identify a large number of Soviet agents and communist sympathizers.
NSA and GCHQ continued to solve important Soviet systems in the period 1945-48, such as the cipher machines Coleridge, Longfellow, Pagoda and a modified version of the Hagelin B-211. They were also able to intercept the Soviet civilian network thanks to German equipment, captured in the last days of the war.
Operations came to a standstill in 1948 when after being warned by one of their agents the Soviets introduced new secure cipher procedures. From then on NSA would continue its efforts against Soviet high level cryptosystems but with little to no success and this despite devoting most of its resources to the Soviet problem.
The author looks into the efforts of the NSA to solve Soviet high level cryptosystems, the investments in new technologies such as high speed computers, the crisis resulting from repeated failures and the huge resources devoted to the Soviet problem (at the expense of other targets). In the end the failure to solve Soviet ciphers using the ‘standard’ methods meant that more resources had to be directed to ELINT satellites, ‘bugs’ and traffic analysis. It was only in the late 1970’s that a combination of new supercomputers (built by the Cray corporation) and mathematical research (from the Institute for Defense Analyses) that allowed the NSA to solve Soviet high level ciphers.
Overall the book covers NSA operations from WWII till the end of the Cold War and looks into all aspects of the agency’s work, their codebreaking successes, the relationship with the CIA, their investment into high speed computers, operations in Korea and Vietnam and even the organizational and security problems of running an organization of such size.
There are also five appendixes with short explanations of enciphered codebooks, the Soviet cipher teleprinter (from TICOM sources), cryptanalysis of the Hagelin machines, Turing’s deciban method and Friedman’s Index of Coincidence.
Considering the information presented in the book it is a valuable contribution to Cold War cryptologic history.
Q&A with Stephen Budiansky
The author was kind enough to answer some of my questions.
1). You’ve written several books on signals intelligence and codebreaking. How did you become interested in this subject and how did you go from writing ‘Battle of Wits: The Complete Story of Codebreaking in World War II’ to ‘Blackett's War: The Men Who Defeated the Nazi U-Boats and Brought Science to the Art of Warfare’ and now ‘Code Warriors: NSA's Codebreakers and the Secret Intelligence War Against the Soviet Union’?
I think what first drew me to the subject, and what has been the common theme of all of these books of mine that you mention, is my abiding interest in the intersection of science and warfare. Since World War II, science has dominated warfare, and it’s simply impossible to understand modern military history without the technical story of scientific developments that have shaped conflicts, weapons, and strategy and tactics. It also involves an often fascinating intersection of very different cultures and personalities.
2). There are only a few books that cover the Cold War operations of the NSA. What new information have you discovered for your latest book that set it apart from previous efforts? How hard is it to research cryptologic history compared to social, economic or standard military history?
NSA has been regularly releasing and declassifying documents from the post-World War II period. It’s admittedly slim pickings, and a very frustrating process. I did file several Mandatory Declassification Review requests with NSA in the course of my research and actually received several of the important reports I was seeking to have declassified—only to find that NSA’s declassifiers had redacted so much from the documents that they needn’t have bothered even pretending they were releasing anything: so much was chopped out that there was literally nothing of significant historical information left by the time they were done.
And we’re talking about information that is as much as 70 years old, involving for example Soviet rotor cipher machines from 1947. You can buy a Soviet “Fialka” machine on the collectors’ market these days, but NSA still refuses to acknowledge that such a thing even exists, much less anything about its cryptanalysis.
That said, there are a few significant things that have come out recently, notably in the William Friedman Papers released (or partially released I should say — these too were subjected to the usual heavy-handed redaction censorship) last year.
But the main thing I tried to do which I think is different from earlier books was to pull the clues from disparate sources together, provide essential context, and try as much as possible to synthesize the technical story of cryptanalysis in the Cold War, as I did for World War II in my earlier book “Battle of Wits.” There’s a lot that one can figure out from context, correlating sources, and applying a basic knowledge of cryptology
3). During the Cold War the NSA’s budget and manpower rivaled those of the largest companies in the world. Do you think that this investment paid off for the US government? How can one evaluate the operations of an agency that works in secret?
I’ve repeatedly argued that NSA would be much better off if they were more open. It’s very difficult to get them to reveal their successes, and the result is exactly the problem you note: Why should the American public continue to support these agencies and their activities if they can’t know what they’re getting for the investment? The public tends to hear about NSA when there’s a failure or a scandal.
That said, I do think NSA’s greatest success in the Cold War was preventing us all from being blown to bits in World War III. That’s a negative argument which you can't prove of course. But until the advent of real-time photoreconnaissance satellites in the 1970s and 80s, SIGINT was the primary source of early warning of Soviet military activity and in particular was the only real source of strategic warning of Soviet preparations that would precede a nuclear attack. The reassurance that NSA’s surveillance gave US leaders that the Soviets could not launch a first strike without us having significant warning greatly reduced the hair trigger of the Cold War nuclear standoff. And we specifically know, as I note in the book, that during some key crises in the Cold War—such as the Suez Crisis in 1956—the information NSA provided was crucial in convincing US leaders that Soviet threats of military intervention were a bluff, not backed up by any actual movement or mobilization of its forces, which greatly helped to defuse those crises. It’s not hard to imagine an escalation that could quickly have gotten out of hand had we been in the dark and left to guess what the Soviets were up to.
4). What are your thoughts on the recent Snowden revelations regarding the NSA interception of US civilian communications?
I think they show NSA to be very much a creature of its history. The problems that the Snowden revelations point to regarding NSA’s efforts to “get everything,” to exaggerate the effectiveness of its bulk collection activities, its willingness to press a maximal and at times highly dubious view of its legal authorities, and to misrepresent the truth when confronted with embarrassing facts, are strikingly similar to the mindset and institutional culture of the agency (and its predecessors) going back as early as World War II.
5). What areas of intelligence history do you find most interesting and what topics do you plan to research for future books?
I feel in a way that intelligence history is at a real crisis point. We’ve beaten to death World War II—I mean how many books about breaking the Enigma or Operation Zig Zag do we need? — and the completely broken system of official declassification of post–World War II documents has left intelligence historians with precious little to work on. Until some fundamental change occurs—and it has to come from the top, because the entire FOIA and declassification process is the *problem*, not the solution—I think I’m going to head for another field entirely, if I decide to write another book. I’m a bit worn out from reading documents with every other word crossed out!
↧
Netherlands Intelligence Studies Association 25th anniversary conference 1991 – 2016
On Friday 28 and Saturday 29 October 2016 the Netherlands Intelligence Studies Association (NISA) celebrates its 25th anniversary with an inspiring two-days conference.
Main theme is the strongly changed environment of the intelligence analyst during these past 25 years.
In other words: the 25th anniversary as a symbol for the revolutionary changes in the intelligence world with which analysts have to deal; both external developments (the onset of a multipolar world, asymmetric conflicts, the information revolution), and internal changes (in collecting, processing, dissemination,legitimization and supervision).
These developments forced intelligence analysts and organisations to adapt work processes and methods and techniques. Intelligence analysts still mostly operate in secret, but the demands of intelligence consumers and the public have changed over the last 25 years. Social and technological developments have changed the playing field and the rules of the game for the intelligence analyst, leading to an enormous growth in (publicly) available information and means of communication, and demands for more transparency and accountability. Aim of the conference is to touch on the consequences of this changed environment, and to look ahead.
These developments forced intelligence analysts and organisations to adapt work processes and methods and techniques. Intelligence analysts still mostly operate in secret, but the demands of intelligence consumers and the public have changed over the last 25 years. Social and technological developments have changed the playing field and the rules of the game for the intelligence analyst, leading to an enormous growth in (publicly) available information and means of communication, and demands for more transparency and accountability. Aim of the conference is to touch on the consequences of this changed environment, and to look ahead.
Participants are invited to listen to distinguished experts in the field, and to enter into discussions on various topics relating to intelligence analysis.
The Conference will be held at the Nationaal Archief (the National Archive),
Prins Willem Alexanderhof 20,
The Hague, the Netherlands
Prins Willem Alexanderhof 20,
The Hague, the Netherlands
↧
Update
↧
The US military attaché double transposition cipher
In The compromise of the communications of General Barnwell R. Legge, US military attache to Switzerland I’ve added the following:
Apart from the standard cryptosystems (Military Intelligence Code, War Department Confidential Code. M-138-A strip cipher) US attaches also had an emergency double transposition cipher. According to the instructions for this system, found in the files of Pers Z (decryption department of the German Foreign Ministry) (2):
Use of cipher. To enable M/As to exchange safely secret or confidential messages with other attaches or with assistants or agents acting under their direction, the double transposition cipher is prescribed.
Keys. The keys will be determined by the M/A. They will consist of short phrases consisting of from five to twenty letters. They will be changed at frequent intervals.
However the same numerical sequence was used for both cages, which means that this system would have been vulnerable to cryptanalysis.
↧
↧
Typex security measures
In The British Typex cipher machine i added information from report FO 850/171 (mentioned in the book ‘Alan Turing: The Enigma’):
Countermeasures against cribbing
As an ENIGMA type device (with a reflector) Typex was also vulnerable to the plaintext-ciphertext attacks used by the Allied codebreakers against the German plugboard Enigma. In order to hinder such attacks several measures were employed, such as burying the address in the middle of the text, cyclic encipherment for short messages and insertion of random letters in the text.
For example report FO 850/171 ‘Preparation of telegrams: use of code words: cypher machines and traffic: teleprinter services: en clair messages. Code 651 file 1 (to paper 4968)’ (25) says:
‘When encyphering on the Typex machine, the encyphered version of a letter can never be the letter itself. This sometimes makes it possible to assign with absolute accuracy even a small number of words known or estimated to be in a message to the actual letters of the cypher version by which they are represented. To obviate this danger operators must from time to time press a key not demanded by the text of the message; the additional letters resulting will make the accurate fitting to the cypher version of a piece of clear text quite impossible. Such an insertion should be made on average once in every 10 words while the body of the message is being encyphered; it should be made on average once in every three words during the encypherment of the codress, the prefatory details and the beginnings and endings, whichever of the methods of encypherment in paragraph 25 is being followed; it should also be made on average once in every three words throughout very short messages when they have to be encyphered separately in Typex (see paragraph 27). The insertion should be made within words and not between them.’
↧
Wrong!
During WWII Weber worked at OKW/Chi (Oberkommando der Wehrmacht/Chiffrier Abteilung – Codebreaking department of the Supreme Command of the Armed Forces) and he solved important Japanese diplomatic cipher systems.
The first major system solved was the transposed code J-19 FUJI, used in the period 1941-43. Although some TICOM reports state that he was unable to solve the successor to FUJI this is not correct.
The next main system was also a transposed code (Japanese designation TOKI) and it was solved in the period 1943-45 by OKW/Chi and by the Pers Z agency (decryption department of the German Foreign Ministry).
Why did some Germans say in postwar interrogations that they could not solve it? I don’t know.
Why did the Allied interrogators believe them? I don’t know.
Just clearing things up…
↧
SRH-368 ‘Evaluation of the Role of Decryption Intelligence in the Operational Phase of the Battle of the Atlantic’
The very interesting Special Research History SRH-368‘Evaluation of the Role of Decryption Intelligence in the Operational Phase of the Battle of the Atlantic, U.S. Navy OEG Report #68’ is available from the NSA website.
I’ve also uploaded the pdf to my Google drive and Scribd accounts.
Enjoy!
↧
What happened to my FOIA requests? Progress report
When I started researching WWII cryptologic history I not only copied stuff from the archives but I also requested a lot of material from the NSA’s FOIA office.
Although I have received some interesting reports from them (I-172, I-89, DF-111, DF-105, DF-176, DF-169) they are still reviewing a lot of the TICOM files that I’ve requested. Why do they need to ‘review’ material from the 1940’s? I don’t know.,,
So what’s the progress of these cases?
Let’s see, so far in 2016 i have received information on the following cases:
State Department M-138-A instructions
The M-138-A strip cipher system was used by the State Department during WWII for enciphering messages classified SECRET. The 1944 instructions can be found at NARA but the previous editions from 1939 and 1942 are not there (or at least I haven’t been able to find them).
The NSA FOIA office stated that ‘a thorough search of our archival files was conducted but no records responsive to your requests were located’.
However the 1939 instructions can be found in TICOM report DF-15 so that’s not a complete loss.
Note that the 1939 instructions describe the straight board system (30 letters taken from one cipher column) while the 1944 instructions state that channel elimination is to be used for each message.
Dr Wilhelm Vauck, head of Referat 12 (Agents section)
In 1942 the German Army’s codebreaking agency OKH/Inspectorate 7/VI created a new department tasked with the solution of enemy agents codes. This was Referat 12 (Department 12), headed by the mathematician dr Wilhelm Vauck. According to the short bio of dr Vauck found at the website of Dresden University he survived the war and was a POW for the years 1945-48.
I thought that in that period he would have been interrogated by the Allies about his wartime activities so I requested information from the NSA. The recent response was ‘a thorough search of our historical files was conducted but no records responsive to your request were located’.
Still it is possible that other agencies might have some information on Vauck. We’ll see…
Dr Herbert Lotze, head of Wa Pruef 7/IV/E
Dr Lotze was head of research on speech privacy systems at the German Army’s Ordnance, Development and Testing Group, Signal Branch Group IVe. His team solved the US Bell Labs A-3 speech scrambler and similar Soviet devices.
The response of the NSA to my request for any postwar interrogations of dr Lotze was ‘a thorough search of our historical files was conducted but no records responsive to your request were located’.
However there seems to be material on Lotze at NARA so this isn’t the end of this case.
Other cases
During the year I was able to copy from NARA some of the reports that I had requested from the FOIA office years ago. This is material that was given to NARA in 2015.
Apart from these cases there are about a dozen TICOM reports that are being reviewed by the FOIA office. Let’s hope that they are released soon.
↧
↧
Czechoslovak not Czech – my mistake…
I changed the title of my essay Svetova Revoluce and the codes of the Czech resistance into Svetova Revoluce and the codes of the Czechoslovak resistance.
↧
Another victory!
In my recent essay on The ciphers of Czechoslovakia’s government in exile I mentioned a report titled ‘Dopady lúštenia šifrovacieho systému čs. londýnskeho MNO z rokov 1940-1945 na domáci odboj’.
This report had information on the compromise of Czechoslovak communications and the impact on resistance activities.
It has taken me some time get a copy but thanks to my friend Jozef Krajcovic I finally have the report.
I will soon add some of this information in my essays on Czechoslovak ciphers.
↧
Update
On the contrary the recent article ‘О ВКЛАДЕ СОВЕТСКИХ КРИПТОГРАФОВ В ПОБЕДУ ПОД МОСКВОЙ’, says that in late 1942 the Soviet codebreakers analyzed the Enigma cipher machine and developed ways of solving it. However their efforts failed in January 1943 due to German security measures.
This information seems to be confirmed by the war diary of the German Army’s Inspectorate 7/VI. The March 1943 report of Referat 13 (security of German cipher machines) says that based on the published radio dispatches from Stalingrad Inspectorate 7/VI was asked to give an opinion from the point of view of decipherment.
Schlüsselüberwachung
Auf grund der veröffentlichten Funksprüche asus Stalingrad wurde In 7/VI um ein allgemeines Gutachten gebeten, das die Stellungnahme vom Standpunkt der Entzifferung enthält.
Thus it seems that the Soviet effort to decrypt Enigma messages was identified early and countered by the Germans.
2). In Soviet signals intelligence and the German Enigma cipher machine i added:
According to the recent article ‘О ВКЛАДЕ СОВЕТСКИХ КРИПТОГРАФОВ В ПОБЕДУ ПОД МОСКВОЙ’, in late 1942 the Soviet codebreakers analyzed the Enigma cipher machine and developed ways of solving it. However their efforts failed in January 1943 due to new German security measures.
This information seems to be confirmed by the war diary of the German Army’s Inspectorate 7/VI. The March 1943 report of Referat 13 (security of German cipher machines) says that based on the published radio dispatches from Stalingrad Inspectorate 7/VI was asked to give an opinion from the point of view of decipherment.
Schlüsselüberwachung
Auf grund der veröffentlichten Funksprüche asus Stalingrad wurde In 7/VI um ein allgemeines Gutachten gebeten, das die Stellungnahme vom Standpunkt der Entzifferung enthält.
Thus it seems that the Soviet effort to decrypt Enigma messages was identified early and countered by the Germans.
↧
More Pages to Explore .....
