VENONA Declassified

The National Security Agency's Center for Cryptologic History published a large number of documents about the VENONA project on its Declassification Initiatives section. The VENONA story is a summary of the Intelligence, derived from deciphered VENONA messages, and explains how the codebreakers succeeded in deciphering these important messages.

The top secret VENONA project was initiated in 1943 by the U.S. Army Signal Intelligence Service in Arlington Hall, Virginia, and was continued by its successor, the NSA, until 1980. What started as an attempt to exploit and decipher Soviet diplomatic and trade communications would soon become a vital source of information about Soviet Intelligence operations in the United States. Analysts discovered that portions of the encrypted Soviet diplomatic communications contained espionage related information.

Miniature one-time pad

Richard Hallock, Cecil Phillips and Meredith Gardner were the key players in the VENONA decryption efforts. Analysis identified five different ciphering systems on the diplomatic traffic. The messages were encoded into digits with the aid of different sets of codebooks and additionally enciphered with so-called one-time pads (see image right). These one-time pads, containing series of truly random numbers, are added to the message digits. A one-time pad provides mathematically unbreakable encryption, if used only once.

However, the codebreakers discovered that the Soviets mistakenly reused a small portion of these pads. Time pressure and tactical circumstances during the Second World War lead in some cases to the distribution of more than two copies of certain keys. Although VENONA is often referred to as the project that broke Soviet one-time pads, they never actually broke one-time pads, but exploited a most fatal implementation error: you should never ever reuse a one-time pad.

Nonetheless, the codebreakers faced an enormous challenge. Due to the vast quantity of intercepted messages, the few reused pads and the lack of Soviet codebooks they had to decipher and reconstruct the messages and codebooks painstakenly, piece by piece, solely relying on cryptanalysis. It took 37 years before they closed project VENONA.

From 1946 on, they began to read portions of KGB (Soviet Security Service) messages that had been sent between the KGB station (rezidentura) in New York and Moscow Center. The derived Intelligence was sensational. When VENONA ended, around 3,000 messages (only a fraction of the intercepted traffic) were partially or completely deciphered. These were mostly communications between the KGB's First Chief Directorate (Foreign Intelligence) and its KGB Station Chiefs.

The messages revealed critical information on KGB and GRU (Military Intelligence) operations in the United States and Great Britain, and the KGB's role in the Soviet consulates, the TASS news agency, COMINTERN and the AMTORG Trading Corporation. The decrypts disclosed massive espionage efforts against the U.S. Departments of State and Justice, the Department of the Treasury, the Office of Strategic Services (OSS), and the War Department.

Kim Philby

Information, derived from VENONA, identified many Soviet Intelligence operations, hundreds of Soviet agents and people who collaborated with the Soviets. This enabled the arrest of major Soviet spies such as Klaus Fuchs and Harry Gold (MANHATTAN Project and A-Bomb), the Rosenberg's spy ring, and the identification of Donald Maclean, which lead to the unmasking of "Cambridge Five" members Kim Philby and Guy Burgess.

Because of its importance, and the difficulty to decipher and identify the covernames and codenames in the messages, the VENONA project lasted until 1980, providing the FBI and CIA over the years with vital counter-intelligence information to solve many spy cases. VENONA is a good example of "we will get you, sooner or later", as many spies were arrest upto decades after they stopped spying.

The VENONA story (pdf), many of its deciphered messages and other related documents are found on NSA's VENONA project page. Another very good reference is The Secret Sentry, recently declassified by The National Security Archive. It contains the extensive 66 page VENONA document (alt. link) and other previously top secret documents, related to the Korean war and Vietnam.

Update: A great tip from Mark Stout is the VENONA Names Cross Reference, created by John Taber. It comprises an index of names and identifications, and an index of names to decrypts, both as Excel files. A very extensive work that links thousands of files, names and locations from the VENONA decrypts.