How do you secure email? (Part Deux)

My last posting on securing email ended on a heart-stopping cliff-hanger, and I know the suspense has had you reaching for the soothing medication. But rest easy, dear reader, as our tale may now resume.

As you may recall, Toby and Violet’s plans to exchange secure email have run aground. Neither Toby nor Violet can figure out how to trust each other’s public keys. The security of secure email is built on the assumption that public keys are genuine – no trust, no secure email.

And so Violet and Toby do the simple thing. They meet at Tully’s for coffee and physically exchange public keys. Violet knows Toby’s public key is genuine because he gave it to her. She can safely verify his digital signatures and accept his email.

But what works for two doesn’t work for 637. For Toby wants to send secure email to 637 of his closest Facebook friends (the remaining 1039 worthy only of the Wall), many of whom aren’t even in the same city. Will Toby travel the world having coffee with each of his friends? Toby is eager to sip cappuccino in Florence and Lisbon, but his financial advisor (me) mandates otherwise.

Certificates of Authenticity

Toby recognizes that his public key needs a proper certificate of authenticity. Yes, sort of like the one that came with your prized William & Kate commemorative plate (I know you have one).

Toby heads to Certificates-R-Us, a well known Certificate Authority. Certificates-R-Us issues X509 Certificates – standardized digital packages that contain a public key, a statement about who the key belongs to, and evidence of the key’s authenticity.

Toby’s certificate acquisition process goes something like this:

  • Certificates-R-Us asks Toby for his papers. Toby produces his Government issued ID (Driver’s License) as evidence that he is the famed canine.
  • Certificates-R-Us has a higher bar for identity proofing than its competitor Certificate-Mart. It demands more information from Toby, such as his bank account or credit card number. Certificates-R-Us uses Toby’s personal information to conduct a series of validation checks:
    • It hires a credit agency to verify Toby’s identity by confirming his financial credentials.
    • It verifies Toby’s home address (taken from his license) by sending him snail mail
    • It calls Toby’s home phone number and listens to his melodious bark
  • Fortunately for Toby, his identity checks out.
  • Certificates-R-Us now proceeds with the certificate issuance:
    • It creates a new (public, private) key pair for Toby
    • It very carefully transfers the private key to Toby. Toby stores this key in a very secure store (Attila the Hound is always on the prowl, after all) and uses it to sign his email.
    • It issues a X509Certificate containing Toby’s public key and a statement that he is the Subject to whom the certificate was issued.
    • It collects processing fees from Toby (darn capitalism).


X509 Certificates in daa email

Toby signs his email to Violet with his spiffy new private key, then attaches his shiny new certificate to it. The certificate bears his name, and he is very proud of it. He also posts his X509 Certificate in a public directory to enable his many friends to download and use to encrypt the email they send him.

But how does any of this help? How does Violet know this isn’t another of Attila the Hound’s forgeries?

You guess rightly. Digital signatures to the rescue once again. What works for secure email, and documents, also works for certificates.

Certificates-R-Us signs Toby’s X509 Certificate with its own private key. The signature certifies the authenticity of Toby’s public key. Certificates-R-Us can confidently certify Toby’s public key because Certificates-R-Us created both Toby’s (public, private) key pair and the certificate.

And we’re done, right? Nope. I still have half my weekly quota of words to write.

To trust Toby’s certificate, Violet checks that the Subject of the certificate matches the sender (Toby’s) name. She validates the digital signature of the certificate issuer. She decrypts the signature with the issuer’s public key so that she can compare the issuer’s hash……and behold, we’re right back to where we started.

Because while verifying the issuer’s signature, Violet encounters yet another public key she cannot trust – the issuer’s !

Who do you trust?

In the real world, you trust someone because:

  • You chose to trust them explicitly
    • You know the person
    • The person has an honest face (more likely, good looking face – trust isn’t always rational)
    • You take their word for it (possibly because they honestly have a good looking face)
    • Etc .
  • You trust them because you explicitly trust somebody else who trusts them. Or attests to their trustworthiness. Or issues them a Photo ID. Or “likes” them on that website. Somebody vouches for somebody who vouches for somebody who vouches for…. This is called trust delegation.

You always end up trusting somebody explicitly. You can delegate and federate (and complicate) trust all day long – but in the end, that chain of delegation must end somewhere. The trust chain has to terminate at somebody you explicitly trust.

What is true in real life is also true for certificates. Toby’s certificate also has a certification path or trust chain.


Chains of Trust

Toby’s certificate issuer – Certificates-R-Us – also has a certificate; an intermediate or subordinate certificate issued by the well regarded Certificate Authority Woo Hoo Corp. This certificate contains – yes – the public key for Certificate-R-Us.


The Woo Hoo Corp Root Authority also has a certificate. This root certificate is special because no other authority certifies the public key the certificate contains. But since all X509 Certificates must be signed by someone, a root certificate is self-attested or self-signed. Woo Hoo Corp issues certificates to multiple subordinate Certificate Authorities – such as “Certificates-R-Us Asia” or “Certificates-For-The-Masses”.


How do you trust?

Violet can trust Toby’s certificate in 3 ways:

  • She can trust Toby’s certificate explicitly
  • She can trust the certificate for the authority that issued Toby’s certificate – Certificates-R-Us.
  • She can trust the certificate for the authority (Woo Hoo Corp) that issued the certificate for the authority (Certificates-R-Us) that issued Toby’s certificate (say that really quickly – thrice)

In the Direct Project parlance, Violet’s trusted certificates are her Trust Anchors. Violet picks the anchors she trusts, and stores them in her trust anchor store, or anchor list.

Your browser and your computer come pre-installed with the certificates of several highly trusted large Certificate Authorities, managed by vendors such as VeriSign. Every time you create an SSL connection to order your prescription from, your browser verifies the server’s identity, by going through a procedure very similar to Violet’s. There are a few other details involved, naturally, so how SSL works is best saved for another post.


Circles of Trust

The higher up the trust chain you decide to anchor your trust at, the wider the circle of trust you belong to. By trusting a Certificate Authority (CA), you make it simple for yourself to exchange secure email with an entire trust community – the community of individuals and organizations who hold certificates issued by the CA. The community admits only trusted parties– only those who meet the security and privacy policies outlined by the community members. The benefits of admission is an admission card – your X509 Certificate issued by the community’s Certificate Authority. Use the keys associated with member X509 Certificates to sign and encrypt email and have it accepted and trusted anywhere in the community.

You don’t have to pay Certificates-R-Us to set up a community or to issue certificates to your members – you can do it for free (now I have your attention – see below). Your community can set up its own Certificate Authority by running one of many commercial and open source certificate management servers. It doesn’t really matter as long as your community members install your CA certificate in their trust store.

The beauty and simplicity of this model is the powerful premise behind the Direct Project. Its not a new idea. We owe it to the geniuses who invented modern crypto & asymmetric encryption. And the designers of S/MIME.

Trusting a Certificate Authority has its costs. If you trust Certificates-R-Us, you trust any and all certificates issued by Certificates-R-Us – a possibly large community. If you trust Woo Hoo Corp, you trust half the planet. If you trust too broadly, you become too trusting. So trust wisely.

Creating your own Circle of Trust

To create a trust community, you must first create a Certificate Authority (public, private)key pair and X509 Certificate. You can then start issuing certificates (and keys) to your members. You can do this very cheaply by using the free tools: OpenSSL (open source) or MakeCert.exe (free in Windows).

Learn how to use makecert.exe using these sample batch files.

  • genca.bat: Create a new certificate authority certificate (root).
  • gencert_exchange.bat: Issue a new certificate to use for secure email – issued signed by the authority you created in genca.bat

You do need to be careful with the all the private keys you generate for your community members. After securely transfering them to the owning member using a secure out of band mechanism (such as password protected PFX files) – do ensure they are wiped from the machine you created them on. For Attila the Hound prowls endlessly.

You can find detailed documentation on makecert and PFX files MSDN and the Web.


How encrypted email works

I’ve been working on the Direct Project for the past year or more. The Direct Project is a federally sponsored initiative that uses secure email as the foundation for the ubiquitous nationwide exchange of health information.

To secure an email, you have to, among other things, encrypt the message content. It is no surprise that many newcomers to Direct want to know how encrypted email works. Others, who are comfortable with classic message security, notice that unlike point to point messaging (one sender, one receiver), email is inherently multicast (one sender, many receivers). They ask: how do you encrypt email sent to multiple recipients?

In this inaugural posting for my new blog, I will try to answer both questions in plain English.

Encryption Basics

First, a quick refresher on encryption concepts:

  1. Key: An array of carefully generated bits, used to encrypt and decrypt data.
  2. Encryption: You use a key (secret) and a precise series of complicated steps (encryption algorithm or cipher) to mangle (encrypt) data into undecipherable gibberish.
  3. Decryption: You use a key (secret – hopefully the right one) and a precise series of complicated steps (decryption algorithm or cipher) to un-mangle (decrypt) gibberish back into your original data. If you use the wrong key, or the wrong algorithm, you turn the source gibberish into more gibberish.
  4. Symmetric Encryption: You use the same key to both encrypt and decrypt the data. Both the sender and the receiver have a copy of the same keya shared secret. To share the secret, the sender and receiver must exchange their shared key securely – without an attacker getting a peek. If an attacker can somehow (silently) intercept an inadequately protected secret as it moves from sender to receiver (steaming open the envelope, so to speak), the attacker can also decrypt your encrypted data.
  5. Asymmetric Encryption: You use one key (public) to encrypt the data and an associated but different key (private) to decrypt the data. Data encrypted with your public key can only be decrypted with your associated private key. You boldly give the public part of your key pair to anybody you want to receive encrypted data from. You keep your private key secret and and use it to decrypt data that people send you. Unlike symmetric encryption, there is no shared secret to exchange. You can distribute your public key to the entire world without fear. Data encrypted with your public key is truly for your eyes only – because only you can decrypt it with the secret private key that only you have.The reverse is also true. Data encrypted with your private key can only be decrypted using your public key. This property has important implications for digital signatures (more in future posts).

Symmetric and Asymmetric encryption work differently, – they use different types of keys and different encryption/decryption algorithms.

Symmetric encryption is fast. Asymmetric encryption is slow.

How does email encryption work?

Violet wants people to encrypt the email they send her. To help them do this, Violet creates a (public, private) key pair. She wraps up her public key in a secure package called an X509 Digital Certificate (more on this in future posts) and gives the certificate containing the public key to those she is corresponding with. To make it easy for others to find her public key, she even publishes her certificate in a public directory.

Violet’s good friend Toby Toby decides to send her some encrypted email.

All Toby has to do is use Violet’s public key to encrypt the message, right? Wrong.

To use Violet’s public key to encrypt his email, Toby must use asymmetric encryption. Which, unfortunately, is slow. Toby cannot practically encrypt the content of his email using Violet’s asymmetric public key – it takes too much work!

To encrypt his email content, Toby needs a faster option – symmetric encryption. Toby generates a new symmetric encryption key and uses this key to efficiently encrypt the content of his email.

But how does Violet decrypt Toby’s email? To decrypt, Violet needs a copy of the symmetric encryption key, which she doesn’t have because Toby generated it on the fly and hasn’t given it to her yet! How does Toby securely send Violet a copy of his encryption key?

Toby cleverly solves the problem by attaching the encryption key to the email itself. The message brings its own key with it.

But isn’t that crazy? Anybody can now grab the key and decrypt the email, right? Wrong.

The clever Toby encrypts the symmetric encryption key before attaching it to the email. He does this using Violet’s public key, which he had obtained earlier. And even though this requires slow asymmetric encryption, the performance conscious Toby doesn’t mind because the encryption key is relatively small – usually only 256 bits long at most.

Toby sends his email to Violet. Naturally, Toby does not encrypt the addressing information on the message – the To & From – which have to travel in the clear, just like the addressing information on the envelope of a sealed snail-mail letter. Email servers use the addressing information to transport the email to its destination.

When Violet receives the email, she decrypts the attached encryption key using her private key. She then uses the encryption key to decrypt the email content and receives Toby’s friendly missive.

How do you encrypt email sent to multiple recipients?

Toby wants to send an email message to both Violet and Margaret. How does he encrypt this message?

Should Toby repeat the encryption process twice? Encrypt the email once for Violet and again for Margaret? And what happens if Toby also puts Gitanjali on the To line? Does Toby have to encrypt the message three times? And send out 3 different copies of the same message? Isn’t that getting really inefficient?

Toby has a much better idea. Just like before, he encrypts the email exactly once, using a symmetric encryption key. Then he attaches multiple copies of the same encryption key to the message – one for each recipient and encrypted with that recipient’s public key. Toby encrypts one copy of the encryption key with Violet’s public key. He encrypts a second copy with Margaret’s public key and third with Gitanjali’s. Then he attaches the 3 copies to the message.

When Margaret receives the email, she locates the copy of the encryption key that was intended for her. She decrypts the encryption key, then uses it to decrypt Toby’s note. Violet and Gitanjali do the same.

You can use the same technique to encrypt email sent to as many recipients as you like. Every new recipient merely means the small overhead of an additional attached copy of the encryption key.


You should now have a high level notion of how email encryption works. Those of you who are interested in the gory details should deep dive into S/MIMEthe defacto standard for securing email. Please do peruse the S/MIME and Direct Transport specs for a bit by bit commentary.

It takes more than encryption to secure email. See my follow up posts to learn how:

Source Code

The open source Direct Project Reference implementation contains a full S/MIME and secure email implementation. To learn how to encrypt and sign email and email content in C#, check out the SMIME source code.