Email 09 - Automatic email encryption

Email Solution · Previous: TLS and certificates · Next: High availability and key synchronization

This is the part of the system that was not installed but written. Every message that leaves the platform for a person outside has to be encrypted for that person, and the applications that send the mail know nothing about keys. So the two internal servers do it for them: a Postfix content filter, about 850 lines of Bash called bash-postfix-encrypt-filter, takes each message, looks for a certificate or a key of the recipient and sends the message on encrypted. The whole script is a Config document: bash-postfix-encrypt-filter.sh.

The requirement

FR12 in Requirements and concept asks for automatic encryption of mail from the addresses of the site's domain (@ad.example.net at site 1, @ad-dc2.example.net at site 2) with S/MIME and PGP/MIME, and for two exception lists, one for senders and one for recipients; mail of a listed address must be forwarded as it is. The requirement and the filter are already in the first concept of 2017–2018, there still in the future tense ("will be written in BASH"). The script header says 2018, version 0.1, and the lines in master.cf that switch it on carry my marker CFG-ON (26.11.2018). The script archived in 2019 is still version 0.1.

MIME, S/MIME and PGP/MIME

MIME is the format that lets a mail carry more than ASCII text: other character sets, attachments, several body parts, each described by a Content-Type header. S/MIME encrypts a MIME entity for the public key in an X.509 certificate of the recipient and wraps the result in a new part of type application/pkcs7-mime. The recipient needs a certificate from a certificate authority, and most mail clients can read S/MIME without an add-on.

PGP/MIME (RFC 3156) does the same with OpenPGP keys: the message becomes multipart/encrypted with two parts, a small one that says "Version: 1" and one that holds the armored OpenPGP data. Because the whole MIME entity is encrypted, attachments are covered, and nobody can tell from outside whether there are any. The recipient needs no authority, only a key pair, and usually an add-on in the mail client. The requirement names both, so the filter does both.

How Postfix hands a message over

Postfix has two ways to run mail through external software. The filter uses the simple one from the Postfix FILTER_README: a pipe service that starts a program for a message, and the program gives the result back with the sendmail command. Three pieces in master.cf and one in main.cf do it.

bash-postfix-encrypt-filter    unix    -   n   n   -   -   pipe
  flags=Rq user=bash-postfix-encrypt-filter null_sender=
  argv=/usr/local/bin/bash-postfix-encrypt-filter.sh -f ${sender} -- ${recipient}

smtp      inet  n       -       n       -       -       smtpd
  -o content_filter=bash-postfix-encrypt-filter:

The smtps service has the same -o content_filter= line.

The last line is what the design means by "Postfix will generate single recipient emails". It has to be so: an encrypted message can only be read with the key it was encrypted for, the script encrypts for exactly one certificate or key, and a message to three people may need S/MIME for one, PGP for the second and nothing for the third.

The script gives the result back with /usr/sbin/sendmail -G -i -f sender -- recipient. -i stops a line with a single dot from ending the message, -G marks it as relayed mail so that Postfix does not rewrite addresses in it, and the comment next to it (from the Postfix example) warns never to add -t: the recipient must come from the envelope and not from the headers. Mail submitted this way enters through pickup, which has no content_filter, so it is not filtered a second time. The other side of that: mail created on the server itself never sees the filter.

The decision flow

The design shows this as figure 5 and as a list of steps. Redrawn from that list and from the script:

flowchart TB
  c["Client: SMTP with STARTTLS, or SMTPS"] --> s["smtpd with content_filter"]
  s --> q["Postfix queue"]
  q --> p["pipe: one filter run per recipient"]
  p --> t["Save message, set To header to this recipient"]
  t --> f{"Sender in exceptions_encrypt_from.txt?"}
  f -- "yes" --> o["sendmail: message as it is"]
  f -- "no" --> r{"Recipient ends with an entry of exceptions_encrypt_to.txt?"}
  r -- "yes" --> o
  r -- "no" --> k{"smime/recipient.cer exists?"}
  k -- "yes" --> sm["openssl smime -encrypt"]
  k -- "no" --> g{"pgp/recipient.asc exists?"}
  g -- "yes" --> pg["gpg --encrypt, wrapped as PGP/MIME"]
  g -- "no" --> inf["Keep headers, replace body by information text"]
  sm --> se["sendmail: result"]
  pg --> se
  inf --> se
  se --> ar["Copy original and result to archive"]
  ar --> cl["Purge old archive, remove work files"]
  o --> cl
  se --> pk["pickup, queue, no filter this time"]
  o --> pk
  pk --> out["Mailbox server or relay"]

Reading the script against the description shows where they differ.

S/MIME: one openssl command

$OPENSSL smime -encrypt -in "$EMAIL_ORIGINAL" -out "$EMAIL_ENCRYPTED" \
    -from "$EMAIL_FROM" -to "$EMAIL_TO" -subject "$EMAIL_SUBJECT" "$EMAIL_TO_CERT" \
    || { echo "$MESSAGE_INSPECT_ENCRYPT_SMIME_PROBLEM"; exit $EX_TEMPFAIL; }

(One line in the script; I wrapped it here.) -encrypt produces an enveloped message for the certificate given as the last argument. -in is the complete original message, headers included, so the original header block travels inside the encrypted part and the mail client finds the original Content-Type and all attachments there after decryption. -from, -to and -subject write a new outer header: the envelope sender, the envelope recipient and the original subject. Nothing else of the old header is outside any more, and the subject stays readable for everybody on the way. No cipher option is given, so the default of the installed OpenSSL decides the algorithm: for the 1.0.2 of RHEL 7 the manual names triple DES.

PGP/MIME: built by hand

GnuPG encrypts data, it does not build mail, so the function encrypt_pgp_mime assembles the MIME structure itself: a boundary from uuidgen, the headers From, To, Subject, Date and MIME-Version copied from the original with formail, a new Content-Type: multipart/encrypted, the fixed first part, and then the encrypted second part.

$GPG --list-keys $EMAIL_TO_KEY
if [ $? -ne 0 ]; then
    $GPG --import $EMAIL_TO_KEY || { echo $MESSAGE_PGP_KEY_CANNOT_IMPORT; exit $EX_TEMPFAIL; }
fi

$GPG --no-verbose --no-tty --quiet --batch --yes --encrypt --output - \
     --armor --textmode --always-trust -r $EMAIL_TO $PGP_MIME_EMAIL_BODY >> $EMAIL_ENCRYPTED

(Shortened: the logging line is left out.) --batch, --no-tty and --yes keep GnuPG from asking anything, --armor gives ASCII output, --output - writes it to standard output so that it can be appended to the message under construction, and -r selects the key by the mail address of the recipient. --always-trust skips the trust check; without it GnuPG would refuse a key nobody has signed. The trust decision is therefore made by whoever copies the key file to the server. What gets encrypted is the original Content-Type header plus the original body, which is a complete MIME entity as long as the message is multipart.

The .asc file is only the trigger and the source for the import. Encryption uses the keyring of the service account in its home directory, and the first block is meant to import the key once. It asks --list-keys for the file path, though, which no key matches, so as far as I can read it the import runs on every message; importing a key again is harmless.

No key: the information message

When there is neither a certificate nor a key, the recipient gets the original headers and, as body, a short text: a message from this sender was addressed to you, our system has no S/MIME certificate or PGP public key of yours, please send one to a named mailbox. The content is not delivered. It is not returned to the sender either: the sender is not told. The original survives only in the archive.

Account, directories and keys

The install notes ran this as root on both internal servers.

$ yum install openssl gnupg2 procmail awk
$ useradd bash-postfix-encrypt-filter
$ cp /tmp/bash-postfix-encrypt-filter.sh /usr/local/bin/
$ chown bash-postfix-encrypt-filter:root /usr/local/bin/bash-postfix-encrypt-filter.sh
$ chmod 500 /usr/local/bin/bash-postfix-encrypt-filter.sh
$ mkdir -p /var/spool/postfix/bash-postfix-encrypt-filter/work
$ chown -R bash-postfix-encrypt-filter:root /var/spool/postfix/bash-postfix-encrypt-filter/
$ chmod -R 700 /var/spool/postfix/bash-postfix-encrypt-filter

The mkdir was repeated for each directory of the table. procmail is installed for its formail tool.

Adding a recipient is a copy, on either node; the file name must be the recipient address exactly as it appears in the envelope.

$ cp /tmp/email@address.com.cer /var/spool/postfix/bash-postfix-encrypt-filter/smime/
$ cp /tmp/email@address.com.asc /var/spool/postfix/bash-postfix-encrypt-filter/pgp/

Owner, mode and the copy to the other node are the business of the synchronization services in High availability and key synchronization. Postfix runs confined, so postfix_pipe_t needed rules to execute gpg and to write under the spool: SELinux module postfix-local.

The exception lists

exceptions_encrypt_from.txt holds fifteen sender addresses at site 2: the notification mailboxes of the platforms, the second-level support mailboxes and the monitoring sender. The Source material does not say why these; the obvious reading is that their mail goes to recipients nobody can ask for a key. The comparison is exact. The list is only as safe as the envelope sender: the design intended sender-login matching, but as archived the order of the sender restrictions defeats it, so any authenticated account in SMTP_ACCESS can send as a listed address and bypass encryption; see Internal servers: Postfix. exceptions_encrypt_to.txt holds one line, @ad-dc2.example.net, compared with the end of the recipient address: mail to the site's own mailboxes is not encrypted. The design notes that entries must be lowercase; nothing in the chain folds the addresses, so the client has to send lowercase too. Neither file is synchronized between the nodes.

Log and archive

Every step writes a line date MAIL-ID=<pid>:text to log/bash-postfix-encrypt-filter.log: message entered, backup made, recipient set, which encryption is possible, what was sent. The file is rotated daily and kept seven days by a logrotate rule; for that the log directory was labelled like /var/log, as root.

$ semanage fcontext -a -e /var/log /var/spool/postfix/bash-postfix-encrypt-filter/log
$ restorecon -R -v /var/spool/postfix/bash-postfix-encrypt-filter/log/

With ARCHIVE_ORIGINAL=1 and ARCHIVE_ENCRYPTED=1 the untouched original and the encrypted or information message are copied to archive/<zone>-<date>/<hour>-<minute>/<recipient>/. Every run ends with find -mtime +1 over the archive and deletes what is older. For that time the server holds, in clear text, exactly the mail that had to be encrypted, readable for the service account and root only.

When something fails

Every error path of the script ends with exit code 75, EX_TEMPFAIL. For the pipe service that means a temporary failure: the message stays in the queue and Postfix tries again later, until the queue lifetime is over and the sender gets a bounce. The script never asks for a bounce itself; EX_UNAVAILABLE is defined and not used. A failed run leaves its work directory behind, with the message in it.

Limits and known weaknesses

These are in the script for anyone who reads it; none of them comes from a recorded incident.

• A failed gpg --encrypt is not noticed. Its exit code is not tested. If no usable key matches the address, the message goes out with an empty encrypted part.
• The closing boundary of the PGP/MIME message has no trailing --. Strictly, the multipart is not terminated.
• A single-part body in base64 or quoted-printable loses its Content-Transfer-Encoding header on the PGP path, because only Content-Type is copied into the encrypted part.
• The information message keeps the old Content-Type. If that was multipart/mixed, the plain text body does not fit the header any more.
• Subject, sender and recipient are never encrypted, with either standard.
• Three error messages use variable names that are not defined, and logging cannot be switched off; details are with the script.
• Every recipient costs a Bash process, a handful of child processes and a find over the archive. The only item in the script's own TODO list is a benchmark of the exception lookup.

Checked against OpenSSL 3.5 and GnuPG 2.5

Current at the time of this check: OpenSSL 4.0.3 with 3.5 as the long-term branch, GnuPG 2.5.24, Postfix 3.11.

The Postfix side has not aged at all. The cryptography has: triple DES and unauthenticated CBC were ordinary in 2018 and are what EFAIL and S/MIME 4.0 moved away from. A filter written today would call openssl cms with an AES-GCM cipher, or would not be written: the CipherMail community gateway is an actively maintained project, and zeyple is one for OpenPGP only. I have not tested either.

• Postfix after-queue content filter
• openssl-cms
• RFC 8551, S/MIME 4.0
• EFAIL