Email 12 - Mailbox server

Email Solution · Previous: Relay servers · Next: Webmail

The internal pair forwards mail in two directions: to the relays, for recipients on the internet, and to the third internal server, for recipients in the site's own domain. DC2-A-VCMSX002 receives everything for @ad-dc2.example.net from the internal pair, hands it to Dovecot, and Dovecot files it into a Maildir on a data disk. The same server carries the webmail, which is the only way anyone reads those mailboxes (see Webmail). This Article covers the part between the SMTP port and the file on disk.

The server

Site 1 has the counterpart DC1-A-VCMSX002 for @ad.example.net; the design names it and says no more than that the concept is the same. Only the third server has a data disk. It is also the only one of the five without a twin: the design's requirement of high availability on the application layer is met for sending mail, not for the mailboxes, and the table of affinity rules has an empty cell for this server.

From the pair to the Maildir

flowchart TB
  subgraph pair["Internal pair"]
    q["Postfix, after the encryption filter"]
    t1["transport, ad-dc2.example.net to smtp 10.12.19.43"]
  end
  subgraph mbx["Mailbox server DC2-A-VCMSX002"]
    smtpd["Postfix smtpd, port 25, STARTTLS required"]
    chk["Recipient check, virtual_mailbox_domains and LDAP maps"]
    grp["virtual_alias_maps, distribution groups"]
    t2["transport, ad-dc2.example.net to dovecot"]
    pipe["pipe service dovecot, user vmail"]
    lda["Dovecot deliver -d recipient"]
    auth["Dovecot auth, socket auth-userdb"]
    md["Maildir under /data/vmail"]
    imap["Dovecot IMAP, port 143"]
    rc["Roundcube"]
  end
  ad["Domain controllers, LDAPS"]
  q --> t1
  t1 -->|"SMTP with STARTTLS"| smtpd
  smtpd --> chk
  chk --> grp
  grp --> t2
  t2 --> pipe
  pipe --> lda
  lda -->|"where is the mailbox"| auth
  lda --> md
  chk -.-> ad
  grp -.-> ad
  auth -.-> ad
  rc --> imap
  imap --> md

Postfix: the same file, turned around

The Postfix configuration is the one of the internal pair with a handful of lines changed. The install notes show exactly those lines and refer to the attached files for the rest.

Everything else is identical, down to the files: virtual_domains, restricted_senders and the five LDAP tables are byte for byte those of the pair (virtual_domains, mailbox map, group map). The Config documents of this server: Postfix main.cf, mailbox server, Postfix master.cf, mailbox server, transport, mailbox server.

Because the restrictions are the same, the mailbox server treats the pair like the pair treats a client, with one difference: the pair is in mynetworks and needs no login. It still has to use STARTTLS, since smtpd_tls_security_level = encrypt applies to everybody (see TLS and certificates). The loopback address is not in mynetworks, so Roundcube, which sends through 127.0.0.1:465, logs in with the user's own name and password. That is why Dovecot's authentication service is needed here too (see Dovecot authentication).

The domain is declared a virtual mailbox domain, and the recipients are looked up in Active Directory:

virtual_mailbox_domains = hash:/etc/postfix/virtual_domains
virtual_mailbox_maps = proxy:ldap:/etc/postfix/ad_virtual_mailbox_maps.cf
virtual_alias_maps = proxy:ldap:/etc/postfix/ad_virtual_group_maps.cf
transport_maps = hash:/etc/postfix/transport
dovecot_destination_recipient_limit = 1

virtual_mailbox_maps serves one purpose here, the recipient check: an address is accepted when an object of class person has it as userPrincipalName. The table also returns a path (%d/%u/Maildir/), which would matter to Postfix's own virtual delivery agent. That agent is never asked. There is no virtual_transport in main.cf; delivery is redirected by the transport table, and the path that counts is the one Dovecot computes.

The dovecot transport

The design says that Dovecot acts as the mail delivery agent. Dovecot offers two ways to be one: the LMTP service, to which Postfix speaks over a socket, and the delivery program, which Postfix starts once per message. As built it is the second. /etc/postfix/transport points the domain at a service named dovecot, and master.cf defines it:

# CFG-ON  -> Dovecot daemon is responsible for local mail delivery
dovecot   unix  -       n       n       -       -       pipe
        flags=ODRhu user=vmail:vmail argv=/usr/libexec/dovecot/deliver -e -f ${sender} -d ${recipient}

Postfix's pipe daemon starts deliver as vmail, with the envelope sender and one recipient; dovecot_destination_recipient_limit = 1 makes sure it is one. The flags add X-Original-To, Delivered-To and Return-Path headers and fold the address to lower case. -e makes deliver report a refusal to Postfix instead of composing a bounce itself.

LMTP is nevertheless running. dovecot.conf says protocols = imap lmtp, so Dovecot opens its LMTP socket, and Postfix has the stock lmtp client in master.cf; nothing connects the two. The Source material does not say whether LMTP was tried and dropped or simply came with the template. The same dovecot service line is also in the master.cf of the internal pair, where no transport refers to it.

Dovecot delivery

deliver -d knows a recipient address and nothing else. To find the mailbox it asks Dovecot's authentication service over the socket auth-userdb, which 10-master.conf creates with owner vmail for exactly this caller. The service runs the user databases of auth-ldap.conf.ext, first against the users' OU and then against the devices' OU:

userdb {
  driver = ldap
  args = /etc/dovecot/dovecot-ldap.conf.users
  default_fields = home=/data/vmail/%Ld/%Ln/Maildir/,=mail=maildir:/data/vmail/%Ld/%Ln/Maildir/
}

The directory supplies no path. It only answers whether the account exists, is enabled and is a member of IMAP_ACCESS; the location is assembled from the address, domain and local part in lower case. mail_location = maildir:~/Maildir in 10-mail.conf is the fallback that this field overrides. The files: auth-ldap.conf.ext, dovecot-ldap.conf.users, Dovecot 10-mail.conf, Dovecot 10-master.conf.

The two lookups are not the same question, and the gap between them is worth knowing. Postfix accepts a recipient that is any person with that userPrincipalName. Dovecot delivers only to an enabled account in one of the two OUs that is in IMAP_ACCESS, directly or through nested groups. Reading the filters, a message to an account that has an address but no webmail access is accepted by Postfix and then refused by deliver as an unknown user, which ends as a bounce to the sender. The Source material contains no test of this case; the design only says that webmail access is given by the IMAP_ groups (see Accounts, clients and operations).

Maildir layout and the vmail account

The design fixes the path as /data/vmail/<domain>/<user>/Maildir:

/data/vmail/                          home of the service account vmail
  ad-dc2.example.net/                 one directory per mail domain
    user01/                           one directory per account
      Maildir/                        the mailbox, Maildir format
        cur/  new/  tmp/              one file per message

Maildir was chosen for the reason the design gives: one file per message and one directory per folder, so the file system does the locking. All mailboxes belong to one system account. The users are virtual, they exist in Active Directory and not in /etc/passwd, and Dovecot accesses every mailbox as vmail:

mail_uid = vmail
mail_gid = vmail
first_valid_uid = 501
last_valid_uid = 501

The account and the mail store were created by hand, as root:

$ groupadd -g 501 vmail
$ adduser --system --home /data/vmail --uid 501 --gid 501 --shell /sbin/nologin vmail
$ mkdir -p /data/vmail
$ chown vmail:vmail /data/vmail
$ chcon -R -t mail_home_rw_t /data/vmail/

The install notes list the directory before the account; the chown can only have worked in this order. The uid and gid are fixed at 501 so that first_valid_uid and last_valid_uid can pin Dovecot to this one account. The chcon gives the tree the SELinux type that Dovecot may write to, and a local policy module adds permissions for directories that still carry the default label: dovecot-local-1.1.te. Nothing creates the directories of a domain or of a user. Dovecot does that on the first delivery or the first login.

IMAP, for the webmail only

The design is explicit: IMAP runs on TCP port 143 and is "reachable only from localhost", because its one client is Roundcube on the same host, which connects to 127.0.0.1:143. The configuration is less strict than the sentence:

Taken together, the mailbox server offers unencrypted IMAP with plaintext logins on its network address, and its own firewall lets it through. What keeps it "local" is the network firewall in front of the subnet, whose rule table in the design has no IMAP entry (see Network, DNS and firewall). The install notes have two firewall sections for this server, the first without IMAP and the second with it; they do not say why the second was added. dovecot.conf is the same on the internal pair, so IMAP and LMTP are started there as well, but the pair's host firewall opens only SSH, 25 and 465.

A login over IMAP uses the same password check as SMTP and then the same user database as delivery, so IMAP_ACCESS decides who can open the webmail.

Folders and quota

There is one namespace, the stock inbox. The special-use folders are the distribution's list in 15-mailboxes.conf (Drafts, Junk, Trash, Sent, Sent Messages), without auto = create, so Dovecot creates none of them by itself; they appear when Roundcube creates them. Neither that file nor 15-lda.conf, 20-imap.conf and 20-lmtp.conf carries a local setting, which is why they have no Config documents.

There is no quota. 90-quota.conf is the stock file with empty plugin blocks, and no mail_plugins line loads the quota plugin. The limits that exist are of another kind: message_size_limit = 20971520 in Postfix (20 MiB per message), the size of the /data file system, and the monitoring check on its usage that the design lists for this server.

Distribution groups

Mailing lists are groups in Active Directory, in OU=DISTRIBUTION_GROUPS. Postfix expands them with virtual_alias_maps:

search_base     = OU=DISTRIBUTION_GROUPS,DC=ad-dc2,DC=example,DC=net
query_filter    = (&(objectClass=group)(sAMAccountName=%u))
special_result_attribute = member
result_attribute = mail

The local part of the recipient address (%u) is compared with the sAMAccountName of a group. special_result_attribute = member makes Postfix follow every member DN, and result_attribute = mail takes the mail address of each. The result replaces the original recipient, and each address is then routed on its own: local ones to Dovecot, others to the pair.

Three properties follow from the file. Members are listed by their mail attribute, while mailboxes are found by userPrincipalName, so a group works for local members only as long as the two are equal. The filter looks at the local part alone, with no condition on the domain, so the name of a group matches in front of any domain. And the table is identical on all three internal servers, so a message to a group that is submitted to the pair is already expanded there, and what reaches the mailbox server are the single recipients.

Mail leaving the mailbox server

The only mail that originates here is what users write in the webmail. Messages to the own domain are delivered on the spot. Everything else goes to relayhost = [10.12.19.41], node A of the pair by its own address and not the VIP, with node B as smtp_fallback_relay. On the pair such a message passes the encryption filter like any other and continues to the relays (see Automatic email encryption).

The install notes, step by step

Two things are missing in the notes. transport is a hash: table and needs postmap /etc/postfix/transport after every edit; the command is not there. And the header of the notes describes the server as the mailbox store for @ad.example.net, the domain of site 1, two lines above the correct @ad-dc2.example.net: the notes, like the configuration, were copied from the first server and adapted.

What I would do differently

• The promise "IMAP on localhost only" should be kept by the service itself: listen = 127.0.0.1 for IMAP and no imap service in the host firewall, instead of relying on a firewall elsewhere.
• Postfix and Dovecot should ask the directory the same question. One filter for "has a mailbox", used by virtual_mailbox_maps and by user_filter, would refuse a recipient without a mailbox at the SMTP stage instead of producing a bounce after acceptance.
• The mailbox server is a single machine in datacenter A with all mailboxes on one virtual disk, next to a requirement that asks for high availability on the application layer. The design does not say that the mailboxes are exempt from it; that should have been written down as a decision.