Credits

Home
Exim Filter Specification - Testing a new filter file Go to the first, previous, next, last section, table of contents.

1.2 Testing a new filter file

Filter files, especially the more complicated ones, should always be tested, as it is easy to make mistakes. Exim provides a facility for preliminary testing of a filter file before installing it. This tests the syntax of the file and its basic operation, and can also be used with ordinary `.forward' Exim Overview
The Exim Mail Transport Agent
Table of Contents Exim Overview Background Availability Limitations Main features Performance Interface Method of operation Mail filtering Directors Routers Transports Exim logs Exim databases SMTP batching Retries Header rewriting Host verification SMTP port reservation Control of relaying Sender verification Sender lock out Receiver verification The `percent hack' Security The Exim Monitor Exim Overview Date: 10 July 1997 Exim is a mail transport agent (MTA) developed at the University of Cambridge for use on Unix systems connected to the Internet. It is freely available under the terms of the GNU General Public Licence. In style it is similar to Smail 3, but its facilities are more extensive, and in particular it has some defences against mail bombs and unsolicited junk mail, in the form of options for refusing messages from particular hosts, networks, or senders. Exim is in production use on a number of sites that move tens of thousands of messages per day. This document contains an overview description of the way Exim works, with a certain amount of omission and simplification to keep it fairly short. Please address any enquiries about Exim to Philip Hazel: Email: <ph10@cus.cam.ac.uk> Phone: +44 1223 334714 Fax: +44 1223 334679 University of Cambridge Computer Laboratory Pembroke Street Cambridge CB2 3QG United Kingdom This document is copyright (c) University of Cambridge 1997, but copying permission is granted to all. ------------------------------------------------------------------------- "If I have seen further it is by standing on the shoulders of giants." (Isaac Newton) Background Exim owes a great deal to Smail 3 and its author, Ron Karr. Without the experience of running and working on the Smail 3 code, I could never have contemplated starting to write a new mailer. Many of the ideas and configuration interfaces are taken from Smail 3, though the actual code of Exim is entirely new. My intention was to write a mailer that had more functionality than Smail 3, but which retained the simple lightweight approach, as this seemed to me to be all that was needed for systems directly connected to the Internet, where most messages are delivered almost immediately. Availability The current distribution of Exim is available from $st{ftp://ftp.cus.cam.ac.uk/pub/software/programs/exim/exim-$si{n.nn}.tar.gz} where n.nn is the version number. The distribution contains an ASCII copy of the documentation; other formats are available from $st{ftp://ftp.cus.cam.ac.uk/pub/software/programs/exim/exim-postscript-$si{n.nn}.tar.gz} $st{ftp://ftp.cus.cam.ac.uk/pub/software/programs/exim/exim-texinfo-$si{n.nn}.tar.gz} The following operating systems are currently supported: AIX, BSDI, DGUX, FreeBSD, HI-OSF (Hitachi), HP-UX, IRIX, Linux, MIPS RISCOS, NetBSD, OpenBSD, DEC OSF1 (aka Digital UNIX), SCO, SCO SVR4.2 (aka UNIX-SV), SunOS4, SunOS5, Ultrix, and Unixware. Limitations For the benefit of those reading this overview to see whether Exim is of interest to them, its limitations are listed first. Exim is written in ANSI C. This should not be much of a limitation these days. However, to help with systems that lack a true ANSI C library, Exim avoids making any use of the value returned by the `sprintf()' function, which is one of the main incompatibilities. It has its own version of `strerror()' for use with SunOS4 and any other system that lacks this function, and a macro can be defined to turn `memmove()' into `bcopy()' if necessary. Exim uses file names that are longer than 14 characters. Exim is intended for use as an Internet mailer, and therefore handles addresses in RFC 822 domain format only. It cannot handle `bang paths', though simple two-component bang paths can be converted by a straightforward rewriting configuration. Exim insists that every address it handles has a domain attached. For incoming local messages, domainless addresses are automatically qualified with a configured domain value. Configuration options specify from which remote systems unqualified addresses are acceptable. The only external transport currently implemented is an SMTP transport over a TCP/IP network (using sockets), suitable for machines on the Internet. However, a pipe transport is available, and there are facilities for writing messages to files in `batched SMTP' format; both of these can be used to send messages to some other transport mechanism such as UUCP. Batched SMTP input is also catered for. Main features Exim follows the same general approach of decentralized control that Smail 3 does. There is no central process doing overall management of mail delivery. However, unlike Smail, the independent delivery processes share data in the form of `hints', which makes delivery more efficient in some cases. The hints are kept in a number of DBM files. If any of these files are lost, the only effect is to change the pattern of delivery attempts and retries. Here is a summary of Exim's main features. More details are given in the sections which follow. Many configuration options can be given as expansion strings, and as these can include file lookups, much of Exim's operation can be made table-driven if desired. For example, it is possible to do local delivery on a machine on which the users do not have accounts. Regular expressions are available in a number of configuration parameters. Domain lists can include file lookups, making it possible to support a large number of local domains. Exim has flexible retry algorithms, applicable to mail routing as well as to delivery. Exim contains header and envelope rewriting facilities. Unqualified addresses are accepted only from specified hosts or networks. Exim can perform multiple deliveries down the same SMTP channel after deliveries to a host have been delayed. Exim can be configured to do local deliveries immediately but to leave remote deliveries until the message is picked up by a queue-runner process. This increases the likelihood of multiple messages being sent down a single SMTP connection. When copies of a message have to be delivered to more than one remote host, up to a configured maximum number of remote deliveries can be done in parallel. Exim supports optional checking of incoming return path (sender) and receiver addresses as they are received by SMTP. SMTP calls from specific hosts and networks, optionally from specific idents, can be locked out, and incoming SMTP messages from specific senders can also be locked out. It is possible to control which hosts may use the Exim host as a relay for onward transmission of mail; the control can be made to depend on the address domain. Messages on the queue can be `frozen' and `thawed' by the administrator. The maximum size of message can be specified. Exim can handle a number of independent local domains on the same machine; each domain can have its own alias files, etc. These are commonly called virtual domains. Exim stats a user's home directory before looking for a `.forward' file, in order to detect the case of a missing NFS mount. Exim contains an optional built-in mail filtering facility. This enables users to set up their own mail filtering in a straightfoward manner without the need to run an external program. There can also be a system filter file that applies to all messages. There is support for multiple user mailboxes controlled by prefixes or suffixes on the user name, either via the filter mechanism or through multiple `.forward' files. Periodic warnings are automatically sent to messages' senders when delivery is delayed -- the time between warnings is configurable. A queue run can be manually started to deliver just a particular portion of the queue, or those messages with a recipient whose address contains a given string. Exim can be configured to run as root only when needed; in particular, it need not run as root when receiving incoming messages or when sending out messages over SMTP. It always uses `setuid()' when doing local deliveries. I have tried to make the wording of delivery failure messages clearer and simpler, for the benefit of those less-experienced people who are now using email. Exim contains support for IPv6. The Exim Monitor is an optional extra; it displays information about Exim's processing in an X window, and an administrator can perform a number of control actions from the window interface. Performance Although I did not specifically set out to write a high-performance MTA, Exim does seem to be fairly efficient. The busiest site I know of is an mailing list exploder that sometimes handles over 100,000 deliveries a day on a big Linux box, the record being 177,000 deliveries (791MB in total). Up to 13,000 deliveries in an hour have been reported. Interface Like many MTAs, Exim has adopted the Sendmail interface so that it can be a straight replacement for `/usr/lib/sendmail'. All the relevant Sendmail options are implemented. There are also some additional options that are compatible with Smail 3, and some further options that are new to Exim. The runtime configuration interface is a single file which is divided into a number of sections. The entries in this file consist of keywords and values, in the style of Smail 3 configuration files. Control of messages on the queue can be done via certain privileged command line options. There is also an optional monitor program called `eximon', which displays current information in an X window and contains interfaces to the command line options. Method of operation When Exim receives a message, it writes two files in its spool directory. The first contains the envelope information, the current status of the message, and the headers, while the second contains the body of the message. The status of the message includes a complete list of recipients and a list of those that have already received the message. The header file gets updated during the course of delivery if necessary. A message remains in the spool directory until it is completely delivered to its recipients or to an error address, or until it is deleted by an administrator or by the user who originally created it. In cases when delivery cannot proceed -- for example, when a message can neither be delivered to its recipients nor returned to its sender, the message is marked `frozen' on the spool, and no more deliveries are attempted. The administrator can thaw such messages when the problem has been corrected, and can also freeze individual messages by hand if necessary. As delivery proceeds, Exim writes timestamped information about each address to a per-message log file; this includes any delivery error messages. This log is solely for the benefit of the administrator. All the information Exim itself needs for delivery is kept in the header spool file. The message log file is deleted with the spool files. If a message is delayed for more than a configured time, a warning message is sent to the sender. This is repeated whenever the same time elapses again without delivery being complete. The main delivery processing elements of Exim are called directors, routers, and transports. Code for a number of these is provided, and compile-time options specify which ones are actually included in the binary. Directors handle addresses that include one of the local domains, routers handle remote addresses, and transports do actual deliveries. When a message is to be delivered, the sequence of events is roughly as follows: If there is a system filter file, it is obeyed. This can check on the contents of the message and its headers, and cause delivery to be abandoned or directed to alternative or additional addresses. Each address is parsed and a check is made to see if it is local or not, by comparing the domain with the list of local domains, which can be wildcarded, or even held in a file if there are a large number of them. If an address is local, it is passed to each configured director in turn until one is able to handle it. If none can, the address is failed. Directors can be targeted at particular local domains, so several local domains can be processed independently of each other. A director that accepts an address may set up a local or a remote transport for it, or it may generate one or more new addresses (typically from alias or forward files). New addresses are fed back into this process from the top, but in order to avoid loops, a director will ignore any address which has an identically-named ancestor that was processed by itself. If an address is not local, it is passed to each router in turn until one is able to handle it. If none can, the address is failed. A router that accepts an address may set up a transport for it, or may pass an altered address to subsequent routers, or it may discover that the address is a local address after all. This typically happens when an partial domain name is used and (for example) the DNS lookup is configured to try to extend such names. In this case, the address is passed back to the directors. Routers normally set up remote transports for messages that are to be delivered to other machines. However, a router can pass a message to a local transport, and by this means messages for remote hosts can be routed to other transport mechanisms such as UUCP. When all the directing and routing is done, addresses that have been successfully handled are passed to their assigned transports. Local transports handle only one address at a time, but remote ones can handle more than one. Each local transport runs in a separate process under a non-privileged uid. If there were any errors, a message is returned to an appropriate address (the sender in the common case). If one or more addresses suffered a temporary failure, the message is left on the queue, to be tried again later. Otherwise the spool files and message log are deleted. Mail filtering Exim can be configured to allow users to set up filter files as an alternative to the traditional `.forward' files. A filter file can test various characteristics of a message, including the contents of the headers and the start of the body, and direct delivery to specified addresses, files, or pipes according to what it finds. The system-wide filter file uses the same control syntax. Directors The existing directors are listed below. I use the RFC 822 term local-part to mean that portion of an address that comes before the @ character. `aliasfile': This director handles local-part expansion via a traditional alias file. The name of the file is obtained by string expansion, and may therefore depend on the local-part or the domain. Generated pipe and file addresses can be (independently) locked out. The `aliasfile' director can also be used to test a list of local parts and direct any messages for them to a specific transport. In this case the data associated with the local part in the file is not used for address expansion, but is available for other purposes. For example, files containing records of the form foo: uid=1234 gid=5678 mailbox=/home_1/foo/inbox could be used on a Exim Specification - 11. Main configuration Go to the first, previous, next, last section, table of contents. 11. Main configuration The first part of the run time configuration file contains the main configuration settings. Each setting occupies one line of the file, possibly continued by a terminating backslash. If any setting is preceded by the word `hide', the -bP option displays its value to admin users only (see section 7.3). All macro definitions must be in this part of the file -- they differ from options settings by starting with an upper-case letter (see section 7.2). The available options are listed in alphabetical order below, along with their types and default values. accept_8bitmime Type: boolean Default: false This option causes Exim to send 8BITMIME in its response to an SMTP EHLO command, and to accept the BODY= parameter on MAIL commands. However, though Exim is 8-bit clean, it is not a protocol converter, and it takes no steps to do anything special with messages received by this route. Consequently, this option is turned off by default. accept_timeout Type: time Default: 0s This sets the timeout for accepting a non-SMTP message, that is, the maximum time that Exim waits when reading a message on the standard input. If the value is zero, it will wait for ever. This setting is overridden by the -or command option. The timeout for incoming SMTP messages is controlled by smtp_receive_timeout. admin_groups Type: string list Default: unset If the current group or any of the supplementary groups of the caller is in this list, the caller has admin privileges. If all your system programmers are in a specific group, for example, you can give them all Exim admin privileges by putting that group in admin_groups. However, this does not permit them to read Exim's spool files (whose group owner is the Exim gid). To permit this, you have to add individuals to the Exim group. allow_mx_to_ip Type: boolean Default: false It appears that more and more DNS zones are breaking the rules and putting IP addresses on the right hand side of MX records. Exim follows the rules and rejects this, giving an error message that explains the mis-configuration. However, some other MTAs support this practice, so to avoid `Why can't Exim do this?' complaints, allow_mx_to_ip exists, in order to enable this heinous activity. It is not recommended, except when you have no other choice. always_bcc Type: boolean Default: false Exim adds a To: header to messages whose recipients are given on the command line when there is no To:, Cc:, or Bcc: in the message. In other cases of missing recipient headers, it just adds an empty Bcc: header to make the message conform with RFC 822. Setting always_bcc causes it to add an empty Bcc: in all cases. This can be helpful in conjunction with mailing list software that passes recipient addresses on the command line. auth_always_advertise Type: boolean Default: true This option is available only when Exim is compiled with authentication support. Normally, if any server authentication mechanisms are configured, Exim advertises them in response to any EHLO command. However, if auth_always_advertise is set false, Exim advertises availability of the AUTH command only if the calling host is in auth_hosts, or if it is in host_auth_accept_relay and not in host_accept_relay. In other words, it advertises only when the host is required always to authenticate or to authenticate in order to relay. Otherwise, Exim does not advertise AUTH, though it is always prepared to accept it. Certain mail clients (for example, Netscape) require the user to provide a name and password for authentication if AUTH is advertised, even though it may not be needed (the host may be in host_accept_relay). Unsetting auth_always_advertise makes these clients more friendly in these circumstances, while still allowing you to use combinations such as host_auth_accept_relay = * host_accept_relay = 10.9.8.0/24 without needing to fill up host_auth_accept_relay with exceptions. auth_hosts Type: host list Default: unset Any hosts in this list that connect to an Exim server as clients are required to authenticate themselves using the SMTP AUTH command before any commands other than HELO, EHLO, HELP, AUTH, NOOP, RSET, or QUIT are accepted. See chapter 35 for details of SMTP authentication. auth_over_tls_hosts Type: host list Default: unset Any hosts in this list must start an encrypted TLS session before issuing an SMTP AUTH command, but it does not of itself require them to authenticate. See chapter 38 for details of SMTP encryption. auto_thaw Type: time Default: 0s If this option is set to a time greater than zero, a queue runner will try a new delivery attempt on any frozen message if this much time has passed since it was frozen. This may result in the message being re-frozen if nothing has changed since the last attempt. It is a way of saying `keep on trying, even though there are big problems'. See also timeout_frozen_after, ignore_errmsg_errors, and ignore_errmsg_errors_after. bi_command Type: string Default: unset This option supplies the name of a command that is run when Exim is called with the -bi option (see chapter 5). The string value is just the command name, it is not a complete command line. If an argument is required, it must come from the -oA command line option. check_log_inodes Type: integer Default: 0 See check_spool_space below. check_log_space Type: integer Default: 0 See check_spool_space below. check_spool_inodes Type: integer Default: 0 See check_spool_space below. check_spool_space Type: integer Default: 0 The four check_... options allow for checking of disc resources before a message is accepted: check_spool_space and check_spool_inodes check the spool partition if either value is greater than zero, for example: check_spool_space = 10M check_spool_inodes = 100 The spool partition is the one which contains the directory defined by SPOOL_DIRECTORY in `Local/Makefile'. check_log_space and check_log_inodes check the partition in which log files are written if either is greater than zero. These should be set only if log_file_path and spool_directory refer to different partitions. If there is less space or fewer inodes than requested, Exim refuses to accept incoming mail. In the case of SMTP input this is done by giving a 452 temporary error response to the MAIL command. If ESMTP is in use and there was a SIZE parameter on the MAIL command, its value is added to the check_spool_space value, and the check is performed even if check_spool_space is zero, unless no_smtp_check_spool_space is set. For non-SMTP input and for batched SMTP input, the test is done at start-up; on failure a message is written to stderr and Exim exits with a non-zero code, as it obviously cannot send an error message of any kind. collapse_source_routes Type: boolean Default: false From version 3.10, this option is obsolete and does nothing. Formerly, it caused source-routed mail addresses to be stripped down to their final components. This now happens automatically, and cannot be suppressed. daemon_smtp_port Type: string Default: unset This option specifies the numerical port number or the service name equivalent on which the daemon is to listen for incoming SMTP calls. It is overridden by -oX on the command line. If this option is not set, the service name `smtp' is used. daemon_smtp_service Type: string Default: unset This option is a synonym for daemon_smtp_port. debug_level Type: integer Default: 0 This option sets the debug level, thus enabling it to be set when calling Exim from an MUA, but it is overridden by the use of -d on the command line. delay_warning Type: time list Default: 24h When a message is delayed, Exim sends a warning message to the sender at intervals specified by this option. If it is set to a zero, no warnings are sent. The data is a colon-separated list of times after which to send warning messages. Up to 10 times may be given. If a message has been on the queue for longer than the last time, the last interval between the times is used to compute subsequent warning times. For example, with delay_warning = 4h:8h:24h the first message is sent after 4 hours, the second after 8 hours, and subsequent ones every 16 hours thereafter. To stop warnings after a given time, set a huge subsequent time. delay_warning_condition Type: string, expanded Default: see below The string is expanded at the time a warning message might be sent. If all the deferred addresses have the same domain, it is set in $domain during the expansion. Otherwise $domain is empty. If the result of the expansion is a forced failure, an empty string, or a string matching any of `0', `no' or `false' (the comparison being done caselessly) then the warning message is not sent. The default is delay_warning_condition = \ ${if match{$h_precedence:}{(?i)bulk|list|junk}{no}{yes}} which suppresses the sending of warnings about messages that have `bulk', `list' or `junk' in a Precedence: header. Note that the colon to terminate the header name cannot be omitted, because brace characters may legally occur in header names. deliver_load_max Type: fixed-point Default: unset When this option is set, no message deliveries are ever done if the system load average is greater than its value, except for deliveries forced with the -M option. If deliver_queue_load_max is not set and the load gets this high during a queue run, the run is abandoned. There are some operating systems for which Exim cannot determine the load average (see chapter 1); for these this option has no effect. deliver_queue_load_max Type: fixed-point Default: unset If this option is set, its value is used to determine whether to abandon a queue run, instead of the value of deliver_load_max. delivery_date_remove Type: boolean Default: true Exim's transports have an option for adding a Delivery-date: header to a message when it is delivered -- in exactly the same way as Return-path: is handled. Delivery-date: records the actual time of delivery. Such headers should not be present in incoming messages, and this option causes them to be removed, to avoid any problems that might occur when a delivered message is subsequently sent on to some other recipient. dns_again_means_nonexist Type: domain list Default: unset DNS lookups give a `try again' response for the DNS error `non-Authoritive host found or SERVERFAIL'. This can cause Exim to keep trying to deliver a message, or to give repeated temporary errors to incoming mail. Sometimes the effect is caused by a badly set up nameserver and may persist for a long time. If a domain which exhibits this problem matches anything in dns_again_means_nonexist then it is treated as if it did not exist. This option should be used with care. dns_check_names Type: boolean Default: true This option causes Exim to check domain names for illegal characters before handing them to the DNS resolver, because some resolvers give temporary errors for bad names. If a domain name contains any illegal characters, a `not found' result is forced. The check is done by matching the domain name against the regular expression specified by the dns_check_names_pattern option. dns_check_names_pattern Type: string Default: see below This option defines the regular expression that is used when the dns_check_names option is set. The default value is dns_check_names_pattern = \ (?i)^(?>(?(1)\.|())[^\W_](?>[a-z0-9-]*[^\W_])?)+$ which permits only letters, digits, and hyphens in components, but they may not start or end with a hyphen. dns_retrans Type: time Default: 0s The options dns_retrans and dns_retry can be used to set the retransmission and retry parameters for DNS lookups. Values of zero (the defaults) leave the system default settings unchanged. The first value is the time between retries, and the second is the number of retries. It isn't totally clear exactly how these settings affect the total time a DNS lookup may take. I haven't found any documentation about timeouts on DNS lookups; these parameter values are available in the external resolver interface structure, but nowhere does it seem to describe how they are used or what you might want to set in them. dns_ipv4_lookup Type: boolean Default: false When Exim is compiled with IPv6 support, it looks for IPv6 address records (AAAA and A6) as well as IPv4 address records when trying to find IP addresses for hosts. However, if dns_ipv4_lookup is set, it disables DNS lookups for AAAA and A6 records. This is a fudge to help with name servers that give big delays or otherwise do not work for these new record types. If Exim is handed either of these record types as part of an MX lookup (for example), it still handles them, and may as a result make outgoing IPv6 calls. All this option does is to make it look only for IPv4-style A records when it needs to find an IP address for a host name. In due course, when the world's name servers have all been upgraded, there should be no need for this option. dns_retry Type: integer Default: 0 See dns_retrans above. envelope_to_remove Type: boolean Default: true Exim's transports have an option for adding an Envelope-to: header to a message when it is delivered -- in exactly the same way as Return-path: is handled. Envelope-to: records the original recipient address in the envelope that caused the delivery. Such headers should not be present in incoming messages, and this option causes them to be removed, to avoid any problems that might occur when a delivered message is subsequently sent on to some other recipient. errmsg_text Type: string Default: unset If errmsg_text is set, its contents are included in the default error message immediately after `This message was created automatically by mail delivery software.' It is not used if errmsg_file is set. errmsg_file Type: string Default: unset This option defines a template file containing paragraphs of text to be used for constructing the message which is sent by Exim in the case of a delivery failure. Details of the file's contents are given in chapter 39. See also warnmsg_file. errors_address Type: string Default: "postmaster" The mail address to which Exim will send certain error reports. As the default is specified without a domain, it will be sent to the domain specified by the qualify_recipient option. If this address is specified with a domain, it must be a fully qualified domain. There are actually only a few situations where this address is used: When freeze_tell_mailmaster is set, and a message that is not a failing, locally generated bounce message is frozen. However, if the errors_address is one of the recipients of the frozen message, nothing is sent, in order to avoid potential loops. Delivery failed, and there is no other address to which a bounce message can be sent, except for bounce messages that are timing out (they are just discarded). -Mg was used to cancel delivery, and there is no other address to which to send a message. errors_copy Type: string list, expanded Default: unset Setting this option causes Exim to send bcc copies of delivery failure reports that it generates to other addresses. The value is a colon-separated list of items; each item consists of a pattern and an address list, separated by white space. If the pattern matches the recipient of the delivery error report, the message is copied to the addresses on the list. The items are scanned in order, and once a matching one is found, no further items are examined. For example: errors_copy = spqr@mydomain postmaster@mydomain :\ rqps@mydomain mailmaster@mydomain,\ postmaster@mydomain Each pattern can be a single regular expression, indicated by starting it with a circumflex; alternatively, either portion (local part, domain) can start with an asterisk, or the domain can be in any format that is acceptable as an item in a domain list, including a file lookup. A regular expression is matched against the entire (fully qualified) recipient; non-regular expressions must contain both a local part and domain, separated by @. The address list is a string which is expanded, and must end up as a comma-separated list of addresses. It is used to construct a Bcc: header which is added to the error message. The expansion variables $local_part and $domain are set from the original recipient of the error message, and if there was any wildcard matching, the expansion variables $0, $1, etc. are set in the normal way. errors_reply_to Type: string Default: unset Exim's delivery error messages contain the header From: Mail Delivery System <Mailer-Daemon@${qualify_domain}> (where string expansion notation is used to show a va

The Exim Mail Transport Agent

Table of Contents

`The Exim Mail Transport Agent`