Credits

Home
Exim Filter Specification - Testing a new filter file Go to the first, previous, next, last section, table of contents.

1.2 Testing a new filter file

Filter files, especially the more complicated ones, should always be tested, as it is easy to make mistakes. Exim provides a facility for preliminary testing of a filter file before installing it. This tests the syntax of the file and its basic operation, and can also be used with ordinary `.forward' Exim Overview
The Exim Mail Transport Agent
Table of Contents Exim Overview Background Availability Limitations Main features Performance Interface Method of operation Mail filtering Directors Routers Transports Exim logs Exim databases SMTP batching Retries Header rewriting Host verification SMTP port reservation Control of relaying Sender verification Sender lock out Receiver verification The `percent hack' Security The Exim Monitor Exim Overview Date: 10 July 1997 Exim is a mail transport agent (MTA) developed at the University of Cambridge for use on Unix systems connected to the Internet. It is freely available under the terms of the GNU General Public Licence. In style it is similar to Smail 3, but its facilities are more extensive, and in particular it has some defences against mail bombs and unsolicited junk mail, in the form of options for refusing messages from particular hosts, networks, or senders. Exim is in production use on a number of sites that move tens of thousands of messages per day. This document contains an overview description of the way Exim works, with a certain amount of omission and simplification to keep it fairly short. Please address any enquiries about Exim to Philip Hazel: Email: <ph10@cus.cam.ac.uk> Phone: +44 1223 334714 Fax: +44 1223 334679 University of Cambridge Computer Laboratory Pembroke Street Cambridge CB2 3QG United Kingdom This document is copyright (c) University of Cambridge 1997, but copying permission is granted to all. ------------------------------------------------------------------------- "If I have seen further it is by standing on the shoulders of giants." (Isaac Newton) Background Exim owes a great deal to Smail 3 and its author, Ron Karr. Without the experience of running and working on the Smail 3 code, I could never have contemplated starting to write a new mailer. Many of the ideas and configuration interfaces are taken from Smail 3, though the actual code of Exim is entirely new. My intention was to write a mailer that had more functionality than Smail 3, but which retained the simple lightweight approach, as this seemed to me to be all that was needed for systems directly connected to the Internet, where most messages are delivered almost immediately. Availability The current distribution of Exim is available from $st{ftp://ftp.cus.cam.ac.uk/pub/software/programs/exim/exim-$si{n.nn}.tar.gz} where n.nn is the version number. The distribution contains an ASCII copy of the documentation; other formats are available from $st{ftp://ftp.cus.cam.ac.uk/pub/software/programs/exim/exim-postscript-$si{n.nn}.tar.gz} $st{ftp://ftp.cus.cam.ac.uk/pub/software/programs/exim/exim-texinfo-$si{n.nn}.tar.gz} The following operating systems are currently supported: AIX, BSDI, DGUX, FreeBSD, HI-OSF (Hitachi), HP-UX, IRIX, Linux, MIPS RISCOS, NetBSD, OpenBSD, DEC OSF1 (aka Digital UNIX), SCO, SCO SVR4.2 (aka UNIX-SV), SunOS4, SunOS5, Ultrix, and Unixware. Limitations For the benefit of those reading this overview to see whether Exim is of interest to them, its limitations are listed first. Exim is written in ANSI C. This should not be much of a limitation these days. However, to help with systems that lack a true ANSI C library, Exim avoids making any use of the value returned by the `sprintf()' function, which is one of the main incompatibilities. It has its own version of `strerror()' for use with SunOS4 and any other system that lacks this function, and a macro can be defined to turn `memmove()' into `bcopy()' if necessary. Exim uses file names that are longer than 14 characters. Exim is intended for use as an Internet mailer, and therefore handles addresses in RFC 822 domain format only. It cannot handle `bang paths', though simple two-component bang paths can be converted by a straightforward rewriting configuration. Exim insists that every address it handles has a domain attached. For incoming local messages, domainless addresses are automatically qualified with a configured domain value. Configuration options specify from which remote systems unqualified addresses are acceptable. The only external transport currently implemented is an SMTP transport over a TCP/IP network (using sockets), suitable for machines on the Internet. However, a pipe transport is available, and there are facilities for writing messages to files in `batched SMTP' format; both of these can be used to send messages to some other transport mechanism such as UUCP. Batched SMTP input is also catered for. Main features Exim follows the same general approach of decentralized control that Smail 3 does. There is no central process doing overall management of mail delivery. However, unlike Smail, the independent delivery processes share data in the form of `hints', which makes delivery more efficient in some cases. The hints are kept in a number of DBM files. If any of these files are lost, the only effect is to change the pattern of delivery attempts and retries. Here is a summary of Exim's main features. More details are given in the sections which follow. Many configuration options can be given as expansion strings, and as these can include file lookups, much of Exim's operation can be made table-driven if desired. For example, it is possible to do local delivery on a machine on which the users do not have accounts. Regular expressions are available in a number of configuration parameters. Domain lists can include file lookups, making it possible to support a large number of local domains. Exim has flexible retry algorithms, applicable to mail routing as well as to delivery. Exim contains header and envelope rewriting facilities. Unqualified addresses are accepted only from specified hosts or networks. Exim can perform multiple deliveries down the same SMTP channel after deliveries to a host have been delayed. Exim can be configured to do local deliveries immediately but to leave remote deliveries until the message is picked up by a queue-runner process. This increases the likelihood of multiple messages being sent down a single SMTP connection. When copies of a message have to be delivered to more than one remote host, up to a configured maximum number of remote deliveries can be done in parallel. Exim supports optional checking of incoming return path (sender) and receiver addresses as they are received by SMTP. SMTP calls from specific hosts and networks, optionally from specific idents, can be locked out, and incoming SMTP messages from specific senders can also be locked out. It is possible to control which hosts may use the Exim host as a relay for onward transmission of mail; the control can be made to depend on the address domain. Messages on the queue can be `frozen' and `thawed' by the administrator. The maximum size of message can be specified. Exim can handle a number of independent local domains on the same machine; each domain can have its own alias files, etc. These are commonly called virtual domains. Exim stats a user's home directory before looking for a `.forward' file, in order to detect the case of a missing NFS mount. Exim contains an optional built-in mail filtering facility. This enables users to set up their own mail filtering in a straightfoward manner without the need to run an external program. There can also be a system filter file that applies to all messages. There is support for multiple user mailboxes controlled by prefixes or suffixes on the user name, either via the filter mechanism or through multiple `.forward' files. Periodic warnings are automatically sent to messages' senders when delivery is delayed -- the time between warnings is configurable. A queue run can be manually started to deliver just a particular portion of the queue, or those messages with a recipient whose address contains a given string. Exim can be configured to run as root only when needed; in particular, it need not run as root when receiving incoming messages or when sending out messages over SMTP. It always uses `setuid()' when doing local deliveries. I have tried to make the wording of delivery failure messages clearer and simpler, for the benefit of those less-experienced people who are now using email. Exim contains support for IPv6. The Exim Monitor is an optional extra; it displays information about Exim's processing in an X window, and an administrator can perform a number of control actions from the window interface. Performance Although I did not specifically set out to write a high-performance MTA, Exim does seem to be fairly efficient. The busiest site I know of is an mailing list exploder that sometimes handles over 100,000 deliveries a day on a big Linux box, the record being 177,000 deliveries (791MB in total). Up to 13,000 deliveries in an hour have been reported. Interface Like many MTAs, Exim has adopted the Sendmail interface so that it can be a straight replacement for `/usr/lib/sendmail'. All the relevant Sendmail options are implemented. There are also some additional options that are compatible with Smail 3, and some further options that are new to Exim. The runtime configuration interface is a single file which is divided into a number of sections. The entries in this file consist of keywords and values, in the style of Smail 3 configuration files. Control of messages on the queue can be done via certain privileged command line options. There is also an optional monitor program called `eximon', which displays current information in an X window and contains interfaces to the command line options. Method of operation When Exim receives a message, it writes two files in its spool directory. The first contains the envelope information, the current status of the message, and the headers, while the second contains the body of the message. The status of the message includes a complete list of recipients and a list of those that have already received the message. The header file gets updated during the course of delivery if necessary. A message remains in the spool directory until it is completely delivered to its recipients or to an error address, or until it is deleted by an administrator or by the user who originally created it. In cases when delivery cannot proceed -- for example, when a message can neither be delivered to its recipients nor returned to its sender, the message is marked `frozen' on the spool, and no more deliveries are attempted. The administrator can thaw such messages when the problem has been corrected, and can also freeze individual messages by hand if necessary. As delivery proceeds, Exim writes timestamped information about each address to a per-message log file; this includes any delivery error messages. This log is solely for the benefit of the administrator. All the information Exim itself needs for delivery is kept in the header spool file. The message log file is deleted with the spool files. If a message is delayed for more than a configured time, a warning message is sent to the sender. This is repeated whenever the same time elapses again without delivery being complete. The main delivery processing elements of Exim are called directors, routers, and transports. Code for a number of these is provided, and compile-time options specify which ones are actually included in the binary. Directors handle addresses that include one of the local domains, routers handle remote addresses, and transports do actual deliveries. When a message is to be delivered, the sequence of events is roughly as follows: If there is a system filter file, it is obeyed. This can check on the contents of the message and its headers, and cause delivery to be abandoned or directed to alternative or additional addresses. Each address is parsed and a check is made to see if it is local or not, by comparing the domain with the list of local domains, which can be wildcarded, or even held in a file if there are a large number of them. If an address is local, it is passed to each configured director in turn until one is able to handle it. If none can, the address is failed. Directors can be targeted at particular local domains, so several local domains can be processed independently of each other. A director that accepts an address may set up a local or a remote transport for it, or it may generate one or more new addresses (typically from alias or forward files). New addresses are fed back into this process from the top, but in order to avoid loops, a director will ignore any address which has an identically-named ancestor that was processed by itself. If an address is not local, it is passed to each router in turn until one is able to handle it. If none can, the address is failed. A router that accepts an address may set up a transport for it, or may pass an altered address to subsequent routers, or it may discover that the address is a local address after all. This typically happens when an partial domain name is used and (for example) the DNS lookup is configured to try to extend such names. In this case, the address is passed back to the directors. Routers normally set up remote transports for messages that are to be delivered to other machines. However, a router can pass a message to a local transport, and by this means messages for remote hosts can be routed to other transport mechanisms such as UUCP. When all the directing and routing is done, addresses that have been successfully handled are passed to their assigned transports. Local transports handle only one address at a time, but remote ones can handle more than one. Each local transport runs in a separate process under a non-privileged uid. If there were any errors, a message is returned to an appropriate address (the sender in the common case). If one or more addresses suffered a temporary failure, the message is left on the queue, to be tried again later. Otherwise the spool files and message log are deleted. Mail filtering Exim can be configured to allow users to set up filter files as an alternative to the traditional `.forward' files. A filter file can test various characteristics of a message, including the contents of the headers and the start of the body, and direct delivery to specified addresses, files, or pipes according to what it finds. The system-wide filter file uses the same control syntax. Directors The existing directors are listed below. I use the RFC 822 term local-part to mean that portion of an address that comes before the @ character. `aliasfile': This director handles local-part expansion via a traditional alias file. The name of the file is obtained by string expansion, and may therefore depend on the local-part or the domain. Generated pipe and file addresses can be (independently) locked out. The `aliasfile' director can also be used to test a list of local parts and direct any messages for them to a specific transport. In this case the data associated with the local part in the file is not used for address expansion, but is available for other purposes. For example, files containing records of the form foo: uid=1234 gid=5678 mailbox=/home_1/foo/inbox could be used on a Exim Specification - 11. Main configuration Go to the first, previous, next, last section, table of contents. 11. Main configuration The first part of the run time configuration file contains the main configuration settings. Each setting occupies one line of the file, possibly continued by a terminating backslash. If any setting is preceded by the word `hide', the -bP option displays its value to admin users only (see section 7.3). All macro definitions must be in this part of the file -- they differ from options settings by starting with an upper-case letter (see section 7.2). The available options are listed in alphabetical order below, along with their types and default values. accept_8bitmime Type: boolean Default: false This option causes Exim to send 8BITMIME in its response to an SMTP EHLO command, and to accept the BODY= parameter on MAIL commands. However, though Exim is 8-bit clean, it is not a protocol converter, and it takes no steps to do anything special with messages received by this route. Consequently, this option is turned off by default. accept_timeout Type: time Default: 0s This sets the timeout for accepting a non-SMTP message, that is, the maximum time that Exim waits when reading a message on the standard input. If the value is zero, it will wait for ever. This setting is overridden by the -or command option. The timeout for incoming SMTP messages is controlled by smtp_receive_timeout. admin_groups Type: string list Default: unset If the current group or any of the supplementary groups of the caller is in this list, the caller has admin privileges. If all your system programmers are in a specific group, for example, you can give them all Exim admin privileges by putting that group in admin_groups. However, this does not permit them to read Exim's spool files (whose group owner is the Exim gid). To permit this, you have to add individuals to the Exim group. allow_mx_to_ip Exim Specification - Concept Index Go to the first, previous, next, last section, table of contents. Concept Index Jump to: $ - * - + - / - 8 - ` - a - b - c - d - e - f - g - h - i - j - l - m - n - o - p - q - r - s - t - u - v - w - x $ $key, $key, $key, $key, $key $value, $value, $value * *@ + +allow_unknown +caseful +warn_unknown / /dev/null, /dev/null /etc/passwd 8 8-bit characters, 8-bit characters, 8-bit characters 8BITMIME ` `From', `From', `From', `From', `From' `percent hack', `percent hack', `percent hack', `percent hack' `sticky' bit, `sticky' bit a abandoning mail, abandoning mail adding drivers additional groups, additional groups, additional groups, additional groups address list case forcing address list format address: constructed address: duplicated address: qualification address: rewriting, address: rewriting address: sender address: source-routed, address: source-routed address: testing, address: testing address: verification, address: verification admin user, admin user, admin user, admin user, admin user, admin user, admin user, admin user, admin user alias errors alias file:, alias file: alias file: backslash in alias file: building, alias file: building alias file: defaults alias file: multi-domain alias file: one-time expansion alias file: ownership alias file: per-domain default alias file: repeated expansion alias files: broken alias for host aliasfile director, aliasfile director, aliasfile director, aliasfile director alternate configuration file, alternate configuration file angle brackets, excess appendfile transport, appendfile transport appending to a file architecture type arguments, logging asterisk after IP address Athena authentication:, authentication:, authentication: authentication: advertising authentication: client authentication: generic options authentication: id authentication: relaying authentication: sender, authentication: sender, authentication: sender authentication: server authentication: testing auto_thaw automatic mail processing autoreply transport, autoreply transport b background delivery backlog of connections backslash in alias file backslash in forward file bad senders (fixing) bang paths: bang paths: rewriting banner for SMTP batch delivery, batch delivery batched SMTP input, batched SMTP input, batched SMTP input batched SMTP output, batched SMTP output Bcc: header, Bcc: header, Bcc: header bcopy Berkeley DB BIN_DIRECTORY bind IP address, bind IP address bitnet black hole body of message: editing body of message: expansion variable, body of message: expansion variable body of message: line count body of message: size body of message: transporting body of message: visible size broken alias or forward files build directory building DBM files building Exim c C header files caching lookup data callback for verification, callback for verification carriage return, carriage return, carriage return, carriage return, carriage return case forcing in address lists case forcing in strings case of local parts, case of local parts cc compiler Cc: header, Cc: header cdb, cdb, cdb certificate: certificate: verifying character code checking disc space, checking disc space cipher: logging, cipher: logging cipher: verifying closed mailing lists command line options common option syntax compiler name configuration file: alternate, configuration file: alternate configuration file: common option syntax configuration file: editing configuration file: errors configuration file: format configuration file: macros configuration file: ownership configuration options configuration: changes configuration: default configuration: main configuration: pre-building configuration: retry configuration: run time, configuration: run time CONFIGURE_FILE, CONFIGURE_FILE, CONFIGURE_FILE connection backlog constructed address control of incoming mail control of relaying copy of message (unseen option) Courier CR, CR, CR, CR, CR cram_md5 authenticator creating directories current directory, current directory current_directory customizing: Received: header customizing: SMTP banner customizing: SMTP error messages cycling logs Cyrus, Cyrus d daemon, daemon, daemon, daemon daemon, process id, daemon, process id, daemon, process id, daemon, process id, daemon, process id database: lookup database: maintenance Date: header DBM DBM building DBM libraries debugging debugging output default default configuration defaults for aliases defaults for lookups deferred delivery, forcing delay_after_cutoff delay_warning_condition deliveries, maximum number of delivery to single file Delivery-date: header, Delivery-date: header, Delivery-date: header delivery: by external agent delivery: cancelling all delivery: cancelling by address delivery: deferral delivery: failure, delivery: failure, delivery: failure, delivery: failure, delivery: failure, delivery: failure, delivery: failure delivery: failure report, delivery: failure report, delivery: failure report, delivery: failure report, delivery: failure report, delivery: failure report, delivery: failure report delivery: failure report, discarding, delivery: failure report, discarding delivery: failure, long-term delivery: fake delivery: first delivery: forcing delivery: forcing deferral delivery: forcing failure, delivery: forcing failure delivery: from given sender delivery: problems with delivery: procmail delivery: sorting remote delivery: to given domain delivery_date_remove dialup directing loop, directing loop director: director: skipping, director: skipping directories, multiple directory creation, directory creation, directory creation, directory creation disc space, checking, disc space, checking discarded messages DNS DNS reverse lookup, DNS reverse lookup, DNS reverse lookup, DNS reverse lookup dnsdb domain literal, domain literal, domain literal, domain literal domain: definition domain: delivery to domain: extraction domain: in alias file domain: in alias or forward file, preserving domain: in smartuser, preserving domain: list format domain: virtual domainless addresses domainlist router dot handling, dot handling dots in local parts driver specifications drivers duplicate addresses e EACCES editing configuration files editing message body EDQUOT EHLO, EHLO, EHLO, EHLO EHLO verification encrypted comparison encrypted SMTP connections encryption, encryption ENOSPC ENOTDIR Envelope-to: header, Envelope-to: header, Envelope-to: header, Envelope-to: header, Envelope-to: header, Envelope-to: header, Envelope-to: header envelope_to_remove environment for local transports environment for pipe transport, environment for pipe transport ERRNO_QUOTA error messages: error messages: customizing, error messages: customizing, error messages: customizing error messages: discarding, error messages: discarding error reporting, error reporting, error reporting, error reporting, error reporting errors: in configuration file errors: in outgoing SMTP errors: skipping bad syntax, errors: skipping bad syntax errors_to ETRN, ETRN, ETRN, ETRN ETRN: command ETRN: serializing exceptions: alias defaults exceptions: RBL checking exceptions: rejected recipients, exceptions: rejected recipients, exceptions: rejected recipients exec failure exicyclog exigrep exim monitor exim_dbmbuild exim_dumpdb exim_fixdb EXIM_GROUP exim_lock exim_monitor/EDITME, exim_monitor/EDITME exim_tidydb EXIM_USER eximon eximstats eximstats options exinext exiqsumm exiwhat expansion of strings: expansion of strings: alias data expansion of strings: character translation expansion of strings: conditions, expansion of strings: conditions expansion of strings: file lookup expansion of strings: headers expansion of strings: numeric comparison expansion of strings: operators, expansion of strings: operators expansion of strings: query lookup expansion of strings: string comparison expansion of strings: substitution expansion of strings: testing expansion of strings: variables, expansion of strings: variables EXPN, EXPN external local delivery external transports EXTRALIBS f failing delivery, forcing failure of exec fallback, fallback fallover, fallover fifo (named pipe) file: appending file: checking existing format file: locking, file: locking, file: locking file: lookup, file: lookup, file: lookup file: MBX format file: too many open file: transport filter: system filter, filter: system filter filter: testing filter: transport filter, filter: transport filter, filter: transport filter, filter: transport filter, filter: transport filter, filter: transport filter filter: user filter, filter: user filter filtering all mail, filtering all mail first delivery fixed point format fixing bad senders forcing delivery foreground delivery format: address list format: alias file format: configuration file format: domain list format: fixed point format: forward file format: group name format: host list format: integer format: list format: message format: octal integer format: spool files format: string format: time interval format: user name forward file: forward file: backslash in forward file: ownership forward file: testing forward files: broken forward files: repeated expansion forwardfile director, forwardfile director, forwardfile director, forwardfile director forwardfile: one-time expansion freezing messages, freezing messages From: header, From: header, From: header frozen messages:, frozen messages:, frozen messages:, frozen messages: frozen messages: display frozen messages: forcing delivery, frozen messages: forcing delivery frozen messages: moving frozen messages: spool data frozen messages: thawing, frozen messages: thawing, frozen messages: thawing, frozen messages: thawing, frozen messages: thawing frozen messages: timing out g gateway gcc generic options: generic options: director, generic options: director generic options: router, generic options: router generic options: transport gid: caller gid: Exim's own, gid: Exim's own gid: forward file gid: in spool file gid: local delivery, gid: local delivery, gid: local delivery, gid: local delivery, gid: local delivery gid: of originating user gid: queryprogram gid: system filter, gid: system filter giving up on messages group name format groups, additional, groups, additional, groups, additional, groups, additional h hash function: numeric hash function: textual header files header syntax checking header verification, header verification headers: adding, headers: adding, headers: adding headers: listing headers: removing, headers: removing, headers: removing headers: transporting HELO, HELO, HELO, HELO HELO verification HOME home directory, home directory, home directory, home directory, home directory, home directory home_directory HOST host: alias host: error host: heavily loaded host: list format host: locally unique number host: lookup failures host: maximum number to try host: name, host: name host: name verification host: randomized list, host: randomized list host: serialising connections host_accept_relay host_reject host_reject_recipients hosts_treat_as_local HP-UX hub i id of message ident, ident, ident, ident ignoring faulty addresses, ignoring faulty addresses included address list, included address list, included address list incoming SMTP over TCP/IP incorporated code inetd, inetd, inetd, inetd initgroups option installing exim integer format, integer format interface address interfaces, network intermittently connected hosts IP address:, IP address: IP address: binding, IP address: binding IP address: discarding IP address: masking, IP address: masking IP address: translating IP options IP source routing, IP source routing ipliteral router iplookup router IPv6, IPv6 IPv6 addresses in lists IPv6 DNS lookup j Exim Specification - 9. String expans

Exim Internet Mailer

1.2 Testing a new filter file

`The Exim Mail Transport Agent`

Table of Contents

Exim Overview

Background

Availability

Limitations

Main features

Performance

Interface

Method of operation

Mail filtering

Directors

11. Main configuration

accept_8bitmime

accept_timeout

admin_groups

allow_mx_to_ip Exim Specification - Concept Index Go to the first, previous, next, last section, table of contents.

Concept Index

$

*

+

/

8

`

a

b

c

d

e

f

g

h

i

j Exim Specification - 9. String expans