Many strings in Exim's run time configuration are expanded before use. Some of them are expanded every time they are used; others are expanded only once.
When a string is being expanded it is copied verbatim from left to right except when a dollar or backslash character is encountered. A dollar specifies the start of a portion of the string which is interpreted and replaced as described below in section 11.5 onwards. Backslash is used as an escape character, as described in the following section.
An uninterpreted dollar can be included in an expanded string by putting a backslash in front of it. A backslash can be used to prevent any special character being treated specially in an expansion, including itself. If the string appears in quotes in the configuration file, two backslashes are required because the quotes themselves cause interpretation of backslashes when the string is read in (see section 6.12).
A portion of the string can specified as non-expandable by placing it between two occurrences of \N. This is particularly useful for protecting regular expressions, which often contain backslashes and dollar signs. For example:
deny senders = \N^\d{8}[a-z]@some\.site\.example$\N
On encountering the first \N, the expander copies subsequent characters without interpretation until it reaches the next \N or the end of the string.
A backslash followed by one of the letters “n”, “r”, or “t” in an expanded string is recognized as an escape sequence for the character newline, carriage return, or tab, respectively. A backslash followed by up to three octal digits is recognized as an octal encoding for a single character, and a backslash followed by “x” and up to two hexadecimal digits is a hexadecimal encoding.
These escape sequences are also recognized in quoted strings when they are read in. Their interpretation in expansions as well is useful for unquoted strings, and for other cases such as looked-up strings that are then expanded.
Many expansions can be tested by calling Exim with the -be option. This takes the command arguments, or lines from the standard input if there are no arguments, runs them through the string expansion code, and writes the results to the standard output. Variables based on configuration values are set up, but since no message is being processed, variables such as $local_part have no value. Nevertheless the -be option can be useful for checking out file and database lookups, and the use of expansion operators such as sg, substr and nhash.
Exim gives up its root privilege when it is called with the -be option, and instead runs under the uid and gid it was called with, to prevent users from using -be for reading files to which they do not have access.
A number of expansions that are described in the following section have alternative “true” and “false” substrings, enclosed in curly brackets. Which one is used depends on some condition that is evaluated as part of the expansion. If, instead of a “false” substring, the word “fail” is used (not in curly brackets), the entire string expansion fails in a way that can be detected by the code that requested the expansion. This is called “forced expansion failure”, and its consequences depend on the circumstances. In some cases it is no different from any other expansion failure, but in others a different action may be taken. Such variations are mentioned in the documentation of the option that is being expanded.
The following items are recognized in expanded strings. White space may be used between sub-items that are keywords or substrings enclosed in braces inside an outer set of braces, to improve readability. Warning: Within braces, white space is significant.
$<variable name> or ${<variable name>}
Substitute the contents of the named variable, for example
$local_part
${domain}
The second form can be used to separate the name from subsequent alphanumeric characters. This form (using curly brackets) is available only for variables; it does not apply to message headers. The names of the variables are given in section 11.9 below. If the name of a non-existent variable is given, the expansion fails.
The string is first itself expanded, and then the operation specified by <op> is applied to it. For example,
${lc:$local_part}
The string starts with the first character after the colon, which may be leading white space. A list of operators is given in section 11.6 below. The operator notation is used for simple expansion items that have just one argument, because it reduces the number of braces and therefore makes the string easier to understand.
${extract{<key>}{<string1>}{<string2>}{<string3>}}
The key and <string1> are first expanded separately. Leading and trailing whitespace is removed from the key (but not from any of the strings). The key must not consist entirely of digits. The expanded <string1> must be of the form:
<key1> = <value1> <key2> = <value2> ...
where the equals signs and spaces (but not both) are optional. If any of the values contain white space, they must be enclosed in double quotes, and any values that are enclosed in double quotes are subject to escape processing as described in section 6.12. The expanded <string1> is searched for the value that corresponds to the key. The search is case-insensitive. If the key is found, <string2> is expanded, and replaces the whole item; otherwise <string3> is used. During the expansion of <string2> the variable $value contains the value that has been extracted. Afterwards, it is restored to any previous value it might have had.
If {<string3>} is omitted, the item is replaced by an empty string if the key is not found. If {<string2>} is also omitted, the value that was extracted is used. Thus, for example, these two expansions are identical, and yield “2001”:
${extract{gid}{uid=1984 gid=2001}}
${extract{gid}{uid=1984 gid=2001}{$value}}
Instead of {<string3>} the word “fail” (not in curly brackets) can appear, for example:
${extract{Z}{A=... B=...}{$value} fail }
This forces an expansion failure (see section 11.4); {<string2>} must be present for “fail” to be recognized.
${extract{<number>}{<separators>}{<string1>}{<string2>}{<string3>}}
The <number> argument must consist entirely of decimal digits, apart from leading and trailing whitespace, which is ignored. This is what distinguishes this form of extract from the previous kind. It behaves in the same way, except that, instead of extracting a named field, it extracts from <string1> the field whose number is given as the first argument. You can use $value in <string2> or fail instead of <string3> as before.
The fields in the string are separated by any one of the characters in the separator string. These may include space or tab characters. The first field is numbered one. If the number is negative, the fields are counted from the end of the string, with the rightmost one numbered -1. If the number given is zero, the entire string is returned. If the modulus of the number is greater than the number of fields in the string, the result is the expansion of <string3>, or the empty string if <string3> is not provided. For example:
${extract{2}{:}{x:42:99:& Mailer::/bin/bash}}
yields “42”, and
${extract{-4}{:}{x:42:99:& Mailer::/bin/bash}}
yields “99”. Two successive separators mean that the field between them is empty (for example, the fifth field above).
${hash{<string1>}{<string2>}{<string3>}}
This is a textual hashing function, and was the first to be implemented in early versions of Exim. In current releases, there are other hashing functions (numeric, MD5, and SHA-1), which are described below.
The first two strings, after expansion, must be numbers. Call them <m> and <n>. If you are using fixed values for these numbers, that is, if <string1> and <string2> do not change when they are expanded, you can use the simpler operator notation that avoids some of the braces:
${hash_<n>_<m>:<string>}
The second number is optional (in both notations).
If <n> is greater than or equal to the length of the string, the expansion item returns the string. Otherwise it computes a new string of length <n> by applying a hashing function to the string. The new string consists of characters taken from the first <m> characters of the string
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQWRSTUVWXYZ0123456789
If <m> is not present the value 26 is used, so that only lower case letters appear. For example:
${hash{3}{monty}} yields jmg
${hash{5}{monty}} yields monty
${hash{4}{62}{monty python}} yields fbWx
$header_<header name>: or $h_<header name>:
$bheader_<header name>: or $bh_<header name>:
$rheader_<header name>: or $rh_<header name>:
Substitute the contents of the named message header line, for example
$header_reply-to:
The newline that terminates a header line is not included in the expansion, but internal newlines (caused by splitting the header line over several physical lines) may be present.
The difference between rheader, bheader, and header is in the way the data in the header line is interpreted.
rheader gives the original “raw” content of the header line, with no processing at all, and without the removal of leading and trailing whitespace.
bheader removes leading and trailing whitespace, and then decodes base64 or quoted-printable MIME “words” within the header text, but does no character set translation. If decoding of what looks superficially like a MIME “word” fails, the raw string is returned. If decoding produces a binary zero character, it is replaced by a question mark – this is what Exim does for binary zeros that are actually received in header lines.
header tries to translate the string as decoded by bheader to a standard character set. This is an attempt to produce the same string as would be displayed on a user's MUA. If translation fails, the bheader string is returned. Translation is attempted only on operating systems that support the iconv() function. This is indicated by the compile-time macro HAVE_ICONV in a system Makefile or in Local/Makefile.
In a filter file, the target character set for header can be specified by a command of the following form:
headers charset "UTF-8"
This command affects all references to $h_ (or $header_) expansions in subsequently obeyed filter commands. In the absence of this command, the target character set in a filter is taken from the setting of the headers_charset option in the runtime configuration. The value of this option defaults to the value of HEADERS_CHARSET in Local/Makefile. The ultimate default is ISO-8859-1.
Header names follow the syntax of RFC 2822, which states that they may contain any printing characters except space and colon. Consequently, curly brackets do not terminate header names, and should not be used to enclose them as if they were variables. Attempting to do so causes a syntax error.
Only header lines that are common to all copies of a message are visible to this mechanism. These are the original header lines that are received with the message, and any that are added by an ACL warn statement or by a system filter. Header lines that are added to a particular copy of a message by a router or transport are not accessible.
For incoming SMTP messages, no header lines are visible in ACLs that are obeyed before the DATA ACL, because the header structure is not set up until the message is received. Header lines that are added by warn statements in a RCPT ACL (for example) are saved until the message's incoming header lines are available, at which point they are added. When a DATA ACL is running, however, header lines added by earlier ACLs are visible.
Upper case and lower case letters are synonymous in header names. If the following character is white space, the terminating colon may be omitted, but this is not recommended, because you may then forget it when it is needed. When white space terminates the header name, it is included in the expanded string. If the message does not contain the given header, the expansion item is replaced by an empty string. (See the def condition in section 11.7 for a means of testing for the existence of a header.)
If there is more than one header with the same name, they are all concatenated to form the substitution string, up to a maximum length of 64K. A newline character is inserted between each line. For the header expansion, for those headers that contain lists of addresses, a comma is also inserted at the junctions between lines. This does not happen for the rheader expansion.
${hmac{<hashname>}{<secret>}{<string>}}
This function uses cryptographic hashing (either MD5 or SHA-1) to convert a shared secret and some text into a message authentication code, as specified in RFC 2104. This differs from ${md5:secret_text...} or ${sha1:secret_text...} in that the hmac step adds a signature to the cryptographic hash, allowing for authentication that is not possible with MD5 or SHA-1 alone. The hash name must expand to either md5 or sha1 at present. For example:
${hmac{md5}{somesecret}{$primary_hostname $tod_log}}
For the hostname mail.example.com and time 2002-10-17 11:30:59, this produces:
dd97e3ba5d1a61b5006108f8c8252953
As an example of how this might be used, you might put in the main part of an Exim configuration:
SPAMSCAN_SECRET=cohgheeLei2thahw
In a router or a transport you could then have:
headers_add = \
X-Spam-Scanned: ${primary_hostname} ${message_id} \
${hmac{md5}{SPAMSCAN_SECRET}\
{${primary_hostname},${message_id},$h_message-id:}}
Then given a message, you can check where it was scanned by looking at the X-Spam-Scanned: header line. If you know the secret, you can check that this header line is authentic by recomputing the authentication code from the host name, message ID and the Message-id: header line. This can be done using Exim's -be option, or by other means, for example by using the hmac_md5_hex() function in Perl.
${if <condition> {<string1>}{<string2>}}
If <condition> is true, <string1> is expanded and replaces the whole item; otherwise <string2> is used. The available conditions are described in section 11.7 below. For example:
${if eq {$local_part}{postmaster} {yes}{no} }
The second string need not be present; if it is not and the condition is not true, the item is replaced with nothing. Alternatively, the word “fail” may be present instead of the second string (without any curly brackets). In this case, the expansion is forced to fail if the condition is not true (see section 11.4).
If both strings are omitted, the result is the string true if the condition
is true, and the empty string if the condition is false. This makes it less
cumbersome to write custom ACL and router conditions. For example, instead of
condition = ${if >{$acl_m4}{3}{true}{false}}
you can use
condition = ${if >{$acl_m4}{3}}
${length{<string1>}{<string2>}}
The length item is used to extract the initial portion of a string. Both strings are expanded, and the first one must yield a number, <n>, say. If you are using a fixed value for the number, that is, if <string1> does not change when expanded, you can use the simpler operator notation that avoids some of the braces:
${length_<n>:<string>}
The result of this item is either the first <n> characters or the whole of <string2>, whichever is the shorter. Do not confuse length with strlen, which gives the length of a string.
${lookup{<key>} <search type> {<file>} {<string1>} {<string2>}}
${lookup <search type> {<query>} {<string1>} {<string2>}}
These items specify data lookups in files and databases, as discussed in chapter 9. The first form is used for single-key lookups, and the second is used for query-style lookups. The <key>, <file>, and <query> strings are expanded before use.
If there is any white space in a lookup item which is part of a filter command, a retry or rewrite rule, a routing rule for the manualroute router, or any other place where white space is significant, the lookup item must be enclosed in double quotes. The use of data lookups in users' filter files may be locked out by the system administrator.
If the lookup succeeds, <string1> is expanded and replaces the entire item. During its expansion, the variable $value contains the data returned by the lookup. Afterwards it reverts to the value it had previously (at the outer level it is empty). If the lookup fails, <string2> is expanded and replaces the entire item. If {<string2>} is omitted, the replacement is the empty string on failure. If <string2> is provided, it can itself be a nested lookup, thus providing a mechanism for looking up a default value when the original lookup fails.
If a nested lookup is used as part of <string1>, $value contains the data for the outer lookup while the parameters of the second lookup are expanded, and also while <string2> of the second lookup is expanded, should the second lookup fail.
Instead of {<string2>} the word “fail” can appear, and in this case, if the lookup fails, the entire expansion is forced to fail (see section 11.4). If both {<string1>} and {<string2>} are omitted, the result is the looked up value in the case of a successful lookup, and nothing in the case of failure.
For single-key lookups, the string “partial” is permitted to precede the search type in order to do partial matching, and * or *@ may follow a search type to request default lookups if the key does not match (see sections 9.5 and 9.6 for details).
If a partial search is used, the variables $1 and $2 contain the wild and non-wild parts of the key during the expansion of the replacement text. They return to their previous values at the end of the lookup item.
This example looks up the postmaster alias in the conventional alias file:
${lookup {postmaster} lsearch {/etc/aliases} {$value}}
This example uses NIS+ to look up the full name of the user corresponding to the local part of an address, forcing the expansion to fail if it is not found:
${lookup nisplus {[name=$local_part],passwd.org_dir:gcos} \
{$value}fail}
${nhash{<string1>}{<string2>}{<string3>}}
The three strings are expanded; the first two must yield numbers. Call them <n> and <m>. If you are using fixed values for these numbers, that is, if <string1> and <string2> do not change when they are expanded, you can use the simpler operator notation that avoids some of the braces:
${nhash_<n>_<m>:<string>}
The second number is optional (in both notations). If there is only one number, the result is a number in the range 0--<n>-1. Otherwise, the string is processed by a div/mod hash function that returns two numbers, separated by a slash, in the ranges 0 to <n>-1 and 0 to <m>-1, respectively. For example,
${nhash{8}{64}{supercalifragilisticexpialidocious}}
returns the string “6/33”.
${perl{<subroutine>}{<arg>}{<arg>}...}
This item is available only if Exim has been built to include an embedded Perl interpreter. The subroutine name and the arguments are first separately expanded, and then the Perl subroutine is called with those arguments. No additional arguments need be given; the maximum number permitted, including the name of the subroutine, is nine.
The return value of the subroutine is inserted into the expanded string, unless the return value is undef. In that case, the expansion fails in the same way as an explicit “fail” on a lookup item. The return value is a scalar. Whatever you return is evaluated in a scalar context. For example, if you return the name of a Perl vector, the return value is the size of the vector, not its contents.
If the subroutine exits by calling Perl's die function, the expansion fails with the error message that was passed to die. More details of the embedded Perl facility are given in chapter 12.
The redirect router has an option called forbid_filter_perl which locks out the use of this expansion item in filter files.
${readfile{<file name>}{<eol string>}}
The file name and end-of-line string are first expanded separately. The file is then read, and its contents replace the entire item. All newline characters in the file are replaced by the end-of-line string if it is present. Otherwise, newlines are left in the string. String expansion is not applied to the contents of the file. If you want this, you must wrap the item in an expand operator. If the file cannot be read, the string expansion fails.
The redirect router has an option called forbid_filter_readfile which locks out the use of this expansion item in filter files.
${readsocket{<name>}{<request>}{<timeout>}{<eol string>}{<fail string>}}
This item inserts data that is read from a Unix domain socket into the expanded string. The minimal way of using it uses just two arguments:
${readsocket{/socket/name}{request string}}
Exim connects to the socket, writes the request string (unless it is an empty string) and reads from the socket until an end-of-file is read. A timeout of 5 seconds is applied. Additional, optional arguments extend what can be done. Firstly, you can vary the timeout. For example:
${readsocket{/socket/name}{request-string}{3s}}
A fourth argument allows you to change any newlines that are in the data that is read, in the same way as for readfile (see above). This example turns them into spaces:
${readsocket{/socket/name}{request-string}{3s}{ }}
As with all expansions, the substrings are expanded before the processing happens. Errors in these sub-expansions cause the expansion to fail. In addition, the following errors can occur:
Failure to create a socket file descriptor;
Failure to connect the socket;
Failure to write the request-string;
Timeout on reading from the socket.
By default, any of these errors causes the expansion to fail. However, if you supply a fifth substring, it is expanded and used when any of the above errors occurs. For example:
${readsocket{/socket/name}{request-string}{3s}{\n}\
{socket failure}}
You can test for the existence of the socket by wrapping this expansion in ${if exists, but there is a race condition between that test and the actual opening of the socket, so it is safer to use the fifth argument if you want to be absolutely sure of avoiding an expansion error for a non-existent socket.
The redirect router has an option called forbid_filter_readsocket which locks out the use of this expansion item in filter files.
$rheader_<header name>: or $rh_<header name>:
This item inserts “raw” header lines. It is described with the header expansion item above.
${run{<command> <args>}{<string1>}{<string2>}}
The command and its arguments are first expanded separately, and then the command is run in a separate process, but under the same uid and gid. As in other command executions from Exim, a shell is not used by default. If you want a shell, you must explicitly code it. If the command succeeds (gives a zero return code) <string1> is expanded and replaces the entire item; during this expansion, the standard output from the command is in the variable $value. If the command fails, <string2>, if present, is expanded. If it is absent, the result is empty. Alternatively, <string2> can be the word “fail” (not in braces) to force expansion failure if the command does not succeed. If both strings are omitted, the result is the standard output on success, and nothing on failure.
The return code from the command is put in the variable $runrc, and this remains set afterwards, so in a filter file you can do things like this:
if "${run{x y z}{}}$runrc" is 1 then ...
elif $runrc is 2 then ...
...
endif
If execution of the command fails (for example, the command does not exist), the return code is 127 – the same code that shells use for non-existent commands.
Warning: In a router or transport, you cannot assume the order in which option values are expanded, except for those pre-conditions whose order of testing is documented. Therefore, you cannot reliably expect to set $runrc by the expansion of one option, and use it in another.
The redirect router has an option called forbid_filter_run which locks out the use of this expansion item in filter files.
${sg{<subject>}{<regex>}{<replacement>}}
This item works like Perl's substitution operator (s) with the global (/g) option; hence its name. However, unlike the Perl equivalent, Exim does not modify the subject string; instead it returns the modified string for insertion into the overall expansion. The item takes three arguments: the subject string, a regular expression, and a substitution string. For example
${sg{abcdefabcdef}{abc}{xyz}}
yields “xyzdefxyzdef”. Because all three arguments are expanded before use, if any $ or \ characters are required in the regular expression or in the substitution string, they have to be escaped. For example
${sg{abcdef}{^(...)(...)\$}{\$2\$1}}
yields “defabc”, and
${sg{1=A 4=D 3=C}{\N(\d+)=\N}{K\$1=}}
yields “K1=A K4=D K3=C”. Note the use of \N to protect the contents of the regular expression from string expansion.
${substr{<string1>}{<string2>}{<string3>}}
The three strings are expanded; the first two must yield numbers. Call them <n> and <m>. If you are using fixed values for these numbers, that is, if <string1> and <string2> do not change when they are expanded, you can use the simpler operator notation that avoids some of the braces:
${substr_<n>_<m>:<string>}
The second number is optional (in both notations). If it is absent in the simpler format, the preceding underscore must also be omitted.
The substr item can be used to extract more general substrings than length. The first number, <n>, is a starting offset, and <m> is the length required. For example
${substr{3}{2}{$local_part}}
If the starting offset is greater than the string length the result is the null string; if the length plus starting offset is greater than the string length, the result is the right-hand part of the string, starting from the given offset. The first character in the string has offset zero.
The substr expansion item can take negative offset values to count from the right-hand end of its operand. The last character is offset -1, the second-last is offset -2, and so on. Thus, for example,
${substr{-5}{2}{1234567}}
yields “34”. If the absolute value of a negative offset is greater than the length of the string, the substring starts at the beginning of the string, and the length is reduced by the amount of overshoot. Thus, for example,
${substr{-5}{2}{12}}
yields an empty string, but
${substr{-3}{2}{12}}
yields “1”.
When the second number is omitted from substr, the remainder of the string is taken if the offset is positive. If it is negative, all characters in the string preceding the offset point are taken. For example, an offset of -1 and no length, as in these semantically identical examples:
${substr_-1:abcde}
${substr{-1}{abcde}}
yields all but the last character of the string, that is, “abcd”.
${tr{<subject>}{<characters>}{<replacements>}}
This item does single-character translation on its subject string. The second argument is a list of characters to be translated in the subject string. Each matching character is replaced by the corresponding character from the replacement list. For example
${tr{abcdea}{ac}{13}}
yields “1b3de1”. If there are duplicates in the second character string, the last occurrence is used. If the third string is shorter than the second, its last character is replicated. However, if it is empty, no translation takes place.
For expansion items that perform transformations on a single argument string, the “operator” notation is used because it is simpler and uses fewer braces. The substring is first expanded before the operation is applied to it. The following operations can be performed:
The string is interpreted as an RFC 2822 address, as it might appear in a header line, and the effective address is extracted from it. If the string does not parse successfully, the result is empty.
The string must consist entirely of decimal digits. The number is converted to base 62 (sic) and output as a string of six characters, including leading zeros. Note: Just to be absolutely clear: this is not base64 encoding.
The string must consist entirely of base-62 digits. The number is converted to decimal and output as a string.
The string is interpreted as an RFC 2822 address and the domain is extracted from it. If the string does not parse successfully, the result is empty.
If the string contains any non-printing characters, they are converted to escape sequences starting with a backslash. Whether characters with the most significant bit set (so-called “8-bit characters”) count as printing or not is controlled by the print_topbitchars option.
${eval:<string>}
${eval10:<string>}
These items supports simple arithmetic in expansion strings. The string (after expansion) must be a conventional arithmetic expression, but it is limited to the four basic operators (plus, minus, times, divide) and parentheses. All operations are carried out using integer arithmetic. Plus and minus have a lower priority than times and divide; operators with the same priority are evaluated from left to right.
For eval, numbers may be decimal, octal (starting with “0”) or hexadecimal (starting with “0x”). For eval10, all numbers are taken as decimal, even if they start with a leading zero. This can be useful when processing numbers extract