Regular expressions are available i
Exim 4.00 Specification chapter 11
Previous
Next
Contents
(Exim 4.00 Specification)
Many strings that are given in Exim's run time configuration are expanded
before use. Some of them are expanded every time they are used; others are
expanded only once.
When a string is being expanded it is copied verbatim from left to right except
when a dollar or backslash character is encountered. A dollar specifies the
start of a portion of the string which is interpreted and replaced as described
below in section 11.4 onwards.
An uninterpreted dollar can be included in an expanded string by putting a
backslash in front of it. A backslash can be used to prevent any special
character being treated specially in an expansion, including itself. If the
string appears in quotes in the configuration file, two backslashes are
required because the quotes themselves cause interpretation of backslashes when
the string is read in.
An entire portion of the string can specified as non-expandable by placing it
between two occurrences of \N. This is particularly useful for protecting
regular expressions, which often contain backslashes and dollar signs. For
example:
deny senders = \N^\d{8}[a-z]@some\.site\.example$\N
On encountering the first \N, the expander copies subsequent characters
without interpretation until it reaches the next \N or the end of the
string.
A backslash followed by one of the letters ``n'', ``r'', or ``t'' in an expanded
string is recognized as an escape sequence for the character newline, carriage
return, or tab, respectively. A backslash followed by up to three octal digits
is recognized as an octal encoding for a single character, and a backslash
followed by ``x'' and up to two hexadecimal digits is a hexadecimal encoding.
These escape sequences are also recognized in quoted strings when they are read
in. Their interpretation in expansions as well is useful for unquoted strings,
and for other cases such as looked-up strings that are then expanded.
Many expansions can be tested by calling Exim with the -be option. This takes
the command arguments, or lines from the standard input if there are no
arguments, runs them through the string expansion code, and writes the results
to the standard output. Variables based on configuration values are set up, but
since no message is being processed, variables such as $local_part have no
value. Nevertheless the -be option can be useful for checking out file and
database lookups, and the use of expansion operators such as substr and
hash.
Exim gives up its root privilege when it is called with the -be option, and
instead runs under the uid and gid it was called with, to prevent users from
using -be for reading files to which they normally do not have access.
The following items are recognized in expanded strings. White space may be used
between sub-items that are keywords or substrings enclosed in braces inside an
outer set of braces, to improve readability. Within braces, however, white
space is significant.
$<variable name> or ${<variable name>}
Substitute the contents of the named variable, for example
$local_part
${domain}
The second form can be used to separate the name from subsequent alphanumeric
characters. This form (using curly brackets) is available only for variables;
it does not apply to message headers. The names of the variables are given
in section 11.8 below. If the name of a non-existent variable is given,
the expansion fails.
${<op>:<string>}
The string is first itself expanded, and then the operation specified by <op>
is applied to it. For example,
${lc:$local_part}
A list of operators is given in section 11.5 below. The string starts
with the first character after the colon, which may be leading white space.
${extract{<key>}{<string1>}{<string2>}{<string3>}}
The key and <string1> are first expanded separately. The key must not
consist entirely of digits. The expanded <string1> must be of the form:
<key1> = <value1> <key2> = <value2> ...
where the equals signs and spaces (but not both) are optional. If any of the
values contain white space, they must be enclosed in double quotes, and any
values that are enclosed in double quotes are subject to escape processing as
described in section 6.10. The expanded <string1> is searched for
the value that corresponds to the key. The search is case-insensitive. If the
key is found, <string2> is expanded, and replaces the whole item; otherwise
<string3> is used. During the expansion of <string2> the variable $value
contains the value that has been extracted. Afterwards, it is restored to any
previous value it might have had.
If {<string3>} is omitted, the item is replaced by an empty string if the
key is not found. If {<string2>} is also omitted, the value that was
extracted is used. Thus, for example, these two expansions are identical, and
yield ``2001'':
${extract{gid}{uid=1984 gid=2001}}
${extract{gid}{uid=1984 gid=2001}{$value}}
Instead of {<string3>} the word ``fail'' (not in curly brackets) can appear,
for example:
${extract{Z}{A=... B=...}{$value} fail }
{<string2>} must be present for ``fail'' to be recognized. When this syntax
is used, if the extraction fails, the entire string expansion fails in a way
that can be detected by the code in Exim which requested the expansion. This is
called ``forced expansion failure'', and its consequences depend on the
circumstances. In some cases it is no different from any other expansion
failure, but in others a different action may be taken. Such variations are
mentioned in the documentation of the option which is expanded.
${extract{<number>}{<separators>}{<string1>}{<string2>}{<string3>}}
The <number> argument must consist entirely of decimal digits. This is what
distinguishes this form of extract from the previous kind. It behaves in the
same way, except that, instead of extracting a named field, it extracts from
<string1> the field whose number is given as the first argument. The first
field is numbered one. If the number is negative, the fields are counted from
the end of the string, with the rightmost one numbered -1. If the number given
is zero, the entire string is returned. If the modulus of the number is greater
than the number of fields in the string, the result is the expansion of
<string3>, or the empty string if <string3> is not provided. The fields in
the string are separated by any one of the characters in the separator string.
For example:
${extract{2}{:}{x:42:99:& Mailer::/bin/bash}}
yields ``42'', and
${extract{-4}{:}{x:42:99:& Mailer::/bin/bash}}
yields ``99''. Two successive separators mean that the field between them is
empty (for example, the fifth field above).
$header_<header name>: or $h_<header name>:
Substitute the contents of the named message header line, for example
$header_reply-to:
The terminating newline is not included in the expansion, but internal newlines
(caused by splitting the header line over several physical lines) may be
present.
The header names follow the syntax of RFC 2822, which states that they may
contain any printing characters except space and colon. Consequently, curly
brackets do not terminate header names, and should not be used to enclose
them as if they were variables. Attempting to do so causes a syntax error.
Only header lines that are common to all copies of a message are visible to
this mechanism. These are the original header lines that are received with the
message, and any that are added by a system filter. Header lines that are added
to a particular copy of a message by a router or transport are not accessible.
Upper case and lower case letters are synonymous in header names. If the
following character is white space, the terminating colon may be omitted, but
this is not recommended, because you may then forget it when it is needed. When
white space terminates the header name, it is included in the expanded string.
If the message does not contain the given header, the expansion item is
replaced by an empty string. (See the def condition in section 11.6
for a means of testing for the existence of a header.) If there is more than
one header with the same name, they are all concatenated to form the
substitution string, with a newline character between each of them. However, if
the length of this string exceeds 64K, any further headers of the same name are
ignored.
${if <condition> {<string1>}{<string2>}}
If <condition> is true, <string1> is expanded and replaces the whole item;
otherwise <string2> is used. For example,
${if eq {$local_part}{postmaster} {yes}{no} }
The second string need not be present; if it is not and the condition is not
true, the item is replaced with nothing. Alternatively, the word ``fail'' may be
present instead of the second string (without any curly brackets). In this
case, the expansion is forced to fail if the condition is not true. The
available conditions are described in section 11.6 below.
${lookup{<key>} <search type> {<file>} {<string1>} {<string2>}}
${lookup <search type> {<query>} {<string1>} {<string2>}}
These items specify data lookups in files and databases, as discussed in
chapter 9. The first form is used for single-key lookups, and the
second is used for query-style lookups. The <key>, <file>, and <query>
strings are expanded before use.
If there is any white space in a lookup item which is part of a filter command,
a retry or rewrite rule, a routing rule for the manualroute router, or any
other place where white space is significant, the lookup item must be enclosed
in double quotes. The use of data lookups in users' filter files may be locked
out by the system administrator.
If the lookup succeeds, <string1> is expanded and replaces the entire item.
During its expansion, the variable $value contains the data returned by the
lookup. Afterwards it reverts to the value it had previously (at the outer
level it is empty). If the lookup fails, <string2> is expanded and replaces
the entire item. If {<string2>} is omitted, the replacement is null on
failure. Alternatively, <string2> can itself be a nested lookup, thus
providing a mechanism for looking up a default value when the original lookup
fails.
If a nested lookup is used as part of <string1>, $value contains the data
for the outer lookup while the parameters of the second lookup are expanded,
and also while <string2> of the second lookup is expanded, should the second
lookup fail.
Instead of {<string2>} the word ``fail'' can appear, and in this case, if the
lookup fails, the entire expansion is forced to fail. If both {<string1>}
and {<string2>} are omitted, the result is the looked up value in the case
of a successful lookup, and nothing in the case of failure.
For single-key lookups, the string ``partial-'' is permitted to precede the
search type in order to do partial matching, and * or *@ may follow a
search type to request default lookups if the key does not match (see sections
9.4 and 9.5).
If a partial search is used, the variables $1 and $2 contain the wild
and non-wild parts of the key during the expansion of the replacement text.
They return to their previous values at the end of the lookup item.
This example looks up the postmaster alias in the conventional alias file:
${lookup {postmaster} lsearch {/etc/aliases} {$value}}
This example uses NIS+ to look up the full name of the user corresponding to
the local part of an address, forcing the expansion to fail if it is not found:
${lookup nisplus {[name=$local_part],passwd.org_dir:gcos} \
{$value}fail}
${perl{<subroutine>}{<arg>}{<arg>}...}
This item is available only if Exim has been built to include an embedded Perl
interpreter. The subroutine name and the arguments are first separately
expanded, and then the Perl subroutine is called with those arguments. No
arguments need be given; the maximum number permitted is eight.
The return value of the subroutine is inserted into the expanded string, unless
the return value is undef. In that case, the expansion fails in the same way
as an explicit ``fail'' on a lookup item. If the subroutine exits by calling
Perl's die function, the expansion fails with the error message that was
passed to die. More details of the embedded Perl facility are given in
chapter 12.
The redirect router has an option called forbid_filter_perl which locks
out the use of this expansion item in filter files.
${readfile{<file name>}{<eol string>}}
The file name and end-of-line string are first expanded separately. The file is
then read, and its contents replace the entire item. All newline characters in
the file are replaced by the end-of-line string if it is present. Otherwise,
newlines are left in the string.
The redirect router has an option called forbid_filter_readfile which
locks out the use of this expansion item in filter files.
${run{<command> <args>}{<string1>}{<string2>}}
The command and its arguments are first expanded separately, and then the
command is run in a separate process, but under the same uid and gid. As in
other command executions from Exim, a shell is not used by default. If you want
a shell, you must explicitly code it.
If the command succeeds (gives a zero return code) <string1> is expanded and
replaces the entire item; during this expansion, the standard output from the
command is in the variable $value. If the command fails, <string2>, if
present, is expanded. If it is absent, the result is empty. Alternatively,
<string2> can be the word ``fail'' (not in braces) to force expansion failure
if the command does not succeed. If both strings are omitted, the result is the
standard output on success, and nothing on failure.
The return code from the command is put in the variable $runrc, and this
remains set afterwards, so in a filter file you can do things like this:
if "${run{x y z}{}}$runrc" is 1 then ...
elif $runrc is 2 then ...
...
endif
The redirect router has an option called forbid_filter_run which locks
out the use of this expansion item in filter files.
${sg{<subject>}{<regex>}{<replacement>}}
This item works like Perl's substitution operator (s) with the global (/g)
option; hence its name. It takes three arguments: the subject string, a regular
expression, and a substitution string. For example
${sg{abcdefabcdef}{abc}{xyz}}
yields ``xyzdefxyzdef''. Because all three arguments are expanded before use, if
any $ or \ characters are required in the regular expression or in the
substitution string, they have to be escaped. For example
${sg{abcdef}{^(...)(...)\$}{\$2\$1}}
yields ``defabc'', and
${sg{1=A 4=D 3=C}{\N(\d+)=\N}{K\$1=}}
yields ``K1=A K4=D K3=C''.
${tr{<subject>}{<characters>}{<replacements>}}
This item does single-character translation on its subject string. The second
argument is a list of characters to be translated in the subject string. Each
matching character is replaced by the corresponding character from the
replacement list. For example
${tr{abcdea}{ac}{13}}
yields ``1b3de1''. If there are duplicates in the second character string, the
last occurrence is used. If the third string is shorter than the second, its
last character is replicated. However, if it is empty, no translation takes
place.
The following operations can be performed on portions of an expanded string.
The substring is first expanded before the operation is applied to it.
${address:<string>}
The string is interpreted as an RFC 2822 address, as it might appear in a
header line, and the effective address is extracted from it. If the string does
not parse successfully, the result is empty.
${base62:<digits>}
The string must consist entirely of decimal digits. The number is converted to
base 62 and output as a string of six characters, including leading zeros.
${domain:<string>}
The string is interpreted as an RFC 2822 address and the domain is extracted
from it. If the string does not parse successfully, the result is empty.
${escape:<string>}
If the string contains any non-printing characters, they are converted to
escape sequences starting with a backslash. Whether characters with the most
significant bit set (so-called ``8-bit characters'') count as printing or not is
controlled by the print_topbitchars option.
${expand:<string>}
The expand operator causes a string to be expanded for a second time. For
example,
${expand:${lookup{$domain}dbm{/some/file}{$value}}}
first looks up a string in a file while expanding the operand for expand, and
then re-expands what it has found.
${hash_<n>_<m>:<string>}
This is Exim's original hashing function. There are also numeric and MD5
hashing functions (see below). The two items <n> and <m> are numbers. If
<n> is greater than or equal to the length of the string, the operator
returns the string. Otherwise it computes a new string of length <n> by
applying a hashing function to the string. The new string consists of
characters taken from the first <m> characters of the string
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQWRSTUVWXYZ0123456789
and if <m> is not present the value 26 is used, so that only lower case
letters appear. These examples:
${hash_3:monty}
${hash_5:monty}
${hash_4_62:monty python}
yield
jmg
monty
fbWx
respectively. The abbreviation h can be used instead of hash.
${lc:<string>}
This forces the letters in the string into lower-case, for example:
${lc:$local_part}
${length_<number>:<string>}
The length operator can be used to extract the initial portion of a string.
It is followed by an underscore and the number of characters required. For
example
${length_50:$message_body}
The result of this operator is either the first <number> characters or the
whole string, whichever is the shorter. The abbreviation l can be used
instead of length.
${local_part:<string>}
The string is interpreted as an RFC 2822 address and the local part is
extracted from it. If the string does not parse successfully, the result is
empty.
${mask:<IP address>/<bit count>}
If the form of the string to be operated on is not an IP address followed by a
slash and an integer (that is, a network address in CIDR notation), the
expansion fails. Otherwise, this operator converts the IP address to binary,
masks off the least significant bits according to the bit count, and converts
the result back to text, with mask appended. For example,
${mask:10.111.131.206/28}
returns the string ``10.111.131.192/28''. Since this operation is expected to be
mostly used for looking up masked addresses in files, the result for an IPv6
address uses full stops (periods) to separate components instead of colons,
because colon terminates a key string in lsearch files. So, for example,
${mask:3ffe:ffff:836f:0a00:000a:0800:200a:c031/99}
returns the string
3ffe.ffff.836f.0a00.000a.0800.2000.0000/99
Letters in IPv6 addresses are always output in lower case.
${md5:<string>}
The md5 operator computes the MD5 hash value of the string, and returns it as
a 32-digit hexadecimal number.
${nhash_<n>:<string>}
The string is processed by a hash function which returns a numeric value in the
range 0 to <n>-1.
${nhash_<n>_<m>:<string>}
The string is processed by a div/mod hash function which returns two numbers,
separated by a slash, in the ranges 0 to <n>-1 and 0 to <m>-1,
respectively. For example,
${nhash_8_64:supercalifragilisticexpialidocious}
returns the string ``6/33''.
${quote:<string>}
The quote operator puts its argument into double quotes if it contains
anything other than letters, digits, underscores, full stops (periods), and
hyphens. Any occurrences of double quotes and backslashes are escaped with a
backslash. For example,
${quote:ab"*"cd}
becomes
"ab\"*\"cd"
The place where this is useful is when the argument is a substitution from a
variable or a message header.
${quote_<lookup-type>:<string>}
This operator applies lookup-specific quoting rules to the string. Each
query-style lookup type has its own quoting rules which are described with
the lookups in chapter 9. For example,
${quote_ldap:two + two}
returns ``two%20%5C+%20two''. For single-key lookup types, no quoting is
necessary and this operator yields an unchanged string.
${rxquote:<string>}
The rxquote operator inserts a backslash before any non-alphanumeric
characters in its argument. This is useful when substituting the values of
variables or headers inside regular expressions.
${substr_<start>_<length>:<string>}
The substr operator can be used to extract more general substrings than
length. It is followed by an underscore and the starting offset, then a
second underscore and the length required. For example
${substr_3_2:$local_part}
If the starting offset is greater than the string length the result is the null
string; if the length plus starting offset is greater than the string length,
the result is the right-hand part of the string, starting from the given
offset. The first character in the string has offset zero. The abbreviation s
can be used instead of substr.
The substr expansion operator can take negative offset values to count
from the right-hand end of its operand. The last character is offset -1, the
second-last is offset -2, and so on. Thus, for example,
${substr_-5_2:1234567}
yields ``34''. If the absolute value of a negative offset is greater than the
length of the string, the substring starts at the beginning of the string, and
the length is reduced by the amount of overshoot. Thus, for example,
${substr_-5_2:12}
yields an empty string, but
${substr_-3_2:12}
yields ``1''.
If the second number is omitted from substr, the remainder of the string is
taken if the offset was positive. If it was negative, all characters in the
string preceding the offset point are taken. For example, an offset of -1 and
no length yields all but the last character of the string.
${uc:<string>}
This forces the letters in the string into upper-case.
The following conditions are available for testing by the ${if construct
while expanding strings:
!<condition>
Preceding any condition with an exclamation mark negates the result of the
condition.
<symbolic operator> {<string1>}{<string2>}
There are a number of symbolic operators for doing numeric comparisons. They
are:
| = | equal |
| == | equal |
| > | greater |
| >= | greater or equal |
| < | less |
| <= | less or equal |
For example,
${if >{$message_size}{10M} ...
Note that the general negation operator provides for inequality testing. The
two strings must take the form of optionally signed decimal integers,
optionally followed by one of the letters ``K'' or ``M'' (in either upper or lower
case), signifying multiplication by 1024 or 1024*1024, respectively.
crypteq {<string1>}{<string2>}
This condition is included in the Exim binary if it is built to support any
authentication mechanisms (see chapter 32). Otherwise, it is
necessary to define SUPPORT_CRYPTEQ in Local/Makefile to get crypteq
included in the binary.
The crypteq condition has two arguments. The first is encrypted and compared
against the second, which is already encrypted. The second string may be in the
LDAP form for storing encrypted strings, which starts with the encryption type
in curly brackets, followed by the data. If the second string does not begin
with ``{'' it is assumed to be encrypted with crypt(), since such strings
cannot begin with ``{''. Typically this will be a field from a password file.
An example of an encrypted string in LDAP form is:
{md5}CY9rzUYh03PK3k6DJie09g==
If such a string appears directly in an expansion, the curly brackets have to
be quoted, because they are part of the expansion syntax. For example:
${if crypteq {test}{\{md5\}CY9rzUYh03PK3k6DJie09g==}{yes}{no}}
Two encryption types are currently supported:
{md5} computes the MD5 digest of the first string, and expresses this as
printable characters to compare with the remainder of the second string. If the
length of the comparison string is 24, Exim assumes that it is base-64 encoded
(as in the above example). If the length is 32, Exim assumes that it is a
hexadecimal encoding of the MD5 digest. If the length not 24 or 32, the
comparison fails.
{crypt} calls the crypt() function (the same action as when the string
does not start with ``}'').
def:<variable name>
The def condition must be followed by the name of one of the expansion
variables defined in section 11.8. The condition is true if the named
expansion variable does not contain the empty string, for example
${if def:sender_ident {from $sender_ident}}
Note that the variable name is given without a leading $ character. If the
variable does not exist, the expansion fails.
def:header_<header name>: or def:h_<header name>:
This condition is true if a message is being processed and the named header
exists in the message. For example,
${if def:header_reply-to:{$h_reply-to:}{$h_from:}}
Note that no $ appears before header_ or h_ in the condition,
and that header names must be terminated by colons if white space does not
follow.
exists {<file name>}
The substring is first expanded and then interpreted as an absolute path. The
condition is true if the named file (or directory) exists. The existence test
is done by calling the stat() function. The use of the exists test in
users' filter files may be locked out by the system administrator.
eq {<string1>}{<string2>}
The two substrings are first expanded. The condition is true if the two
resulting strings are identical, including the case of letters. Use the lc or
uc expansion operators to force both strings to the same case if you want to
do a caseless comparison.
first_delivery
This condition, which has no data, is true during a message's first delivery
attempt. It is false during any subsequent delivery attempts.
ldapauth {<ldap query>}
This condition supports user authentication using LDAP. See section 9.9
for details of how to use LDAP in lookups and the syntax of queries. For this
use, the query must contain a user name and password. The query itself is not
used, and can be empty. The condition is true if the user name and password are
accepted by the LDAP server, and false otherwise. See chapter 32
for details of SMTP authentication, and chapter 33 for an example
of how this can be used.
match {<string1>}{<string2>}
The two substrings are first expanded. The second is then treated as a regular
expression and applied to the first. Because of the pre-expansion, if the
regular expression contains dollar, or backslash characters, they must be
escaped. Care must also be taken if the regular expression contains braces
(curly brackets). A closing brace must be escaped so that it is not taken as a
premature termination of <string2>. The easiest approach is to use the
\N feature to disable expansion of the regular expression.
For exa