SnertSoft: We Serve Your Server

milter-cli/0.19 (beta)
A Command Line Interface Milter


Description & Usage ° Installation & Notes ° License & Support

WARNING

THIS IS MAIL FILTERING SOFTWARE AND WILL BLOCK MAIL THAT FAILS TO PASS A GIVEN SET OF TESTS. SNERTSOFT AND THE AUTHOR DO NOT ACCEPT ANY RESPONSIBLITY FOR MAIL REJECTED OR POSSIBLE LOSS OF BUSINESSS THROUGH THE USE OF THIS SOFTWARE. BY INSTALLING THIS SOFTWARE THE CLIENT UNDERSTANDS AND ACCEPTS THE RISKS INVOLVED.

Description

This Sendmail mail filter provides a means by which client connection and envelope details and/or message headers and content can be filtered using shell commands or scripts. Intended as a way to implement quick & dirty content filtering solutions when there is no other suitable milter available. For example it could be used with command-line based anti-virus scanners.

Options

access-db=/etc/mail/access.db
The type and location of the read-only access key-value map. It provides a centralised means to black and white list hosts, domains, mail addresses, etc. The following methods are supported:
text!/path/map.txtR/O text file, memory hash
/path/map.dbBerkeley DB hash format
db!/path/map.dbBerkeley DB hash format
db!btree!/path/map.dbBerkeley DB btree format
sql!/path/databaseAn SQLite3 database
socketmap!host:portSendmail style socket-map
socketmap!/path/local/socketSendmail style socket-map
socketmap!123.45.67.89:portSendmail style socket-map
socketmap![2001:0DB8::1234]:portSendmail style socket-map

If :port is omitted, the default is 7953.

The access-db contains key-value pairs. Lookups are performed from most to least specific, stopping on the first entry found. Keys are case-insensitive.

An IPv4 lookup is repeated several times reducing the IP address by one octet from right to left until a match is found.

tag:192.0.2.9
tag:192.0.2
tag:192.0
tag:192

An IPv6 lookup is repeated several times reducing the IP address by one 16-bit word from right to left until a match is found.

tag:2001:0DB8:0:0:0:0:1234:5678
tag:2001:0DB8:0:0:0:0:1234
tag:2001:0DB8:0:0:0:0
tag:2001:0DB8:0:0:0
tag:2001:0DB8:0:0
tag:2001:0DB8:0:0
tag:2001:0DB8:0
tag:2001:0DB8
tag:2001

A domain lookup is repeated several times reducing the domain by one label from left to right until a match is found.

tag:[ipv6:2001:0DB8::1234:5678]
tag:[192.0.2.9]
tag:sub.domain.tld
tag:domain.tld
tag:tld
tag:

An email lookup is similar to a domain lookup, the exact address is first tried, then the address's domain, and finally the local part of the address.

tag:account@sub.domain.tld
tag:sub.domain.tld
tag:domain.tld
tag:tld
tag:account@
tag:

If a key is found and is a milter specific tag (ie. milter-cli-Connect, milter-cli-From, milter-cli-Auth, milter-cli-To), then the value is processed as a pattern list and the result returned. The Sendmail variants cannot have a pattern list. A pattern list is a whitespace separated list of pattern-action pairs followed by an optional default action. The supported patterns are:

[network/cidr]actionClassless Inter-Domain Routing
!pattern!actionSimple fast text matching.
/regex/actionPOSIX Extended Regular Expressions

The CIDR will only ever match for IP address related lookups.

A !pattern! uses an astrisk (*) for a wildcard, scanning over zero or more characters; a question-mark (?) matches any single character; a backslash followed by any character treats it as a literal (it loses any special meaning).

!abc!exact match for 'abc'
!abc*!match 'abc' at start of string
!*abc!match 'abc' at the end of string
!abc*def!match 'abc' at the start and match 'def' at the end, maybe with stuff in between.
!*abc*def*!find 'abc', then find 'def'

For black-white lookups, the following actions are recognised: OK or RELAY (white list), REJECT or ERROR (black list), DISCARD (accept & discard), SKIP or DUNNO (stop lookup, no result), and NEXT (opposite of SKIP, resume lookup). Its possible to specify an empty action after a pattern, which is treated like SKIP returning an undefined result. Other options may specify other actions.

Below is a list of supported tags. Other options may specify additional tags.

  
milter-cli-Connect:client-ip  value   § Can be a pattern list.
Connect:client-ip  value
 
milter-cli-Connect:[client-ip]  value   § Can be a pattern list.
milter-cli-Connect:client-domain  value   § Can be a pattern list.
milter-cli-Connect:  value   § Can be a pattern list.
Connect:[client-ip] value
Connect:client-domain value
 
All mail sent by a connecting client-ip, unresolved client-ip address or IP addresses that resolve to a client-domain are black or white-listed. These allows you to white-list your network for mail sent internally and off-site, or connections from outside networks. Note that Sendmail also has special semantics for Connect: and untagged forms.
 
milter-cli-From:sender-address  value   § Can be a pattern list.
milter-cli-From:sender-domain  value   § Can be a pattern list.
milter-cli-From:sender@  value   § Can be a pattern list.
milter-cli-From:  value   § Can be a pattern list.
From:sender-address value
From:sender-domain value
From:sender@ value
 
All mail from the sender-address, sender-domain, or that begins with sender is black or white-listed. In the case of a +detailed email address, the left hand side of the +detail is used for the sender@ lookup. Note that Sendmail also has special semantics for From: and untagged forms.
 
milter-cli-Auth:auth_authenvalue   § Can be a pattern list.
milter-cli-Auth:value   § Can be a pattern list.
 
All mail from the authenticated sender, as given by sendmail's {auth_authen} macro, is black or white-listed. The string searched by the pattern list will be the sender-address. The empty form of milter-cli-Auth: allows for a milter specific default only when {auth_authen} is defined.
 
milter-cli-To:recipient-address  value   § Can be a pattern list.
milter-cli-To:recipient-domain  value   § Can be a pattern list.
milter-cli-To:recipient@  value   § Can be a pattern list.
milter-cli-To:  value   § Can be a pattern list.
Spam:recipient-address value   * (FRIEND or HATER are recognised)
Spam:recipient-domain value   * (FRIEND or HATER are recognised)
Spam:recipient@ value   * (FRIEND or HATER are recognised)
To:recipient-address value
To:recipient-domain value
To:recipient@ value
 
All mail to the recipient-address, recipient-domain, or that begins with recipient is black or white-listed. In the case of a +detailed email address, the left hand side of the +detail is used for the recipient@ lookup. Note that Sendmail also has special semantics for Spam:, To:, and untagged forms.
 

The milter-cli-Connect:, milter-cli-From:, and milter-cli-To: tags provide a milter specific means to override the Sendmail variants. For example, you normally white list your local network through any and all milters, but on the odd occasion you might want to actually scan mail from inside going out, without removing the Connect: tag that allows Sendmail to relay for your network or white listing for other milters. So for example if you have Sendmail tags like:

To:mx.example.comRELAY

You might have to add milter specific overrides in order to make sure the mail still gets filtered:

To:mx.example.comRELAY
milter-cli-To:mx.example.comSKIP

Some additional examples:

milter-cli-Connect:80.94 [80.94.96.0/20]OK  REJECT
 
Accept connections from the netblock 80.94.96.0/20 (80.94.96.0 through to 80.94.111.255) and rejecting anything else in 80.94.0.0/16.
 
milter-cli-Connect:192.0.2 /^192\.0\.2\.8[0-9]/OK  REJECT
 
Accept connections from 192.0.2.80 through to 192.0.2.89, reject everything else in 192.0.2.0/24.
 
milter-cli-From:example.com /^john@.+/OK  /^fred\+.*@.*/OK  REJECT
 
Accept mail from <john@example.com> and <fred@example.com> when fred's address contains a plus-detail in the address. Reject everything else from example.com.
 
milter-cli-To:example.net !*+*@*!REJECT  !*.smith@*!REJECT  /^[0-9].*/REJECT
 
Reject mail to example.net using a plus-detail address or to any user who's last name is "smith" or addresses starting with a digit. No default given, so B/W processing would continue.
 

Normally when the access.db lookup matches a milter tag, then the value pattern list is processed and there are no further access.db lookups. The NEXT action allows the access.db lookups to resume and is effectively the opposite of SKIP. Consider the following examples:

milter-cli-From:com
From:com
/@com/REJECT  NEXT
OK
 
Reject mail from places like compaq.com or com.com if the pattern matches, but resume the access.db lookups otherwise.
 
milter-cli-From:aol.com  
From:fred@aol.com  
/^[a-zA-Z0-9!#$&'*+=?^_`{|}~.-]{3,16}@aol.com$/NEXT  REJECT
OK
 
AOL local parts are between 3 and 16 characters long and can contain dots and RFC 2822 atext characters except % and /. The NEXT used above allows one simple regex to validate the format of the address and proceed to lookup white listed and/or black listed addresses.
 
content-filter=
A shell command line or script used to filter the message headers and content.

The command first reads the message queue ID, followed by the message headers and body from standard input and may write to standard output a report. Standard error is merged with standard output. The command must exit with one of the following values:

 0 Accept the message, ignore standard output.

 1 Temporary reject of the message with 450 4.7.1; use no more than the first 32 lines from standard output for the SMTP response.

Note that RFC 2821 4.1.1.4 DATA paragraph 4 states that its not possible to return partial failure after receiving the message; either accept or reject. Therefore use of this return code may have undesired results.

 2 Reject the message with 550 5.7.1; use no more than the first 32 lines from standard output for the SMTP response.
 3 Discard the message, log standard output

 4 Tag the subject line with the first line of output. Subsequent output lines are added as a X-milter-cli-Report header.
 5 Copy the message to the mail addresses given by standard output, one address per line. Note that report lines can be interleaved with recipients lines provided they contain at least one space. A X-milter-cli-Report header is then added to the message.
 6 Redirect the message to the mail addresses given by standard output, one address per line. Note that report lines can be interleaved with recipients lines provided they contain at least one space. A X-milter-cli-Report header is then added to the message.
 value Any other value results in a temporary reject of the message with 450 4.7.1; log standard output.
content-max-size=64
The maximum number of kilo bytes to pass to the content-filter= command. Specify 0 for unlimited. For efficiency, normally only the first 64KB is passed, however some sites might prefer more accuracy instead of speed.
envelope-filter=
A shell command line or script used to filter the client connection and envelope information. This command is executed when the DATA command is sent by the client and so executes before the content-filter= command. If this command exits with a value greater than 0, then the content-filter= command is never executed.

The command reads the following lines from standard input: the client IP, the client name, the HELO/EHLO argument, the MAIL FROM: argument, the message queue ID, and one or more RCPT TO: arguments until EOF. The command may write to standard output a report. Standard error is merged with standard output. The command must exit with one of the following values:

 0 Accept the message, ignore standard output.

 1 Temporary reject of the message with 450 4.7.1; use no more than the first 32 lines from standard output for the SMTP response.
 2 Reject the message with 550 5.7.1; use no more than the first 32 lines from standard output for the SMTP response.
 3 Discard the message, log standard output

 4 Tag the subject line with the first line of output. Subsequent output lines are added as a X-milter-cli-Report header.
 5 Copy the message to the mail addresses given by standard output, one address per line. Note that report lines can be interleaved with recipients lines provided they contain at least one space. A X-milter-cli-Report header is then added to the message.
 6 Redirect the message to the mail addresses given by standard output, one address per line. Note that report lines can be interleaved with recipients lines provided they contain at least one space. A X-milter-cli-Report header is then added to the message.
 value Any other value results in a temporary reject of the message with 450 4.7.1; log standard output.
+daemon
Start as a background daemon or foreground application.
file=/etc/mail/milter-cli.cf
Read the option file before command line options. This option is set by default. To disable the use of an option file, simply say file=''
filter-timeout=30
The filter I/O timeout in seconds, 0 for indefinite.
-help or +help
Write the option summary to standard output and exit. The output is suitable for use as an option file.
±info
Write the configuration and compile time options to standard output and exit.
mitler-id=
Specify a milter instance ID string. This allows multiple instances of the milter to be run, by changing the milter name to milter-cli-id. This affects the .pid file, access database lookups by milter tag, and the default unix domain socket (Internet sockets must be uniquely specified).
milter-socket=unix:/var/run/milter/milter-cli.socket
A socket specifier used to communicate between Sendmail and milter-cli. Typically a unix named socket or a host:port. This value must match the value specified for the INPUT_MAIL_FILTER() macro in the sendmail.mc file. The accepted syntax is:
{unix|local}:/path/to/file
A named pipe. (default)
inet:port@{hostname|ip-address}
An IPV4 socket.
inet6:port@{hostname|ip-address}
An IPV6 socket.
milter-timeout=7210
The sendmail/milter I/O timeout in seconds.
pid-file=/var/run/milter/milter-cli.pid
The file path of where to save the process-id.
±quit
Quit an already running instance of the milter and exit. This is equivalent to: kill -QUIT `cat /var/run/milter/milter-cli.pid`
±restart
Terminate an already running instance of the milter before starting.
run-group=milter
The process runtime group name to be used when started by root.
run-user=milter
The process runtime user name to be used when started by root.
verbose=info
A comma separated list of how much detail to write to the mail log. Those mark with § have meaning for this milter.
§ all All messages
§ 0 Log nothing.
§ info General info messages. (default)
§ trace Trace progress through the milter.
§ parse Details from parsing addresses or special strings.
  debug Lots of debug messages.
§ dialog I/O from Communications dialog
  state State transitions of message body scanner.
  dns Trace & debug of DNS operations
  cache Cache get/put/gc operations.
§ database Sendmail database lookups.
  socket-fd Socket open & close calls
  socket-all All socket operations & I/O
§ libmilter libmilter engine diagnostics
±version
Show version and copyright.
work-dir=/var/tmp
The working directory of the process. Normally serves no purpose unless the kernel option that permits daemon process core dumps is set.

SMTP Responses

This is the list of possible SMTP responses.

553 5.1.0 imbalanced angle brackets in path
The path given for a MAIL or RCPT command is missing a closing angle bracket
553 5.1.0 address does not conform to RFC 2821 syntax
The address is missing the angle brackets, < and >, as required by the RFC grammar.
553 5.1.0 local-part too long
The stuff before the @ is too long.
553 5.1.[37] invalid local part
The stuff before the @ sign contains unacceptable characters.
553 5.1.0 domain name too long
The stuff after the @ is too long.
553 5.1.7 address incomplete
Expecting a domain.tld after the @ sign and found none.
553 5.1.[37] invalid domain name
The domain after the @ sign contains unacceptable characters.

Installation

  1. Download:

    milter-cli/0.19 md5sum Change Log
    LibSnert md5sum Change Log
    Sendmail 8.14   http://www.sendmail.org/
    Berkeley DB   http://www.sleepycat.com/
  2. If you have never built a milter for Sendmail, then please make sure that you build and install libmilter, which is not built by default when you build Sendmail. Please read the libmilter documentation. Briefly, it should be something like this:

    cd (path to)/sendmail-8.14.0/libmilter
    sh Build -c install
    
  3. The build process for libsnert and milter-cli is pretty straight forward once you have libmilter installed:

    cd (path to)/com/snert/src/lib
    ./configure
    make build
    cd ../milter-cli
    ./configure
    make build
    make install
    

    Both configuration scripts have some options that allow you to override defaults. Those options are listed with:

    ./configure --help
    
  4. An example /usr/local/share/examples/milter-cli/milter-cli.mc is supplied. This file should be reviewed and the necessary elements inserted into your Sendmail .mc file and sendmail.cf rebuilt. Please note the comments on the general milter flags.

    
    
  5. Once installed and configured, start milter-cli and then restart Sendmail. An example startup script is provided in /usr/local/share/examples/milter-cli/milter-cli.sh. The default options can be altered by specifying them on the command-line or within a /etc/mail/milter-cli.cf. The milter-cli.cf is parsed first followed by the command-line options.

Notes

  • Currently tested platforms:

    Cobalt Qube 1 with Linux RH 5.1 (mips 2.0.34 kernel); OpenBSD 3.6 (Intel x386)
  • The /usr/local/share/examples/milter-cli/ directory contains the following scripts:

     env.awk  Rejects mail that appears to come from a dynamic address pool based on a POSIX regex pattern:

    envelope-filter='/usr/local/share/examples/milter-cli/env.awk'

     env2.awk  Tag mail that appears to come from a dynamic address pool using a diffent scheme for identifying them:

    envelope-filter='/usr/local/share/examples/milter-cli/env2.sh'

     ext.sh  Rejects mail contain attachments with Windows executable extensions:

    content-filter='/usr/local/share/examples/milter-cli/ext.sh'

     ext.awk  A more involved example using AWK to reject Windows executable extensions with the file name and a multiline reply. Reject anti-virus notices. Also redirect mail with .zip or .rar extensions to a spam box:

    content-filter='/usr/bin/awk -f /usr/local/share/examples/milter-cli/ext.awk'

     tags.awk  Another AWK example that rejects mail containing an IFRAME with inline content and tracking bugs. Also tags subjects lines of messages using inline style sheets:

    content-filter='/usr/bin/awk -f /usr/local/share/examples/milter-cli/tags.awk'

    Note that ${prefix} at ./configure time defaults to /usr/local.

  • The minimum desired file ownership and permissions are as follows for a typical Linux system. For FreeBSD, NetBSD, and OpenBSD the binary and cache locations may differ, but have the same permissions.

    Process user ``milter'' is primary member of group ``milter'' and secondary member of group ``smmsp''. Note that the milter should be started as root, so that it can create a .pid file and .socket file in /var/run; after which it will switch process ownership to milter:milter before starting the accept socket thread.

    /etc/mail/root:smmsp0750 drwxr-x---
    /etc/mail/access.dbroot:smmsp0640 -rw-r-----
    /etc/mail/sendmail.cfroot:smmsp0640 -rw-r-----
    /etc/mail/milter-cli.cfroot:root0644 -rw-r--r--
    /var/run/milter/milter-cli.pidmilter:milter0644 -rw-r--r--
    /var/run/milter/milter-cli.socketmilter:milter0644 srw-r--r--
    /var/db/milter-climilter:milter0644 -rw-r--r-- (*BSD)
    /var/cache/milter-climilter:milter0644 -rw-r--r-- (linux)
    /usr/local/libexec/milter-cliroot:milter0550 -r-xr-x---
  • I would like to express my thanks to Derek Balling for his support at http://www.milter.org/.

License Agreement 1.8

SNERTSOFT & CO. ARE WILLING TO LICENSE THE SOFTWARE IDENTIFIED ABOVE TO YOU ONLY UPON THE CONDITION THAT YOU ACCEPT ALL OF THE TERMS CONTAINED IN THIS LICENSE AGREEMENT. PLEASE READ THE AGREEMENT CAREFULLY. BY DOWNLOADING OR INSTALLING THIS SOFTWARE, YOU ACCEPT THE TERMS OF THE AGREEMENT.

  1. Definitions

    1. ``Package'' means the identified above in source and/or binary form, any other machine readable materials provided (including, but not limited to documentation, sample files, data files), any updates or error corrections, and its derivative works.

    2. ``Organisation'' means a legal entity or an individual.

    3. ``You'' (or ``Your'') means an Organisation exercising rights under, and complying with all of the terms of, this License or a future version of this License issued under Section 6.1. For legal entities, ``You'' includes any entity which controls, is controlled by, or is under common control with You. For purposes of this definition,``control'' means (a) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (b) ownership of more than fifty percent (50%) of the outstanding shares or beneficial ownership of such entity.

    4. ``SnertSoft'' means Anthony C. Howe situated in Cannes, France (SIRET #489 259 937 00014).

    5. ``SnertSoft & Co.'' means SnertSoft and all authorised & licensed partners, such as value-added resellers or appliance manufacturers.

  2. Statement of Original Work

    1. The Package is an original work written by SnertSoft, with exception of following third party code:

      1. MD5 routines written by L. Peter Deutsch, based on RFC 1321;

      2. SQLite3 package is written by D. Richard Hipp of Hwaci and is in the public domain.

      3. strnatcmp.c is written by Martin Pool of sourcefrog.net and has a BSD style license.

      4. Protothreads headers originally written by Adam Dunkels and has a BSD style license.

  3. License To Use

    1. You may install and use this Package, without modifications, exclusively on machines for which You have purchased a license, provided You retain this notice, SnertSoft's copyright notice, any and all license control methods (see below), and any links within the Package back to the most current online versions of this License and Disclaimer.

    2. You may copy, share, distribute, modify, and create derivative works from the user manuals and any related documentation solely for Your internal business purposes, such as in-house documentation, training manuals, or reference material.

  4. Restrictions

    1. Redistribution, including but not limited to books, CDROMS, download mirrors, floppy diskettes, hard disks, hardcopy print outs, online archives, solid state disks, streaming tapes, or other current or future forms of storage or communication media of the Package, with or without modifications, including any and all derivative works such as source patches, binaries, binary patches, or similar is expressly forbidden without prior written permission in hardcopy (ie. letter or fax) signed and dated by SnertSoft.

    2. It is expressly forbidden for You to use the Package, in whole or in part, in any other software or appliance without prior written permission in hardcopy (ie. letter or fax) signed and dated by SnertSoft.

    3. It is expressly forbidden for You to use the Package to develop any software or other technology having the same primary function as the Package, including but not limited to using the Package in any development or test procedure that seeks to develop like software or other technology, or determine if such software or other technology performs in a similar manner as the Package.

    4. You may not sell, rent, lease, or transfer the Package to third parties without prior written permission in hardcopy (ie. letter or fax) signed and dated by SnertSoft.

  5. Termination

    1. This Agreement is effective until terminated. You may terminate this Agreement at any time by destroying all copies of the Package.

    2. This Agreement will terminate immediately without notice from SnertSoft if You fail to comply with any provision of this Agreement.

    3. Either party may terminate this Agreement immediately should any portion of the Package become, or in either party's opinion be likely to become, the subject of a claim of infringement of any intellectual property right. Upon Termination, You must destroy all copies of the Package.

  6. Versions Of The License

    1. New Versions. SnertSoft may publish revised and/or new versions of the License from time to time. Each version will be given a distinguishing version number.

    2. Effect of New Versions. Once a version of the Package has been published under a particular version of the License, You may always continue to use it under the terms of that License version. You may also choose to use such Package under the terms of the most current version of the License published by SnertSoft.

    3. No one other than SnertSoft has the right to modify the terms applicable to the Package created under this License.

Disclaimer

THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO WAY SHALL SNERTSOFT OR LICENSEE BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

License Control

The Package may use one or more license control methods including, but not limited to, license key activation, periodic reporting of Package details and IP address of installation to SnertSoft & Co., or remote license verification by SnertSoft & Co.. Any information reported to or gathered by SnertSoft & Co. shall remain strictly confidential and the private property of SnertSoft & Co.. Under no circumstances will SnertSoft & Co. resell or release this information to third parties, unless demanded by court order.

Support

Support is provided for one year from date of purchase and only for SnertSoft's original Package that was purchased directly from SnertSoft. Additional support beyond the first year can be obtained from SnertSoft on time & materials basis or from one of SnertSoft's authorised partners.

Support for the Package obtained from authorised partners, such as value-added resllers or appliance manufacturers, will be supplied by those partners. SnertSoft will not support the Package without proof of purchase from SnertSoft, such as an Order N° or Invoice N°.

Package enhancements requests and product suggestions are always welcome. A community mailing list is available; please refer to SnertSoft web site Support area for details.

Gifts

Gifts from the author's Amazon US or Amazon UK wishlist (search by mail address <achowe at snert dot com>) are welcomed for the continued encouragement, moral support, and ego pumping needed to work in foreign non-english speaking lands.

Daleks since 7 May 2005