Setting up `procmail` on XMission

Contents:

Creating the basic files and directories.
A basic .procmailrc file.
Turning on procmail.
Tips and tricks.

On a RedHat Linux box, procmail is the default mail delivery agent, the software that drops incoming messages into your inbox. On XMission's servers, other methods are used by default but you can still make procmail work. Setting up procmail has a coupld of advantages:

You can pre-process mail, sorting it into folders as it arrives instead of after it's been downloaded to your own machine. If you subscribe to high-volume mailing lists, this can significantly cut down the amount of time it takes to sort out your mail by seperating the list mail from everything else. You can also file mail from particular people into high-priority folders to be checked before everything else.
You can completely discard mail that qualifies as spam or is otherwise undesirable, or signal the mail system to simply refuse to accept delivery of it, avoiding the need to download it at all.
You can keep mail under your home directory instead of in the mail spool, which lets you avoid the much lower mail-spool quota limits by using your personal disk quota for your mail.

All of these make it worthwhile to have procmail running. So, here's a summary of how to do it and some tips on things you can do to make life easier for you.

What follows may sometimes appear to be in the opposite order from what you might think. That's because things will break badly if you, for example, turn on forwarding of mail into procmail before having things configured and places to put mail ready.

Creating the basic files and directories

You'll need a few things set up first. One thing is a directory to put mail folders into. Often this will correspond to the mail folder directory for e-mail programs like Pine, so you can read mail in your folders directly. I conventionally use the Mail directory under my home directory for this, and configure all my mail clients to use this directory for mail folders ( Pine uses it by default ). To do this, do:

cd
mkdir Mail
chmod u=rwx,go= Mail
ln -s Mail mail

This will create an empty directory for mail folders. I make the symbolic link from mail to Mail because some programs use the all-lower-case form and this makes them happily refer to the real directory without headaches. The permissions are set to allow only you to access the directory, because under normal conditions nobody but you should be reading your e-mail. This directory will only be used for mail you put into folders, mail sent to or left in your default inbox will still appear in your mail spool file as it did before you turned on procmail.

Another thing procmail will likely use is a log file to hold summary information about all e-mail processed and what was done with it. This should also only be accessible by you, so do:

cd
touch procmail.log
chmod u=rw,go= procmail.log

If you turn on logging in procmail, every message processed will cause an entry to be written here telling who it was from, when it arrived, what the subject line was, how big the message was and what procmail did with it.

Creating a basic `.procmailrc` file

Now that you've got places to put your mail, you need to create the .procmailrc file that tells procmail how to process your messages. For basic things this doesn't have to be complex. A very simple one looks like:

MAILDIR=${HOME}/Mail
LOGFILE=${HOME}/procmail.log

:0 :
* ^X-RBL-Warning
rbl-warning

The first line tells procmail that mail folders should be in the Mail directory under your home directory. The second says that it should log information about processed messages in the procmail.log file you created in the previous section.

The last 3 lines are a recipe. The first line begins with a colon ':' to mark it as the start of a recipe. The 0 tells procmail to automatically determine how many header match lines there are. The trailing colon ':' indicates that it should use a lockfile to insure that it doesn't try to deliver 2 messages to a folder at the same time. The second line is a header match line. The asterisk '*' marks it as such, and after the space is a regular expression. Case is not relevant in it. The headers of the mail message will be examined, and if every header match line matches a header in the message then this recipe will be executed. The final line specifices a folder name to deliver the message to if it matches this recipe. This particular recipe says that if the message contains the X-RBL-Warning header that XMission's mail software puts in messages coming from sources on the MAPS Realtime Blackhole List or listed in the Open Relay Blocking System, it'll be dumped into the Mail/rbl-warning folder instead of being put into your inbox. This should make a noticeable dent in your spam. The only potential problem is if people you know use the same ISP as a spammer, in which case their messages will be dropped into this folder as well. You can delete the recipe, leaving only the first two lines in the .procmailrc file, if you think this will be a serious problem.

The mail system is sensitive to permissions on the .procmailrc file, so once you've created it you should do:

chmod u=rw,go= .procmailrc

to insure that only you can access it and that procmail will not ignore it due to excessively-open permissions.

All mail not handled by a recipe in the file will be delivered to your default inbox in the mail spool directory, the same place it is delivered without procmail

Turning on `procmail`

Now that you've got the configuration file created, it's time to turn on procmail and let it work. To do this, you set up a .forward file telling the mail system to send all your mail through procmail instead of delivering it. procmail will then handle delivery per the rules you've set up in your .procmailrc file. The .forward file in your home directory should contain exactly this:

"|/usr/local/bin/procmail -f-"

The double-quotes around it are neccesary. Once it's been created, you need to set it's permissions correctly:

chmod u=rw,go= .forward

If the permissions are not set to exactly this, the mail system will ignore the .forward file and procmail will not be used.

At this point you should be able to send yourself an e-mail message, using your favorite mail client. Check that it showed up in your inbox, and that lines were added to procmail.log showing the summary information about the message. If nothing was added to procmail.log then check the permissions and contents of the .forward and .procmailrc files and make sure they are correct. If procmail.log has information but the message didn't appear in your inbox, check the last line of the log entry to see where procmail delivered the message.

Tips and tricks

The first tip is that, in .procmailrc, order isn't important, it's critical. Processing stops, usually, with the first recipe that a message matches. This means that you should put more-specific recipes before less-specific ones, for example, and that you should put recipes that explicitly accept particular types of messages before ones that reject broad classes of messages.

For example, I subscribe to the RedHat announcement mailing list. This mailing list doesn't put my address as a recipient of mail, but it's legitimate mail to me. I have a recipe in my .procmailrc that looks like this:

:0 :
* ^To:.*redhat-announce-list@redhat\.com
* ^Sender:.*redhat-announce-list-admin@redhat\.com
* ^List-Id:.*redhat-announce-list.redhat.com
$DEFAULT

This recipe specifies header lines that identify the 'signature' of a message from the RedHat announcement list, the set of headers that occur in those messages and not in any others ( barring someone deliberately setting out to get past my rules ). Those messages are then dropped into my default inbox. I also have a rule that says that all mail not addressed explicitly to me should be considered spam:

:0 :
* !^TO_tknarr
spam

To work correctly, I need to put the recipe to accept the RedHat announcements before the one to dump spam into it's own folder. If I put the spam-rejecting one first, all mail from the RedHat list would match it and go into the spam folder and processing would never reach the one to accept RedHat announcements.

This usually shows up when you have a spam-rejecting rule like mine in place and subscribe to a new mailing list, only to find that all the mail from it is getting rejected. If you just dump it into a folder then you can extract those messages from it and add appropriate rules to accept them, but if you sent them to /dev/null instead you've lost mail.

A hint about the last line of a recipe: filenames that don't begin with a '/', like all the folder names I've used so far, are considered to be in or below the directory set by the MAILDIR setting. Filenames that do begin with a '/', such as /dev/null can also be used. Specifying /dev/null as the place to deliver mail to sends that mail into the bit bucket, gone forever. This is good for disposing of mail you didn't want, but mistakes can be uncorrectable ( or worse, undetectable, which is one reason for using the log file and checking it occasionally to see which messages might have gotten discarded that shouldn't have been ).

If you sort your mail into folders using procmail on XMission's servers and use fetchmail to retrieve mail, you probably wonder how to get to those folders. It's actually fairly simple: just use the folders keyword in .fetchmailrc to specify the folders to retrieve from. For example, if you file mail into the important folder if it's from family, friends or other people you know, then just after the password entry in .fetchmailrc you should add this line:

    folders INBOX,Mail/important

This tells fetchmail to get mail from the important folder in your Mail directory in addition to your regular inbox. Pine users who occasionally fire it up directly on the XMission shell machine can also make use of this to retrieve mail from the received folder.

Of course, the problem is that all that information about which folder you filed things in gets lost when fetchmail brings the mail down to your local machine. Wouldn't it be nice to be able to keep that info? It's not really that hard. procmail can deliver to filter programs as well as files, specifically the formail program designed to edit headers of mail messages. For example, you could decide to use the X-folder header to contain the name of the folder you filed the mail in. Say I wanted to file all that RedHat announcement mail in the rh-announce folder, and flag it with a header for when it arrives on the local machine. I could modify the recipe from earlier to read:

:0 :
* ^To:.*redhat-announce-list@redhat\.com
* ^Sender:.*redhat-announce-list-admin@redhat\.com
* ^List-Id:.*redhat-announce-list.redhat.com
|/usr/local/bin/formail -i "X-folder: rh-announce" >>${MAILDIR}/rh-announce

procmail will see the vertical bar ( pipe ) character and treat the remainder of the delivery line as a command line to run. The message will be sent to the standard input of the command. formail in this instance will insert an X-folder header as specified and append the message to the rh-announce folder. The .procmailrc file on your Linux box can then just look for appropriate X-folder lines instead of having to repeat the more complex header-matching rules.

For more information, check the procmail, procmailrc, procmailex and procmailsc man pages. These give details about how procmail operates, the exact syntax of the .procmailrc file and examples of recipes to do various things with procmail. You can also check the man page for formail for it's options. It's a rather useful program, used for adding/editing headers and generating automatic replies to incoming e-mail. Remember to check your work before turning things loose. This is one place where having a Linux box is nice, you can test procmail recipes and tricks locally without risking having things blow up where someone else will be affected.

tknarr@silverglass.org

Setting up procmail on XMission

Creating the basic files and directories

Creating a basic .procmailrc file

Turning on procmail

Tips and tricks

Setting up `procmail` on XMission

Creating a basic `.procmailrc` file

Turning on `procmail`