handle-auto

This is a script for qmail systems, which handles aliases of the form "auto-ext@domain.xyz", where "auto" is a fixed name, "ext" is a variable extension, and "domain.xyz" is your domain name. Any messages sent to such an address will be stored in a sub-folder named "ext" within a specific Maildir. It's kind of convoluted, but when you see an example it makes perfect sense.

So here's an example of how I'm using it.

I receive "cron" reports from several servers every day. It gets rather irritating to have to wade through these messages, especially when they're mixed in with my normal mail. So what I've done is set this script up to handle mail sent to any "cron-_____@domain.xyz" addresses. (I've obviously changed the domain name- the last thing I need is a spammer to start filling up my cron boxes with crap.)

My INBOX folder has a folder called "cron". I've configured the script to point to this folder, which means that any messages sent to (for example) "cron-xyzmail@domain.xyz" is delivered to an "xyzmail" folder within the "cron" folder... and if the "xyzmail" folder doesn't exist, it is automatically created.

The script can also be used in cases where a web site demands an email address, and you're pretty sure they're just going to use it to send you spam- or even worse, they're going to give or sell your address to others so they can send you spam. You might give them a fake email address, but if they want to send you some kind of link to click on, to first prove that you own the address, it doesn't do any good to give them a fake address.

You can give them one of these "auto" addresses, and their messages will show up in a folder within your INBOX. You can click on the link to prove that the address is real, and then if they start spamming you (or if others start spamming you) you will have proof that they are the cause of the spam and you can "deactivate" that address later on, by making it not deliver messages anywhere.

This solution is not as elegant as something like SpamGourmet (which I've been using for several years now), but it's easy enough to set up on your own system, and if you're good with scripting you can make your own server do anything SpamGourmet does and more.

Requirements

This script is written to work on a qmail system. If you are using something else, you should be aware that the script relies on environment variables which are set by qmail's "qmail-local" program. Unless your MTA happens to set the same variables, chances are this script will not work on your server.

The script may or may not work on Plesk systems, since Plesk's MTA was originally built from qmail's source code. However, I don't have a Plesk system to test it with, and to be honest, given the fact that for years Plesk was distributing their binaries in violation of DJB's original qmail license, I'm not even interested in trying to support it... at least not for free. If you're using Plesk, then you are also presumably paying them for a support contract, and you should ask them for help with your server.

The script should work on any qmail system, however the directions below are being written under the assumption that you are also using vpopmail to manage virtual domains. If this is not the case, the script can still be used, however you will need to figure out the correct settings for some of the variables within the script on your own. If you're not sure about something, feel free to join my qmail-patch mailing list and ask questions there; the list exists to support my combined patch as well as the scripts and other information on this web site.

Notes

• Be careful using "-", or whatever your conf-ext character is, as part of the "base name" of the email address which automatically creates the folders. If you do this, you will find that the names of your automatically created folders will all start with whatever is after the first hyphen in the name (so if your base name is "auto-from@domain.xyz" and a message arrives for "auto-from-server@domain.xyz", the script will create a folder called "from-server", rather than "server" as you might expect.

  Although now that I think about it, this might be something people might WANT to do. Hrmmmm... Just make sure that if you do it, you're doing it on purpose rather than accidentally.

  Thanks to Jason Sumner for pointing out this quirk.

• 2009-12-18 One problem I've run into with this script on my own server is the fact that some mailers will change the "case" of the email address. I have one sender who, even though I gave them the address in all lowercase, will sometimes send messages to the "all uppercase" version of the address.

  I've modified the script so it can now lowercase or uppercase the folder name. Personally, I'm lowercasing all of the names, however you may want to uppercase them, or you may not want to touch them at all. You can set the behaviour you want by un-commenting the "CASE=UPPER" or "CASE=LOWER" line within the script. If both are commented (which is the default) then the case will not be changed, and you may end up with separate folders named "ABC", "abc", and "Abc". It's up to you.

Download

Download the script and store it on your system, where any user is able to read and execute it. On my own server, and in this example, the script is stored as /usr/local/bin/handle-auto.

# cd /usr/local/bin
# wget http://qmail.jms1.net/scripts/handle-auto
...
# chmod 0755 handle-auto

Setting up the script

The examples below will assume the following:

• The server uses vpopmail, and the vpopmail user's home directory (shown in red in the examples below) is "/home/vpopmail".

• We are handling email for a fictional domain called "domain.xyz".

• We also manage a few dozen servers, and would like the cron output from those servers to be separated by server name.

• We want to set up a "cron-_____@domain.xyz" handler, which stores incoming messages in automatically created folders within a "cron" folder in the "postmaster" mailbox.

Decide on the PARENT location

The script creates folders automatically within a given PARENT location. This location is specified on the command line when the script is executed (i.e. it will be in the ".qmail-whatever" file which causes the script to be executed.) There are two possibilities for a PARENT:

• If the PARENT is the root of a particular mailbox, then the automatically created folders will be direct children within the INBOX of that mailbox. To configure this, the PARENT value MUST end with a "/" character, and must point to an already-existing Maildir.

  For example, this value will cause folders to be automatically created as first-level children in the INBOX of "postmaster@domain.xyz":

  /home/vpopmail/domains/domain.xyz/postmaster/

• If the PARENT is a folder within a particular INBOX, then the automatically created folders will be created as child folders within the folder you specify. To configure this, the PARENT value MUST NOT end with a "/" character. The specified folder does not have to already exist, however the Maildir which contains the folder must exist.

  For example, this value will cause folders to be automatically created as children of an "auto" folder within the INBOX of "postmaster@domain.xyz":

  /home/vpopmail/domains/domain.xyz/postmaster/.auto

Create the .qmail file(s)

Before we do this, we need to disable deliveries to the domain. This is so that any mail which arrives while we're working on the domain won't be processed through a halfway-finished .qmail-_____ file and accidentally be sent to the wrong place. We do this by setting the "sticky bit" on the directory.

The qmail-local program checks the sticky bit on the user's home directory before trying to deliver any messages to that user. (The way vpopmail uses the virtual user mechanism, qmail-local sees the domain directory as the home directory of a virtual user.) If the sticky bit is turned on, qmail-local will defer any deliveries, and the messages will stay in the queue and be delivered at some point in the future.

# cd /home/vpopmail/domains/domain.xyz
# chmod +t .

We are going to be creating two files... sorta. We're actually only going to create one file, but it's going to have two names. It may sound strange, but it makes perfect sense in the world of *nix filesystems.

The first filename will be ".qmail-", followed by the alias root name (i.e. for "cron-_____@domain.xyz", the word "cron".) So for this example, the filename is ".qmail-cron".

The file needs to contain a single line. The first character of that line must be a pipe character (i.e. "|", usually SHIFT-BACKSLASH on the keyboard.) It should be followed by the full path to where you installed the script, then a space, and then the PARENT value we figured out above. For example:

|/usr/local/bin/handle-auto /home/vpopmail/domains/domain.xyz/postmaster/.auto

You can create the file using any text editor (i.e. nano, vim, etc.) or even a command like this:

Make sure to use SINGLE QUOTES, otherwise the shell will try to interpret the pipe character and become confused.
# echo '|/usr/local/bin/handle-auto /home/vpopmail/domains/domain.xyz/postmaster/.auto' > .qmail-cron

The file should have the same ownership and permissions as the other ".qmail-_____" files in the same directory.

First find the correct ownership and permissions...
# ls -la .qmail-*
-rw------- 1 vpopmail vchkpw 46 Sep 24 2008 .qmail-default

Now copy that ownership to the new file...
# chown vpopmail:vchkpw .qmail-cron

... and copy the permissions as well.
# chmod 0600 .qmail-cron

Now we create the "second file"... which really means we're going to create a second name pointing to the same file. The "ln" command is used for this. You may have seen it before, for creating symbolic links. You could use a symbolic link for this, but it's easier and faster for the system if you use a "hard link", which is a new name pointing to the same inode number as an existing filename (remember that under *nix filesystems, the inode is the real file, and the filenames are just pointers to inodes.)

The second filename is the same as the first one, with "-default" added to the end of it. This is because the first name, ".qmail-cron", only handles the address "cron@domain.xyz". The second filename, ".qmail-cron-default", handles all "cron-_____@domain.xyz" addresses.

# ln .qmail-cron .qmail-cron-default

And now, if you look at a directory listing, you should see both of the files there.

$ ls -la .qmail-* -rw------- 2 vpopmail vchkpw 79 Nov 30 19:35 .qmail-cron -rw------- 2 vpopmail vchkpw 79 Nov 30 19:35 .qmail-cron-default -rw------- 1 vpopmail vchkpw 46 Sep 24 2008 .qmail-default

You may also notice the column between the permissions and the owner- that's the "link count" for the inode. The "2" means that the inode knows it has two names pointing to it. If you add the "-i" option to the "ls" command, it will show you the actual inode numbers. You will see that ".qmail-cron" and ".qmail-cron-default" have the same inode number.

$ ls -lai .qmail-* 6213923 -rw------- 2 vpopmail vchkpw 79 Nov 30 19:35 .qmail-cron 6213923 -rw------- 2 vpopmail vchkpw 79 Nov 30 19:35 .qmail-cron-default 6211862 -rw------- 1 vpopmail vchkpw 46 Sep 24 2008 .qmail-default

Now that we're done, we need to remove the "sticky bit" from the domain directory, so that incoming mail deliveries will happen normally again. DO NOT FORGET to do this, otherwise the entire domain will have no incoming mail at all.

# cd /home/vpopmail/domains/domain.xyz
# chmod -t .

Testing it all

You should now be able to send an email to the automatic alias address, as well as any extension of that address, and watch your qmail-send logs to make sure that the messages are being stored in folders where you expect them. Our testing procedure is simple- we're going to try sending a few messages, and see what happens.

# echo 'testing with no ext' | mail cron@domain.xyz # mtrack -p cron /service/qmail-send/log/main/current | tai64nlocal 2009-11-30 20:04:57.345043500 new msg 573850 2009-11-30 20:04:57.345045500 info msg 573850: bytes 218 from qp 17311 uid 0 2009-11-30 20:04:57.345047500 starting delivery 9679: msg 573850 to local domain.xyz-cron@domain.xyz 2009-11-30 20:04:57.361225500 delivery 9679: success: did_0+0+1/ 2009-11-30 20:04:57.361237500 end msg 573850 Looks good. Now try another message, this one with an extension. # echo 'testing with abc' | mail cron-abc@domain.xyz # mtrack -p cron /service/qmail-send/log/main/current | tai64nlocal 2009-11-30 20:04:57.345043500 new msg 573850 2009-11-30 20:04:57.345045500 info msg 573850: bytes 218 from qp 17311 uid 0 2009-11-30 20:04:57.345047500 starting delivery 9679: msg 573850 to local domain.xyz-cron@domain.xyz 2009-11-30 20:04:57.361225500 delivery 9679: success: did_0+0+1/ 2009-11-30 20:04:57.361237500 end msg 573850 2009-11-30 20:05:23.897388500 new msg 573850 2009-11-30 20:05:23.897390500 info msg 573850: bytes 219 from qp 17335 uid 0 2009-11-30 20:05:23.897392500 starting delivery 9680: msg 573850 to local domain.xyz-cron-abc@domain.xyz 2009-11-30 20:05:23.897416500 delivery 9680: success: did_0+0+1/ 2009-11-30 20:05:23.897426500 end msg 573850

If you look in that Maildir, you should see that it created the automatic folders.

# ls -la postmaster/Maildir drwx------ 29 vpopmail vchkpw 4096 Nov 30 16:29 . drwx------ 3 vpopmail vchkpw 4096 Sep 9 2007 .. drwx------ 5 vpopmail vchkpw 4096 Nov 30 20:04 .cron drwx------ 5 vpopmail vchkpw 4096 Nov 30 20:04 .cron.abc drwx------ 2 vpopmail vchkpw 4096 Oct 14 18:56 cur -rw------- 1 vpopmail vchkpw 36 May 13 2008 dovecot-keywords -rw------- 1 vpopmail vchkpw 84 Oct 14 18:56 dovecot-uidlist -rw------- 1 vpopmail vchkpw 384 Oct 14 18:56 dovecot.index -rw------- 1 vpopmail vchkpw 13412 Oct 14 23:14 dovecot.index.cache -rw------- 1 vpopmail vchkpw 9496 Oct 14 23:14 dovecot.index.log drwx------ 2 vpopmail vchkpw 4096 Sep 9 2007 new -rw------- 1 vpopmail vchkpw 61 Nov 1 01:09 subscriptions drwx------ 2 vpopmail vchkpw 4096 Oct 14 18:56 tmp

If you look further, you should find that the messages themselves were stored in those folders.

# ls -la postmaster/Maildir/.cron/new total 12 drwx------ 2 vpopmail vchkpw 4096 Nov 30 20:04 . drwx------ 5 vpopmail vchkpw 4096 Nov 30 20:04 .. -rw------- 1 vpopmail vchkpw 208 Nov 30 20:04 1259629497.P17313.phineas # cat postmaster/Maildir/.cron/new/1259629497.P17313.phineas Received: (qmail 17311 invoked by uid 0); 30 Nov 2009 20:04:57 -0500 Date: 30 Nov 2009 20:04:57 -0500 Message-ID: <20091201010457.17310.qmail@a.mx.jms1.net> From: root@jms1.net To: cron@domain.xyz testing with no ext # ls -la postmaster/Maildir/.cron.abc/new total 12 drwx------ 2 vpopmail vchkpw 4096 Nov 30 20:05 . drwx------ 5 vpopmail vchkpw 4096 Nov 30 20:05 .. -rw------- 1 vpopmail vchkpw 208 Nov 30 20:05 1259629523.P17337.phineas # cat postmaster/Maildir/.cron.abc/new/1259629523.P17337.phineas Received: (qmail 17335 invoked by uid 0); 30 Nov 2009 20:05:23 -0500 Date: 30 Nov 2009 20:05:23 -0500 Message-ID: <20091201010523.17334.qmail@a.mx.jms1.net> From: root@jms1.net To: cron-abc@domain.xyz testing with abc

One last thing...

At this point, the "cron-_____@domain.xyz" aliases should be working. However, if your system uses a validrcptto.cdb file, or some other mechanism which makes qmail-smtpd only accept messages intended for mailboxes which exist, you may need to rebuild the files it relies on, so that it knows about the "cron-_____@domain.xyz" addresses.

On my own system, with a qmail-updater service, I just need to touch the pipe it's watching, and it rebuilds everything for me automatically.

$ touch /tmp/update-qmail

Configuring the other servers

Now that we have this system for automatically sorting the incoming messages, the other half of the picture is to configure the other systems to send their messages to the new "cron-_____@domain.xyz" addresses. Each system will have to be changed by hand, and may need to be changed in multiple locations. This section will show how to change the target for a few of the commonly used server scenarios, however it cannot cover every possible scenario.

In other words, there is no substitute for you knowing how to administer your own systems.

cron

There are three different types of cron jobs:

• System-wide cron jobs run from /etc/crontab.

  Edit the /etc/crontab file. If there is a MAILTO= line, change it to point to the appropriate "cron-_____@domain.xyz" address. If not, add one near the top of the file, which looks like this:

  MAILTO=cron-_____@domain.xyz

  You may or may not need to restart or send a signal to "crond" after changing this file, see your system's documentation.

• System-wide cron jobs run from individual files in the /etc/cron.d/ directory.

  This is very similar to the /etc/crontab files above. Edit the /etc/cron.d/_____ file, and change or add the "MAILTO=" line.

  Again, you may or may not need to restart or send a signal to "crond" after editing these files.

• Per-user cron jobs.

  The user should run "crontab -e" to edit their personal crontab file. Again, either edit or add the "MAILTO=" line as needed.

  Most systems' crontab commands will automatically send whatever signals are necessary for crond to know that a change has been made, so you shouldn't have to do anything else once you are done editing the file. However, you should consult your system's documentation just to be sure.

Other automatic jobs

There may be other processes on the system which generate email whenever something happens. For example, smartd can send emails whenever it sees a new error or warning involving a SMART-capable hard drive. Each program will have its own method of configuring where those emails are sent. You should check with the documentation for these programs to find out how to configure them.

In many cases, these programs will simply send their messages to "root" on the machine. It may be helpful to change where "root" mail goes in general, just to catch all of these at once.

For a qmail system, edit the /var/qmail/alias/.qmail-root file. It should contain an "&" character followed by the email address where you want the messages to be forwarded. You may want to edit the .qmail-postmaster, .qmail-mailer-daemon, and .qmail-anonymous files in the same directory as well.

For systems running sendmail, create or edit a file called ".forward" in root's home directory, which contains just the email address where you want the messages to be forwarded. (This may also apply to other MTAs as well, I don't honestly know.)

De-activating aliases

Let's say you're using this script to create temporary addresses for web sites which you know are going to send you spam after they send you your password or software key or whatever. And now they've started sending their spam, or they've "allowed" this email address to get out to the spammers, and the address is basically worthless. How do you turn it off?

• If you are using my combined qmail patch version 6c6 or newer, you can configure a validrcptto.cdb file to reject incoming mail for that alias. This would be done by adding keys for the address you wish to reject, and the "-default" for that address, with "-" as the value. If you're using cdbmake-12 to build your validrcptto.cdb file, you need to add these two lines to text file which makes up the input:

  cron-badguys@domain.xyz:-
  cron-badguys-default@domain.xyz:-

  When you rebuild your validrcptto.cdb file with these lines in there, qmail-smtpd will refuse to accept any mail for these addresses, so they won't ever get into the queue to begin with.

• Another way, if your qmail patch supports it (my combined patch supports it as of version 6c, and several other commonly used patches have this as well) is to use a "badrcptto" file. Check the documentation for your patch- it may only take plain-text patterns, but it may take regular expressions.

• This method should work on any qmail system. Create a ".qmail-whatever" file, with a more specific name than the default catch-all name, whose contents make the messages simply disappear without being delivered.

  For example, let's say you have the "cron-_____@domain.xyz" we used above, and you need to "turn off" the address "cron-badguys@domain.xyz" but leave "cron-everythingelse@domain.xyz" working. The process looks like this:

  # cd /home/vpopmail/domains/domain.xyz
  # chmod +t . Prevent deliveries for now
  # echo '#' > .qmail-cron-badguys
  # chown vpopmail:vchkpw .qmail-cron-badguys
  # chmod 0600 .qmail-cron-badguys
  # chmod -t . Allow deliveries again
  

  Once this is done, any messages sent to "cron-badguys@domain.xyz" will be silently dropped on the floor. The qmail-send log will show them as being successfully delivered, however they won't actually be stored anywhere. They will just plain disappear.

Other considerations

When a mailbox is automatically created by manipulating the filesystem, it is not automatically added to the list of "subscribed" folders. This is because:

• Different IMAP servers (courier-imap, dovecot, binc, etc.) store their subscription lists differently. Some use a simple text file, but others might use some other file format, or a SQL database, or who knows what else. Some IMAP clients might even keep the subscription list on the local machine rather than the server, making it impossible for a server-side program like this to update the subscription list.

• If any IMAP client sessions are open and happen to be changing the subscription list at the same time a new folder is being added, the processes might try to update the file at the same time and the file can be corrupted. (I'm sure there's some kind of locking mechanism in place, but again it's going to be different for each IMAP server.)

• I haven't had time to research every IMAP server out there, to see how it stores subscriptions and how to safely update that information.

Until/unless I have time to do the research and figure out how subscriptions (and file locking on the subscriptions file itself) are handled, I recommend that you not use subscriptions on your INBOX to begin with. (To be honest, I'm not sure why somebody would have a folder in their INBOX without subscribing to it to begin with...)

Either that, or if you know that you're going to start using a particular automatic address (i.e. you're building a new server), send a test message to the automatic address (to create the folder, so you know that it exists) and then manually subscribe to the folder, using the mechanism provided by your MUA.

Copyright 1995-2024 John M. Simpson KG4ZOW (email) (PGP key)
Last updated 2009-12-18 21:49:13 +0000