mkvalidrcptto

The most recent version of the script is dated 2012-09-16. The changelog file contains a list of what changes were made, each time I made a change.

The most common questions I receive about the validrcptto.cdb patch involve how to create the validrcptto.cdb file in the first place, or how to use the same file on multiple servers. The mkvalidrcptto script can be a major part of the answer to both questions. This web page will show how I'm using mkvalidrcptto on my own server.

It should be noted that you are not required to use mkvalidrcptto in order to benefit from the validrcptto.cdb patch- any mechanism which produces a list of all valid email addresses on your system will work, even something as simple as manually editing a text file with one email address on each line. As long as you correctly turn this text file into a cdb file, it will work.

Installing the cdb package

Before you install the script, there are a few other packages which need to be installed on the system. The first is djb's cdb library and tools, which contains the cdbmake-12 program, which converts a text file into a cdb file. This package should be installed using the directions on djb's web site. A quick walk-through is shown here:

If you are using a system with glibc version 2.3 or later, the original djb source code will cause an error in the file error.h, because the newer versions of glibc implement the errno variable differently than how it has been done in the past. (errno is a global variable which libc uses to tell the caller whether the last I/O function call was successful or not, and if there was an error, it will contain a number identifying the error.)

There is a simple fix, which can actually be done on all systems whether it's necessary or not, without causing any problems. If you know (or suspect) that you need this fix, or if you have tried to compile the code and gotten an error about error.h, or if you just want to do it anyway, here's the fix:

Open the file error.h with a text editor.

$ vi error.h

Find this line:

#ifndef ERROR_H
#define ERROR_H

extern int errno;

extern int error_intr;
extern int error_nomem;

Comment the line out, and add this new line below it:

#ifndef ERROR_H
#define ERROR_H

/* extern int errno; */ #include <errno.h>

extern int error_intr;
extern int error_nomem;

You can also make this change with a one-line sed command, useful for scripts:

$ sed -i '/int errno/{s/^/\/* /;s/$/ *\//;G;s/$/#include <errno.h>/;}' error.h

Then finish compiling and installing the package as normal. Note that this same problem can occur, and the same procedure will fix it, in most of djb's programs, including qmail, ucspi-tcp, daemontools, and djbdns.

Installing the CDB_File module

The mkvalidrcptto script reads several cdb files in order to do its job, which means that you need to install the CDB_File perl module, available through CPAN, the Comprehensive Perl Archive Network, which is an archive of Perl modules which are not included with Perl itself, but which others have decided to share in the hope that they will prove useful.

$ su -
Password: you should not see the root password as you type it

# perl -MCPAN -e shell

If you have never run the CPAN shell before, you will be asked to configure the module first. This will involve setting the path to the various utilities that CPAN uses for downloading and building modules, as well as selecting one or more CPAN "mirrors". Mirrors are web and FTP sites around the world which have copies of all of the CPAN modules. You should go through this process if it asks you to- for most of the questions you can just hit ENTER and be okay, and for mirrors you should try to choose at least three of them, preferably ones which are in your part of the world.

cpan> install CDB_File
...
cpan> exit
# exit

After installing any Perl modules, it's a good idea to run the pfix script, available on the code section of my normal (non-qmail) web site. Some Perl modules don't correctly set the permissions on their files when you install them- the pfix script resets the permissions on all directories in Perl's default library path so that everything is owned by root and is world-readable but only writable to root.

Installing mkvalidrcptto

Believe it or not, this is actually the easiest part of the whole process. All you need to do is download it into a directory which is in your PATH and set the permissions.

One thing that wouldn't hurt is to make sure that your installation of perl is happy with the script and can find the modules. You can do this by running this command as a non-root user:

You should then run it once as root and make sure the output makes sense for your system. The output should be a list of every valid email address on your system, one on each line.

# mkvalidrcptto
alan@domain1.xyz
jms1@domain1.xyz
postmaster@domain1.xyz
thomas@domain1.xyz
@domain2.xyz
postmaster@domain3.xyz
user@domain3.xyz
...

Scripting for one system

The mkvalidrcptto script itself just reads the information it needs from your system and prints a list of email addresses. The other half of the equation is turning that list of email addresses into a validrcptto.cdb file, so that qmail-smtpd can use it.

The original versions of mkvalidrcptto worked by simply printing the list of email addresses, and relied on another program called cdbmake-12 to produce the actual validrcptto.cdb file. However, as of 2007-06-06, mkvalidrcptto can write the validrcptto.cdb file by itself. This makes the scripting much simpler.

In addition, as of 2007-06-18, mkvalidrcptto has some extra code which detects whether or not the data has changed since the last time the file was written, and only writes a new file if the data has changed. Not only does this make scripting simpler, but the timestamp on the validrcptto.cdb file correctly tells the last time it was updated, just as the original script here did.

The idea situation would be to have vpopmail run a certain command whenever it changes something. When I originally wrote this page, vpopmail did not have that kind of hook, however I have since written a patch for vpopmail called the ONCHANGE patch, which is officially part of vpopmail as of vpopmail version 5.4.15.

It is possible to write a script like the one below, which can run as a cron job, in response to an ONCHANGE event, or as part of a general qmail-updater service, to rebuild the validrcptto.cdb file.

#!/bin/sh PATH=/usr/bin:/bin:/usr/local/bin:/var/qmail/bin umask 022 mkvalidrcptto -c /var/qmail/control/validrcptto.cdb

Automating the process

I have a script called cron.qmail which I was using for several years to keep other files updated automatically, and I added this logic to it after I wrote the validrcptto.cdb patch.

However, a few of my clients' servers have issues with crond either not recognizing when a crontab file has changed, or just plain stopping- which means that in some cases, these machines were not running this cron job when they were supposed to.

So now, instead of using a cron job, I use a daemontools service called qmail-updater to update these files on a regular basis. The web page listed here has a link to download the script, along with instructions on how to turn it into a daemontools service.

Scripting for multiple systems

One of the biggest advantages of having validrcptto.cdb as a stand-alone file is that it can be copied to other machines. The most common scenario is to have one machine handling the mailboxes, and one or more other machines (which I call "mailhubs") which handle incoming email from the outside world. These mailhubs' only job is to do RBL, virus, and/or spam checking on the incoming email, and only accept messages which are "clean". If a mailhub has a validrcptto.cdb file available, it can check the RCPT TO arguments to make sure the email addresses actually exist, without having to bother the mailbox machine for every single RCPT TO command from every client on every mailhub.

The mailhub page explains this scenario in more detail. It also includes scripts to handle automatically replicating certain qmail control files from the mailbox machine to the mailhubs- specifically the rcpthosts and morercpthosts.cdb files, the smtproutes file, and the validrcptto.cdb file.

Troubleshooting

The number one problem people seem to be having with this script is not understanding how it interacts with vpopmail's "catch all" feature. Here's the deal: if a vpopmail domain's catch-all is set to "bounce", this script will list the individual mailboxes. Otherwise, it will simply generate an "@domain" line.

Most people ask why it doesn't list the mailboxes when the domain is set to "catch-all deleted". This is because "catch-all deleted" means something totally different- it means that the messages should be accepted by the SMTP server, and vpopmail will delete them without actually delivering them to a mailbox. Some people prefer doing this- if a message is refused, some servers will try the message over and over for up to a week.

Accepting messages with non-existent recipients will keep the remote servers from trying the same bogus messages over and over... and if you know you're only going to delete them without storing them anywhere, you end up spending less CPU time to accept and not deliver the message than you would if you were to reject the message once an hour for a week.

Basically, if you want to reject mail for non-existent mailboxes in a vpopmail domain, you need to set the domain to "catch-all bounce".

2007-06-02 People keep asking about why I wrote the script to do this, and why "catch-all delete" can't also mean to refuse messages. There are some cases where the owner of a domain might not want to refuse messages, but they also don't want those messages to end up in a mailbox- in other words, they want "delete" to work the way it normally does. That's why vpopmail was written with "bounce" and "delete" as separate options.

However, if you have an existing server which previously did not have the validrcptto.cdb functionality available and are adding it, and that server has enough domains with "catch-all delete" that you don't want to manually change them by hand, you have two options:

Thanks to Joey DiJulio on the qmailrocks list for suggesting the change- I can see where it might be a good idea for some people, but I don't think it's a good enough idea to justify removing the "delete" functionality from vpopmail entirely.

Also thanks to Patrick "marlowe" MacDonald for suggesting a warning message about domains which don't reject mail to non-existent mailboxes.

2007-08-24 If you're doing something "different" with a vpopmail domain, such as using maildrop to control the deliveries, mkvalidrcptto doesn't know how to interpret any changes you make to the .qmail-default file- the only thing it knows is to look for a line containing the word "vdelivermail", with the word "bounce-no-mailbox" after it on the same line. If you want this script to generate a list of mailboxes instead of a single "@domain" line for the domain, then instead of removing the original "|vdelivermail ... bounce-no-mailbox" line, you should comment it out (i.e. put a "#" character at the beginning of the line.) Thanks to Bob Walter for pointing this out.

File:	mkvalidrcptto
Size:	24,057 bytes
Date:	2012-09-16 22:49:52 +0000
MD5:	`b0538e451fd9516353a776d81b2c5c9d`
SHA-1:	`2bd2e84d47d5585044221352cc80c57c63fdff81`
RIPEMD-160:	`13eaa9c6f280b03a64453b4c0d8dd96b59cd5326`
PGP Signature:	mkvalidrcptto.asc