http:// qmail.jms1.net / vpopmail /

vpopmail

I have been using vpopmail to manage my mailboxes for several years. For the most part I have been happy with it, however I have found a few things that I wish it could do. However, these things were not major enough for me to devote any time to writing patches for the things I needed, since I had other work-arounds available for the problems I was trying to solve.

One of the programs that Inter7 wrote as part of the vpopmail package is called vpopmaild. I had read about it for several months, and was interested in using it- but it was only available through CVS, not available with a released version of vpopmail- until version 5.4.15 was released.

When vpopmail version 5.4.15 was released, I downloaded it and started playing with vpopmaild. My first project was to get vpopmaild running as a service under daemontools. Once this was done, I played with vpopmaild and started finding little problems with it- minor things, like if the first command a client entered was not a successful login, it would hang up on you.


The "onchange" patch

I also write patches for qmail. One of the qmail patches I have written is the validrcptto.cdb patch, which modifies qmail-smtpd to verify the full email address entered in a RCPT command against a file called validrcptto.cdb, which contains every valid email address on the server. If somebody tries to send a message to an email address which isn't listed in this file, qmail-smtpd will refuse the RCPT command. The advantage here is that the cdb file can be copied to other servers (in a mailhub scenario, for example) and these other servers can correctly accept or reject recipient addresses without having to communicate with any other system, whether it's vpopmail, mysql, LDAP, or anything.

The problem with this, of course, is knowing when to rebuild the validrcptto.cdb file. What I have been doing is running a daemontools service which periodically builds a new list and compares it to the current list. If the list has changed, it finishes the process and turns the new list into a new validrcptto.cdb file.

There are two problems with this. (1) The computer wastes a lot of time and CPU by building the list of addresses over and over when it's not needed, and (2) when it IS needed (when a mailbox is added, for example) there is a delay before the new validrcptto.cdb file is actually built.

The ideal solution, since vpopmail is managing the mailboxes, would be for vpopmail to somehow trigger the creation of the new validrcptto.cdb file whenever something changes.

Vpopmail currently has no such "trigger" functionality- so I decided to write a patch which gives it this functionality.

The first version of this patch simply provided a mechanism for an external command (called ~vpopmail/etc/onchange) to be executed whenever a change was made. For my purposes, this was all I needed.

However, Robin Bowes asked if it would be possible for the external script to find out what change had actually been made. I have added the code so that when the onchange script is run, it will receive two items on its command line- the vpopmail function which caused the change, and the target (mailbox or domain) which was changed.

(The list has changed, see the updated list below.)

2006-04-15 We found a bug where the "add domain" operation was causing the notifications to be sent in the wrong order- first "add_domain", then "mod_user" when the postmaster mailbox's quota was being set, then "add_user" for the postmaster mailbox itself. I have moved the "add_user" notification so that it now happens in the correct order.

2006-04-16 David Treau reported a problem compiling vpopmail with this patch. He was compiling vpopmail to use mysql (which I do not do on my own server) and found a typo and two bugs (wrong variable names) in vmysql.c, which I didn't catch as part of my own testing process (again, because I don't use vpopmail with mysql.) After fixing the typos, I checked the other files which are changed by this patch, and found and fixed the same bugs in vpgsql.c as well.

I have updated the patch yet again. If you have a version earlier than "4" (the version which was committed to the vpopmail CVS tree on 2006-04-15 was version "2") you should be aware that it will not work with mysql or pgsql.

2006-04-16 Rick Widmer brought up an interesting point- the notifications for "del_user" and "del_domain" should happen BEFORE anything is deleted, so that the handler can run a backup of what's about to disappear if needed. Version 5 includes this change.

2006-04-16 I have updated the patch so that the vadddomain() and vadduser() functions, which call other functions from within themselves, can "mask" the notifications that those child calls would normally send. I have also changed the code so that it depends on one or two #define statements being present in config.h in order to enable the "onchange" functionality.

I don't know how to add this functionality to configure, so for now I have been "faking it" by adding these two lines to the end of config.h after running ./configure and before running make:

#define ONCHANGE_SCRIPT
#define ONCHANGE_ONCE

2006-04-17 There was a discussion about standardizing the names of the commands which are sent to the notification script. I have updated the patch (to version 7) to use the names suggested by Rick. This is the new list of the messages sent:

Command Arg1 Arg2
add_user user@domain  
del_user
mod_user
add_domain domain  
del_domain
add_alias_domain  alias_domain  real_domain 
insert_alias alias@domain alias_line
remove_alias
del_alias
del_all_alias domain

vpopmail-onchange.txt is a list of the functions which generate "onchange" notifications, along with the commands and arguments they generate. If anybody finds any missing or duplicate calls, please let me know (or even better, join the vpopmail-devel mailing list and let everybody know.)

File: vpopmail-onchange.txt
Size: 1,534 bytes
MD5: 9851916f5bd99d0a303271b37f27ef25
SHA-1: 32eb28364c7f92f4aa5f41cfa0df9da39e6b7518
RIPEMD-160: 3e311cb09dd1b22c076d34192c16dbe1d3cc05fa
PGP Signature: vpopmail-onchange.txt.asc

2006-06-28 People are asking for a concrete example of how the "onchange" functionality works. My server is running a qmail-updater service, which watches a named pipe called /tmp/update-qmail. Whenever any process writes data to (or just "touches") that named pipe, the service launches a script which rebuilds my validrcptto.cdb and auth.cdb files.

Why go to the trouble of having a separate service to do this? Two reasons- (1) the process requesting the upate (i.e. vpopmail) doesn't have to wait for it to finish, and (2) the service can run the scripts as root without it being necessary for the process requesting the update to also have root privileges.

Here is the ~vpopmail/etc/onchange script I am using on my system, which "triggers" the qmail-updater service. The qmail-updater service page has the other scripts needed to get the service running.

2006-12-31 This patch is included in vpopmail 5.4.18, and is no longer available for download here. If you need this functionality, you should use the newer version of vpopmail.

2007-01-09 Joshua Megerman found a rather serious bug in the onchange patch... basically if the "onchange" script exists but cannot be executed (if it's not in a format that the kernel understands how to run) then the child process did not exit cleanly. This patch fixes the problem.

File: vpopmail-5.4.18-onchange.fix.patch
Size: 2,050 bytes
MD5: b1eda6b07649b9f59d4f5aa19b51abeb
SHA-1: d49c84404ef1f7769ab70a2067a8a3ec49afe6ec
RIPEMD-160: e91a33972618a8cb24e6a0d3e281002f3511c7ec
PGP Signature: vpopmail-5.4.18-onchange.fix.patch.asc

2007-02-07 Adam Sloan found a bug in the onchange patch as well... turns out if you delete a domain which is actually an alias rather than a physical domain, the onchange action is never triggered. This bug happened because the lousy formatting of vpopmail.c (one or two characters per tab, and that not consistently applied, grrr...) prevented me from seeing the real structure of the code I was working on when I wrote the original onchange patch. The patch below fixes this problem.

This file has changed since I originally put it here... the original idea was to have a single patch which fixed both (or all) of the problems, however I realized that many people may be using the first patch (above) already and just need to add this fix. The file here is JUST the fix for the second problem. If you happen to be the ONE PERSON who (according to the web server's logs) downloaded a copy of the original patch, I apologize for the confusion.

File: vpopmail-5.4.18-onchange.fix2.patch
Size: 1,073 bytes
MD5: 9bd135bef406b8c93040e06989f2dc10
SHA-1: a3ffcf966da3807ab6d0f3bc383808509e9e7bde
RIPEMD-160: defd6c890bf10cf145de9132756f8a29f66c0a3b
PGP Signature: vpopmail-5.4.18-onchange.fix2.patch.asc

2007-07-14 I have just upgraded to vpopmail 5.4.19, and found that it does not have the bug fixes contained in the two patches above. I have made a single patch file (below) which fixes these issues, and will be sending it to the other vpopmail developers to be applied to the next version of vpopmail.

File: vpopmail-5.4.19-onchange-fix.patch
Size: 2,429 bytes
MD5: 008efcff599cfa208d06080b078f4397
SHA-1: 86e5ec4892aee5502b3fdeee9d56e9f40d157c1d
RIPEMD-160: 647894df92c811e4a5263442a3af74434f7a973c
PGP Signature: vpopmail-5.4.19-onchange-fix.patch.asc

And because people have asked about it, here is a sample onchange script, designed to be used with a qmail-updater service (which would run actions which may need to be done when something changes- building a validrcptto.cdb file, running qmail-newu or qmail-newmrh, and so forth.

File: onchange
Size: 1,421 bytes
MD5: 84769178e6fdd9b1691bdb3c3ff6b976
SHA-1: 1ab93be9511ba932a2b91441ac3b38999d643f04
RIPEMD-160: c05951205477ce30a9a00c1c7cbf11277c8b7315
PGP Signature: onchange.asc

The "onchange" patch for qmailadmin

2007-07-12 I received a report from Adam Cantwell that qmailadmin doesn't call the onchange function when adding or removing a mailing list. This makes perfect sense, since qmailadmin doesn't use the functions in libvpopmail.a to add or remove mailing lists. I didn't think about this when I wrote the onchange patch because I was concentrating on vpopmail, and qmailadmin is a separate program.

2008-09-24 I received another report from Kai Gallasch that qmailadmin doesn't call the onchange function when changing a domain's default delivery status (i.e. from "catch-all deleted" to "catch-all bounce", or setting a specific mailbox or remote address to receive all of the domain's main.

2008-12-21 The qmailadmin guys have released a newer version, 1.2.12... it's still listed as a "development" release, but it does seem to work fairly well. One of the changes they made was that the .qmail-default file for a default alias now contains an email address, even if the default delivery target is a maildir. I personally don't like the idea, because it forces the message back through the queue, slows down the delivery process, and adds extra headers to the message, but nobody asked me about it (and to be honest, I haven't been paying much attention to the qmailadmin mailing list either.) This change kept the existing 1.2.11v2 patch from working, so I have fixed it.

While making the correction, I also noticed that when qmailadmin created an autoresponder (they call it a "mail robot") the ONCHANGE script wasn't being called. I have fixed this as well, and have updated the patch to "version 3" for both 1.2.11 and 1.2.12.

The patches below will make qmailadmin call the onchange function when needed, IF it's compiled with a version of vpopmail which calls the onchange function. It sends the following command names and arguments to the "onchange" script:

Command Arg1 Arg2
addmailinglist listname@domain  
delmailinglist
setdefault (directory name)  
delete
bounce-no-mailbox
(remote address)
addautorespond name@domain  

This patch replaces the earlier versions which were here (which didn't contain all of the bug fixes from the current version.) Anybody using an older version should upgrade.

For qmailadmin-1.2.11
File: qmailadmin-1.2.11-onchange.3.patch
Size: 2,694 bytes
Date: 2008-12-21 21:12:59 +0000
MD5: 9a1dafd1881d9bba9a8b40a4c444d2a5
SHA-1: 7968372043437ff8f7f2e87b418226be832a21ce
RIPEMD-160: 8dd36b1d384e2c9e4d69a009b889e9c77561a01a
PGP Signature: qmailadmin-1.2.11-onchange.3.patch.asc
For qmailadmin-1.2.12
File: qmailadmin-1.2.12-onchange.3.patch
Size: 1,725 bytes
Date: 2008-12-21 21:05:26 +0000
MD5: 626b42f0485c0d9fbccd5ed085f10cbb
SHA-1: 2962e8d4412fa04db6db178e221aade6123f706c
RIPEMD-160: e1f976214b04e591385d0383095f00cf53c33b21
PGP Signature: qmailadmin-1.2.12-onchange.3.patch.asc

The vpopmaild "cslogin" patch

While reading a message from Rick Widmer on the vchkpw mailing list, I noticed that the semantics of the login command would prevent users whose passwords contained spaces from being able to authenticate. The original vpopmaild protocol had a login user@domain password compact option which would send less data back to the client in case of a successful login.

I asked about how this would handle passwords with spaces, and provided a suggestion in case the answer was "it won't handle passwords with spaces", which of course ended up being the answer. Rick agreed that the problem existed and I wasn't seeing things, and suggested a slightly different way to solve it (different command names). I took his suggestion and have written a patch which hopefully solves the problem.

With the patch, instead of using login user@domain password compact, you would use clogin user@domain password. This way if the password happens to contain a space, that space is not parsed as a separator for the word "compact".

While I was in there, I also added a slogin user@domain password command (s meaning "silent"), which sends back no information other than the standard "+OK " response, which tells the client that the login attempt was succesful.

I also found it annoying that the help command (1) showed every single command to everybody, which can be confusing if you are using vpopmaild via manual telnet, and (2) caused vpopmaild to disconnect if you entered the command before a successful login command.

Since I was already modifying the code, I went ahead and changed the help command so that it only shows the commands which are actually available to the client, and I also fixed the logic surrounding the login process so that the help command (or a login command which is not successful) does not disconnect the client- it allows the client to try again. I also added a counter for invalid logins, with a limit of 3 attempts to prevent an attacker from conducting a dictionary attack.

2006-04-04 It turns out that the add_user command within vpopmaild was expecting a human-readable name (a "gecos" name) as input after the password, even though this was not documented anywhere- not even in the "help" text. Even more surprising was the fact that I didn't see this while making the cslogin patch.

I have change the code so that the add_user command no longer looks for a gecos name. Any programs which are expecting the previous behaviour will be surprised to find that their "real name" ends up becoming part of the password assigned to the user. The gecos name assigned to a newly created mailbox will now be the same as the mailbox name.

If you need to set the gecos name for a new mailbox, use the mod_user command's comment sub-command. This is the only way to set the gecos name for a mailbox.

2006-04-16 Jeffrey Lim emailed me to point out that the clogin command does more than just modify the format of the information you see when you log into the service- it also modifies the output of the user_info command. This makes sense to me. It also turns out that using slogin to log in will modify the user_info output in the same manner. This also makes sense, although I will admit that I didn't consciously design it to work like this... just call it a happy coincidence.

So if your client needs to have the "uidflags" bitmap explained line by line, you need to use the login command to log into the service instead of using clogin or slogin.

2006-12-31 This patch is included in vpopmail 5.4.18, and is no longer available for download here. If you need this functionality, you should use the newer version of vpopmail.


Bug fix for vdominfo

2006-04-07 I noticed that the vdominfo -n command lists the wrong output in cases where one domain name is an alias for another. For example...

# vadddomain domain.one password
# vaddaliasdomain domain.one domain.two
# vdominfo -n
domain.one

domain.one
alias of: domain.one

As you can see, the output doesn't make much sense. When printing the second domain name (the one which is an alias), it prints the real domain name instead of the alias domain name.

Looking at the code, it looks like two different people were working on the program, and made changes to try and fix an issue in the past, but fixed it in the wrong place.

I have written a patch which fixes the problem, and makes the output look correct:

# vdominfo -n
domain.one

domain.two (alias of domain.one)

2006-05-07 This patch is included in vpopmail 5.4.16, and is no longer available for download here. If you need this functionality, you should use the newer version of vpopmail.


Bug fix for vpgsql.c

While test-compiling the "onchange" patch, I came across a series of errors in vpgsql.c. It looks like, when support was added to vmysql.c to differentiate between a read-write server and a read-only server, somebody added the same will_update parameter to the vauth_open() function within vpgsql.c, but didn't update everything else within vpgsql.c to actually pass that parameter.

I have written a patch which updates all of the other calls within vpgsql.c so that they pass a parameter to vauth_open(), even though vauth_open() doesn't currently do anything with that parameter (other than log its value, if you have debugging turned on.)

2006-05-07 This patch is included in vpopmail 5.4.16, and is no longer available for download here. If you need this functionality, you should use the newer version of vpopmail.


Bug fix for vuserinfo

A user on the qmailrocks mailing list found a discrepancy in how the vuserinfo command works. If you call it with a specific mailbox address, it shows the "flags" correctly, however if you call it with -D and a domain name, the individual entries shown in the resulting list show the "flags" as if they were zero.

I have tracked this back to a problem in how vauth_getall() was written in most of the authentication modules. Basically, there's a function called vlimits_setflags() which sets the pw_flags field by combining the entry's pw_uid value with any domain limits. The vauth_getpw() function (which retrieves the information for a single mailbox) correctly calls this function before returning, however the vauth_getall() function doesn't call vlimits_setflags() in any of the modules.

The patch below corrects the issue by adding a call to vlimits_setflags() in the vauth_getall() function of each of the four affected modules (the vldap.c module is not affected, since its vauth_getall() function calls vauth_getpw() to get the individual mailbox data.)

This patch is included in vpopmail version 5.4.20, and is no longer available for download here.


Maximum password length

One of the things which has always bothered me about vpopmail is that, by default, it limits you to using passwords which were no longer than 16 characters long. I've always worked around this by editing vpopmail.h before compiling the program and changing the values of MAX_PW_PASS and MAX_PW_CLEAR_PASSWD to allow longer passwords.

I suggested this on the vpopmail mailing list as a permanent thing, and Jeremy Kister replied with a related suggestion, that if you compile vpopmail with the "--disable-md5-passwords" option, it should limit the passwords to eight characters, in order to prevent people from getting a false sense of security by thinking "password#*!%@^::" is a secure password. It's not, because only "password", the first eight bytes, are used, and the remaining characters are ignored.

This patch changes the maximum password length to 128 characters, and re-defines the limit as 8 if "--diable-md5-passwords" was used when configuring the software.

Note that the change from 128 characters to 8 characters only happens within vpopmail.c. For now I think this is safe, since that's the only file which uses the value of MAX_PW_CLEAR_PASSWD to begin with. However, it might make more sense to move the original definition from vpopmail.h to config.h.in, and have the ./configure script set it to either 8 or 128 before building the software, just in case a future version of vpopmail happens to use the value of MAX_PW_CLEAR_PASSWD in another source file.

2007-09-18 2128 -0400 The original here had a typo. Thanks to Joshua Megerman for pointing it out, and for eyeballing the patch to begin with.

This patch is included in vpopmail version 5.4.24, and is no longer available for download here.


Skeleton directory patch

This patch causes vpopmail to copy the contents of ~vpopmail/skel into every new mailbox it creates, similar to how most *nix systems copy the contents of /etc/skel in new users' home directories. This can be used to automatically create IMAP folders within new mailboxes.

While thinking about how this patch doesn't compile cleanly with a new version of vpopmail, I was reminded of the fact that everything this patch does (basically, copy a skeleton of files into the newly created mailbox) can be done from a script run by the ONCHANGE mechanism. And to me, that makes more sense than trying to keep updating a patch which only a handful of people seem to still be using.

Therefore, I don't intend to update this patch any more. If you need its functionality, write it into an ONCHANGE script. Here's a simple example if you need one. (Note that I haven't tested this myself, but the idea should be clear enough for any qualified system administrator to understand. In particular, the "cp" command may need to be adjusted to match where and how you store your skeleton.)

#!/bin/sh

PATH="/home/vpopmail/bin:/usr/bin:/bin"
SKEL="/home/vpopmail/skel"

if [ "${1:-}" = "add_user" ]
then
    DIR=`vuserinfo -d ${2:?No email address specified}`
    cp -r "$SKEL" "$DIR"
fi

Below is the previous contents of this web site, from when I was maintaining the patch, in case somebody needs it.

There was an existing patch floating around on the net, written by Brandon "mrjackson" Jackson (and "pushed" on the qmailrocks community by Bill Olson) which claimed to do basically the same thing. However, I looked at it and couldn't make heads or tails of some parts of the code. The patch did a lot more than just the skeleton function, and it wasn't very clearly written- I've been programming in C since 1985, and there were a few parts of the patch which were very difficult to follow. And as near as I could see, its recursive directory-copy function was calling itself with the parameters not modified at all, which suggests a potential endless loop of some kind.

I wrote a program several years ago, part of which involved copying an entire directory tree. So I pulled those functions out of my old code, slapped on the same "GPL v2+" license that vpopmail uses, and added a bare minimum of code to vpopmail.c in order to use those functions. Then I figured out the correct way to add a --enable-skeleton option to vpopmail's ./configure script, and rolled the whole thing into the patch you see below.

Because it changes the configure script's options, you should re-run "autoconf" to generate a new configure script. If you don't, the "make" step will do this for you, however if your system doesn't have exactly the same version of the "autoconf" tools, it will complain and possibly not work. The solution is to run this command after applying the patch, and before running your "./configure" command:

> autoreconf -i

I have done some testing of this on my own server, and it does seem to work, however I invite anybody and everybody to look through the code and let me know if you spot any potential problems with it.

This patch does not include any code from the old "mrjackson" patch. The recursive directory-copy function was lifted from one of my own programs from the 2002-2003 time frame, and the modifications to vpopmail.c I wrote specifically for this patch.

2007-10-05 I've updated the patch so it now copies symbolic links within the skeleton directory (it skipped them before.) Also removed some memory leaks from the code. Again, I'm fairly confident, but I'd feel better if other people were to bang on it for a while and let me know what their experience is with it- good, bad, or otherwise. Thanks to Geoff Garside for the suggestion about copying symlinks, it's not something I had even considered.

2007-12-15 Updated for vpopmail 5.4.26.

2007-12-28 I received an email from Rance Hall, saying that the code in copydir.c was treating things as symlinks which weren't actually symlinks. I had never heard of this before, and the code had been working for me for years, but after digging through the code I vaguely remember avoiding the S_ISLNK macro for some reason- it didn't exist on certain systems or something like that, so I avoided it then, and then forgot all about it because it was working.

I've updated the patch to use the S_ISLNK macro, and that seems to have fixed the problem.

2008-02-12 A few people have reported a bug, with symptoms usually involving files being created with the wrong ownerships when creating new domains and/or mailboxes. I tracked it back to the same issue with the S_ISLNK macro, but in vpopmail.c this time.

File: vpopmail-5.4.26-skel4.patch
Size: 20,114 bytes
Date: 2008-02-12 22:42:00 +0000
MD5: 88b8c135e31c1c73fd662a45e73d27c6
SHA-1: 6c2e1f4d10b8cb8688c15d4aa73788bdd441540a
RIPEMD-160: 8345cee5b0e91b7fd107580df4870cdc356b888e
PGP Signature: vpopmail-5.4.26-skel4.patch.asc