![[Valid HTML 4.01 Strict]](/valid-html401-blue.png){kind=small}
![[hacker emblem]](/hacker.png)
![[donate]](/donate.png)
Copyright
1995-2024
John M. Simpson KG4ZOW
(email)
(PGP key)Last updated 2008-08-08 03:43:25 +0000
One of the questions I've been asked is how to migrate from an existing sendmail system (using system accounts) to a qmail system (using vpopmail.)
djb's web site has a page whose title sounds like it may describe how to do this, but it actually covers a situation where you are installing qmail on the same machine where sendmail was formerly running, and it lists a bunch of what I consider to be "crutches" for people who want qmail to run but not to the fullest of its abilities. In particular, this page describes a system where incoming mail is still delivered to /var/spool/mail files, owned by system userid's...
It has been my experience that these types of migrations almost always involve the mail service being physically moved from one server to another. Usually this is done in conjunction with a hardware upgrade.
This page will show how I have, in the past, migrated existing sendmail systems to a new server running qmail. I'm writing a lot of this from memory, and I don't claim that my memory is perfect- I am hoping that people will try these directions and let me know whether they work, or if you run into problems, what problems you are having.
The first step is to create the mailboxes on the new server. In order to do this, on the old server we need to build a list of the userid's and their passwords. In order to make our job easier, the list that we build on the old server will be formatted as a series of commands which we can run through a shell on the new machine.
The sendmail-list script produces such a list. You may want to edit the script in order to match how your system is set up- as listed on the site, it will ignore users with a numeric uid lower than 500 or higher than 63999, or which have "!" or "*" in their encrypted password (which are usually suspended or inactive accounts.) The script is very simple, it shouldn't take a lot of work to customize it.
On the old server, run the script and save the output to a file.
(as root)
# ./sendmail-list domain.xyz > makeusers
# cat makeusers
vadduser -e '$1$yt5hE5rz$NXn6X6t29ci/8CYk4S44d.' jms1@domain.xyz
vadduser -e '$1$6io3bpHS$ALqBpZbsrXI2YVTQDeG0S.' tom@domain.xyz
vadduser -e '$1$aJJ9jz5W$Xj5hTvO3f.NXMZT4tniWy/' stev0@domain.xyz
vadduser -e '$1$iFc/1eP0$PCdFL/va4brzQUXSJLLcr/' frogger@domain.xyz
vadduser -e '$1$oBPqXQ4n$b8e2kZy9H5WBkLOc/YSNh.' kg4zow@domain.xyz
...
The next step is to copy this file to the new server, using a secure method such as scp, USB memory stick, or floppy disk. The encrypted data in this file cannot be directly reversed to find the real passwords, but they would enable an attacker with a fast computer and enough time to literally try every possible password until they find the one that works.
After copying the file to the new machine, the next step is to create the domain, and then create the mailboxes (using the file you just copied.)
First create the domain...
# vadddomain domain.xyz
Please enter password for postmaster:
enter password again:
... then create the mailboxes. The "-x" tells the shell to show the
commands as it executes them.
# sh -x makeusers
+ vadduser -e '$1$yt5hE5rz$NXn6X6t29ci/8CYk4S44d.' jms1@domain.xyz
+ vadduser -e '$1$6io3bpHS$ALqBpZbsrXI2YVTQDeG0S.' tom@domain.xyz
+ vadduser -e '$1$aJJ9jz5W$Xj5hTvO3f.NXMZT4tniWy/' stev0@domain.xyz
+ vadduser -e '$1$iFc/1eP0$PCdFL/va4brzQUXSJLLcr/' frogger@domain.xyz
+ vadduser -e '$1$oBPqXQ4n$b8e2kZy9H5WBkLOc/YSNh.' kg4zow@domain.xyz
...
At this point, if your users were to try and access an POP3 or IMAP server on the new machine, they would be able to log into their mailboxes- but they would see an empty box, rather than seeing whatever messages were on the old server.
This step actually copies the contents of the mailboxes (i.e. the INBOX folders) from the sendmail machine to the qmail machiine, and then converts the messages from sendmail's "mbox" format (where the entire mailbox is one file, with the messages stored one after the other) to qmail's "maildir" format (where each message is a separate file, stored in a directory structure which allows for safe access while incoming messages are being written.)
The first step is to copy the mbox files to the new server. This can be done with any file-copy mechanism, this example shows how to use "scp" (the "secure copy" program from the ssh package) to copy the files across.
On the NEW server, create a directory to hold the incoming files. This
should be owned by a normal user. This user should have no disk quota, and
there should be enough free space on the disk holding their home directory
to hold all of the mailbox files.
As a normal user...
% cd
% mkdir mailboxes
On the OLD server, log in as root and copy the mailbox files to the
directory on the new server. Assuming that the non-root user we just
used on the new server was jms1 ...
# cd /var/spool/mail
# scp * jms1@newserver:mailboxes/
...
...
...
Once the files are copied across, the last step is to convert the mbox files into individual files located within each vpopmail mailbox. There are several scripts on the net which do this- each one works a little bit differently from the others, and of course I have written a script called mbox2vpopmail which works specifically for the job at hand- it converts one mbox file to individual messages in a vpopmail mailbox.
The best way to use this script when converting an entire set of users is, of course, to write a quick script. (If you're starting to get the idea that running a server involves a lot of script-writing, you're right.) Here's an example of what such a script might look like- note that this is only an example and is not guaranteed to work.
#!/bin/sh
cd ~jms1/mailboxes
for user in *
do
mbox2vpopmail $user@domain.xyz < $user | tee -a ~/output
done
I'm using "tee" here to catch all of the output. tee is one of those little programs which has been with *nix for years and years, but most people either don't know it's there, or they just never use it... tee works by reading standard in and sending the data to standard out (just like "cat") but it also writes a copy of the data to a file. In this case, it lets you see the results on the screen when you run the script, and also gives you a file containing the results that you can then inspect if the whole thing scrolls by too quickly for you to read.
My mbox2vpopmail script prints the word ERROR in capital letters whenever it encounters an error. After running your script you should search the output for "ERROR", and if it finds any errors, take care of them.
The biggest issues that people seem to run into with a migration like this seem to revolve around the users' passwords- where to find them and how to copy them are the important ones, but people often seem to wonder about how to "decrypt" the passwords as well.
Historically, users' passwords were stored in a file called /etc/passwd, and some systems still store them there. However, this file must be world-readable for the system to work correctly, because programs like "ls" use it to convert numeric uid's to the userids for directory listings.
There are two problems with using /etc/passwd: (1) the encrypted passwords are readable to anybody on the machine, and (2) the encryption used is rather weak. People started writing programs to "crack" these passwords, and then abuse other peoples' accounts.
There are two algorithms which are commonly used to encrypt passwords in a *nix password database- the original unix crypt() function implements a weakened form of DES encryption (weakened in that it ignores everything past the first eight bytes of the password), and MD5 hashes (which, on most systems, ignores everything past the first 127 bytes of the password.)
The first and easiest solution was to simply move the encrypted passwords to a file which was not readable to normal users. This file is normally called /etc/shadow on most systems, and is only readable to the root user.
The second solution was to change to a much stronger form of encryption. Instead of using a single-round DES on the first eight bytes of the password, they started using the MD5 hashing algorithm and storing an MD5 hash instead of an actual encrypted version of the password. This has the advantage of being able to potentially work with a password of any length, the longer the password the harder it is to crack, and the implementation used on most systems has a limit of 127 bytes for a password.
Another problem, not as critical as a security issue but still a major issue on systems with thousands of users, is the fact that the /etc/passwd and /etc/shadow files are text files, and whenever a program needs information from the files, they have to read the entire file until they find the line they are looking for. On a system with thousands of users, this tends to slow things down, so some of the BSD systems started storing this information in a /etc/master_passwd.db file instead.
One of the nice things about writing my scripts in perl is that perl's built-in "getpwent()" function already knows how to deal with whatever kind of file(s) may be in use on the system- I don't need to worry about, or even know, what files are involved.