Setting up tinydns replication using SSH

This page explains how my main DNS server (running tinydns, of course) is replicating its data to the slave DNS servers (also running tinydns) using SSH.

The hardest part of the process for most people is understanding how the process works. It involves a bit of familiarity with SSH. I will start with a high-level overview of the process, and then explain each part in more detail.

Note that djb has his own directions for how to do what this page describes. I encourage you to read his directions, either before or after reading mine, and proceed with whichever option you prefer.

Obviously I prefer to do things my way, for the following reasons:

• My method involves only making one connection to each remote server instead of two connections (one to copy the data file and one to activate it.)

• djb's method requires that the slave server allow full root logins with a standard shell. This means that anybody who either knows your root password, or gets a copy of the key file we will be generating, will be able to remotely get a root shell on the slave server. My method uses a non-root user, and only allows the key to have access to do one thing- feed new DNS data. The potential damage is limited to the DNS data themselves, rather than the entire system.

• My directions give a more complete description of how to set up the whole system, including the SSH portions.

Again, I encourage you to examine djb's instructions, and if you feel more comfortable with them because they're from a "more official source", then by all means use those instructions. His method does work, it just opens up a bit more of a security hole than I'm comfortable with.

Overview

I maintain all of my DNS data on one primary server. However, rather than using "add-*" commands, or editing a single text file called "data", I edit multiple files called "zones.something", where {something} is a description of which person's data is in that particular file. For example, my own domains are stored in a "zones.jms1" file, while other clients' DNS data are stored in their own files.

I have written a perl script which reads all of these files and builds one combined "data" file from them, and in the process converts all "Z" and "." records to "Z" records with explicit SOA serial numbers (as well as "&" records, and "+" records if the "." record has an IP address.) This way, when the resulting data file is copied to the other servers, all of the servers will be serving data with the same serial numbers.

The script is run using the same "make" command that tinydns users are already familiar with. I have obviously modified the Makefile to run my script, and to call a second script to actually push the new data to the child servers.

The mechanism for sending the data to the child servers is done using SSH, authenticated with a key. The main server has the private key, and each child server has a line in root's ".ssh/authorized_keys" file which not only allows that key to access the root account, but forces a certain script to be run instead of a normal shell.

The script on the child server writes the data from its standard input (which is the DNS data written by the main server) to a temporary file, renames the temporary file to "data", and runs "make" to build the data.cdb file. While this is happening, the output side of the SSH tunnel is still being used, so that the output of everything happening on the slave machine is visible on the server. The script also allows up to ten recent copies of the "data" file to be kept on the server, while deleting anything older than the ten most recent versions.

Setting up the master server

Setting up tinydns

The process starts by setting up a tinydns server. This should be done in accordance with the first part of djb's instructions. I am including a sample walk-through of what the process looks like. Note that instead of using "Gtinydns" and "Gdnslog" as the userids, I use "dnsrun" and "dnslog" instead. This example assumes that the IP address of your ethernet interface is 1.8.7.200.

# tinydns-conf dnsrun dnslog /etc/tinydns 1.8.7.200
# ln -s /etc/tinydns /service
Wait about five seconds
# svstat /service/tinydns
/service/tinydns: up (pid 18905) 3 seconds

You should see the service "up", with a run-time of two or more seconds. If the service is not "up", figure out what the problem is before continuing. If the service is "up" but the count is less than two seconds, wait a few seconds and try the "svstat" command again. If it stays below two seconds, figure out what the problem is before continuing.

Remove the add-* scripts

The next step is to remove the "add-*" scripts that tinydns-conf creates in the service directory. This is not strictly required, but the scripts will be fairly useless when we switch away from maintaining the "data" file directly and start using the "zones.*" files instead.

# cd /service/tinydns/root
# rm add-*

Install the make-data program

Now we need to install the make-data program, which joins the "zones.*" files together to produce the "data" file.

2008-05-09 Added some error checking to the "Z" lines. Thanks to Bruce McAlister for pointing out the problem (even though the bug isn't what he thought it was.) Also cleaned up the logic for "." lines while I was in there.

Even if you plan to maintain all of your DNS data in one file, this program is still necessary since the script also fixes all "Z" and "." lines so that they have explicit SOA serial numbers. This is critical- if you are running multiple servers, the serial numbers MUST be identical on all of your servers.

# cd /service/tinydns/root
# wget http://qmail.jms1.net/djbdns/make-data
# chmod 755 make-data

Edit the Makefile

The next step is to edit the Makefile. The changes will do the following:

• Build "data" whenever any of the "zones.*" files change, or the header file (which becomes the header of the resulting data file, to remind people not to edit the data file directly) changes.

• Build "data.cdb", AND push the new "data" file to the remote servers, whenever "data" has been changed.

• Include an option to force the changes to be made, in case you change something that the Makefile can't detect (i.e. you delete one of the "zones.*" files, or you add a file with a timestamp older than "data".)

Rather than walking through each change individually, I have a modified Makefile available for download.

# cd /service/tinydns/root
# wget -O Makefile http://qmail.jms1.net/djbdns/replication-Makefile
# wget http://qmail.jms1.net/djbdns/header
# chmod 644 Makefile header

Install the push-changes program

Looking at the new "Makefile", you will see that it calls a program called "push-changes" to push the new data out to the child nameservers. We also need to install this program as well.

# cd /service/tinydns/root
# wget http://qmail.jms1.net/djbdns/push-changes
# chmod 755 push-changes

You need to edit this script to set the IP addresses of your child nameservers. The comments in the file itself will show you what to change. It should be obvious that this step is required, but just in case, the script as downloaded from this web site has two nonsense IP addresses in there, so that if somebody tries to run it without editing it, it will fail miserably.

Handling existing data

If you already have an existing tinydns server and are converting it to become a master server, you will need to rename your existing "data" file to "zones.something". You may wish to break it up into several files, or you may wish to keep it as one file- but every file containing tinydns data must have a filename starting with "zones." in order for this to work. I would recommend using the name "zones.original", but again you are free to use any "zones.*" name you want.

Testing

To test what you have so far, you can comment out the "./push-changes" line in the Makefile, and then run "make all". You should see it build a new data file from the header and the zones.* files, and then run tinydns-data to generate the "data.cdb" file.

Inspect the resulting data file and make sure it worked correctly. One thing you will notice is that if your "zones.*" files contain lines which begin with ".", they will be replaced in the data file with a "Z" line, a "&" line, and a "+" line for the nameserver (if the original "." line had an IP address.)

Remember to un-comment the "./push-changes" line in the Makefile, or your new data files will never be sent to your child nameserver(s).

Generating the SSH keys

Before we go any further, we need to generate an SSH key pair, which will allow the master server to log into the slave server without a password.

Note that if you have an existing SSH key pair that you already use to log into the server, you should not use it for this purpose. If you do, the limitations we will place on the key will prevent you from using it as a general purpose login key on the slave server.

Also note that this key pair should ONLY be used for updating your child DNS servers, and not for any other purpose.

When you create the key pair, it will ask you for a passphrase. Just hit ENTER when it asks. We are specifically NOT creating a passphrase on the private key, because if we did, the push-changes script would ask us to type that passphrase for every child nameserver. This would make the system more secure, but it's also a pain have to use. As long as we take care not to allow the private key file to fall into the wrong hands, we should be okay.

# cd /service/tinydns/root
# ssh-keygen -t dsa -b 1024 -f id_dsa_dnsupdate -C 'dns update key'
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): Just hit ENTER
Enter same passphrase again: Just hit ENTER again
Your identification has been saved in id_dsa_dnsupdate.
Your public key has been saved in id_dsa_dnsupdate.pub.
The key fingerprint is:
8c:04:2c:f4:a4:c4:8d:6f:eb:aa:f4:f9:76:13:1a:f0 dnsupdate key
Note: your key fingerprint will almost certainly be different from the example shown here.
# chmod 600 id_dsa_dnsupdate

This process created two files: "id_dsa_dnsupdate" is the private key file, and "id_dsa_dnsupdate.pub" is the public key file. The final chmod command makes sure that nobody except root would be able to read the private key file.

Setting up a slave server

Install tinydns

Obviously the slave server needs to be running tinydns. The setup procedure is the same as the one you followed to install tinydns on the master machine. However, don't worry about creating any "data" files right now. (And yes, it's okay to start the service, even if you don't have a "data.cdb" file ready. It will start, it just won't be able to answer any queries.)

Configure sshd to support logins with public keys

Before we can install anything on the slave server, we need to make sure that it has sshd configured to allow logins when the client uses a key to authenticate.

The configuration for sshd is done with a file called "sshd_config". This file is normally in the /etc/ssh/ directory, although on some systems you may find it in another location such as /etc or /usr/local/etc. Look through this file and see if you have a line starting with "PubKeyAuthentication". If so, make sure it has the word "yes" at the end, like this:

PubKeyAuthentication yes

If it's not there, or if it is there and it says "no", you need to change it to say "yes". (Technically, if it's not there you're still okay, since the default is "yes". However, I like to explicitly spell out the options that I'm relying on, just in case.)

If you make any changes to the sshd_config file, you need to stop and restart your sshd process to make the changes take effect. This process depends on how your system handles stopping and starting services- for my own system (CentOS, which is a clone of RedHat Enterprise Linux) this is done with the following command:

# /etc/init.d/sshd restart
Stopping sshd:           [ OK ]
Starting sshd:           [ OK ]

Install the delbut script

The update-dns script keeps copies of the last ten versions of the data file in the /service/tinydns/root directory, as a sort of backup. It uses another script I have written, called delbut, to delete all but the ten most recent versions of the file. Since this script is useful for more than just this one application, I normally install it in /usr/local/bin on my server and on my clients' servers.

You can read more about the delbut script on the random code page of my normal (non-qmail) web site.

# cd /usr/local/bin
# wget http://www.jms1.net/code/delbut
# chmod 755 delbut

Create the DNS slave user(s)

When the master server sends the updated DNS data to the slave server, it needs to use a non-root user on the slave server. This is to limit the amount of damage that an attacker would be able to do if they were to get a copy of the SSH private key file we created above.

If the slave will only be accepting incoming data from a single master, you can simply create a single user and be done with it. Create this user as you would any other user- and yes, this means it needs a real shell. If you give it something like "/bin/false" as a login shell, it will not be able to run the forced command we will be setting up.

If the slave will be accepting incoming data from multiple master machines (for example, if you are offering "secondary DNS" service to clients who update their own "data" files) you will need multiple userids, one for each "master" machine which will be sending you data.

If it prompts you for a password when creating the new user(s), make something up. It doesn't matter what it is, we're going to get rid of it anyway.

# useradd dnsslave
New UNIX password:
Retype new UNIX password:

The next step is to "lock" the passwords of these users. We will do this by replacing the users' encrypted passwords with one or more characters which cannot be part of an encrypted password. For example, the unix crypt() function generates results consisting of upper- and lower-case letters, digits, and the characters "." and "/". If we use one or more "@" characters in the encrypted password, it becomes physically impossible for ANY password to encrypt to that value.

However, we need to be careful in the characters we choose. On most systems, the "!" character means that the account is "locked", and will prevent sshd from allowing that userid to log in, even if they authenticate using a key. On many systems, the "*" character serves as a flag that the user's encrypted password is stored in an NIS database. I'm not aware of "@" having any special meaning, therefore that's what I use.

Most systems provide a command which allows you to manually set a user's encrypted password. Most Linux machines have a "usermod" command, and the *BSD systems have a "chpass" command. Check the documentation on your system, and make sure you understand what you're doing, before using these commands.

For a CentOS system:
# usermod -p '$1$xxxxxxxx$@@-NO-REAL-PASSWORD-@@' dnsslave

For a FreeBSD system:
# chpass -p '$1$xxxxxxxx$@@-NO-REAL-PASSWORD-@@' dnsslave

If your system doesn't have such a command, you may be able to edit the /etc/shadow file directory. I do not recommend this, especially if you aren't used to editing the low-level system config files. If you edit this file and mess it up, you can lock yourself and your users out of the systeme entirely.

The rest of these directions are written for the "one userid" situation. If you are setting up multiple userids, simply follow the same steps for each userid.

Install the update-dns script

The next step is to install the update-dns command in the dnsslave user's home directory. This script is what will be running when the master server connects in order to send new data.

# cd ~dnsslave
# wget http://qmail.jms1.net/djbdns/update-dns
...
# chmod 755 update-dns

Test the update-dns script

You can test the update-dns script by copying the data file from the master server by hand, using whatever method you are most comfortable with- scp, floppy disk, USB memory stick, carrier pigeons (with or without QOS entensions), morse code, semaphore flags... whatever works for you. This example assumes you have copied the "data" file to the dnsslave user's home directory on the slave server.

# cd ~dnsslave
# ./update-dns < data
Writing incoming data data to data.2005-06-04.053921
Updating data file
Activating new data
Cleaning up old files

You should see a new "data" file and a new "data.cdb" file. The "data" file should contain the header (to remind you that you need to edit the file on the master server,) followed by the non-comment lines from the original "zones.*" files on the master server, with the "." lines converted to "Z", "&", and "+" lines, and all of the "Z" lines should have the same serial number (which is the unix time when the "make" command was run on the master server.)

Install the ssh public key

Next we need to copy the public key (which we generated above) from the master machine. Copy the "id_dsa_dnsupdate.pub" file from the master server to the "dnsslave" user's home directory on the slave server, again using whatever method works for you. The contents of the public key file are not critical and do not need to be kept a secret. Copy the file to the .ssh directory inside of root's home directory.

After copying the public key file, we need to edit it. It's already in the correct format to be usable for general SSH logins, but we need to add a "forced command" to the file so that the key can only be used to run that one command. This way, if somebody manages to get a copy of the private key file from the master server, the only damage they can do on the slave server is tampering with the DNS data- they would not be able to use that key to gain full access to the system.

The id_dsa_dnsupdate.pub file itself consists of one very long line of text, most of which will look like a string of garbage. Edit the file using your editor of choice, and add the following to the beginning of the line:

The file will already contain...
ssh-dss AAAAB3NzaC1kc3MAAACBAN65...ep/8= dns update key

Add to the beginning:
command="/home/dnsslave/update-dns" ssh-dss AAAAB3NzaC1kc3MAAACBAN65...ep/8= dns update key

Note that this assumes that the "dnsslave" user's home directory is "/home/dnsslave". If not, you will need to make the appropriate change when adding the forced command.

Also note, and this is very important- the file itself MUST contain EXACTLY ONE LINE of text. Before saving your changes, make sure your editor didn't use a "word wrap" feature to turn the file into two or more lines of text. If you're not sure, the command "wc -l id_dsa_dnsupdate.pub" (after you save it) will count how many lines are in the file. If it gives you any number except one, you need to fix the file before continuing.

Create the authorized_keys file

Once you have added the forced command to the public key file, you need to install it as .ssh/authorized_keys within the dnsslave user's home directory.

# cd ~dnsslave
# mkdir -m 700 .ssh
# chown dnsslave:dnsslave .ssh
# mv id_dsa_dnsupdate.pub .ssh/authorized_keys
# chown dnsslave:dnsslave .ssh/authorized_keys
# chmod 600 .ssh/authorized_keys

Test the data transfer

On the master machine, run "make all". You should see it build a new "data" file on the master machine, and then push the changes out to the slave(s). It will look something like this:

On the "master" machine
# cd /service/tinydns/root
# make all
cp header data.tmp
./make-data zones.alfa zones.bravo zones.charlie >> data.tmp && mv data.tmp data
chmod 644 zones.alfa zones.bravo zones.charlie data header
/usr/local/bin/tinydns-data
chmod 644 data.cdb
svc -t /service/dnscache
./push-changes
/---- Start of messages from 1.8.7.301
Writing incoming data data to data.2005-06-04.053921
Updating data file
Activating new data
Cleaning up old files
\---- End of messages from 1.8.7.301

Linking the data into the service

The last piece of the puzzle is, possibly, to create a symlink so that the running tinydns process on the slave machine is reading from the data that the "master" server(s) are sending.

Set the slave server up in the same manner as a "master" server. The first difference is that, instead of symlinking the "data.cdb" file, you will create a symlink from a "zones.whatever" name to the "data" file in the dnsslave user's home directory.

# cd /service/tinydns/root
# ln -s ~dnsslave/data zones.something Choose an appropriate name

You should also set up the /service/tinydns/root directory with the same files (i.e. make-data, Makefile, header, and possibly push-data) that you installed on the master server.

You will also need to install the trigger-make program, which allows the "update-dns" script to run the "make" command in the /service/tinydns/root directory as root.

Before you compile the program, you may need to edit the following lines in the code:

#define DIR  "/service/tinydns/root"
#define MAKE "/usr/bin/make"
#define PATH "/usr/bin:/bin:/usr/local/bin"

These strings should contain the full path to the "/service/tinydns/root" directory (some people may use different service names for this) and the full path to the "make" executable. You may also need to modify the PATH which will be in effect when the "make" command runs. The default values (shown here) should work for most systems.

Once you have configured these strings as you need them, we can compile the code and then make the resulting binary "setuid", so that when it runs, it will run as root.

# cd ~dnsslave
# wget http://qmail.jms1.net/djbdns/trigger-make.c
...
Edit the DIR, MAKE, and/or PATH strings in the file if needed.
# gcc -Wall -o trigger-make trigger-make.c
# chown root:dnsslave trigger-make Substitute the GROUP ID of the dnsslave user. The user should have its own group, not shared with other userids on the system.
# chmod 4110 trigger-make

Once this is done, the "update-dns" script in the dnsslave user's home directory will see the program there, and will run it instead of building a "data.cdb" file. This does the same thing would happen if you were to log in as root and run "make" in the "/service/tinydns/root" directory- the new data will be combined with other "zones.whatever" files (or symlinks) to produce the data.cdb file that the service will actually use, and if the server is pushing data to other slaves, the data will be sent out.

Conclusion

That's all there is to it! Whenever you add or change the "zones.*" files on the master, run "make" and it will bring the master and all of the slaves up to date in a matter of seconds. If you ever delete a "zones.*" file entirely, the Makefile will not be able to detect it, so use "make all" instead.

Copyright 1995-2024 John M. Simpson KG4ZOW (email) (PGP key)
Last updated 2008-12-07 16:55:14 +0000