simscan

simscan is a program developed by Inter7, the same people who wrote vpopmail, qmailadmin, and several other programs which are commonly used on qmail servers. It serves the same purpose for which many people use qmail-scanner, however because it's written in C instead of Perl it runs much more quickly, making it a better choice for high-volume mail servers.

I've written a web page which has more detail about troubleshooting the problems people commonly run into when installing simscan.

Patrick "marlowe" McDonald has written a web page similar to this one, detailing issues that people commonly run into when setting up simscan on a system. Some of the content here may mirror things on his site, and vice-versa... it's well worth your time to check out his page as well.

Patch to work with clamav 0.90 and higher

The patch has been updated for ClamAV 0.93, however I'm not able to test the changes directly because clamav 0.93 on my server is not producing the new "daily.cld" files which people are talking about. Please let me know if you have any problems with it.

When clamav version 0.90 was released, they changed the location and filenames used for the virus definition files. The "configure" script from simscan has not yet been updated to look for the new names, so people who try to install simscan while the newer clamav is installed are receiving an error message like this when they run their "./configure" command:

configure: error: Unable to find your clamav databases, specify --enable-clamavdb-path.

I ran into the same issue, and ended up writing a patch to fix the problem. I then joined the simscan mailing list to announce the patch, but I decided to first search the archives to make sure somebody else hadn't already written one. I did find that somebody had already written a patch, however I found a few problems with their patch and decided to finish testing and using my own patch instead.

2007-08-14 The patch which was originally here had a potential problem. While testing another issue I discovered that if the files in the daily.inc directory (i.e. the new storage format) become corrupted or were missing, freshclam would remove the directory and just download a daily.cvd file (i.e. the old storage format) instead.

The original patch hard-coded either "the old filenames" or "the new filenames" into the binaries, which could potentially cause two problems:

• If something happened to the main.inc or daily.inc directories and freshclam replaced them with main.cvd and/or daily.cvd files for some reason, "simscanmk -g" would not be able to read the version numbers.

• If somebody were to upgrade from a pre-0.90 version of clamav to a newer version, "simscanmk -g" would not be able to read the version numbers.

2008-05-02 It looks like clamav 0.93 has introduced a third format for storing the virus databases, this time using a "daily.cld" file. I've updated the patch so it should recognize and work with these files as well, however "freshclam" on my own server is not producing these files, so it's rather difficult to test the updated patch. If you're using clamav 0.93, have a "daily.cld" file, and are having issues, please let me know.

The patch below will work correctly with any of the three known storage methods, and can therefore be applied by everybody, regardless of what version of clamav you may be using (up to 0.93 anyway... who knows what the clamav guys are going to change in the future.)

2008-07-09 Like I said above, I wasn't able to fully test the ".2" version of the patch, but I couldn't see any reason why it wouldn't work. I have since heard from man people that it was working without any problems, however one person reported it wasn't working... and when I looked on my own server (which was also using ".cld" files by then) I found that it also wasn't working for me.

After doing some testing, I was able to fix the problem by changing the structure of the code just a little bit. It still works in exactly the same way, but now it explicitly holds the return value of the access() function in a variable before testing it, rather than allowing the value to be "swallowed" into an if statement by the compiler. There is no valid reason why the older patch shouldn't have worked- the only thing I can think is that the compiler "over-optimized" the code somehow, and bypassed one of the checks.

Note that the previous patches will no longer be available, once I have verified that this works with the new clamav version. If you're still using simscan 1.3.1, you should upgrade to 1.4.0.

Patch to explicitly set umask()

I ran into a problem while testing simscan- basically it wasn't allowing mail to be accepted. Clients were receiving the error message "mail server temporarily rejected message (#4.3.0)", and setting the "SIMSCAN_DEBUG=2" variable only gave the following messages:

I use tcsh instead of bash.
# setenv QMAILQUEUE /var/qmail/bin/simscan
# setenv SIMSCAN_DEBUG 2
# setenv NOP0FCHECK 1
# qmail-inject < mail.txt
...
simscan: calling clamdscan
simscan: cdb looking up version clamav
simscan: fatal error executing clamdscan
simscan: exit error code: 71
qmail-inject: fatal: mail server temporarily rejected message (#4.3.0)

I added some debugging messages to the code (see the debug patch below) and was able to at least see what was coming back from clamdscan:

# qmail-inject < mail.txt
...
simscan: calling clamdscan
simscan: clamdscan: /var/qmail/simscan/1186199934.284560.19969: Access denied. ERROR
simscan: clamdscan:
simscan: clamdscan: ----------- SCAN SUMMARY -----------
simscan: clamdscan: Infected files: 0
simscan: clamdscan: Time: 0.000 sec (0 m 0 s)
simscan: cdb looking up version clamav
simscan: fatal error (2) executing clamdscan
simscan: exit error code: 71
qmail-inject: fatal: mail server temporarily rejected message (#4.3.0)

First, let me suggest that this extra layer of debugging IS needed from time to time- perhaps a SIMSCAN_DEBUG=3 could be created to turn on this kind of logging? (This is how I did it for my own testing- a patch for this is below.)

As you can see, the problem is that clamd didn't have permission to read the temporary directory. However, simscan deleted the directory before I could look at it, so I couldn't tell what permissions were and were not there.

So I added some more debugging code, this time bypassing the deletion of the temporary directory if SIMSCAN_DEBUG_FILES is non-zero and an error occurs, or if SIMSCAN_DEBUG_FILES is greater than one. (Again, see the patch below.)

With the code no longer deleting the temporary directory I was able to see the contents of the temporary directoy, and the problem was immediately evident:

# ls -laF /var/qmail/simscan/1186200783.537875.20730/ total 20 drwx--S--- 2 simscan simscan 4096 Aug 4 00:13 ./ drwxr-s--- 3 simscan simscan 4096 Aug 4 00:13 ../ -rw------- 1 simscan simscan 31 Aug 4 00:13 addr.1186200783.537875.20730 -rw------- 1 simscan simscan 173 Aug 4 00:13 msg.1186200783.537875.20730 -rw------- 1 simscan simscan 18 Aug 4 00:13 textfile0

As you can see, the directory and the files within it were created with the correct group, but with the group not having any permissions at all. The problem is that the "clamd" process, running as the "clamav" user (which is part of the "simscan" group) needs to have "group read" permission on the files, and both "group read" and "group execute" permissions on the directories, in order to be able to scan the files.

So I looked at the code which creates the temporary directory, and I see that when it calls mkdir(), it does explicitly use 0750 as the permissions on the new directory. However, the permissions for any directories created by the mkdir() function are "filtered" through a value called the "umask" in order to come up with the actual permissions assigned to the new directory.

If you're not familiar with the term "umask", the idea is that any items (files, directories, named pipes, UNIX sockets, etc.) you may create will have their initial permissions "filtered" by this value. Some people may have a specific policy about the permissions given to new items- for example, any new files might need to be readable to users in their "group", but not accessible to other users. The umask() mechanism gives the user a way to enforce their own policy- programs would create new items using "0777" permissions (i.e. full access to anybody) and the umask value would "subtract" the bits they don't want added by default.

I normally use "umask 077" on my system, because whenever any files or directories are created, I don't want them to be accessible to other users on the system. I figure if I want a file to be available to others, I will explicitly "chmod" the files as needed.

The simscan program doesn't mention the umask() function at all, which means that it simply uses whatever value was set by its parent process. This is normally not a problem, beacuse most OS distributions have startup and/or login scripts which give the user a umask of 022.

However, in this case we need to be absolutely sure that the temporary files created by the "simscan" user (the "simscan" and "ripmime" programs) are readable by the "clamav" user (the "clamd" process), and my umask value was forcing those files to not be readable to anybody except the userid which owns the file (which is "simscan".) In order to do this, we need to do two things:

• The /var/qmail/simscan directory is "owned" by the "simscan" group instead of the default "root" or "wheel" group, and the "group sticky" bit is turned on for the directory. The "group sticky" bit causes the kernel to assign the group which owns the directory as the "group owner" of any files and directories created within that directory (and in addition, any new directories also have their "group sticky" bit turned on as well, so this behaviour cascades into all child directories.)

  This step is already a normal part of setting up simscan to work with clamav version 0.90 or later.

• We need to change the "umask" value to one which DOES NOT block the "group read" and "group execute" permissions.

  The "execute" permission on a directory actually means "traversable"- basically a process which has "execute" but not "read" permissions on a directory can access individual files within that directory if it knows the exact filenames (and the files permissions allow the access, of course) but it cannot use opendir() (used by the "ls" command, for example) to read a list of the filenames in the directory.

  This step may be accomplished in one of two ways:

  • Explicitly add a "umask 027" line to the "run" script(s) for your SMTP service(s).

  • Explicitly add a call to "umask()" within the simscan program itself.

  The first option is self-explanatory. The patch below implements the second option- it adds an explicit call to the umask() function within the simscan program, just before it calls mkdir() to create the temporary directory.

debug patch

Here's a quick patch which implements the debugging messages I mentioned above. The changes are:

• Setting SIMSCAN_DEBUG=3 will show the actual text returned from clamdscan.

• Setting SIMSCAN_DEBUG_FILES=1 will bypass deleting the work directory (within /var/qmail/simscan) if there was an error with clamav.

• Setting SIMSCAN_DEBUG_FILES=2 will always bypass deleting the work directory.

I just noticed that I had SIMSCAN_DEBUG_FILE here- the variable is SIMSCAN_DEBUG_FILES, with the letter "S" at the end. Sorry for the inconvenience. -jms1 2007-11-23

Downloads...

Combined patch

A few people have asked for a single patch file which combines the three patches above (clamav, umask, and debug.) Since I was updating things for 1.4.0 anyway, I went ahead and rolled one. Enjoy.

The ".3.patch" version below includes the clamav ".3" patch which works with ClamAV 0.93.

For what it's worth, I've been told that the clamav and umask patches will be part of simscan-1.4.1 when it's released. I haven't heard whether the debug patch will be added or not, but I think it would be a good idea to have it in there, and then add lower-level debug messages to other subsystems (spamassassin, dspam, etc.) which call external programs.

On 2008-10-07, Felix Buenemann sent a patch to the simscan mailing list (local copy below) which fixed a bug in simscanmk, where the read() function's value was not being checked for errors. This is a legitimate bug in the original simscan code, so I have added this patch to my combined simscan patch.

update-simscan

update-simscan is a C program which is meant to be used as a wrapper for systems running both simscan and clamav (as I am now), to allow freshclam to run the simscanmk -g command, which updates simscan's database of virus scanner versions (so the correct version numbers show up in the "Received:" header that simscan adds to each message it processes.)

Here's a quick example of how to compile the program and set the permissions correctly:

Running as root. First build and install the executable...
# gcc -s -o /usr/local/sbin/update-simscan update-simscan.c
# chown root:simscan /usr/local/sbin/update-simscan
# chmod 4110 /usr/local/sbin/update-simscan

Once it's compiled and installed, you need to configure freshclam to run the program whenever new virus definitions are received. We also need to make sure that it runs the program with the proper group permissions. As you can see above, members of the "simscan" group are allowed to run it, so we first need to make sure that the "clamav" user is a member of this group.

First see what groups the clamav user belongs to.
# groups clamav
clamav : clamav

If simscan is not listed, you need to add it to the group. This command works with Linux, there should be something similar on your system.
# usermod -a -G simscan clamav
# groups clamav
clamav : clamav simscan

Once this is done, we need to make two changes to freshclam's configuration file.

# nano /usr/local/etc/freshclam.conf Your freshclam.conf file may be in a different location.

Find this line...
OnUpdateExecute

... and make it look like this (add the full path to the update-simscan program):
OnUpdateExecute /usr/local/sbin/update-simscan


Find this line...
#AllowSupplementalGroups yes

... and make it look like this (i.e. remove the "#" and make sure it says "yes"):
AllowSupplementaryGroups yes

Then, if you are running freshclam as a background service (as opposed to running it from a cron job) you need to restart that service. This may involve running a command like one of these:

# svc -t /service/freshclam if you're running it under daemontools, as I do

# service freshclam restart if you're running it from a sysvinit script

Of course, if you're running freshclam from a cron job, there is nothing to restart- the next time cron runs your job, it will read the new freshclam.conf file and use the new settings automatically.

Here's the link to download the source code.

Ramdisk for /var/qmail/simscan

One thing that I used to do with qmail-scanner, and I'm now doing with simscan, is to mount the "work" directory as a ramdisk. This does away with having to wait for a physical disk to spin around and the heads to move to the right place in order to write out a file which will only exist long enough to be scanned for viruses. I have found it significantly reduces the amount of time it takes for the virus scans to finish.

I'm doing this on my server (running CentOS Linux) with this line in my /etc/fstab file:

none   /var/qmail/simscan   tmpfs   nodev,noexec,noatime,uid=169,gid=164,mode=2750 0 0
On my server, 169 is the numeric UID of the "simscan" user, and 164 is the numeric GID of the "simscan" group.

Like I mentioned above, I did the same thing with qmail-scanner, except that the mount point was /var/qmail/scan/tmp (I used to compile qmail-scanner to use "/var/qmail/scan" instead of "/var/spool/qscan"), and I had the numeric UID and GID of the "qscand" user. (I still had "mode=2750" in there.)

Of course, in order for this to work, there has to be a physical "/var/qmail/simscan" directory on the underlying filesystem. The permissions and ownership don't really matter- I have mine owned by "root:root" and 0700, so that simscan will complain in case the ramdisk doesn't mount- you may wish to do the same, or you may wish to make it owned by "simscan:simscan" with 2750, so that things will still work, albeit more slowly, if the ramdisk isn't mounted.

2009-01-02 There is a problem with ripmime version 1.4.0.9, where the files it creates within the work directory are not group-readable. Most people have been working around this by using version 1.4.0.7 instead, however Jukka Kurkela wrote a patch to make version 1.4.0.9 act correctly. The patch is available on his web site http://qmail.tjc.fi/ if you need it.

Note that I'm still using ripmime 1.4.0.7, so I haven't tested his patch myself. I have, however, looked at it, and it looks fairly sane to me (i.e. it's almost identical to what I would have done- the only thing I would have done differently is added a command-line option to ripmime so the user could choose whether or not to turn on the group-readable permission bit at run time, rather than compiling it into the code so it's always turned on. Some ripmime users may not want to turn on group-read permissions, and by not forcing the option on those users, the patch has a much better chance of being added to the next official version of ripmime.)

I'm providing the link to his patch here as a courtesy to those who may need it. If you have any questions or problems with it, you can reach him through my qmail-patch mailing list (where he is a fairly regular member) or by direct email (his email address is listed on that web page.)

Copyright 1995-2024 John M. Simpson KG4ZOW (email) (PGP key)
Last updated 2009-01-20 09:41:09 +0000