http:// qmail.jms1.net / simscan /

simscan

simscan is a program developed by Inter7, the same people who wrote vpopmail, qmailadmin, and several other programs which are commonly used on qmail servers. It serves the same purpose for which many people use qmail-scanner, however because it's written in C instead of Perl it runs much more quickly, making it a better choice for high-volume mail servers.

I've written a web page which has more detail about troubleshooting the problems people commonly run into when installing simscan.

Patrick "marlowe" McDonald has written a web page similar to this one, detailing issues that people commonly run into when setting up simscan on a system. Some of the content here may mirror things on his site, and vice-versa... it's well worth your time to check out his page as well.


Patch to work with clamav 0.90 and higher

The patch has been updated for ClamAV 0.93, however I'm not able to test the changes directly because clamav 0.93 on my server is not producing the new "daily.cld" files which people are talking about. Please let me know if you have any problems with it.

When clamav version 0.90 was released, they changed the location and filenames used for the virus definition files. The "configure" script from simscan has not yet been updated to look for the new names, so people who try to install simscan while the newer clamav is installed are receiving an error message like this when they run their "./configure" command:

configure: error: Unable to find your clamav databases, specify --enable-clamavdb-path.

I ran into the same issue, and ended up writing a patch to fix the problem. I then joined the simscan mailing list to announce the patch, but I decided to first search the archives to make sure somebody else hadn't already written one. I did find that somebody had already written a patch, however I found a few problems with their patch and decided to finish testing and using my own patch instead.

2007-08-14 The patch which was originally here had a potential problem. While testing another issue I discovered that if the files in the daily.inc directory (i.e. the new storage format) become corrupted or were missing, freshclam would remove the directory and just download a daily.cvd file (i.e. the old storage format) instead.

The original patch hard-coded either "the old filenames" or "the new filenames" into the binaries, which could potentially cause two problems:

2008-05-02 It looks like clamav 0.93 has introduced a third format for storing the virus databases, this time using a "daily.cld" file. I've updated the patch so it should recognize and work with these files as well, however "freshclam" on my own server is not producing these files, so it's rather difficult to test the updated patch. If you're using clamav 0.93, have a "daily.cld" file, and are having issues, please let me know.

The patch below will work correctly with any of the three known storage methods, and can therefore be applied by everybody, regardless of what version of clamav you may be using (up to 0.93 anyway... who knows what the clamav guys are going to change in the future.)

2008-07-09 Like I said above, I wasn't able to fully test the ".2" version of the patch, but I couldn't see any reason why it wouldn't work. I have since heard from man people that it was working without any problems, however one person reported it wasn't working... and when I looked on my own server (which was also using ".cld" files by then) I found that it also wasn't working for me.

After doing some testing, I was able to fix the problem by changing the structure of the code just a little bit. It still works in exactly the same way, but now it explicitly holds the return value of the access() function in a variable before testing it, rather than allowing the value to be "swallowed" into an if statement by the compiler. There is no valid reason why the older patch shouldn't have worked- the only thing I can think is that the compiler "over-optimized" the code somehow, and bypassed one of the checks.

Note that the previous patches will no longer be available, once I have verified that this works with the new clamav version. If you're still using simscan 1.3.1, you should upgrade to 1.4.0.

File: simscan-1.4.0-clamav.3.patch
Size: 2,869 bytes
Date: 2008-07-07 18:30:47 +0000
MD5: 657528ee3727526345e8b89f9d297afc
SHA-1: 68b591ec8a487e7147d62a567428d5023e227570
RIPEMD-160: 4f07e43bad6967234b665bc30a492271a401528c
PGP Signature: simscan-1.4.0-clamav.3.patch.asc

Patch to explicitly set umask()

I ran into a problem while testing simscan- basically it wasn't allowing mail to be accepted. Clients were receiving the error message "mail server temporarily rejected message (#4.3.0)", and setting the "SIMSCAN_DEBUG=2" variable only gave the following messages:

I use tcsh instead of bash.
# setenv QMAILQUEUE /var/qmail/bin/simscan
# setenv SIMSCAN_DEBUG 2
# setenv NOP0FCHECK 1
# qmail-inject < mail.txt
...
simscan: calling clamdscan
simscan: cdb looking up version clamav
simscan: fatal error executing clamdscan
simscan: exit error code: 71
qmail-inject: fatal: mail server temporarily rejected message (#4.3.0)

I added some debugging messages to the code (see the debug patch below) and was able to at least see what was coming back from clamdscan:

# qmail-inject < mail.txt
...
simscan: calling clamdscan
simscan: clamdscan: /var/qmail/simscan/1186199934.284560.19969: Access denied. ERROR
simscan: clamdscan:
simscan: clamdscan: ----------- SCAN SUMMARY -----------
simscan: clamdscan: Infected files: 0
simscan: clamdscan: Time: 0.000 sec (0 m 0 s)
simscan: cdb looking up version clamav
simscan: fatal error (2) executing clamdscan
simscan: exit error code: 71
qmail-inject: fatal: mail server temporarily rejected message (#4.3.0)

First, let me suggest that this extra layer of debugging IS needed from time to time- perhaps a SIMSCAN_DEBUG=3 could be created to turn on this kind of logging? (This is how I did it for my own testing- a patch for this is below.)

As you can see, the problem is that clamd didn't have permission to read the temporary directory. However, simscan deleted the directory before I could look at it, so I couldn't tell what permissions were and were not there.

So I added some more debugging code, this time bypassing the deletion of the temporary directory if SIMSCAN_DEBUG_FILES is non-zero and an error occurs, or if SIMSCAN_DEBUG_FILES is greater than one. (Again, see the patch below.)

With the code no longer deleting the temporary directory I was able to see the contents of the temporary directoy, and the problem was immediately evident:

# ls -laF /var/qmail/simscan/1186200783.537875.20730/ total 20 drwx--S--- 2 simscan simscan 4096 Aug 4 00:13 ./ drwxr-s--- 3 simscan simscan 4096 Aug 4 00:13 ../ -rw------- 1 simscan simscan 31 Aug 4 00:13 addr.1186200783.537875.20730 -rw------- 1 simscan simscan 173 Aug 4 00:13 msg.1186200783.537875.20730 -rw------- 1 simscan simscan 18 Aug 4 00:13 textfile0

As you can see, the directory and the files within it were created with the correct group, but with the group not having any permissions at all. The problem is that the "clamd" process, running as the "clamav" user (which is part of the "simscan" group) needs to have "group read" permission on the files, and both "group read" and "group execute" permissions on the directories, in order to be able to scan the files.

So I looked at the code which creates the temporary directory, and I see that when it calls mkdir(), it does explicitly use 0750 as the permissions on the new directory. However, the permissions for any directories created by the mkdir() function are "filtered" through a value called the "umask" in order to come up with the actual permissions assigned to the new directory.

If you're not familiar with the term "umask", the idea is that any items (files, directories, named pipes, UNIX sockets, etc.) you may create will have their initial permissions "filtered" by this value. Some people may have a specific policy about the permissions given to new items- for example, any new files might need to be readable to users in their "group", but not accessible to other users. The umask() mechanism gives the user a way to enforce their own policy- programs would create new items using "0777" permissions (i.e. full access to anybody) and the umask value would "subtract" the bits they don't want added by default.

I normally use "umask 077" on my system, because whenever any files or directories are created, I don't want them to be accessible to other users on the system. I figure if I want a file to be available to others, I will explicitly "chmod" the files as needed.

The simscan program doesn't mention the umask() function at all, which means that it simply uses whatever value was set by its parent process. This is normally not a problem, beacuse most OS distributions have startup and/or login scripts which give the user a umask of 022.

However, in this case we need to be absolutely sure that the temporary files created by the "simscan" user (the "simscan" and "ripmime" programs) are readable by the "clamav" user (the "clamd" process), and my umask value was forcing those files to not be readable to anybody except the userid which owns the file (which is "simscan".) In order to do this, we need to do two things:

File: simscan-1.3.1-umask.patch
Size: 473 bytes
Date: 2007-08-04 05:19:52 +0000
MD5: 546dcdc855a4bad02fb9af072e18a16e
SHA-1: eed42fdb156f340bf3c576df2d4a63142a615d2f
RIPEMD-160: a433d29c7bf598c09ba02fd5b7702951f65990fc
PGP Signature: simscan-1.3.1-umask.patch.asc
File: simscan-1.4.0-umask.patch
Size: 473 bytes
Date: 2007-11-23 06:45:24 +0000
MD5: 4a2e6424123869eb9b720c77dbc57a62
SHA-1: 4075859b2b3bf9004000d4e76ae1aa65826a797e
RIPEMD-160: e32b99b07038427a96269257223c9b4c6597c941
PGP Signature: simscan-1.4.0-umask.patch.asc

debug patch

Here's a quick patch which implements the debugging messages I mentioned above. The changes are:

I just noticed that I had SIMSCAN_DEBUG_FILE here- the variable is SIMSCAN_DEBUG_FILES, with the letter "S" at the end. Sorry for the inconvenience. -jms1 2007-11-23

Downloads...

File: simscan-1.3.1-debug.patch
Size: 2,210 bytes
Date: 2007-08-07 23:33:25 +0000
MD5: b99cbfdc478c9aeb3f6e0e680753683d
SHA-1: 0497f75c4d0befc03eea3c9489e213b46bfab951
RIPEMD-160: ec4fd4b0c0e5c81f6082d03921466e68f5a864e5
PGP Signature: simscan-1.3.1-debug.patch.asc
File: simscan-1.4.0-debug.patch
Size: 2,176 bytes
Date: 2007-11-23 06:45:37 +0000
MD5: 8bb7782eb91e2a5d670727cc5136fa1f
SHA-1: c057d0ea63a8e6bc62bacff5dcc40816ad256456
RIPEMD-160: 4d0aedd55713eef17721141e024ad1c4ac1f9011
PGP Signature: simscan-1.4.0-debug.patch.asc

Combined patch

A few people have asked for a single patch file which combines the three patches above (clamav, umask, and debug.) Since I was updating things for 1.4.0 anyway, I went ahead and rolled one. Enjoy.

The ".3.patch" version below includes the clamav ".3" patch which works with ClamAV 0.93.

File: simscan-1.4.0-combined.3.patch
Size: 5,300 bytes
Date: 2008-07-07 18:33:40 +0000
MD5: 6ea52a9438e4391c3cb3e3dba29d6c8f
SHA-1: 395f7ef9910e747a3cfca74440f7528160d96eeb
RIPEMD-160: 7f18e2c33089a844386f1bf67532425f578e86c8
PGP Signature: simscan-1.4.0-combined.3.patch.asc

For what it's worth, I've been told that the clamav and umask patches will be part of simscan-1.4.1 when it's released. I haven't heard whether the debug patch will be added or not, but I think it would be a good idea to have it in there, and then add lower-level debug messages to other subsystems (spamassassin, dspam, etc.) which call external programs.

On 2008-10-07, Felix Buenemann sent a patch to the simscan mailing list (local copy below) which fixed a bug in simscanmk, where the read() function's value was not being checked for errors. This is a legitimate bug in the original simscan code, so I have added this patch to my combined simscan patch.

File: simscan-1.4.0-combined.4.patch
Size: 7,092 bytes
Date: 2008-10-08 01:49:08 +0000
MD5: 9b90f6b3f02ab9e987b68fd4a09fc0e9
SHA-1: aa08ad59abec1d14698d511daf8f73ecea801a7b
RIPEMD-160: 52e0bd53fc770efa7586517aa16f596767d464bc
PGP Signature: simscan-1.4.0-combined.4.patch.asc
File: read.patch
Size: 2,010 bytes
Date: 2008-10-08 01:48:55 +0000
MD5: 90f5ce60a42c35ba3876ccb9223a79a7
SHA-1: 5f6000089bd60cc758661f22f7e7764d88a7df66
RIPEMD-160: 780202638522d7a1b2cbe328e4ccc9bab6bb6abe
PGP Signature: read.patch.asc

update-simscan

update-simscan is a C program which is meant to be used as a wrapper for systems running both simscan and clamav (as I am now), to allow freshclam to run the simscanmk -g command, which updates simscan's database of virus scanner versions (so the correct version numbers show up in the "Received:" header that simscan adds to each message it processes.)

Here's a quick example of how to compile the program and set the permissions correctly:

Running as root. First build and install the executable...
# gcc -s -o /usr/local/sbin/update-simscan update-simscan.c
# chown root:simscan /usr/local/sbin/update-simscan
# chmod 4110 /usr/local/sbin/update-simscan

Once it's compiled and installed, you need to configure freshclam to run the program whenever new virus definitions are received. We also need to make sure that it runs the program with the proper group permissions. As you can see above, members of the "simscan" group are allowed to run it, so we first need to make sure that the "clamav" user is a member of this group.

First see what groups the clamav user belongs to.
# groups clamav
clamav : clamav

If simscan is not listed, you need to add it to the group. This command works with Linux, there should be something similar on your system.
# usermod -a -G simscan clamav
# groups clamav
clamav : clamav simscan

Once this is done, we need to make two changes to freshclam's configuration file.

# nano /usr/local/etc/freshclam.conf Your freshclam.conf file may be in a different location.

Find this line...
OnUpdateExecute

... and make it look like this (add the full path to the update-simscan program):
OnUpdateExecute /usr/local/sbin/update-simscan


Find this line...
#AllowSupplementalGroups yes

... and make it look like this (i.e. remove the "#" and make sure it says "yes"):
AllowSupplementaryGroups yes

Then, if you are running freshclam as a background service (as opposed to running it from a cron job) you need to restart that service. This may involve running a command like one of these:

# svc -t /service/freshclam if you're running it under daemontools, as I do

# service freshclam restart if you're running it from a sysvinit script

Of course, if you're running freshclam from a cron job, there is nothing to restart- the next time cron runs your job, it will read the new freshclam.conf file and use the new settings automatically.

Here's the link to download the source code.

File: update-simscan.c
Size: 2,677 bytes
Date: 2007-07-31 22:52:48 +0000
MD5: 81c17227ac71c5af5bfb594c9c0459dd
SHA-1: 67b231052167dd452a0d5f9edfbefa5434fb77f8
RIPEMD-160: ce9d5956b874155e62c92215617f0f5806bc484f
PGP Signature: update-simscan.c.asc

Ramdisk for /var/qmail/simscan

One thing that I used to do with qmail-scanner, and I'm now doing with simscan, is to mount the "work" directory as a ramdisk. This does away with having to wait for a physical disk to spin around and the heads to move to the right place in order to write out a file which will only exist long enough to be scanned for viruses. I have found it significantly reduces the amount of time it takes for the virus scans to finish.

I'm doing this on my server (running CentOS Linux) with this line in my /etc/fstab file:

none   /var/qmail/simscan   tmpfs   nodev,noexec,noatime,uid=169,gid=164,mode=2750 0 0
On my server, 169 is the numeric UID of the "simscan" user, and 164 is the numeric GID of the "simscan" group.

Like I mentioned above, I did the same thing with qmail-scanner, except that the mount point was /var/qmail/scan/tmp (I used to compile qmail-scanner to use "/var/qmail/scan" instead of "/var/spool/qscan"), and I had the numeric UID and GID of the "qscand" user. (I still had "mode=2750" in there.)

Of course, in order for this to work, there has to be a physical "/var/qmail/simscan" directory on the underlying filesystem. The permissions and ownership don't really matter- I have mine owned by "root:root" and 0700, so that simscan will complain in case the ramdisk doesn't mount- you may wish to do the same, or you may wish to make it owned by "simscan:simscan" with 2750, so that things will still work, albeit more slowly, if the ramdisk isn't mounted.


2009-01-02 There is a problem with ripmime version 1.4.0.9, where the files it creates within the work directory are not group-readable. Most people have been working around this by using version 1.4.0.7 instead, however Jukka Kurkela wrote a patch to make version 1.4.0.9 act correctly. The patch is available on his web site http://qmail.tjc.fi/ if you need it.

Note that I'm still using ripmime 1.4.0.7, so I haven't tested his patch myself. I have, however, looked at it, and it looks fairly sane to me (i.e. it's almost identical to what I would have done- the only thing I would have done differently is added a command-line option to ripmime so the user could choose whether or not to turn on the group-readable permission bit at run time, rather than compiling it into the code so it's always turned on. Some ripmime users may not want to turn on group-read permissions, and by not forcing the option on those users, the patch has a much better chance of being added to the next official version of ripmime.)

I'm providing the link to his patch here as a courtesy to those who may need it. If you have any questions or problems with it, you can reach him through my qmail-patch mailing list (where he is a fairly regular member) or by direct email (his email address is listed on that web page.)