HowTos/Mailwasher

From Scalix Wiki
Jump to: navigation, search

Scalix Wiki -> How-Tos -> Mailwasher

Contents

MailWasher and Scalix

Disclaimer

I am not, nor do I profess to be any sort of sendmail, Scalix or MailWasher expert. Furthermore, while the techniques I describe below worked for me, your mileage may vary. To that extent, please be sure to read all documentation available to you as I will not take responsibility for any mistakes or typos made. I have checked this document for its accuracy, but I may have missed something during the process, or your configuration may require additional steps or options not discussed in this HOWTO.

Neither Scalix, MailWasher, nor Sendmail have officially supported this setup in any fashion, and they probably do not claim it to be functional and probably will NOT offer any support. If you have any problems, I can assist you with basic help, but you're pretty much on your own. I may offer more detailed instructions as time permits, however, I am mainly writing this as reference for myself in the event I need to do it again.

Please note that MailWasher is setup as a Sendmail Milter, which may not be recommended by Scalix.

I am NOT a MailWasher representative nor am I affiliated with them in any way outside of being a paying customer and fan of their excellent product. The statements I am about to make are my opinion only and their validity and accuracy may or may not be true. Please contact a FireTrust MailWasher representative for any questions, comments or clarification.

Links and References

What is MailWasher Server?

<exerpt from oss.firetrust.com>

Code: 

MailWasher Server is an open-source, server-side junk mail filter package for businesses. MailWasher
Server differs from other open-source server anti-spam packages in that it offers a polished,
well-integrated web interface and built-in quarantine management facilities. MailWasher Server is
easier for administrators to set up and manage and easier for users to use the product on a day-to-day
basis. MailWasher Server fully supports Windows Server/Exchange Server systems, in addition to
Unix-based systems such as Linux and Solaris.

Why MailWasher Server?

Well, as the description above indicates, MailWasher Server possesses an extremely well polished web interface for quarantined messages. It is simple, easy and effective. It's ease of use hits a home run with non-expert users and it provides very powerful system for efficient administration of the SPAM filtering process.

Additionally, some of its nice features include a global white and black list, per user white and black list, as well as per user configuration changes. While complete integration with Scalix is not there, end users will be right at home with the easy and simple to use interface.

MailWasher Server is also Open Source with commercial support available at the per incident or contract level, which may ease the concern of support for some users.

I prefer it because of the ability to send Quarantined Message Summaries on a great number of customizeable (on the global level and per user level) options, such as, once per week, month, or per any number of quarantined messages. It also allows users to manage their own spam retrieval instead of forcing the administrator to assist or otherwise control it.

Assumptions

For the sake of NOT repeating installation instructions that can be found elsewhere, I will stop here and say that you MUST have a working Scalix installation. I will point you to the document on proper installation of MailWasher which you must follow up to the "Installing the mail conduit" and "Sendmail" portions of the MailWasher Linux Installation document listed above.

I personally downloaded and installed the MailWasher RPM for Fedora Core 4, which I would recommend as the easiest method of MailWasher installation.

As of 8/7/2006, the current version of MailWasher is 2.1.4 and you can obtain it from the links above.

Installing the mail conduit

This assumes you are using Scalix in the STOCK configuration, meaning Sendmail is your MTA, and you have completed all installation tasks up to Installing the mail conduits in the MailWasher installation documents or installed via RPM.

Let's turn on MailWasher debugging by editing /etc/mwserver.conf and adding the following line to the end.

Code: 

log_level=debug


Setup MailWasher to start at boot.

(On Fedora at least) Code: 

chkconfig --level 345 mwserver-mpd.sh on
chkconfig --level 345 mwserver-mwi.sh on


(On SuSE Linux OSS 10.2) Code: 

chkconfig -a mwserver-mpd.sh 
chkconfig -a mwserver-mwi.sh

Additionally, I had to perform the following after the RPM installation of MailWasher, but prior to starting the services.

Code: 

chown -R mwserver:mwserver /var/run/mwserver
chmod 750 /var/run/mwserver
chown -R mwserver:mwserver /var/log/mwserver
chown -R mwserver:mwserver /var/lib/mwserver

Start MailWasher

To start MailWasher Server, use the following commands:

(at least on Fedora) Code: 

/etc/init.d/mwserver-mpd.sh start
/etc/init.d/mwserver-mwi.sh start

Configuring MailWasher

Be sure to configure MailWasher if you have not done so already. Configuration is performed via a web based configuration process which is fairly simple and provides online help.

Point a web browser to:

http://your.server.name.local:4044 and follow the instructions.

Note: During the installation process, you will be prompted for FirstAlert! information. While you do NOT have to set this up at this time, you can either purchase a new subscription to FA! or get a trial account by visiting http://oss.firetrust.com/shop/. If you opt to not configure it now, you can always add the support at any time by logging into MailWasher as an administrator and setting it up at a later date.

Backup your sendmail configuration.

Code: 

cp -Rp /etc/mail /etc/mail.backup
cp -p /var/opt/scalix/sys/smtpd.cfg /var/opt/scalix/sys/smtpd.cfg.orig

If you are using Scalix 11, the file is: 

 /var/opt/scalix/{host}/s/sys/smtpd.cfg

where {host} is a 2-character string consisting of the first character and last character of your hostname. So, for "server.mail.com" the filename would be: 

 /var/opt/scalix/sr/s/sys/smtpd.cfg

Modify sendmail.mc to configure MailWasher

Next, we need to add two lines to the end of your /etc/mail/sendmail.mc file.

Code: 

INPUT_MAIL_FILTER(`mailwasher_server',    `S=unix:/var/run/mwserver/mpd.sock, F=T, T=S:4m;R:4m')
define(`confINPUT_MAIL_FILTERS', `mailwasher_server')

NOTE: For the sake of argument, do not include any blank lines between commands in the sendmail.mc file!

Compile sendmail.mc into a fresh sendmail.cf

Now, we need to compile the sendmail configuration. This step requires the sendmail-cf package to be installed, so if it isn't installed, please do so before proceeding.

Code: 

m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Configure Scalix

Now, let's tell Scalix to use an SMTPFILTER, edit /var/opt/scalix/sys/smtpd.cfg and add the following to the end of the file, but before the [SUBMIT] and [LMPT] block, especially if you have theire directive set to NO.

Code: 

SMTPFILTER=TRUE (if there appears an statement called SMTPMILTER change it to SMTPFILTER)

Configure sendmail and Scalix

Now, configure Scalix to modify your sendmail.cf file and restart all the required services by running the following command.

Code: 

omsendin


Be sure to say YES to install it.

Restart all dependant services

Restart sendmail and Scalix with the following:

Code: 

/etc/init.d/sendmail restart
/etc/init.d/scalix restart
/etc/init.d/scalix-tomcat restart

Troubleshooting Notes

I had issues with MailWasher quarantining local mail going to the Internet. I fixed it by adding a global whitelist for the address *@domain.tld which took effect immediately.

Important log files to watch are:

Code: 

/var/log/mwserver/mpd.log
/var/log/mwserver/mwi.log
/var/log/maillog


Finally, after you have completed the initial training, make sure you turn off debug logging by deleting the log_level=debug from your /etc/mwserver.conf and restart MailWasher once you are positive it is working properly as extremely verbose logging has a tendency to fill up hard drives ;-)

Congratulations, you should now have a fully functional MailWasher/Scalix system!

Frequently Asked Questions

Q: What BlackList services can I utilize with MailWasher?

Currently, you can utilize any BlackList service with MailWasher. The most common ones I've seen in use are:

  • IP Based
    • relays.ordb.org
    • sbl.spamhaus.org

You can however use other ones, a brief list can be found at http://www.email-policy.com/Spam-black-lists.htm.

Q: Where do I find the source or rpm for this? Am i missing something?

You must go to the Source Forge page for the MailWasher program and download the current version from there. If you need help locating it, please go to http://oss.firetrust.com and click on the "Downloads" link.

Q: Has anyone run into an issue where the SWA or SAC doesn't work after installing the rpm?

I would suspect your Apache, and/or Scalix installation may need to be restarted. You can also attempt a telnet to the server on port 80 and see if it connects.

Q: Is it possible to have MailWasher scan for keywords in the subject line or body text?

At this time, I do not believe so. You are however encouraged to participate in the development of MailWasher and contribute such a feature back to the community!

Q: Do you have to make any changes to sendmail.cf?

To start, yes, you will need to modify your sendmail.cf file. That said, I HIGHLY advise against editing it directly. Instead, please follow the directions in this HOWTO and use m4 to generate your sendmail.cf, all of the steps required are listed.

Q: Does the junkmail submission work with Scalix?

I have NOT been able to get the junk mail submission to work properly with Scalix SWA. I have had success using it with Outlook or any other client supporting the "Forward As Attachment" option.

As a note, I did NOT create a local scalix user for this task. I believe I created a MailWasher user account however.

According to Firetrust, the "Junk Mail Submission Service" is not hosted by the local installation. All reported messages are forwarded to Firetrust's FirstAlert! Spam Filtering Service (a pay mail filter hashing log thing). When a new message arrives, it is scanned locally by the local installation and a hash of that email is queried against the FirstAlert! service which flags e-mails as either ham or spam. If you are using the demo service, you'll notice that the reason the email was quarantined was because of "FirstAlert! Match: Category" or something like that.

Forwarding messages to the "Junk Mail Submission Service" really only benefits those of us who have a license to use the FirstAlert! service.

Q: How can I check if MailWasher is actually scanning messages?

As far as I know, MailWasher does NOT modify any of the headers of e-mails. The only way I know of to check to see if it is handling any messages is to do one of the following:

Assuming you are running mwserver with debug logging: 

tail -f /var/log/mwserver/mpd.log

You can also view your mail log: 

tail -f /var/log/maillog

However, I usually log into the MWI (MailWasher Interface) and go to System Statistics. The number of messages analyzed will tell you if its checking messages ;-) Also, if you followed the HOWTO including adding the local users to the global whitelist, their messages are more than likely NOT being filtered, which could explain why / if their outgoing messages are being scanned.

Q: Does MailWasher modify the message headers or the message body?

As far as I know, it does not modify the message or headers.

Q: What does cursor->c_pget returned -30976: DB_SECONDARY_BAD: Secondary index inconsistent with primary mean?

Apparently there are syncronization issues with "Purging Messages Since Last Login" or whatever that means. This is not however the end of the world. I'll refer you to this thread which explains the simple fix: http://www.firetrust.org/phpBB2/viewtopic.php?t=564

Basically, I got tired on my large installation of manually performing those steps, so I have a cron job that runs every 5 minutes that does the following: 

Checks to see if mpd is running.
If mpd is not running:
Stops mwi if it is running.
Prints a message saying that it was not running.
Deletes /var/lib/mwserver/quarantine*
Restarts mpd

Checks to see if mwi is running.

If mwi is not running:
Start mwi.

Don't worry about deleting the quarantine* files, I have not noticed any users loosing their quarantines as MailWasher rebuilds the database if it is not there.

Q: Do I have to use the FirstAlert! service?

The short answer is: NO. I find that the filter works equally well with out it. And specifically in most cases, works better than SpamAssassin alone. However, if you would like to test the service, they provide a user account that you can use on a trial basis, ie try before you subscribe.

You can obtain the current trial username/password from:

http://oss.firetrust.com/shop/

However, the pricing for the FirstAlert! service seems quite reasonable for small to large installations.

Q: Do I have to configure the FirstAlert! service during installation?

You do not have to configure it at all upon initial installation. You can disable it or skip it during the installation process and still activate it at any time afterwards by logging into the MWI (web interface) as an "administrator".

Q: Why isn't my BlackList service working properly?

I will say that there seemed to be some discussion on the MailWasher forums regarding v 2.1.0 and its inability to utilize blacklist services. The problem was fixed for v2.1.1. Please upgrade to the newest version of MailWasher.

If you are running a current version of MailWasher, using the black-hole list servers does in fact play a role in the filtering process. I'm not entirely sure to what extent however.

Utilizing the black-hole servers AFAIK is the LAST STEP in the MailWasher filtering process. If the FirstAlert! service or statistical analysis analyzes mail as spam, it is more than likely not going through any of the black listing services that are configured. You can however, according to some of the documentation I've read, opt to utilize the same black listing services in sendmail instead of using MailWasher to query them first.

That being said, I cannot confirm or even assume how that can be accomplished as I am NOT a sendmail fan or an expert on sendmail. I do know that that question has been asked on the FireTrust MailWasher forums. I would refer you to there for further questioning.

Q: Why do I see "mail loops back to me (MX problem?)" in my maillog?

Typically the error you are describing is caused by a bad relay that is forcing mail loops. You may have to reconfigure Scalix and sendmail to ensure your relays are set up properly.

I'm wondering if omsendin properly modified your sendmail.mc file. I would attempt re-running it and see if there is any possibilities that it did not modify it for the Scalix additions.

Try rerunning omsendin and restarting sendmail and Scalix.

Q: What exactly happens when my users submit junk messages?

According to Firetrust, the "Junk Mail Submission Service" is not hosted by the local installation. All reported messages are forwarded to Firetrust's FirstAlert! Spam Filtering Service (a pay mail filter hashing log thing). When a new message arrives, it is scanned locally by the local installation and a hash of that email is queried against the FirstAlert! service which flags e-mails as either ham or spam. If you are using the demo service, you'll notice that the reason the email was quarantined was because of "FirstAlert! Match: Category" or something like that.

Forwarding messages to the "Junk Mail Submission Service" really only benefits those of us who have a license to use the FirstAlert! service.

Lastly, I will say that if 60% of your mail is being identified as spam by the FirstAlert! service, and you've been submitting samples to them, you are helping your spam filter out. I am a paying subscriber and am quite happy with their service.

I mentioned that the FirstAlert! service is optional and that it is a pay service with reasonable corporate pricing plans. This service contributes to the well being of the entire project as well as allows you to compare local spam with a much larger spam database.

Without sounding like a commercial or an ad, please support all open source projects as much as possible.

Notes

My installation was using all default values for all installations. MailWasher required the first 100 messages to be classified in order to apply a statistical filter to incoming e-mails. This was easy enough as it sent my administration user an e-mail letting me know when it required this to be done (after it recieved 100 e-mails). I marked them as Junk, Legitimate or Unsure and suddenly the statistical filtering started with out a hitch.

There is no synchronization between MailWasher and Scalix. When a user recieves e-mail, if they do NOT have a MailWasher account, it creates them an account and e-mails them their password which they can change. Users are able to retrieve quarantined messages from the e-mail (after verifying their password) without administrator assistance.

If you run into any problems or have any comments, questions or compliments, please send me an e-mail to: joshua@prestoncentral.com

Thanks!

Joshua Preston.

Changelog

  • Fixed typographical error in backing up Scalix configuration
  • Added information regarding the FirstAlert! service offered by FireTrust
  • Added recommendation to turn off debug logging in MailWasher
  • Created Wiki on Scalix Wiki page
  • Added Frequently Asked Questions
Retrieved from "https://www.scalix.com/wiki/index.php?title=HowTos/Mailwasher&oldid=4356"