Antivirus / SPAM Filter recomendations

[SidebandSamurai]

Hello Everyone,

I want to look at installing anti virus / SPAM filter for Scalix,

First: What are the recommendations?

Second: there is a real feeling inside the company that they might miss important email's from clients because an email was accidentally flagged as "Spam" is there anyway of recovering mis-classified messages?

Third: can I control "roll out" on a per user basis, in other words can I turn it on for one user while keeping it off for all the rest (while I test this).

forth: can e-mail messages classified as spam be placed in the users Junk Email folder so the end user can see or catch miss-classified e-mail on outlook?

Thanks a lot for all your help.

Sincerely,

Sideband Samurai

[les]

Hello Everyone,

I want to look at installing anti virus / SPAM filter for Scalix,

First: What are the recommendations?

Second: there is a real feeling inside the company that they might miss important email's from clients because an email was accidentally flagged as "Spam" is there anyway of recovering mis-classified messages?

Third: can I control "roll out" on a per user basis, in other words can I turn it on for one user while keeping it off for all the rest (while I test this).

forth: can e-mail messages classified as spam be placed in the users Junk Email folder so the end user can see or catch miss-classified e-mail on outlook?

Thanks a lot for all your help.

Sincerely,

Sideband Samurai

Easy Solution:
spamassassin/clamav/amavisd/amavisd-milter integrated into sendmail. All opensource.
whitelisting/blacklisting is global, can be managed through webmin easily.
emails can be tagged and server side rules setup in scalix to redirect tagged messages to Junk E-mail folder (sxaa). In general, spamassassin scores >6 = bad stuff, 3<6 = probably spam, but some false postivies, so redirect to Junk e-mail, <3 = good - straight to inbox.
Amavis does have a "raw" quarantine area, but users can't get to it.
False positives generally only happen for the first couple of weeks (teething time), until regular senders are added to whitelists (if necessary).
I use this almost everywhere.

More Difficult solution:
Per user control and whitelisting can be done, per user quarantine, reporting etc.
probably something like Mailscanner. Check the scalix wiki for some howto's.

[PrisonMind]

hi,

i wouldn´t use spam-tagging.
it´s better to hard reject spam!

[les]

hi,

i wouldn´t use spam-tagging.
it´s better to hard reject spam!

you misunderstand.

Spamassassin scores each message from -999 to 1000. Spamassassin will "Tag" the header of each message going through with various lines like....

X-Spam-Level: *****

The stars represent the score, rounded.

So, anything scoring more than 6 = auto delete or "hard reject"
anything scoring between 3 and 6 = probably spam, but might be legit. The header "tagged" in the message for something scoring 4 would be....
X-Spam-Level: ****

So you write an sxaa rule (or outlook) which looks for "X-Spam-Level: ***" in the "tagged" header and if found, redirect to Junk E-mail.

Anything scoring under three will just go into the Inbox. The header will be tagged like "X-Spam-Level: *" or "X-Spam-Level: " if it scored 0 or below.

[PrisonMind]

hey les,

i mean that you have possible mail lost with spam-tagging.

The Theory of SpamTagging is that you prevent false positive
Practice: Users should/must check really (!) All (!) Tagged mails in the junk(spam) folder.
How many users really do this???!?!?

If you use only hard reject the false positives are after 10 seconds by the sender, he knows and can respond.

[les]

hey les,

i mean that you have possible mail lost with spam-tagging.

The Theory of SpamTagging is that you prevent false positive
Practice: Users should/must check really (!) All (!) Tagged mails in the junk(spam) folder.
How many users really do this???!?!?

If you use only hard reject the false positives are after 10 seconds by the sender, he knows and can respond.

i don't think you're following what i am saying. I used the word "tagging" because that's the word you used. So lets forget the word tagging for a minute.

mail gets scored by spamassassin, a header is added to the message. anything scoring more than 6 is deleted automatically.
anything scoring between 3 and 6 is redirected into Junk E-mail via a rule checking for the spam header.
users do check junk email folders.
mail scoring below 6 doesn't trigger the rule, even though the header is added, and it ends up in the inbox.
No messages are "tagged" as spam like in a subject tag.
Any false positives are reported to the IT tech and they put adddresses in the whitelist.

This is a standard setup.

[William]

Use 5-6 good dnsbl to reject spam in sendmail (or elsewhere) and importantly use a whitelist (in the same place so that whitelisted domains/ or email addresses are not checked against any dnsbl).
You can make a whitelist from all your users contacts, (and also a grep of to addresses in your sendmail logs if you want to get a really complete whitelist). I posted a script here a few years ago.

[William]

ClamAV plus sendmail is ok.

Rejecting email - means you'll not be able to get the email back if in the unlikely event you did not have the persons domain whitelisted and the MX for the domain was listed in respected DNSBL at the time of them sending you the email. The presumably completely new client can resend the email, or going from the error message returned to them, get themselves delisted very easily, or use a different email account or phone you etc etc.

I would suggest - go spam free , tagging spam only means uncertainty and wasted time.

[SidebandSamurai]

Hi everyone sorry for the late response. I had a family emergency that took me out of town for 2 weeks.

First of all thanks for the discussion. This is becoming very informative.

As I begin to form an implementation plan, I will be asking more specific questions.

@les ->
Thanks for your suggestions, I have some additional questions for you:

Easy Solution:
spamassassin/clamav/amavisd/amavisd-milter integrated into sendmail. All opensource.

Really this is the easy solution? sounds pretty complicated to me. I have not read the docs yet but I am looking at least four different programs to install and configure. Too me it looks pretty complicated on first glance to setup and get right the first time.

whitelisting/blacklisting is global, can be managed through webmin easily.

What about allowing the users to control the white/black listing, Doesn't Exchange (oh I said the dirty word here ) allow users to do that? Maybe not, I could be totally wrong. The reason I ask about this is Its going to be a pain to add a new client's domain every time an client hires them for work.

emails can be tagged and server side rules setup in scalix to redirect tagged messages to Junk E-mail folder (sxaa). In general, spamassassin scores >6 = bad stuff, 3<6 = probably spam, but some false postivies, so redirect to Junk e-mail, <3 = good - straight to inbox.

I Imagine that you are telling me what one of these rules are, is this global for the whole system or on a per-user basis?
What is SXAA?

False positives generally only happen for the first couple of weeks (teething time), until regular senders are added to whitelists (if necessary).
I use this almost everywhere.

Yea I was anticipating this, that is why I want to implement this and roll out to only a couple of users right now (mainly my account and the Office Manager) so that I can get the system trained and all the bugs worked out. You never mentioned if this can be rolled out per user or not.

Thanks for your thoughts.

Sincerely,

Sideband Samurai

[les]

Easy Solution:
spamassassin/clamav/amavisd/amavisd-milter integrated into sendmail. All opensource.

Really this is the easy solution? sounds pretty complicated to me. I have not read the docs yet but I am looking at least four different programs to install and configure. Too me it looks pretty complicated on first glance to setup and get right the first time.

I guess, if you are installing it from scratch, the first time IS complicated, but once you get it right you have a recipe for next time and next time etc. Some of the other solutions are even more complicated (like mailscanner)

whitelisting/blacklisting is global, can be managed through webmin easily.

What about allowing the users to control the white/black listing, Doesn't Exchange (oh I said the dirty word here ) allow users to do that? Maybe not, I could be totally wrong. The reason I ask about this is Its going to be a pain to add a new client's domain every time an client hires them for work.

I don't think Exchange does that, could be wrong though. Funnily enough, when i run exchange (oops, i mean i don't really....i always run scalix ) I still have a front end linux box running spam and AV filtering. Never done much with the inbuilt exchange content filtering.
What we are actually talking about here is adding a whitelisted domain to the spamassassin user_prefs file. In general, not every client needs to be whitelisted, it depends on what mail clients they use to send, what format they compose in, how "spammy" there messages look (i.e. unsubscribe links CAPS in subject, newsletter-ish feel might trip scores over the threshold). But some do. With this setup, there is no way to manage per user spamassassin user_prefs files, it is just one global file. The only way to do this on a per-user basis is with Mailscanner or some other front-end filter which has these features.

emails can be tagged and server side rules setup in scalix to redirect tagged messages to Junk E-mail folder (sxaa). In general, spamassassin scores >6 = bad stuff, 3<6 = probably spam, but some false postivies, so redirect to Junk e-mail, <3 = good - straight to inbox.

I Imagine that you are telling me what one of these rules are, is this global for the whole system or on a per-user basis?
What is SXAA?

Rules are per user and server side. Spamassassin adds headers like "X-Spam-Level: ****" to messages, so a rule, created either by sxaa, swa or outlook with scalix connect to look for messages with that header, say containing 3 * (which means scored 3 or more) can be setup to move to the Junk E-mail folder.
IMHO sxaa is the best way to do this as it creates the folder if it doesn't exist and avoids the rule being disabled if the user deletes there Junk E-mail folder (yep that has happened, even though its a special folder and outlook does recreate it, if you did the rule in Outlook it needs to be told to go to the "new" Junk E-mail folder again).

p.s. while the rules for moving messages to junk folders are per user, if you tell amavis (which calls spamassassin) to auto delete message scoring above 6 or 8 or whatever, that is global.

sxaa is a scalix command line script to create user based rules. It is easier to use to setup email redirect and forwarding rules. do a "man sxaa" on a scalix server.
these commands can be added into the SAC via means of plugins to help make life a little easier.

False positives generally only happen for the first couple of weeks (teething time), until regular senders are added to whitelists (if necessary).
I use this almost everywhere.

Yea I was anticipating this, that is why I want to implement this and roll out to only a couple of users right now (mainly my account and the Office Manager) so that I can get the system trained and all the bugs worked out. You never mentioned if this can be rolled out per user or not.

see above.

In essence, a spam filter setup is fairly involved to get working, regardless of which option you choose. There are some other options out there, commercial based like commtouch, which Scalix endorse or internet based filtering like messagelabs or dyndns spam filter (which funnily enough uses all the same opensource tools above and is global). They all have their own "complications". Maybe commtouch is easier, but you do need to pay a license for it, and i think its annual to. Could be wrong though. Messagelabs is really expensive, but solid and does AV scanning as well. Call it the rolls royce model. dyndns is relatively cheap ($50us per year) but it doesn't include AV scanning and while it uses the same opensource tools as above you have a an interface which cant change as many options (i.e. you cant change certain global spamassassin settings) as you might like to.

[joako]

hey les,

i mean that you have possible mail lost with spam-tagging.

The Theory of SpamTagging is that you prevent false positive
Practice: Users should/must check really (!) All (!) Tagged mails in the junk(spam) folder.
How many users really do this???!?!?

If you use only hard reject the false positives are after 10 seconds by the sender, he knows and can respond.

It's not a good idea either to DSN spam mails. Better to reject connections based on RBL, send the borderline mails to the junk email folder, delete the ones high on the spam scale. In the past that works really well for me.

[SidebandSamurai]

Thanks for the comments so far.

Another question comes to mind. I really need to get this right the first time. I don't want to implement this and then dial it in.

Is there a way to setup a secondary Scalix server and forward copies of all the email from the primary server to the "Sandbox" server so I could work with a "Sandbox" instead of a production system?

One thing I don't want to do is implement this and start loosing a ton of legitimate Email. I have seen posts where administrators have implemented a solution and the result was ALL email was flagged as SPAM. My goal would be to train this "Sandbox" and then copy what it had learned to the production system. It would also allow me to make mistakes with out loosing important email.

MailScanner looks to be a great program and is open source, a lot of major corporations use MailScanner, why would you not implement MailScanner?

Sincerely,

Sideband Samurai

[les]

It's not a good idea either to DSN spam mails. Better to reject connections based on RBL, send the borderline mails to the junk email folder, delete the ones high on the spam scale. In the past that works really well for me.

amavis allows you to not send DSN's for junk mail.

[les]

MailScanner looks to be a great program and is open source, a lot of major corporations use MailScanner, why would you not implement MailScanner?

The last time i looked at Mailscanner (many many moons ago) the installation was quite involved. Lots of components to pull together into 1 system. I agree its a good product, but i've never used it, stuck to my original recommendations which i use everywhere and i get little or no issues with it. And none of my clients have requested a quarantine area, per use spam filter settings etc.

Maybe someone else who is using Mailscanner can weigh in and let you know how easy/hard it is to get going and how it performs etc.

[RSisco]

I guess, if you are installing it from scratch, the first time IS complicated, but once you get it right you have a recipe for next time and next time etc.

You're assuming that we all keep excellent documentation while implementing new software.

Richard