Page 1 of 2

Create a white list and black list of domains/email addresse

Posted: Fri Jul 21, 2006 9:52 am
by chrisdebono
Hi

I was wondering if it is possible to enable a white list based on the addressbook or other config file and possibly a black list too in order to cut as much spam as possible.

Regards

Chris

Posted: Wed Aug 30, 2006 4:19 am
by William
I too would be interested in a method of extracting just the email addresses from all the contacts databases.

export of domains/email addresses from address book

Posted: Wed Aug 30, 2006 5:27 am
by William
I have got so far but am a little stuck:

omsearch -e "INTERNET-ADDR=*@*" -m IA-FORMAL -U -S

returns just all the Scalix users email addresses. Running omshowcda only shows the following:

# omshowcda
** SYSTEM Shared 15 minutes 30.08.06 10:04:40

I tried adding '-t s' and '-d SYSTEM' to the omsearch command to try and access all the other contacts we have.

What am I missing here?

Posted: Wed Aug 30, 2006 6:00 am
by Valerion
The internal directories (used by omsearch) doesn't store the local contacts folders in the mailboxes (I assume that's what you're after).

In Scalix 11 you can get access to that via the Platform API, though you need to authenticate as the user to read that information.

In Scalix 10 you can access it via LDAP. Authenticate to the LDAP service on the Scalix server as your user, set your bind DN to o=MyContacts and use that information. The rfc822Mailbox attribute contains the email address of the contact.

Posted: Wed Aug 30, 2006 8:06 am
by William
I have tried

omldapsearch -v '(rfc822Mailbox=*)' -D o=MyContact rfc822Mailbox

but I get this meeage:

ldap_open( mail, 389 )
ldap_bind: Invalid credentials

How do I authenticate against the LDAP service?

Posted: Wed Aug 30, 2006 8:45 am
by Valerion
Sorry, it's not Bind DN, but base DN in my previous reply ...

This works for me:

Code: Select all

omldapsearch -v '(rfc822Mailbox=*)' -b 'o=MyContacts' -D 'cn=<MY CN>' -w '<MY PASSWORD>' rfc822Mailbox

Posted: Wed Aug 30, 2006 9:50 am
by William
That works as expected, by returning my personal contacts.

We have a shared contacts list (public folder) with 1800 contacts in that everyone can access - how would I access this from the command line?

How would I find out it's name?

Posted: Wed Aug 30, 2006 9:52 am
by Valerion
I don't think you have any way other than a client interface (IMAP, maybe?) to access that at the moment.

Posted: Thu Aug 31, 2006 10:11 am
by William
Found a way to access the shared contacts via the command line, by searching for VCARD contents:

cd /pathto/scalix/data/
fgrep -r "EMAIL;PREF;INTERNET:" * > domains.txt
cut --delimiter=@ --only-delimited --fields=2 domains.txt | tr -d [:blank:] | tr [:upper:] [:lower:] | sort | uniq > white_list_domains.txt

the frgrep command takes about 10mins to run (on our ~60k files in the data folder tree hierarchy - ls -R /pathto/scalix/data/ | wc -l)

Where are the personal contacts stored ?

Posted: Thu Aug 31, 2006 10:35 am
by florian
I don't think you'll get all the data there - it highly depends on how the object has been created.

Contacts created by Outlook e.g., no matter if personal or shared, are stored as binary Attribute-Value-pair MAPI objects - no vCard information here.

I'd suspect it probably makes sense to wait for Scalix 11 here and then use our REST web services API which will allow you to access all contact-type folders in vCard format.

Cheers,
Florian.

Posted: Thu Aug 31, 2006 10:52 am
by William
I was uncertain as to its completeness untill I exported all the shared contacts from Scalix via Outlook to Excel and filtered for uniques and so on and actually ended up with the same number (minus 3 for some perhaps unimportant reason).

They were and are all created and manged from within Outlook AFAIK.

We do not exacty have a deluge of UCE getting through to users at the moment - but this was one of the steps to getting an essential white list inplace before we added other antispam measures.

Posted: Thu Aug 31, 2006 10:55 am
by florian
Yeah, I think what might be happening here is that they are rendered into vCards for optimized access by IMAP clients, Evolution and SWA - but again, however, no guarantees....

-- Florian.

Posted: Mon Sep 04, 2006 8:15 am
by William
Here is an improvement over the previous posting:


cd /pathto/scalix/data/
fgrep -r "EMAIL;PREF;INTERNET:" * > domains.txt

# this formats the file for use in access.db within sendmail
cut --delimiter=@ --only-delimited --fields=2 domains.txt | tr -d [:blank:] | tr [:upper:] [:lower:] | sort | uniq | sed '/yahoo/d; /hotmail/d; /gmail/d; /aol/d' | sed -e 's/^/From:/' -e 's/$/\t\tOK/' > /etc/mail/white_list.txt

# Include an external file into access.db:
cat /etc/mail/access /etc/mail/white_list.txt | makemap hash /etc/mail/access.db

# view access.db contents
makemap -u hash /etc/mail/access.db | more

Posted: Wed Sep 13, 2006 10:26 am
by William

Code: Select all

#!/bin/sh
fgrep -r "EMAIL;PREF;INTERNET:" /pathto/scalix/data/* | cut --delimiter=@ --only-delimited --fields=2 | tr -d [:blank:] | tr [:upper:] [:lower:] | sort | uniq > /root/contacts_whitelist.txt
fgrep -i " TO" /var/log/mail* | cut --delimiter="@" --only-delimited --fields=2 | cut --delimiter=">" --only-delimited --fields=1 | tr -d [:blank:] | tr [:upper:] [:lower:] | sort | uniq > /root/sendmail_whitelist.txt
cat /root/sendmail_whitelist.txt /root/contacts_whitelist.txt | sort | tr -d [:blank:] | tr [:upper:] [:lower:] | uniq | sed '/yahoo/d; /hotmail/d; /gmail/d; /aol/d' | sed -e 's/^/From:/' -e 's/$/\t\tOK/' > /etc/mail/white_list.txt
cat /etc/mail/access /etc/mail/white_list.txt | makemap hash /etc/mail/access.db


This also extracts all the domains from email addresses that email was sent to, it searches the sendmail maillogs, and includes them into the whitelist.
The first command takes a little while to run, the others take no time at all.

Posted: Tue Sep 19, 2006 8:45 am
by William
does anyone have code / pointers on how to iterate through all the users, getting all of their personal contacts?