Page 1 of 1

What mailbox format is being used

Posted: Tue May 02, 2006 3:42 pm
by enquirer
What kind of mailbox format does the server use? I guess, it is not mbox file format. Is it maildir? Or is it someting proprietary?

Thanks.

Posted: Tue May 02, 2006 7:42 pm
by leigh
I would suggest it is the latter.

Posted: Wed May 03, 2006 7:54 pm
by enquirer
Hasn't this quetion interested those who installed and usef Saclix?

What if you need to migrate to another mail server? Are there any tools to export mail from Scalix to some standard format, like mbox or maildir?

Posted: Wed May 03, 2006 7:57 pm
by leigh
That's easy.
If you're using outlook, export to a .PST and then import it into your new server.
If not, use IMAP.
I have migrated a significant number of POP3 mailboxes to scalix using these methods, and I see no real reson why it wouldn't work the other way around.

Posted: Wed May 03, 2006 8:15 pm
by enquirer
Well, that is a solutin, of course. But if I need to migrate gigabytes of emails for different users, using IMAP for that is not a great idea.
In case of mbox or maildir, one can just grab the files and move them to any location.

But seriously, how does Scalix store emails? Are the files human-readable and parseable?

Posted: Wed May 03, 2006 8:32 pm
by leigh
Perhaps the development or support team can give you a better answer, but It looks to me like all messages are stored in /vat/opt/scalix/data/
Under that are several directories with what appears to be arbitrary names. They are all binary files, ie non-human-readable.
You would also need the various database files and message tracking files from the other directories.
The short answer to your question, I think, is No.

However, there's nothing to stop anyone from pulling apart the CE RAW package and telling us all how to extract messages. Isn't that what it's for?

Standard disclaimers apply here. I'm just a user, I don't do development on the thing, so I could be talking through my hat.

Posted: Thu May 04, 2006 3:40 am
by Valerion
Well ... this is working from memory only, from docs I read YEARS ago when Scalix was something comepletely different. And Scalix may correct me if I'm wrong :)

When a message enters the system, the responsible queue runner (internet mail gateway, rci, etc) writes it to disk at once, in ~/data somewhere. Scalix writes a container file (containing a list of body parts), and a file for each body part (headers, body and each attachment). The container file has links to these parts. A pointer is then used to route that message through the system.

When it gets handled by local delivery, the service just writes a pointer to the container file into each user's mailbox. This allows for a single copy of the message for multilple recipients.

When it comes to mail folders, the root of your mailbox is in your user dir (omshowu -f will point that out), and each folder under that is a container file in ~/data somewhere, containing other container files.

If you want to trace the structure, you can use tf.browse to some extent, or of course, omcontain (the best tool).

The advantages are IMO:
1) A bad sector on the disk may make you lose an attachment, not a whole mailbox or mailstore, unlike DB-based or mbox-based systems
2) Standard file handling are typically cheap operations, and unlike a mbox you don't have to go sequencially search a large file
3) A 50MB email to 20 users only gets written to disk once, and only takes up that amount of space (plus FS overhead)

Disadvantages:
1) Easy migration to / from other systems (I still think it's easy enough, though)
2) You can't use standard daemons (IMAP, POP3, etc), since only Scalix's daemons will understand the mailstore (But then Scalix HAS all the standard daemons ...)
3) Watch your inode allocation on your filesystem (Been bitten by this once)

Posted: Fri May 05, 2006 5:18 am
by ScalixSupport
Excellent explanation, thanks B.

Just to add to that, ReiserFS allocates inodes dynamically, so you will never run out of inodes on that FS.

Cheers,

Sascha.

Posted: Thu May 25, 2006 10:16 pm
by leigh
By the way, if you're really interested in the structure of the mailstore, have a look at the Scalix Knowledgebase. There's a chapter there from the admin guide which details the message store layout. Interesting reading.

Posted: Fri May 26, 2006 11:21 am
by pete
So, Sascha:

Are you suggesting that we use Reiser instead of Ext3? Just curious.

P