Scalix Performance

[jprangi]

Hello,
I have a performance issue and it is so bad that it can totally change our idea of moving to scalix if we don't find the solution.

1. Deleting more that 1000 email terribly slow down the system. That's even with one test user. Now I am scared with 40 heavy users and more than 100 mailing lists.

2. I did a very small crash test. Send 500 email email in just few seconds. The uptime was more that 20 in just few second and I was having hard time connecting to scalix server for next 15 minutes. Imap and outlook were timing out.
It took almost 1 hour 30 minutes before I receive all the emails in my Inbox.

3. I have noticed the the performance is much better with small size emails than large size emails. Is that true or its just me.

4. Tomcat crashes frequently. I have to restart tomcat and apache to enable webmail.

I am running dual CPU Xeon 2.0 GHz with 1GB RAM and 2GB swap space.

I want to know if I can do some performance tunning on the server. Cause in the real world this server is going to handle more that 20K emails every day and sometimes email quakes of 5000 emails in few seconds. Do you think scalix can handle that kind of traffic.

Thank you,
-Jai

[ScalixSupport]

Please take a look at the disk test binary I wrote at http://www.scalix.com/wiki/index.php?title=Admin_Resource_Kit#Disk_performance_test

I suspect you have a performance problem with your disks. How is the disk subsystem set up ? IDE, SCSI, SATA ? Raid 1+0 or Raid 5 ? NFS ?

Cheers

Dave

[jprangi]

Thank you for response, We are using sata drives. No RAID, No NFS
Here are the results from the scripts. Does not look too bad.
[root@mail hub]# ~/./disktest
Test Started
Test file: disktest.data
File size: 100 Mb
Num write: 10000
PRO: 0.000 Create file: disktest.data
PRO: 0.240 Zero fill
PRO: 1.980 Sync data
PRO: 0.000 Seek to start
PRO: 0.000 Fill write buffer
Starting write tests
PRO: 23.085 Write tests complete
Test complete


-Jai

[jprangi]

Hello All,

Has any one done the performance (ofcourse I am not talking about price/performance) evaluation between scalix, exchange and postfix with courier IMAP.

I will appreciate if some can share their experience.


Thank you,
-Jai

[btisdall]

If you're only concerned about performance go for Postfix with Dovecot, the latter of which is much more efficient than Courier in my experience.

BUT Postfix/Dovecot is a "pure mail" solution which can't really be compared to Scalix or Exchange, both of which are focused on business-orientated features such as calendaring & contacts & are designed to work in multi-server environments across enterprises.

I'm afraid my exposure to Exchange is very limited so I'll let someone else jump in to comment on the performance of this vs Scalix.

[Brian_K]

What are optimal numbers I should be seeing when running this disktest binary in a RAID 1+0 Enviorment. I ran it and came up with the following results.

Test Started
Test file: disktest.data
File size: 100 Mb
Num write: 10000
PRO: 0.000 Create file: disktest.data
PRO: 1.272 Zero fill
PRO: 3.531 Sync data
PRO: 0.000 Seek to start
PRO: 0.000 Fill write buffer
Starting write tests
PRO: 34.113 Write tests complete
Test complete

Any info would be appreciated.

Thanks,
Brian

[David.be]

Hi,

I'm running Scalix 11.1 on FC5 (1Gb RAM, SATA Disks, Software Raid 1) and I have a lot of performance issues. Connection timeouts, slow SWA, etc.

Here's the output of disktest :

Test Started
Test file: disktest.data
File size: 100 Mb
Num write: 10000
PRO: 0.068 Create file: disktest.data
PRO: 0.825 Zero fill
PRO: 3.880 Sync data
PRO: 0.000 Seek to start
PRO: 0.000 Fill write buffer
Starting write tests
PRO: 278.910 Write tests complete
Test complete

This is amazingly bad. I only have 10 or 15 users connected via IMAP (Thunderbird).

Could someone please help me improve performance on my system ?

Thanks,

David

[jprangi]

Hello,
Has anyone moved from scalix from exchange. I think the whole community will benefit if some can post their experience in this regard. I guess interesting point will be to see the comparison between performance, and features regardless of how important are they.

[kjakkanen]

Hi,

The recent posts from Scalix people about the impact of the disk system (including the possible RAID-solution) seems not to be in vain, it really makes a difference on what HW and RAID (especially taking care of the controller chosen and possible write cache), this is what we get with a HP ProLiant DL380 G5 (8 x 72 GB SAS 15K RPM + HP SmartArray P500 with 512 MB battery-backed write cache):
---
Test Started
Test file: disktest.data
File size: 100 Mb
Num write: 10000
PRO: 0.002 Create file: disktest.data
PRO: 0.230 Zero fill
PRO: 0.329 Sync data
PRO: 0.000 Seek to start
PRO: 0.000 Fill write buffer
Starting write tests
PRO: 0.779 Write tests complete
Test complete
---

So not bad, even though we use the questionable RAID-5 on the LVM-array.

I'd definitely steer off from using any kind of software RAID-solutions, go for 15K SCSI/SAS-disks for /var/opt/scalix mount and make sure the controller is a very decent piece of hardware. This all costs some, but it's a nightmare to fix later and in the end hardware is rather cheap nowadays - it's the downtime and burned-down admins that will cost you a heck of more $$$ in the end.

And of course it will depend on the system usage as well, so your mileage may vary depending on whether you have 10 or 100 and more users. Check out posts on this forum from KAnderson@Scalix, you'll see some more opinions regarding the subject.

-Kimmo

[KevinG]

Hi,

I'm new to Scalix, I am just evaluating it as a potential replacement for Exchange 5.5.

I must say that I think you all appear to be too accepting of the high demands that Scalix is making on the hardware. I have a thread running in the Web Access section on this.

If I open a large shared calendar through the Outlook Client it works within a few seconds.

If I open the same shared calendar through Web Access it takes a very long time (tens of seconds), sometimes so long that the Web Client reports the failure "Failed to display messages for folder ..." While this is happening there is very heavy disk i/o. Bare in mind that the only stuff that it needs from disk is the same as it looked up for the Outlook Client within a couple of seconds.

While Web Access is opening a single folder for a single user it is creating hundreds of temporary files (I have seen over 1000 created for one public folder lookup) in /var/opt/scalix/cs/s/data with names like /var/opt/scalix/cs/s/data/000003j/000e5jt and it is also creating many (I've seen up to 200) in /var/opt/scalix/cs/s/temp with names like /var/opt/scalix/cs/s/temp/000dbuj and /var/opt/scalix/cs/s/temp/0006bef.mim. I can't help thinking that it should be able to do this by creating one or two temporary files, not 1200.

I really think, particularly for those of you paying for support, that you should be asking does Web Access really need to be so i/o intensive, before you go out and change your hardware.

To be fair to Scalix I would admit that this could just be down to a strange quirk with my particular setup. Surely I can't be the only person seeing this problem. Or am I?!

[jaime.pinto]

If I open a large shared calendar through the Outlook Client it works within a few seconds.

You could be enjoying what Scalix claims to be one of the advantages of using SmartCache.

I wouldn't so much try to compare performance of a locally ran application (OL) with that of a web client/server model.
I do agree there could be something improved on the SWA on its own merit:
viewtopic.php?t=7954#36094

[KevinG]

Hi again Jaime,

I have spent a lot of time on this and I think I have taken reasonable care to make sure that I am not looking at cached data (by setting up new users and so on).

I think what I need to do is setup Exchange 2003 on exactly the same hardware and look at the performance on the same folders. If Exchange 2003 performs and Scalix doesn't then I would have no choice but to recommend Exchange. Please believe me when I say that I really want Scalix to work. It looks like a great product apart from these performance issues.

Kevin

[kjakkanen]

Hi,

I agree with the number of temp files, for what are the millions (new persons to Scalix, there can indeed be millions of temp files under the s-directory unless you periodically run omscan process) of temps actually used? Or perhaps I should as why it needs so many? That might explain the need for high-intense disk I/O... Just looking to understand the internals.

-Kimmo

[Richard Hall]

Hi KevinG,

The large number of files in the temp directory created during the listing of a large Public Folder is almost certainly due to the one-off creation of the MIME structure information for those messages. These files are a by product of that information generation and are not recreated when the folder is subsequently opened/listed.

Whether this information needs to be generated depends on how the messages were created, any messages that enter the system via delivery to the Inbox will already have this info, however other ways of creating messages (sxmboximp, import from PST, etc) will produce messages that will require the information to be generated when the message is first listed by an IMAP client.

Richard

[KevinG]

Richard,

Thanks for that reply, it is good to find out what is going on. I'm not convinced that I only get problems the first time that I access a folder, I will do some more testing, and I still think that that Web Access should not be so much more demanding on the hardware than the Outlook client - does it really need to use IMAP for example? Do you need to store the public folder information in a database in one format and then also save it in a poorly performing cache in a different format?

One quick question. The folders I have been seeing problems with were exported from Exchange 5.5 and then imported into Scalix (using Export/Import in Outlook 2000). Can I force this MIME structure to be created without having to manually open each folder through Web Access and having to keep retrying each time it fails? If I went live with end users seeing the number of failiure messages in Web Access that I have seen in testing then the product would soon be thrown out of the window.

Kevin