15400 mails - SWA slow for everybody - sended mails busy

[kmeyer2]

Now here I'm with a Scalix 11.1
Suse linux 10.1 4000MB Ram 2,6 Dualcore and the scalix-server has a lot of problems as I tryed to go life - since I copied 15400 e-mails into a mailbox (the 8th user).
(the scalix-tomcat.conf is configured for 2000M.)

Followings: the swa works very slow on "looking for new mails" on every folder of each user and if you try to close that window after 15 seconds or 20 the IE7 crashes - or if you wait longer the swa says it isn't able so show that mails - next time it works after 15 seonds. (and that aree the mailboxes with 14 mails in it...)

yesterday some e-mails were busy at the recipient - outlook - clients.

I tryed 11.03 11.04 11.1 and it seems to be finer by every version. 11.1 starts great and if there isn't a easy way of solving that problem - I'll skip that implementation, cause it seems not to be stable enought.

I changed the Network - Card, testet the memory and the rest of the hardware - it is okay.

Fetchmail has problems too, by resolving mails - it slowes down after popping some mailboxes - and then it needs a long time to manage the rest. - reindexing does not help .

Is there someone who has problems like that? Is there someone who solved all this problems?

need help

BR Amok

[jaime.pinto]

Slow SWA has always been an issue for us as well, since 11.0.1 till 11.0.4. I have not had the chance to upgrade to 11.1.0, so I don't know if things have improved with this release. But I do know that our server is swapping a lot, with only 1GB of RAM. You have 4GB, so this is somewhat of a surprise to hear.

What seems consistent with SWA is that the first time a user logs in via SWA it takes a long time for the messages to show up every time the users clicks on a folder, in particular users with lots of email.This is really an unpleasant surprise for most users. If the user happens to click on another folder before the red scalix logo appears on the top right corner the browser goes nuts!
Although not as drastic, but just as annoying, if a user has been using outlook (or another email client) for a long time (a couple of weeks), next time he/she returns to SWA the same thing happens.

What is the result of the first 7 lines of the "top" command on the server?

[kmeyer2]

Hi jaime,
Thanks for your fast reply:

here is a screenshot of the top. Any Idea?

BR Amok

Tasks: 158 total, 2 running, 156 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 3367736k total, 1288672k used, 2079064k free, 83664k buffers
Swap: 2007992k total, 0k used, 2007992k free, 641216k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6577 root 16 0 8512 2348 1872 R 0 0.1 0:00.30 sshd
7697 root 16 0 2308 1060 764 R 0 0.0 0:01.05 top
1 root 16 0 716 280 244 S 0 0.0 0:01.46 init
2 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1
5 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1
6 root 10 -5 0 0 0 S 0 0.0 0:00.02 events/0
7 root 10 -5 0 0 0 S 0 0.0 0:00.00 events/1
8 root 11 -5 0 0 0 S 0 0.0 0:00.00 khelper
9 root 13 -5 0 0 0 S 0 0.0 0:00.00 kthread
12 root 10 -5 0 0 0 S 0 0.0 0:00.00 kblockd/0
13 root 10 -5 0 0 0 S 0 0.0 0:00.00 kblockd/1
14 root 13 -5 0 0 0 S 0 0.0 0:00.00 kacpid
142 root 20 0 0 0 0 S 0 0.0 0:00.00 pdflush
143 root 15 0 0 0 0 S 0 0.0 0:00.85 pdflush
144 root 16 0 0 0 0 S 0 0.0 0:00.00 kswapd0
145 root 13 -5 0 0 0 S 0 0.0 0:00.00 aio/0
146 root 13 -5 0 0 0 S 0 0.0 0:00.00 aio/1
356 root 11 -5 0 0 0 S 0 0.0 0:00.00 cqueue/0
357 root 11 -5 0 0 0 S 0 0.0 0:00.00 cqueue/1
358 root 10 -5 0 0 0 S 0 0.0 0:00.01 kseriod
394 root 10 -5 0 0 0 S 0 0.0 0:00.00 kpsmoused
745 root 10 -5 0 0 0 S 0 0.0 0:00.00 ata/0
746 root 10 -5 0 0 0 S 0 0.0 0:00.00 ata/1
758 root 12 -5 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
759 root 12 -5 0 0 0 S 0 0.0 0:00.00 scsi_eh_1
865 root 10 -5 0 0 0 S 0 0.0 0:00.00 md1_raid1
871 root 10 -5 0 0 0 S 0 0.0 0:01.02 md2_raid1
888 root 10 -5 0 0 0 S 0 0.0 0:00.00 reiserfs/0
889 root 10 -5 0 0 0 S 0 0.0 0:00.00 reiserfs/1
983 root 21 -4 1884 596 348 S 0 0.0 0:00.36 udevd
1746 root 10 -5 0 0 0 S 0 0.0 0:00.00 md0_raid1
1789 root 15 0 0 0 0 S 0 0.0 0:00.00 kjournald
2165 root 15 0 1800 660 528 S 0 0.0 0:00.00 resmgrd
2185 root 15 0 1892 620 432 S 0 0.0 0:00.06 syslog-ng
2188 root 16 0 1652 516 324 S 0 0.0 0:00.00 klogd
2210 root 16 0 1520 496 416 S 0 0.0 0:00.00 acpid
2227 messageb 16 0 3420 948 728 S 0 0.0 0:00.04 dbus-daemon
2276 root 16 0 4380 2976 1440 S 0 0.1 0:01.04 hald
2531 root 16 0 1412 392 336 S 0 0.0 0:00.00 dhcpcd
2617 root 20 0 1816 600 524 S 0 0.0 0:00.00 hald-addon-acpi
2808 root 17 0 1816 576 496 S 0 0.0 0:00.02 hald-addon-stor
2863 root 19 0 2283m 374m 15m S 0 11.4 2:29.23 java
2945 root 16 0 2808 632 508 S 0 0.0 0:00.00 kdm
2954 root 15 0 275m 12m 3648 S 0 0.4 0:00.62 X
2966 root 17 0 3148 1172 892 S 0 0.0 0:00.01 kdm
[/img]

[jaime.pinto]

Your cpu is 100% idle, you have 2GB of free RAM and no swap at all. No indexing going on. Very intriguing ...
This sounds like a job for one of the Scalix fellows on this forum.

[kmeyer2]

Hi jaime,

thanks for your interrest in that problem.

I'll wait for other people which have some ideas .. (I hope there are some)

BR Amok

[jillrae]

When my SWA runs slow I double check to make sure postgres is running:

Type lsof -i:5733

And then ps -ef|grep postmaster. You should see a short list. If postgres is not in the list, you need to startit.

Go to /etc/init.d, open a shell and type ./scalix-postgres start

Sign into SWA and see if that hlped with the speed problem

You can run the ps -ef|frep postmaster to see if postgres is listed now.

I have to do this everytime I reboot my server. Haven't figured out why.

jillrae

[kmeyer2]

hi jillrae,

you are great! The swa is fast and nice now!

I'll go and check the other problems ... many many thanks ..

BR Amok

[kmeyer2]

following testings:

there are no damaged send emails till yet. I hope that problem is fixed too.

At the big Mailbox with over 15400 mails:
I copied 9400 (all in the postbox) into another folder to reach more spead on login into the swa.
But I think the swa isn't created to manage copy or delete-jobs like that. The swa got problems - doesn't react on "folder clicks" - doesnt refresh any folder.
After stopping and restarting the services (scalix/scalix-tomcat) - a lot of mailes are transfered to the new folder (It needs a lot of time) .

I told that clerk that he has to use his outlook for jobs like that.

When I look at the tool "top" there is the ln.imap41d running. Is it real that the swa uses "imap" to copy - create etc? Cause imap isn't fast! (surely not on my system)

any ideas?

BR amok

[daniel.sand]

following testings:

there are no damaged send emails till yet. I hope that problem is fixed too.

At the big Mailbox with over 15400 mails:
I copied 9400 (all in the postbox) into another folder to reach more spead on login into the swa.
But I think the swa isn't created to manage copy or delete-jobs like that. The swa got problems - doesn't react on "folder clicks" - doesnt refresh any folder.
After stopping and restarting the services (scalix/scalix-tomcat) - a lot of mailes are transfered to the new folder (It needs a lot of time) .

on scalix 11.1.0 SWA reacts really slow, but only on mailboxes with heavy package ( 10k mails, 10+ calendars, huge load of contacts...etc )

SWA screws up on loading this mailbox, soap errors and i can barely move after its finished loading and clicking away the erros.

[kmeyer2]

so after a few hours the - job was done and everything works fine.
It needs really a long time to cut and paste 9400 Emails into another folder.
The hard - disk lite burns all the time. (The whole e-mail box has 1900Mb.) And during this job you better do not click on other folder - or the close button - cause the IE/Java will crash. (by the way, scalix is still doing his copy-job after IE down!) .

Is it normal, that an sxexport has much more speed than the creating and transportation into a new folder with 9400 e-mails ? (copy 9400 mails needs 2-3 hours / sxeport 1-2 minutes for the whole mbox)

So what is the reason? Using imap? or is there a bug? need help ...

BR Amok

[Shredder]

SWA does you imap internally for its connection to the server.

Shredder

[kmeyer2]

hi shredder

in the file : /var/opt/scalix/m2/s/sys/general.cfg .. are two lines as imap - options.

It says:
# These tweaks limit the number and rate of IMAP connections to the
# server. The IMAP_CONNECTION_LIMIT simply restricts the total number of
# connections to the server -- note that many IMAP clients have several
# connections for each IMAP session. The IMAP_CONNRATE_LIMIT restricts
# the rate at which clients can connect to the server, in this case, at
# most ten connections per second; if clients try to connect faster
# than that, the IMAP server simply slows down the rate at which it will
# accept new connections.
IMAP_CONNECTION_LIMIT=500 (I've changed to 5000)
IMAP_CONNRATE_LIMIT=10 (I've changed to 100)

Here is the "top" result as I started to delete 800 messages:
23604 60534 17 0 22724 6196 4272 D 4 0.2 0:01.11 in.imap41d
...there is only a virt memory of 22724 for this process
is there a way to tune up the in.imap41d ? I found nothing else than the general.cfg
at the moment .. (SWA works okay, but not fantastic at this time and the swa needs to be re-logged to let show the "new 800 messages" correctly.)

Best regards Amok ..

[Shredder]

Does the Virtual Memory increase as the command runs?

Shredder

[kmeyer2]

hi shredder

no it is not increasing, it comes and is the same, it looks like a concrete max value - definied somewere, I don't know. Sometimes there appeares a second and third value of the imap process, between 18000 an the 22724 mem but I can not say if it belongs to the copying process, or another user that uses the swa.

hopeful Amok

[kmeyer2]

Hi shreder,

now I found the slapd.conf in then directory /var/scalix.../s/sys/.

there you can spezify more mem etc.
then
omoff imap
omon imap ---to restart the imapd

since I conifigured higher values the swa works better - faster on big email - grounds and better imap-connections from the outlook2000 - clients.

Hope that helps a few other people too! - now I look what happend the next days ..

BR - virgin's care - Amok