Heavy IMAP Usage is Killing the Server

[thecowster]

Thanks Valerion. I have reported this to Scalix Support, so hopefully it will get resolved one day.

[seanyseansean]

As a FYI - I have just come back from 2 week holiday, and neither myself nor thecowster have heard anything from either scalix or our reseller on this issue.

We tried 11.4 today and the problem is not fixed.

Is it possible to get any idea on whether this issue is being looked at? Killing the server with an out-of-control imap process by importing a single email would be top priority i'd have thought.

Thanks,

Sean

[florian]

Can you guys send me a PM or email with the Support Ticket Number?

Tx,
Florian.

[florian]

Just as a general information.

I am now in private communication with seany. Turns out that so far he hasn't worked with the Scalix Technical Support team directly, but just talked to his reseller about it. Don't know why this wasn't then forwarded/processed to us yet, but will find out. Also don't know anything about thecowster's ticket yet, but I can say that there is at least no case of negligence on Scalix Support's side of things with regards to Seany's issues, so that at least clarifies that part as I am pretty sensitive to our teams not providing responses in appropriate timeframe.

Cheers,
Florian.

[seanyseansean]

Ah, apologies. We never stood to say that Scalix were at fault - only that our reseller had promised to put the issue forward to Scalix, and we had heard nothing from either party since.

Scalix have actually been pretty good

[florian]

Ouch, no offense taken, so apologies to you if my wording seemed to be indicating this.

This was basically just because we had a period last year during the Scalix-Xandros transition, where a good number of people were unhappy with our responsiveness. While we still see a few cases falling through cracks sometimes - given the volume of stuff we handle, that's probably a necessary evil - in general we have worked hard and invested quite a bit over the last 12 months to make those processes more stable, if not foolproof.

For this reason I really like to understand every single case where we miss our targets that way.

Cheers, thx again,
Florian.

[florian]

Ouch, no offense taken, so apologies to you if my wording seemed to be indicating this.

This was basically just because we had a period last year during the Scalix-Xandros transition, where a good number of people were unhappy with our responsiveness. While we still see a few cases falling through cracks sometimes - given the volume of stuff we handle, that's probably a necessary evil - in general we have worked hard and invested quite a bit over the last 12 months to make those processes more stable, if not foolproof.

For this reason I really like to understand every single case where we miss our targets that way.

Cheers, thx again,
Florian.

[Hibbelharry]

Hi People,
we're evaluating scalix again and have quite some clients connecting via imap. Everything works okay when people are using Outlook or SWA. But when imapd is stressed scalix breaks down. Imap Clients get messages like unable to get MYRIGHTS (seen very often) and omcheck reports broken imap cache. It's not working again before at least imap cache is cleaned up. Some files in the mailstore also belong to wrong unknown users with uid in a range of 60000+ while scalix hat uid in the normal under 1024 system range.
I can get that bad behaviour triggered easily when i move some bigger mail folders via imap to the server. It didn't make any difference which imap client i used, I tested with thunderbird and imapsync. I also found no single mail which explicitly repeatable kills the server. It's just happening regularly when the system is stressed. omshowlog offers some slightly different messages, only OM 3433 is relatively common.
CPU Usage is spiking a lot while importing messages happens. Server is a Dual Core Opteron, memory usage is okay, Disk usage seems also okay. Scalix is running on an uptodate fresh installation of fedora 9.
Is this still some kind of known issue ? What should I do to solve this ?

Greetings
Hibbelharry

[florian]

Hibbel,

thanks for your feedback; I think this needs to be looked at in detail as there are various different things referred to in the post.

First, some files with UIDs in the 60000 range showing up are normally. Scalix assigns each mailbox owner on a server a unique unix UID out in that range out of a pool; these are not registered in /etc/passwd. See omadmidp and omshowidp man pages for details. So this is healthy behaviour.

Second, all omcheck looks at is file permissions; it does not detect any cache corruption. If you can provide an example of the omcheck command line you're using and the error message it outputs we may be able to look at it, but again I would assume that whatever it shows is benign and we may have missed out on some file in the description config file it compares against, so it's probably not a problem and, if anything at all, a cleanliness issue.

Third, if you were experiencing an IMAP cache corruption, you would see certain messages in omshowlog and in logs/fatal, so if you have any specific information here, it would be good to know that to analyze what may be causing this. If so, it would also be relevant to know more about the type of load you're applying to the system.

Fourth - what version of Scalix 11 are you using? We recently published 11.4.1-U1, which was a patch update to 11.4.1, and one of the 3 fixes in there adresses specifically a type of IMAP cache corruption seen with a number of customers; therefore, you should be using the very latest version. here.

And finally - if this is an evaluation for commercial use, it may be good to get in touch with us because we may also be able to evaluate any issue you're experiencing through our Tech Support and see if there is anything environmental involved in the issue.

Tx,
Florian.

[florian]

... can you indicate what version of Scalix you were using when seeing those issues? Again, the difference between 11.4.1 and 11.4.1-U1 may be relevant here.

Florian.