Imap cache issues

[techsharp]

Hello,

Just about everyone in the company is having problems with their webmail and not seeing all the messages in the inbox. I clear the imap-cache and it works for the day and then returns the next day.

I really would like a solution that works and not having to reset everyones imap-cache everyday.

Thank you

[mikevl]

Hi

What version of Scalix are you using?

Mike

[techsharp]

Hi Mike,

Currently running 11.3 - looks like they have a nice fix in 11.4 - will be upgrading next week.

I did do a restart of Tomcat and that seemed to work for a bunch of users as well.

[anthony.hardy]

I am seeing the same thing in 11.4. I was NOT seeing this in 11.3 though:(. Tomcat has started to hang up on us as well in 11.4. We only have 768M allocated to Tomcat, but I have more RAM ordered so we can give it 3-4G and see if that helps with speed and / or imap cache.

[skygriffin]

Hi,

we are starting to see the same issue with 11.4 and SWA. The web interface takes a long time to load eventually resulting in an empty InBox.

[schmoe90]

You might want to check that there aren't a bunch of zombie in.imap41d processes confusing everything.

[bcarruthers]

I am seeing this problem too. My users login and everything works fine until the email content is loaded to be shown in the content window.

My logs say things along the lines of below for the users that are effected.

ERROR IMAP Server Da(IMAP Server Pr) 07.10.08 16:32:45
[OM 24070] Debug message for Lab use :
mbcl_connect: Failed to connect to unix socket: /var/opt/scalix/mv/s/temp/mime_browse
User Name: Malcolm Jackon / mailserv/CN=Malcolm Jackson

What is the problem? It seems to be a degredation issue as restarting scalix cleaned things up? Memory leak? Not cleaning up TCPIP connections and flooding?

Regards,
Brett

[alvaro]

I just upgraded from 11.3 to 11.4 and things got worse with SWA.
Same symptoms .

[zmediaguru]

I'm also experiencing issues with lots of zombie IMAP processes. Last week the total count of IMAP processes spiraled to +500 in one day (the average is 50-60). When I shutdown the IMAP server (omoff imap) I still had tons of imap processes out there that I had to kill. Once I restarted the imap service (omon imap) everything returned to normal for awhile.

The same problem cropped back up last night, I caught the server at +200 processes, followed the steps above, and everything returned to normal. Eventually with all these processes out there, the system runs out of resources and users are no longer able to connect via webmail or SWA.

I'm still waiting on feedback from Scalix on what could be causing all of these IMAP processes out there.

[alvaro]

Perhaps you could try clearing the IMAP cache for your users.
There is an article in the Scalix Wiki on how to.

[mikevl]

Hi

Did clearing the IMAP cache clear up your problems?

Mike

[toni31]

Same problems here, omshowlog shows OM 3433 The open container reference...
Users get error messages , when deleting or sending messages in Thudnerbrid via imap. New mails aren't visible in Iinbox.
It's good how to know to fix this, but it's much better to avoid such corruption.
Is it possible to never use the imapcache?

Thanks
Flep

[florian]

Nope. The IMAP folder cache is not a feature, but part of the core IMAP server design. There is clearly some bug involved here and we're trying to chase it down. It's been an ugly one as we don't have a repeatable scenario to produce the corruption and once it's there, there is no evidence to describe how it came to that point. We have put some additional logging code into 11.4.1 hoping that this may give us some further information on the root cause of the issue. Once we understand and fix the issue, we would provide a patch for this as quickly as possible, but as per above unfortunately we are not there yet.

If somebody has a way to get that cache into corrupted state reproducibly, starting with a clean mailbox, please let us know.... I'll then assign you a very distinctive 'Scalix Master Of Corruption' logo!

Florian.

[mmas]

Hi,

scalix 11.4.1 same problem with a user, he can view "inbox" but not "sent items" or "delete items". If I go with mouse on "sent items" a popup appear telling me that 5058 messages are present in "sent items" but no messages are showed. He has same problem also to authenticate with message "username/password incorrect" for rhis problem I need to restart tomcat.

I noticed these errors in /var/opt/scalix/sx/tomcat/logs/scalix-swa.log when I access to the mailbox via swa and click on "sent items":

2008-08-06 09:16:23,633 ERROR [MailFolderTree.get:119] MIME:Problem getting MailFolder
com.oddpost.soap.SoapFault: faultCode: SOAP-ENV:SERVER.MessagingException; faultString: can't open folder [#Scalix/Oddpost]

; detailName: MessagingException; detailMessage: can't open folder [#Scalix/Oddpost]

; detailDebugInfo: javax.mail.MessagingException: A15 NO cannot SELECT that mailbox;
nested exception is:
com.sun.mail.iap.CommandFailedException: A15 NO cannot SELECT that mailbox
at com.sun.mail.imap.IMAPFolder.open(IMAPFolder.java:945)
at com.scalix.swa.service.MailServices.selectFolder(MailServices.java:235)
at com.oddpost.server.mail.MailFolder.<init>(MailFolder.java:78)
at com.oddpost.server.mail.MailFolderTree.get(MailFolderTree.java:116)
at com.oddpost.server.objects.UserInfo.load(UserInfo.java:108)
at com.oddpost.server.partner.Authenticate.init2(Authenticate.java:319)
at com.oddpost.server.partner.Authenticate.init(Authenticate.java:265)
at com.oddpost.server.partner.PartnerImplementations.createAuthenticate(PartnerImplementations.java:213)
at com.oddpost.server.SoapAuth.isAuthenticated(SoapAuth.java:175)
at com.oddpost.server.HttpRequestHandler.handleRequest(HttpRequestHandler.java:196)
at com.oddpost.server.SoapServlet.doPost(SoapServlet.java:44)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.oddpost.server.filter.HttpConfFilter.doFilter(HttpConfFilter.java:181)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
... skipping the last 18 lines as irrelevant...
Caused by: com.sun.mail.iap.CommandFailedException: A15 NO cannot SELECT that mailbox
at com.sun.mail.iap.Protocol.handleResult(Protocol.java:328)
at com.sun.mail.imap.protocol.IMAPProtocol.select(IMAPProtocol.java:642)
at com.sun.mail.imap.IMAPFolder.open(IMAPFolder.java:869)
... 37 more
; isHeaderFault: false; faultActor: null
at com.oddpost.server.mail.MailFolder.<init>(MailFolder.java:80)
at com.oddpost.server.mail.MailFolderTree.get(MailFolderTree.java:116)
at com.oddpost.server.objects.UserInfo.load(UserInfo.java:108)
at com.oddpost.server.partner.Authenticate.init2(Authenticate.java:319)
at com.oddpost.server.partner.Authenticate.init(Authenticate.java:265)
at com.oddpost.server.partner.PartnerImplementations.createAuthenticate(PartnerImplementations.java:213)
at com.oddpost.server.SoapAuth.isAuthenticated(SoapAuth.java:175)
at com.oddpost.server.HttpRequestHandler.handleRequest(HttpRequestHandler.java:196)
at com.oddpost.server.SoapServlet.doPost(SoapServlet.java:44)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.oddpost.server.filter.HttpConfFilter.doFilter(HttpConfFilter.java:181)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:133)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Unknown Source)



2008-08-06 09:29:14,791 ERROR [MailFolderTree.get:119] MIME:Problem getting MailFolder
com.oddpost.soap.SoapFault: faultCode: SOAP-ENV:SERVER.MessagingException; faultString: can't open folder [Sent Items]

; detailName: MessagingException; detailMessage: can't open folder [Sent Items]

; detailDebugInfo: javax.mail.MessagingException: A7 NO cannot SELECT that mailbox;
nested exception is:
com.sun.mail.iap.CommandFailedException: A7 NO cannot SELECT that mailbox
at com.sun.mail.imap.IMAPFolder.open(IMAPFolder.java:945)
at com.scalix.swa.service.MailServices.selectFolder(MailServices.java:235)
at com.oddpost.server.mail.MailFolder.<init>(MailFolder.java:78)
at com.oddpost.server.mail.MailFolderTree.get(MailFolderTree.java:116)
at com.oddpost.server.module.SoapMail$StatCommand.exec(SoapMail.java:348)
at com.oddpost.server.module.SoapMail.status(SoapMail.java:213)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.oddpost.soap.SoapModule.invokeMethod(SoapModule.java:219)
at com.oddpost.soap.SoapRequestImpl.execute(SoapRequestImpl.java:145)
at com.oddpost.server.HttpRequestHandler.handleRequest(HttpRequestHandler.java:222)
at com.oddpost.server.SoapServlet.doPost(SoapServlet.java:44)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.oddpost.server.filter.HttpConfFilter.doFilter(HttpConfFilter.java:181)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
... skipping the last 18 lines as irrelevant...
Caused by: com.sun.mail.iap.CommandFailedException: A7 NO cannot SELECT that mailbox
at com.sun.mail.iap.Protocol.handleResult(Protocol.java:328)
at com.sun.mail.imap.protocol.IMAPProtocol.select(IMAPProtocol.java:642)
at com.sun.mail.imap.IMAPFolder.open(IMAPFolder.java:869)
... 40 more
; isHeaderFault: false; faultActor: null
at com.oddpost.server.mail.MailFolder.<init>(MailFolder.java:80)
at com.oddpost.server.mail.MailFolderTree.get(MailFolderTree.java:116)
at com.oddpost.server.module.SoapMail$StatCommand.exec(SoapMail.java:348)
at com.oddpost.server.module.SoapMail.status(SoapMail.java:213)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.oddpost.soap.SoapModule.invokeMethod(SoapModule.java:219)
at com.oddpost.soap.SoapRequestImpl.execute(SoapRequestImpl.java:145)
at com.oddpost.server.HttpRequestHandler.handleRequest(HttpRequestHandler.java:222)
at com.oddpost.server.SoapServlet.doPost(SoapServlet.java:44)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at com.oddpost.server.filter.HttpConfFilter.doFilter(HttpConfFilter.java:181)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:133)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Unknown Source)

[florian]

Are you sure it is the same problem, i.e. does clearing the IMAP folder cache fix the issue for the user?

As a more general update - we have a candidate fix for the cache corruption issue that we're starting to test internally today. As we still don't have a well-defined way to reproduce the problem, we don't know whether or not it addresses it in full, but we'll deploy it on our internal servers and our support team will then work with a number of select customers to test it out in production. If this goes well, we'll publish an updated server version.

Cheers,
Florian.