Error Message "Java heap space"

[lamers]

Hi

At the morning, my SWA stopt working with the error message "Java Heap Spacer". After this I couldn't start the swa again.
I have restartet the scalix-tomcat server and recieve this error message.

Using JRE_HOME:
29.04.2008 12:04:37 org.apache.catalina.startup.Catalina stopServer
SCHWERWIEGEND: Catalina.stop:
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:344)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:435)
Killing: 21964

I have take a look in scalix logs and found many errors like this in the last time.
Whats the Problem with java or catalina?

[lamers]

since them my scalix-tomcat crashes every hour.

The scalix-swa.log:

Code: Select all

2008-04-30 12:36:23,462 ERROR [HttpRequestHandler.sendResponse:293] ip: (ip unavailable); username: henrich; message: Failed Response

Could not write SOAP response: org.apache.catalina.connector.ClientAbortException: null
        at com.oddpost.soap.SoapResponseImpl.flushAndHandleException(SoapResponseImpl.java:371)
        at com.oddpost.soap.SoapResponseImpl.send(SoapResponseImpl.java:350)
        at com.oddpost.server.HttpRequestHandler.sendResponse(HttpRequestHandler.java:291)
        at com.oddpost.server.HttpRequestHandler.handleRequest(HttpRequestHandler.java:262)
        at com.oddpost.server.SoapServlet.doPost(SoapServlet.java:29)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at com.oddpost.server.filter.HttpConfFilter.doFilter(HttpConfFilter.java:181)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
        ... skipping the last 17 lines as irrelevant...
Caused by: ClientAbortException:  java.net.SocketException: Broken pipe
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:366)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:352)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:392)
        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:381)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:83)
        at com.oddpost.soap.SoapResponseImpl.serializeElementListToStream(SoapResponseImpl.java:204)
        at com.oddpost.soap.SoapResponseImpl.flush(SoapResponseImpl.java:292)
        at com.oddpost.soap.SoapResponseImpl.flushAndHandleException(SoapResponseImpl.java:359)
        ... 30 more
Caused by: java.net.SocketException: Broken pipe
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(Unknown Source)
        at java.net.SocketOutputStream.write(Unknown Source)
        at com.sun.net.ssl.internal.ssl.OutputRecord.writeBuffer(Unknown Source)
        at com.sun.net.ssl.internal.ssl.OutputRecord.write(Unknown Source)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Unknown Source)
        at com.sun.net.ssl.internal.ssl.AppOutputStream.write(Unknown Source)
        at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:747)
        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:432)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:347)
        at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:770)
        at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
        at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:580)
        at org.apache.coyote.Response.doWrite(Response.java:560)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:361)
        ... 38 more


Scalix-api.log:

Code: Select all

2008-04-30 19:56:32,766 ERROR [Imap.stopNotification:1484] Error stopping notification C45C7C217AEDC5239076AC1947EA8A99 DB460D530FC2374D410DF8583185EAC6
javax.mail.MessagingException: A6 NO X-NOTIF-CANCEL failure: can't cancel notifications for that client-id;
  nested exception is:
        com.sun.mail.iap.CommandFailedException: A6 NO X-NOTIF-CANCEL failure: can't cancel notifications for that client-id
        at com.sun.mail.imap.IMAPFolder.doCommand(IMAPFolder.java:2631)
        at com.scalix.api.imap.Imap.stopNotification(Imap.java:1479)
        at com.scalix.api.imap.Imap.shutdown(Imap.java:336)
        at com.scalix.api.imap.ImapService.cleanup(ImapService.java:65)
        at com.scalix.api.mailbox.ImapDataCache.synchFolder(ImapDataCache.java:660)
        at com.scalix.api.mailbox.ImapDataCache.getMessages(ImapDataCache.java:316)
        at com.scalix.api.mailbox.ImapDataCache.getMessages(ImapDataCache.java:293)
        at com.scalix.api.mailbox.Mailbox.getMessages(Mailbox.java:322)
        at com.scalix.api.rest.MailboxServlet.doGet(MailboxServlet.java:163)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:174)
        at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
        at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
        at com.scalix.api.PlatformDispatcherServlet.service(PlatformDispatcherServlet.java:74)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:133)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
        at java.lang.Thread.run(Unknown Source)
Caused by: com.sun.mail.iap.CommandFailedException: A6 NO X-NOTIF-CANCEL failure: can't cancel notifications for that client-id
        at com.sun.mail.iap.Protocol.handleResult(Protocol.java:328)
        at com.scalix.api.imap.XNotifyCancel.doCommand(XNotifyCancel.java:68)
        at com.sun.mail.imap.IMAPFolder.doProtocolCommand(IMAPFolder.java:2672)
        at com.sun.mail.imap.IMAPFolder.doCommand(IMAPFolder.java:2626)
        ... 37 more


Ihave read about this problem in this post [http://www.scalix.com/forums/viewtopic.php?t=10648&highlight=scalix+log]
but I become not the Error 23

Please can anyone help me? It's very important!

Thanks

[lamers]

Is ther no one who have a idea whats the Problem with Scalix-Tomcat?

At the moment, i must restart my scalix-tomcat two times a day. After a restart, tomcat runs about 4-5 hours and then the server didn't crash but it is unreachable.

Please have anyone a tip for me? It is very important!

[computernay]

I'm not sure if this will help, but this is what usually fixes Java heap space errors for me.

run this:

Code: Select all

omcheck -s -d > omcheck.s

Then look through the omcheck.s file it spits out. Usually for me there are lots of chmods and chowns to fix permissions. If it all looks ok, make the file executable and run it. Restart scalix-tomcat and see what that does for it. Sometimes running

Code: Select all

omscan -a -f -x

helps as well.

Hope that helps.

[lamers]

Hi,

Big Thanks for your reply! But it dosen't help. ;-(
I think it is not a problem with rights but rather with the Java memory space.
It seems that java allocate not enouth memory space at startup for scalix

[Valerion]

How much memory is allocated to tomcat? You need at least 256MB, but 512MB to 1GB is recommended.

[lamers]

Im not sure where i can look for the memory size of tomcat?

My system have 1.5 Gbyte physical memmory is that enouth? This machine is only running scalix.

[Valerion]

Look in /etc/opt/scalix-tomcat/scalix-tomcat.conf, at the -Xms and -Xmx parameters. Test it with eg. 512MB first (-Xms512m -Xmx512m) and then boost it higher if you still get issues. The more memory you give it the better tomcat and the applications will perform. I have mine at 1024MB and it works perfectly.

Code: Select all

JAVA_OPTS="-server -Djava.net.preferIPv4Stack=true -Xms1024m -Xmx1024m"

[lamers]

I think i have found my answer in an other post
http://www.scalix.com/forums/viewtopic.php?t=10709&highlight=java+lang+outofmemoryerror

I have added these two parameters to the java_opts
"-Xms1024m -Xmx1024m"
Now it looks like:
JAVA_OPTS="-server -Djava.net.preferIPv4Stack=true -Xms1024m -Xmx1024m"[/url]

[Valerion]

With only 1.5GB of RAM it may be risky to allocate 1GB to tomcat. I would leave it as is, but check the memory usage (free). If you start swapping excessively, or at all, then you need to reduce this. Don't go over half your total RAM in the beginning, I found. The machine I use this on has 4GB physical RAM, so 1GB for tomcat is not that significant.

[lamers]

ok i have wrote my last post bevor i seen yours.

now i have reduced it to 512mb

Here is my free output:

Code: Select all

             total       used       free     shared    buffers     cached
Mem:       1572864    1500600      72264          0      13548     373772
-/+ buffers/cache:    1113280     459584
Swap:      1951856       1048    1950808



[Eugene]

Hi Valerion, everyone

You say your server runs happily with 1GB RAM for Tomcat....how many SWA users do you have?

How many SWA users should a 3Ghz Xeon server with 3GB RAM be able handle comfortably?

Our server of the above spec is battling with its ~140 SWA users.

Webmin reports virtually no swapping whatsoever, there is free ram available but the wait time goes through the roof every now and again...

I guess a good place to start would be to determine whether our hardware can cope with the load.

Thanks in advance
Eugene

[kanderson]

LAMERS

Personally, I'd recommend adding RAM. Even if you have 32 bit hardware, you can still go to 2G of RAM with very little cost. I'd do that, and leave it at 1024 as Valerion showed. 512 will likely slow the frequency of the problems, but my guess is, that unless you have very few users, that won't solve it.


EUGENE

Your situation is different. I don't believe your problem is hardware based on the specs you posted in the other thread. We should move this back to that thread...

Kev.

[Eugene]

Hi Kev

Agreed...we should carry on on that thread.

I just tried to get as much input as possible to try and resolve my issues as soon as I can.

Thanks again,
Eugene