Number of open files

[Stepko]

Hi everyone,

I try to install scalix 11.0.2 for 5 - 10 users on a virtual host system running debian sarge. When I start scalix the number of open files in the system raises from 1200 to 8800. The server is limited to have a maximum of 10656 open files at the same time. If I logon to webmail scalix opens even more files so that the maximum is reached.
Is there a way to reduce the number of files scalix opens? Is it perhaps possible to run some parts of scalix using xinetd instead of running them as daemons? Can somebody check how many files are opened when scalix is running and 2 or 3 people are using webmail?

Any help is appreciated. Thanks in advance
Stepko

[ssiekmann]

There is no way Scalix would open that many files for 5-10 users. Can you show us omshowu -m all | wc -l please and also copy and paste the relevant portion of the Tomcat logfiles stating you are running out of handles.

Sascha.

[Stepko]

Scalix opens that many files on my system when it is running, even without a user logged in...
There are some things I don´t understand:
I got the number of open files from /proc/user_beancounters, which is 9680 when scalix is running. But "lsof | wc -l" shows only 5615. The limitation is set in /proc/user_beancounters, so that value counts.

"omshowu -m all | wc -l" returned 2. (I have only 2 users right now, sxadmin and sxqueryadmin)

The tomcat-log is at "/var/opt/scalix/<instance>/tomcat/logs/catalina.out", right? Nothing is written there, when the maximum number of open files is reached. So I switched the loglevel to debug.

I found some error-messages in other logfiles. Perhaps this is important, too??

Since you wrote that Scalix would not open so many files I still have hope to get it running

Thanks
Stepko


And here are the logfiles:
catalina.out (debug)

Code: Select all

INFO: Server startup in 12515 ms
Apr 3, 2007 9:30:05 PM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
SIS received contextDestroyed event
log4j:ERROR Error occured while converting date.
java.lang.NullPointerException
        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:488)
        at java.lang.StringBuffer.append(StringBuffer.java:302)
        at org.apache.log4j.helpers.ISO8601DateFormat.format(ISO8601DateFormat.java:124)
        at java.text.DateFormat.format(DateFormat.java:314)
        at org.apache.log4j.helpers.PatternParser$DatePatternConverter.convert(PatternParser.java:436)
        at org.apache.log4j.helpers.PatternConverter.format(PatternConverter.java:56)
        at org.apache.log4j.PatternLayout.format(PatternLayout.java:495)
        at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:292)
        at org.apache.log4j.WriterAppender.append(WriterAppender.java:150)
        at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:221)
        at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:57)
        at org.apache.log4j.Category.callAppenders(Category.java:187)
        at org.apache.log4j.Category.forcedLog(Category.java:372)
        at org.apache.log4j.Category.debug(Category.java:241)
        at com.scalix.sac.res.events.Notifier.run(Notifier.java:52)
Exception in thread "Notifier-thread-1" java.lang.NullPointerException
        at org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:95)
        at org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:182)
        at org.apache.log4j.helpers.PatternParser$ClassNamePatternConverter.getFullyQualifiedName(PatternParser.java:530)
        at org.apache.log4j.helpers.PatternParser$NamedPatternConverter.convert(PatternParser.java:503)
        at org.apache.log4j.helpers.PatternConverter.format(PatternConverter.java:56)
        at org.apache.log4j.PatternLayout.format(PatternLayout.java:495)
        at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:292)
        at org.apache.log4j.WriterAppender.append(WriterAppender.java:150)
        at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:221)
        at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:57)
        at org.apache.log4j.Category.callAppenders(Category.java:187)
        at org.apache.log4j.Category.forcedLog(Category.java:372)
        at org.apache.log4j.Category.debug(Category.java:241)
        at com.scalix.sac.res.events.Notifier.run(Notifier.java:52)
Apr 3, 2007 9:30:05 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: Failed shutdown of Apache Portable Runtime
Apr 3, 2007 9:31:02 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/j2sdk1.5-
sun/jre/lib/i386/server:/usr/lib/j2sdk1.5-sun/jre/lib/i386:/usr/lib/j2sdk1.5-sun/jre/../lib/i386
Apr 3, 2007 9:31:02 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 897 ms
Apr 3, 2007 9:31:02 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Apr 3, 2007 9:31:02 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.16
Apr 3, 2007 9:31:02 PM org.apache.catalina.core.StandardHost start
SWA:reading configuration @ /var/opt/scalix/v7/webmail/swa.properties
SIS received contextInitialized event
SIS initialized with config from /var/opt/scalix/v7/sis/sis.properties
Apr 3, 2007 9:31:04 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/var/opt/scalix/v7/tomcat/webapps/caa/WEB-INF/lib/j2ee.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: jav
ax/servlet/Servlet.class
CAA: scalix.instance is /var/opt/scalix/v7
CAA: caa-root-directory is /var/opt/scalix/v7/caa/
CAA: Loaded web.xml init params
CAA: caa-config-directory is /var/opt/scalix/v7/caa/config
CAA: log4j-init-file is /var/opt/scalix/v7/caa/config/log4j.properties
CAA: caa-framework-init-file is /var/opt/scalix/v7/caa/config/framework-config.xml
RES: Initializing log4j using file /var/opt/scalix/v7/res/config/log4j.properties
log4j:WARN No appenders could be found for logger (org.hibernate.proxy.BasicLazyInitializer).
log4j:WARN Please initialize the log4j system properly.
Apr 3, 2007 9:31:13 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on vs162037.vserver.de/62.75.162.37:8009
Apr 3, 2007 9:31:13 PM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/224  config=null
Apr 3, 2007 9:31:13 PM org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
Apr 3, 2007 9:31:13 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 11516 ms
Apr 3, 2007 9:41:24 PM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
SIS received contextDestroyed event
Exception in thread "Notifier-thread-1" java.lang.NullPointerException
        at com.scalix.sac.res.events.Notifier.run(Notifier.java:52)
Apr 3, 2007 9:41:24 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: Failed shutdown of Apache Portable Runtime
Apr 3, 2007 9:45:09 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent
INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/j2sdk1.5-
sun/jre/lib/i386/server:/usr/lib/j2sdk1.5-sun/jre/lib/i386:/usr/lib/j2sdk1.5-sun/jre/../lib/i386
Apr 3, 2007 9:45:09 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 954 ms
Apr 3, 2007 9:45:09 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Apr 3, 2007 9:45:09 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.16
Apr 3, 2007 9:45:09 PM org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
SWA:reading configuration @ /var/opt/scalix/v7/webmail/swa.properties
SIS received contextInitialized event
SIS initialized with config from /var/opt/scalix/v7/sis/sis.properties


The logfile contains the NullPointerException only once. The lines below are repeated several times.


/var/opt/scalix/<instance>/s/logs/daemon.stderr

Code: Select all

/opt/scalix/bin/sxmkindex: error while loading shared libraries: libom_nf.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_hash.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_exual.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_fstr.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_dstring.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_dstring.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_tf.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_sml.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_t61.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_ufcv.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_nfda.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_sfl.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_pool.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_tf.so: cannot open shared object file: Error 23
exec: No such file or directory
/opt/scalix/bin/mime.browse: error while loading shared libraries: libom_tfl.so: cannot open shared object file: Error 23
omslapdeng: error while loading shared libraries: libglib-2.0.so.0: cannot open shared object file: Error 23
/opt/scalix/bin/omeval: error while loading shared libraries: libom_msg.so: cannot open shared object file: No such file or directory
/opt/scalix/bin/index.browse: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_enc.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_ml.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_vi.so: cannot open shared object file: Error 23
/opt/scalix/bin/index.browse: error while loading shared libraries: libom_vista.so: cannot open shared object file: Error 23
omchown: error while loading shared libraries: libpthread.so.0: cannot open shared object file: Error 23
/opt/scalix/bin/unix.in: error while loading shared libraries: libom_aud.so: cannot open shared object file: No such file or directory


/var/opt/scalix/<instance>/s/logs/fatal

Code: Select all

SERIOUS ERROR           Service Router(Service Router) Sun Apr  8 11:13:08 2007
[SYS 43] Identifier removed
Pid of logging process: 10207
Current errno value: 43


The are lots of these errors in /var/opt/scalix/<instance>/s/logs/fatal

[Stepko]

I found out, that more than 3000 library-files are opened when scalix is running. All files are stored under "/opt/scalix/lib". The files are opened serveral times from different processes (omsessd, omctmon, omsmdm ....). Is this scalix normal behavior? Shouldn´t the library files only be loaded, when they are needed?

I found the same messages described here: http://www.scalix.com/forums/viewtopic.php?t=6978

To install scalix on my box I used the debian-packages. Is there any hope that the problem can be solved building scalix from sources?

Will scalix work on Suse Linux 9.3? I can install Suse 9.3 on my server, how many files are opened on a Suse 9.3 system at the same time (for 5-10 users).

Stepko

[swordfish]

Does anyone know how to check the open files on RHEL 4.0 Scalix server? I don't have /proc/user_beancounters on this version of Linux.

[Stepko]

Does anyone know how to check the open files on RHEL 4.0 Scalix server? I don't have /proc/user_beancounters on this version of Linux.

/proc/user_beancounters is a Virutozzo-specific thing. You can check the open files using "lsof | wc -l". Perhaps you need to install lsof.