Scalix-tomcat wont start

[mito]

Hello,

I have just upgraded from Scalix 10 to Scalix 11 (finally) and am having yet another problem that I didn't have in either of my last 3 attempts (2 trials that eventually went smooth, and one production upgrade that failed miserably, which resulted in a restore of scalix 10).

I am running on CentOS 4.4, I did the installation/upgrade via the command line, and it went through without problems. At the end of the install I tried to access the webmail, but it wasn't active. I checked the scalix services via 'service scalix status' and all processes were stopped. so I started them all via 'service scalix start' and 'service scalix-tomcat start', and both returned OK apropriately. I then tried to access the page again and still could not get anything (getting 404 error). So at this point I check the status of the services again to see if anything isn't running... all scalix processes are running fine, but the scalix-tomcat service says it's not running.

My first easy attempt to fix it was to reboot the server and let everything start as normal, that didn't work. I then checked to make sure scalix was running, it was. Checked to make sure scalix-tomcat was running, it wasn't. checked further and see this:

Code: Select all

# ps -ef|grep tomcat
root      3810     1  1 16:37 ?        00:00:25 /usr/java/jre1.5.0_06/bin/java -server -Djava.net.preferIPv4Stack=true -Xms512m -Xmx512m -Dscalix.instance=/var/opt/scalix/ml -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/var/opt/scalix/ml/tomcat/conf/logging.properties -Djava.endorsed.dirs=/opt/scalix-tomcat/common/endorsed -classpath /usr/java/jre1.5.0_06/lib/tools.jar:/opt/scalix-tomcat/bin/bootstrap.jar:/opt/scalix-tomcat/bin/commons-logging-api.jar -Dcatalina.base=/var/opt/scalix/ml/tomcat -Dcatalina.home=/opt/scalix-tomcat -Djava.io.tmpdir=/var/opt/scalix/ml/tomcat/temp org.apache.catalina.startup.Bootstrap start
root      7259  3850  0 17:03 pts/0    00:00:00 grep tomcat


This then makes me think that it is, running, but if so, then why am I getting this?

Code: Select all

# service scalix-tomcat status
Instance (mail) is not running

I did check the Scalix 11 FAQ wiki, and ran the 'omwebconf -l AMERICAN' command hoping it would fix this, but it didn't.

Anyone have any ideas? Thanks!

[mito]

Oh, I found a post that said to use this command to check a log, and I'm getting some strange errors...

Code: Select all

# omshowlog

WARNING                        Administration(ommodu        ) 01.27.07 16:25:57
[OM.DMON 2205] Failed to connect to Session Monitor: mdc_connect: No such file or directory


SERIOUS ERROR                  Administration(ommodu        ) 01.27.07 16:25:57
[OM 8047] Cannot contact Session Monitor Daemon
Current errno value: 2
        <- nm_AppendFieldMem
        <- nm_PutFieldMem
        -> nm_ParseORN
        <- nm_ParseORN
        <- ul_utUnpackUserEnt
        <- ul_FindPrimeUser
        -> nm_ParseORN
        <- nm_ParseORN
        -> nm_PutFieldMem
        -> nm_AppendFieldMem
        <- nm_AppendFieldMem
        <- nm_PutFieldMem
        -> usr_PidSignon
        <- usr_PidSignon
        <- /build/10.0.5.3/src/lib/cu/cu_mod.c:1491[100,8047]


WARNING                        Administration(omgetmsg      ) 01.27.07 16:26:27
[OM 23559] An unrecognized line (561) was found when an OMSTDID line was expected.


WARNING                        Administration(omaddent      ) 01.27.07 16:27:07
[OM 29022] Could not generate internet address or part of address
for entry /mail,mydomain using domain mydomain.com and rule "C" <gS>


WARNING                        Shared memory (Shared memory ) 01.27.07 16:27:32
[OM 24142] Conversion from KSC5601 to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:27:32
[OM 24142] Conversion from ISO2022CN to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:27:32
[OM 24142] Conversion from SJIS to IA5 not found.


WARNING                        Administration(omstat        ) 01.27.07 16:32:43
[OM.DMON 2205] Failed to connect to Session Monitor: mdc_connect: No such file or directory


WARNING                        Administration(omstat        ) 01.27.07 16:32:43
[OM.DMON 2205] Failed to connect to Session Monitor: mdc_connect: No such file or directory


WARNING                        Administration(omstat        ) 01.27.07 16:32:43
[OM.DMON 2205] Failed to connect to Session Monitor: mdc_connect: No such file or directory


WARNING                        Shared memory (Shared memory ) 01.27.07 16:32:54
[OM 24142] Conversion from KSC5601 to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:32:54
[OM 24142] Conversion from ISO2022CN to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:32:54
[OM 24142] Conversion from SJIS to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:37:18
[OM 24142] Conversion from KSC5601 to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:37:18
[OM 24142] Conversion from ISO2022CN to IA5 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 16:37:18
[OM 24142] Conversion from SJIS to IA5 not found.


Any idea why these errors are happening, what I can do to fix them, and if they are causing my problems with not being able to connect to the webmail/SAC ?

[mito]

Ok, well I had tried doing a reconfigure of the scalix-tomcat earlier, but I finally went ahead and totally uninstalled scalix and the scalix-tomcat, and then reinstalled it.

After doing so, the scalix-tomcat does say that it is running when checked with 'service scalix-tomcat status' command, but I am still unable to get any sac or webmail access to work (still showing 404 errors).

I have noticed that now every time I restart the scalix services (service scalix restart) I get these errors from omshowlog

Code: Select all

WARNING                        Shared memory (Shared memory ) 01.27.07 18:04:01
[OM 24142] Conversion from KSC5601 to ISO8859_1 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 18:04:01
[OM 24142] Conversion from ISO2022CN to ISO8859_1 not found.


WARNING                        Shared memory (Shared memory ) 01.27.07 18:04:01
[OM 24142] Conversion from SJIS to ISO8859_1 not found.

Anyone have any ideas?

P.S. During this time pop3 access is available, but 90% of my users use the webmail exclusively, so this isn't good enough... help!

[mito]

For the time being I downgraded back to Scalix 10.

Which works, for the most part, but now it will not accept new mail...

[kanderson]

Downgrading from 11 to 10 is HIGHLY not recommended. There are 1 way changes made to upgrade the message store, so unless you retrieve the message store from a restore, I'd be worried.

In any case. Probably the first thing I'd check is that apache is running. Both apache and tomcat need to be running, or you won't see http connections.

Kev.

[mito]

Well, I'd already downgraded by the time I'd posted. The downgrade did actually work btw, at least as far as the message store itself went.

I ended up having problems afterwards where all of my incoming email was being stored in the sendmail inbound queue waiting to be scanned by my MailScanner, but it kept having problems after problems.

I ended up copying the inbound message queue to a network store and reverting back to my old OS drive again, and then dumping the messages into there and letting it deliver them etc, which worked out good.

I never did figure out / fix this problem where I couldn't get the sac or webmail to be recognized. As soon as I saw that tomcat was actually running after the reinstall I was so happy, but it didn't fix it after all.

I just don't understand it though... I've done 3 tests now where I've done a complete wipe/reload of my OS, and then restore my scalix etc and then upgrade it, and all 3 tests worked smoothly, without any major problems (though I have to admit, I only tested receiving emails in one of those 3 tests). But here, where I try to do a wipe/reload of my production system, I get problems after problems (I was at the office over 12 hours today working on it today) and every time I have to revert back to my original backup-copy. Which is bad, because this backup copy isn't under any kind of raid etc!

Anyways, that's off topic... but I've listed my entire installation/upgrade procedure on my website for my own documentation. Everything is correct to my knowledge... I just don't know what's going wrong, and why always on my production system?

[kanderson]

Glad that it's working...

Kev.

[mito]

Well, was working... had to revert back to my backup due to the mail never being delivered...

Anyways, the httpd was running every time I checked it, and I was able to get the standard apache test pages without problem. I was even able to get my MailWatch (web-interface/monitor for MailScanner) up and running, just not the sac/webmail.

I'm thinking of doing yet another wipe/reload of it again just to get rid of the half-installed versions etc. That and I've never installed Scalix *after* I've had MailScanner etc up and running, I don't know what kinds of issues that might bring up...

[florian]

the area of highest risk in your scenario of a scalix 10 to scalix 11 upgrade is actually the scalix-tomcat-connector (or apache mod_jk). In Scalix 10, we just placed a file into the apache config tree from the installer, in Scalix 11 we've cleanly wrapped everything in packages.

one thing to try for such an upgrade is to uninstall all scalix 10 components, just leaving the message store intact. then deleting everything that's left in /etc/opt/scalix* and /opt/scalix, only leaving /var/opt/scalix.

Then... making sure all scalix config files and the mod_jk and mod_jk2 and workers.properties and workers2.properties files are gone from /etc/httpd.

then starting a fresh scalix 11 install, preserving the message store. that should work.

before you try all this, please have a backup.

and kev is right, once a message store has ever been touched by a running 11.x system, there is no safe way to get back to scalix 10 in place. the containers will have been upgraded and i assume this is why your local delivery stopped working. check the logs and you should see ugly things.

Florian.

[mito]

Thanks Florian for the info.

I did uninstall all and remove /etc/opt/scalix/ before reinstalling, but I guess I forgot about /opt/scalix and I didn't even know about the new files in the /etc/httpd so those stayed there as well.

It doesn't look like I'll be able to try a production upgrade again for another 2 weeks or so, so it gives me plenty of time to retry this all on my test systems over and over and over again :).

I just wish it would break in my testing and not my production for once! My tests upgrade perfectly!

Thanks again!

[kanderson]

Uninstall 10 completely, but leave the message store.

Then delete /opt/scalix and /etc/opt/scalix.

Then do the upgrade to 11.

What does "cat /etc/hosts" and "hostname -f" show?

Kev.

[mito]

Uninstall 10 completely, but leave the message store.

Then delete /opt/scalix and /etc/opt/scalix.

Then do the upgrade to 11.

Well here's the question... will a Scalix 11 install find a Scalix 10 message store if Scalix 10 has never been installed? I ask because what I am doing is a clean wipe of the server (I want a different hosting OS than what I have currently) so I always install 10 to restore it, and then upgrade. I guess I never even tried to restore the message store but then install straight to Scalix 11... I could try that next.

What does "cat /etc/hosts" and "hostname -f" show?

I can't access it right now as it is restored to the old drive, but I tried it two ways, one way as:

Code: Select all

127.0.0.1 localhost.localdomain   localhost
x.x.x.x    mail.mydomain.com   mail

as well as just:

Code: Select all

127.0.0.1 localhost.localdomain    localhost

I tried it the correct way first (the first one I mentioned) and then tried it the 2nd way, both had the same problem with not seeing the sac/webmail. Then I thought that it could be because I had my firewall blocking access from my WAN IP's (I have it setup with 3 nics, 2 for 2 different ISP's, 1 for internal) BUT, then I remembered that both sac/webmail worked fine like that after I installed 10 but before I upgraded it to 11, while the firewall was in place blocking the nics, so it shouldn't be that.

Could the tomcat have gotten confused by my multiple nics/IP's? I wouldn't think so, but it's worth a thought...

[kanderson]

If you have a v10 message store at /var/opt/scalix, you can install 11 directly.

The instructions I gave you were designed to remove anything relating to your v10 install except the message store.

[mito]

Wow, I can't believe it, but I ran into this EXACT problem again....

even though I've done yet another 3 test upgrades since the last time this happened, and they all went perfectly... this time, yet again on the production machine in my production upgrade, I get this problem again...

Any ideas?

[florian]

most likely problem area is still hostname resolution. give us... hostname, hostname -s, hostname -f and the contents of /etc/hosts.

cheers,
Florian.