SWA running slower than molasses in January

[jillrae]

Got SWA up and running on Friday (See post "11.10.1 upgrade "broke" SWA) but today it is taking about 5-8 minutes for people's inbox to load and sometimes in doesn't even load at all. I checked to make make sure only one Tomcat was running (it is) I also looked to make sure the swa.soap.soapRequestTimeout=60 line was installed correctly, I aslso checked the CPU usage time and nothing is taking more than 1-2% or less time. What else should I check to see why SWA is taking so long to load. Basically my users can't use SWA.

Thanks

jillrae

[jillrae]

BTW, is there anyway to "un-do" the 11.0.1 upgrade and go back to 10?

[kanderson]

Scalix 11 is a one way upgrade. You CANNOT go back. The changes made to the mailstore are permanent. It's better to wait to try to resolve the slowness.

Does the PostgreSQL log in /var/opt/scalix/??/postgres show any errors?

[jillrae]

I didn't find a PostgreSQL log in /var/opt/scalix/??/postgres. But did find a postgresstart.log Here is its output:

2007-02-05 08:54:20 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "folder_status_pkey" for table "folder_status"
2007-02-05 08:54:20 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "message_pkey" for table "message"
2007-02-05 08:54:20 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "deleted_pkey" for table "deleted"
2007-02-05 10:31:47 ERROR: schema "sx_ba99d8b48badfa783e8171f0a51addca" does not exist
2007-02-05 10:31:47 ERROR: current transaction is aborted, commands ignored until end of transaction block
2007-02-05 10:31:47 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "folder_status_pkey" for table "folder_status"
2007-02-05 10:31:47 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "message_pkey" for table "message"
2007-02-05 10:31:47 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "deleted_pkey" for table "deleted"
2007-02-05 11:16:20 ERROR: schema "sx_cd2b47601d526187d3dc89baa2b2b15c" does not exist
2007-02-05 11:16:20 ERROR: current transaction is aborted, commands ignored until end of transaction block
2007-02-05 11:16:20 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "folder_status_pkey" for table "folder_status"
2007-02-05 11:16:20 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "message_pkey" for table "message"
2007-02-05 11:16:20 NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "deleted_pkey" for table "deleted"
2007-02-05 11:31:23 LOG: recycled transaction log file "0000000000000001"
2007-02-05 12:16:28 LOG: recycled transaction log file "0000000000000002"
2007-02-05 12:21:06 LOG: unexpected EOF on client connection
2007-02-05 12:21:06 LOG: unexpected EOF on client connection
2007-02-05 12:21:06 LOG: unexpected EOF on client connection
2007-02-05 12:21:06 LOG: unexpected EOF on client connection
2007-02-05 12:21:06 LOG: unexpected EOF on client connection
2007-02-05 12:22:06 LOG: received fast shutdown request
2007-02-05 12:22:06 LOG: shutting down
2007-02-05 12:22:08 LOG: database system is shut down
2007-02-05 12:25:00 LOG: could not bind IPv4 socket: Cannot assign requested address
HINT: Is another postmaster already running on port 5733? If not, wait a few seconds and retry.
2007-02-05 12:25:00 FATAL: could not create listen socket for "linux"

There were no other log files under /var/opt/scalix/??/postgres

[jillrae]

BTW what does the error 2007-02-05 11:16:20 ERROR: schema "sx_cd2b47601d526187d3dc89baa2b2b15c" does not exist
mean? What schema?

Do you need any more log outputs to help figure out what's going on?

[florian]

the schema "error" message is normal, it's just a namespace in the DB that is created for each users. the first time,it's not there but we try to access it, hence the error.

- is postgres running correctly? (lsof -i :5733, ps -ef | grep postmaster)
- does your scalix-api logfile (in tomcat/logs) have any errors about trying to connect to the database?

Florian.

[jillrae]

I ran lsof -i:5733 (no response) but then ran PS -ef|grep postmaster got the following:

root 7076 7075 0 08:38 pts/1 00:00:00 grep postmaster

I checked the scalix-api log. There are lots of :

-Problem encountered during authenication M00021 could not connect to database
-java.sql.SQLException: Connection could not be aquired from underlying databse
-599 FATAL [DbUtils.getSession:51] could not create database connection


All of these errors repeat many times over in the log.

Hope this helps.

[florian]

well, the first two commands indicate that your postgres database server is not running.

if you run /etc/init.d/scalix-postgres start, does the result for the ps and lsof commands change?

Cheers,
Florian.

[jillrae]

Started the ppstgress
Here are the results:

linux:~ # lsof -i :5733
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
java 4025 root 27r IPv4 797420 TCP linux.garrettcontainer.com:40972->linux.garrettcontainer.com:5733 (ESTABLISHED)
java 4025 root 364u IPv4 797423 TCP linux.garrettcontainer.com:40973->linux.garrettcontainer.com:5733 (ESTABLISHED)
java 4025 root 388u IPv4 797426 TCP linux.garrettcontainer.com:40974->linux.garrettcontainer.com:5733 (ESTABLISHED)
java 4025 root 390u IPv4 797429 TCP linux.garrettcontainer.com:40975->linux.garrettcontainer.com:5733 (ESTABLISHED)
java 4025 root 391u IPv4 797432 TCP linux.garrettcontainer.com:40976->linux.garrettcontainer.com:5733 (ESTABLISHED)
postmaste 8648 postgres 3u IPv4 797189 TCP linux.garrettcontainer.com:5733 (LISTEN)
postmaste 8654 postgres 8u IPv4 797421 TCP linux.garrettcontainer.com:5733->linux.garrettcontainer.com:40972 (ESTABLISHED)
postmaste 8655 postgres 8u IPv4 797424 TCP linux.garrettcontainer.com:5733->linux.garrettcontainer.com:40973 (ESTABLISHED)
postmaste 8656 postgres 8u IPv4 797427 TCP linux.garrettcontainer.com:5733->linux.garrettcontainer.com:40974 (ESTABLISHED)
postmaste 8657 postgres 8u IPv4 797430 TCP linux.garrettcontainer.com:5733->linux.garrettcontainer.com:40975 (ESTABLISHED)
postmaste 8658 postgres 8u IPv4 797433 TCP linux.garrettcontainer.com:5733->linux.garrettcontainer.com:40976 (ESTABLISHED)


linux:~ # ps -ef|grep postmaster
postgres 8648 1 0 09:17 pts/1 00:00:00 /usr/bin/postmaster -i -h linux -p 5733 -D /var/opt/scalix/lx/postgres/data -k /var/opt/scalix/lx/postgres/data
root 8794 8563 0 09:19 pts/1 00:00:00 grep postmaster


Looks like things are running now. Tested loging into SWA. Instead of the 1 minutes to show the folder list and another minute to show the INBOX messages (this is not exagerating, I timed it) it now takes about 5-7 seconds for the folders to load and the messages to come up. Big improvement.

Thanks

Will I have to double check to make sure this postgres database starts if I do a server e-boot?

[florian]

it should, you might want to see if it's set to do so using

Code: Select all

stingray:/var/opt/scalix/sy/s # chkconfig --list scalix-postgres
scalix-postgres           0:off  1:off  2:off  3:on   4:off  5:on   6:off

Should look similar for you. What Linux distro is this on?

Florian.

[jillrae]

Do you think the 11.0.1 upgrade I did maybe didn't quite install correctly? Maybe the downloaded file took a hit? Just thinking.....

[jillrae]

Sorry. Posted to wrong topic. Will move.

[florian]

very unlikely; the downloaded file is a tar.gz, which hardly unpacks when it is corrupted. we also publish the MD5 checksums on our website, those you could verify using the linux-supplied md5 tool.

you might want to check the latest /var/log/scalix-installer log if something went wrong during installation of the update.

Cheers,
Florian.

[jillrae]

Yes the scalix-postgres is set correctly.

thanks so much for all feedback. Still learning how everything works, so I ask a lot of questions.....

[friedmar]

have the same thing. Very, very slow webmail!

On 11 it worked like a charm.

Friedmar