high load server slow after sxmboimp

[packets]

I do sxmboximp last 12:00AM (midnight) and then when I check around 11:AM. The load is still high even though sxmboximp is no longer running. Here is the load

Code: Select all

top - 12:23:17 up 8 days, 21:37,  1 user,  load average: 12.31, 14.13, 11.09
Tasks: 258 total,   1 running, 257 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.7%us,  0.7%sy,  0.0%ni,  2.2%id, 92.4%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3114632k total,  3017528k used,    97104k free,   159596k buffers
Swap:  5177336k total,      136k used,  5177200k free,  2297076k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
19496 root      25   0  583m 230m  15m S  4.3  7.6   0:51.74 java
20338 postgres  15   0 22468  11m 9.9m D  4.3  0.4   0:00.72 postmaster
22454 55032     17   0 19904 4168 2844 S  1.0  0.1   0:00.03 in.imap41d
21655 55063     15   0 22156 6688 3620 S  0.3  0.2   0:00.24 in.imap41d
21931 root      15   0  2428 1120  796 R  0.3  0.0   0:00.88 top
    1 root      15   0  2160  632  548 S  0.0  0.0   0:00.72 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:01.23 migration/0
    3 root      39  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0

I got 3GB of memory. Dual Core 1.6Ghz. Current login was 81 (omstat -u all | wc -l). I tried already restart scalix,scalix-tomcat and scalix-postgres but still it was high load

Any idea?

[PrisonMind]

Hi,

top is not the best tool for i/o activity

you should try iotop or with systemtap(redhat/centos)

cat disktop.stp

#!/usr/bin/env stap
#
# Copyright (C) 2007 Oracle Corp.
#
# Get the status of read/write every 5 seconds, print top ten entries
#
# This is free software,GNU General Public License (GPL); either version 2, or (at your option) any
# later version.
#
# Usage:
# ./iotop.stp
#

global io_stat
global read_bytes,write_bytes

probe syscall.read.return {
if ($return > 0) {
io_stat[pid(),execname(),uid(),ppid(),"R"] += $return
read_bytes += $return
}
}

probe syscall.write.return {
if ($return >0) {
io_stat[pid(),execname(),uid(),ppid(),"W"] += $return
write_bytes += $return
}
}

probe timer.ms(5000) {
printf("\n%-25s, %-8s%4dKb/sec, %-7s%6dKb, %-7s%6dKb\n\n",ctime(gettimeofday_s()),"Average:",
((read_bytes+write_bytes)/1024)/5,"Read:",read_bytes/1024,"Write:",write_bytes/1024)

/* print header */
printf("%8s %8s %8s %25s %4s %12s\n","UID","PID","PPID","CMD","T","BYTES")

/* print top ten I/O */
foreach ([process,cmd,userid,parent,action] in io_stat- limit 10)
printf("%8d %8d %8d %25s %4s %12d\n",userid,process,parent,cmd,action,io_stat[process,cmd,userid,parent,action])

/* clear data */
delete io_stat
read_bytes = 0
write_bytes = 0
}

probe end{
delete io_stat
delete read_bytes
delete write_bytes
}

[packets]

Thanks for the info. I'll try that one. However, it is weird. While I'm checking for a solution, load just drop to 0 to 3.