SMTP Relay Fatal Error

[dresdn]

Seems like our server is having more problems (rough morning).

Anyways, the SMTP Relay died and it came back up when doing a omoff and omon.

I'm just concerned as to *why* it died as I don't want to keep having to monitor all the processes to be sure everything is working okay.

Code: Select all

ERROR                          SMTP Relay    (SMTPD Relay Pr) 05.30.06 11:18:49
[OM 6027] Timed out waiting for database
Current errno value: 2
        -> dr_ACIModContextFlags
        <- dr_ACIModContextFlags
        <- ul_FindAuthId
        -> ul_CloseUL
        <- ul_CloseUL
        -> ul_OpenUL
        -> dr_ACISetDefaultContext
        -> dr_ACIModContextFlags
        <- dr_ACIModContextFlags
        <- dr_ACISetDefaultContext
        <- ul_OpenUL
        -> ul_SetBadPwdCount
        <- /build/10.0.1.3/src/lib/ombase/vi/vi_locks.c:352[3,16082]
        <- /build/10.0.1.3/src/lib/ombase/vi/vi_locks.c:201[3,16082]
        <- /build/10.0.1.3/src/lib/ombase/ul/ul_Locks.c:53[3,6027]
        <- /build/10.0.1.3/src/lib/ombase/ul/ul_ChgAttr.c:223[3,6027]


WARNING                        Internet Mail (Incoming      ) 05.30.06 11:18:56
[OM.UX 1401] Cannot read message data.
        <- tf_AppendRecord
        <- mp_PutObjectFile
        <- ux_InParseMimeFile
        <- ux_InParseMimeContent
        -> im_UnlinkItem
        -> im_ItemRef2FName
        <- im_ItemRef2FName
        <- im_UnlinkItem
        -> im_GetNewItemNum
        <- im_GetNewItemNum
        -> ux_InParseContentHeader
        -> GetMailText
        <- /build/10.0.1.3/src/bin/ux/ux_mimein.c:284[102,1401]
        <- /build/10.0.1.3/src/bin/ux/ux_mimein.c:5426[102,1401]
        <- /build/10.0.1.3/src/bin/ux/ux_in.c:4766[102,1401]
        <- /build/10.0.1.3/src/bin/ux/ux_in.c:2187[102,1401]


Thanks,
Mike

[ScalixSupport]

What does your mail configuration look like? Is the scalix server what your mx record points to or is there a smarthost routing to the scalix server? Is your server experiencing abnormal load?

What OS? What scalix version?

I'm just concerned as to *why* it died as I don't want to keep having to monitor all the processes to be sure everything is working okay.

Monitoring is generally a good idea. We provide a monitoring script called ommaint. You will find it in the admin_resource_kit of your install medium.

Thanks,
Don

[dresdn]

What does your mail configuration look like? Is the scalix server what your mx record points to or is there a smarthost routing to the scalix server?

Scalix is the mail server for our organization (84 mailboxes). It is what the MX record points to, and it's what users use for outgoing mail.

Is your server experiencing abnormal load?

In terms of mail load, no. But in terms of real load, yes. The server has been hovering around a load of 8 or 9, with it going as high as 41. Everyone's at lunch now, and it's down near 1.

What OS? What scalix version?

Red Hat Enterprise Linux v4 using Scalix 10.0.1.

Monitoring is generally a good idea. We provide a monitoring script called ommaint. You will find it in the admin_resource_kit of your install medium.

Thanks for pointing me to this. I'll take a look at it.

[ScalixSupport]

With the load going to 41 would you say that is normal for your environment?

The error is indicating that the smtpd process wasn't getting a response from the server. I think the warning is spurious -- just a by product of the error.

The smtpd process is listening for incoming messages and for messages submitted via webmail, imap and pop clients.

Is it possible you are getting some sort of DOS attack or being used as a spam relay? What do you get with

lsof -i:25

Have you modified your smtpd.cfg? Specifically have you uncommented any of these lines:

Code: Select all

RELAY Log_Reject ALL

# extra rules added to prevent open relay usage
RECIPIENT Log_Reject *@*@*
RECIPIENT Log_Reject *%*
RECIPIENT Log_Reject *!*
RECIPIENT Log_Reject *#*@*

Thanks,
Don

[dresdn]

Hi Don,

Thank you for your responses so far.

With the load going to 41 would you say that is normal for your environment?

Absolutely not. The thing is, yesterday was actually the day we went live with our Scalix install. Some people have huge mailboxes (400-800M). Since the imapsync re-ordered the mails, each client had to re-download all of their mail. I tried to stagger this as much as possible, but of course people just kept trying to check their mail.

I'm willing to bet that was what was causing the problem. I'm not sure why the relay process died rather than the imap processes, but under load strange things happen. ;)

Is it possible you are getting some sort of DOS attack or being used as a spam relay? What do you get with

lsof -i:25

DOS'd internally I think. lsof is now just returning:

Code: Select all

# lsof -i:25
COMMAND    PID USER   FD   TYPE DEVICE SIZE NODE NAME
omsmtpd  12083 root    4u  IPv4 513137       TCP mail.contentconnections.com:smtp (LISTEN)
sendmail 16538 root    4u  IPv4 101373       TCP localhost.localdomain:smtp (LISTEN)
sendmail 17243 root    9u  IPv6 589698       TCP mail.contentconnections.com:47983->216.52.184.240:smtp (SYN_SENT)


Have you modified your smtpd.cfg? Specifically have you uncommented any of these lines:

Code: Select all

RELAY Log_Reject ALL

# extra rules added to prevent open relay usage
RECIPIENT Log_Reject *@*@*
RECIPIENT Log_Reject *%*
RECIPIENT Log_Reject *!*
RECIPIENT Log_Reject *#*@*

Yes, all of those line are uncommented with some additional ones for our internal people.

The one thing that does concern me is the number of imap processes running.

Code: Select all

]# ps -ef | grep imap41d | wc -l
101


Could there be imap sessions not timing out fast enough, or dying, etc. that are taking up or wasting resources? Where can I look to tune the imap settings?

Thanks in advance, and I believe I was a little panicy yesterday ;)

-Mike

[ScalixSupport]

I can understand panic if things aren't behaving as expected so no worries.

You've just migrated. Did you use

omtidyallu -m

to build the mime cache in advance of opening the server for user sign on? If not then swa has to do it for the user when they log in which will put extra stress on the server.

Typically you will see four imap processes per user. There is the core imap process which is the inbox. Then as other folders are accessed other imap processes are spawned. The idea is to increase performance so the data doesn't have to be constantly reread. Much of the memory used by the imap processes is shared so you needn't be terribly concerned by what you are seeing. It's normal.

Let's see how things go the next few days.

Regards,
Don

[dresdn]

You've just migrated. Did you use

omtidyallu -m

Actually, no I didn't. Most (if not all) users are actually using Thunderbird for now, since most are Macs and I really only want to do one thing at a time here. Would that have helped even though SWA wasn't being initially used?

One thing that has concerned me is that for some reason IMAP "times out" every so often (only once today). What happens is if I close Thunderbird and load it back up, it sits there at "Sending authentication information ..." To resolve it, I had to do a omoff imap, killall imap41d (since it took forever to stop them all), and then omon imap.

Not sure if this is something to worry about or not, but just thought I would mention it.

Thanks,
Mike