Sendmail expert advice needed

[grubi]

Hi.

During the last days I was affected by the following problem, which I
think everyone could be affected using scalix/sendmail and having lower
bandwidth internet connections (lets say less than 1 mbit/s). Meanwhile I have
done many tests and read a lot but did not find a suitable solution. I
would be glad if someone more experienced with sendmail could point me
in the right direction.

One user submitted one large mail (about 10 MB) for delivery to
30 external recipients (BCC) via Scalix SMTPD. When looking at
the Sendmail queue is saw that this mail was splitted into 30 seperate
mails and sendmail spawned one child process for every mail
delivery ending up in 30 processes trying to deliver the mail
simultaneously. The delivery processes will share the bandwith to the
internet for delivery which leads to the fact that every submission
takes a huge amount of time before finishing. On testing I found that
many receiving MTAs on the internet operate on rather short timeout
values for the whole mail transaction to prevent DOS attacks. This leads
to the fact that all submissions one after the other were terminated with
"dsn=4.0.0, stat=Deferred" and were requeued for submission. The
situation was even getting worse as I had a short queue runner
interval and end up having multiple queue runners picking up the
queued mails for resending leading to the initial problem of deferring
and requeueing undtil the 5 day limit is reached.

The easy solution would be to have a max number of children doing
delivery but this seems not to be possible to configure
(neither confMAX_DAEMON_CHILDREN nor confMAX_QUEUE_CHILDREN will do this).

Another solution would be to set the delivery mode to queue-only,
have a short queue runner interval (say 3 minutes) and limit the number
of queue runners e.g to 5 with confMAX_QUEUE_CHILDREN and
confMAX_RUNNERS_PER_QUEUE

This would have prevented the timeout/requeue problem however
until 25 of the 30 mails were delivered not even very small mails would have
been delivered. You also should ensure that confMAX_DAEMON_CHILDREN
is larger than confMAX_RUNNERS_PER_QUEUE in the case you use sendmail
also as receiving MTA, otherwise inbound mails are rejected until the number
of queue runners again drops lower than confMAX_DAEMON_CHILDREN
(in this case when 25 of the 30 mails were sent).

The only optimum solution I see at the moment would be to work in
queue-only mode as described above and have two queue groups, one
for small messages and one for large messages both with a different
number of queue runners assigned. The problem is how to write a rule
which assigns the message to the queue group according to it's size?
As I'm not familiar with the whole sendmail rule thing can anybody
show me how I could achive this goal.

If all this works maybe some good stuff for the wiki.

Regards,
grubi.

[swordfish]

I think this is not exactly Scalix topic. You rather visit the Sendmail forums and ask for advice there.

[dougp23]

There is a way to do this better in sendmail.
It involves creating mulitiple queue directories.
Def go to a sendmail forum and ask for some help.

You may also want to try and restrict users from sending 10M attachments to 30 people! I think that on low bandwidth systems, this is always going to be a problem for you. (and make sure your clamav settings aren't scanning super large attachments, they might be scanning that attachment slowing the whole thing down even more.)

[grubi]

I think this is not exactly Scalix topic. You rather visit the Sendmail forums and ask for advice there.

As Scalix and Sendmail are tied together closely I was not aware of that fact that this would be OT here. Instead I was under the impression that it would be interresting for other Scalix users as they could also be affected by this. I will take your advice and discuss this elsewhere.

grubi.

[grubi]

There is a way to do this better in sendmail.
It involves creating mulitiple queue directories.

This is not true. Multiple queue directories will not solve that problem.

Def go to a sendmail forum and ask for some help..

Already understood.

You may also want to try and restrict users from sending 10M attachments to 30 people! I think that on low bandwidth systems, this is always going to be a problem for you.

That's true, however this scenario will also match for larger bandwith systems under heavyer load. So it's a good idea to think about it in general.

I already found where to limit msgsize, but what is the best way to limit number of recipients. From what I understand how it works in sendmail it would not be a solution to do it there. For doing this in scalix I only found a way to limit numer of BCC addresses via message filter but not the total number of recipients.

(and make sure your clamav settings aren't scanning super large attachments, they might be scanning that attachment slowing the whole thing down even more.)

Already done of course although system load caused by calmav and amavis is not the problem here. But allways a good idea to keep it in mind

Regards,
grubui

[swordfish]

Grubi,

Don't take my advice as something bad. As this is a Scalix forum and your question is specific for Sendmail and not about interaction between Sendmail and Scalix the chances are that you'll find better answers in the Sendmail forum.