sendmail segfaults

[borenius]

Hello there,

We're trying out Scalix 11 RC2 on a Fedora Core 5 Athlon 64 architecture.
We've tried sendmail versions 8.13.5 and the latest, 8.13.8 releases but with both we're getting the same error, which is that sendmail segfaults.
Note that throughout the message, we have replaced the real domain with DOMAIN due to extreme paranoia

Code: Select all

/var/log/messages
Dec 17 20:13:08 serv0 kernel: sendmail[3276]: segfault at 0000000055822788 rip 00002b0823670826 rsp 00007fffffa6e640 error 4


We've set it up so that sendmail listens to the loopback interface and smtpd listens to port 25 normally.

Code: Select all

sendmail  2901     root    4u  IPv4  10095       TCP localhost.localdomain:smtp (LISTEN)
omsmtpd   2937     root    4u  IPv4  10204       TCP DOMAIN.UNDISCLOSED:smtp (LISTEN)


In other words, telnetting to port 25 on the looback interface will yield:

Code: Select all

[root@serv0 ~]# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 serv0.DOMAIN ESMTP Sendmail 8.13.8/8.13.8; Sun, 17 Dec 2006 20:19:58 +0200
HELO test
250 serv0.DOMAIN Hello serv0.DOMAIN [192.168.2.130], pleased to meet you
MAIL FROM: test@DOMAIN
Connection closed by foreign host. <<SEGFAULT>>


and when telnetting to the network interface

Code: Select all

[root@serv0 ~]# telnet 192.168.2.130 25
Trying 192.168.2.130...
Connected to serv0.DOMAIN (192.168.2.130).
Escape character is '^]'.
220 serv0.DOMAIN ESMTP Scalix SMTP Relay 11.0.0.417; Sun, 17 Dec 2006 20:23:03 +0200 (EET)
HELO test
250 serv0.DOMAIN Hello serv0.DOMAIN [192.168.2.130], pleased to meet you
MAIL FROM: test@DOMAIN
250 test@DOMAIN... Sender ok
RCPT TO: test@DOMAIN <<SEGFAULT>>

We've tried with several clean installs of both Scalix (always RC2) and different versions of sendmail. We've had sendmail up and running for several months on the same box previously.

Any ideas regarding this issue would be greatly appreciated.
The strange thing is that sendmail segfaults even if we telnet to smtpd, and it segfaults in a different place.
As soon as we remove the MAILER(scalix)dnl line from sendmail.mc (and remake sendmail.cf, restart the service etc) the segfaults stop, but of course also make the whole system meaningless.

EDIT: Here are the relevant lines of sendmail.mc

Code: Select all

[root@serv0 mail]# grep -v "dnl " sendmail.mc
divert(-1)dnl
include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
VERSIONID(`setup for linux')dnl
OSTYPE(`linux')dnl
define(`confDEF_USER_ID',``8:12'')dnl
define(`confTO_CONNECT', `1m')dnl
define(`confTRY_NULL_MX_LIST',true)dnl
define(`confDONT_PROBE_INTERFACES',true)dnl
define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnl
define(`ALIAS_FILE', `/etc/aliases')dnl
define(`STATUS_FILE', `/var/log/mail/statistics')dnl
define(`UUCP_MAILER_MAX', `2000000')dnl
define(`confUSERDB_SPEC', `/etc/mail/userdb.db')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confAUTH_OPTIONS', `A')dnl
define(`confTO_IDENT', `0')dnl
FEATURE(`no_default_msa',`dnl')dnl
FEATURE(`smrsh',`/usr/sbin/smrsh')dnl
FEATURE(`mailertable',`hash -o /etc/mail/mailertable.db')dnl
FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable.db')dnl
FEATURE(redirect)dnl
FEATURE(always_add_domain)dnl
FEATURE(use_cw_file)dnl
FEATURE(use_ct_file)dnl
FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl
FEATURE(`access_db',`hash -T<TMPF> -o /etc/mail/access.db')dnl
FEATURE(`blacklist_recipients')dnl
EXPOSED_USER(`root')dnl
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl
FEATURE(`accept_unresolvable_domains')dnl
LOCAL_DOMAIN(`DOMAIN')dnl
MAILER(smtp)dnl
MAILER(procmail)dnl
MAILER(scalix)dnl


[Xlot]

EDIT: To save you time searching/reading, this issue affects x86_64 versions of sendmail from FC5 and FC6. See further down this thread for details and a workaround.

Hi,

Did you get a resolution to this problem? Its one I'm encountering too - I had a thread in the now-locked Feedback forum for version 11 here:

http://www.scalix.com/community/viewtopic.php?t=5330

Since that thread, I've installed a clean new machine with FC5 X86_64, updated all packages on the host from the fedora-updates repo, and then installed java 1.5.0-9, and then scalix 11 GA for FC5 (along with the prereqs).

The same problem occurs - I still get the segmentation faults from sendmail in the system log, and outbound internet messages consequently sit in the Internet Mail Gateway queue, not going anywhere.

I was previously running Scalix 11 Beta 1 with no issues - I'm about to remove Scalix 11 GA from my test server and install Beta1 to see if it works, to make sure its not my environment (pretty sure its not - dns etc. are all left untouched and running on other hosts).

Receiving mail from the internet works just fine, as does sending to domains hosted in the scalix environment (i.e. using a client to send to another local address routes correctly and messages appear in the relevant mailbox)

The only changes to sendmail config were adding entries to my local-host-names file to support the domains I'm running.

Code: Select all

 kernel: sendmail[17053]: segfault at 0000000055824668 rip 00002aaaab6e5826 rsp 00007fff0f4deac0 error 4


[Xlot]

So I cleaned all references to scalix off the new FC5 host, including removing message stores etc.

Then I installed Scalix 11 Beta1 from scratch (filename: scalix-core-fc5-11.0.0-beta.tgz).

It works - I no longer get the segmentation fault with sendmail, and outbound internet mail is routed properly, and delivered externally to my test gmail address.

So.. what changed since beta1?

I just noticed that Beta1 still used the old /sbin/sendmail -bs transport in the unix.mapper file.

Code: Select all

transport-service = /usr/sbin/sendmail -bs


This is instead of the 11 GA's unix.mapper which has:

Code: Select all

transport-service = inet:localhost:25

So I'm guessing the problem lies with ldapmapper? Perhaps I need to install the i386 version of a package to support the Scalix ldapmapper?

[Xlot]

And just to confirm, I upgraded my working Beta1 installation to GA11.

Code: Select all

kernel: sendmail[11729]: segfault at 0000000055824ed8 rip 00002aaaab6e5826 rsp 00007fff3f47a460 error 4

The problem is back.

Also, to confirm, I'm running no firewall on the host (its behind a firewall, and is port-forwarded to so that port 25 from an external interface reaches it). I also have Selinux disabled.

[borenius]

Thank you for your reply.
Seems we're experiencing the same problems on the same OS and pretty much the same hardware. We haven't delved further on the problem since the server is in production and cannot have any downtime during normal working hours.
I guess the easiest solution is to wait for the next version now.

[Xlot]

Thank you for your reply.
Seems we're experiencing the same problems on the same OS and pretty much the same hardware. We haven't delved further on the problem since the server is in production and cannot have any downtime during normal working hours.
I guess the easiest solution is to wait for the next version now.

I suspect its something as simple as needing to install the i386 version of a package or 2 to make the ldapmapper happy - given that I noticed the FC5 build is for the i386 architecture.

Hopefully one of the excellent scalix support people can advise on this!

I now have a separate FC5 64-bit test system to experiment on without having to take down my mail server, so I'm happy to try things out to aid in getting a solution.

In the meantime, I've reverted my main server to Scalix 11 Beta1 so that outbound is working.

[Xlot]

Ok, after running with Beta1 for a while, I just spent a bit of time investigating this issue further.

I found a successful workaround - replace the sendmail binary with the 32-bit version of it.

i.e. do the following:

# remove the existing sendmail binary, but not all the things that require sendmail
rpm -e sendmail --nodeps

# then grab the 32-bit sendmail from an Updates repository, and install it
rpm -ivh sendmail-8.13.8-1.fc5.i386.rpm

Then just use "omsendin" to reapply the scalix bits to sendmail.mc and sendmail.cf, and it all works.

I would like to know what it is about the 64-bit binary thats causing the segmentation fault - but at least I have the release version of Scalix 11 working now, instead of the fairly broken Beta1.

[ScalixSupport]

We've encountered a similar problem on RHEL3 with no updates. See bug:

http://bugzilla.scalix.com/show_bug.cgi?id=14327

I'll ask that 64 bit FC5 be tested again.

Regards,
Don

[ScalixSupport]

Is this a fresh install of FC5 or have updates been applied? If so, which updates.

Thanks,
Don

[Xlot]

Is this a fresh install of FC5 or have updates been applied? If so, which updates.

Thanks,
Don

Originally I hit this problem on FC6 - thinking it may be FC6 that was the problem, I then built a clean FC5 test host.

It was an install off the FC5 x86_64 DVD, followed by a "yum update" to update packages from the Fedora Updates repo.

The only package other than the Scalix ones that wasn't from the Fedora Core or Fedora Updates repo was the Java Runtime Environment, which came from Sun (1.5.0-9).

[jch]

There's a bug in the FC5 and FC6 x86_64 sendmail that makes it crash as soon as you do something with LDAP. I logged a bug (https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=206288) as soon as I found it, and while a fix was posted pretty quickly, it hasn't been pushed out yet.

The bug contains details of what you need to do to solve the problem and it's nice and easy to do. I've added a comment to that bug indicating that it would be nice to get the new sendmail RPMs pushed out, but at the time of writing there's no response.

jch

[kennethburgener]

This is an alternative to installing the 32bit edition of sendmail...

The problem has to do with sendmail 8.13.8-1 (and prior?) and LDAP. Scalix 11.0.3 just happens to want this configuration. Using sendmail 8.13.8-2 will fix this problem, but at the moment the sendmail 8.13.8-2 package is not available for Fedora Core 5 x86_64 edition, and there is no RPM for FC5 x86_64 that I could find. There is a source RPM available though. For a quick tutorial that got me up and running:

1. $ uname -a
Linux mail.mydomain.com 2.6.20-1.2312.fc5 #1 SMP Tue Apr 10 15:14:58 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
2. Uninstall Scalix
3. $ yum remove sendmail sendmail-cf
4. $ wget http://people.redhat.com/twoerner/BZ/20 ... .1.src.rpm
5. $ rpmbuild --rebuild sendmail-8.13.8-2.1.src.rpm
1. You will likely have to install some dependencies:
$ yum install rpm-build db4-devel hesiod-devel openssl-devel cyrus-sasl-devel openldap-devel
6. $ cd /usr/src/redhat/RPMS/x86_64/
7. $ rpm -Uvh sendmail-8.13.8-2.1.x86_64.rpm
8. $ rpm -Uvh sendmail-cf-8.13.8-2.1.x86_64.rpm
9. Reinstall Scalix

I assume this would also work for FC6 x86_64 users as well.

[jch]

That should work Couple of comments though:

You don't need to uninstall scalix while you're building and installing sendmail.

You may need to combine the final two rpm -Uvh commands: install both sendmail and sendmail-cf together -- on some versions at least there's a dependency that requires the same version of both to be installed and rpm will complain if it tries to have two different versions iinstalled.

I can confirm that it works for FC6 x86_64 though ... that's exactly the version I'm running and I know at one stage I built it myself.

jch

[kennethburgener]

This is probably true, but I experienced problems when I attempted removing sendmail from under Scalix. So for simplicity I choose to write up the method that worked flawlessly the first time.

[jch]

Ahh ... I missed the very first "yum remove sendmail" -- I blame it on Doctor Who. You don't want to do that And then you won't need to uninstall Scalix either. But you definitely will need to modify the installation to say:

rpm -Uvh sendmail-8.13.8-2.1.x86_64.rpm sendmail-cf-8.13.8-2.1.x86_64.rpm

Anyway, it's good to know that you can get it working on FC5.

jch