attachments are being renamed

[markrich]

We have a very odd problem with our new mail system.

When PDF attachments are received by the server they do not arrive with the correct name in the destination mailbox.

Doing some tests we noticed Webmail presents the attachment as attachement.pdf where Thunderbird shows it as 1.2.PDF.

The PDF should be report-server1.customername.pdf.

Dragging an old email from a local folder to the mailbox ones does not exhibit the same problem so that leads me to beleive the problem is with the way Scalix receives it's email. Is that through Sendmail?

It's quite important we fix this as we have many services which send reports from our Customers systems each day.

All help appreciated.

Marky

[florian]

can you capture the incoming message on the SMTP relay level by using the DEBUG_LOG=TRUE option in smtpd.cfg (put it in at the top, restart SMTP relay, find resulting file in /var/opt/scalix/??/s/tmp/smtpd-SMTP.log) and re-submitting such a message? after finished, don't forget to turn this off again as it will create large tracefiles.

once done, please copy/paste the MIME header for the body part, I'm interested in something similar to this:

Code: Select all

--------------090900040603080508020203
Content-Type: text/x-vcard;
   charset="utf-8";
   name="name.vcf"
Content-Disposition: attachment;
   filename="name.vcf"


Please note that it doesn't help if you get the mime source when looked at it from a scalix client - at that point it will already have passed through the Scalix server and possibly be modified. We need to look at it as it enters the system.

Florian.

[markrich]

I think I may be missing something obvious here.

My system has 4 copies of that file:

/opt/scalix/template/release.sys/smtpd.cfg
/var/opt/scalix/sa/s/release.sys/smtpd.cfg
/var/opt/scalix/sa/s/sys/smtpd.cfg
/var/opt/scalix/sa/s/tmp/smtpd.cfg

I am uncertain which is the one you mean. I tried the bottom two of the four and restarted SENDMAIL but the log only contains:

[2009-07-30 11:24:08] Logging Started pid 30780
[2009-07-30 11:31:21] Logging Started pid 31500
[2009-07-30 11:31:23] Logging Started pid 31513
[2009-07-30 11:31:29] Logging Started pid 31533
[2009-07-30 11:52:15] Logging Started pid 1572
[2009-07-30 11:52:17] Logging Started pid 1581

Nothing like what you mention in the last message.

Confused.

[markrich]

Never mind, I restarted all the Scalix services the log has started gathering data.

[florian]

the relevant file is

/var/opt/scalix/sa/s/sys/smtpd.cfg

the other ones are templates.

Florian.

[markrich]

Okay...we've had some thinking here and here's what we have learnt.

Example 1:

An email is sent in with an attachment from a webmail account.

In the MIME settings of that message coming in, via the log, the following is visable:

Code: Select all

--B_3331801349_858650
Content-type: video/x-flv; name="sxbackup.pdf"
Content-disposition: attachment;
        filename="sxbackup.pdf"
Content-transfer-encoding: base64

The attachment will arrive at the destination with the correct name.

Example 2:

In the MIME settings of a message sent in via the automated app Logicity the following is visable:

Code: Select all

----boundary_0_e31aeac6-1096-4be6-8dbc-bec22c887b32
Content-Type: application/octet-stream; name=vod_consumption.pdf
Content-Transfer-Encoding: base64

The attachment will arrive with the email but without name. Instead it has attachment.pdf or 1.2.pdf or attachment1234.pdf.

The obvious thing which stands out is the absence of a MIME command which states FILENAME= and it may be easy to point the finger at this lack of MIME and say there's the problem, it's with that app, were it not for some other facts. a) the old mail system based on POSTFIX and IMAP worked. b) the customers receiving this message from the external servers receive the attachment correctly through their mail servers.

The problem only occurs when the message reaches Scalix. At this point, the system looses the ability to apply NAME= to the attachment, which the other mail systems can.

Assuming this is the case then I am hopeful a config file exists someplace to allow this function.

All help appreciated.

[florian]

The problem is as-stated.

The message is not standards conformant; disposition=attachment with a filename= attribute is the correct passing of file-based attachments.

Scalix does not use MIME as it's internal format; the message needs to be translated to Scalix Container format. This translation uses a standards-conformant parsing. The conversion is necessary due to Scalix' multiple client approach, including Outlook, which does not rely on MIME at all.

The comparison to a Postfix system is irrelevant as Postfix does not interpret or change message bodies; message is provided to the IMAP client as stored. As such, interpretation of these headers will be subject to the client, some clients may accent non-standard-conformant headers, while others may not, the behavior is undefined. I assume you may have tested with something like Thunderbird, which may work in certain versions - other clients may not work and fail in various and unpredictable ways.

There is currently no config-level workaround or solution to allow this to be parsed correctly; if the system sending the message is under the same control, it is recommended to change this system to send out standards-conformant Headers.

Scalix has received a number of customer reports from customers using non-standards compliant applications that are generating such messages, typically dated fax software or some embedded devices like multifunction scanners/copiers/faxes that send these. To better interop with them, an improvement-level bug has been raised as https://bugzilla.scalix.com/show_bug.cgi?id=20326 and it is planned to improve Scalix' behavior for such situations in the next release of Scalix, 11.5, currently scheduled for sometime 2H/2009.

Cheers,
Florian

[markrich]

This leaves us in a very difficult position.

Our customers in Moscow use Exchange/Outlook and have no issues receiving the attachments. Likewise those in Australia. We moved to Scalix because we wanted to keep costs low and remain outside of a ActiveDirectory network. That cost saving now looks like it will cost us much.

The Logicity app is required to take a Crystal Report app, fill in the report and send it on. If Exchange has no difficulties understanding the attachment despite the lack of a MIME setting then why should Scalix also not, especially as you suggest MIME is not the defining action here.

Something must dictate the name of the attachment if MIME is not it. And if so, why are we looking at it?

I am perhaps cluthcing at straws here but this app is mission critical to our business and I really need a resolution which could work.

Is there perhaps a mechanism which could be applied to assign the missing attribute to the email as it comes in so that Scalix can process it properly?

Find a way to work around the problem?

[florian]

Well, you can obviously put some software in front of Scalix that processes the message body. Scalix even provides an interface - in the form of the MILTER interface on the SMTP relay where such a plugin can be installed. I have no knowledge whether a plugin that can be configured to do such mapping exists anywhere or whether you will have to write your own code for that.

Short of that, if this is a true issue and you are a commercial Scalix customer with an active support entitlement, I would recommend that you open a formal support ticket on this. This would obviously be linked to the bug and would influence it's priority. Depending on your support status and type of entitlement, you may even have the option to see an escalation openened and possibly progress the proposed change to a point where you would get a hotfix or patch.

Tx,
Florian.

[markrich]

Okay...thank you for your help.

I have dropped a forum question at the Logicity website asking them to add the extra two headers to outbound email on the next release.

We don't really have the time to hack something ourselves and we would probably keep our support instances from the paid for version we have for something later.

[markrich]

The Logicity people were excellent.
They produced a new version of the app within 2 hours of me posting the problem on their forums.
The new app now provides the missing details to the headers and Scalix now understands the attachment names.

Thank you for your help.