CSV Attachment Displayed Inline in UUENCODE Format

[billrcorley]

I am recieving a CSV attachment from a sender in a MIME message. The message has been UUENCODED due to some special characters in the CSV file. The problem is that in Outlook this message is not being decoded, but rather is displayed completely inline with the CSV attachment in UUENCODE format.

Why is this not being detected, decoded and placed into an attachment rather then being left in the message as if it were plain text? Shouldn't this be detected by the server and translated prior to being placed in the message store?

Just for completeness, this message is displayed the same way in both the SWA and in an IMAP client as well. The UUENCODED material is still encoded and displayed inline.

[ScalixSupport]

Hi Bill,

What system is sending the message? Shouldn't it be using base64 rather than uuencoding? Base64 is more system transparent than uuencoding which is why it was chosen as the encoding scheme for MIME.

In the message header, what is the Content-Transfer-Encoding: for the uuencoded body part set to? Can you post a snippet of the MIME message which shows the body part header and the beginning of the uuencoded body part?

Thanks,
Rachel

[billrcorley]

The email is originating from one of our company partners, so I don't have access to what they are using to send this email. Because it is a routine email I suspect it is an automated system that is sending it.

We have been as supprised as you seeing this format. Haven't seen it for some time now. But our current email infrastructure is able to handle it transparently. (I am supprised at that too.)

Here is a snippet of the message headers and content:

Code: Select all

MIME-Version: 1.0
Content-Type: text/plain
Content-Disposition: inline

**********************************************************************************************
This e-mail and any attachments thereto, is intended only for use by the addressee(s) named herein and may
contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any
dissemination, distribution or copying of this email, and any attachments thereto, is strictly prohibited. If you
receive this email in error please immediately notify me at 555-555-5555 and permanently delete the
original copy and any copy of any e-mail, and any printout thereof.

**********************************************************************************************
begin 755 FILENAME.csv
M4%)/1%5#5"Q&04-)3$E462Q,3U1?240L3U=.15(L0U)%051)3TY?1$%412Q#
M4D5!5$E/3E]15%DL0U524D5.5%]15%DL3U!%4D%424].+%-(3U)47T1%4T,L
... content omitted ...
M("`@("`@-3$L("`@("`@("`@("`@-#$L*BHJ*BHL*BHJ*BHJ*BHJ*BHJ*BHL
M*BHJ*BHJ+"HJ*BPJ*BHJ*BHJ*BHJ+"HJ*BHJ*BHJ*BHL("`@("`@("`@("`@
$-#$-"@``
end


[ScalixSupport]

Hi Bill,

Okay, I see the problem. The MIME header is saying that this message is text/plain, so Scalix is pushing it through as such. Is there anyway your partner company can drop the MIME portion of the header? In other words, if they just sent the message through like this:

Code: Select all

From: system@partner.com
To: you@yourdomain.com
Subject: CSV

**********************************************************************************************
This e-mail and any attachments thereto, is intended only for use by the addressee(s) named herein and may
contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any
dissemination, distribution or copying of this email, and any attachments thereto, is strictly prohibited. If you
receive this email in error please immediately notify me at 555-555-5555 and permanently delete the
original copy and any copy of any e-mail, and any printout thereof.

**********************************************************************************************
begin 755 FILENAME.csv
M4%)/1%5#5"Q&04-)3$E462Q,3U1?240L3U=.15(L0U)%051)3TY?1$%412Q#
M4D5!5$E/3E]15%DL0U524D5.5%]15%DL3U!%4D%424].+%-(3U)47T1%4T,L
... content omitted ...
M("`@("`@-3$L("`@("`@("`@("`@-#$L*BHJ*BHL*BHJ*BHJ*BHJ*BHJ*BHL
M*BHJ*BHJ+"HJ*BPJ*BHJ*BHJ*BHJ+"HJ*BHJ*BHJ*BHL("`@("`@("`@("`@
$-#$-"@``
end

The message would be uudecoded properly.

Thanks,
Rachel

[billrcorley]

I have been attempting this and I am running into a problem. If I delete the MIME related headers from this message, or build a new message without any mime headers and attache this uuencoded section, I can see this message arrive in Scalix and then promptly disappear.

When I look at the audit log I can see it come in on the SMTP-Relay, but it never gets anything else logged. I have checked all of the queues and it doesn't show up there. I have checked the fatal log and I don't see any problems there. Nothing shows up with omshowlog.

I am at a loss to figure out where this message is disappearing to. Any thoughts?

At any rate I believe that your comment about why it is showing up in line rather than being decoded makes sense. I expect that to be the answer. As soon as I can figure out where it is going.

Thanks

[ScalixSupport]

Hi Bill,

What I've done to test this is:

Code: Select all

telnet SCALIX_SERVER 25
helo localhost
mail from: root@localhost
rcpt to: someone@myserver.com
data
From: root@localhost
To: someone@myserver.com
Subject: CSV

**********************************************************************************************
This e-mail and any attachments thereto, is intended only for use by the addressee(s) named herein and may
contain confidential information. If you are not the intended recipient of this e-mail, you are hereby notified any
dissemination, distribution or copying of this email, and any attachments thereto, is strictly prohibited. If you
receive this email in error please immediately notify me at 555-555-5555 and permanently delete the
original copy and any copy of any e-mail, and any printout thereof.

**********************************************************************************************
begin 755 FILENAME.csv
M4%)/1%5#5"Q&04-)3$E462Q,3U1?240L3U=.15(L0U)%051)3TY?1$%412Q#
M4D5!5$E/3E]15%DL0U524D5.5%]15%DL3U!%4D%424].+%-(3U)47T1%4T,L
... content omitted ...
M("`@("`@-3$L("`@("`@("`@("`@-#$L*BHJ*BHL*BHJ*BHJ*BHJ*BHJ*BHL
M*BHJ*BHJ+"HJ*BPJ*BHJ*BHJ*BHJ+"HJ*BHJ*BHJ*BHL("`@("`@("`@("`@
$-#$-"@``
end
.
quit



and it arrives correctly as an attachment called FILENAME.csv.

When you're sending, are you including the original RFC-822 header? If so, you need to bump the Message-ID: otherwise the SMTP Relay will think it's a dupe and not deliver it.

Thanks,
Rachel

[billrcorley]

I have resent the original message, changing the message id. In fact there was a couple of times when we didn't and the audit logs show a duplicate message line and don't deliver in that case. But with new message IDs they do not get delivered.

I have also created brand new messages from a unix command line, sending to a non-Scalix mailserver and then using the exact same input with a scalix user address. The messages to the non-Scalix server come through decoded. The mail to the Scalix server is lost.

In every case, literally, This message is in the audit log for the SMTP-Relay. But it never makes it into the log as unix-in, routing or delivery. It is just gone. I have confirmed as well that I can send plain messsages from the command line methods to Scalix accounts just fine. They do make it through.

Just not sure where else to look. But I really apprecxiate the help.

[ScalixSupport]

Hi Bill,

I'm assuming you checked in the error queue, but did you also check in the poison and smerr queues? Another you can do is turn the logging on local delivery, service router and the internet mail gateway:

omconfaud ld 15 sr 15 unix 15

then restart the services:

omoff -d0 ld sr unix
omon ld sr unix

Now send the message through. If you send it like the example I posted, does it still disappear?

Thanks,
Rachel

[billrcorley]

Those 3 items were already configured for level 9 logging. I increased them to level 15. No change. The email is still just gone after the SMTP-Relay.

I was thinking that it could be something like ClamAV. So I changed the logging for it to level 3 as well. There was no indication that Clam caught anything on that email. And I am not running spam assasssin or anything else I can think of.

Art (from Scalix) will be here in the morning on another topic. I will have him look at it while he is here.

Thanks again.

[ScalixSupport]

Hi Bill,

Just to verify, the RCPT TO: value you're entering is definitely a Scalix user, right? If you check in /var/log/maillog, you don't see the message in there do you? Let me know if Art doesn't find anything and we'll dig into this some more.

Thanks,
Rachel

[billrcorley]

The recipient was definitely a scalix user.

Art helped me track this and it definitely is having problems getting from the SMTP Relay to the unix.in queue. Here is a trace of the command session that I used in testing this.

Code: Select all

9 host1 > telnet scalix_svr 25
Trying 10.188.134.33...
Connected to scalix_svr.domain.com.
Escape character is '^]'.
220 scalix_svr.domain.com ESMTP Scalix SMTP Relay 10.0.0.175; Wed, 21 Jun 2006 19:28:13 +0000 (GMT)
helo localhost
250 scalix_svr.domain.com Hello host1.domain.com [10.188.37.146], pleased to meet you
mail from: senderid@host2
250 senderid@host2... Sender ok
rcpt to: mailid@scalix_svr.domain.com
250 Ok
data
354 Enter mail, end with "." on a line by itself (relay)
From: Bill
To: mailid@scalix_svr
Subject: Test Message

*************
some stuff
more stuff
*************

begin 755 FILENAME.csv
M4%)/1%5#5"Q&04-)3$E462Q,3U1?240L3U=.15(L0U)%051)3TY?1$%412Q#
M4D5!5$E/3E]15%DL0U524D5.5%]15%DL3U!%4D%424].+%-(3U)47T1%4T,L
... content ommitted ...
M("`@("`@("`T,2PJ*BHJ*BPJ*BHJ*BHJ*BHJ*BHJ*BPJ*BHJ*BHL*BHJ+"HJ
D*BHJ*BHJ*BHL*BHJ*BHJ*BHJ*BP@("`@("`@("`@("`T,0T*
end

.

.
^]
telnet> quit
Connection to scalix_svr.domain.com closed.


And here is the corresponding smtpd.log from that same session.

Code: Select all

SMTP Relay 10.0.0.175 started (5411): Wed, 21 Jun 2006 19:28:11 +0000 (GMT)
Accepted client 1 from  (host1.domain.com)
Cli 5411/1 sent  : 220 scalix_svr.domain.com ESMTP Scalix SMTP Relay 10.0.0.175; Wed, 21 Jun 2006 19:28:13 +0000 (GMT)
   
Cli 5411/1 rcvd: 'helo localhost'
Cli 5411/1 sent  : 250 scalix_svr.domain.com Hello host1.domain.com [10.188.37.146], pleased to meet you
   
Cli 5411/1 rcvd: 'mail from: senderid@host2'
Cli 5411/1 sent  : 250 senderid@host2... Sender ok
   
Cli 5411/1 rcvd: 'rcpt to: mailid@scalix_svr.domain.com'
  ... new process 5434
5411: Allocate new unix.in process 5434
Cli 5411/1 rcvd U: '220 Scalix ESMTP Ready.'
Cli 5411/1 rcvd U: '250 Ok'
Cli 5411/1 sent U: rcpt to:  mailid@scalix_svr.domain.com
   
Cli 5411/1 rcvd U: '250 Ok'
Cli 5411/1 sent  : 250 Ok
   
Cli 5411/1 rcvd: 'data'
Cli 5411/1 sent  : 354 Enter mail, end with "." on a line by itself (relay)
   
Cli 5411/1 sent U: data
   
Cli 5411/1 rcvd U: '354 Enter mail, end with "." on a line by itself.'
Cli 5411/1 sent U: Received: from localhost (host1.domain.com 10.188.37.146)
       by scalix_svr.domain.com (Scalix SMTP Relay 10.0.0.175)
       via ESMTP; Wed, 21 Jun 2006 19:28:22 +0000 (GMT)
   From: Bill
   
Cli 5411/1 sent U: To: mailid@scalix_svr
   
Cli 5411/1 sent U: Subject: Test Message
   
Cli 5411/1 sent U: Message-Id: <5413.10001150918112.scalix_svr.domain.com>
   
   
Cli 5411/1 sent U: *************
   some stuff
   more stuff
   
Cli 5411/1 sent U: *************
   
   begin 755 TOWER.csv
   
Cli 5411/1 got .
Cli 5411/1 sent U: M4%)/1%5#5"Q&04-)3$E462Q,3U1?240L3U=.15(L0U)%051)3TY?1$%412Q#
   M4D5!5$E/3E]15%DL0U524D5.5%]15%DL3U!%4D%424].+%-(3U)47T1%4T,L
   ... content ommitted ...
   M("`@("`@("`T,2PJ*BHJ*BPJ*BHJ*BHJ*BHJ*BHJ*BPJ*BHJ*BHL*BHJ+"HJ
   D*BHJ*BHJ*BHL*BHJ*BHJ*BHJ*BP@("`@("`@("`@("`T,0T*
   end
   
   .
   
SMTP Relay 10.0.0.175 finished (5411): Wed, 21 Jun 2006 19:28:56 +0000 (GMT)


Notice how the log shows that the '.' was recognized before it was sent.

Thanks,
Bill

[ScalixSupport]

Hi Bill,

It looks like in your example you have an extra .<crlf> in there. I've made a simple CSV file (basically 1,2,3,4,5 and uuencoded it). Can you use this text as the message:

Code: Select all

From: root@localhost
To: user@yourdomain.com
Subject: Testing

*************
some stuff
more stuff
*************

begin 644 test.csv
*,2PR+#,L-"PU"@``
`
end



Use the same HELO, MAIL FROM, RCPT TO and DATA commands that you were using and after pasting in my message after the DATA section type a period (with no leading or trailing spaces) on a line by itself.

Thanks,
Rachel

[billrcorley]

The extra .<crlf> you are seeing is me trying to get the smtp session to close. After entering the data I pressed '.' and enter but it did not end the data entry the way it should have. I had to ctrl-] to get to a telnet prompt and quit the telnet session. Things were wrong way before the extra .<crlf>.

But on a better note I have finally figured out why I was not getting uuencode to work at all. But it is not clear why it should be a problem.

The attachments I am getting are originating on a Windows box and the original CSV file has the Windows line endings, both the CR and LF. If I take the uuencoded data and decode it on a UNIX machine, then run dos2unix on it to remove the extra line endings, and finally reencode the file with uuencode, I can send and recieve the file just fine as a decoded CSV file attachment. And it doesn't matter if the file is in a MIME segment or not.

Of course for my testing I was using the recieved file. That is why I could not get anything to work, mime encode or not. With the above modifications it works fine.

It seems to me that the extra line ending character in the original CSV file is causing the problem. But that doesn't really make any sense. That has to be a very common thing.

So why does the smtp session hang on that character? Thoughts?

Another note ... I am running ClamAV. Not sure why that would make a difference, but thought you should know.

[jandrews]

I am still experiencing this same problem. Unfortunately we don't have the option to decode, dos2unix, encode before the messages go out.
Is there any other way to work around this or any further trouble shooting I can do to help find a solution.

--John

[dkelly]

Can you detail which problem you are seeing ?

The reason I ask is that Bill's problem was related to the fact that the uuencoding did not conform to accepted standards and there is no blank line between the end of the uunencoding and the "end" character. Our decoding didn't take this into account and would go into a permanent loop.

If you are an Enterprise or SBE customer, we can provide you with a patch that addresses this.

If it's not that problem, ignore what I said above and tell me what it is

Cheers

Dave