Errors transfereing messges from old imap server to new

[SidebandSamurai]

Hello everyone,

I have attempted to transfer one e-mail account from the old imap server to the new imap server. While transfering imapsync, and iamp copy report errors.

Imaysync reports the following errors:

Code: Select all

Couldn't append msg #196 (Subject:[=?ISO-8859-1?B?VHdvIE5ldyBNaWNyb3NvZnSuIFBhdGNoZXMgUmU=?= =?ISO-8859-1?B?bGVhc2VkIEluY2x1ZGUgT25lIENyaXRpY2Fs?=]) to folder INBOX: Error trying to append string: 202 BAD syntax error

Couldn't append msg #197 (Subject:[FREE DOWNLOAD: Recover deleted files in seconds]) to folder INBOX: Error trying to append string: 203 BAD syntax error

Couldn't append msg #198 (Subject:[Your Remediator Security Digest Has Arrived]) to folder INBOX: Error trying to append string: 204 BAD syntax error

Couldn't append msg #199 (Subject:[New free network utility from Executive Software]) to folder INBOX: Error trying to append string: 205 BAD syntax error

Couldn't append msg #200 (Subject:[Research help needed]) to folder INBOX: Error trying to append string: 206 BAD syntax error

Couldn't append msg #201 (Subject:[Security Gumshoe Meets the Falcon Cracker ]) to folder INBOX: Error trying to append string: 207 BAD syntax error

The above is a small sample. imap sync ends up not transfering about 50% of the messages.

Imapcopy reported the following errors also:

Code: Select all

Error saving message 607, continuing copy, Server returned "BAD syntax error"
Error saving message 608, continuing copy, Server returned "BAD syntax error"
Error saving message 609, continuing copy, Server returned "BAD syntax error"
271 Messages copied, 342 Errors

1 User processed, 641 Messages copied, 386 Error(s)
2 Folder(s) created, 0 Folder create errors, 0 Folder not copied


I have attempted to use outllook to copy the messages, and that process locked up outlook.

On my last test I started monitoring the maillog log file, and nothing showed up there.

All the messages can be seen by the mail client, weather it be outlook or thunderbird or netscape 7.2 It seems I can't transfer them. Why?

Has anybody ran across this problem before?

Thank you all for your help

Sincerely,

Sideband Samurai

[jhamill]

we migrated an exchange 5.5 server for about 20 users over to scalix. This had about 10 years of mail :-) so we had a lot of issues getting it to work reliably. imapsync seemed to be about the best of the imap mapping tools I tried although far from perfect.

i found the thing that seemed to work best was doing a justfolders first so that all the structure was created so I made a little script to do this, then I did the messages.

imapsync --host1 192.168.20.1 --user1 MyDomain/blah --password1 SECrEt --host2 192.168.20.4 --user2 blah --password2 StillASecrtet --noauthmd5 --justfolders --nofoldersize

imapsync --host1 192.168.20.1 --user1 MyDomain/blah --password1 SECrEt --host2 192.168.20.4 --user2 jhamill --password2 StillASecrtet --noauthmd5

I still found some folders that won't migrate correctly, so for those ones I exported to a pst file and imported back into outlook with the scalix connector.

[SidebandSamurai]

Hello everyone,

I have attempted the suggestion. and this was a improvement. One of the other problems I was having is imapsync would create the folders and then not transfer the message, creating the folders first, solves that problem

Thanks for the suggestion.

but...

I am still getting transfer errors from the old imap server.

here is a copy of the --debugimap option

Code: Select all

Sending: 1994 UID FETCH 704 FLAGS

Sent 26 bytes
Read: * 609 FETCH (UID 704 FLAGS (Junk))
1994 OK UID FETCH completed

flags: line = '1994 UID FETCH 704 FLAGS
'
flags: line = '* 609 FETCH (UID 704 FLAGS (Junk))
'
flags: line = '* 609 FETCH (UID 704 FLAGS (Junk))
' and 1,2,3,4 = 609,704,Junk,
flags: line = '1994 OK UID FETCH completed
'
flags from : [Junk][]
Sending: 929 APPEND Inbox (Junk) {3259}

Sent 32 bytes
Read: 929 BAD syntax error

Error trying to append string: 929 BAD syntax error

Sending: 1995 UID FETCH 704 body.peek[header.fields (Subject)]

Sent 55 bytes
LITERAL: received literal in line * 609 FETCH (UID 704 BODY[HEADER.FIELDS ("SUBJECT")]  of length 56; attempting to retrieve from the 88 bytes in: Subject: Microsoft Releases Two Critical New Patches

)
1995 OK UID FETCH completed
<END_OF_iBuffer>
Read: * 609 FETCH (UID 704 BODY[HEADER.FIELDS ("SUBJECT")] Subject: Microsoft Releases Two Critical New Patches

)
1995 OK UID FETCH completed

Couldn't append msg #704 (Subject:[Microsoft Releases Two Critical New Patches]) to folder Inbox: Error trying to append string: 929 BAD syntax error


Sending: 1996 UID FETCH 705 FLAGS

Sent 26 bytes
Read: * 610 FETCH (UID 705 FLAGS (\Seen))
1996 OK UID FETCH completed

flags: line = '1996 UID FETCH 705 FLAGS
'
flags: line = '* 610 FETCH (UID 705 FLAGS (\Seen))
'
flags: line = '* 610 FETCH (UID 705 FLAGS (\Seen))
' and 1,2,3,4 = 610,705,\Seen,
flags: line = '1996 OK UID FETCH completed
'
Autoloading: UID store 3567 +FLAGS (\Seen)
Sending: 930 UID store 3567 +FLAGS (\Seen)

Sent 35 bytes
Read: * 274 FETCH (FLAGS (\Seen $MdnSent))
930 OK STORE completed

Sending: 931 UID FETCH 3567 FLAGS

Sent 26 bytes
Read: * 274 FETCH (FLAGS (\Seen $MdnSent) UID 3567)
931 OK FETCH completed

flags: line = '931 UID FETCH 3567 FLAGS
'
flags: line = '* 274 FETCH (FLAGS (\Seen $MdnSent) UID 3567)
'
flags: line = '* 274 FETCH (FLAGS (\Seen $MdnSent) UID 3567)
' and 1,2,3,4 = 274,,\Seen $MdnSent,3567
flags: line = '931 OK FETCH completed
'
Sending: 1997 UID FETCH 706 FLAGS

Sent 26 bytes
Read: * 611 FETCH (UID 706 FLAGS (\Seen \Answered))
1997 OK UID FETCH completed

flags: line = '1997 UID FETCH 706 FLAGS
'
flags: line = '* 611 FETCH (UID 706 FLAGS (\Seen \Answered))
'
flags: line = '* 611 FETCH (UID 706 FLAGS (\Seen \Answered))
' and 1,2,3,4 = 611,706,\Seen \Answered,
flags: line = '1997 OK UID FETCH completed
'
Autoloading: UID store 3571 +FLAGS (\Seen \Answered)
Sending: 932 UID store 3571 +FLAGS (\Seen \Answered)

Sent 45 bytes
Read: * 275 FETCH (FLAGS (\Seen \Answered $MdnSent))
932 OK STORE completed

Sending: 933 UID FETCH 3571 FLAGS

Sent 26 bytes
Read: * 275 FETCH (FLAGS (\Seen \Answered $MdnSent) UID 3571)
933 OK FETCH completed

flags: line = '933 UID FETCH 3571 FLAGS
'
flags: line = '* 275 FETCH (FLAGS (\Seen \Answered $MdnSent) UID 3571)
'
flags: line = '* 275 FETCH (FLAGS (\Seen \Answered $MdnSent) UID 3571)
' and 1,2,3,4 = 275,,\Seen \Answered $MdnSent,3571
flags: line = '933 OK FETCH completed
'
Sending: 1998 UID FETCH 707 FLAGS

Sent 26 bytes
Read: * 612 FETCH (UID 707 FLAGS (\Seen))
1998 OK UID FETCH completed

flags: line = '1998 UID FETCH 707 FLAGS
'
flags: line = '* 612 FETCH (UID 707 FLAGS (\Seen))
'
flags: line = '* 612 FETCH (UID 707 FLAGS (\Seen))
' and 1,2,3,4 = 612,707,\Seen,
flags: line = '1998 OK UID FETCH completed
'
Autoloading: UID store 3575 +FLAGS (\Seen)
Sending: 934 UID store 3575 +FLAGS (\Seen)

Sent 35 bytes
Read: * 276 FETCH (FLAGS (\Seen $MdnSent))
934 OK STORE completed

Sending: 935 UID FETCH 3575 FLAGS

Sent 26 bytes
Read: * 276 FETCH (FLAGS (\Seen $MdnSent) UID 3575)
935 OK FETCH completed

flags: line = '935 UID FETCH 3575 FLAGS
'
flags: line = '* 276 FETCH (FLAGS (\Seen $MdnSent) UID 3575)
'
flags: line = '* 276 FETCH (FLAGS (\Seen $MdnSent) UID 3575)
' and 1,2,3,4 = 276,,\Seen $MdnSent,3575
flags: line = '935 OK FETCH completed
'
Sending: 1999 UID FETCH 708 FLAGS

Sent 26 bytes
Read: * 613 FETCH (UID 708 FLAGS (\Seen))
1999 OK UID FETCH completed

flags: line = '1999 UID FETCH 708 FLAGS
'
flags: line = '* 613 FETCH (UID 708 FLAGS (\Seen))
'
flags: line = '* 613 FETCH (UID 708 FLAGS (\Seen))
' and 1,2,3,4 = 613,708,\Seen,
flags: line = '1999 OK UID FETCH completed
'
Autoloading: UID store 3579 +FLAGS (\Seen)
Sending: 936 UID store 3579 +FLAGS (\Seen)

Sent 35 bytes
Read: * 277 FETCH (FLAGS (\Seen $MdnSent))
936 OK STORE completed

Sending: 937 UID FETCH 3579 FLAGS

Sent 26 bytes
Read: * 277 FETCH (FLAGS (\Seen $MdnSent) UID 3579)
937 OK FETCH completed

flags: line = '937 UID FETCH 3579 FLAGS
'
flags: line = '* 277 FETCH (FLAGS (\Seen $MdnSent) UID 3579)
'
flags: line = '* 277 FETCH (FLAGS (\Seen $MdnSent) UID 3579)
' and 1,2,3,4 = 277,,\Seen $MdnSent,3579
flags: line = '937 OK FETCH completed
'
Time : 3346 s
++++ Statistics ++++
Time                   : 3349 sec
Messages transfered    : 0
Messages skipped       : 271
Total bytes transfered : 0
Total bytes skipped    : 10663997
Total bytes error      : 15437264
Detected 342 errors
Please, rate imapsync at http://freshmeat.net/projects/imapsync/
You have new mail in /var/spool/mail/root
[root@sbgk-2 imapsync-1.145]#


As you can see above, 342 messages are not being transferd, as all of the errors are exactly the same. I have tried the --nofastio option with no success

Does anybody know what the error "BAD syntax error" means It seems to transfer the other messages just fine. This issue also appears to be unrelated to attachments.

The old server is a University of Washington IMAP4 server which is installed by default on Redhat 8.

Here is the justconnect information:

Code: Select all

$RCSfile: imapsync,v $ $Revision: 1.145 $ $Date: 2005/12/04 01:58:54 $
Mail::IMAPClient version used here is 2.2.9
From imap server [sbgk-2.sbgk.com] port [143] user [hrobinson]
To   imap server [grumpy.sbgk.com] port [143] user [harold.robinson@sbgk.com]
CRAM-MD5 not wanted by you
CRAM-MD5 not wanted by you
From software : * OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS AUTH=LOGIN] sbgk-2.sbgk.com IMAP4rev1 2001.315rh at Tue, 17 Jan 2006 07:13:36 -0800 (PST)
To   software : * OK Scalix IMAP server 9.4.2.4 ready on grumpy.sbgk.com
From capability : IMAP4REV1 IDLE NAMESPACE MAILBOX-REFERRALS SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND LOGIN-REFERRALS STARTTLS AUTH=LOGIN
To   capability : IMAP4 IMAP4rev1 X-SCALIX-1 X-SCALIX-2 X-SCALIX-3 X-SCALIX-4 ID IDLE LOGIN-REFERRALS NAMESPACE UIDPLUS ACL AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=PLAIN AUTH=CRAM-MD5 AUTH=GSSAPI
From separator and prefix : [/][]
To   separator and prefix : [/][]
++++ Calculating sizes ++++
From Folder [Inbox]                             Size:  26101261 Messages:   613
Total size: 26101261
Total messages: 613
Time : 1 s
++++ Calculating sizes ++++
To   Folder [Calendar]                          Size:         0 Messages:     0
To   Folder [Contacts]                          Size:         0 Messages:     0
To   Folder [Deleted Items]                     Size:         0 Messages:     0
To   Folder [Drafts]                            Size:    678646 Messages:     1
To   Folder [INBOX]                             Size:  10673858 Messages:   277
To   Folder [Notes]                             Size:         0 Messages:     0
To   Folder [Redhat Network]                    Size:    364485 Messages:    80
To   Folder [Sent]                              Size:  50961008 Messages:    99
To   Folder [Sent Items]                        Size:     55878 Messages:    10
To   Folder [Tasks]                             Size:         0 Messages:     0
To   Folder [Trash]                             Size:         0 Messages:     0
To   Folder [mbox]                              Size:         0 Messages:     0
Total size: 62733875
Total messages: 467
Time : 0 s


Thanks everyone for all your suggestions,

Sincerely,

Sideband Samurai

[jhamill]

I gave up finally for those "recalcitrant messages" as I was having the same issues for some users, so I exported the affected folders to a PST file using outlooks export with the exchange profile. Changed profile to scalix then imported the messges.

Hello everyone,

I have attempted the suggestion. and this was a improvement. One of the other problems I was having is imapsync would create the folders and then not transfer the message, creating the folders first, solves that problem


[/code]

Thanks everyone for all your suggestions,

Sincerely,

Sideband Samurai

[SidebandSamurai]

My fear in that process was Outlook can be so unpredicitable when working with large batches of messages that it would just hang in the middle.

The good news was I Migrated the whole account (my test acount) with out issue. I did not have to use the Outlook connecter either. I just used Outlooks abiity to connect to the imap server directly.

Moving the messages to the PST first acutally prevents outlook from locking up because its doing to much at once.

Thanks for the suggestion.

Sincerely

Sideband Samurai

[SidebandSamurai]

Hello eveyone,

I have come to the end of my migration and wanted to report back what i have discovered so that if someone has issues similar to mine this article will be helpful.

I have discoverd that the errors stated in previous posts were not a result of a faulty migration program. Imapsync works excellent.

My problem was with the imap files them selves. eveidently they had gotten corrupted but not enough to cause the e-mail client to report any errors.

In some instances, I used a utiltiy called mbxcvt from the RPM package called
imap-utils-2002d-6mdk.i586.rpm After installing this package, I re-converted the imap mailboxes to mbx format. The problem then disappeared.

Now I want to caution anybody who reads this. My imap server (university of Washington imap) was configured to handle mbx formated files. these files were placed in each users home directory under the directory /home/<userid>/Mail you have to configure a special .imaprc file in order to use these unsupported features. The unsupported features I was using was the ability to place the users e-mail files in a directory called "Mail". UWIMAP will read by default mbx formatted files as well as its standard files.

Some times this process would not work. I had at first attemted to use outlook to migrate the mail (using the imap accounts). The problem with this method was outlook appends winmail.dat files as attachments to all messages. This winmail.dat file contains message formatting information only understood by outlook. So now if the end user saw a winmail.dat file he could not see his attachments.

To solve this isssue, I used thunderbird and migrated to a local e-mail file. Once that completed I migrated to the new scalix server. Over 10GB if mail data was moved using this method successfuly.

I did use imapsync, when I could, to duplicate the folder directory structure, which helped emensely when migrating with thunderbird.

The file server has had only one glitch which I adressed today. My ethernet driver managed to get itself hosed up, and stopped communicating on the network. I had to reboot the sever to get it to communicate again.

Other than that Everything has been working just fine.

Sincerley,

Sideband Samurai

[ScalixSupport]

Thanks for that Sideband Samurai!

Best wishes,
Don