IMAP is broken

[jason-meers]

Hi all,
I've noticed that every time I delete something using an Imap client it never really gets deleted and is still in my mailbox when viewed from another computer or client. The only way I can delete items is to use a MAPI connection from Outlook.

I have tested this at great length using Thunderbird on Linux with a direct connection to the server, Outlook via IMAP, Outlook via an IMAP proxy (up-imapproxy) and from the imap client shipped on the Nokia 66xx series mobile phones.

The problem only occurs when trying to delete and item using IMAP when the item is in the Inbox folder (other folders are fine). We had a simlar problem removing attachments from items in the Inbox folder and had to move them temporarily to another folder to make it work.

Scalix support told us that the Inbox has different "ownership" that all other folders and that would be why moving items into another folder trying again makes a difference.

We are using a "figaro" installation upgraded from v9.21

Thanks

Jason Meers

[ScalixSupport]

IMAP is not broken.

This is a hazard of using other clients that do not understand the IMAP protocol on the same mailbox.

When you delete a message using IMAP you are setting the \Deleted flag. Even in IMAP this is just a logical deletion but clients are designed to filter out those messages. To undelete the message, the client will unset that \Deleted flag. The best way to see that is to use Outlook or Outlook Express in IMAP mode. If you delete a message, you will see the message displayed with a strikethrough line.

IMAP has a command called EXPUNGE which will physically delete all messages marked with the \Deleted flag.

Using a client, to physically delete the message, select the "Purge" option in the client or, if you are using Thunderbird, right click the folder and select "Compact Folder". This forces the client to issue the EXPUNGE IMAP command to the server.

Cheers

Dave

[jason-meers]

If this is as you say then how can it be the client.

Assume I have two messages in my inbox, and I move one of the messages to folder XYZ. If I then delete the message in my inbox and then delete the message in folder XYZ I get different results, the one in XYZ is deleted (as far as the user is concerend) and the one in the inbox is still there.

I am using EXACTLY the same client with EXACTLY the same settings in both cases.

As I said earlier, how can this be the fault of the client, it is scalix that is behaving inconsistently.

Thanks

Jason Meers

[jch]

What client(s) are you using?

Let me paraphase your example slightly: with two messages in the inbox, move the second one to a folder called "XYZ", then delete the message in the inbox and delete the mesage in XYZ. This IMAP conversation would do that:

Code: Select all

1 select inbox
2 copy 2 XYZ
3 store 2 +flags \deleted
4 select XYZ
5 store 1 +flags \deleted
6 select inbox
7 store 1 +flags \deleted


Notice that IMAP doesn't have a "move" command, it only has copy and a mechanism for marking a message to be deleted.

So, what you're left with at the end of this is two messages in the INBOX, both of which have the \deleted flag set and one message in XYZ which has the \deleted flag set. I know this works because I created a user to test it and I did this (and this time I'm inclusing what the IMAP server reports):

Code: Select all

a select inbox
...
a OK [READ-WRITE] SELECT completed.
b fetch 1:* flags
* 1 FETCH (FLAGS (\Deleted))
* 2 FETCH (FLAGS (\Deleted))
b OK FETCH completed
c select XYZ
...
c OK [READ-WRITE] SELECT completed.
d fetch 1:* flags
* 1 FETCH (FLAGS (\Deleted $MdnSent))
d OK FETCH completed


The "$MdnSent" flag is set because we mustn't generate more than one MDN for a message so to avoid that we ensure that only one message has the "$MdnSent" flag unset. You don't need to worry about that, the point is that all three messages have the \deleted flag set. It doesn't matter if I'd "moved" message 1 or message 2, as far as the flags are concerned the end result is the same. I could fetch the contents of all three messages and discover which one had been copied (OK, I did that, it was the second one).

Now, if you have an IMAP client that displays one message in the inbox after all this then it must know something that the IMAP server doesn't: it must know which message was "moved" so that it can avoid displaying it.

If you look at the inbox with a MAPI client you'll see two messages and you'll see one message in the XYZ folder. That's because MAPI doesn't know about the \deleted flag.

Now, there is a minor white lie in all of this. For SWA we implemented an IMAP extension, there's an X-MOVE command: it actually does move a message from one folder to another. So if you move a message from the inbox to XYZ then you'll see that the message really has moved and that there are still only two copies of the message. SWA does something else as well; it cleans up (issues an EXPUNGE or a CLOSE) quite frequently so it doesn't leave messages lying around that are marked for deletion. As as result it is quite surprised (if software can be surprised) when it sees a message with the \deleted flag set -- in fact it tends to ignore that flag which catches me out from time to time.

Does that make what might be going on any clearer? Or is it just totally confusing? :-)

jch