Script to delete corrupt data

[HansR100]

Hey Com,

do someone have a script to delete corrupt messages?
Like during the archiving process appears corrupt data in the scalixmail.csv, the data should get deleted by the script?

Thx & Regards,
Hans

[joako]

Unfortunatly Scalix server will store corrupt data (within an non-corrupt mailstore) and there's no process to identify, eliminate or correct this. If data integrity is your desire, don't use Scalix.

[William]

Not sure if it will fix the situation described, but you could try this:

omscan -avtfx
and omtidyu

[joako]

omscan won't do anything to correct corrupt entries inside the mailstore. It will only correct mailstore's corrupt structure.

Maybe if you export the mailbox as PST and then import it to Exchange 2010 server, it will not import corrupt data. Then you could export it again and import it back to your Scalix server. This seems rather time-consuming.

Good luck!

[florian]

Joako,

interesting - even after such a long time your posts appear as useless as ever.

1. You are correct in that omscan restores referential integrity of the mailstore and does very little to validate data contents; the reason is that some of the content data is only ever modified or evaluated by the client, not the server, and the server cannot interpret it, let alone fix it if it's broken. As the poster, however, did not indicate in detail what kind of problem he is facing that he describes as "corruptness", the diagnosis that omscan would not help is wrong. It is not guaranteed to help, but it can.

Unfortunatly Scalix server will store corrupt data (within an non-corrupt mailstore) and there's no process to identify, eliminate or correct this. If data integrity is your desire, don't use Scalix.

2. Compared to other similar applications, Scalix is one of the friendliest systems when it comes to deal with datastore corruption, caused by whatever reason. Not sure if you've ever had to manage an Exchange or Domino server (by market share the largest two players in Email). In case any of their data files is corrupted in whatever way, typically their support will tell you to restore pretty much the whole store from backup, as they don't have much of a toolchain to do anything inside. With the Scalix data structure, on the other hand, you can do a lot with omcontain and other tools, and sometimes even deleting a single message file from the file system, followed by an omscan, does fix a problem by deleting a single broken message, i.e. corruption is much more limited than in any other comparable system I've seen.

So again, if you want to make noise here (and after all this time you still seem to be the only person on this forum having that particular desire), why don't you go away and annoy other people on Microsoft's or someone else's forums and leave people here alone? I think I would personally appreciate that.

Florian

[HansR100]

hey,

so if more infomartion is needed:

iam facing corrupt data during archiving, the archiving aborts due to corrupt data and its not only a few...
omscan and omtidiyu doesn´t work in this case.
nearly all users cannot archive data... a script which deletes appearing corrupt data(scalixmail.csv) would be good.

maybe someone have a script or another idea.

Regards Hans

[joako123]

Joako,

Unfortunatly Scalix server will store corrupt data (within an non-corrupt mailstore) and there's no process to identify, eliminate or correct this. If data integrity is your desire, don't use Scalix.

With the Scalix data structure, on the other hand, you can do a lot with omcontain and other tools, and sometimes even deleting a single message file from the file system, followed by an omscan, does fix a problem by deleting a single broken message, i.e. corruption is much more limited than in any other comparable system I've seen.

I suppose you are correct. However I haven't found a method in Scalix to identify exactly which message is corrupt. I strongly believe there isn't such a method.

[florian]

The problem is the definition of corruptness. As it looks in this thread, Outlook is falling over some message. In this specific case, the problem may be invalid values in so-called MAPI properties, i.e. Outlook-specific attributes attached to the message. These are name-value pairs that the server stores in a transparent way, i.e. it doesn't know how to interpret them. If the values of these are not what Outlook expects, it can fail badly (i.e. from it's viewpoint, the message is "corrupt"). The server cannot identify such messages.

Any problem isolation will therefore have to happen on the client. It should typically be possible to increase logging so that the logs contain information (Message ID, DRef, etc.) on which message is currently being loaded or accessed on the server. If the client-side error then occurs, the last message accessed is very likely to be the one to cause the issue and it can typically be identified and possibly deleted.

-- Florian.

[mikethebike]

Although I agree with Florian with regards clients, I have found in the past that an omcpoutu can show issues that omscan or omtidy does not. The fact that several users are affected may point to a single message shared by those users.
Try copying one of the users out (omcpoutu -n "Hans R100" -f /dev/null) and check the event log (omshowlog -p 10 -s admin) if it fails.
I have also seem BBs cause some weird errors that apear to be corrupt mailboxes, so maybe worth a look in there for weird looking items.

Mick

[HansR100]

Hello,

i tried the way you described, but the corrupt data didn´t show up in the log, also sxmboximp/exp work fine.
problem is the archiving via outlook which stops at corrupt data... so the user cannot archive their mailboxes
unless the corrupt data will deleted.

maybe some got a script or another idea

thanks
Hans

[RickC]

I have the exact same issue -

Outlook (2k7) stops archiving when it finds corruption in my inbox, but I cannot find any corruption on my account using omscan.

Rick

[HansR100]

Hey Rick,

to find them use the scalixmail log, there appears the dref, but to do it by yourself it took its time.

HansR