Data stays on disc after user is deleted?

[derfraenk]

Hello!


We've been testing the Scalix Community Edition 11.2.0.52 for some time and now I've come across an issue for which I was unable to find an answer in the forums or the wiki.

I created a user and filled his mailbox with about 12000 mails to test backup procedures for huge mailboxes. The total size reported by sxdu was about 3.6GB. At some point I deleted the user via the admin webinterface to test disaster recovery possibilities. However I noticed that the free space on the data store partition only increased marginally and indeed, the /var/opt/scalix/xx/x/data/ directory still holds 4,6GB. I have tried various runs of omscan and other things like rebuilding indexes but I seem to be unable to clear the unused data.

Is this behavior normal? If yes, what is the reason for this, is there a possibility to recover his mail data with these left over files? What is the correct procedure to delete a user and all of his data at the same time? Is there a way I can clean up the data store now that the user is gone?


It might be interesting to note that when the user's mailbox was that full, I was unable to login via the webmail interface. I kept getting "username or password incorrect" messages. Accessing the mailbox via Outlook- and Evolution-connectors as well as via IMAP worked fine though. I don't know if that's related but I thought I'd mention it.


Thanks for any insights!

[derfraenk]

I apologize for reviving this old thread but we have since installed Scalix on was is going to be the actual mail server very soon and the issue still persists.


The version is also Community Edition 11.2.0.52.

We have tried filling various mailboxes with 50-100 MB of emails (using omsend) and deleting the users, measuring the space taken up by the /var/opt/scalix/xx/x/data/ directory after each step. Indeed, after the emails are delivered, the space in use increases very precisely by the size of the actual emails.

Now if I log into a user's mailbox via any client and delete the emails, it takes a few minutes and the space is cleared from the data directory, again precisely the amount of space that was taken up by the emails.

However if I delete the user himself, without deleting his emails first, almost no space gets freed (~1 MB), even after a full weekend with next to zero server load. Browsing the data directory, I can even verify that the old emails of the deleted user are still physically present on disc. Also, the used up space doesn't seem to get "recycled" in any way, ie if I create a new user and start filling his mailbox, more space gets used.


I would really appreciate any thoughts, suggestions or reports of how the server behaves in your case.

[Mikev]

Hi
You need to run

omscan -Aavfx

Mike

[derfraenk]

Thank you, Mikev.

However we have already read through most related threads and tried running omscan in various combinations, among them -Aavfx. We have also run it multiple times which was suggested in another thread. Whatever we did, there were no orphans or any other errors reported and no significant amount of space was freed.

It's as if only the user account gets deleted but not the data from the disc. After the user is gone, the files remain on disc but don't show up anywhere (sxdu, etc).

[techsharp]

Try a omdelent -e "G=John/S=Doe"

Input the user in question in the above and see if that helps.

[jaime.pinto]

Even after you "delete" a user, the user information as well as the data in the mailstore remains on the system for about a week, in case you change your mind.

[derfraenk]

Thanks a lot for the replies.


techsharp, if my memory serves me right, we did try that and got told that such a user didn't exist. Righly so, because we deleted him earlier. However I will certainly try again tomorrow to verify.


jaime.pinto, that's interesting to know. I will continue to keep a very close look on the size of the data directory. The problem is, since I made the first post at the end of October, our test system with the first Scalix installation has been running all the time and to this day it doesn't seem to have cleaned up anything.

On another note, is it possible to trigger the deletion of the data manually? Or maybe configure the time frame the server keeps it on disc?

[jaime.pinto]

Read this:
viewtopic.php?t=7606#34558

[derfraenk]

Ok, I have just run omscan -Aavfx multiple times again, the size of the data folder before and after that was exactly the same, according to du.

I also tried omdelent -e "G=Firstname/S=Lastname" but as expected I got "Entry not in the Directory".


jaime.pinto, sorry I'm not sure what you're getting at. Do you mean to say that the user's data is still in the recovery folder? I thought that feature wasn't available in the Community Edition? Also, if so, why is it that the server on the test machine still hasn't cleared up the deleted user's data after so much time?

I still checked sys/general.cfg for RECOVERY_FOLDER_EXPIRY_TIME. We do not have set the option, so that means items should get cleared after 7 days which is the default.

Additionally I did three runs of omtidyallu -d -T r -a 0, but each time it only reported "Scalix Recovered Items: 0 items" for each user that still has an account, ie the deleted users do not show up in the list. Again, no space was freed by these runs.


For reference, sxdu -s corrently returns 471 KB while du -s /var/opt/scalix/xx/x/data returns 1497308 KB.
On our test system sxdu -s returns 589642 KB and du -s /var/opt/scalix/xx/x/data returns 6351064 KB.

[mikevl]

Hi

Orphans are also purged fro the system after a period of 30 days. Omscan moves them from the data directory(s) to /var/opt/scalix/xx/s/orphans, haven't checked if that has changed recently.

Mike

[derfraenk]

Thanks, I just checked and we do not have an orphans subdirectory in there. I'm guessing this means that no orphans were found so far and thus the directory was not created yet?

[derfraenk]

Well, the problem still persists and after some very time consuming tests I seem to have found the cause: it seems to be related to the "stale imap connections" problem in this thread: viewtopic.php?t=4234


This is how I have come to this conclusion:
If I create a new user, fill his mailbox and immediately delete him again, without ever logging in as said user, the used space gets cleared from the disc immediately after I delete him.

However, if I fill the user's mailbox and then log in and out of it via SWA, that leaves open 2-3 in.imap41d processes on the server that belong to that user. Since I cannot delete him while these exist, they won't disappear by themselves and there doesn't seem to be any proper way to end them, I have to kill them. I can now delete the user but none of the data will be deleted and thus no space freed. This is completely reproducable.

This does not happen if I connect to the mailbox with an IMAP-client, for example Thunderbird. In that case, all imap processes are properly ended after I log out and deletion of the user also clears the data from the disc. Outlook+connector is also no problem. That is, if the user has never logged in via SWA before. The only thing that causes this is SWA.

If I restart Scalix with /etc/init.d/scalix stop/start or omshut/omrc, the imap processes are properly ended and after I log into the SCA and delete the user, the data is deleted properly. However this is not a viable option, I cannot restart the whole server every time I want to delete a user!


In my opinion this seems to be a bug. We would need a proper way to end the imap processes left behind by an SWA session without restarting the whole server. Unfortunately I don't think I can justify purchasing a commercial version of Scalix in this state.

[mikevl]

Hi

There is an easier way

First you need to do

omstat -u rci to find if that user (john doe) is logged on

then find all the unix processes associated with john doe

the kill -9 x x x x each process

Try this and see if it works

Mike

[derfraenk]

Thanks mikevl. But the problem is that once I use kill to end the imap processes, something seems to go wrong and Scalix does not delete the user's data. Or are you saying there will be other (non-in.imap41d) processes that also need to be killed?

I cannot test it right now but I will as soon as possible. I really need to set up a Scalix server at home.


[EDIT]
Ok, I have just tested it, but it was essentially what I have been doing the whole time. Only in.imap41d processes show up (usually two of them). I kill -9 them, delete the user and the data stays on the disc. The issue still stands.

I'd like to ask anyone who has some time on their hands to please test this on their servers:

• Create a test user
• Fill his mailbox with at least a few MB
• Log in as that user through the webmail interface, let it load and close it again (This is important)
• Check the size of the data directory and write it down: du -s /var/opt/scalix/xx/x/data
• Try to delete the user through the admin web interface
• If you cannot, kill the imap processes associated with that user. Now you should be able to delete him.
• Check the size of the data directory again. Did the space that was occupied by his emails get freed?

I have tested this on three seperate systems now and I'm quite sure it's not isolated to our setup.

[Mikev]

Hi

Are you sure you have deleted all the users IMAP processes as listed for that user in
omstat -u rci?

The user is then logged off for sure.

Mike