Feature Request: Directory Migration for server migration

[jplorier]

Hi there,

I've been having some trouble this weeks trying to get everything running after setting up a new server and then getting all the currently in production mailstore to it.
In order to do it I had to move all /var/opt/scalix/xx directory and then follow the howto/changeip and howto/changefqdn, which got me most things running but not everything.
I think there should be a procedure to get ldap directory replicated to a brand new install with all the fqdn and ip and other stuff settled from fresh install and the just by using sxmboximp get the users data in. That should be something clean and kind of simple for a not so uncommon task such as migrating to a new server and maybe having to change fqdn and/or ip.
thanks,

[Valerion]

LDAP is the wrong way to go about this, as Scalix doesn't use LDAP as a management tool as much as people expect. You can't create users or similar using it.

What I did was to write a perl script that compiled a complete userlist with attributes, then on the new machine I would run a corresponding import script that goes and creates the users again. It looks something like this (this is pre-Scalix 11). The code is definitely not the cleanest I have written, but it does the job.

Code: Select all

#!/usr/bin/perl

# Copyright Conversant Systems (Pty) Ltd

$base_dir = "ScalixBackup";
$bin_dir = "/opt/scalix/bin";

mkdir $base_dir;

@all_mailnodes = `$bin_dir/omshowmn -D -f`;

foreach $mailnode (@all_mailnodes) {
        $mailnode =~ s/^\s+//g;
        $mailnode =~ s/\*\*//g;
        $mailnode =~ s/^\s+//g;
        $mailnode =~ s/\s+$//g;
        $mailnode =~ s/\s+/ /g;

        ($mailnode_name,$mailnode_domain) = split (/ /,$mailnode);
        ($ou1,$ou2,$ou3,$ou4) = split (/,/,$mailnode_name);
        $mailnode_fqn = "OU1=".$ou1;
        if ($ou2) {
                $mailnode_fqn = $mailnode_fqn."/OU2=".$ou2;
                if ($ou3) {
                        $mailnode_fqn = $mailnode_fqn."/OU3=".$ou3;
                        if ($ou4) {
                                $mailnode_fqn = $mailnode_fqn."/OU4=".$ou4;
                        }
                }
        }
        $mailnode_dir = $base_dir."/".$mailnode_name;
        mkdir $mailnode_dir;
        @all_users = `$bin_dir/omshowu -m $mailnode_name`;
                foreach $user (@all_users) {
                        $user =~ s/\/[\w\W]+//g;
                        $user =~ s/\s+$//g;
                        if ($user =~ / /) {
                                $has_firstname = "yes";
                                ($firstname,$surname) = split (/ /,$user);
                                $name_fqn="S=".$surname."/G=".$firstname;
                        }
                        else {
                                $has_firstname = "no";
                                $surname = $user;
                                $name_fqn="S=".$surname;
                        }
                        $user =~ s/[\s\_]/./g;
                        $name_file = $mailnode_dir."/".$user;
                        $name_fqn =~ s/_/ /g;
                        $fqn = $name_fqn."/".$mailnode_fqn;
                        $user_details = `$bin_dir/omshowu -n "$fqn" -S`;
                        open OUTFILE, ">".$name_file.",details";
                        print OUTFILE $user_details;
                        close OUTFILE;
                        $file = $name_file.",store";
                        `$bin_dir/omcpoutu -n "$fqn" -f "$file"`
                }
}

Still uses omcpoutu instead of sxmboxexp. It's fairly easy to write a import counterpart for this. Here's mine, in any event

Code: Select all

#!/usr/bin/perl

# Copyright Conversant Systems (Pty) Ltd

$bin_dir = "/opt/scalix/bin";

sub process_store
{
        $cmd = "$bin_dir/omcpinu -f $file";
        print "$cmd\n";
        `$cmd\n`;
}
sub process_file
{
        my $value, $key, $firstname, $surname, $username, $mailnode, $ia;
        my $lang, $passwd, $alias, $alias_addr, $add, $cmd;

        open FILE, $file;

        while (<FILE>) {
                ($key, $value) = split ":";
                chomp $key;
                $key =~ s/^\s+//;
                $key =~ s/\s+$//;

                chomp $value;
                $value =~ s/^\s+//;
                $value =~ s/\s+$//;

                if ($key eq "User Name") {
                        $username = $value;

                        if ($value =~ /\s/) {
                                ($firstname, $surname) = split " ", $value;
                                $passwd = $firstname;

                        } else {
                                $firstname = "";
                                $surname = $value;
                                $passwd = $surname;
                        }

                        $passwd = lc $passwd;

                }

                if ($key eq "MailNode") {
                        $mailnode = $value;
                }

                if ($key eq "Internet Address") {
                        $ia = $value;
                }

                if ($key eq "Language") {
                        $lang = $value;
                }

                if ($key eq "Admin Capabilities") {
                        if ($value eq "YES") {
                                $add = $add . "-c admin,mboxadmin ";
                        }
                }

                if ($key eq "System Login") {
                        if ($value !~ /\d+/) {
                                $add = $add . "-u $value ";
                        }
                }

                if ($key =~ /\/INTERNET-ADDR/) {
                        $key =~ s/\s+//g;
                        $add = $add . "-a \"$key\" ";
                }

        }

        close FILE;

        $add =~ s/^\s+//;
        $add =~ s/\s+$//;

        $cmd = "$bin_dir/omaddu -n \"$username/$mailnode/INTERNET-ADDR=$ia\" -l ENGLISH -p $passwd $add";
        #print "$cmd\n";
        #`$cmd\n`;

        $add = "";
}

@dirs = <*>;

foreach $dir (@dirs) {
        @files = <$dir/*details>;

        foreach $file (@files) {
                process_file;
        }
}

foreach $dir (@dirs) {
        @files = <$dir/*store>;

        foreach $file (@files) {
                process_store;
        }
}

[jplorier]

Thanks Valerion, you were really fast on this.
I'll give it a try and tell you about it. Have you though of "giving" scalix this script so they can provide users with it? It can really help people not so expert like me to get the job done more easyly and thus more bosses happy about implementing scalix.
Thanks again and I'll come back to tell.

[Valerion]

Well, if you keep the copyright notice you are free to use it yourself, and Scalix is as well, of course.

[jplorier]

I'll do it so. I'll also try Scalix to do it, maybe they'll make some changes, but then, is for the greater good.
Thanks again for the hand

[jplorier]

Valerion,

Does the script actually create the user on the new server?. As I see it created a backup for every user, but if I'm not wrong, you can't import the backup unless you have the user created, and thus won't work for a fresh install.
Thanks,

[Valerion]

$cmd = "$bin_dir/omaddu -n \"$username/$mailnode/INTERNET-ADDR=$ia\" -l ENGLISH -p $passwd $add";

The user is added with this command

[jplorier]

Sorry, I didn't take time to read the import script, just the export. Thanks again

[jplorier]

Well, I'm affraid this won't do. It won't create the groups or asociate the users to them (in case I decide to add them manualy). It doesn't either copy the user mailbox size, or password, not even the rules for rejecting mails when over the limit.
I'm in the task to find if there are different commands that can hanndle this info and use them all together to "move" all the users to the new server.
I'll keep also investigating for a way to allow to the new server to collect all the directory structure and make it his own.
This should be the best way, at least as I see it, so I'll keep in this task.
If you happen to find anything to make things faster, I'd apreciate it because I'm running really out of time.
Thanks

[Valerion]

The password you can /maybe/ do by extracting the hash from the USERLIST directory. One of the Scalix developers did comment on that on a different forum.

Swap omcpinu/omcpoutu for sxmboximp/sxmboxexp and check the manual pages - it transfers more things than the older tools.

Groups you can do in a similar way (omshowpdl -l all to get a list, omshowpdln -l <name> to get a list of members, with omaddpdl and omaddpdln to add it on the new server)

[jplorier]

Thanks Valerion. You've been really helpfull. I think I must go to the "move all" method that most people uses, although it gives me some trouble getting sac running, the rest works and maybe I can get things done without having to ask every user to reenter his password.
I've read about sxmboximp/exp and does include more user data than the old tools, but still there are many settings that are not moved and thus meaning that the user has to notice what is missing and restore it by hand (which is not an option because I had a hard time just to make Scalix the company's server and there's still too many people unhappy with it to try to anoy them anymore).
Thanks again, and in case I'm not able to get things done, I'll get back to bother just a little more.