|
|
| (3 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| | + | [[Scalix Wiki]] -> [[How-Tos]] -> '''Backup and Restore''' |
| | + | |
| | == Exporting/Importing user account data == | | == Exporting/Importing user account data == |
| | | | |
| Line 5: |
Line 7: |
| | === Handling large mailboxes === | | === Handling large mailboxes === |
| | | | |
| − | The dump files that omcpoutu creates are limited in size to a maximum of 2GB, i.e. you will get an error message if you try to export a mialbox that is larger than this size. | + | The dump files that omcpoutu creates are limited in size to a maximum of 2GB, i.e. you will get an error message if you try to export a mailbox that is larger than this size. |
| | | | |
| | However, with Scalix versions of 9.4 and later, you can redirect the output of the command to "Standard Output" and then redirect the output to a file on the OS side. This will allow for backup and restore of large mailboxes. You can even compress the output on the fly, resulting in much smaller filesizes. | | However, with Scalix versions of 9.4 and later, you can redirect the output of the command to "Standard Output" and then redirect the output to a file on the OS side. This will allow for backup and restore of large mailboxes. You can even compress the output on the fly, resulting in much smaller filesizes. |
| Line 17: |
Line 19: |
| | | | |
| | '''Note:''' ''On some OS distributions, you will still not be able to reimport mailboxes larger than 2 GB in size; this is through a bug in Scalix versions < 11.0 in conjunction with some OS file handling default characteristics. For Scalix SBE and EE users with active subscriptions, there is a Hotfix available for release 10.0 through Scalix Support. | | '''Note:''' ''On some OS distributions, you will still not be able to reimport mailboxes larger than 2 GB in size; this is through a bug in Scalix versions < 11.0 in conjunction with some OS file handling default characteristics. For Scalix SBE and EE users with active subscriptions, there is a Hotfix available for release 10.0 through Scalix Support. |
| − |
| |
| − |
| |
| − | === Automatic Backup Script ===
| |
| − | Will add description later when I have some free time. Feel free to modify.
| |
| − |
| |
| − |
| |
| − | #!/bin/sh
| |
| − | ###############################################################################
| |
| − | # ombackup:
| |
| − | # a backup script for scalix mail servers
| |
| − | #
| |
| − | # This script is used to backup Scalix mail servers; it exports each
| |
| − | # user to a gzip compressed file using the 'omcpoutu' command, then
| |
| − | # duplicates the scalix data directory using rsync.
| |
| − | #
| |
| − | # Before using this program you should set the values of the variables
| |
| − | # below to match your server/preferences.
| |
| − | #
| |
| − | # For detailed descriptions of the available command line switches,
| |
| − | # execute the program with the -h flag.
| |
| − | #
| |
| − | #
| |
| − | # Copyright (C) 2006 Jon Allie <jon@jonallie.com>
| |
| − | #
| |
| − | # This program is free software; you can redistribute it and/or modify
| |
| − | # it under the terms of the GNU General Public License as published by
| |
| − | # the Free Software Foundation; either version 2 of the License, or
| |
| − | # (at your option) any later version.
| |
| − | #
| |
| − | # This program is distributed in the hope that it will be useful,
| |
| − | # but WITHOUT ANY WARRANTY; without even the implied warranty of
| |
| − | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
| |
| − | # GNU General Public License for more details.
| |
| − | #
| |
| − | # You should have received a copy of the GNU General Public License
| |
| − | # along with this program; if not, write to the Free Software
| |
| − | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
| |
| − | #
| |
| − | #
| |
| − | #
| |
| − | # ------------------------------ Modifications ----------------------------
| |
| − | #
| |
| − | # Ianaré Sévi <ianare@gmail.com> on 2006-10-11:
| |
| − | #
| |
| − | # Combined various incarnations and patches of the original script
| |
| − | # found here: http://www.scalix.com/community/viewtopic.php?t=1922
| |
| − | # into a fully working script. Huge thanks to all the original
| |
| − | # contributors - I am but a shadow in their footsteps.
| |
| − | #
| |
| − | # Added -r option and associated programming to allow rotation of backups
| |
| − | # based on the day of the week (7 working backups)
| |
| − | #
| |
| − | ###############################################################################
| |
| − |
| |
| − |
| |
| − |
| |
| − | ### main variables
| |
| − |
| |
| − | MAILNODE="mail,lorientech"
| |
| − | ROOT_BACKUP_DIR=/mnt/ScalixBackUp
| |
| − | SCALIX_DIR=/var/opt/scalix
| |
| − | SCALIX_BIN=/opt/scalix/bin
| |
| − | LOGFILE=/tmp/ombackup.log
| |
| − | USERFILE=/tmp/userfile.$$
| |
| − | DATE=`date +%Y%m%d`
| |
| − | PROCESS_BLOCK_SIZE=5
| |
| − | ROTATE_BACKUP=Y
| |
| − |
| |
| − |
| |
| − | ### function declarations
| |
| − |
| |
| − | function usage
| |
| − | {
| |
| − | printf $"
| |
| − | Usage: ombackup [-h] [-b backup dir] [-d scalix data dir] [-s scalix bin dir]
| |
| − | [-l logfile] [-u user file] [-m mailnode] [-r rotate backups (Y|N)]
| |
| − |
| |
| − | ombackup comes with ABSOLUTELY NO WARRANTY. This is free software, and you
| |
| − | are welcome to redistribute it under certain conditions. See the GNU
| |
| − | General Public Licence for details.
| |
| − |
| |
| − | ombackup is a shell script to perform both user level and system level backups
| |
| − | of a Scalix mail server. User mailboxes are backed up via the 'omcpoutu' utility
| |
| − | and are stored in a configurable backup directory in a subdirectory named the
| |
| − | same as the mailnode being backed up. Systems level backups are performed by
| |
| − | copying the whole Scalix data dir (usually /var/opt/scalix) to a backup directory
| |
| − | using rysnc.
| |
| − |
| |
| − | Most options can be configured by setting the values of the variables in the script
| |
| − | or can be passed to the script at runtime
| |
| − |
| |
| − | Options:
| |
| − | -h : print this message and exit
| |
| − |
| |
| − | -m <mailnode> : mailnode to dump users from
| |
| − |
| |
| − | -b <backup dir> : backup directory. This directory will store both the user and
| |
| − | system level backups. User backups are stored in a subdirectory
| |
| − | under this directory users/<mailnode>/<userfile>.
| |
| − |
| |
| − | -d <scalix dir> : scalix data dir. Defaults to /var/opt/scalix
| |
| − |
| |
| − | -s <bin dir> : scalix bin dir. Contains scalix utility binaries. Defaults to
| |
| − | /opt/scalix/bin
| |
| − |
| |
| − | -l <logfile> : path to a logfile for logging backup actions.
| |
| − |
| |
| − | -u <userfile> : userfile. This file is created during the user mailbox
| |
| − | backup. Defaults to /tmp/userfile.[pid]
| |
| − |
| |
| − | -r <Y|N> : wether or not to rotate backups on 7 day schedule.
| |
| − |
| |
| − |
| |
| − | Copyright (C) 2006 by Jon Allie <jon@jonallie.com>\n\n"
| |
| − |
| |
| − | exit ${1:-0}
| |
| − | }
| |
| − |
| |
| − |
| |
| − | function badInput
| |
| − | {
| |
| − | echo "Use -h for more information."
| |
| − | echo
| |
| − | exit 1
| |
| − | }
| |
| − |
| |
| − |
| |
| − | function log_it
| |
| − | {
| |
| − | echo "[ `date` ]: $*" >>$LOGFILE
| |
| − |
| |
| − | }
| |
| − |
| |
| − | function echo_and_log
| |
| − | {
| |
| − | echo $*
| |
| − | log_it $*
| |
| − | }
| |
| − |
| |
| − | function clean_up
| |
| − | {
| |
| − | echo_and_log "Cleaning up temporary files"
| |
| − | [ -f $USERFILE ] && rm -f $USERFILE
| |
| − | }
| |
| − |
| |
| − |
| |
| − | function exit_with_error
| |
| − | {
| |
| − | echo_and_log "Error: $*"
| |
| − | clean_up
| |
| − | exit 1
| |
| − | }
| |
| − |
| |
| − | function pre_check
| |
| − | {
| |
| − |
| |
| − | # look for scalix directories
| |
| − | for dir in $SCALIX_BIN $SCALIX_DIR
| |
| − | do
| |
| − | [ -d $dir ] || exit_with_error "A required Scalix directory $dir doesn't exist."
| |
| − | done
| |
| − |
| |
| − | # make sure that the $BACKUP_DIR structure exists, try to create it if not.
| |
| − | for dir in $BACKUP_DIR $BACKUP_DIR/users $BACKUP_DIR/users/$MAILNODE
| |
| − | do
| |
| − | if [ ! -d $dir ]
| |
| − | then
| |
| − | echo_and_log "$dir doesn't exist: creating it"
| |
| − | mkdir $dir || exit_with_error "Unable to create required directory $dir"
| |
| − | fi
| |
| − | done
| |
| − |
| |
| − | # clear out user backup files
| |
| − | rm -rf $BACKUP_DIR/users/$MAILNODE/*
| |
| − |
| |
| − | }
| |
| − |
| |
| − | function dump_users
| |
| − | {
| |
| − | #index for processing block
| |
| − | let i=1
| |
| − | let index=1
| |
| − |
| |
| − | # Build userfile
| |
| − | $SCALIX_BIN/omshowu -m $MAILNODE|cut -f1 -d'/' >$USERFILE
| |
| − | [ "$?" != "0" ] && exit_with_error "Unable to build userfile $USERFILE from mailnode $MAILNODE"
| |
| − |
| |
| − | # Loop over userfile and create backups. Use 'while read' instead of 'for' because of spaces in names
| |
| − | while read sc_username
| |
| − | do
| |
| − | # Create a version of the username without spaces and other crappy characters
| |
| − | nospaces=`echo $sc_username|sed -e "s/[ \.;=*'?_!]//g"`
| |
| − |
| |
| − | BACKUP_FILE="$BACKUP_DIR/users/$MAILNODE/${nospaces}-mail.gz"
| |
| − |
| |
| − | if [ $i -le $PROCESS_BLOCK_SIZE ]
| |
| − | then
| |
| − | echo "Adding Process: Number $index of $PROCESS_BLOCK_SIZE -- User: $sc_username"
| |
| − | ## BACKGROUND PROCESS
| |
| − | $SCALIX_BIN/omcpoutu -n "$sc_username/$MAILNODE" -f - -F | gzip | cat > $BACKUP_FILE || echo_and_log "Error: Unable to complete backup operation for $sc_username" &
| |
| − | PIDs[$index]=$!
| |
| − | let i+=1
| |
| − | let index=$i
| |
| − | else
| |
| − | echo "Process block is full."
| |
| − | echo "Waiting for first complete process..."
| |
| − | while [ $i -gt $PROCESS_BLOCK_SIZE ]
| |
| − | do
| |
| − | for p in `seq 1 $PROCESS_BLOCK_SIZE`
| |
| − | do
| |
| − | ps ${PIDs[$p]} > /dev/null
| |
| − | if [ "$?" != "0" ]
| |
| − | then
| |
| − | echo "Process number $p of $PROCESS_BLOCK_SIZE has completed. -- User: $sc_username"
| |
| − | unset PIDs[$p]
| |
| − | let index=$p
| |
| − | echo "Adding Process: Number $index of $PROCESS_BLOCK_SIZE -- User: $sc_username"
| |
| − | ## BACKGROUND PROCESS
| |
| − | $SCALIX_BIN/omcpoutu -n "$sc_username/$MAILNODE" -f - -F | gzip | cat > $BACKUP_FILE || echo_and_log "Error: Unable to complete backup operation for $sc_username" &
| |
| − | PIDs[$index]=$!
| |
| − | break 2
| |
| − | fi
| |
| − | done
| |
| − | done
| |
| − | fi
| |
| − | done < $USERFILE
| |
| − |
| |
| − | echo "All processes have been added."
| |
| − | echo "Waiting for those still running..."
| |
| − | wait
| |
| − | echo "All done!"
| |
| − | }
| |
| − |
| |
| − | function sync_files
| |
| − | {
| |
| − | echo_and_log "Beginning rsync of $SCALIX_DIR to $BACKUP_DIR"
| |
| − | rsync -av --delete $SCALIX_DIR $BACKUP_DIR/ >>$LOGFILE
| |
| − |
| |
| − | if [ "$?" != "0" ]
| |
| − | then
| |
| − | exit_with_error "Rsync operation of $SCALIX_DIR to $BACKUP_DIR did not complete successfully"
| |
| − | else
| |
| − | echo_and_log "Completed rsync of $SCALIX_DIR to $BACKUP_DIR"
| |
| − | fi
| |
| − | }
| |
| − |
| |
| − | # process command line arguments
| |
| − | # -h : show help
| |
| − | # -b <dir> : backup directory
| |
| − | # -l <file> : log file
| |
| − | # -u <userfile> : userfile
| |
| − | # -m <mailnode> : main mailnode
| |
| − | # -d <dir> : location of the scalix data dir
| |
| − | # -s <dir> : location of the scalix bin dir
| |
| − | # -r <Y|N> : rotate backups or not
| |
| − |
| |
| − | while getopts hb:l:u:m:s:r: opt
| |
| − | do
| |
| − | case "$opt" in
| |
| − | h) usage ;;
| |
| − | b) BACKUP_DIR=$OPTARG ;;
| |
| − | l) LOGFILE=$OPTARG ;;
| |
| − | u) USERFILE=$OPTARG ;;
| |
| − | m) MAILNODE=$OPTARG ;;
| |
| − | d) SCALIX_DIR=$OPTARG ;;
| |
| − | s) SCALIX_BIN=$OPTARG ;;
| |
| − | r) ROTATE_BACKUP=$OPTARG ;;
| |
| − | \?) badInput ;;
| |
| − | esac
| |
| − | done
| |
| − |
| |
| − | # validate that all required options are set
| |
| − | for x in "$LOGFILE" "$ROOT_BACKUP_DIR" "$MAILNODE" "$SCALIX_DIR" "$SCALIX_BIN" "$USERFILE" "$ROTATE_BACKUP"
| |
| − | do
| |
| − | if [ -z "$x" ]
| |
| − | then
| |
| − | echo "A required parameter is missing, please check your command arguments."
| |
| − | badInput
| |
| − | fi
| |
| − | done
| |
| − |
| |
| − | # rotate backups or not
| |
| − | if [ "$ROTATE_BACKUP" = "Y" ]
| |
| − | then
| |
| − | DAYNUMBER=`date +%A`
| |
| − | BACKUP_DIR=$ROOT_BACKUP_DIR/$DAYNUMBER
| |
| − | else
| |
| − | BACKUP_DIR=$ROOT_BACKUP_DIR
| |
| − | fi
| |
| − |
| |
| − | # initialize the logfile
| |
| − | >$LOGFILE
| |
| − |
| |
| − | # call pre_check function to verify backup directory structure
| |
| − | pre_check
| |
| − |
| |
| − | # shutoff remote client connections
| |
| − | echo_and_log "Shutting down remote client interface"
| |
| − | $SCALIX_BIN/omoff -d 0 rci
| |
| − | [ "$?" != "0" ] && exit_with_error "Unable to halt remote client interface"
| |
| − |
| |
| − | # call dump_users function to make backups of user mailboxes
| |
| − | dump_users
| |
| − |
| |
| − | #echo_and_log "Stopping scalix services"
| |
| − | /etc/init.d/scalix stop
| |
| − | [ "$?" != "0" ] && exit_with_error "Unable to halt scalix services"
| |
| − |
| |
| − | # call sync_files function to make a backup of the $SCALIX_DIR
| |
| − | sync_files
| |
| − |
| |
| − | # restart scalix services
| |
| − | echo_and_log "Starting Scalix services"
| |
| − | /etc/init.d/scalix start
| |
| − | [ "$?" != "0" ] && exit_with_error "Error restarting scalix services"
| |
| − |
| |
| − | # explicily call the clean_up function to erase leftover files
| |
| − | clean_up
| |
| − |
| |
| − | # exit successfully
| |
| − | echo_and_log "All operations complete"
| |
| − | exit 0
| |
To export/import a single user's mailbox to a dump file, the omcpoutu/omcpinu (Scalix < 11.0) or sxmboxexp/sxmboximp (Scalix >= 11.0) can be used. Please see the respective man pages for details.
The dump files that omcpoutu creates are limited in size to a maximum of 2GB, i.e. you will get an error message if you try to export a mailbox that is larger than this size.
However, with Scalix versions of 9.4 and later, you can redirect the output of the command to "Standard Output" and then redirect the output to a file on the OS side. This will allow for backup and restore of large mailboxes. You can even compress the output on the fly, resulting in much smaller filesizes.
