HowTos/Setting up for High Availability How-To

Overview

The most common multi-server setup is dual servers clustered for failover. In this scenario, server A fails over to Server B and vice versa. Each machine has an active physical instance (A or B) and a virtual instance (A’ or B’) that takes the load if the other fails.

Both servers mount to a single shared storage solution and clustering software is responsible for relocating the Scalix instances among the machines in case of failure. This includes automated unmounting and mounting of the shared storage solution and automated shutdown and startup of the necessary Scalix services. The clustering software you can use includes:

• RedHat Cluster Suite
• SUSE Heartbeat
• Veritas Cluster Server

Each instance should be a complete vertical stack, with the Scalix Server, Postgres database, Management Agent, Tomcat and Search and Index Server packages installed. And connectivity from either of the servers to the shared storage must be through direct means such as scsc cable, fiber channel or ISCSI. NFS is not recommended or supported at this time.

Alert: There are three distinct components to the high availability setup outlined here. These steps and Scalix as a company are only responsible for one: Multi-instance awareness. The others are the responsibility of the user or the third-party provider.

• Shared storage (customer responsibility)
• Server relocation software (customer responsibility)
• Multi-instance (Scalix responsibility)

Setting up a dual-server failover cluster requires the following distinct steps. More detailed sample instructions are provided below.

• Plan for both physical and virtual hostnames and IP addresses. For example:

   Physical server with IP address 10.17.96.55 = mail1
   Physical server with IP address 10.17.96.56 = mail2
   Virtual server IP address 10.17.96.57 = virtual1
   Virtual server IP address 10.17.96.58 = virtual2

The virtual instances will be accessed by the email clients and referenced during configuration of the Scalix server software.

• Build the two as if they are simply two servers within the same organization. When installing the operating system, use the virtual hostname and IP address.
• Using the operating system’s DNS configuration process, add the virtual instances to the DNS just as you would a physical server.
• Install Scalix on both servers. One should have the Management Console installed and the other should not. For more on how to install without the Management Console, see the Scalix Installation Guide section about custom installations.
• Disable automatic startup of services.
• Change the name of the server back to the name of the physical server, still retaining the physical server’s IP address.
• Install and configure the cluster software, which recognizes the physical servers as real servers, and the virtual instances as services.
• Fix the instance.conf file

Sample Setup Instructions

The instructions outlined below are provided as an example only. They are specific to RedHat. If you have a different operating system, use them simply as guidelines.

Perform all of these steps on both servers.

To set up a dual-server failover cluster for Scalix:

1. Build the two servers as if they are simply two servers within the same organization. When installing the operating system, use the virtual hostname and IP address.

2. Using the operating system’s DNS configuration process, add the virtual instances to the DNS just as you would a physical server. To add the virtual instance, launch the RedHat Network Configuration window to change the name of the server from the name of the physical machine to the name of the virtual instance.

   # system-config-network

In the window that launches, select the DNS tab and enter the name of the virtual instance, then save and exit the Network Configuration window. For other operating systems, use KDE or the GNOME network setting GUI.

3. Back at the command line, edit the /etc/hosts file to enter the IP address of the physical server.

   # vi etc/hosts
   # ifconfig

Type in the IP address of the physical server, followed by the virtual domain name and the virtual name. For example:

   10.17.120.52   virtual1.scalixdemo.com   virtual1

Save the new information and exit vi.

4. Run a check to ensure that the physical server is functioning with the virtual IP address.

   # ifconfig

5. Install Scalix on both servers. One should have the Management Console installed and the other should not. For more on how to install without the Management Console, see the Scalix Installation Guide section about custom installations. Before moving on, check that Scalix is up and running.

   # omstat -s

6. Disable automatic startup of services. This requires a stop first.

To stop the services:

     # service ccsd stop
     # service rgmanager stop
     # service cman stop
     # service fenced stop

To disable Scalix from restarting upon reboot:

     # chkconfig -level 35 scalix off
     # chkconfig -level 35 scalix-tomcat off
     # chkconfig -level 35 scalix-postgres off  

Alert: If you do not complete this step, you may corrupt your installation.

7. Reboot the server.

   shutdown -r now

8. Verify that the physical server is now offline and the virtual service is up and running.

   # clustat

9. Rename the server back to the name of the physical server, still retaining the physical server’s IP address. For instructions, see steps 2 and 3 above. In this case, you remove the entry in the etc/hosts file that you created in step 3, then save and exit vi.

   # vi /etc/hosts

10. Reboot the server again.

   # shutdown -r now 

11. Restart the cluster.

   # service ccsd start
   # service rgmanager start
   # service cman start
   # service fenced start

12. Run another check to ensure that the physical server once again is functioning with the physical IP address.

   # ifconfig

13. Install and configure the cluster software, which recognizes the physical servers as real servers, and the virtual instances as services. Configure the cluster software to include the two physical machines, and then configure the virtual instance of Scalix with your cluster configuration software program. This typically consists of:

• An IP address
• A shared file system
• A script

14. On both machines, define the virtual instances to Scalix by editing the /etc/opt/scalix/<instance.cfg> and changing the omname to the actual virtual machine name, then changing the value of autostart to FALSE.

15. Relocate the information store to a shared directory so that it can switch back and forth between the two clustered servers.

   # cd ~/mail1 (or mail2)
   # mkdir temp
   # mv /mail1* ./temp (or mail2)

16. Relocate the virtual instance to this machine.

   # clusvcadm -r <virtual hostname> -n <physical domain name>

17. Move everything from temp to the shared disk.

   # mv ./temp/* ./mail1

18. Edit the file /opt/scalix/global/config to change the omautostart value from true to false.

   # vi /opt/scalix/global/config

19. Restart the virtual instance.

   # service scalix start <virtual server hostname>

20. Telnet into the virtual instance to make sure it’s running

   # telnet <virtual server hostname> <port>

21. Stop Scalix.

   service scalix stop <hostname>

22. Copy the configuration files from each machine to the other so that both have the exact same files.

   cd /etc/opt/scalix-tomcat/connector/jk
   scp *-<hostname 1>.* <hostname 2>:/etc/opt/sclaix-tomcat/connector/jk

Then repeat, reversing the hostnames.

   scp mail1:/etc/opt/scalix-tomcat/connector/jk/*-virtual1.*

23. Edit the file workers.conf to include both mailnodes of the virtual hosts. Do this on both physical hosts.

   cd /etc/opt/scalix-tomcat/connector/jk/workers.conf
   vi workers.conf

24. Go to the following directory to make the shutdown port 8006.

   cd /var/opt/scalix/<nn>/tomcat/conf/server.xml
   vi server.xml

Look for the following value:

   Server port =”8005” shutdown=”SHUTDOWN”

Change the port number from 8005 to 8006. While there, also change the short name of the instance to the fully qualified domain name in the three separate places that it appears.

25. Repeat the entire sequence on the other virtual machine.

26. Change the server names in the following file to the virtual machine names.

   cd /etc/httpd/conf.d
   vi scalix-web-client.conf

At the bottom of the file, notice that the diretory refers to the physical directory instead of the virtual. You’re going to move these lines to one of the tomcat.conf files.

Select the bottom seven lines and copy then delete, them.

   Alias /omhtml/ /var/opt/scalix/n1/s/omhtml/ <Directory "/var/opt/scalix/n1/s/omhtml">
       AllowOverride None
       Order allow,deny
       Allow from all
       AddDefaultCharset off
   </Directory>

27. Open the etc/opt/scalix-tomcat/connector/jk/instance-name1.conf

   cd /etc/opt/scalix-tomcat/connector/jk/app-virtual1.*.conf

Paste these seven lines in, beginning with the third line. The first nine lines of the modified instance-name1.conf file should look as follows.

   <VirtualHost virtual1.scalixdemo.com:80>
     Include /etc/opt/scalix-tomcat/connector/jk/app-virtual1.*.conf
   Alias /omhtml/ /var/opt/scalix/n1/s/omhtml/ <Directory "/var/opt/scalix/n1/s/omhtml">
       AllowOverride None
       Order allow,deny
       Allow from all
       AddDefaultCharset off
   </Directory>

Perform this step on both physcial hosts.