CMU Migrations

This guide explains best practices on how to export and import sites, users and settings using CMU, the "Cobalt Migration Utility".

Export:

1.) Be sure your CMU is up to date:

If your source server is a BlueQuartz, be sure to have the latest CMU installed. You should have CMU v2.90-0BX01 or newer installed. If not, you can get it here. To check the version of the installed CMU, run this command: "rpm -qa|grep cmu".

If the source server is a BlueOnyx, run "yum clean all" and "yum update" to install all updates first. That will also make sure that CMU is fully updated.

2.) Make sure no site or user is over quota (optional):

Run this two commands as "root" from SSH:

/usr/sausalito/sbin/disk_restorequotas.pl
/usr/sbin/swatch

The first command syncs the CODB stored quota settings for users and sites with the file system quota. The second command then checks the actual quota usage and stores it into CODB.

Then go to the GUI, click on "Active Monitor" / "Status" / "Disk Usage" and check the "Sites" and "All users" tab to make sure that no site or user is over quota. If a site or user is over quota, increase their disk quota allowance until all sites and users are showing green.

Please note: Since CMU v2.90 it is possible to migrate sites and users even if they are over quota. The import will go through nonetheless.

3.) Directory preparation:

Create an export directory into which you export all data that needs to be migrated. We suggest the following:

mkdir /home/export
mkdir /home/export/data
mkdir /home/export/extra 

That will create a directory named /home/export with two subdirectories in it: "data" and "extra".

4.) Service Preparation:

Through the GUI disable SMTP, POP3, IMAP and FTP, so that no new emails or files can arrive while you do the export.

5.) Export MySQL:

Stop the MySQL daemon:

On all but 5209R the command for this is this one:

/sbin/service mysqld stop

On 5209R you will need to use this:

systemctl stop mysqld

Find out where your MySQL directory is located. It is usually /var/lib/mysql or /home/mysql.

Pack the directory up and store it into /home/export/extra/mysql.tar.gz:

tar czvf /home/export/extra/mysql.tar.gz /var/lib/mysql

Find out your MySQL "root" password in case you have forgotten it by using CCEclient:

/usr/sausalito/bin/cceclient 

This will answer with something like this:

100 CSCP/0.80
200 READY

Type the following:

find MySQL

This will answer with something like this:

104 OBJECT 72
201 OK

Note the reported "object ID" - in this case "72". Yours will be different. Poll the object ID with the"get" command like this (but substitute the reported object ID instead!):

get 72

CCEclient will then fetch the data stored in that object and will display it like this:

102 DATA NAMESPACE = ""
102 DATA sql_rootpassword = "the-supersecret-password"
102 DATA CLASSVER = "1.0"
102 DATA sql_host = "localhost"
102 DATA timestamp = "1322165402"
102 DATA CLASS = "MySQL"
102 DATA savechanges = "1322165402"
102 DATA sql_port = "3306"
102 DATA OID = "72"
102 DATA sql_root = "root"
201 OK

Your MySQL "root" password (if it was set through the GUI) will be reported in the line beginning with "102 DATA sql_rootpassword".

Store the password into a textfile in /home/export/extra/mysql.txt - just to be sure.

6.) Export DNS:

Stop the DNS server:

/sbin/service named stop

On 5209R that would be this command:

systemctl stop named

Pack up the directory containing the DNS zone files and (although we don't really need it) your named.conf:

tar czvf /home/export/extra/named.tar.gz /var/named/chroot/etc/named.conf /var/named/chroot/var/named

If (and only if!) your are using DNS Slave zones, use the following procedure to export the DNS slave zones into a textfile for easy import:

cd /home/export/extra/
wget http://d2.smd.net/scripts/dns-slaves/dns-slavezones.txt
mv dns-slavezones.txt dns-slavezones.pl
chmod 700 dns-slavezones.pl
./dns-slavezones.pl > /home/export/extra/slaves.txt

That will create the textfile /home/export/extra/slaves.txt with the DNS slave zones in them.

7.) Use CMU to export all sites and users:

/usr/sbin/cmuExport -d /home/export/data

The command will tell you that the directory already exists and will ask you if you want to use it. Confirm this with "Yes" and let the export run. This will take a while.

8.) Copy all files to the target server of the migration:

Be sure that the target server for the migration has SSH enabled and that SSH logins as "root" are allowed. That can be configured through the GUI. Then run this command on the source server of the migration:

scp -r /home/export root@target.of.migration.com:/home/

That will copy the entire "/home/export" directory, the subdirectories and all files within to /home/export on the target server.

Import:

Login to the target server by SSH as "admin" and use "su -" to gain root access. Also login to the GUI as "admin".

On the shell confirm that /home/export exists and contains all our data from the source server.

1.) MySQL import:

Stop the MySQL server first:

/sbin/service mysqld stop

On 5209R use this command instead:

systemctl stop mysqld

Remove the existing MySQL database directory:

rm -R /var/lib/mysql

Unpack the MySQL directory from the mysql.tar.gz tarball:

cd /home/export/extra
tar zxvf mysql.tar.gz

Move the extracted "mysql" directory to /var/lib/:

mv /home/export/extra/var/lib/mysql /var/lib/

Start the MySQL server again:

/sbin/service mysqld start

On 5209R that  would be:

systemctl start mysqld

Fix the MySQL privilege table and do any conversion that may be required, but substitute "your-mysql-root-password" with your actual MySQL root password:

/usr/bin/mysql_fix_privilege_tables your-mysql-root-password 

Go the the GUI interface, click on "Network Services" / "MySQL" and enter the MySQL root password (twice) in the first tab to let the GUI know how to connect to MySQL. Afterwards the "MySQL Status" will report: "Connection possible with this settings".

2.) Import the sites using CMU:

/usr/sbin/cmuImport -a -s -d /home/export/data/

This will import all sites and users, the files of user "admin" and the "Extra Admin's" (Reseller) onto the same IP address(es) as before. To import to a different IP address you can use this command instead:

/usr/sbin/cmuImport -a -s -i 192.168.100.1 -d /home/export/data/

That would import all sites onto the IP address 192.168.100.1 instead.

3.) Import DNS:

Remove all existing DNS record. The CMU-Import may have created some records, so do not skip this step:

/usr/sausalito/sbin/dnsDeleteAllRecords.pl --delete-confirm

Unpack the named.tar.gz tarball:

cd /home/export/extra
tar zxvf named.tar.gz

Next we need to do some cleanup and just copy what we need over to /home/export/extra/dns:

mkdir /home/export/extra/dns
cp /home/export/extra/var/named/chroot/var/named/db*.* /home/export/extra/dns/
rm /home/export/extra/dns/*~

Now we have all primary records in the directory /home/export/extra/dns/ ready for import.

Import all DNS zone files:

/usr/sausalito/sbin/dnsImport.pl /home/export/extra/dns

If you have no DNS slave zones, you can skip the next couple of steps and can directly restart the name server again.

Optional DNS slave zone import:

cd /home/export/extra
wget http://d2.smd.net/scripts/dns-slaves/dns-import-slavezones.txt
mv dns-import-slavezones.txt dns-import-slavezones.pl
chmod 700 dns-import-slavezones.pl
./dns-import-slavezones.pl

That will import your DNS slave zones from the optional /home/export/extra/slaves.txt file.

Restart the DNS server:

/sbin/service named restart

On 5209R that would be:

systemctl restart named-chroot

Confirm that the DNS server is running:

/sbin/service named status

On 5209R that would be:

systemctl status named-chroot

Also check the GUI to see if the checkbox for the DNS server is ticked.

4.) Post import cleanup:

After the import it is suggested to perform the following steps on the command line:

/usr/sausalito/sbin/fix_user_UID_and_GID.pl
/usr/sausalito/sbin/web_alias_redirects.pl --enabled
/usr/sausalito/sbin/fix_user_suspension.pl
/usr/sausalito/sbin/fix_web-and-email-server-aliasses.pl

These scripts do the following:

a) Fixes potential UID and GID issues of sites and users.
b) Web aliases for all sites will redirect to the main site name.
c) Users who are supposed to be suspended will be suspended.
d) Web server aliases and Email server aliases will be fixed.

5.) Post import checks:

Use the GUI interface to check that all services which are supposed to be enabled are enabled (Email, POP3, IMAP, SMTP-Auth, FTP, DNS and what not).

Check that automatic updates are enabled under "Software updates" / "YUM updates".

Check that all sites are working and that no site is missing.

Lastly, when everything is working, you may want to remove /home/export to conserve space:

rm -R /home/export

If you have any questions about this procedure or need help, then please don't hesitate to ask for help on the BlueOnyx users mailing list.