Easy Migrate
This guide explains how to easily migrate all relevant content such as Vsites, Users, Emails, Cronjobs, MySQL databases and users, DNS and Server Configuration from one BlueOnyx to another.
Easy Migrate is a replacement for the older Cobalt Migration Utility (CMU) and was designed from the start to do a much better job at getting your data from one server to another. The goal was to make it easy to literally clone entire Vsites from one box to another - with all relevant data. With Easy Migrate it is now possible to set up multiple BlueOnyx and have them serve the same content by simply using "Easy Migrate" to once get a Vsite duplicated and to then keep it synchronized.
Opposed to CMU the new Easy Migrate will use RSYNC over SSH for file transfers and this allows you to copy over just the changes between Vsites and Users. During an Easy Migrate run the CODB settings that contain data about the configuration of a Vsite and its Users are also (once cloned) synchronized via SET transactions. Additionally Easy Migrate is able to copy data and configuration of installed Web Applications from the BlueOnyx Shop across - provided both servers have the same PKGs installed.
At this time "Easy Migrate" is a command line script. This guide explains the command line interface and how you can use Easy Migrate.
Easy Migrate can export data from: BlueOnyx 5207R, 5208R, 5209R, 5210R and 5211R
Easy Migrate can import data to: BlueOnyx 5209R, 5210R and 5211R
Server Preparation
Easy Migrate operates in a "Pull" fashion. So you perform the migration FROM the target server of the migration - the box you want to migrate to. This server must be a BlueOnyx 5209R, BlueOnyx 5210R or BlueOnyx 5211R. Easy Migrate (RPM-name: easy-migrate) came aboard with a recent YUM update and you can find the executable in /usr/sausalito/sbin/easy-migrate.pl. You must be user "root" to execute this script and the script itself must be able to connect to the source server of the migration via SSH. You must first install the SSH key of user "root" of the target server on the source server of the migration. You can easily do so this way (if you haven't already done so):
~]# ssh-copy-id root@<source-server-name-or-ip>
Afterwards confirm that you can login as "root" to the source server without being prompted for a password:
~]# ssh root@<source-server-name-or-ip>
If that works, press CTRL+D or type exit to return to the target server of the migration.
Please note: 5211R (EL9) uses a more modern SSH and some older ciphers have been deprecated. This is no issue if you're trying to import from BlueOnyx 5210R or BlueOnyx 5209R. But if you still have a BlueOnyx 5207R or 5208R and want to Easy-Migrate straight to BlueOnyx 5211R? In that case you need to create the file /root/.ssh/config on your BlueOnyx 5211R and add the following text into it to allow user "root" to use older HostKeyAlgorithms and key-types:
HostKeyAlgorithms +ssh-rsa,ssh-dss
PubkeyAcceptedKeyTypes +ssh-rsa,ssh-dss
That will also allow Easy-Migrate to work. When you're done with the migration, just remember to remove the file again.
Before you get started please use the GUI on both source and target server and make sure that the target server has the same shop PKGs and Web Applications installed that the source server has - provided you want to continue to use them on the target server as well. The version numbers of the PKGs don't need to be the same. If something is a bit newer or older it doesn't really matter as far as the migration is concerned.
Now let us look at easy-migrate.pl. If you run it without options or with the -h switch, it'll show you the available options:
As a first step you should test if Easy Migrate can establish a connection to the source server of the migration and if that server is compatible for an import of data:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --check
This will tell you if something went wrong. If not, then you can continue with the migration.
Migration of Server Configuration
Often your source server has many small config changes to services and features and you've done them so long ago that you don't even remember half of them or where they are to find. Easy Migrate takes care of that . if you let it:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --config
This optional step tasks Easy Migrate to fetch the configuration of the following services from the source server and to adjust the settings on the target server accordingly:
- Anything Email related: SMTP, SMTP-Auth, POP3, IMAP, relay configuration and so on.
- The general configuration of Apache, Nginx (if present) and Subdomain defaults.
- PHP settings.
- FTP settings.
- DNS settings.
- SiteStats settings and default settings for new Vsites
- Your name and email address in the "Support" settings.
- YUM configuration for the daily updater.
- Active Monitor settings and notification email addresses.
What it does NOT touch are the SSH settings as well as the GUI desktop settings (like forcing the GUI to HTTPS). For security reasons Easy Migrate will not touch these two.
DNS Migration
Optional step. During a Vsite migration (or a migration of all Vsites) Easy Migrate will directly import the DNS records of the zone that the Vsite belongs to. Provided the source server had DNS enabled and had the records for that zone. Easy Migrate will also copy the DNS zone include file over, so any custom additions you made to those files will not get lost either. However, if you want to just migrate the DNS or also need to migrate DNS zones which are not associated with Vsites on the source server, then you can do so this way:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --dnsonly
That will migrate all SOA records, all DNS Records and also all DNS Slave Zones.
Vsite and User migration
If you want to migrate just a single Vsite and all its related data (website, SSL certificate, all settings, all Users, their data, emails and settings as well as the associated MySQL and DNS data and all emails), then you can run Easy Migrate this way:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --ip <new-ip-of-vsite> --vsites www.site.com
That will migrate just the Vsite "www.site.com" and it will directly give it a new IPv4 IP address as specified with the --ip switch. If the source server is carrying DNS records for that Vsite, these records will be imported as well and their IP will be changed to the new IP address, too. If you do NOT want to import DNS, you can add the --nodns switch. If you want to assign or change an IPv6 IP address, then you can add the --ipv6 switch and specify the new IPv6 IP of the Vsite.
Please note: A Vsite will fail to import if it has a MySQL user and/or database who already exist on the target server. If that happens, please delete the MySQL user and database manually or via phpMyAdmin and try the import again.
Example for importing two Vsites, giving them new IPv4 and IPv6 IPs and NOT importing any DNS data:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --ip <new-ip-of-vsite> --ipv6 <IPv6 IP> --vsites www.site1.com,www.site2.com --nodns
If you want to migrate ALL Vsites simply use --vsites all instead:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --ip <new-ip-of-vsite> --vsites all
If Easy Migrate gets interrupted during a migration? Don't worry. You can simply run it again with the same options. If it partially migrate a Vsite it will just migrate it again by updating the settings and RSYNC'ing the rest of the data.
MySQL Migration
Optional step. You should *only* run this *after* all Vsites have been imported, because a Vsite import will fail if it has a MySQL user and/or database who already exist.
If all your Vsites use MySQL databases and users were created via the GUI or by the WebApp installer, then you may not need to perform this step. But if you have any manually created MySQL users and databases, then you should run it. In any case it won't hurt if you run it:
~]# /usr/sausalito/sbin/easy-migrate.pl --source <IP-of-Source-Server> -p 22 --mysql
That will create SQL dumps of all MySQL users and their grants (except for user "root"!) as well as MySQL dumps for all databases. It then copies the SQL-dumps over and directly imports them on the target server. This will replace any user or database of the same name that are already present on the target server.
Example output from a single Vsite migration
This shows a successful migration of a single Vsite:
First the Vsite was created on the target server - using the same CODB settings as it had on the source server - with the exception of the IP address, which we had changed during the migration. During the creation of the Vsite on the target server Easy Migrate checked if the group name "site17" was available. It was the same group that the Vsite had used on the source server. As that group name was free, the Vsite was created with the same group name. Had it not been free, the first free group name would have been used instead.
Next Easy Migrate configured all so called NameSpaces of the Vsite. These NameSpaces covered the PHP settings, SiteStats settings, Userweb configuration, Redirects, Shell configuration, CGI, Disk, SSi, FTP access, DNS, Subdomains, defaults for new Users and (in this case) the settings of the Shop PKGs WebApp Installer, Wordpress, Roundcube and OwnCloud.
However: Easy Migrate encountered several NameSpaces that existed on the source server, but which do not exist on the target server. AnonFTP for example (which has been deprecated) and also NameSpaces for PKGs which we chose not to install on the target server. Easy Migrate reports this, but it doesn't fail during that stage. So this is just an info for your benefit.
Easy Migrate then created and updated the three Users of this Vsite and adjusted all their settings accordingly.
After that Easy Migrate started several RSYNC's over SSH to copy the SSL certificates and all Vsite related folders over to the target server. It then imported the SSL certificate. And if this was a Let's Encrypt certificate, then that certificate will automatically renew at the configured renewal date, because Easy Migrate imported the settings for that as well. As you can see, it then imported the SQL database for the RoundCube Web Application, because this Vsite was actively using it.
In the final step the contends of the home directories of all Users of that Vsite are imported via RSYNC over SSH and of course that includes their emails as well. Should an individual user have a cronjob, then that cronjob is imported, too.
Potential problems during a migration:
Username conflicts:
If you try to import a User with the same username as is already in use by an existing Vsite and that Vsite does not have the same fully qualified domain name as the same user belongs to on the source server. Example: Vsite www.company.com has user "charly". But on the target server the Vsite www.happycompany.com also already has a User named "charly". At the stage where user "charly" is being imported Easy-Migrate will stop with an error message, telling you that user "charly" already exists and what Vsite he belongs to. This is a conflict that you need to resolve on your own, sorry. You need to delete one of the two users if you want to continue the migration of that Vsite.
MySQL/MariaDB database import problems:
There are a few things that can go wrong during the MySQL database and user imports. Say the source server was really old and used MySQL user passwords with the old, short hashes from MySQL >4.1? Support for that has been removed from newer MySQL versions, but you may still have the odd old user here and there on really old boxes that were using even older database imports from before. Easy-Migrate will import these users, but their passwords won't work. If you run into this, then you have to manually reset the passwords of the affected users.
Other things that may go wrong? Say you imported MySQL first with --mysql and then import all the Vsites. Which will import the MySQL users and databases again and during each of these imports it will try to drop the database, re-create it empty and then imports the SQL dump from the source server. Sometimes it fails to drop the databases and that might cause Easy-Migrate to fail. If that happens: Use phpMyAdmin and delete the problematic database(s) first and then run Easy-Migrate again.
Conclusion
Easy Migrate lives up to the promise of making a BlueOnyx to BlueOnyx migration really easy. It gets all the relevant data over in a fast and efficient way. All you need for the migration is just /usr/sausalito/sbin/easy-migrate.pl and that single tool does all the heavy lifting for you. It also opens new doors like in those cases where you need to clone a Vsite across multiple servers for load balancing purposes or as a creative backup solution when you want poor man's clustering over two servers.
Try it out and let it surprise you with a perfect and easy migration.