The Joomla! Community Magazine™

Complex Joomla! 1.5 Migration With Minimal Downtime

Written by | Saturday, 01 March 2014 00:00 | Published in 2014 March
Joomla 1.5 is not supported by the Joomla project for quite some time now. You still can continue to use Joomla 1.5, but (security) updates will no longer be released. Furthermore, both Joomla 2 and Joomla 3 have many new features and improvements. Perhaps you recognize it: once you use Joomla 2 or Joomla 3, managing a Joomla 1.5 website really feels old-fashioned and you notice how greatly Joomla actually improved.

You should really migrate Joomla 1.5 websites to a Joomla version that is currently supported by the Joomla project. Unfortunately there are still many Joomla 1.5 websites around, because of various reasons. No time, no budget, no motivation, hold off, etc..

Sometimes it’s because of the complexity of a migration. Recently a Joomla 1.5 website with complex conditions needed to be migrated. A great challenge and I would like to share the knowledge acquired to help you migrating any complex Joomla websites you might have.

Joomla website Exact Netherlands

Exact is a leading global supplier of business software and provides a complete ERP offering for small and medium-sized businesses. Exact develops industry-specific on-premise and cloud solutions for manufacturing, wholesale and distribution, professional services and accountancy. Exact is using Joomla for many of their websites for several years now. The Exact Netherlands website was powered by Joomla 1.5 and needed to be migrated.

The website is one of the largest Joomla websites in The Netherlands and the main marketing, communication and sales tool of Exact. Exact has a great importance for a stable website that is always available for its visitors. For this reason, the hosting environment is fully redundant for example.

Many content editors work on a daily basis on the website which is in constant motion. The website uses several (custom) extensions and contains a large number of menu items, articles and modules.

Strong requirements for the migration

For the migration strong requirements were made​​: migrate the website within 1,5 day and two hours was the maximum downtime allowed during the evening hours. The expected result was a website with identical layout, functionality, URLs, menus, articles, etc. but powered by Joomla 2.5. In short: the visitor should not see or experience differences between the Joomla 1.5 and Joomla 2.5 version of the website.

But how do you achieve that? A “typical” Joomla migration (make a copy, migrate it, correct settings and put it online) was not an option for Exact. This would take too long and the copy of the website in migration would be outdated within hours.

Typical Joomla Migration

To get a good overview of the migration the project started with the typical migration. To migrate the Joomla core data I prefer to use the extension SP Upgrade. This paid extension is installed in a fresh Joomla 2.5 installation without sample data. All 1500 articles, 650 menu items, and 950 modules were imported by using this extension.

The website uses several third party extensions, fortunately all with a Joomla 2.5 version available. These extensions were migrated by copying the database tables to the Joomla 2.5 database, something that can be done with SP Upgrade as well, and then install the extensions. The extensions will detect outdated database structures and update them if needed. For the template, a Joomla 2.5 version was already available.

New features, new settings

Until now all steps are part of a typical Joomla migration. The website data is now migrated, but when checking the website you will notice that it does not look as it should yet. This is often caused by different or new settings in Joomla compared to older Joomla versions. A typical example is the Joomla menu module. In Joomla 1.5 you configured to display level 0 to 3, in Joomla 2.5 you need to configure level 1 to 4 to achieve the same output. Level 0 does not exist anymore in Joomla 2.5.

Normally you check all these settings and make adjustments where necessary. Depending on the size of the website this may take hours or even days. Once everything is checked and adjusted, the website is ready put live. This part of the migration was the main problem for Exact, it would take way to long to adjust all settings and it is too unpredictable.

A custom migration script

Still it must be clear which data and setting adjustments were needed to ensure that the Joomla 2.5 version would be identical to the Joomla 1.5 version. Instead of making changes through the interface of Joomla, the changes where made directly in the database, while keeping track of these changes.

All these changes have been incorporated into a customized script that consists out of a set of SQL queries. Collecting all these necessary adjustments will take time, but once the script is completed, the SQL queries are executed in seconds.

Menu level changes

Let's create a simple custom migration script and use the module level setting adjustments as an example. Create a file called “migrate.php” and place that in the root of the website. Add the code below in that file. The code will change modules with ID’s 233, 294 and 357. The parameters of these modules are modified in a loop, and after that saved in the database. The “startLevel” of module with ID 233 is changed from “2” to “3”. As additional examples the “menutype” is added for module with ID 294 and the “You are here” text is set to “hide” for the breadcrumbs module with ID 357.

define('_JEXEC', 1);

// Connect with Joomla
define('JPATH_BASE', __DIR__);
require_once JPATH_BASE . '/includes/defines.php';
require_once JPATH_BASE . '/includes/framework.php';

// Database connection
$db = JFactory::getDBO();

 * Module changes
// Get the modules that need changes
$query = $db->getQuery(true);
	->where('id IN (233,294,357)');
$modules = $db->loadObjectList('id');

foreach($modules as $module)
	$module->params = json_decode($module->params);

// Change startLevel
if ($modules[233]->params->startLevel == 2)
	$modules[233]->params->startLevel = 3;
	echo('Module [233]: startLevel: 3 
'); } // Set menutype if (empty($modules[294]->params->menutype)) { $modules[294]->params->menutype = 'mainmenu'; echo('Module [294]: menutype: mainmenu
'); } // Hide breadcrumb "You are here" if (!$modules[357]->params->showHere) { $modules[357]->params->showHere = 0; echo('Module [357]: showHere: 0
'); } // Save new module params foreach($modules as $id=>$module) { $params = json_encode($module->params); $mod = new JObject(); $mod->id = $id; $mod->params = $params; $result = $db->updateObject('#__modules', $mod, 'id'); }

Full script as Gist:

Extension specific changes

Similar to this adjustments can be made anywhere in the database. For the extension ZOO all items needed adjustments for the access setting. Because of the migrations the items were not visible anymore, which is obviously not the intention. You can add the following code to the migration script to ensure that the access for the ZOO items is corrected in the database.

 * Component changes
// ZOO access fixes
$query 		= $db->getQuery(true);
$fields 	= array('access=1');
$conditions = array('access=0');

$result = $db->query();
echo('ZOO: Access set to 1 

Full script as Gist:

Drastically reduce migration time

You can probably imagine that the script eventually used for the Exact migration is many times bigger, but above gives you an idea about the power of a custom migration script: all actions that you would normally perform for a migration in the Joomla backend are done in just seconds, instead of hours or days. This drastically reduces the time needed for the migration, something we could really use for the migration of the Exact Netherlands website.

An additional advantage of the migration script is that you can test and improve the migration repeatedly until the desired result is achieved.

Joomla 2.5 base

In addition to database changes it is often necessary to make server files changes, such as template overrides. This is done during the testing process as well and resulted in a Joomla 2.5 base installation that included all files necessary for the final Joomla 2.5 version of the website.

For extensive testing of the custom migration script each time a fresh copy of the Joomla 2.5 base installation is used. In this way the migration is well prepared and unexpected surprises during the final migration are eliminated.

The big migration day

After weeks of preparations we were ready for the final migration day. Several last test migrations were made and the Joomla 2.5 base installation was prepared in a subfolder of the domain. Everything was ready to for the final migration of the website, for which we used the following steps:

  1. Back-up Joomla 1.5 website
  2. Take (Joomla 1.5) website offline
  3. Migration of the Joomla core data by SP Upgrade (+/- 1 minute)
  4. Copy extension tables to the Joomla 2.5 database (+/- 1 minute)
  5. Operations in the Joomla 2.5 website (+/- 2 minutes)
    • Installation of 3rd party extensions
    • Execute upgrade scripts of 3rd party extensions
    • Start indexing content by Joomla Smart Search
    • Solve asset issues with ACL Manager
  6. Execute custom migration script (+/- 0,5 minute)
  7. Move Joomla 1.5 website files to archive folder (+/- 0,5 minute)
  8. Move Joomla 2.5 website files to website root (+/- 0,5 minute)
  9. Change the /tmp/ en /log/ paths in configuration.php, and delete the custom migration script from server (+/- 0,5 minuut)
  10. Put (Joomla 2.5) website back online

Successful Joomla migration in six minutes

As you can see the Exact Netherlands website was down for just six minutes. During this downtime the the entire website with all its data was migrated from Joomla 1.5 to Joomla 2.5.

Visitors noticed no difference, forms submissions from minutes before the downtime were processed and migrated successfully. Content editors that made changes in the Joomla 1.5 website earlier that day saw a renewed Joomla administrator after login, but found all changes back in the migrated website. No data was lost.

The successful migration with the very short downtime needed to be celebrated. For that we enjoyed the one and only freshly baked Joomla Stroopwafels!


A couple migration take-aways

  1. Clean up: make sure the website is completely cleaned up, Remove articles, categories, modules and menus that are no longer in use and empty the trash before the migration.
  2. Backup, and test backup: always make a backup of the website and test if restoring the backup is working correctly.
  3. Test, test, test and test: I’m sure you won’t like surprises during a migration. Know what to expect, always test a migration before migrating a live website.
  4. Emergency revert: make sure you can always put back the old version of the website at any moment during the migration.
  5. Don’t hold off: migrate your non-supported Joomla websites now. Joomla 2.5 and Joomla 3.2 are the versions currently supported by the Joomla project. They are safe, stable and have loads of great new features and improvements.
Read 18517 times
Tagged under Administrators, English
Sander Potjer

Sander Potjer

The passion for Joomla began for Sander Potjer in 2005 with building a site for his rowing team. He was immediately impressed with Joomla and the community surrounding it. Soon he became personally involved and co-founded and started the Dutch Joomla User Groups together with others.

From this community the Dutch foundation "Stichting Sympathy" emerged, to promote and support Joomla in the Netherlands, which Sander chairs. This foundation also organizes the annual Dutch JoomlaDays. Internationally he volunteer to Joomla as a member of the Community Leadership Team (CLT).

Sander is co-owner of Perfect Web Team where they develop, maintain and support web solutions for complex situations based on Joomla. He is also the creator and developer of ACL Manager, a popular Joomla extension that simplifies the management of access permissions.

Very good article and very similar to the project plan we have for the migration of our website (planned for later this year to go from 1.5 to 3.5 - when released and stable). Not sure we would achieve the 6 minutes cut over time and it is not a requirement.
The only thing I found missing in the article (which I know has to be quite short for the magazine) was more details of the planning, migration development and pre-migration testing timescales and effort that you planned for/actually experienced in the migration. I realise you may not want to share this but even some high-level details would be useful.
Hi Brian,

Thank you for your comment! Happy to share more details about the timescales.

As this was the first time working with a custom migration script (as in the example) it took more time to find out the best approach for a migration with minimal downtime. Without taking that into account the timescale was like:
- 2 days: initial scan to find out what extensions are in use, and how (if possible) to migrate each of them
- 1-2 weeks: test-migrations while writing & testing the custom migration scripts (this will really depend on the website that needs to be migrated)
- 1 day: actual migration day
- 3 days: after care (not needed, mainly used for further improvements of the site)

Does that helps you further? Let me know if you have additional questions.

Sounds like a really fantatic adventure ! :-)
Fadi (itoctopus) Monday, 10 March 2014
Hi Sander,

The "Emergency revert" point is a very important point. I have seen many Joomla migrations gone wrong when made live (technical errors, performance issues, etc...)

Always have your old database ready, and a backup of your old filesystem. Should anything go wrong, identify the issues, revert to the backup, fix the issues on the migrated website, then go live again.

Generally, the Migration should be done in the weekend during off hours.
Che Linh Nguyen Tuesday, 11 March 2014
Thank you very much for all your useful information !!!
Great, i have personally migrated a fairly large Automobile site with over 250,000 users pm. It requires lot of detailing

Just a thing to make your client happpy - Site is running poor as having a duplicate - both page versions are there and without canonical

may put canonical or redirect - for better search engine perspective
Well written article once again. Compliments !!
Hi Saahil, thanks for your feedback. Forwarded your suggestion to the client!
Why was the project not updated to version 3 ?
Thanks for the pointers on this, it all makes sense and is very logical.
The point about having the original available is very important, although I was wondering why you did not do this on a development area first then simply upload the 100% working new site first... would be interested to know the reasons for doing this on live.
Hi Matthew.

The client prefers to use the Joomla Long Term Support (LTS) versions, so 1.5, 2.5 and later on 3.5. So thats why the website is migrated to 2.5 as Joomla 3.5 is not available yet.

Actually the migration was done on a "development area", but in a subfolder of the live website. Once successful migrated the site was moved to the root directory, and the 1.5 website moved to a subfolder (see steps in "The big migration day" section).

This is much quicker than downloading & uploading a large website, and it eliminates the risk of live environment being different from live. If the migration would have failed, the 1.5 site would simple be turned online again.

Hope this makes it clear, let me know if not.

Wow, impressive Sander and very nice to read what it takes when upgrading a site like this with minimal downtime.