The Joomla! ® Community Magazine

Automating Your Component Demo Site

Written by David Hurley | Saturday, 01 December 2012 00:00 | Published in 2012 December
Level of Difficulty:Advanced These days almost all Joomla! developers have a demo site to display their component for testing. It really doesn’t matter if you give your component away or if you are offering it for purchase; most users expect to be able to test the functionality and “see it in action” before they purchase or waste their time downloading and installing on their own site. This means as a component developer you have to keep a demo site up and running at all times with the latest version of your component.

There are several different ways to accomplish this. I’d like to present the way our shop at www.crmery.com has found to be most successful for us. I don’t imply this to be the only way to run a demo site (or even the best way – we seem to be always tweaking things), but this has proven to be extremely efficient for our purposes. I thought it may be helpful to others as well.

We use GitHub for managing our software versions and as a result we have some very powerful tools that we can take advantage of to make our demo site always stay in sync. You can configure your repository in a variety of ways and in another post I will go into detail about how Joomla! handles their repositories and how we have chosen to handle ours. The bottom line is that you need to have a master or production branch. This branch only contains stable code. You should be able to download your code from this branch and it will install and run without any problems. This is the branch we want to reference for our demo site.

GitHub provides hooks to a variety of applications and functions. You can find these by going to the admin section of your repository. Once in the admin side you will find the “Service Hooks” on the left hand menu as referenced in the image below.



Clicking on the Service Hooks will give you a list of available services you can reference. GitHub hooks provide a convenient way to receive information about each commit as it’s made on your repository. Any hook you define will be triggered every time you commit to your repo. Don’t be overwhelmed. We are only going to look at one, the very first option, Web Hook URL’s.  What we are going to do is enter a path to a location on our server where we have a special git folder. You will need to enter the URL in the standard browser accessible format:

e.g. http://demo.crmery.com/gitrepo/git.php

Save your changes and you’re done for the moment on GitHub. We’ll revisit this later to test the service hook and add our SSH keys. Now you need to go setup this special git folder on your server. This will involve a couple of steps. The first thing you should probably do is make sure that git is enabled and running on your server. You can do this by opening up a terminal window and simply typing

git --version

You should see a result similar to the following:

git version 1.8.0

If you get a command not found error then you will need to setup git prior to continuing.  Once you know git is installed on your server you’re ready to move to the next step. You will need to create the directory and file which you entered at GitHub. In our case we need to create a folder called gitrepo and then add a file named git.php.

For the contents of the git.php file we add the following:

<?php
if($_REQUEST['payload']) {        shell_exec('/usr/local/bin/git fetch --all &&
        /usr/local/bin/git reset --hard origin/master');
}
?>

This is the file and code that will be run each time your GitHub hook is triggered. GitHub will deliver a set of post variables, namely one called payload. This contains the details of the commit. If you have a valid payload then this file runs a git command to interpret the payload.

Before you test your hook and script there are a few more things we need to set-up. Don’t give up, we’re almost there.

Your GitHub requires authentication before making any pulls remotely so you need to configure your GitHub and your server to authenticate silently and automatically using SSH keys. GitHub has a wonderful article on configuring your SSH keys so I don’t have to go into detail about that process, suffice to say, it’s simple to follow their steps.

*One thing to note here – be sure that the user you use to create your id_rsa key is the same user that will run your git command in your git.php file above! In most cases this is the apache user. This is critical for authentication.

After you have configured your SSH keys you will need to test this by running a git command as this user and make sure your connection to GitHub authenticates and works correctly. Since we need a starting copy of the repository to begin with anyway we can test our authentication at the same time. Navigate into your gitrepo folder you’ve created previously and run the following commands:

git -clone 
 This email address is being protected from spambots. You need JavaScript enabled to view it.
 /your_repo_path/repo.git
git -fetch --all origin/master

You should see a connection established to github and a clone of your repository begin. You have now copied all the files from your repository into your gitrepo folder. Congratulations! Your git settings and your SSH keys are working correctly. Now you can test your GitHub hook by posting a commit to your repository and checking the files in your gitrepo folder on your server. You should see your changes almost instantaneously. If you do then you have only one small step remaining and your automatic demo site updating will be configured.  If you do not then you will need to step back and see at what point the process is failing. You can always use the GitHub guides to answer any specific questions.

The last step to setting up your automated component demo is to make sure that your Joomla! site uses those files located in your gitrepo folder. The easiest way to accomplish this is to set a few symlinks. By setting symlinks you can keep your directory structure of your github repository the same within your gitrepo folder on your server. In case you need an example, here is one way that might look:

/gitrepo/
git.php
manifest.xml
/package/
/component_folder/
install.php
component.xml
/admin/
/site/

Simply create a symlink in the Joomla! component folder on the front side to the site folder within the above structure.

ln -sv /home/yourpath/public_html/components/com_component 
/home/yourpath/public_html/gitrepo/package/component_folder/site

Create a second symlink for your admin folder on the administrator side of Joomla! and you are done.  Now every time you push a new version to your master branch of your GitHub repository you will automatically update your demo site with the newest code.

There is one little matter of the database that needs to be configured but you can handle the updating of that by simply running your install.php script as part of your git.php file.

Leave a comment if you’d like to see a follow-up post describing these steps and how to keep your databases always in sync through GitHub as well.

I hope this article has helped you see one way in which you can reduce your workload by updating your component demo site automatically. Obviously the same methodology holds true for modules and plugins as well. Be sure to watch for my next article on how to simplify your deployment procedures from your desktop to your member’s only file repository.

Read 45348 times
Tagged under Developers, English
David Hurley

David Hurley

David is the Community Development Manager for Joomla! and liaison with both the Production Leadership Team and the Community Leadership Team. He works at WebSpark and has produced several well-known extensions for Joomla! CMS (CRMery, Banter, JForce), as well as Cobalt, considered to be the first complete web application built on the Joomla! Platform.

Leave a comment

Make sure you enter the (*) required information where indicated.

[b] [i] [u] [s] [url] [quote] [code] [img]   

Comments (11)

  • avatar
    • 1
    • 0
    Nick Savov

    Thanks for writing this, David! I was surprised by how simple the process is.

    I'd like to see a follow-up post describing how to keep the databases always in sync through GitHub :)

  • avatar
    • 0
    • 0
    David Hurley

    Thanks Nick!

    I appreciate the comment. It does feel so simple once it's been done. But isn't that the way it goes? It's always easy in the end ;) - We had a real time with some of the authentication and getting it updating out correctly. The simple solutions are always the best it seems. :)

    I hope to write the next post for the next issue!

  • avatar
    • 0
    • 0
    Phil

    Good article on hooks, but this article should simply be retitled - "Deploying a site with git" as it describes that process good, but it doesn't really describe the correct and secure way of "Automating Your Component Demo Site"

    This is a dangerous article if your demo site has any kind of user upload/admin login privileges then your demo site can be used to hack your server or worse.

    Any demo site should be completely destroyed (removing all files) on updating it (every hour/day/deployment) if user login is allowed (and uploads enabled) as a git reset will not remove uploaded hacker files.

    You should also remove and replace the database on refresh - incase someone has already created a super admin or worse

    If you are just running a demo site with no login then also DELETE the components you are not using com_config, com_user, etc...

    I speak from almost 10 years of running a Joomla based Demo site for extensions :-)

  • avatar
    • 0
    • 0
    David Hurley

    Hi Phil!

    You're absolutely right that there is quite a bit that goes into securing a website, demo or otherwise. And quite certainly there are more steps involved with fully setting up a demo site, however the title of this article was automating your component demo site, not setting one up. :) You raised some valuable insights that users should consider when setting up a demo site and you have some great points which would be awesome to see in an article discussing fully setting up a demo site and the security aspects as well. This article is obviously not mean to be an all inclusive guide to configuring a demo environment but rather some tips on automating merely one aspect of such! Of course a demo site is refreshed quite consistently (hourly, daily etc...). This article only discusses updating versions of your extension as you create them, which would then follow the same refresh pattern as you mentioned.

    I'd love to see you write an article going into more depth regarding site security of a demo and live site!

  • avatar
    • 1
    • 0
    Dinh Hung

    Thank you David for sharing your experience.

    If you look on the left of our demo server at http://www.joomlart.com/demo/ you will also see a Demo Site builder Tool which allow you make a clone of our template quickstart with full administration access.

    The interesting feature of the tool is that you can pick up any version of any joomla extension (available in the inventory) to install with any version of the core Joomla version. If you have a look at one of our extension, eg: JA Slideshow Lite http://update.joomlart.com/#products.list//JA Slideshow Lite Module/, you can see which template is using that module at which versions.

    @Phil: You are right about security issues. That is why we decided to put the demo builder site to a separate dedicated server, all installations will be removed after some time and the whole server software will be reloaded from a clean copy every 3/4 weeks.

    Our solutions works both with SVN & GIT, if I have a chance, I hope I can share our solution with all of you guys as well.

  • avatar
    • 0
    • 0
    Chris Paschen

    Just a quick clarification on where to find the 'service hooks' (at least right now.
    You need to click on the 'Settings' tab in your repository.
    (There isn't anything called 'Admin', at least now right now)

  • avatar
    • 0
    • 0
    Dwi Rianto

    Worth to try. Thank you so much for sharing @David.

  • avatar
    • 0
    • 0
    Mike Fatica

    We've had a good amount of success with the same deployment technique, though for an actual demo site, where users need to login with provided credentials, I think the following are also necessary:

    1) Make the entire site's filesystem read only,
    2) Create a script to drop the entire site's database hourly, and reload from your starting point restore point.
    3) Delete com_install

  • avatar
    • 0
    • 0
    Scott Greenwald

    Great article David! Can you explain more about how to sync the mysql database with git? I've seen very few [good] articles on this topic. Thanks!

  • avatar
    • 0
    • 0
    Olivier NOLBERT

    Thanks for this great article.

    I think the same approach to automate the creation of extension installation zip file would be great.

  • avatar
    • 0
    • 0
    Tim

    Nice article!

    I'm not very familiar with the ln command but aren't the source_directory and target_directory in the wrong order? Shouldn't the command be:

    Code:

    ln -sv /home/yourpath/public_html/gitrepo/package/component_folder/site /home/yourpath/public_html/components/com_component

    That would put your repo code in your Joomla's component directory.

    But there's a good chance I missed something.