In Joomla!, we currently use three kinds of tests.
Unit Tests – testing small chunks of code (units) on the lowest possible level, usually methods, without including other classes or methods. These tests ensure that methods do their job as expected.
Integration Tests – testing the integration of two or more units. These tests cover the interaction between several units.
System Tests – testing the system as a whole. Testing an application through the browser belongs to this group.
To ensure the overall quality of Joomla!, a unit test coverage of more than 90% is needed. The current code base does not always allow unit tests. Caused by the current tight coupling of objects, we are mostly restricted to integration and system tests.
Test Driven Development
For the future it must be the goal to enforce Test Driven Development, ie., a developer writes a unit test first to document the expected behavior, and after that implements the code. Thus, a very high unit test coverage is guaranteed. Evolving the code base will never again break existing functionality, because that would be detected instantly.
Utilizing Dependency Injection instead of the currently high coupling will enable unit tests throughout the project. An easy way to achieve this would be to make JFactory's methods non-static and pass an (the) instance of the factory as an argument. That way it is no problem to inject a mock factory that delivers mock objects for testing.
No Test – No Fix
To fix a bug, a test must be written to reproduce the problem. Ideally this would be a unit test, but often this will end in a system test. Then a solution is created to make the test succeed, which proves the fix.
CMS Unit Test Working Group
The platform has done a great job adding automated test coverage for their code. With Joomla! 3.0, it is our goal to introduce automated tests for the CMS as well. Within the next couple of weeks, the CMS Unit Test Working Group coordinated by Michael Babker will work on setting up the test environment for the CMS with version 3.0 as the base for development of the test structure.
A guideline for writing and contributing tests, including some tutorials, will be published on the wiki as soon as the setup is ready.
If you'd like to join the effort, you just need a GitHub account, so you can watch or fork the repository and send pull requests with your contributions.