The Pizza bugs and Fun event in February 2024 is over, Pizza is eaten and most likely digested, but Bugs and Fun persist. You might have caught the bug testing and enjoy the feeling of marking a test as successful. When one of your tests is crucial for the merging of a PR or prevents a catastrophe - how cool is that?
However, as a non-developer, you might not know how to continue the PR testing on your own, so here are some hints.
Join the Bug Squad Channel
If you enjoy working in a team, join the Bug Squad channel on Mattermost. Bug Squad is a https://docs.joomla.org/Bug_Squad team in Joomla that deals with issues. Here, you can see discussions, and calls if urgently needed tests are requested, or you can find help if you are stuck in with difficult test.
Prepare a testing environment
The best testing method for non developers is: make a copy of one of your websites where you can play and test and where you have real data: the more, the better, the more extensions and languages, the better. It must have one of the supported stable versions.
Then download the latest Patchtester version from https://github.com/joomla-extensions/patchtester/releases and install the component like you would install any other extension.
Find issues and Pull Requests
You have already made your first steps with GitHub and the Issue Tracker during the Pizza Bugs and Fun event. You may be wondering why we use two systems. The reason is: once upon a time, there was no GitHub. Joomla needed a system for managing issues, and the issue tracker was developed.
About 12 years ago, GitHub emerged and it is now the home of Joomla development. All relevant information for developers is on GitHub, but we keep the issue tracker for collecting test results. It has a friendlier interface and an easy-to-use search tool.
It is not as easy to find an issue or Pull Request you can work on. There are over 800 issues and more than 200 open pull requests - so you have the choice. You can filter:
- Every issue can be checked
- A pr ready to commit (RTC) is already tested
- A DRAFT is not ready for testing
- The milestone / branch above your testing environment or your PHP version helps you to find out if a PR is in scope for you. For example: A test of a pull requests for Joomla 6.0 won't work when your site is still on 4.4.
- If you are not a developer you also can ignore pull requests with an NPM label, they need to be built before testing. But if you want to, you can test these PRs using the prepared package.
Check new issues
Testing PRs is important but there is another task which could be interesting for you: checking new issues.
Everything can be reported as an issue, also a question or a feature request. Feature requests are not in scope here, but in general, issues are about bugs. The descriptions are not always clear, sometimes due to language difficulties, in any case they must be read carefully and must be replicated if possible. It helps if someone can confirm a bug and add precise information to the description.
If you see a new issue, try to reproduce it on your own site. If you are able to replicate the error – add a comment on GitHub confirmed, otherwise could not replicate and some more information if you can.
Test Pull Requests
Testing with Patchtester
The usage of the patchtester is well known from PBF. If you forgot - see the Video tutorial https://www.youtube.com/watch?v=nXNN5rpddcA
Testing with prebuilt installations
This is the easiest way of testing very big pull requests which also have the NPM label and contain css and javascript changes. Here you can download and install a complete Joomla or an update package and update your test installation with this package.
You can find the download on GitHub. Select the PR in GitHub and scroll down until here:
You get a page that lets you download a package or lets you directly update your site with the PR.
Testing... go, go, go…
Testing itself is up to you - and sometimes it is not easy. If you have a question, the Bug Squad channel is a good place to ask for information.
Don’t think that only a successful test is important. Maybe you doubt if the whole thing makes sense and is user friendly? Then write a comment.
Tests can become invalid
Suppose you have tested a PR and proudly gave it a successful test in the issue tracker.
A short time later you find out that your test is no longer valid, This is frustrating.
As soon as something changed in the code after your test, the system invalidates previous tests. If it was an easy test, you simply could repeat it but if it was a difficult and time consuming test you can ask in Bug Squad. They can decide to re-validate your former test if the changes in code are without risk.
And a last warning…
Please never think: I don’t understand what’s going on here, the test is not ok, but this PR comes from SANTACLAUS. She is famous, this is surely perfect and does not wait for my humble test, I click on test success. This is wrong. Who works makes mistakes and surely SANTACLAUS can make mistakes. Test carefully and be honest, if in doubt, ask Bug Squad or write a comment!
Have Fun!