Thank you Harald Leithner!
Joomla exists thanks to the dedication of its volunteers. In this article we would like to express our gratitude and respect to Harald Leithner who served for long time as Release Leader for the Joomla 3.9.x series. Harald has served the project in several capacities over the years, he was also the Production Department Coordinator in the 2019-20 term. But let's get to know in depth the daily job of the Release Leader, with a short interview directly to Harald.
Could you describe the usual “Release Day” for a Release Leader?
The usual release day begins one week earlier, by collecting the last RTC (Ready to Commit) PRs (Pull Requests) for the release. Depending on the complexity of the PR I do at least a code review on all PRs I merge. After this I check if we have pending Issues/Pull Requests set for the upcoming release milestone. If a PR which should go into the release has a missing test or some open discussion I try to solve the situation. That means I try to find a tester or test it myself. If there is no common sense on a PR I check with other maintainers what to do with the PR and merge it if we say it’s ok. After finishing this part I create the Release Candidate. That’s basically some scripts I execute and upload the tag and the files to github. (Actually it’s more work then pressing some buttons). Then I notify the Release Team about the upload of the release candidate and give them some hints about what I would like to have tested a bit more carefully. For example PRs that look complex or risky to me. Depending on the feedback from the Release Team I fix fundamental issues reported and create another RC or build the final release packages a couple of days before the actual release.
Creating the actual release is normally split into 2 days. On the first day I check with the JSST which Security fixes are ready and can be merged. This happens on my laptop only. I tag the release version and build the packages. Further I prepare several Joomla properties for the release day. That means I create the download entries on downloads.joomla.org creating the release article on joomla.org and share it with the marketing team for translations into other languages. Also the final packages get distributed to the Release Team again for the final tests especially for the security patches. Also a small group of other teams get the final files a bit earlier. On the release day, normally a Tuesday, I notify the Release Team for the upcoming release and do the final cleanups on the websites. The file upload to 3 different locations (Amazon S3, update.joomla.org, github) starts the actual release. And set the prepared Security articles and Joomla.org article public, I also push the combined security/release commit to github. This is basically the last task of the release itself. After this I merge some prepared commit for docker and update the nightly build and api documentation. I may also update Wikipedia depending on the overall situation. This is the release if everything works perfect but it’s not over, now it’s time to watch the forum and issue tracker. As long as nothing really bad is reported my job is done for the day. If not I have a long bad day creating another release.
In your opinion, what does it take to be a good release leader?
I don’t know what a good release lead needs to know. I don’t know if I’m a good or a bad one. Most of the time I only get feedback when something bad happens. If everything is fine I can read some success stories on the forum. In the case of Joomla! I would say you need a hard shell and must not take it all to heart (which is really hard for me).
How did you get involved as a Joomla Volunteer?
After using Joomla since Mambo and some loose chat sessions on the IRC channel, I visited the 2nd JoomlaDay Austria in 2016. After this I started to be a bit active by visiting JoomlaDay Germany and joined the glip chat. So actually I’m not so long in this community.
You’ve been around in this community for several years now. What are the moments you will remember forever?
I think it was talking to a guy with a black basecap on my first JoomlaDay Austria. And the Joomla World Conference in Rome. There are some situations but in fact it’s the people I met and like to talk to and become friends with.
If somebody would like to help out on a technical level, what would you advise them?
Just start, it’s not too hard to understand Joomla. It’s following common patterns and tries to make it easy for developers to understand.
How come you’re in so many teams? And how do you manage that?
Unable to say no? I mean I’m Release Lead this automatically adds you to 3 teams. Beside that I was Production Department Coordinator which adds you to another 100 team channels… but basically if you don’t say no you will be sucked into every team in Joomla.
What is the biggest misunderstanding people have when it comes down to new releases?
People think everything is self-evident but in fact it’s really hard to find people helping you to create a solid and stable release. Sometimes it’s really hard to find testers for the release candidates. Lucky me Philip Walton did a great job building up the Release Team. I never got any feedback from extension developers that really should test their own extensions on the upcoming release.
What can we as Joomlers do to make the work of a release leader easier?
Testing the release candidates and pull requests on release sites.
And what makes it harder?
Complaining about postponed features that maybe not fit into a release.
What is needed for Joomla to be ready for the future?
Volunteers, it’s really simple we need more people in all areas with every skill that exists. Of course I would say we need developers but I think much more important is a working marketing team and fundraising people.
I would like to say thank you to you Luca because you are doing an amazing job in your role as Secretary. I think without you Joomla would look completely different and not in a good way. I have 2 other people I want to thank for their time they invest in Joomla and in me. First one is George Wilson who is a really smart guy and tries really hard to get J4 out and of course the guy with the black basecap Benjamin Trenkle without him I wouldn’t be able to manage all my engagement in Joomla!
Joomla RC releases really do catch us 3PDs by surprise. There's no indication an RC will be published until we see a Joomla tweet late on a Thursday evening. Even if we sacrifice our weekend this leaves us with 4 days to test all of our software on it and report issues. That's the best case scenario because given the surprise nature of the RC we might be in the middle of a release cycle and don't have the luxury to drop everything and test Joomla.
Even if we do test the Joomla RC, what's the point? It doesn't include any security fixes which, in our experience, are the primary reason that would break things in a patch level release. Even if that wasn't the case our experience is that filing an issue with Joomla is an ordeal. We will most definitely be told we are doing it wrong. If we somehow manage to persevere and by some miracle the release manager sees the issue the Joomla release will still go through. Therefore we have wasted our time and we still have to work around any issues you introduced. We will still be told that we should have tested with the RC — oh, the irony.
What's better from a business perspective is do nothing. Come release Tuesday things will break. We will learn about that within hours and find out why they are broken. We will wait a day or two to see if Joomla makes a new release to fix what's broken. If not, we release a new version of our software with a workaround to the latest core breakage. No wasted time, less work, same result as before.
Little wonder 3PDs don't bother testing the RCs. With the current timeframe and implementation testing the RCs is a frustrating exercise in futility.