Perl 5 core development was in a downward spiral for a couple of years. Releases happened less and less frequently, the number of contributors dropped and the general outlook was getting worse and worse. This is the story how the downward trend has been reversed and how fun, optimism and even excitement has returned to Perl 5 language development.
Loss of momentum
Major Perl releases used to happen every 2 years: 5.005 in 1998, 5.6 in 2000, 5.8 in 2002. Then it took 5 years to get to 5.10 in 2007 (5.7 and 5.9 were internal development tracks). Sure, bug fixes and changes that didn’t break binary compatibility were published as 5.8.x maintenance releases. But any other enhancements to the codebase were effectively unavailable to the public user. This discouraged external contributors to spend much time on improving Perl.
Detraction by Perl 6
Development of Perl 6 was announced in Summer 2000. Attempts to change Perl 5 in any major way were forthwith redirected to the Perl 6 project, as Perl 5 was now supposed to go into maintenance mode. Of course nobody knew at that time that even 10 years later Perl 6 would be in no position to replace Perl 5 for the vast majority of its users.
Vanishing contributors
So some people stopped contributing to Perl altogether, some decided to work on Perl 6 instead, and some of the developers most knowledgeable about Perl 5 internals spent a lot of time reviewing bug fixes and changes and integrating them into older maintenance branches. Major releases become harder and harder the more time passes between them because the amount of deferred work grows: changelogs need to be updated, local changes to external modules must be pushed back upstream, and updated modules from CPAN need to be integrated back into the core. The future of another major public Perl 5 release started to look quite bleak indeed.
Divide and Conquer
In summer 2009 Jesse Vincent (Founder and President of Best Practical Solutions) decided to do something about this sorry state of affairs. He set up the changelogger web application to let people collaborate on sifting through old check-in messages to create an annotated list of all important changes since the release of Perl 5.10. He proceeded to release a Perl 5.11.0 development snapshot and announced that from then on a development release would be made each month by a different volunteer. Each monthly iteration would lead to more refinements to the newly created release manager’s guide, making it easier to delegate this task in the future to interested volunteers.
Halloween and The Great Pumpkin
After successfully completing the first two development releases Jesse sent his Halloween message, formally taking on the role of the Perl development manager (aka the Perl Pumpking) and announced his plan to get to a public Perl 5.12 release.
More crowdsourcing
Paul Fenwick and the Melbourne Perl Mongers jumped in to help triage some of the accumulated bug reports, categorizing them as either “showstoppers“, “need expert assessment”, “non-critical”, or “can be closed”, leaving the core developers to work on fixing bugs or reviewing the “need expert assessment” cases. In order to eventually have a public release the main branch of the Perl source repository was closed to all changes unless they addressed a showstopper bug. A public release could then be done once all showstopper bugs were closed. Meanwhile the monthly developer releases continued, keeping the perldelta documents up-to-date and external modules in sync.
Source code reorganization
The source code layout in the Perl repository was modified to simplify maintenance of these dual-lifed modules that exist both on CPAN and in the Perl 5 core. Previously the source code and the regression tests had been mixed in with the rest of the core, making it hard to automatically integrate changes between the Perl 5 source tree and the CPAN version. Now dual-lifed modules are distributed between the cpan/ and dist/ directory trees. The former contains all modules where CPAN is considered the upstream source whereas the later contains the modules that are primarily maintained as part of the Perl 5 distribution. This division makes it a lot simpler to spot when a bug fix is mistakenly being applied to the cpan/ tree instead of being reported to the upstream source.
Time based releases
To keep the regained momentum, Perl will also switch from feature-driven releases to time-based ones. A Perl 5.12.1 maintenance release containing only bug fixes is planned for about 1 month after 5.12, and then the 5.13 development track will be active for another 11 month, with a public Perl 5.14 release planned approximately one year after the 5.12 one.
Good Fortune
Other good news for Perl 5 development happened earlier this year when David Mitchell’s grant proposal was accepted by The Perl Foundation. For a long time Dave has worked on hard and gnarly Perl internals issues in his free time. He will now be paid for the next 6 month to work half-time on “hard bugs” in the Perl 5 codebase. These are the bugs that never got fully diagnosed and fixed because they require more knowledge and time commitment than most volunteers can offer and that have therefore been open bugs for a long period of time.
Bright Future
Most of the interesting things in the Perl world have been outside the core development in the last few years: the creation of the Moose object system, which is quickly becoming the de facto standard, the development of the Catalyst web framework, the DBIx::Class Object/Relational Mapper, or the new Perl Web Server infrastructure provided by Plack, to give just a few examples. But with all the focused activities directed at releasing Perl 5.12 people have returned to core Perl development again. A direct indicator of the growing enthusiasm is the number of requests to get the release out of the door and re-open the repository so that people can submit new changes for the next version of Perl. Fortunately, Perl 5.14 should be released in just about another year or so. Share your comments on how you use Perl below and enjoy!
Title photo courtesy of Alice Donovan Rouse on Unsplash.