Requests for comment/Release notes automation

Most of the developers working MediaWiki have encountered a merge conflict due to  at some point. As a result it has become common to no longer add release notes. This RFC seeks to find a solution to this problem.

Proposal 1: Declare in commit message
Using either a keyword or a footer, we'd declare them in the commit message. An automated process (commit bot or pre-merge hook in Jenkins perhaps) will then aggregate these periodically and write them to the  file.


 * Disadvantages:
 * Depending on whether we go with the pre-merge hook or commit bot, this would add junk to the commit message in the repository.

Keyword format
foo: The subject line of the commit

Commit message body.

RELEASE New features that spans multiple lines.
 * Bullet point to add to the release notes

Blank lines and lines not part of a bullet point are ignored.

RELEASE Bug fixes
 * (bug 123) Another item to go into a different section.
 * (bug 345) Another item to go into a different section.

Change-Id: I15915e04bfed744931e05b6493c14b82cd0a41a9

RELEASE === Bug fixes === RELEASE Bug fixes
 * All of that text will be automagically added to the RELEASE-NOTES sections once the bot detects that a branch has been merged with the master
 * It would be ideal if the bot checks if such header actually exists in the file and emails user in case of an error (on branch commit, not on merge)
 * It would be great if Jenkins detected those errors
 * Bot should understand any of these forms of headers:

Footer format
foo: The subject line of the commit

Commit message body.

Lines not directly connected to the footer are ignored. They will be line-wrapped later.

Release-New-Features: Bullet point to add to the release notes that spans multiple lines. Release-Bug-Fixes: (bug 123) Another item to go into a different section. Release-Bug-Fixes: (bug 345) Another item to go into a different section. Change-Id: I15915e04bfed744931e05b6493c14b82cd0a41a9

Commit bot implementation

 * Bot runs periodically
 * Looks through all commits to master since the last run
 * Extracts the bullet points associated with their respective sections to go in
 * Update repository
 * Edit file
 * Commit, push to gerrit, auto-merge (like i18n-bot)

Jenkins hook implementation

 * Based on the jenkins-bot in OpenStack's Gerrit.


 * In the submit pipeline
 * Look through the commit message
 * Edit release notes file
 * Submit amended patch set with the updated release notes file and the lines removed from the commit message
 * Submit patchset


 * Advantage:
 * Leaves commit message clean
 * Keeps modification of release notes in the same commit
 * Less pollution in the commit history

Preamble
In the previous discussions on the "release notes conflict" on the wikitech-l mailing list, it was said that
 * 1) We can't use the commit message subject because they aren't always the same as what you want to write in the release notes.
 * 2) Not all commits should become an entry in the release notes.

On the first point: Though this is not true in most cases, it is true in some scenarios. A typical case for such scenario is a commit that fixes a bug by performing a change that would not be adequately summarised by merely mentioning the problem it fixed. For example commit "mw.hook: Use jQuery.Callbacks instead of mw.legacy.events" fixing "(bug 12345) mw.hook.bind is broken if mw.hook.once was called with the same function". In this case what you want is two entries:
 * One for the backend change.
 * One for the bug fix.

That's exactly what the below described workflow will do. I'll add an entry for change the commit describes in its subject. And an entry under "Bug fixes" with the bug id from the commit message (fetching the bug title from Bugzilla's API).

On the second point: Absolutely correct. However that doesn't mean we should manually do it just because some commits are not noteworthy.

Workflow
My proposed workflow is as follows:

Commit messages should be in a consistent style and form (like most contributors do already). Though everyone is welcome to improve the commit message at any time, it is the final duty of the project owners who merge changes to make sure the commit message is good prior to approval.

We periodically run a script that appends the list of commit summaries (since the previous run) to a designated wiki page. This script can use similar algorithms we already use for the wmf-branch release notes. The wiki page could be sectioned like "New", "Removed", "Changed", "Bugs fixed" etc. (like we already do for release notes). The script could default to one of the sections or we could have a generic "Unprocessed" section that acts like a queue.

Once on this page it will be a lot easier to reason about and maintain. Changes like the following:


 * Omit entries not noteworthy (e.g. trivial changes)
 * Remove obsolete entries (e.g. changes that were reverted in the same release cycle)
 * Enforce consist style and formatting
 * Mark entries with the relevant labels ("New", "Removed", "Breaking" etc.)

We'll maintain this as a community with the release manager having the final say about it. Part of the release process for major releases will be taking the wiki page contents and prepending them to the  file and creating (since it'll no longer exist during the cycle) the   file.

For backporting to release branches, this workflow makes the process slightly easier. A commit will not contain release notes. This makes the change more atomic and easier to backport to other branches. Since the "wiki page appender" bot will also monitor release branches, the wiki page will still be updated accordingly.

When a minor release is made the regular process is simply repeated: Drop-in replace the section in the  file and replace contents of the   file. This will also incidentally include unrelated improvements made to the release notes, which is expected and intended. Traditionally those changes would be separate commits in master that are then backported as well.