Developer Wishlist/2017/Documentation

Problem
There are some widely used information templates on mediawiki.org which display some software information that could be easily extracted from the source code, but those templates are currently created and maintained manually. This is inefficient and ineffective - it's a significant amount of manual work and often not done, or not updated when code changes. Example: extension, extension install, hook and config variable templates.

Who would benefit

 * Developers who don't need to spend time on creating or updating hook/config var pages whenever they
 * Extension maintainers (especially people who routinely change others' extensions) who don't need to track things like required MW version in two different places
 * Developers and site admins who will gain access to correct and complete documentation
 * Possibly site admins running older MW versions because this would allow a much more user-friendly presentation of MW compatibility information

Proposed solution

 * 1) find a place to store structured mw.org infobox data - there are several possible approaches, see T155024: Store structured needed for MediaWiki documentation for discussion
 * 2) write a bot to extract the data from git (,  ,   phpdoc) and upload it
 * 3) fetch the data in the infoboxes
 * 4) for hook/configvar pages, make the bot create them automatically with stub content when they don't exist
 * 5) run a bot with some regular frequency (e.g. weekly) to update the pages accordingly

Problem
There seems to be an undocumented and inconsistent hierarchy of rights. For example, if I cannot view a page, I cannot edit it. But: if I can't edit, can I still move? In particular, the API has no concept of a hierarchy of rights. This leads to potential security issues in read protected MediaWikis.

Who would benefit
Extension developers and MediaWiki maintainers will have a more clear cut security model.

Proposed solution
We should add a minimum hierarchy in our rights, such as read > edit > other actions, similar to the way we have a hierarchy in the user groups: * > user > other groups. If one cannot read, they cannot do anything else. If one cannot edit, they cannot do any other modifications. I know this is too simplistic, so we need to sketch out a proper hierarchy. The hierarchy should also be documented.

Problem
Wikimedia Security Team/Security reviews lists three separate processes which are either recommended or required in order to get an extension deployed on Wikimedia servers.

For non-WMF developers it can be unclear what is needed for each review step and how the Security team expects the information to be presented.

Who would benefit

 * Anyone unfamiliar with the current review practices, likely meaning extension developers outside of the WMF.
 * The security team: Clearer expectations/instructions should hopefully help streamline new external review requests.

Proposed solution
Identify one or a few good real-life examples for each process (or at least the latter two) to promote as case studies.

Problem
As an extension developer, I often ran into issues where my code would not be executed / updated due to some cache hit in earlier parts of the code. Documentation is spread out all over MediaWiki, and incomplete.

Who would benefit
Extension developers who write code that dynamically changes page content and need to know when and how to invalidate the caches.

Proposed solution
A good starting point is https://www.mediawiki.org/wiki/Manual:Caching. We need to complete the information missing for some cache types. A rough description of which caches are used when in the execution order would be extremely helpful (maybe in relation to some central hooks). Also, some information of how these caches can be invalidated within the code and in general should be added.

Problem
The process for updating users about technical changes is far more complex than it should be. If I have a technical newsletter (Tech News, specific newsletters for teams or products) and want to reach out in several languages, this is the process to send it out:


 * Generate the text on Meta using a Lua module that combines all languages and adds a switch. See example.
 * Realize you had too many languages, the module can't handle more than 20.
 * Generate the first 20.
 * Generate the next batch.
 * Carefully insert the new languages.
 * Find the duplication of the source text (typically English) and remove one of them.
 * Test it.
 * Find a mistake.
 * Fix it on Meta.
 * Redo the entire process.
 * Finally send it out.

Who would benefit
Anyone keeping users updated about technical changes: upcoming, things they can give feedback on, or new changes.

Proposed solution
At least make sure we can send out messages to more than 20 languages without inviting mistakes by manual insertion prior to MassMessaging all the wikis.

Module:Assemble multilingual message

Review and update the Examples extension
Other than ContentAction, which was touched during GCI, the code in the Examples extension hasn't really been touched for a few years. It would be nice to give it a spring clean, updating code to follow modern coding practices for MediaWiki extensions.

Problem
Extensions are (supposed to be) documented on a page on MediaWiki. This documentation includes Template:Extension to provide a summary of important information about the extension, including the "required version of MediaWiki".

This version field is inadequate: does specifying "1.19+" mean that the extension's master branch is intended to be compatible with MediaWiki 1.19, or that the extension has release branches going back to REL1_19 while the extension's master branch is only intended to be compatible with MediaWiki core's master branch, or perhaps some middle ground?

Some extensions have solved this by including detailed text in the existing "version" parameter, such as "1.26+ . Flow master is only supported with core's master; otherwise, use matching branches (e.g. Flow REL1_26 with core REL1_26, or matching WMF release branches).", but this is currently done inconsistently and infrequently.

Who would benefit
Developers making updates to extensions for a core change in compliance with the new deprecation policy will no longer have to guess at whether backwards compatibility with older versions of MediaWiki core should be maintained.

Developers of extensions that don't maintain compatibility with old versions of MediaWiki won't have unnecessary and sometimes complex backwards-compatibility code added.

Developers of extensions that do maintain compatibility with old versions of MediaWiki won't have to -1 or -2 as many changes that break backwards compatibility.

Proposed solution

 * 1) Update Template:Extension with parameters to indicate the MediaWiki versions for which release branches exist versus the MediaWiki versions the extension's master branch maintains compatibility with.
 * 2) Update existing extension pages to use these parameters, which may involve determining the master-compatibility policy where it's not already specified.

Create a developer documentation special interest group
Problem Documentation is out of date, incomplete (T2001: Documentation is out of date, incomplete (tracking))

Wikimedia produces a large number of FLOSS software products targeted at various audiences. The audiences need documentation in the form of tutorials, API references, platform overviews, etc to effectively use and extend the products. MediaWiki itself is a highly extensible platform for creating solutions for collaborative, open sharing of information. Sadly, this platform itself is under-documented which slows down the rate at which developers can produce innovative solutions based on it. "Read the source code" is not an acceptable way to gain an initial understanding of a product offering, but many software developers lack the time, motivation, and skills needed to effectively and completely document the solutions they produce. Improved technical documentation of the internal and external interfaces that can be used to extend or programmatically consume MediaWiki hosted content should make use and development easier for existing and future technical contributors.

Solution Create special interest group in charge of:
 * Triaging/grooming the #documentation workboard.
 * Identifying top priorities based on impact and interest.
 * Organizing activities (including lobbying to the Wikimedia Foundation) in order to complete the top priorities.
 * Attracting people interested in working on documentation.
 * Connecting documentation volunteers with people and projects who can help get them started.
 * Advocating documentation best practices for Wikimedia products.
 * Developing resources to make contributing technical documentation easier.

See also
 * Community with outreach potential: Technical Writers

Problem
Extension developers and even core MediaWiki developers want to, or should want to, use a standardized design. Even if we don't like the standard itself, by using a standard we can then inherit improvements to the standard with little to no additional work.

OOjs UI is the standard. However, it is difficult to work with, owing to poor documentation. This discourages its use and adoption as a standard.

Compare this to other frameworks such as Semantic UI that make it very easy to use its design. You can use its JavaScript widgets for specialized features, or just use the CSS to get the look and feel if you are dealing with static HTML.

Who would benefit
People developing new extensions and those working on improvements to MediaWiki core and existing extensions. The Wikimedia Foundation would benefit from less time wasted on figuring out this system.

Proposed solution
Better documentation that makes it easier for developers to integrate OOjs UI.

Organize a MediaWiki Documentation Day (similar to the Gerrit Cleanup Day)
It would be awesome if we devoted 1 entire day to doing nothing but documenting MediaWiki. There are countless classes and functions in MediaWiki core and its extensions that have no documentation. There is also a serious lack of high level documentation for developers. The problem is that documentation is often an after-thought and rarely gets focused attention from developers.

Some suggested projects that could be part of MediaWiki Documentation Day:
 * Tackle some of the blocking tasks at T2001: Documentation is out of date, incomplete (tracking) or #tracking.
 * Add in-code function descriptions for important functions that don't have them.
 * Add in-code class descriptions to classes that don't have them.
 * Create documentation pages on MediaWiki.org for important classes in core that don't have them. See Manual:User.php for an example of a page that does exist. High-level documentation, like Manual:Title.php#Title_structure is especially useful.
 * Clean up our outdated documentation on MediaWiki.org.
 * Create README files for all the extensions that don't have them and make sure that all configuration variables are documented there.
 * Create some high-level documentation on how to write new API modules.

See Also:
 * T85592: Sort out &#91;&#91;:Category:Tutorials&#93;&#93;
 * T101659: Run a documentation sprint for Labs
 * T117526: Improve the generated PHPdoc (by considering alternatives to Doxygen?)
 * T149372: MediaWiki Documentation

Relocate CI generated docs and coverage reports
Part of / blocker of T133300: Target architecture without gallium.wikimedia.org

Another blocker, and potentially a prerequisite, is publishing the generated documentation. From the drawing and https://www.mediawiki.org/wiki/Continuous_integration/Documentation_generation

CI Target 2016 - Relocation of Doc / coverage reports (private) gives an overview of the documentation flow.

{Image size=full}

As summarized by thcipriani :

Current:
 * Doc and coverage reports are generated on Nodepool instances (they have lot of build dependencies)
 * Building instance rsync to a labs instance
 * A Jenkins job  is executing on gallium to rsync the doc from the publisher instance.

We would need an instance to host the material with PHP5 (some docs need that). Most probably out of the labs support host / on an isolated network. We would need some intermediary system to have the Nodepool building instances to push to.
 * Doc and coverage reports are still generated on Nodepool instances
 * Building instance push to a publisher system
 * might reuse
 * Jenkins (or another system) runs a task that fetch from the publisher system to doc.wikimedia.org document root.

We need to find a target host on which to migrate documentation to. It would have Apache / PHP5 code and run code as generated by the various code repository that have doc/coverage enabled.

Create an authoritative and well promoted catalog of Wikimedia tools
Suggested by Qgil at https://lists.wikimedia.org/pipermail/wikitech-l/2015-October/083580.html

Brief list of catalogs that have been suggested thus far:
 * https://tools.wmflabs.org/
 * https://tools.wmflabs.org/hay/directory/#/
 * https://outreach.wikimedia.org/wiki/GLAM/Resources/Tools
 * https://de.wikipedia.org/wiki/Benutzer:Atlasowa/edit_history_visualization
 * http://seealso.org/
 * https://www.wikidata.org/wiki/Q6584911 ("Wikipedia:Tools", particularly the large lists at:
 * https://de.wikipedia.org/wiki/Wikipedia:Helferlein
 * https://en.wikipedia.org/wiki/Wikipedia:Tools

Somewhat related:
 * https://www.mediawiki.org/wiki/Manual:Extensions
 * https://www.mediawiki.org/wiki/Extension:Gadgets/Roadmap#Gadgets_3.0

- See also: T1238: Central Code Repository for code used on wikis (Templates, Lua modules, Gadgets)

Document common low level site requests (namespace + logo changes, IP throttling expectations) for potential technical contributors and advertise those docs in tasks
A good bunch of #Wikimedia-Site-Requests items are pretty common / repetitive: Namespace translation changes, namespace additions, site logo changes, IP throttling exceptions for editathons etc.

Let's make sure they are documented for the patch writers (maybe they already are?) so aklapper can link to such docs everytime aklapper sees a request via his Phabricator stock answers. We don't want to burn out our existing contributors in this area like Dereckson etc. and we want to grow our technical contributor base.

Run a documentation sprint for Labs
Currently documentation is way too out of date and inconsistent. Setup a weeklong sprint with specific tasks to fix documentation, and run the weeklong sprint, with help from Admins and volunteers.

Should involve both general labs and tool labs.

See also:
 * T126500: Organize a MediaWiki Documentation Day (similar to the Gerrit Cleanup Day)

Define the architecture areas for MediaWiki core and platform extensions
We really miss a high level architecture overview of MediaWiki core and platform extensions (the ones that provide APIs and enable user features). Reasons to have one:


 * MediaWiki and relatives form a very complex family. https://www.mediawiki.org/wiki/Developers/Maintainers lists 210 components only for core, and there is no subdivision to be seen. Without a shared high level map it is a lot more complex to have shared high level discussions and plans.
 * New potential contributors need an overview to understand the basics of MediaWiki and find the area where they want to work on. Such diagram would be really helpful to identify an area as a starting point. After many conversations with many newcomers with different skill levels, we know that most of them get simply lost / overwhelmed in their first contact.
 * Defining areas is a premise required to discuss whether we want to have architects / owners / maintainers for a specific area that would work with the #ArchCom or be part of it.

How a v0.1 could look like:

MediaWiki Core
 * Configuration
 * Database
 * i18n/L10n
 * Parser
 * Skins
 * Media
 * API

Platform extensions
 * Parsoid