Extension:ZeroPortal/Conventions

This document lays out the strategies, conventions, and best practices currently followed by the Wikipedia Zero engineering team. The intent of this document is threefold: 1) inform team members outside of engineering of our development process, 2) share workflow strategies within the engineering team (so that an extended absence slows, but does not halt development), and 3) provide a resource for rapidly assimilating new hires. This document should evolve over time as new tools are utilized and productive workflows are ironed out.

Strategies
This section contains the development strategies implemented for both the Partners Portal and the End User Experience. The Wikipedia Zero engineering team uses its unique deviation from the traditional mobile experience to employ the latest technologies within Mediawiki in an atmosphere of limited scope. This decision helps to find some of the kinks that can occur during assimilation of new projects to long-standing ones. An example of this is the integration of OO-UI.js, a javascript library still in development, into Mobile Frontend.

Partners Portal
Unlike most projects within WMF engineering, development of the portal utilizes a unique combination of a Mediawiki extension, Lua modules, and on-wiki development. The Mediawiki extension (ZeroPortal) contains the browser tests, LESS files, and back-end logic used by the portal. This logic passes any url parameters to the Lua module "Portal" which then acts as a state machine and router to determine the current page. Finally, one or more wiki pages or templates is rendered and the final page is displayed to the user. This strategy of development enables rigorous testing, utilization of the Mediawiki infrastructure, easy beta testing, and a clean Single Page Application experience for the user, all with the ability of instant deployment thanks to on-wiki editing.

End User Experience
The ZeroBanner Mediawiki extension contains all of the changes to the Wikipedia mobile experience of zero-rated users. These changes include the following: As part of the Wikipedia Zero team's dedication to employing the latest innovations of the Mediawiki infrastructure, the information and warning screens both utilize OO-UI to display temporary overlays to the user.
 * A banner alerting the user that Wikipedia is being offered free of charge by their mobile provider
 * A screen providing more information about the Wikipedia Zero project
 * A warning notice upon leaving Wikipedia that the user is leaving zero-rated coverage

Portal operation details and guidelines
This section covers some more technical details on the operation of the Zero Portal. This section covers some more technical details on the operation of the Zero PortalThe full-stack operation of the portal can seem a bit complicated or daunting at first glance. It is, however, quite the opposite and is important to understand during development discussions of the portal. Consider the following descriptions to also serve as a set of rigorous guidelines when adding features to the portal or modifying its pages.

PHP (ZeroPortal.php)

 * Located within the ZeroPortal Mediawiki extension
 * Sets up Special:ZeroPortal behavior to allow running of Module:Portal (Lua)
 * Passes user information to Module:Portal, including which partner IDs are viewable
 * Additionally passes certain URL parameters to Module:Portal (those with the prefix "zp")
 * Includes LESS files from extension for reseting default Mediawiki styling

Wiki Special Page (Special:ZeroPortal)

 * Located on-wiki
 * Direct URL endpoint for all end users
 * Runs Module:Portal (Lua) to render wiki templates to the page
 * Styling of the page is controlled by a combination of extension LESS files and Mediawiki:Common.css

Lua (Module:Portal)

 * Located on-wiki
 * Includes any additional Lua modules that may be required
 * Acts as a router/state machine
 * Determines which "page" the user is requesting -- more correctly, which templates to render on the special page -- based on given URL parameters
 * Authorizes or rejects rendering the page based on the user's given "state" (admin, applicant, pending partner, launched partner, etc.)
 * Contains logic for returning raw graph data to the Graph extension
 * Planned actions:
 * Acting as an endpoint for AJAX requests to read from/write to partner configurations

Javascript (not yet developed)
The above guidelines are still in a state of constant flux as the framework for the portal is being discussed and developed. Frequent changes are to be expected.
 * Location: TBD (probably on-wiki)
 * Controls animations and validation
 * Performs AJAX requests for reading from and writing to partner configuration files
 * Additional actions that are planned or under consideration:
 * Dynamically generating the news page feed
 * Overriding all links and converting the entire portal into a Single Page Application
 * Several CRM/admin panel actions

Editing
All partner configuration will be stored as JSON blobs in Zero: namespace. Whenever javascript needs to update them, it has to send a post request to the ZeroPortal special page. Since only Lua code knows the legal transitions from one state to another, only Lua can decide if transition is acceptable. Thus, a lua module function, e.g. Module:Portal.edit will return either 1 or 0.

Edit commands can be either set or remove, and will specify the path to the JSON's location, e.g. "banners/fr" or "configs/0/proxies/1". The new value will always be passed as a JSON blob.


 * set command will replace old value at the given path with the new one, or add it if the path does not exist.
 * Remove command will remove given value at the given path

Lua will be able to access both the old configuration - getExistingConfig, and the new, pending configuration - getModifiedConfig. Both of these functions return the entire config blob. getModifiedPath will return an array where each element will be either a string (dictionary key), or an integer (array position).

In order to prevent multiple edits and ensure security, a token will always be needed to modify content. The token will include the hash of the current config value (e.g. SHA1 of the whole config JSON blob), plus the edit token. The token will be available to Lua via an extension function (portal only).

Conventions and best practices
What follows is a comprehensive list of development conventions and best practices sorted by language. As always, fall back to the general coding conventions for Mediawiki as a default. Try to document as much as possible, including labeling any implied globals as such explicitly.

PHP

 * No deviations from standard Mediawiki coding conventions (Yurik to confirm)

Cucumber (Gherkin)

 * Test-driven development is encouraged
 * Write all features from high-level perspective using natural language
 * Be specific when writing step definitions and mimic wording of similar definitions whenever possible
 * Ignore DRY instincts at times where explicitness is more natural/useful
 * When testing for content presence (text, media, etc.), search for container elements over actual content
 * Step definition and page support files always have the prefix "_steps" and "_page", respectively. Feature files have no prefix.
 * Always include a description below feature definitions describing expectations from a user story perspective

Lua

 * Separate significant features as individual modules
 * When creating a new page for the portal, create one or more templates and list them in the betaTemplates dictionary. Once the page is ready for deployment, simply move its template(s) to the productionTemplates dictionary.
 * Use the same process for the Lua modules - the dictionary will contain the list of current and beta pages for each module, and a new module may be deployed once reviewed by other Zero developers by simply changing the version (page name) in the dictionary.
 * Be verbose with variable and function names
 *  DO NOT  reflect user input without input sanitization and output sanitization / escaping or a guarantee of as much from the exported PHP functions. Avoid Cross-Site Scripting (XSS).

HTML and WikiText (portal pages)

 * All portal pages are composed of a series of templates
 * Partial templates -- those that do not describe a full page and/or can be reused across the application -- reside in the Template namespace (e.g. Template:PortalHeader)
 * Each portal page has its own template that resides in the main namespace and can include partial templates (e.g. Home)
 * Once a template has been moved to production it should be considered read-only and never altered again (except for emergency scenarios). Instead, copy the template and attach "-1" to the end (e.g. Template:PortalHeader-1), or increment the number if one already exists (e.g. Template:PortalHeader-2).
 * Templates are written in pure HTML except when WikiText markup or parser functions are required
 * Only assign IDs to elements that are identified by CSS or Javascript
 * Only include images via the tag when referenced by the page content
 * It is best practice to write the full markup for a page and test it before including any scripts or styles
 * Element IDs and classnames should reference the major content element to which they belong via prefix. This does not necessarily imply any sort of hierarchy. For example, if a given content element with an ID of "feature" has a parent wrapper element, the parent's ID would be "feature-container" (unless it had some other major identifying characteristic). Likewise, if element "feature" has any child elements, they would be named by their identifying characteristic in relation to feature, e.g., "feature-text" and "feature-image." See the "stat" objects in the [//zero.wikimedia.org/wiki/Home Home portal template] for a more structured example of this.
 * Do not use tags or inline styles unless absolutely necessary
 * Do not use tags or inline Javascript unless absolutely necessary (to be reviewed once Javascript is implemented)

LESS/CSS

 * All styles for the portal are to be specified within "body.page-Special_ZeroPortal" so as not to accidentally interfere with other pages
 * Use LESS variables for all colors (besides black and white) and sizes of notable elements (i.e. those that affect the sizes of other elements)
 * Use em units whenever possible (unless a property should not change with font size, e.g., border widths)
 * Use mixins for properties that still require vendor prefixes
 * Give preference to pure HTML+CSS solutions over Javascript/jQuery or image editing
 * For portal development, it is best practice to develop in LESS and then compile to CSS and copy the output to Mediawiki:Common.css for rapid testing. Once a page is complete, a patch should be pushed to Gerrit including a LESS file for the page. Upon deployment, the CSS should be removed from Mediawiki:Common.css.

Javascript

 * Use OO-UI whenever possible (ZeroBanner extension contains multiple examples)
 * Follow Mobile Web conventions with the notable exception that URL routing is done via parameters instead of hashes
 * ... More to come