User:Salvatore Ingala/Notes

= Intro = Scope of the project is to make gadgets customizable, that is:
 * Allow gadget writers to easily export their gadget's customization variables.
 * Provide the users with a nice UI for customizing those variables.

This page contains a newer design than the one I proposed in my application, since the scope of the project has slightly changed.

Places
I'm working on this branch: http://svn.wikimedia.org/viewvc/mediawiki/branches/salvatoreingala/Gadgets/

From email 23 Aug: [M]y plan [is] to merge it to trunk only after the Gadget rewrite was complete (or almost)....

I expect integration to be easy, since most of the backend code is in a static class with just a few public methods; so the code to be rewritten just has to manage saving/loading and some other goodies which should be pretty straightforward. Previous brainstorming was done on EtherPad. This is the last useful revision.

Roan Kattouw is working on merging this code as part of his ResourceLoader 2 work, probably in February. From Roan's email 14 Dec 2011: "There is lots of stuff to do in RL2 and integrating Salvatore's work is one of the tasks.". See MediaWiki's Roadmap for a more up to date schedule.

= Features definition = These are the needed features:
 * 1) Ability to specify what preferences do we have for a gadget;
 * 2) *Will do this in JSON format.
 * 3) *Where? For now, I'm using a fixed page MediaWiki:Gadget-mygadget.preferences or something similar. Likely to change (with low impact)
 * 4) *STATUS: implemented (with a limited number of preference type, documented below; more will be added); server-side validation done; client site validation done with jquery.validate (link).
 * 5) Hooking up to Special:Preferences to add per-gadget prefs
 * 6) *Currently done like this: every enabled gadget shows a "configure" link near his checkbox in Special:Preferences. Clicking on the link pops up a modal dialog with the configuration interface. This may need partial rewrite after Gadgets rewrite in RL2.
 * 7) *STATUS: working UI mock-up
 * 8) Store preferences somewhere
 * 9) *Using the existing user_properties table. Gadget preferences are retrieved and then hidden in UserLoadOptions hook, and reinserted in UserSaveOptions.
 * 10) *Unused preferences need to be deleted (lazily is good enough). Saving policy to maintain coherency:
 * 11) **When saving the configuration of a gadget, throw away any previous configuration for that gadget (so to clean up old values for no-longer-existing settings).
 * 12) **When reading the configuration of a gadget, check it against its specification and assume default for missing values or values that fail validation (if specifications changed).
 * 13) *STATUS: implemented
 * 14) Provide the preferences to gadgets. Concepts:
 * 15) *mw.gadgets.options.get( 'gadgetname' )?
 * 16) *mw.loader.options.get( 'modulename' )?
 * 17) *binding the configuration to this in gadget's context. (may have issues with RL2)
 * 18) *STATUS: implemented; current solution is a combination of two ideas: gadget metadata (including configs) retrievable via mw.gadgets.info.get( 'gadgetname' ), but also bound to this in gadget's context.
 * 19) Internationalization of strings used by the gadget
 * 20) *There are two types of "messages" to handle:
 * 21) *#messages shown in the configuration dialog; these don't need to be passed to the gadget's module, and may be interpreted server-side, when building the dialog code during the AJAX request. Should fetch their names form the configuration JSON (maybe adding some common prefix);
 * 22) *#other messages used by the gadget; this is in the scope of RL2.
 * 23) *STATUS: implemented (with some issues to fix)

= Preference description syntax =

The "preference description" is an object in JSON format which describes all relevant info about gadget's settings (and some other settings related to the configuration system); when all the rest of the project will be finished, a tool to create and maintain preference descriptions may be built, so to relieve gadget developers from the annoyance of learning its syntax.

The general format is this:

"global" settings will be settings that apply to the preference dialog (e.g.: min/max width, min/max height, title...). It will be useful for future advanced features, too.

is an array containing field description objects.

The syntax of a field description object is as follows:

is compulsory for any field description. According to, more members may be allowed (or needed).

Raw strings and messages
The ability to specify messages in the "MediaWiki:" namespace is absolutely necessary; nevertheless, sometimes raw strings are needed. Instead of making complex syntaxes to distinguish between raw strings and messages, a preprocessing of strings is done: if the string starts with "@", then it's intended as a message prefixed by "MediaWiki:Gadget-mygadget" (where  is the name of the gadget, case-sensitive); otherwise, it's a raw string. Two "@" at the beginning escape for a single "@". This applies, for example, to s and to options of   fields.

Global settings
Nothing, for now

Field definition specifications
This section describes the exact syntax for each field.

label
Fields of this type don't encode for any gadget preference, and only add an arbitrary text. The syntax is the following

Unary fields
All fields of this section encode for exactly one gadget preference, and support compulsory name</tt>, label</tt> and default</tt> members, that is:

,, and   are compulsory for any option description in this section.

must be a valid JavaScript identifier (that is, it must start with a letter or '_', then contain only letters, numbers or '_'), and must not be more than 40 characters long.

boolean</tt>
This field will be shown as a checkbox. No additional members are allowed.

string</tt>
This will be shown as a single-line text field. There are additional members:


 * required</tt>: (optional boolean, no default value). If true</tt>, a zero-length string will never be accepted; if false</tt>, a zero-length string will always be accepted (even when minlength</tt> is given). If it is not given, none of the above validations is done.
 * minlength</tt> (optional integer, defaults to 0): an integer number specifying the minimum length allowed for this option.
 * maxlength</tt> (optional integer, defaults to a 1024): an integer number specifying the maximum length allowed for this option.


 * TODO: think again for the default value of maxlength</tt>.

number</tt>
This will be shown as a single-line text field. The javascript representation of values of this field is as numeric datatypes (or null). There are additional fields:


 * required</tt>: (optional boolean, defaults to true</tt>). If true</tt>, a valid number must be given. If <tt>false</tt>, an empty string will be accepted (and will be delivered as <tt>null</tt>).
 * <tt>min</tt> (optional number): a number specifying the minimum allowed value.
 * <tt>max</tt> (optional number): a number specifying the maximum allowed value.
 * <tt>integer</tt> (optional boolean, defaults to <tt>false</tt>). If <tt>true</tt>, only integer numbers will be accepted (<tt>min</tt> and <tt>max</tt> must honor this, too). If <tt>false</tt>, decimal number will be allowed, too.

<tt>select</tt>
This will be shown as an HTML <tt>select</tt> element. There is only an additional compulsory field:


 * <tt>options</tt>: (compulsory array): an array with elements of the following shape:

All values should be different (according to the <tt>===</tt> operator), and all names too.

<tt>range</tt>
Shown as a jQuery.UI Slider, even if only its basic functionality is currently supported. There are other fields:


 * <tt>min</tt>: (compulsory number): Minimum allowed number.
 * <tt>max</tt>: (compulsory number): Maximum allowed number.
 * <tt>step</tt>: (optional number, defaults to 1): Gap between allowed numbers.

Negative or decimal numbers are valid values for <tt>min</tt> and <tt>max</tt>; positive decimal numbers are allowed for <tt>step</tt>.

Note that <tt>max</tt> must be coherent with <tt>min</tt> and <tt>step</tt> (that is, the difference between <tt>min</tt> and <tt>max</tt> must be an integer multiple of <tt>step</tt>).

<tt>date</tt>
Shown as a jQuery.UI [http://jqueryui.com/demos/datepicker/ datepicker. There are no other fields.

Dates are delivered as string with the format: <tt>[YYYY]-[MM]-[DD]T[hh]:[mm]:[ss]Z</tt>, which is an UTC time in ISO 8601 standard. It is a format recognized by Javascript's <tt>Date</tt> constructor. Empty dates are delivered as <tt>null</tt>.


 * TODO: add a <tt>required</tt> option.

<tt>color</tt>
Shows a color picker (farbtastic); colors are delivered as hexadecimal strings from "#000000" to "#ffffff" (uppercase letters are not allowed)


 * TODO: add a <tt>required</tt> option; add an option to allow choosing 'transparent' as a color?

<tt>composite</tt>
A <tt>composite</tt> preference is a javascript object, compound of several subfields; better to start with an example:

This example encodes a <tt>composite</tt> field that encodes for a preference with name <tt>position</tt>, whose value is an object with two fields: <tt>position.x</tt> and <tt>position.y</tt>; in this example, both are numbers, each with his own specification, but the types may have been different and there could be an arbitrarily large number of "subfields".

A <tt>composite</tt> field has two compulsory members, <tt>name</tt> (with the same syntax and meaning as in the other fields), and <tt>fields</tt>, which is an array of fields, with the same syntax and semantic of an arbitrary set of preferences; any kind of field is allowed in the <tt>fields</tt> member (yeah, even a <tt>composite</tt>, recursively). It does not have a <tt>default</tt> member, and it deduces the default value as the composition of default values of its subfields.

Visually, all subfield are shown together, but the composite field is not visible to the end user, and its existence may be completely transparent.

<tt>list</tt>
The <tt>list</tt> field type allows the user to create homogeneous ordered sets of things (it implements a generic array). The user can add objects to the set, remove them, or change their order. It has the following members: Example of list field:
 * <tt>name</tt> (compulsory string), the name of the preference encoded by this field.
 * <tt>field</tt> (compulsory object), which is the complete description of an arbitrary unitary field, except that it does not have the <tt>name</tt> field.
 * <tt>default</tt> (compulsory array), which is the default value (an array of items of the type specified by <tt>field</tt>).
 * <tt>required</tt>: (optional boolean, no default value). If <tt>true</tt>, a zero-length array of items will never be accepted; if <tt>false</tt>, a zero-length array of items will always be accepted (even when <tt>minlength</tt> is given). If it is not given, none of the above validations is done.
 * <tt>minlength</tt> (optional integer, defaults to 0): an integer number specifying the minimum number of items allowed for this preference.
 * <tt>maxlength</tt> (optional integer, defaults to a 1024): an integer number specifying the maximum number of items allowed for this preference.

This is an example of a field that allows the choice of an array of colors. The default value is a set of three colors, red, green and blue.

It is possible to create lists of any unitary type; for example, <tt>list</tt> of <tt>composite</tt>s, or even <tt>list</tt> of <tt>list</tt>s, but don't try this at home...


 * TODOs:
 * Add an option to require that all items are different?
 * Add an option to sort items automatically instead of allowing the user to choose the :order?
 * Improve editor support, which is currently limited.

Compound fields
Fields described in this section encode for several gadget preferences.

<tt>bundle</tt>
Bundles allow to subdivide gadget preferences in several named visually distinct sections. Each section has the same format of a full preference description. Bundles are implemented with jQuery.UI tabs, each section in his own tab.

Bundles are especially useful for complex gadgets with lots of preferences.

The syntax is as follows:

Bundles have only one member, named <tt>sections</tt>, which is an array of section descriptions; each section description defines fields of each section, with the same format of full preference descriptions.

Note that whether a gadget preference is encoded inside a bundle section only matters to improve user experience; there is no difference in what will be delivered to gadgets.

Easy to implement

 * multiselects
 * 'url' fields

Ideas for further enhancements
What follows is a collection of unimplemented (and, possibly, not well defined yet) ideas.

Field dependencies
Sometimes one wants to enable a field (or a set of fields) only if some other fields satisfy some contraint. Most typical case is a checkbox which, if checked, enables one or more fields.

Proposed syntax for simple conditions:

The field described by <tt>someField</tt> (in this case a string field) will be enabled only if the value of the field described by <tt>otherField</tt> is <tt>true</tt> (or something that evaluates to <tt>true</tt>, like non-empty strings or non-zero numbers).

A syntax for more advanced conditions may be added later, after evaluating if doing that is worth the added code complexity (it would not be difficult to allow arbitrary boolean predicates on other fields).

Extensibility
No matter how many features will be added to the preference description syntax, there will always be some specific use-case that still needs something that is not provided. Instead of adding tons of new features to cover each and every use-case, it may be better to allow gadget writers to enhance preference dialogs with their own code, which interacts with the dialog with some well defined interface.

This conflicts with an established (and well motivated) policy of not allowing any user defined code to be run on Special:Preferences (even if the "user" is the gadget's writer, which is usually more trusted than ordinary users). A way to solve this is putting user-provided code in an a ResourceLoader module which is registered on Special:Preferences, but not delivered; then, only if the user clicks the "Configure" link for that gadget, the module is downloaded and executed. In this way, the end user would be able to disable the gadget on Special:Preferences, without the risk of malicious code tampering with the interface.

Hidden/restricted fields
There should be a way of marking fields "hidden", so that default is always issued; moreover, there should be the possibility to show certain options only to specific users.

Known issues

 * Gadget preferences are deployed with a delay of 5 minutes after saving them, unless the user purges browser cache.
 * If memcached is enabled, changes to messages used by gadget preferences won't be visible to users untile memcached objects expire (generally 1 hour).

Working example: HotCat v2.9

 * NOTE: syntax of /HotCat.preferences changed in r93975; old syntax won't work anymore.

Steps to reproduce:

svn checkout http://svn.wikimedia.org/svnroot/mediawiki/branches/salvatoreingala/Gadgets
 * Download a copy from the branch I'm working on:
 * and install it.

*HotCat[ResourceLoader]|HotCat.js
 * Add this row to <tt>MediaWiki:Gadgets-definition</tt>:
 * and write a description for the gadget on <tt>MediaWiki:Gadget-HotCat</tt>.


 * Put the content of /HotCat.preferences to <tt>MediaWiki:Gadget-HotCat.preferences</tt> (NOTE: in this example all strings are hard-coded in preferences; in real use, MediaWiki messages would be used instead).
 * Put the content of /HotCat.js to <tt>MediaWiki:Gadget-HotCat.js</tt>.
 * Go to <tt>Special:Preferences</tt> and enable the HotCat gadget: a "Configure" link will (hopefully) appear.

To make the gadget use preferences, these changes were needed.

All the user configuration options documented in commons:Help:Gadget-HotCat (as of 9 Jul 2011) are implemented, with the limitation that the <tt>changedBackground</tt> option currently doesn't allow to choose 'transparent' as a color.

Not much care has been put on improving User Interface appearance and user experience; don't expect it to be great, yet! :)

Preferences editor
The page PreferencesEditor.js contains a simple GUI editor of preferences. After installing Gadgets from my branch, it may be used as a gadget. When ran, it adds a "Preferences editor" link to the Toolbox portlet.

All field types documented here are supported.

Integration howto
This section describes added code, and the changes made to existing classes. Paths refer to the new directory structure in my branch.


 * <tt>/</tt> (root)
 * <tt>Gadgets.alias.php</tt>: no changes
 * <tt>Gadgets.i18n.php</tt>: trivial changes
 * <tt>Gadgets.php</tt>: trivial changes
 * <tt>Gadgets_tests.php</tt>: just added methods to test <tt>GadgetPrefs.php</tt> functions
 * <tt>backend/</tt>
 * <tt>GadgetPrefs.php</tt>: main backend class; static class with these public methods:
 * <tt>isPrefsDescriptionValid($prefsDescription)</tt>: checks if a preferences description is valid (must be called before using <tt>$prefsDescription</tt> with the other methods).
 * <tt>getDefaults($prefsDescription)</tt>: returns the default values for <tt>$prefsDescription</tt>.
 * <tt>getMessages($prefsDescription)</tt>: returns the messages needed to build the preference dialog for <tt>$prefsDescription</tt>.
 * <tt>checkPrefsAgainstDescription($prefsDescription, $prefs)</tt>: checks if <tt>$prefs</tt> are valid preferences for <tt>$prefsDescription</tt>.
 * <tt>matchPrefsWithDescription($prefsDescription, &$prefs)</tt>: changes the <tt>$prefs</tt> array, by putting default value for all missing or invalid values.
 * <tt>simplifyPrefs($prefsDescription, &$prefs)</tt>: removes redundant values from <tt>$prefs</tt>, i.e. values that equal their default (called before saving, to remove unneeded values).
 * <tt>Gadget.php</tt>: old gadget class (was in <tt>Gadgets_body.php</tt>). Added three fields:
 * <tt>$prefsDescription</tt>: description of preferences (decoded from json, <tt>null</tt> if missing or invalid description)
 * <tt>$preferences</tt>: actual preferences for the current user (always valid, "fixed" if needed before saving this field)
 * <tt>$prefsTimestamp</tt>: timestamp of the last time preferences for this gadget have been saved (used to ensure module's freshness when delivering options )
 * Added 8 methods:
 * <tt>getPrefsModule</tt>: returns the module that delivers preferences to the user (<tt>GadgetOptionsResourceLoaderModule</tt>).
 * <tt>getPrefsModuleName</tt>: obvious
 * <tt>getPrefsDescription/setPrefsDescription</tt>, <tt>getPrefs/setPrefs</tt>, <tt>getPrefsTimestamp/setPrefsTimestamp</tt>: accessors for the three new members; <tt>setPrefs</tt> can also save preferences back to DB (if the second param is set to <tt>true</tt>).
 * Also changed things in <tt>loadStructuredList</tt> and <tt>newFromDefinition</tt>, to properly fill the those members (<tt>$preferences</tt> is filled with default values here; this is needed because gadgets may be enabled for anonymous users; prefs for other users are filled in hooks). Code of this methods was somewhat ugly, and now is even uglier: hopefully those don't survive the rewrite :P
 * <tt>GadgetsMainModule.php</tt>: simple module (<tt>'ext.gadgets'</tt>) that creates that <tt>mw.gadgets</tt> object and some non gadget-specific info (e.g.: the list of configurable gadgets).
 * <tt>GadgetResourceLoaderModule</tt>: gadget's module (<tt>'ext.gadget.Foo'</tt>), was in <tt>Gadgets_body.php</tt>. Added dependencies to <tt>'ext.gadgets'</tt> and to <tt>'ext.gadget.Foo.config'</tt> (if needed); overridden <tt>getScript</tt> method to support delivering of preferences (gadget's code is wrapped in a closure).
 * <tt>GadgetOptionsResourceLoaderModule</tt>: actual module (<tt>'ext.gadget.Foo.prefs'</tt>) with user preferences for a gadget; straightforward.
 * <tt>GadgetHooks.php</tt>: was in <tt>Gadgets_body.php</tt>. Changes made:
 * <tt>articleSaveComplete</tt>: if a MediaWiki:Gadget-Foo.preferences is saved, reload gadget (old code only looked for MediaWiki:gadgets-definition).
 * <tt>getPreferences</tt>: just replaced deprecated <tt>wfMsgExt</tt> with <tt>wfMessage</tt>.
 * <tt>registerModules</tt>: added code to register the <tt>'ext.gadgets.preferences'</tt> module; done here because it needs messages returned by <tt>GadgetPrefs::getMessages(...)</tt> for all gadgets. Added code to register the gadgets options module (old code only registered gadget's module).
 * <tt>beforePageDisplay</tt>: only added a couple of lines to load the <tt>'ext.gadgets.preferences'</tt> module on Special:Preferences.
 * <tt>userLoadOptions</tt>: completely new hook. This removes from the <tt>$options</tt> array all the preferences that looks like gadget preferences, unserializes them and saves them in gadget objects (aften ensuring correctness with GadgetPrefs::matchPrefsAgainDescription).
 * <tt>userSaveOptions</tt>: completely new hook. This reinserts gadget's preferences back in the $options array, after calling <tt>GadgetPrefs::simplifyPrefs</tt>, adding timestamp and serializing them.
 * <tt>applyScript</tt>: no changes.
 * <tt>api/</tt>
 * <tt>ApiQueryGadgets.php</tt> and <tt>ApiQueryGadgetCategories.php</tt>: no changes.
 * <tt>ApiGetGadgetPrefs.php</tt> and <tt>ApiSetGadgetPrefs.php</tt>: straightforward new APIs to get preferences (and description) and to set preferences.
 * <tt>ui/</tt>
 * <tt>SpecialGadgets.php</tt>: no changes.
 * <tt>resources/</tt>
 * <tt>jquery.farbtastic.js</tt>, <tt>jquery.farbtastic.css</tt> and the <tt>image/</tt> folder: color picker, adapted from http://acko.net/dev/farbtastic
 * <tt>jquery.formBuilder.js</tt> and <tt>jquery.formBuilder.css</tt>: main cliet-side component, jquery plugin that builds dialogs from preference descriptions. Also contains code for the editor.
 * <tt>ext.gadgets.preferences.js</tt> and <tt>ext.gadgets.preferences.css</tt>: code tweaks for Special:Preferences.
 * <tt>jquery.validate.js</tt>: jQuery plugin for form validation, from http://bassistance.de/jquery-plugins/jquery-plugin-validation/.