Specs/HTML

This page defines a MediaWiki-specific DOM based on HTML5 and RDFa. The semantics of MediaWiki-specific functionality are encoded using RDFa.

See Parsoid/HTML5 DOM with microdata for the general idea and background. This is work in progress, feel free to suggest improvements! See http://rdfa.info/ for RDFa documentation and a live parser.

RDFa structures
Global prefix mappings:
 * Convention: Capital for types, lowercase for attributes.
 * Generally use the prefix instead of vocab definitions to avoid clashes (and allow mixing) with user-supplied RDFa. User-supplied RDFa with the mw prefix is moved to a non-clashing prefix in Parsoid.
 * Generally use the prefix instead of vocab definitions to avoid clashes (and allow mixing) with user-supplied RDFa. User-supplied RDFa with the mw prefix is moved to a non-clashing prefix in Parsoid.

Versioning
An integer version number is set in the head section of the returned HTML document. This version is incremented whenever this DOM spec or any other important aspect of the Parsoid HTML output changes. See bug 52937 for details.

mw:Placeholder and general client behavior
A  protects DOM structures from any editing. Clients are expected to preserve / protect subtrees marked as such. Clients are also expected to preserve any DOM subtrees marked up with  in the http://mediawiki.org/rdf/ namespace they don't understand. This decouples clients from Parsoid development, and lets them concentrate on editing constructs whose special semantics they understand without having to implement all possible content elements.

Images
Status: First patch merged. bug 46576. Followup work to be done.

In the examples below, the original size of the example image is 1941 × 220 pixels (these are the dimensions of the Foobar.jpg used in parserTests). The width and height in the DOM represent the actual scaled image height (not the bounding box dimensions specified in the wikitext). When image dimensions are modified or images with a non-default size are created, we will serialize to a square bounding box around the given width and/or height attributes.

The basic tree structure of all images, regardless of formatting options, alignment, or thumbnails, is: The outer &lt;figure&gt; and inner &lt;figcaption&gt; elements need to become &lt;span&gt; elements when the figure is rendered inline, since otherwise the HTML5 parser will interrupt a surrounding block context. The inner &lt;a&gt; element needs to become a span if there is no link; see bug 44627. A "title" attribute on the &lt;a&gt;/an "alt" attribute on the &lt;img&gt; are present if (and only if) the "title="/"alt=" options are present in the wikitext markup.

Specific examples:

  (Note 1)

Without a link, we use the same basic DOM structure, but use a span instead of an a wrapper (see bug 44627):   (Note 1)

Adding 'left' causes the image to be rendered in block context, so the outer &lt;span&gt; becomes a &lt;figure&gt;:   (Note 2, Note 5)

Scaling, vertical alignment of an inline image:   (Note 1)

Caption (containing disallowed markup) on an inline image:   (Note 2, Note 5)

  (Note 2)

  (Note 3, Note 4)

  (Note 3)

 </tt>

 </tt>

 </tt> (Note 5)

Note that "border" can be combined with "frameless".  </tt> (Note 5)

See enwiki help for all options, see mw for inline/float details

Note 1: The PHP parser adds a default alt attribute to the &lt;img&gt; tag, with content "Foobar.jpg". Client-side post-processing will need to add this for compatibility. (Parsoid does not add this attribute because it does not correspond to anything in the wikitext.)

Note 2: In this case the PHP parser adds a title attribute to the &lt;a&gt; and an alt attribute to the &lt;img&gt;, both with the value "caption". Note that this is a markup-stripped version of the supplied caption in some cases. Client-side post-processing will need to add these.

Note 3: The PHP parser adds a  &lt;a href="./File:Foo.jpg" class="internal sprite details magnify" title="View photo details"&gt;&lt;/a&gt; </tt> element inside the &lt;figure&gt;. Post-processing can add this if needed by a client.

Note 4: The default thumbnail width is a user-specified preference for the PHP parser. Parsoid uses a fixed 180x180px bounding box. The "mw-default-size" class indicates "no size given" and can be used to resize thumbs according to user preferences.

Note 5: In this example, the caption is not visible in PHP output, so the there should be a rule in the default stylesheet like (IE7+ and other modern browsers): In the PHP parser output, the caption does appear as a title attribute on the &lt;a&gt; and an alt attribute on the &lt;img&gt;; client side post-processing should add these (unless there are existing title and alt attributes, resulting from "title=" and "alt=" properties in the wikitext).

Semantic info in HTML/RDFa

 * figure classes: mw-valign-{baseline,middle,sub,super,text-top,text-bottom,top,bottom}, mw-halign-{left,right,center,none} and optionally mw-image-border and mw-default-size for full-size images and thumbs scaled to the wiki's and user's default thumb size
 * figcaption sub-element: The caption
 * resource attribute on image: link to image resource page. TODO: what to use for images from commons?
 * width and / or height on image: scaled image size. Only one of width or height is fine for easier client-side scaling without aspect ratio issues.
 * alt attribute on image: alt property
 * src attribute on image: thumb governed by explicit thumb option or implicit from image
 * href attribute on a around image: link target, normally just the image page- BUT a element can be absent if link is explicitly empty.

Wiki links

 * The href attribute is UTF8 (as everything else), with a relative link prefix that always navigates up to the top of the wiki namespace, especially in subpages / pages containing slashes in the title. Example: './Foo', or (in a subpage) './../Foo'. We percent-encode percents and question marks in hrefs to support following links to wiki pages with question marks in their name. On the way in (when posting HTML to Parsoid) we assume href values to be urlencoded and decode them during serialization. Modified link hrefs without ./ or ../ prefix are temporarily assumed to be absolute to the wiki namespace for now, but will also be interpreted as relative to the page soon to support relative links in other HTML content. After that change, the equivalent of an absolute wikilink  Foo </tt> would need to return an href="/Foo" instead.

 alternate linked content </tt>

 Main Page </tt>

Link with tail:  Potatoes </tt>

Category links
 </tt>

 </tt>

Language links
 Foo </tt>

Interwiki non-language links
Status: In development / not yet implemented! See bug 42160.

 en:Foo </tt>

Autolinked URLs
 http://example.com </tt>

Numbered external link
 </tt>

Named external link
 Link content </tt>

ISBN link
<tt> ISBN 978-1413304541 </tt>

RFC link
<tt> RFC 1945 </tt>

PMID link
<tt> PMID 20610307 </tt>

Nowiki blocks
There are two options to handle nowiki editing:
 * 1) Strip the tags from the DOM and let the serializer add those that are needed after each edit
 * 2) Keep them in the DOM for more accurate round-tripping of manually created nowiki blocks, and prevent non-text content from being entered into these blocks in the editor (TODO)

We picked option 2 for now. The nowiki content remains editable. If the content is modified in a way that makes nowiki unnecessary Parsoid can remove the wrapper in the serializer.

<tt> foo  </tt>

HTML entities
<tt> œ </tt>

Behavior switches
Help:Magic_words. Not yet implemented, tracked in 37909.

<tt> </tt>

<tt> </tt>

<tt> __NEWSECTIONLINK__ </tt>

<tt> __NONEWSECTIONLINK__ </tt>

<tt> __NOGALLERY__ </tt>

<tt> __HIDDENCAT__ </tt>

<tt> __NOCONTENTCONVERT__ </tt>

<tt> __NOCC__ </tt>

<tt> __NOTITLECONVERT__ </tt>

<tt> __NOTC__ </tt>

<tt> </tt>

<tt> __NOINDEX__ </tt>

<tt> __INDEX__ </tt>

<tt> __STATICREDIRECT__ </tt>

Category default sort key

 * See bug 46470. Status: ready for implementation.

<tt> </tt>

Redirects
Status: In discussion, see bug 45808.

<tt> #REDIRECT foo </tt>

<tt> #REDIRECT Category:Foo </tt>

<tt> #REDIRECT </tt>

Template content
Status: Ready for implementation, see bug 44555.

Many parameters contain arbitrary wikitext, styles, template names and other non-semantic / DOM strings. We also have very little information which attributes are semantic and which are presentational. For now, we will thus expose all attributes in a simple JSON attribute:

<tt> </tt>

The (optional) id property will let us associate inline parameters with the JSON data later. This lets us support inline editing of things like infobox parameters in the future without changes to the JSON data structure.

Editing compound content blocks that include output from several templates like this football table would benefit from access to interspersed content from the surrounding page. We will implement this by interspersing wikitext strings with template information in the data-mw.parts array:




 * $$1+1$$
 * }
 * }

Editing support for the interspersed wikitext is difficult to implement on the server side, as those wikitext edits need to be restricted in their effect to the original DOM range. A potential solution to this could be to wrap the multi-template compound block into a template hook that expands its content to a well-balanced DOM structure. Arbitrary wikitext edits within this tag would still only affect the original DOM range, both in Parsoid and the PHP parser. This is lower priority though, so for now the interspersed wikitext will be read-only.

Parameter Substitution at the top-level
This section specifies wrapping for parameter uses in the top-level namespace where all parameter substitutions evaluate to a null value.

Templates in attributes (Current Implementation)
<tt> Some text content </tt>

<tt> <div style="">... </tt>

The exact content of the attribute content for editing purposes could be serialized HTML DOM. Alternatively we could include that directly as a sub-dom in a div-wrapped section at the start or end of the document.

Templates in attributes (New Proposal)
Status: Under discussion and revision

This is a new proposal for handling attributes in links, tables, and html tags whose keys and/or values are fully or partially generated by templates. The affected element will be assigned a "mw:ExpandedAttrs" typeof attribute and the data-wm JSON object will provide additional specific information about the keys or values that are fully or partially generated by templates. It is conceivable to think up use-cases where part of an attribute value is generated by a template (ex: color of a background-color of a style attribute), but not as much for attribute-keys. This spec also assumes that a template can only generate one attribute rather than multiple attributes.

data-mw.expandedVals["href"].html will provide the full transclusion HTML for the href attribute value of the element. A client such as the VisualEditor could use this HTML to provide existing transclusion editing support for that attribute value.

A few examples are worked out below.

Example 1:

Example 2: <div style="">...

Example 3:

Templates in attributes (New Proposal, version 2)
Status: Under discussion and revision

This version differs from the first version by encoding template-affected attributes as tuples rather than two separate objects.

Example 1:

Extension content
Status: In discussion.

<tt> $$1+1$$ </tt>

The data-mw attribute is a JSON object. It is meant as an extensible public interface, so more top-level members can be added. The top-level structure depends on the content type, with the main types being templates and extensions. See also the template content section.

The following formats are valid: In the future, more than one format might be present to provide alternate representations of the content. For example, if there is a experimental editor for mathml, the extension might have both <tt>mathml</tt> and <tt>extsrc</tt> formats in the <tt>data-mw</tt> attribute. Brave users can use the new editor on the <tt>mathml</tt> content; other users will continue to use the raw-text editor on the <tt>extsrc</tt> content.
 * wt: raw wikitext, currently provided practically everywhere
 * extsrc: raw extension body text, used as a fallback when no more specialized parser is available/known.
 * html: MediaWiki HTML+RDFa following this spec

Ref and References
Status: In discussion.

First one Second one Named one Reused Reused again

This results in an RDF graph like this (courtesy of http://rdfa.info/play/):

noinclude / includeonly / onlyinclude
Not yet implemented, tracked in 40305. We only care about these in the actual page context, not in transcluded pages / templates. <tt> foo bar baz </tt>

<tt> foo bar baz </tt>

<tt> foo bar baz </tt>

Language conversion blocks
See bug 41716. Status: provisional / strawman. Also see Writing_systems/Syntax.

<tt> foo-{bar baz}- quux </tt>

<tt> foo-{zh-cn:blog; zh-hk:WEBJOURNAL; zh-tw:WEBLOG;}- quux </tt>

<tt> foo-{zh;zh-hans;zh-hant|blog, WEBJOURNAL, WEBLOG}- quux </tt>

General language conversion plan
The wikitext language variant converter interface documented in Writing_systems/Syntax exposes two classes of operations:
 * 1) Selecting content in place by variant, and
 * 2) dynamic modification of conversion rules that apply from that point in the page on.

In-place content selection is not just used for regular translation pairs, but also for constructs like. Content is mostly well-nested, so we can represent this as an element. The exception from grepping (see regexp used below) are constructs like. Those partly stem from times when the language converter could not be used inside attributes. We can probably fix this automatically by moving the variant block inside the attribute.

In general, we will render content-producing variant code based on the wiki's default variant and the fallback chain. Regular content conversion will only happen as a post-processing step on the saved Parsoid HTML.

Dynamic modification of rules does not seem to be needed in general. Page-global and per-category rules can replace template-based definitions. Until that is implemented, we need however represent existing add / remove rules inline. For also content-producing constructs like  we can both render and record the rule modification in data-mw. Pure modifications (H flag) can be represented as meta tags.

Rule format for separately stored page-global rules
-{H|..}- and -{-|..}- can be represented as metas, others as spans. Block-level content seems to be rare.


 * {"*": "XXX"} for rules migrated from -{A|XXX}-
 * {"zh-cn": "tom hanks", "zh-hk": "SOUP HANS", "zh-tw": "TOM HANKS"}
 * {"zh-cn": {"HUGEBLOCK":"macro"}, "zh-hk": {"BLOCKHUGE":"big"}} for migrated -{H|HUGEBLOCK=>zh-cn:macro;BLOCKHUGE=>zh-hk:big;}-
 * {"zh-cn": {"HUGEBLOCK":"macro"}, "zh-hk": {"HUGEBLOCK":"big"}} for migrated -{H|HUGEBLOCK=>zh-cn:macro;HUGEBLOCK=>zh-hk:big;}-

For consumers of this format:


 * If a rule value is a string, it is a direct translation rule
 * If a rule value is an object, it contains one or more unidirectional nested rules

Other considerations

 * $wgDefaultLanguageVariant and fallback chain for it (search for variantfallbacks in LanguageZh.php, retrievable from . Note that getVariantFallbacks can return a string OR an array for different input... It seems to make more sense to have getVariantFallbacks do array_diff itself but it's not doing so currently... ) is not currently exposed in the API. We'll need both to pick the right content to render for -{zh-tw:foo;zh-cn:bar}-.
 * See also 52700.


 * What to store in  when it contains some other structures?
 * HTML, but the content needs to be properly nested. Run  on a zhwiki dump to find potentially problematic language conversion blocks, then check nesting for them. Problematic cases seem to be
 * different start tags per variant that really only differ in an attribute (title for example). Conversion pairs are now also supported in attributes, so try to fix wikitext to convert attribute only.

Result of : Total revisions: 2234532 Total matches: 773 Ratio: 0.034593373467016804%

<tt> <div title="-{foo}-"> </tt>
 * Need a way to mark up in-place variant conversions in attributes. Idea that might also be useful for transclusion-affected attributes:

Error handling
See bug 48900.

<tt> </tt>

TODO
Some cleanup is needed (from bug 53432):
 * mw:WikiLink/Interwiki becomes mw:ExtLink/Interwiki, and we'll automatically detect interwiki prefixes in new/modified mw:ExtLink content.
 * mw:WikiLink/Category becomes mw:PageProp/Category as these are not really links. They don't render at all in the page, don't accept a caption etc.
 * mw:WikiLink/Language becomes mw:PageProp/Language - same as with categories.

Also under consideration: Simplify mw:ExtLink/* to just mw:ExtLink. The information can mostly be extracted by matching a href prefix. The only problematic area is client-side rendering of numbered external links.

The following constructs still need a RDFa markup definition. They will initially only be marked with typeof="mw:Placeholder" for simple read-only round-tripping.
 * template parameter references (implemented, but not tested much)
 * extension as HTML5 sections (see bug 47936).
 * extension as HTML5 sections (see bug 47936).