User:Nikerabbit/Translate v2

Problem statement. The tags used by the Translate extension to mark translatable parts of translatable wiki pages cannot be easily supported by Parsoid (nor the PHP parser) with their current semantics. First class support in Parsoid is a requirement for implementing a better user interface in Visual Editor.

Issue details
The issues with the tags bear some resemblance with templates: because templates can be mixed freely, for example using multiple templates to construct a table, they cannot be fully isolated from parsing context. In similar manner, one can use the tags to split lists into multiple units each containing multiple list items.

But besides these similarities, there are also other issues, such as how start-of-line context is handled, as documented in T137751. All the three examples below produce a valid heading, but it would be impossible for Parsoid to know this without re-implementing the specifics how the tags are currently parsed.

== A ==

== A ==

== A == For background, Translate uses  hook to massage the wikitext: it removes the tags and the real (PHP) parser never seems them. This was done because the PHP parser, at the time at least, didn't provide enough flexibility to handle the markup gracefully without breaking features like table of contents. This is not the case with Parsoid, which sees the tags.

Proposal
The proposal is to create a new tag with different semantics: it would not be allowed to cross DOM boundaries. By introducing a new tag, we are not forced to break the existing pages (at least until we completely remove the current tag), but instead we can do a gradual migration to the new tag, using automatic scripts and/or linters where appropriate to ease the migration while also allowing us to improve the code in breaking ways before it affects too many pages.

Because this change is purely for technical reasons (to make parsing easier and to enable better future), it is possible it could temporarily make things force, by example of requiring a lot more mark-up in the wikitext. For this reason we propose to consider adding some shortcuts, such as that a long list could be made translatable just by wrapping tags around the whole list (as opposed to marking each list item individually).

It would also be a good idea, while strictly not necessary, to replace the very peculiar  with something more regular.

This proposal is not about removing tags from the wikitext. That is an independent question, although likely easier to implement with the new semantics.

Open questions
Name of the new tag:,  ,  ,  ,  ,  ,  ...

Syntax for.

The specific semantics of the new tag, and how that could be implemented and enforced in Translate/Parsoid/PHP parser.

Timeline
2018-01: Issues discussed by Nikerabbit and SSastry and and initial proposal was created