User:Daniel Kinzler (WMDE)/MCR-PO

While refactoring the MediaWiki storage layer for MCR, questions have arisen regarding the handling of multiple slots in ParserOutput.

This document is intended for discussing the requirements and constraints, and the possible solutions.

Please comment and discuss inline.

Baseline
Goal: provide an MCR-capable interface for updating pages, with a minimum of refactoring of adjecant code. The implementation doesn't need to fully support MCR yet.


 * Use ContentHandler::getParserOutput to generate output for each slot
 * A true "baseline" doesn't require this. You only list it here because of the extra requirements you include below. Anomie (talk) 20:32, 26 February 2018 (UTC)
 * Use Content::getSecondaryDataUpdates to get data updates for each slot
 * A true "baseline" doesn't require this. This seems aimed at "SDC wants two slots blindly concatenated". Anomie (talk) 20:32, 26 February 2018 (UTC)
 * Update links tables with information from all slots. In the future, links tables may also record which slot references what resource.
 * A true "baseline" doesn't require this. This seems aimed at "SDC wants two slots blindly concatenated". Anomie (talk) 20:32, 26 February 2018 (UTC)
 * It's aimed at "SDC wants to be able to place the output of two slots freely on a page", along with "SDC wants to re-use the Wikidata representation and editing interface for the structured output", which I just confirmed as requirements with Mark and Ramsey. Plus "it seems really useful to be able to use output for different slots independently of each other for various use cases", which I confirmed with James. Do you disagree with these things even being requirements? -- Daniel Kinzler (WMDE) (talk) 18:32, 28 February 2018 (UTC)
 * Store a single ParserOutput for the revision in the ParserCache. Can be just the ParserOutput for the main slot for now, but will have to somehow contain all information from all ParserOutputs of all slots eventually. The code composing that combined output may be different for different "kinds" of pages, and may be defined by an extension. The mechanism for that doesn't need to be decided at this stage, though.
 * It won't necessarily have to contain all information from all slots. Even if it does contain some information from a slot, that need not be all the information. Trivial example: if we add a "metadata" slot to hold categories and such, there's no need for the main ParserOutput to contain whatever HTML might be generated to view that metadata. Anomie (talk) 20:32, 26 February 2018 (UTC)
 * If we want to have it in the ParserCache, it has to be in the PO. Or we re-design the ParserCache. -- Daniel Kinzler (WMDE) (talk) 18:32, 28 February 2018 (UTC)
 * The use of ApiStashEdit can be restricted to the main-slot-only case for now.
 * Only if EditPage is restricted to the main slot only. Anomie (talk) 20:32, 26 February 2018 (UTC)
 * No, it could just not use stashing if it's being used for multiple slots. This will be rare anyway, at least in the beginning. -- Daniel Kinzler (WMDE) (talk) 18:32, 28 February 2018 (UTC)

This implies that we need access to POs for individual slots and a combined PO during saving. This does not imply anything about the structure of the PO that gets written to the ParserCache or shown in a preview.

Multi-Slot View
Model use case: Structured Data on Commons (SDoC). Single-slot viewing should also be possible.


 * Allow the rendering of one slots to depend on content of all slots. (Possible restriction: only allow the main slot to depend on other slots. This would allow the main slot to depend on the rendering of other slots, which could otherwise lead to circular dependencies). This needs ContentHandler::getParserOutput to have access to all slots.
 * Allow Article::view (or a replacement of that method) to show the content of each slot, rendered individually, in separate sections of the page. In particular, allow a rendering of the structured data to be shown that is independent of the wikitext, rendered in the user's interface language. That rendering also serves as the editing interface for structured data, just like on a Wikidata page.
 * This, IMO, should be a non-standard view. And it may not need to display all slots at once, versus showing individual slots. Anomie (talk) 20:32, 26 February 2018 (UTC)
 * I asked that question to the MultiMedia team. The result was that they want to at least have the option to have the standard view serve the full HTML of both slots. And it seems likely that this will be the case at least for the initial roll-out. -- Daniel Kinzler (WMDE) (talk) 18:39, 28 February 2018 (UTC)
 * The code composing that combined output, when constructing and/or when using the PO, will be different for different "kinds" of pages, and may be defined by an extension. In the SDoC case, it will be specific to the File namespace, and will be supplied by the WikibaseMediaInfo extension.
 * This is an assumption that need not hold, and either limits the combining to a single extension per page or requires different extensions to hook into or extend each other. What if, for example, TimedMediaHandler wants a slot of its own for subtitles, with its own "combined output" code? Anomie (talk) 20:32, 26 February 2018 (UTC)
 * Since a completely generic combining mechanism seems insufficiently flexible (or would have to be extremely complex), and it's unclear how multiple extensions would cooperate on this, the present idea is to have a singe extension control the rendering of a page. I could also imagine us allowing a single extension to do complex composition, while allowing other extensions to add stuff at the top or bottom.
 * Anyway, what's the alternative? Am I understanding correctly that you want to use wikitext in the main slot as the only composition mechanism? -- Daniel Kinzler (WMDE) (talk) 18:39, 28 February 2018 (UTC)
 * Note that for output in the user language, the ParserCache needs to be split (which is already the case for File pages on Commons)

Multi-Slot View (Anomie's alternative)

 * The default view will show the content of the main slot, as it does now for the non-MCR case.
 * The main slot can explicitly transclude content from other slots, via parser functions or tags, as can happen now. This probably won't generate a ParserOutput for that slot, although that's a possibility if the implementation of the function/tag finds that useful.
 * Content from other slots might also be implicitly "transcluded", much like how Extension:Cite does now when no is present in the wikitext. This need not generate a ParserOutput either, e.g. a "metadata" slot might just call addCategory, addDisplayTitle, and so on to add to the ParserOutput from the main slot. We could locate the code for implicit transclusion in hooks (as Extension:Cite's implicit  does now), or we could add it to Content or ContentHandler to avoid having to use hooks.
 * So, the main implication is that the default view will always be the main slot only. The only way to show things from other slots in the default view would be to explicitly access it from the main slot's wikitext (or other content), or by using hooks to do something very similar to what I am proposing with the combined PO.
 * Since pretty much all the use cases that have been discussed for MCR (Requests_for_comment/Multi-Content_Revisions) assume that the content of all slots is visible per default, it like a good idea to establish a standard mechanism for achieving this easily. I don't see how this can be done in Content or ContentHandler, I'd do it in a separate "PageOutputComposer" or "PageTypeHandler", since the composition logic doesn't have anything to do with the content model. Point in case, for e.g. template documentation, all slots would be wikitext. -- Daniel Kinzler (WMDE) (talk) 22:36, 28 February 2018 (UTC)
 * It should also be possible to view an HTML rendering of a non-main slot. This would generate a ParserOutput for the slot.
 * The parser cache will likely need to be extended to be able to cache non-main slot ParserOutput objects, both for the non-main viewing and for use by potential implicit transclusion. Whether we cache only the main slot's pre-implicit-transclusion ParserOutput (and run implicit transclusion as a post-cache transformation), only the post-implicit-transclusion ParserOutput (and thus re-parse the main slot whenever any slot changes), or both is yet to be determined.
 * I was trying to avoid that. Partially just to avoid additional work, but also because code reading from the ParserCache expects the PO in there to be "the page output". Which it wouldn't be if we only put the main slot there. Basically, my thinking is that all slots contribute to the page's output. Your approach seems to be that the main slot is the page output, but there may be other page output, which may or may not be tagged onto the main output. -- Daniel Kinzler (WMDE) (talk) 22:36, 28 February 2018 (UTC)
 * Access to other slots of the revision will proceed by loading the revision, normally by using Parser::fetchCurrentRevisionOfTitle. If a Parser instance isn't available, ParserOptions::getCurrentRevisionCallback could be used more directly or via a similar helper.

The SDoC case could work like Extension:Cite's : if the wikitext includes a tag to say "insert the structured data here" it will do so, and if not it'll append the structured output to the main slot's wikitext.
 * Appending to the wikitext will not work, it would need to append full HTML, plus registering the necessary JS modules, CSS modules, tracking info for the links table, etc. This sounds a lot like the "combined" PO I was suggesting, but it's unclear to me how the Wikibase content can register its DataUpdates. Probably by hooking in again at some later point. Basically, this sounds like an emulation of my proposal using hooks instead of defining a mechanism for composing a PO from multiple slots.
 * Overall, this seems to be more work: we have to modify the parser cache, introduce new hooks, provide ways to access all kinds of content from withing wikitext, or use several hooks for every slot so it can attach itself to the main PO. I'd rather have a mechanism that just defines how the combine PO is composed, with a per-page-type handler for combining the HTML in the desired way. -- Daniel Kinzler (WMDE) (talk) 22:22, 28 February 2018 (UTC)

Multi-Slot Editing
Model use case: Atomic editing of a Lua module and its documentation. Single-slot editing should also be possible.


 * Allow EditPage to show one editing area (for text based content: one text area) for each slot. Also provide a single-slot editing mode.
 * Allow previews to be rendered for all slots or individual slots.
 * When parsing/rendering after save, re-use the cached rendering of unchanged slots, instead of regenerating it needlessly. However, if the rendering of the unchanged slot depends on a slot that was touched, it needs to be re-rendered anyway.

This implies that ApiStashEdit, ApiParse, and ApiEditPage all accept input for multiple slots at once.

Proposed Changes to ParserOutput

 * Each ParserOutput can record which slots it depends on
 * The combined canonical PO has to return aggregated information of all slots from all getters
 * The combined canonical PO should not contain duplicate information, since it will be serialized and stored in the parser cache, so size matters.

Option 1, compose output just in time:
 * The combined ParserOutput for all slots would have to
 * know the ParserOutput objects for each slot
 * getters aggregate info from all the POs on the fly.
 * know how to combine the HTML from all slot's ParserOutputs into a single output (or return null from getText), by substituting placeholders in an HTML "template". Note that this template may be defined or modified by an extension, and may depend on page type or namespace or some other property of the page.

Options 2, compose output right away:
 * The combined parser output gets fed with the aggregated content of the POs.
 * The code that constructs the combined PO also constructs the combined HTML right away.
 * This means that during saving, the POs of unchanged slots cannot be re-used.

Note that option 1 and 2 offer largely the same interface to calling code. The only code that needs to be aware of this distinction is the PO itself, and the code that constructs it.

This means that we can start with option 2, and change to option 1 later when need arises, for optimizing the rendering during save, or for other purposes. E.g. AbuseFilter may want to apply different rules to different slots, so it would have to process the POs of the individual slots separately.


 * Since all this depends on your model of how things work that I disagree with, I find all this unnecessary as well. Anomie (talk) 20:32, 26 February 2018 (UTC)
 * If I understand correctly, you want to "somehow" merge all info from the extra slots into the ParserOutput of the main slot. That seems very close to my proposal, the only difference being whether we conceptually have a "combined" PO that is different from the main slot's PO or not. I think it would be useful for this discussion if you could state your proposal below, and explain how it meets the requirements above. If you disagree with the requirements, please explain how and why. -- Daniel Kinzler (WMDE) (talk) 18:27, 28 February 2018 (UTC)
 * Actually, for the "combine output using hooks" version of your proposal, the need/opportunity for optimization is the same. The difference is then just whether we have separate keys for each slot in the ParserOutput. I like separate keys in principle, but see little practical advantage, and I expect sticking with a single key is going to be a lot simpler. -- Daniel Kinzler (WMDE) (talk) 23:05, 28 February 2018 (UTC)