Extension talk:Linter

Why is this considered a "Fostered content" error?

{| class="wikitable"
{{{{{subst|}}}!}}<div>a</div>
|}

It's sort of a bug in Parsoid, it's struggling with the templated template name.

The php parser sees this as {{{subst|}}}, a top-level template argument, whose default value is the empty string. So, gets expanded to {{!}}<div>a</div>, which gives the desired wikitext table syntax.

Substitution (subst) is supposed to occur when saving a page, and shouldn't show up in wikitext.

Filed as https://phabricator.wikimedia.org/T200293

I've responded there, but why is the subst showing up in saved source? Or is the complaint that Parsoid isn't handling PST before saving?

The author used the wrong syntax, three { instead of two.

three braces are OK for this case use inside a template. It means taking parameter whose name is subst, and if it is not passed by the page transcluding the template, use a default empty text value.

The role of this parameter in any template "X" is to:

• pass either "|subst=" (i.e. empty text, the default value), so that the code {{!}} will be used (and saved under this form on editing and saving the page making the transclusion of "X")
• or pass "|subst=subst:", so that the | will be used (when saving the page using the template "X", and if the template X itself was "subst:'ed" as {{subst:X|...|subst=subst:}}, then that page will not be saved to contain {{subst:!}} or {{!}}, but its expansion (to a pipe character).

But, {{{subst}}} substs .. i just tested ...

oh .. never mind ... i see what you are saying about top-level argument.

Use of Linter in its current state, results in all zeros displayed in Special:LintErrors on third party wikis, if manual parsoid config (shown in the post) isn't added. This extra parsoid config uses the wikimedia/parsoid version included in MediaWiki.

Does Linter work on third-party wikis? I've installed MW 1.39.1 (on Apache, php 8.0.12) added Linter with the recommended settings:

$wgParsoidSettings = [
    'linting' => true
];

I ran update.php, it shows in Special:Version, but all stats are empty in Special:LintErrors. Editing a page with VisualEditor that has obsolete tags doesn't make the page populate into Special:LintErrors/obsolete-tag. The job queue is empty, there's nothing in the debug logs. I've run rebuildall.php in case it's a category link issue, I can't get this extension to work.

I've found one wiki (farm) on 1.38 (Miraheze) where Linter is functioning correctly, and the folks in MediaWiki discord pointed out that the other third-party wikis found via wikiapiary, using Linter, have all zeros for Linter results like I do--this seems unlikely to be because there's no Linter errors on any of their pages.

The notice "this extension requires linting enabled in parsoid's configuration" in combination with the suggested setting above, lead me to think that the bundled parsoid and its corresponding setting should work, but it doesn't. I see Miraheze is using a manual parsoid config with their (working) version of Linter. Perhaps in its current state, Linter requires the separate parsoid setup/config still, which isn't documented?

If it doesn't work with third-party wikis out of the box, it would be great to add a notice of some kind until the documentation is working and complete.

I've largely given up on making Linter work at this point, I am mostly leaving this here in hopes it might help someone else get farther than I did. I tried a number of variations in the settings (including some I saw in the extension page's history) and these instructions. I also tried making Linter work by following Miraheze's example who got it working in 1.38, loading it as an extension using:

For me, this resulted in a 404 with "The requested relative path () did not match any known handler".

I tried this with and without the extension page's recommended code:

$wgParsoidSettings = [
    'linting' => true
];

This is where I stopped, as I've already been through this same relative path error and most of the others described in task T263928 with manual parsoid/restbase config. I decided I didn't need Linter that badly, but perhaps this info might help someone else.

Okay, so working with the folks in MediaWiki Discord, I figured out how to load Parsoid as an extension (the built-in wikimedia/parsoid version), which lead me to looking at Miraheze's mw-config setup on Github. That got me part of the way there, but broke VE editing.

Fortunately, these links and footwork helped Feivel, the smarty pants at Star Citizen wiki to take it over the finish line, which helped me spot where my own (silly) error was (this particular site required the 'www' be added). The config I ended up with, works now for pages parsed by VE/Parsoid

# Parsoid
wfLoadExtension( 'Parsoid', "$IP/vendor/wikimedia/parsoid/extension.json" );

$wgParsoidSettings = [
    'useSelser' => true,
    'linting' => true,
];

$wgVisualEditorParsoidAutoConfig = false; #(for Linter, to make work)

$wgVirtualRestConfig = [
	'paths' => [],
	'modules' => [
		'parsoid' => [
			'url' => 'https://www.mysite.wiki/w/rest.php',
			'domain' => 'www.mysite.wiki',
			'forwardCookies' => true,
			'restbaseCompat' => false,
			'timeout' => 30,
		],
	],
	'global' => [
		'timeout' => 360,
		'forwardCookies' => false,
		'HTTPProxy' => null,
	],
];

Updated instructions to remove stale information.

The page has a note saying "This extension requires Parsoid to be installed". However, Parsoid is part of MediaWiki core since several stable versions ago, so there's no way to "install" it.

And it says "and linting enabled in Parsoid's configuration", but in the Configuration parameters section, it sets "'linting' => false", which is the opposite of what one would expect for "enable linting on Parsoid".

Good catch! Those instructions were stale. Fixed.

I've installed the extension and modified the config.yaml for Parsoid as per the instructions. VisualEditor is working fine so I know Parsoid and Restbase are both working. However, I am not getting any lint errors at all. This seems unlikely. I've tried deliberately putting an error on a page but it isn't picked up. Any ideas? Is there anything I can do to test that everything is working correctly?

• You can try setting sendAPI: false and rendering exclusively with Parsoid to see if it logs the errors you expect.
• You can check the logs of your MediaWiki API to see if it's rejecting the posted lints
• Maybe you need to whitelist the server Parsoid is running on https://github.com/wikimedia/mediawiki-extensions-Linter/blob/master/extension.json#L166-L169

Thanks for your advice. I think I've solved it. It seems that the version of Parsoid I'm using is expecting the config variable linterSendAPI rather than the current variable. So lint errors were being detected but were going straight to the log, not to the API. However, when I put "linterSendAPI: true" in the config Parsoid won't work at all! Very frustrating! Would I be correct in thinking that this means I'm using an older version of Parsoid and need to update?

I've now established that, for some reason, my server was no longer picking up Parsoid updates from the repository. Fixed that and will be upgrading from the currently installed version (0.7.1) to the latest in the repository. Hopefully that will fix this issue.

The config was renamed between versions 0.7.1 and 0.8.0

You might not want to upgrade as high as 0.9.0 though, see the release notes https://www.mediawiki.org/wiki/Parsoid/Releases#0.9.0_(released_Mar_23,_2018)

Thanks for the pointer. I have now upgraded to 0.8.0. However, it still isn't working. I'm getting the following in the Parsoid log on startup:

WARNING: For config property linter, required a value of type: object

Found null; Resetting it to: {"sendAPI":false,"apiSampling":1,"tidyWhitespaceBugMaxLength":100}

A diff comparing my config.yaml with the distribution version comes up with the following (ignoring domain name stuff);

83,87c83,86

<         linting: true

<         # Send lint errors to MW API instead of to the log

<         linter:

<               sendAPI: true

<               apiSampling: 10

---

>         #linting: true

>         #linter:

>         #  sendAPI: false # Send lint errors to MW API instead of to the log

>         #  apiSampling: 10 # Sampling rate (1 / 10)

97,99d95

<

<       # Enable the batch API

<       useBatchAPI: true

Any ideas? I'm assuming there is something I need to change in config.yaml to make this work?

Maybe a whitespace issue? Do you have any tabs hanging around?

Yes, it was a whitespace issue. I had tabs rather than spaces on the sendAPI and apiSampling lines of config.yaml. I am no longer getting lint errors in the Parsoid log. I can't see anything relevant in the API debug log but I suspect that is because the page I've been using no longer has any lint errors! I'll have to try and provoke one to make sure it is working.

Coming back to this after getting distracted by a problem with subpages...

I'm still not getting any Lint errors. Looking at the Parsoid log, I'm seeing multiple entries saying "Parsoid id found on element without a matching data-parsoid entry". I'm also seeing "Incompatible constraints (merge)" messages. I don't know if the incompatible constraints reports are significant. I know that for a while Parsoid was unable to log errors through the API due to a whitelist problem which I've fixed. Is that the cause of the "element without a matching entry" messages? Is it stopping errors being reported from my test page? If so, what do I do about it? Any advice gratefully received!

Our special page suggests a missing end-tag on this page we use to access an archive by year. The page is mainly a transclusion of this single archive page, that contains no errors. Using lintHint 0 errors are reported for this original page and 4 for the transcluding page! As far as I can see, 3 error messages are generated by the "{{qtu}}" in section 66.1 and 1 by section 79, a WMF message. As there seems to be no problem rendering the page as intended, might this be a bug in Linter?

I have narrowed down the problem to our template: Tl that is used to create a link to a template (description) page, and uses html-entities instead of {}-brackets in order to achieve the desired result. For Linter this is no problem on the page itself, but when the page containing the template is transcluded itself apparently it becomes too much. My question now becomes:

a. Will using the html-entities in the way our template Tl become problematic in future parsers? In that case it would constitute a new error?

b. If the answer to a. is "No!", then isn't this a bug to be fixed in Linter?

Chances are that it isn't a false positive. The problem probably comes from the hacky way you're trying to display the templates, since it ends with }}. It means that if there is another template on the page it will complete the transclusion and have unexpected results, e.g.:

{{ecologie|{{Tl|foo}}

A more sensible approach is to either apply nowiki to each side, or replace all curly brackets there with their html entities:

{{templatename}}

Thanks for the reply and the suggestion. As far as I am concerned my question has been answered and the problem is solved.

Issue:

Currently, the special page doesn't provide same type of paging and sorting available in other special pages. The only way to get more results is to keep clicking next.

Proposed solution

Make it possible to :

• Change number of results , e.g.:https://en.wikipedia.org/wiki/Special:LintErrors/misc-tidy-replacement-issues?namespace=0&limit=5000
• Sort results, e.g. by page name / template, and description, like in https://en.wikipedia.org/w/index.php?title=Special:ListFiles&sort=img_name&limit=50

It might be a general problem in most pages that use the "newer" query list design, or simply a switch that wasn't enabled for this one because it exists in listfiles.

See this query. The error location is wrong, because 278-341 are

<code>」字號以利排版，要複製再利用文獻時，不用複製「<code>*<code>」字號。

the error text should be are

<code>*<code>」字號以利排版，要複製再利用文獻時，不用複製「*</code>」字號。

This location caused my bot to make a wrong repair.

Assuming there are multiple errors on the same page, will the location of other errors be updated after an error is fixed?

It will get updated once the page is reparsed.

Will only change the location, other properties (e.g. error id) will not change?

Error id also changes.

It's bad news for me. If I query out 1, 2, 3, 4 error id before fix, assuming these errors are on the same page. I fixed No.1 error, after the error fix will wrong, because the location of the errors is based on the previous context (before fixed N0.1). Do you have any good ideas?

Now I think to fix only the first error and give up fixing the errors behind. If the error id is not changes..I can append a API query to update location..

Fix it in the reverse order of offsets .. i.e. fix the error with the largest offset first .. that way, offsets for errors earlier in the page won't change.

Issue

Paste the following in a wiki page:

<code>A piece of computer code
asdasdasd
<h2> buuu</h2>
zsadsdasd

Expected:

Output html should be similar to above.

Actual:

<p><code>A piece of computer code
asdasdasd
</code></p><code><h2><span class="mw-headline" id="buuu">buuu</span></h2></code><code><p>zsadsdasd
</p></code>

See it in action on this page (https://www.mediawiki.org/w/index.php?title=Project:Sandbox&oldid=2583108)

Possible solutions:

• Increase priority of this lint error - This seems to be creating huge rendering differences, and seems bad enough to either increase the priority of the "misnested" tag
• Set it as a separate high priority category
• Or decide whether this is actually a bug in the parser - and fix that

Notes: I've been seeing this for quite some time in some help pages (e.g. https://www.mediawiki.org/w/index.php?title=Help_talk:Searching&oldid=2475830 scroll down), and couldn't understand where it came from until I tried it in a different wiki, and it became obvious.

Note that high priority rendering categories are about whether rendering will change when Tidy is replaced with a RemexHTML based tool. It is not about whether the HTML output for a piece of wikitext matches expectations.

In this case, the actual HTML structure is slight different for Tidy, RemexHTML and Parsoid. However, the rendering in the browser is identical.

So, this doesn't belong in the high priority category. Right now, we don't want to confuse editors and add more work to their plate beyond what is required to switch from Tidy to RemexHTML. Once Tidy is replaced, we could change the priority of linter categories based on new desired goals.

Err, rendering did change. Did you actually look at the url?

Try looking at https://www.mediawiki.org/w/index.php?title=Project:Sandbox&oldid=2583108&action=parsermigration-edit. Maybe you've customized the code tag in your browser or personal css, and you don't see the gray areas.

Anyway, it is up to you. Personally I found it so odd, that I began to think there was something wrong with my browser.

Ah, sorry, you are right. I looked at output in my browser outside the mediawiki CSS settings where the differences don't show up. This doesn't happen with the small tag (except for the vertical whitespace diff). Parsoid's analysis for linter errors treats all formatting tags identically, and both code and small are formatting tags. But, looks like CSS for the code tag changes how the heading and paragraph render.

We could add a special case for the code tag, but I am inclined to let this be for just now unless there are a lot of code-tag-related issues in practice.

Hmm, you're right. Those come from the wiki-specific CSS styles. Still, the way the way those code tags are nested results in invalid html, see :

https://validator.w3.org/nu/?doc=https%3A%2F%2Fwww.mediawiki.org%2Fw%2Findex.php%3Ftitle%3DProject%3ASandbox%26oldid%3D2583108

That might be a completely separate problem, maybe https://phabricator.wikimedia.org/T134469. I suppose that code tags aren't used that much in articles, and because the linter detects this it might be reasonable to let editors take their time fixing it.

Output compliance wrt content model constraints is covered in Parsing/Notes/HTML5 Compliance ... that is a separate discussion.

I have installed mediawiki with Linter, VisualEditor and Parsoid extensions.

I get "Your IP address has not been whitelisted to report lint errors" error when I want to edit a page using VisualEditor.

Should I white list my IP anywhere? What is this IP, the visitor IP, server public IP or localhost?

You need to whitelist IPs of your Parsoid server in your mediawiki config.

See https://gerrit.wikimedia.org/r/plugins/gitiles/operations/mediawiki-config/+/d6ca1267a4680a020630e767c4fedd074c8c521e/wmf-config/InitialiseSettings.php#14844 for the wikimedia production config.