Extension talk:SyntaxHighlight/Archive 2021

• Extension talk:SyntaxHighlight/REL1.25 (LTS)

Previous discussion was archived at Extension talk:SyntaxHighlight/Archive 2017 on 29 March 2017.

PHP syntax

This plugin does not support modern PHP syntax:

$person = new class
{
    public $firstName = "John";
    public $lastName = "Smith";

    public function name(){
        return "$this->firstName $this->lastName";
    }
};

37.212.37.47 (talk) 00:23, 26 January 2021 (UTC)

Thank you, but we cannot help that.

Update needs to be made upstream, by everybody, at: https://pygments.org/ Gib Senf dazu! (talk) 12:57, 27 January 2021 (UTC)

Setting $wgSyntaxHighlightModels

On my CentOS Stream release 8 testing environment with PHP 7.4.15 (cli) I made the suggested configuration of $wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua'; in my LocalSettings.php.

Then I ran php maintenance/update.phpand I get the error message:

PHP Warning:  Use of undefined constant CONTENT_MODEL_SCRIBUNTO - assumed 'CONTENT_MODEL_SCRIBUNTO' (this will throw an Error in a future version of PHP) in /var/www/html/w/LocalSettings.php

My edit to update the article with the address the PHP Warning was reverted, per this revision.

Which is correct, the PHP Warning or the revision? Peculiar Investor (talk) 18:46, 18 February 2021 (UTC)

Both. Pppery is right that the solution is not to replace it with a string, but PHP’s also right that you shouldn’t use an undefined constant in the first place. However, PHP’s assumption of what this constant should be is wrong (which is not surprising, since it’s just an assumption)—probably it should be the integer 828. Have you Extension:Scribunto installed at all? Without that, configuring syntax highlight for Scribunto modules makes no sense. Tacsipacsi (talk) 21:59, 18 February 2021 (UTC)

@Tacsipacsi @Pppery Yes Extension:Scribunto is installed and working on MediaWiki 1.35.1. I'm trying to get Syntax Highlighting functioning properly. The Installation and Configuration sections, plus the Talk pages give lots of helpful information but there still seems to be some dark arts to getting everything working. It doesn't help when the maintenance/update.php script throws a PHP Warning when following the configuration information for the extension. Peculiar Investor (talk) 22:17, 18 February 2021 (UTC)

Then maybe you load things in the wrong order? LocalSettings.php is executed from top to bottom, so Scribunto should have already been loaded at the point you want to use its constant.

This won’t work:

wfLoadExtension( 'SyntaxHighlight_GeSHi' );
$wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';
// ...
wfLoadExtension( 'Scribunto' );

But this should:

wfLoadExtension( 'Scribunto' );
// ...
wfLoadExtension( 'SyntaxHighlight_GeSHi' );
$wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';

Tacsipacsi (talk) 22:54, 18 February 2021 (UTC)

@Tacsipacsi Per you suggestion I did change the order in LocalSettings.php. The PHP Warning message still appears however. My LocalSettings.php contains the following snippet:

wfLoadExtension( 'Scribunto' );
$wgScribuntoDefaultEngine = 'luastandalone';
// For a more pleasant user interface, with syntax highlighting and a code editor with autoindent
$wgScribuntoUseGeSHi = true;
$wgScribuntoUseCodeEditor = true;
wfLoadExtension( 'SyntaxHighlight_GeSHi' );
$wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';

FWIW Lua modules are working and <syntaxhighlight> tags are also working on the wiki. Peculiar Investor (talk) 23:27, 18 February 2021 (UTC)

Have you cleared your browser cache, restarted the web server etc.? You might even try to intentionally cause another warning in LocalSettings.php to make sure it’s not just stuck in the cache. If it isn’t a cache issue, then I have no idea why it doesn’t work. By the way, guessing from the name, $wgScribuntoUseGeSHi should enable syntax highlighting on its own, without the line causing the warning. Doesn’t that work? Tacsipacsi (talk) 00:42, 19 February 2021 (UTC)

I can confirm that using $wgScribuntoUseGeSHi = true; does work without also having $wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';. Perhaps the configuration section should clarify how the two settings interact or which one to set.

As a testcase I commented out $wgScribuntoUseGeSHi = true; and then added $wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua'; and I still get the PHP Warning.

I've checked and my extensions are up-to-date 1.35 versions. So the mystery remains as to what's causing the PHP Warning. Peculiar Investor (talk) 20:20, 19 February 2021 (UTC)

Showing line numbers to those reading Module: pages

Feature is new in the forthcoming 1.36 release.

What setting or CSS is required to make line numbers appear on Module pages such as Module:Bananas or Wikipedia's Module:Example?

Our wiki is running 1.35.1, with up-to-date versions of Extension:Scribunto and Extension:SyntaxHighlight.

Our wiki has imported Wikipedia's Module:Example and the code is syntax highlighted as expected but there are no line numbers. Our MediaWiki:Common.css is copied from Wikipedia. We are trying to figure out what setting or CSS is required so that line numbering appears. Thanks in advance. Peculiar Investor (talk) 00:18, 20 February 2021 (UTC)

It's new code from last month. It will be released as part of the 1.36 branch process later this year, sorry. Jdforrester (WMF) (talk) 00:46, 20 February 2021 (UTC)

@Jdforrester (WMF) Thanks. Now I can stop searching. The good news is I've learned quite a bit about the internals. Peculiar Investor (talk) 01:16, 20 February 2021 (UTC)

Re-opening rather than starting new topic.

We have now updated the wiki to MediaWiki 1.39.1 and we still have the same issue, or actually a bigger one as none of the SyntaxHighlight functionality displays.

The wiki is running on shared hosting at a major hosting provided. python3 is installed as an "add-on" functionality.

I have tried the steps listed in the Troubleshooting section without success. I finally set $wgDebugToolbar = true; which reveals the pygmentize cannot find python3.

That's because the hosting provider only has python version 2.7.5. The python3 "add-on" installation is not on the Apache or PHP path.

Is there any additional configuration steps similar to $wgPygmentizePath = that could be used to provide the python3 path? Peculiar Investor (talk) 18:06, 17 January 2023 (UTC)

SOURCE vs. SYNTAXHIGHLIGHT

Not sure if this has already been posted, but for non-Wikimedia installations of MediaWiki 1.35.1 LTS (at least with DreamHost Shared Hosting), the syntaxhighlight element does not work but the source element does. This is very confusing since the extension page here on MediaWiki.org says that the source element is deprecated, though the README file included with the extension says to use the source element (which is how I figured out to get the syntax highlighting to work—otherwise I might not have guessed to try the supposedly deprecated source element). Nicole Sharp (talk) 03:23, 14 March 2021 (UTC)

Based on a quick skim of the page you linked, DreamHost is only a host service and doesn't provide MediaWiki installations for customers, meaning whatever wiki you were working on must have (for whatever reason) customized their copy of the SyntaxHighlight extension to use source instead of syntaxhighlight. Regardless, source is deprecated and support for it will be removed (probably in the REL1_37 branch once that gets cut). Whether individual wikis (or wiki farms) decide to change the extension to continue supporting source, or even to drop syntaxhighlight, is outside the purview of the extension's maintainers here. 「ディノ奴千？！」^{☎ Dinoguy1000} 10:39, 14 March 2021 (UTC)

I installed MediaWiki myself using default settings for MediaWiki and SyntaxHighlight: https://www.nicolesharp.net/wiki/special:version. The only reason I mentioned the webhost is if the error has anything to do with the configurations of PHP or Python on the server. Not sure why syntaxhighlight does not work, but I am glad that source does. Removing support for the source element (regardless of whether it is officially deprecated or not), will remove support for the SyntaxHighlight Extension, at least for some wikis, so this should be avoided at all costs. It doesn't seem harmful to keep support for a redundant tag, especially if removing support will break the extension on some wikis. Nicole Sharp (talk) 13:47, 14 March 2021 (UTC)

I currently only plan to do LTS upgrades of MediaWiki, so at the very least a legacy or LTS version of SyntaxHighlight should be maintained to keep support for the source element. Nicole Sharp (talk) 13:53, 14 March 2021 (UTC)

I solved the problem. syntaxhighlight does not work due to my custom stylesheet at MediaWiki:vector.css, likely due to forcing the text color as black for the pre element. The reason that the source element works and the syntaxhighlight element does not may specifically be because the source element is deprecated, and thus not affected by the Vector stylesheet. All the more reason to keep a legacy element supported, so that users have an option to keep the SyntaxHighlight formatting even when custom stylesheets for the Vector Skin are in use. Nicole Sharp (talk) 14:07, 14 March 2021 (UTC)

"source" is also easier to type and read than "syntaxhighlight" since it has less characters and syllables. Nicole Sharp (talk) 14:10, 14 March 2021 (UTC)

See phab:T237267 for the rationale for deprecating source, and phab:T251116 for suggestions for a new shorthand or alternative to syntaxhighlight. In particular, source is also the name of an unrelated HTML element, and in general, wikitext prefers not to reuse the names of unrelated markup from internet standards. Local styles conflicting with features of core or extensions aren't going to be a strong enough argument for changing those features, or for preserving deprecated functionality, since the fix is to simply remove or change the local styles. 「ディノ奴千？！」^{☎ Dinoguy1000} 19:37, 14 March 2021 (UTC)

SyntaxHighlight on template

Is there a way to apply SyntaxHighlight to content from a MediaWiki template page? Example here when using the SyntaxHighlightPages Extension, so that the template page is already formatted using the syntaxhighlight page content model. The syntaxhighlight element will otherwise strip the wikimarkup so that the template content does not display.

<syntaxhighlight lang="html">
{{example.htm}}
</syntaxhighlight>

Nicole Sharp (talk) 22:00, 14 March 2021 (UTC)

Is there a way to place "<syntaxhighlight>" within a syntaxhighlight element? I tried using nowiki but SyntaxHighlight ignores that and displays "nowiki" instead. Nicole Sharp (talk) 22:21, 14 March 2021 (UTC)

Try Special:permalink/4461333. IKhitron (talk) 01:05, 15 March 2021 (UTC)

Shared hosting issue (OVH)

Hello,

I was using this extension in the past (1.27) but trying to reload a backup to the latest version of mediawiki (1.35.1) I get error. My hosting provider is OVH with a pro offer. I have shell acces over SSH and when navigating to the extension folder, I can run ./pigmentize --help and get the information about it. But when calling it from mediawiki it does not work. I get the following error message :

Invoking Pygments returned blank output with no error response

I have tried the following :

• moving the pygmentize to ./cgi-bin folder setting $wgPygmentizePath accordingly;
• download/reupload the extension, setting binary transfer

Any idea on what else I should try? Symac (talk) 05:37, 16 March 2021 (UTC)

how do disable the "pages with syntax highlighting errors" tracking category?

How can I disable the "pages with syntax highlighting errors" tracking category? Not sure why, but I am experiencing a bug where this tracking category displays for the Pygments language of mma, though admittedly I am using mma incorrectly for Maxima syntax instead of Wolfram Mathematica syntax.

e:limit((1+n^-1)^n,n,inf);
i:sqrt(-1);
pi:integrate((1+x^2)^-1,x,-inf,inf);

Nicole Sharp (talk) 06:14, 27 March 2021 (UTC)

I am not aware that this category is triggered when invalid syntax is occurring in code.

• Otherwise this topic here would be listed in Category:Pages with syntax highlighting errors right now.
• I understood that it will be triggered on missing or unknown lang= attribute value.

The category is configured via MediaWiki:Syntaxhighlight-error-category system message.

• I would not recommend to drop the entire category.
• Usually they are sensitive for template programming.
• Therefore you can evaluate by {{#switch: {{FULLPAGENAME}} |=}} the current page and whitelist some exceptional pages, by assigning - and category title respectively.

Anyway, you might mark the category as __HIDDENCAT__ which will show categorization to registered users only, who opt-in to see such hidden maintenance categorizations. PerfektesChaos (talk) 13:52, 27 March 2021 (UTC)

• Otherwise this topic here would be listed in Category:Pages with syntax highlighting errors right now.

I don’t think so—Flow doesn’t play nice with advanced wikitext features like categories—or syntax highlighting itself.

<!-- this should be highlighted as an HTML comment, yet it’s plain black -->

This doesn’t have any language at all, so if Flow worked properly, it ended up in the category.

Tacsipacsi (talk) 22:51, 27 March 2021 (UTC)

I didn't know you could hide categories. That seems to be an adequate solution to keep the category from public view. I am not sure why it is being triggered though since the extension is being used correctly and not meeting the criteria for syntax highlighting errors. It seems to be a problem with the server or wiki configuration since it is affecting all Pygments languages, not just mma. I am switching over to use Highlightjs_Integration instead though, since HighlightJS supports Maxima and Pygments does not. Nicole Sharp (talk) 14:22, 27 March 2021 (UTC)

Unfortunately, the hidden categories aren't hidden very well, and still show up under "special:categories." Is there a way to disable the SyntaxHighlight tracking categories? Nicole Sharp (talk) 03:12, 28 March 2021 (UTC)

Found an alternative solution. Since something in either the wiki or server configuration is causing every use of syntaxhighlight to be tracked with the category "pages with syntax highlighting errors," regardless of correct or incorrect use of SyntaxHighlight, I used MediaWiki:syntaxhighlight-error-category to change the name of the tracking category to "pages with syntax highlighting." Since there are no errors, at least the categorization is correct now. However, this doesn't solve the underlying problem in that the extension seems to be broken. In addition to thinking the syntax has errors, the highlighting itself doesn't display. Can't figure out why but seems better/easier to switch over to HighlightJS instead. Nicole Sharp (talk) 03:34, 28 March 2021 (UTC)

Installing the Highlightjs_Integration extension (while keeping SyntaxHighlight enabled) removed the (incorrect) SyntaxHighlight tracking category, so the two extensions seem to work well together. Unless a custom element (such as "highlight" instead of "syntaxhighlight") is defined for Highlightjs_Integration, the default behavior is for Highlightjs_Integration to take priority for highlighting over SyntaxHighlight, which avoids conflicts (and any SyntaxHighlight errors for HighlightJS languages not available in Pygments). Nicole Sharp (talk) 04:23, 28 March 2021 (UTC)

Incompatible with hardened PHP installation

Hi Everyone,

This looks like a very useful extension. I hope we can get it working with our installation.

We run a hardened web server at www.cryptopp.com. We remove a bunch of unsafe functions using PHP ini file via disable_functions. You can find the hardened settings here: https://github.com/weidai11/website/blob/master/install/security.ini.

When we attempt to enable SyntaxHighlight in LocalSettings.php, this is the result:

Fatal error: Uncaught ExtensionDependencyError: SyntaxHighlight requires "shell" ability:
Unable to run external programs, proc_open() is disabled in /var/www/html/w/includes/registration/ExtensionRegistry.php:407
Stack trace:
#0 /var/www/html/w/includes/registration/ExtensionRegistry.php(231): ExtensionRegistry->readFromQueue()
#1 /var/www/html/w/includes/Setup.php(161): ExtensionRegistry->loadFromQueue()
#2 /var/www/html/w/includes/WebStart.php(89): require_once('/var/www/html/w...')
#3 /var/www/html/w/index.php(44): require('/var/www/html/w...')
#4 {main} thrown in /var/www/html/w/includes/registration/ExtensionRegistry.php on line 407

PHP does not allow us to whitelist one extension.

My request is, it would be nice if SyntaxHighlight had its own pretty print code so it can handle syntax highlighting within the sandbox that PHP is restricted to.

Thanks in advance. Noloader (talk) 19:37, 11 April 2021 (UTC)

I don’t think it’s a realistic wish, this extension relies on an external program exactly to avoid reinventing the wheel. If it had its own built-in highlighter, then the external one would not be necessary at all, but it would be a large maintenance burden. You can try using the old MediaWiki 1.24 version, which uses GeSHi, written in plain PHP, but I’m sure it won’t work out of the box with modern MW versions. Also, it was for a reason that it was replaced, long-unmaintained software is probably not the best choice if security is an important factor… Tacsipacsi (talk) 21:00, 11 April 2021 (UTC)

Thanks @Tacsipacsi.

As much as I hate to suggest it... Can you consider a PHP compatible library and install it via Composer? Instead of using banned functions and external programs, you would use the PHP library. Or make it a configuration option (external program vs PHP library). That should meet your needs and the needs of folks who wish to harden their installation.

Maybe something like one of these:

• https://github.com/scrivo/highlight.php
• https://github.com/Automattic/syntaxhighlighter
• https://github.com/GeSHi/geshi-1.0

Also see "php" syntax highlighter site:github.com.

Noloader (talk) 01:17, 13 April 2021 (UTC)

Oh, check this out @Tacsipacsi...

https://github.com/ramsey/pygments. It is an actively maintained PHP Wrapper for the Python Pygments.

How difficult would it be to use the PHP bindings rather than shelling out? Noloader (talk) 02:00, 13 April 2021 (UTC)

Probably not that difficult, but at least entirely useless—ramsey/pygments also shells out (through symfony/process). I don’t think it’s possible to run any non-PHP program if you don’t want to allow shelling out; how would you do that? Tacsipacsi (talk) 12:21, 13 April 2021 (UTC)

Thanks @Tacsipacsi.

My bad. I did not realize it was shelling out, too.

It looks like there's an API at https://pygments.org/docs/api/. Using the API does not look too difficult: https://pygments.org/docs/quickstart/#example.

Do you know if PHP can call the Python API? It seems like PHP should be able to do it. (I'm a C/C++/ObjC guy. I don't know PHP).

Thanks in advance. Noloader (talk) 18:42, 13 April 2021 (UTC)

I haven’t tried interacting with non-PHP code from PHP ever except for with proc_open(), so I don’t have any experience, but I wouldn’t hope it can directly interact with Python. Maybe PHP could interact with C and C with Python, as both Zend and CPython interpreters are written in C, but that’s quite a number of hops (and it’s complicated also by the fact that C should be compiled for each platform individually).

However, it just came into my mind that there’s a work in progress to containerize shell execution; maybe it’s acceptable for you security-wise to allow running arbitrary code in an isolated container. This is a priority for Wikimedia as well, so hopefully sooner or later it will be implemented for this extension, too. Tacsipacsi (talk) 22:48, 13 April 2021 (UTC)

Syntax-highlight text from a template?

I want to have a template that contains precisely the source file to be displayed, and to drop it into another page syntax-highlighted.

Conceptually, what I want is something like

<syntaxhighlight> {{msgnw:MySourceFile}} </syntaxhighlight>

but of course that doesn't expand the template.

{{Pre|{{msgnw:MySourceFile}}}} sort of works, but it's not quite the same formatting as <syntaxhighlight>.

Is there a way? Jordan Brown (talk) 01:45, 22 April 2021 (UTC)

If you haven't already, try {{ #tag: syntaxhighlight | {{MySourceFile}} |lang="lang" }}. 「ディノ奴千？！」^{☎ Dinoguy1000} 02:36, 22 April 2021 (UTC)

Thanks but, alas, turns a semicolon into &#59;.

Precisely what I tried - itself inside a template - is

{{ #tag: syntaxhighlight | {{msgnw:User:Jordan Brown/sandbox/{{{name}}}}}|lang="C" }}

where the page pointed at contains cube(10);.

But I didn't know anything about #tag, so I'll look into it further. Jordan Brown (talk) 02:55, 22 April 2021 (UTC)

You might have luck without the msgnw (I haven't tested this myself):

{{ #tag: syntaxhighlight | {{:User:Jordan Brown/sandbox/{{{name}}}}}|lang="C" }}

「ディノ奴千？！」^{☎ Dinoguy1000} 07:53, 22 April 2021 (UTC)

No, didn't work. With

{{ #tag: syntaxhighlight | {{{{BOOKNAME}}/examples/{{{name}}}}}|lang="C"|line="line" }}

and the input being (a deliberate torture test)

cube(10);|| foo && bar {{baz}} <pre></pre> &lt;
x < 5 && y > 6

the result was

cube(10);|| foo && bar [[:Template:Baz]] ?'"`UNIQ--pre-00000000-QINU`"'? <
x < 5 && y > 6

Thanks for your help. Other editors on the book have looked at it and decided that my idea of keeping the examples in separate pages would be too awkward, so I'm abandoning the effort for now. Jordan Brown (talk) 16:17, 22 April 2021 (UTC)

If you see `UNIQ--pre-*-QINU` stripped tags, this is because the transcluded content was parsed by MediaWiki. The only safe way to transclude a content not written in the Mediawiki syntax and that must not be parsed by it, is to use the "msgnw:" prefix before the transcluded full pagename, then you can put it inside a "#tag:" which allows the raw transclusion (via "msgnw:") or the parsed transclusion (without "msgnw:") to work.

However, it should be possible to use a simpler approach with <syntaxhighlight page="fullpagename" />, without even having to set its lang="*" attribute, because every page has metadata specifying its content-type (aka "page content model"): the syntaxhighlight extension should know every supported "content model" of MediaWiki, and notably MediaWiki, JSON, CSS, Javascript, XML, Lua, config/INI files, CSV files (the variant defined by the most common RFC using ASCII commas as field delimiters, ASCII double-quotes for escapes, and numbers in English format using dots and not commas, or otherwise escaping numbers within quotes, and non-mandatory quotation marks surrounding fields, and whitespaces between fields not significant, but where newlines may be present within fields escaped by quotation marks within which whitespaces are significant), and plain text, that should all be mappable to a language code/name supported by a Pygment parser. Verdy p (talk) 11:24, 5 September 2023 (UTC)

how to batch replace <source> with <syntaxhighlight>

There are several options: 1) use a (semi)automated editing tool such as AutoWikiBrowser or Pywikibot; 2) Extension:ReplaceText; or 3) use a tool such as sed to edit the raw article text of a SQL database dump, then re-import from that SQL dump. The first two can be done on a live site; the third would require special care (such as setting $wgReadOnly) to ensure no user edits are lost in the interim.

Hi, I am new to mediawiki. But I have to upgrade an old mediawiki that contains lots of page with source tags. Since it's deprecated by syntaxhighlight tag, how do I replace all source tags in all pages with syntaxhighlight tag in a batch way? Hawk914 (talk) 03:09, 1 June 2021 (UTC)

The way I'd do it is via AutoWikiBrowser (AWB) or a similar (semi)automated editing tool (if you're handy with a programming language and don't mind spending time on it, you could also write such a tool yourself). If you can install extensions yourself, or can request to have them installed, another option is Extension:ReplaceText , though personally I don't recommend this versus running AWB or similar. 「ディノ奴千？！」^{☎ Dinoguy1000} 05:53, 1 June 2021 (UTC)

Another possibility for linux is the pywikibot replace script (see Manual:Pywikibot/replace.py) with the following user-fix.py entry:

fixes['source_tag'] = {
    'regex': False,
    'msg': {
        '_default': 'source-tag ersetzt durch syntaxhighlight',
    },
    'replacements': [
        (r'<source', '<syntaxhighlight'),
        (r'</source', '</syntaxhighlight'),
    ]
}

called like this:

python3 pwb.py replace -fix:source_tag -start:!

2001:A61:BDD:901:C911:EA05:A98:C376 (talk) 14:21, 21 July 2021 (UTC)

that worked for me

cat wiki.sql | sed 's/<source/<syntaxhighlight/g' | sed 's/<\/source/<\/syntaxhighlight/g' > wiki.new.sql

make sure to backup and check thoroughly after importing back Mustakioren (talk) 05:50, 23 August 2021 (UTC)

Linter Error on 1.36.0 ?

Excuse me. On my local MediaWIki, when I install SyntaxHighlighter, I get an error message when displaying Gadget:

Notice: fwrite(): write of 7804 bytes failed with errno=32 Broken pipe in D:\mediawiki-1.36.0\vendor\wikimedia\shellbox\src\Command\UnboxedExecutor.php on line 315

When I save this script, the gadget does not work, but it does work if I remove SyntaxHighlighter from LocalSettings.php beforehand and save it.

Can anyone tell me what is causing this? Pneuma01 (talk) 01:44, 12 June 2021 (UTC)

I'm getting the same error on my new install of 1.36.0

Seems to be looking for the OS type, but not sure how to fix it. 63.245.160.132 (talk) 13:50, 17 June 2021 (UTC)

I "fix" the problem with this code on line 315 : $res = @fwrite( $pipe, $buffers[$fd], 65536 ); 143.196.163.1 (talk) 09:58, 23 September 2021 (UTC)

I don't think that really fixes anything, that just hides/silences the error message? But the error is still there. I am getting the same error now on 1.36.2 after upgrading from 1.35.3. Lwangaman (talk) 10:24, 10 October 2021 (UTC)

The code field is a keyboard trap

I noticed that it is impossible to get out of the code entry field when you have to rely on keyboard navigation. (Mediawiki 1.31). Would it be hard to fix this? MLRodrigue (talk) 09:51, 26 July 2021 (UTC)

The issue is that the tab key is used to indent/outdent the code. I don’t think it would be hard to change this behavior to navigating between form fields, but then there was no way to change the indentation (except for copy-pasting the tabulator character or entering it by ASCII/Unicode codepoint, neither of them is convenient). So there’s no good solution for this; the developers seem to have chosen a solution that’s better for those who prefer indenting their code with tabs but worse for those who rely on keyboard navigation. Tacsipacsi (talk) 14:51, 26 July 2021 (UTC)

Could Shift+Tab be adapted for this purpose? AFAIK it has no special function within the field, and it also doesn't in the browser or OS (Ctrl+Tab switches between open tabs in Firefox and (IIRC) Chrome, and Alt+Tab switches between open program windows in Windows). 「ディノ奴千？！」^{☎ Dinoguy1000} 18:50, 26 July 2021 (UTC)

It has: that’s the outdent. (Shift, as usual, reverses the direction: here it’s outdent instead of indent, if it wasn’t overridden, it would go to the previous form field instead of the next one, Ctrl+Shift+Tab goes to the previous tab instead of the next one, Alt+Shift+Tab goes to the least recently used window instead of the most recently used one, and so on.) Tacsipacsi (talk) 19:10, 26 July 2021 (UTC)

Aah, right, I always forget about outdenting for some reason. So much for that idea, then. 「ディノ奴千？！」^{☎ Dinoguy1000} 19:15, 26 July 2021 (UTC)

Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted [Called from SyntaxHighlight::highlightInner

With MediaWiki 1.35.3, I was able to get the SyntaxHighlight extension to work by using an external pygmentize:

$wgPygmentizePath = '/usr/local/bin/pygmentize';

However after upgrading to MediaWiki 1.36.2, pygmentize is no longer working correctly. If I turn on error reporting:

error_reporting(E_ALL);
ini_set("display_errors", 1);

$wgShowExceptionDetails = true;
$wgShowDBErrorBacktrace = false;
$wgShowSQLErrors = true;
$wgResourceLoaderDebug = false;
$wgDebugToolbar = true;
$wgDevelopmentWarnings = true;
$wgShowDebug = true;

I then get these errors:

Notice: Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted [Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316] in /path/to/mediawiki_1_36/w/includes/debug/MWDebug.php on line 499

Same or similiar problem for lilypond:

Unable to obtain LilyPond version:
Error clone: main.c:2608 main: Operation not permitted

In the debug data I see the following information:

[exec] Executing: '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
[exec] Error running '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': Error clone: main.c:2608 main: Operation not permitted
[objectcache] fetchOrRegenerate(global:highlight:f3eb16b7c32d4a36f595352942cb4e1b): miss, new value computed
[error] [YWLA1o0mAVG7arGqGzx1GAAAAA4] /wiki/User_talk:Johnrdorazio ErrorException: PHP Notice: Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted
[Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316]
[warning] Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted
[Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316]

Lwangaman (talk) 10:47, 10 October 2021 (UTC)

Does MediaWiki 1.36 require the usage of a local shellbox proxy? Or is that optional? I thought I would try adapting the instructions for setting up MediaWiki Shellbox to work in my Plesk setup, but I haven't been successful yet, so I've kept these two lines commented out:

//$wgShellboxUrl = 'http://shellbox.internal/shellbox';
//$wgShellboxSecretKey = 'random_generated_key_here';

But I still can't quite figure out if this is required by MediaWiki 1.36 or simply supported by MediaWiki 1.36. Lwangaman (talk) 10:53, 10 October 2021 (UTC)

Running

/usr/local/bin/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8 < /dev/null

on the command line outputs:

<div class="mw-highlight"><pre><span></span>
</pre></div>

Same goes for

/path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/pygments/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8 < /dev/null

. So they seem to work from the command line. Seems to me the Shell::Command is not working correctly to invoke pygmentize, or lilypond, or perhaps any shell command? Lwangaman (talk) 11:01, 10 October 2021 (UTC)

I'm running MediaWiki/Shellbox 1.0.4, so I don't believe that Phabricator T274474 is the issue.

To further test this, I am attempting to run the same pygmentize under firejail from the command line. I'm explicitly whitelisting limit.sh when running from the command line, and this kind of seems to work:

firejail --profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile --blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php --whitelist=/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh --noroot --seccomp --private-dev --net=none -- /bin/bash /path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh '/usr/local/bin/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8,linenos=inline < /dev/null' SB_INCLUDE_STDERR=; SB_CPU_LIMIT=180; SB_CGROUP=''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes

This produces:

Reading profile /path/to/mediawiki_1_36/w/includes/shell/firejail.profile
Parent pid 3252769, child pid 3252770
Child process initialized in 24.94 ms
<div class="mw-highlight"><pre><span></span><span class="linenos">1</span>
</pre></div>

Parent is shutting down, bye...

However it only works if I add that " < /dev/null". Without the "< /dev/null" the process just hangs. Is it possible that this is missing from MediaWiki Shell Command? Lwangaman (talk) 12:26, 10 October 2021 (UTC)

I tried editing wikimedia/shellbox/src/Command/BashWrapper.php like this:

        if ( $time > 0 || $mem > 0 || $filesize > 0 || $wallTime > 0 ) {
            $cmd = '/bin/bash ' . escapeshellarg( __DIR__ . '/limit.sh' ) . ' ' .
                escapeshellarg( $command->getCommandString() . ' < /dev/null' ) . ' ' .
                escapeshellarg(
                    "SB_INCLUDE_STDERR=" . ( $command->getIncludeStderr() ? '1' : '' ) . ';' .
                    "SB_CPU_LIMIT=$time; " .
                    'SB_CGROUP=' . escapeshellarg( $this->cgroup ) . '; ' .
                    "SB_MEM_LIMIT=$mem; " .
                    "SB_FILE_SIZE_LIMIT=$filesize; " .
                    "SB_WALL_CLOCK_LIMIT=$wallTime; " .
                    "SB_USE_LOG_PIPE=yes"
                );
            $command->unsafeCommand( $cmd )
                ->useLogPipe();
            if ( $command->getAllowedPaths() ) {
                // If specific paths have been allowed, make sure we explicitly
                // allow limit.sh. We don't do this unconditionally because it
                // doesn't work as expected in firejail, see T274474, T182486
                $command->allowPath( __DIR__ . '/limit.sh' );
            }
        }

Notice the extra . ' < /dev/null' after $command->getCommandString(). However this still doesn't help much, I'm getting the same error as before:

[exec] Executing: '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\'' < /dev/null' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
[exec] Error running '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\'' < /dev/null' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': Error clone: main.c:2608 main: Operation not permitted

Lwangaman (talk) 12:44, 10 October 2021 (UTC)

I see others have had a similar issue when running in a docker container (see https://github.com/netblue30/firejail/issues/1856), however I'm not running in a docker container. To make sure permissions are not an issue, I did a 'sudo su - mediawikiApacheUser' and then issued the same pygmentize under firejail from the command line, this time as the mediawiki apache user, and got the same result as above without any permission errors. Lwangaman (talk) 12:50, 10 October 2021 (UTC)

I see someone else mentioned something about an old kernel being the issue, with missing NAMESPACE support.

My current kernel, on Ubuntu 20.04, is 5.4.0-73-generic. If I cd /usr/src/linux-headers-5.4.0-73-generic and egrep '_NS' .config, I seem to have all of the necessary NAMESPACE support:

CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NET_NSH=m
CONFIG_PATA_NS87415=m
CONFIG_PATA_NS87410=m
CONFIG_NS83820=m
CONFIG_GAMEPORT_NS558=m
CONFIG_TCG_NSC=m

So that doesn't seem to me to be the issue... Lwangaman (talk) 13:00, 10 October 2021 (UTC)

_{(edit conflict)}×3

If you don’t add < /dev/null, the process tries to read from the standard input, i.e. it waits for you to type something. Press Ctrl+D to tell it you’re done with typing.

I’m not really familiar with Firejail, but doesn’t its behavior depend on the user executing it? By using

sudo -u www-data firejail ...

instead of

firejail ...

you should become closer to what actually happens when MediaWiki tries to call Pygmentize (assuming you use Apache, different web servers may use different users). Tacsipacsi (talk) 14:31, 10 October 2021 (UTC)

In fact I did give that a try, as mentioned here. Seems to be working ok, without any particular errors... So I'm not sure why the MediaWiki Shellbox::Command is producing Error clone: main.c:2608 main: Operation not permitted. Lwangaman (talk) 15:19, 10 October 2021 (UTC)

For the time being I have just set $wgShellRestrictionMethod to false, rather than to 'firejail'. At least Lilypond and Pygments are working again. It would be nice for it to work with firejail... Lwangaman (talk) 17:54, 12 October 2021 (UTC)

Is it possible to use HTML tags inside the <syntaxhighlight></syntaxhighlight> tags?

I'm dealing with a programming language that's based on Delphi but has some differences.

There are some keywords that do not exist in Delphi, so they won't be colored and highlighted the proper way.

Is it somehow possible to use the HTML tags like <b> or <i> inside the <syntaxhighlight lang=delphi line></syntaxhighlight> blocks?

I would appreciate the help. MrJConstantine (talk) 12:35, 16 November 2021 (UTC)

No, because ‎<syntaxhighlight> is a ‎<nowiki>‑like tag. — ExE Boss (talk) 13:08, 16 November 2021 (UTC)

You would need to extend the upstream variety of offered languages. However, perhaps there is already another Delphi derivative which will fit your needs? PerfektesChaos (talk) 13:14, 16 November 2021 (UTC)

Actually, no. It's our internal system and it is not officially available anywhere except our company. MrJConstantine (talk) 19:59, 16 November 2021 (UTC)

It's possible to work around this by using #tag, e.g. {{ #tag: syntaxhighlight | sample code here with a bit of <b>bolding</b> |lang=delphi |line }}, but it's likely to be pretty finicky (and you're not likely to get much help working around that finickiness). PerfektesChaos' suggestions would be better. 「ディノ奴千？！」^{☎ Dinoguy1000} 16:09, 16 November 2021 (UTC)

I've tried it like this but it doesn't work for me.

I get the following result:

<b>componentlist</b>

MrJConstantine (talk) 20:10, 16 November 2021 (UTC)

Pygments (the library which does the actual highlighting) is open source, so you can write your own lexer based on the Delphi/Pascal one. I haven’t tried doing so, so I don’t know how complicated it is (especially installing the modified Pygments), but it should be possible. Tacsipacsi (talk) 20:56, 16 November 2021 (UTC)