Extension talk:SyntaxHighlight

• Extension talk:SyntaxHighlight/REL1.25 (LTS)

Previous discussion was archived at Extension talk:SyntaxHighlight/Archive 2017 on 2017-03-29.

Hi Everyone,

This looks like a very useful extension. I hope we can get it working with our installation.

We run a hardened web server at www.cryptopp.com. We remove a bunch of unsafe functions using PHP ini file via disable_functions. You can find the hardened settings here: https://github.com/weidai11/website/blob/master/install/security.ini.

When we attempt to enable SyntaxHighlight in LocalSettings.php, this is the result:

Fatal error: Uncaught ExtensionDependencyError: SyntaxHighlight requires "shell" ability:
Unable to run external programs, proc_open() is disabled in /var/www/html/w/includes/registration/ExtensionRegistry.php:407
Stack trace:
#0 /var/www/html/w/includes/registration/ExtensionRegistry.php(231): ExtensionRegistry->readFromQueue()
#1 /var/www/html/w/includes/Setup.php(161): ExtensionRegistry->loadFromQueue()
#2 /var/www/html/w/includes/WebStart.php(89): require_once('/var/www/html/w...')
#3 /var/www/html/w/index.php(44): require('/var/www/html/w...')
#4 {main} thrown in /var/www/html/w/includes/registration/ExtensionRegistry.php on line 407

PHP does not allow us to whitelist one extension.

My request is, it would be nice if SyntaxHighlight had its own pretty print code so it can handle syntax highlighting within the sandbox that PHP is restricted to.

Thanks in advance.

I don’t think it’s a realistic wish, this extension relies on an external program exactly to avoid reinventing the wheel. If it had its own built-in highlighter, then the external one would not be necessary at all, but it would be a large maintenance burden. You can try using the old MediaWiki 1.24 version, which uses GeSHi, written in plain PHP, but I’m sure it won’t work out of the box with modern MW versions. Also, it was for a reason that it was replaced, long-unmaintained software is probably not the best choice if security is an important factor…

Thanks @Tacsipacsi.

As much as I hate to suggest it... Can you consider a PHP compatible library and install it via Composer? Instead of using banned functions and external programs, you would use the PHP library. Or make it a configuration option (external program vs PHP library). That should meet your needs and the needs of folks who wish to harden their installation.

Maybe something like one of these:

• https://github.com/scrivo/highlight.php
• https://github.com/Automattic/syntaxhighlighter
• https://github.com/GeSHi/geshi-1.0

Also see "php" syntax highlighter site:github.com.

Noloader (talk) 01:17, 13 April 2021 (UTC)

Oh, check this out @Tacsipacsi...

https://github.com/ramsey/pygments. It is an actively maintained PHP Wrapper for the Python Pygments.

How difficult would it be to use the PHP bindings rather than shelling out?

Probably not that difficult, but at least entirely useless—ramsey/pygments also shells out (through symfony/process). I don’t think it’s possible to run any non-PHP program if you don’t want to allow shelling out; how would you do that?

Thanks @Tacsipacsi.

My bad. I did not realize it was shelling out, too.

It looks like there's an API at https://pygments.org/docs/api/. Using the API does not look too difficult: https://pygments.org/docs/quickstart/#example.

Do you know if PHP can call the Python API? It seems like PHP should be able to do it. (I'm a C/C++/ObjC guy. I don't know PHP).

Thanks in advance.

I haven’t tried interacting with non-PHP code from PHP ever except for with proc_open(), so I don’t have any experience, but I wouldn’t hope it can directly interact with Python. Maybe PHP could interact with C and C with Python, as both Zend and CPython interpreters are written in C, but that’s quite a number of hops (and it’s complicated also by the fact that C should be compiled for each platform individually).

However, it just came into my mind that there’s a work in progress to containerize shell execution; maybe it’s acceptable for you security-wise to allow running arbitrary code in an isolated container. This is a priority for Wikimedia as well, so hopefully sooner or later it will be implemented for this extension, too.

How can I disable the "pages with syntax highlighting errors" tracking category? Not sure why, but I am experiencing a bug where this tracking category displays for the Pygments language of mma, though admittedly I am using mma incorrectly for Maxima syntax instead of Wolfram Mathematica syntax.

e:limit((1+n^-1)^n,n,inf);
i:sqrt(-1);
pi:integrate((1+x^2)^-1,x,-inf,inf);

Nicole Sharp (talk) 06:14, 27 March 2021 (UTC)

I am not aware that this category is triggered when invalid syntax is occurring in code.

• Otherwise this topic here would be listed in Category:Pages with syntax highlighting errors right now.
• I understood that it will be triggered on missing or unknown lang= attribute value.

The category is configured via MediaWiki:Syntaxhighlight-error-category system message.

• I would not recommend to drop the entire category.
• Usually they are sensitive for template programming.
• Therefore you can evaluate by {{#switch: {{FULLPAGENAME}} |=}} the current page and whitelist some exceptional pages, by assigning - and category title respectively.

Anyway, you might mark the category as __HIDDENCAT__ which will show categorization to registered users only, who opt-in to see such hidden maintenance categorizations.

• Otherwise this topic here would be listed in Category:Pages with syntax highlighting errors right now.

I don’t think so—Flow doesn’t play nice with advanced wikitext features like categories—or syntax highlighting itself.

<!-- this should be highlighted as an HTML comment, yet it’s plain black -->

This doesn’t have any language at all, so if Flow worked properly, it ended up in the category.

I didn't know you could hide categories. That seems to be an adequate solution to keep the category from public view. I am not sure why it is being triggered though since the extension is being used correctly and not meeting the criteria for syntax highlighting errors. It seems to be a problem with the server or wiki configuration since it is affecting all Pygments languages, not just mma. I am switching over to use Highlightjs_Integration instead though, since HighlightJS supports Maxima and Pygments does not. Nicole Sharp (talk) 14:22, 27 March 2021 (UTC)

Unfortunately, the hidden categories aren't hidden very well, and still show up under "special:categories." Is there a way to disable the SyntaxHighlight tracking categories? Nicole Sharp (talk) 03:12, 28 March 2021 (UTC)

Found an alternative solution. Since something in either the wiki or server configuration is causing every use of syntaxhighlight to be tracked with the category "pages with syntax highlighting errors," regardless of correct or incorrect use of SyntaxHighlight, I used MediaWiki:syntaxhighlight-error-category to change the name of the tracking category to "pages with syntax highlighting." Since there are no errors, at least the categorization is correct now. However, this doesn't solve the underlying problem in that the extension seems to be broken. In addition to thinking the syntax has errors, the highlighting itself doesn't display. Can't figure out why but seems better/easier to switch over to HighlightJS instead. Nicole Sharp (talk) 03:34, 28 March 2021 (UTC)

Installing the Highlightjs_Integration extension (while keeping SyntaxHighlight enabled) removed the (incorrect) SyntaxHighlight tracking category, so the two extensions seem to work well together. Unless a custom element (such as "highlight" instead of "syntaxhighlight") is defined for Highlightjs_Integration, the default behavior is for Highlightjs_Integration to take priority for highlighting over SyntaxHighlight, which avoids conflicts (and any SyntaxHighlight errors for HighlightJS languages not available in Pygments). Nicole Sharp (talk) 04:23, 28 March 2021 (UTC)

Hello,

I was using this extension in the past (1.27) but trying to reload a backup to the latest version of mediawiki (1.35.1) I get error. My hosting provider is OVH with a pro offer. I have shell acces over SSH and when navigating to the extension folder, I can run ./pigmentize --help and get the information about it. But when calling it from mediawiki it does not work. I get the following error message :

Invoking Pygments returned blank output with no error response

I have tried the following :

• moving the pygmentize to ./cgi-bin folder setting $wgPygmentizePath accordingly;
• download/reupload the extension, setting binary transfer

Any idea on what else I should try?

Is there a way to apply SyntaxHighlight to content from a MediaWiki template page? Example here when using the SyntaxHighlightPages Extension, so that the template page is already formatted using the syntaxhighlight page content model. The syntaxhighlight element will otherwise strip the wikimarkup so that the template content does not display.

<syntaxhighlight lang="html">
{{example.htm}}
</syntaxhighlight>

Nicole Sharp (talk) 22:00, 14 March 2021 (UTC)

Is there a way to place "<syntaxhighlight>" within a syntaxhighlight element? I tried using nowiki but SyntaxHighlight ignores that and displays "nowiki" instead. Nicole Sharp (talk) 22:21, 14 March 2021 (UTC)

Try Special:permalink/4461333.

Not sure if this has already been posted, but for non-Wikimedia installations of MediaWiki 1.35.1 LTS (at least with DreamHost Shared Hosting), the syntaxhighlight element does not work but the source element does. This is very confusing since the extension page here on MediaWiki.org says that the source element is deprecated, though the README file included with the extension says to use the source element (which is how I figured out to get the syntax highlighting to work—otherwise I might not have guessed to try the supposedly deprecated source element). Nicole Sharp (talk) 03:23, 14 March 2021 (UTC)

Based on a quick skim of the page you linked, DreamHost is only a host service and doesn't provide MediaWiki installations for customers, meaning whatever wiki you were working on must have (for whatever reason) customized their copy of the SyntaxHighlight extension to use source instead of syntaxhighlight. Regardless, source is deprecated and support for it will be removed (probably in the REL1_37 branch once that gets cut). Whether individual wikis (or wiki farms) decide to change the extension to continue supporting source, or even to drop syntaxhighlight, is outside the purview of the extension's maintainers here.

I installed MediaWiki myself using default settings for MediaWiki and SyntaxHighlight: https://www.nicolesharp.net/wiki/special:version. The only reason I mentioned the webhost is if the error has anything to do with the configurations of PHP or Python on the server. Not sure why syntaxhighlight does not work, but I am glad that source does. Removing support for the source element (regardless of whether it is officially deprecated or not), will remove support for the SyntaxHighlight Extension, at least for some wikis, so this should be avoided at all costs. It doesn't seem harmful to keep support for a redundant tag, especially if removing support will break the extension on some wikis. Nicole Sharp (talk) 13:47, 14 March 2021 (UTC)

I currently only plan to do LTS upgrades of MediaWiki, so at the very least a legacy or LTS version of SyntaxHighlight should be maintained to keep support for the source element. Nicole Sharp (talk) 13:53, 14 March 2021 (UTC)

I solved the problem. syntaxhighlight does not work due to my custom stylesheet at MediaWiki:vector.css, likely due to forcing the text color as black for the pre element. The reason that the source element works and the syntaxhighlight element does not may specifically be because the source element is deprecated, and thus not affected by the Vector stylesheet. All the more reason to keep a legacy element supported, so that users have an option to keep the SyntaxHighlight formatting even when custom stylesheets for the Vector Skin are in use. Nicole Sharp (talk) 14:07, 14 March 2021 (UTC)

"source" is also easier to type and read than "syntaxhighlight" since it has less characters and syllables. Nicole Sharp (talk) 14:10, 14 March 2021 (UTC)

See phab:T237267 for the rationale for deprecating source, and phab:T251116 for suggestions for a new shorthand or alternative to syntaxhighlight. In particular, source is also the name of an unrelated HTML element, and in general, wikitext prefers not to reuse the names of unrelated markup from internet standards. Local styles conflicting with features of core or extensions aren't going to be a strong enough argument for changing those features, or for preserving deprecated functionality, since the fix is to simply remove or change the local styles.

Feature is new in the forthcoming 1.36 release.

What setting or CSS is required to make line numbers appear on Module pages such as Module:Bananas or Wikipedia's Module:Example?

Our wiki is running 1.35.1, with up-to-date versions of Extension:Scribunto and Extension:SyntaxHighlight.

Our wiki has imported Wikipedia's Module:Example and the code is syntax highlighted as expected but there are no line numbers. Our MediaWiki:Common.css is copied from Wikipedia. We are trying to figure out what setting or CSS is required so that line numbering appears. Thanks in advance.

It's new code from last month. It will be released as part of the 1.36 branch process later this year, sorry.

@Jdforrester (WMF) Thanks. Now I can stop searching. The good news is I've learned quite a bit about the internals.

On my CentOS Stream release 8 testing environment with PHP 7.4.15 (cli) I made the suggested configuration of $wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua'; in my LocalSettings.php.

Then I ran php maintenance/update.phpand I get the error message:

PHP Warning:  Use of undefined constant CONTENT_MODEL_SCRIBUNTO - assumed 'CONTENT_MODEL_SCRIBUNTO' (this will throw an Error in a future version of PHP) in /var/www/html/w/LocalSettings.php

My edit to update the article with the address the PHP Warning was reverted, per this revision.

Which is correct, the PHP Warning or the revision?

Both. Pppery is right that the solution is not to replace it with a string, but PHP’s also right that you shouldn’t use an undefined constant in the first place. However, PHP’s assumption of what this constant should be is wrong (which is not surprising, since it’s just an assumption)—probably it should be the integer 828. Have you Extension:Scribunto installed at all? Without that, configuring syntax highlight for Scribunto modules makes no sense.

@Tacsipacsi @Pppery Yes Extension:Scribunto is installed and working on MediaWiki 1.35.1. I'm trying to get Syntax Highlighting functioning properly. The Installation and Configuration sections, plus the Talk pages give lots of helpful information but there still seems to be some dark arts to getting everything working. It doesn't help when the maintenance/update.php script throws a PHP Warning when following the configuration information for the extension.

Then maybe you load things in the wrong order? LocalSettings.php is executed from top to bottom, so Scribunto should have already been loaded at the point you want to use its constant.

This won’t work:

wfLoadExtension( 'SyntaxHighlight_GeSHi' );
$wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';

// ...

wfLoadExtension( 'Scribunto' );

But this should:

wfLoadExtension( 'Scribunto' );

// ...

wfLoadExtension( 'SyntaxHighlight_GeSHi' );
$wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';

@Tacsipacsi Per you suggestion I did change the order in LocalSettings.php. The PHP Warning message still appears however. My LocalSettings.php contains the following snippet:

wfLoadExtension( 'Scribunto' );

$wgScribuntoDefaultEngine = 'luastandalone';
// For a more pleasant user interface, with syntax highlighting and a code editor with autoindent
$wgScribuntoUseGeSHi = true;
$wgScribuntoUseCodeEditor = true;

wfLoadExtension( 'SyntaxHighlight_GeSHi' );
$wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';

FWIW Lua modules are working and <syntaxhighlight> tags are also working on the wiki.

Have you cleared your browser cache, restarted the web server etc.? You might even try to intentionally cause another warning in LocalSettings.php to make sure it’s not just stuck in the cache. If it isn’t a cache issue, then I have no idea why it doesn’t work. By the way, guessing from the name, $wgScribuntoUseGeSHi should enable syntax highlighting on its own, without the line causing the warning. Doesn’t that work?

I can confirm that using $wgScribuntoUseGeSHi = true; does work without also having $wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua';. Perhaps the configuration section should clarify how the two settings interact or which one to set.

As a testcase I commented out $wgScribuntoUseGeSHi = true; and then added $wgSyntaxHighlightModels[CONTENT_MODEL_SCRIBUNTO] = 'lua'; and I still get the PHP Warning.

I've checked and my extensions are up-to-date 1.35 versions. So the mystery remains as to what's causing the PHP Warning.

This plugin does not support modern PHP syntax:

$person = new class
{
    public $firstName = "John";
    public $lastName = "Smith";

    public function name(){
        return "$this->firstName $this->lastName";
    }
};

Thank you, but we cannot help that.

Update needs to be made upstream, by everybody, at: https://pygments.org/

The note at the top cautions users that the extensions is included after mediawiki 1.26 and so does not need to be downloaded, then says that the other instructions must still be followed. The installation instructions are missing, and are non-obvious. On the readme on github, it says the following under installation:

Add this line to your LocalSettings.php:

wfLoadExtension( 'SyntaxHighlight_GeSHi' );

Using mediawiki 1.35, including the above line worked for me. It should likely be included on this page, though I'm not sure of all of the correct formatting (or if there's a reason it isn't included).

@Mirinoth: It’s included in the Installation section, although that’s way down on the page (and a bit crowded) compared to other extensions’ simliar sections.

I tried for, I guess, more than a year, make it work, but it simply doesn't work, following all the possible available guides i found on the Internet. So I don't know if someone really made it work normally.

I'm using Windows, Download Mediawiki from Github, as same as Syntaxhighlight, followed all the possible instructions, but it doesn't work.

I replaced the syntax highlighter with this one.

Extension:Highlightjs Integration

It works but doesn't support some of the latest languages.

The languages it supports are:

• Apache
• Bash
• C#
• C++
• CSS
• CoffeeScript
• Diff
• HTML, XML
• HTTP
• Ini
• JSON
• Java
• JavaScript
• Makefile
• Markdown
• Nginx
• Objective-C
• PHP
• Perl
• Python
• Ruby
• SQL
• Shell Session