Extension talk:SyntaxHighlight

• Extension talk:SyntaxHighlight/REL1.25 (LTS)

Previous discussion was archived at Extension talk:SyntaxHighlight/Archive 2017 on 2017-03-29.

Hi,

I've installed Scribunto and then SyntaxHighlight_GeSHi on my local MediaWiki installation. All the extensions are the latest versions from Gerrit, and I'm using Ubuntu and MediaWiki-Docker.

I created a Scribunto module page. When I view, I see this error message at the top:

Notice: Failed to invoke Pygments: [Called from MediaWiki\SyntaxHighlight\SyntaxHighlight::highlightInner in /var/www/html/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 293] in /var/www/html/w/includes/debug/MWDebug.php on line 507

"pygmentize" appears to have the necessary permissions: -rwxrwxr-x  1 amir amir 1063938 May 25 15:15 pygmentize

When I simply use the "syntaxhighlight" tag on a page in the main space, the text is set in a monospace font, but without actual highlighting, and the page is auto-added to the "Pages with syntax highlighting errors" category.

I tried the "Troubleshooting" section here, but couldn't find something that will help. What could be wrong?

Do you have the Python3 executable 'python3' installed on the server and located on the PATH of the PHP interpreter?

See the note at Extension:SyntaxHighlight#Installation.

Oh, that's probably the problem. The docker image doesn't have Python. The solution is supposed to be something like MediaWiki-Docker/Extension/SyntaxHighlight, but it doesn't seem to work. When I run docker-compose build, towards the end I get this:

Status: Downloaded newer image for docker-registry.wikimedia.org/dev/stretch-php72-fpm:2.0.0
 ---> 7f47f1dab953
Step 2/2 : RUN apt update && apt install -y python3
 ---> Running in 5b103ed36b7b

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://security.debian.org stretch/updates InRelease [53.0 kB]
Ign:2 http://mirrors.wikimedia.org/debian stretch InRelease
Get:3 http://security.debian.org stretch/updates/main amd64 Packages [977 kB]
Ign:4 https://apt.wikimedia.org/wikimedia stretch-wikimedia InRelease
Get:5 http://mirrors.wikimedia.org/debian stretch-updates InRelease [93.6 kB]
Get:6 http://mirrors.wikimedia.org/debian stretch-backports InRelease [91.8 kB]
Err:7 https://apt.wikimedia.org/wikimedia stretch-wikimedia Release
  server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Get:8 http://mirrors.wikimedia.org/debian stretch Release [118 kB]
Get:9 http://mirrors.wikimedia.org/debian stretch-backports/main amd64 Packages [790 kB]
Get:10 http://mirrors.wikimedia.org/debian stretch-backports/contrib amd64 Packages [12.0 kB]
Get:11 http://mirrors.wikimedia.org/debian stretch Release.gpg [3177 B]
Get:12 http://mirrors.wikimedia.org/debian stretch/main amd64 Packages [9610 kB]
Reading package lists...
E: The repository 'https://apt.wikimedia.org/wikimedia stretch-wikimedia Release' does not have a Release file.
The command '/bin/sh -c apt update && apt install -y python3' returned a non-zero code: 100
ERROR: Service 'mediawiki' failed to build : Build failed

The instructions at MediaWiki-Docker/Extension/SyntaxHighlight probably need an update, but I know practically nothing about Docker and advanced Debian, so I can't do it myself. Any help?

Sorry, you said that you know "practically nothing" about Docker, and I know zero about it, or about installing MediaWiki with it. I have my test wiki running natively on a machine.

Seems to me that they really need to add python3 to the Docker image so that it works out of the box.

First, Debian Stretch is no longer supported on Wikimedia (when this page was created a year ago, it was still supported), so I think (but haven’t tried it out) that the dockerfile should rather be

FROM docker-registry.wikimedia.org/dev/buster-php72-fpm:2.0.0
RUN apt-get update && apt-get install -y python3

(Or buster-php72-fpm:2.0.0-s1, I don’t know which one is the newer. The second line’s change is just to silence APT’s warnings about the stable CLI interface.) APT’s HTTPS support has been steadily improving in the last years, so if you’re lucky, switching to Buster will automatically fix the actual issue (server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none).

Thanks @Tacsipacsi, this worked! It also required a reboot of docker—python3 appeared not installed at first, but after the reboot it worked.

Hello, I have multiple bugs with this extension, but I don’t know what to search on the internet or what did wrong…

The first time I use this extension everything goes well for the second time it goes wrong (per part of code).

So the first bug it's with visualEditor: when I install it for the first time it works but now the edit following the cursor (so it's very difficult to modify some code).

The second it’s syntax highlight with sql the first time I save it and everything works well. But I edit the code again and the highlight disappears… so I try with sources and the same first time it works second time it disappears…

It is really weird that the first time it works and seconde the bug appears and it still here. I try to change the skin and the bug is still here.

Here is my configuration of this extension.

wfLoadExtension('SyntaxHighlight_GeSHi');

$wgPygmentizePath = "c:\\Python27\\Scripts\\pygmentize.exe";

I install pygmentize with easy_install. Wikimedia 1.37.1 running on xampp 8.1.2 in localhost with a window 10.

Any help is apprised thank you!

Sorry, but using WYSIWYG for editation of the wikicode is the pure evil.

Sometimes I need to create internal links from inside the <syntaxhighlight ></syntaxhighlight > tags. Is it possible and, if yes, than how can it be done?

No, but there is an alternative way... For example, see Template:Translating.

Yes. Solution exists. I use it on my help page for example code, but in wiki must be installed Extension:Variables.

1. First I set {{#vardefine:temp|10 000,-}} value of the variable
2. Next use for view exemple code {{#tag:syntaxhighlight|{{#var:temp}}|lang=haskell}}
3. And finally I use {{#var:temp}} for the interpretation

I'm dealing with a programming language that's based on Delphi but has some differences.

There are some keywords that do not exist in Delphi, so they won't be colored and highlighted the proper way.

Is it somehow possible to use the HTML tags like <b> or <i> inside the <syntaxhighlight lang=delphi line></syntaxhighlight> blocks?

I would appreciate the help.

No, because ‎<syntaxhighlight> is a ‎<nowiki>‑like tag.

You would need to extend the upstream variety of offered languages. However, perhaps there is already another Delphi derivative which will fit your needs?

Actually, no. It's our internal system and it is not officially available anywhere except our company.

It's possible to work around this by using #tag, e.g. {{ #tag: syntaxhighlight | sample code here with a bit of <b>bolding</b> |lang=delphi |line }}, but it's likely to be pretty finicky (and you're not likely to get much help working around that finickiness). PerfektesChaos' suggestions would be better.

I've tried it like this but it doesn't work for me.

I get the following result:

<b>componentlist</b>

Pygments (the library which does the actual highlighting) is open source, so you can write your own lexer based on the Delphi/Pascal one. I haven’t tried doing so, so I don’t know how complicated it is (especially installing the modified Pygments), but it should be possible.

With MediaWiki 1.35.3, I was able to get the SyntaxHighlight extension to work by using an external pygmentize:

$wgPygmentizePath = '/usr/local/bin/pygmentize';

However after upgrading to MediaWiki 1.36.2, pygmentize is no longer working correctly. If I turn on error reporting:

error_reporting(E_ALL);
ini_set("display_errors", 1);

$wgShowExceptionDetails = true;
$wgShowDBErrorBacktrace = false;
$wgShowSQLErrors = true;
$wgResourceLoaderDebug = false;
$wgDebugToolbar = true;
$wgDevelopmentWarnings = true;
$wgShowDebug = true;

I then get these errors:

Notice: Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted [Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316] in /path/to/mediawiki_1_36/w/includes/debug/MWDebug.php on line 499

Same or similiar problem for lilypond:

Unable to obtain LilyPond version:
Error clone: main.c:2608 main: Operation not permitted

In the debug data I see the following information:

[exec] Executing: '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
[exec] Error running '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': Error clone: main.c:2608 main: Operation not permitted
[objectcache] fetchOrRegenerate(global:highlight:f3eb16b7c32d4a36f595352942cb4e1b): miss, new value computed
[error] [YWLA1o0mAVG7arGqGzx1GAAAAA4] /wiki/User_talk:Johnrdorazio ErrorException: PHP Notice: Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted
[Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316]
[warning] Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted
[Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316]

Does MediaWiki 1.36 require the usage of a local shellbox proxy? Or is that optional? I thought I would try adapting the instructions for setting up MediaWiki Shellbox to work in my Plesk setup, but I haven't been successful yet, so I've kept these two lines commented out:

//$wgShellboxUrl = 'http://shellbox.internal/shellbox';
//$wgShellboxSecretKey = 'random_generated_key_here';

But I still can't quite figure out if this is required by MediaWiki 1.36 or simply supported by MediaWiki 1.36.

Running

/usr/local/bin/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8 < /dev/null

on the command line outputs:

<div class="mw-highlight"><pre><span></span>
</pre></div>

Same goes for

/path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/pygments/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8 < /dev/null

. So they seem to work from the command line. Seems to me the Shell::Command is not working correctly to invoke pygmentize, or lilypond, or perhaps any shell command?

I'm running MediaWiki/Shellbox 1.0.4, so I don't believe that Phabricator T274474 is the issue.

To further test this, I am attempting to run the same pygmentize under firejail from the command line. I'm explicitly whitelisting limit.sh when running from the command line, and this kind of seems to work:

firejail --profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile --blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php --whitelist=/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh --noroot --seccomp --private-dev --net=none -- /bin/bash /path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh '/usr/local/bin/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8,linenos=inline < /dev/null' SB_INCLUDE_STDERR=; SB_CPU_LIMIT=180; SB_CGROUP=''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes

This produces:

Reading profile /path/to/mediawiki_1_36/w/includes/shell/firejail.profile
Parent pid 3252769, child pid 3252770
Child process initialized in 24.94 ms
<div class="mw-highlight"><pre><span></span><span class="linenos">1</span>
</pre></div>

Parent is shutting down, bye...

However it only works if I add that " < /dev/null". Without the "< /dev/null" the process just hangs. Is it possible that this is missing from MediaWiki Shell Command?

I tried editing wikimedia/shellbox/src/Command/BashWrapper.php like this:

        if ( $time > 0 || $mem > 0 || $filesize > 0 || $wallTime > 0 ) {
            $cmd = '/bin/bash ' . escapeshellarg( __DIR__ . '/limit.sh' ) . ' ' .
                escapeshellarg( $command->getCommandString() . ' < /dev/null' ) . ' ' .
                escapeshellarg(
                    "SB_INCLUDE_STDERR=" . ( $command->getIncludeStderr() ? '1' : '' ) . ';' .
                    "SB_CPU_LIMIT=$time; " .
                    'SB_CGROUP=' . escapeshellarg( $this->cgroup ) . '; ' .
                    "SB_MEM_LIMIT=$mem; " .
                    "SB_FILE_SIZE_LIMIT=$filesize; " .
                    "SB_WALL_CLOCK_LIMIT=$wallTime; " .
                    "SB_USE_LOG_PIPE=yes"
                );
            $command->unsafeCommand( $cmd )
                ->useLogPipe();
            if ( $command->getAllowedPaths() ) {
                // If specific paths have been allowed, make sure we explicitly
                // allow limit.sh. We don't do this unconditionally because it
                // doesn't work as expected in firejail, see T274474, T182486
                $command->allowPath( __DIR__ . '/limit.sh' );
            }
        }

Notice the extra . ' < /dev/null' after $command->getCommandString(). However this still doesn't help much, I'm getting the same error as before:

[exec] Executing: '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\'' < /dev/null' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
[exec] Error running '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\'' < /dev/null' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': Error clone: main.c:2608 main: Operation not permitted

I see others have had a similar issue when running in a docker container (see https://github.com/netblue30/firejail/issues/1856), however I'm not running in a docker container. To make sure permissions are not an issue, I did a 'sudo su - mediawikiApacheUser' and then issued the same pygmentize under firejail from the command line, this time as the mediawiki apache user, and got the same result as above without any permission errors.

I see someone else mentioned something about an old kernel being the issue, with missing NAMESPACE support.

My current kernel, on Ubuntu 20.04, is 5.4.0-73-generic. If I cd /usr/src/linux-headers-5.4.0-73-generic and egrep '_NS' .config, I seem to have all of the necessary NAMESPACE support:

CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NET_NSH=m
CONFIG_PATA_NS87415=m
CONFIG_PATA_NS87410=m
CONFIG_NS83820=m
CONFIG_GAMEPORT_NS558=m
CONFIG_TCG_NSC=m

So that doesn't seem to me to be the issue...

_{(edit conflict)}×3

If you don’t add < /dev/null, the process tries to read from the standard input, i.e. it waits for you to type something. Press Ctrl+D to tell it you’re done with typing.

I’m not really familiar with Firejail, but doesn’t its behavior depend on the user executing it? By using

sudo -u www-data firejail ...

instead of

firejail ...

you should become closer to what actually happens when MediaWiki tries to call Pygmentize (assuming you use Apache, different web servers may use different users).

In fact I did give that a try, as mentioned here. Seems to be working ok, without any particular errors... So I'm not sure why the MediaWiki Shellbox::Command is producing Error clone: main.c:2608 main: Operation not permitted.

For the time being I have just set $wgShellRestrictionMethod to false, rather than to 'firejail'. At least Lilypond and Pygments are working again. It would be nice for it to work with firejail...

Excuse me. On my local MediaWIki, when I install SyntaxHighlighter, I get an error message when displaying Gadget:

Notice: fwrite(): write of 7804 bytes failed with errno=32 Broken pipe in D:\mediawiki-1.36.0\vendor\wikimedia\shellbox\src\Command\UnboxedExecutor.php on line 315

When I save this script, the gadget does not work, but it does work if I remove SyntaxHighlighter from LocalSettings.php beforehand and save it.

Can anyone tell me what is causing this?

I'm getting the same error on my new install of 1.36.0

Seems to be looking for the OS type, but not sure how to fix it.

I "fix" the problem with this code on line 315 : $res = @fwrite( $pipe, $buffers[$fd], 65536 );

I don't think that really fixes anything, that just hides/silences the error message? But the error is still there. I am getting the same error now on 1.36.2 after upgrading from 1.35.3.

Hi, I am new to mediawiki. But I have to upgrade an old mediawiki that contains lots of page with source tags. Since it's deprecated by syntaxhighlight tag, how do I replace all source tags in all pages with syntaxhighlight tag in a batch way?

The way I'd do it is via AutoWikiBrowser (AWB) or a similar (semi)automated editing tool (if you're handy with a programming language and don't mind spending time on it, you could also write such a tool yourself). If you can install extensions yourself, or can request to have them installed, another option is Extension:ReplaceText , though personally I don't recommend this versus running AWB or similar.

Another possibility for linux is the pywikibot replace script (see Manual:Pywikibot/replace.py) with the following user-fix.py entry:

fixes['source_tag'] = {
    'regex': False,
    'msg': {
        '_default': 'source-tag ersetzt durch syntaxhighlight',
    },
    'replacements': [
        (r'<source', '<syntaxhighlight'),
        (r'</source', '</syntaxhighlight'),
    ]
}

called like this:

python3 pwb.py replace -fix:source_tag -start:!

that worked for me

cat wiki.sql | sed 's/<source/<syntaxhighlight/g' | sed 's/<\/source/<\/syntaxhighlight/g' > wiki.new.sql

make sure to backup and check thoroughly after importing back

I noticed that it is impossible to get out of the code entry field when you have to rely on keyboard navigation. (Mediawiki 1.31). Would it be hard to fix this?

The issue is that the tab key is used to indent/outdent the code. I don’t think it would be hard to change this behavior to navigating between form fields, but then there was no way to change the indentation (except for copy-pasting the tabulator character or entering it by ASCII/Unicode codepoint, neither of them is convenient). So there’s no good solution for this; the developers seem to have chosen a solution that’s better for those who prefer indenting their code with tabs but worse for those who rely on keyboard navigation.

Could Shift+Tab be adapted for this purpose? AFAIK it has no special function within the field, and it also doesn't in the browser or OS (Ctrl+Tab switches between open tabs in Firefox and (IIRC) Chrome, and Alt+Tab switches between open program windows in Windows).

It has: that’s the outdent. (Shift, as usual, reverses the direction: here it’s outdent instead of indent, if it wasn’t overridden, it would go to the previous form field instead of the next one, Ctrl+Shift+Tab goes to the previous tab instead of the next one, Alt+Shift+Tab goes to the least recently used window instead of the most recently used one, and so on.)

Aah, right, I always forget about outdenting for some reason. So much for that idea, then.

I want to have a template that contains precisely the source file to be displayed, and to drop it into another page syntax-highlighted.

Conceptually, what I want is something like

<syntaxhighlight> {{msgnw:MySourceFile}} </syntaxhighlight>

but of course that doesn't expand the template.

{{Pre|{{msgnw:MySourceFile}}}} sort of works, but it's not quite the same formatting as <syntaxhighlight>.

Is there a way?

If you haven't already, try {{ #tag: syntaxhighlight | {{MySourceFile}} |lang="lang" }}.

Thanks but, alas, turns a semicolon into &#59;.

Precisely what I tried - itself inside a template - is

{{ #tag: syntaxhighlight | {{msgnw:User:Jordan Brown/sandbox/{{{name}}}}}|lang="C" }}

where the page pointed at contains cube(10);.

But I didn't know anything about #tag, so I'll look into it further.

You might have luck without the msgnw (I haven't tested this myself):

{{ #tag: syntaxhighlight | {{:User:Jordan Brown/sandbox/{{{name}}}}}|lang="C" }}

No, didn't work. With

{{ #tag: syntaxhighlight | {{{{BOOKNAME}}/examples/{{{name}}}}}|lang="C"|line="line" }}

and the input being (a deliberate torture test)

cube(10);|| foo && bar {{baz}} <pre></pre> &lt;
x < 5 && y > 6

the result was

cube(10);|| foo && bar [[:Template:Baz]] '"`UNIQ--pre-00000000-QINU`"' <
x < 5 && y > 6

Thanks for your help. Other editors on the book have looked at it and decided that my idea of keeping the examples in separate pages would be too awkward, so I'm abandoning the effort for now.

Hi Everyone,

This looks like a very useful extension. I hope we can get it working with our installation.

We run a hardened web server at www.cryptopp.com. We remove a bunch of unsafe functions using PHP ini file via disable_functions. You can find the hardened settings here: https://github.com/weidai11/website/blob/master/install/security.ini.

When we attempt to enable SyntaxHighlight in LocalSettings.php, this is the result:

Fatal error: Uncaught ExtensionDependencyError: SyntaxHighlight requires "shell" ability:
Unable to run external programs, proc_open() is disabled in /var/www/html/w/includes/registration/ExtensionRegistry.php:407
Stack trace:
#0 /var/www/html/w/includes/registration/ExtensionRegistry.php(231): ExtensionRegistry->readFromQueue()
#1 /var/www/html/w/includes/Setup.php(161): ExtensionRegistry->loadFromQueue()
#2 /var/www/html/w/includes/WebStart.php(89): require_once('/var/www/html/w...')
#3 /var/www/html/w/index.php(44): require('/var/www/html/w...')
#4 {main} thrown in /var/www/html/w/includes/registration/ExtensionRegistry.php on line 407

PHP does not allow us to whitelist one extension.

My request is, it would be nice if SyntaxHighlight had its own pretty print code so it can handle syntax highlighting within the sandbox that PHP is restricted to.

Thanks in advance.

I don’t think it’s a realistic wish, this extension relies on an external program exactly to avoid reinventing the wheel. If it had its own built-in highlighter, then the external one would not be necessary at all, but it would be a large maintenance burden. You can try using the old MediaWiki 1.24 version, which uses GeSHi, written in plain PHP, but I’m sure it won’t work out of the box with modern MW versions. Also, it was for a reason that it was replaced, long-unmaintained software is probably not the best choice if security is an important factor…

Thanks @Tacsipacsi.

As much as I hate to suggest it... Can you consider a PHP compatible library and install it via Composer? Instead of using banned functions and external programs, you would use the PHP library. Or make it a configuration option (external program vs PHP library). That should meet your needs and the needs of folks who wish to harden their installation.

Maybe something like one of these:

• https://github.com/scrivo/highlight.php
• https://github.com/Automattic/syntaxhighlighter
• https://github.com/GeSHi/geshi-1.0

Also see "php" syntax highlighter site:github.com.

Noloader (talk) 01:17, 13 April 2021 (UTC)

Oh, check this out @Tacsipacsi...

https://github.com/ramsey/pygments. It is an actively maintained PHP Wrapper for the Python Pygments.

How difficult would it be to use the PHP bindings rather than shelling out?

Probably not that difficult, but at least entirely useless—ramsey/pygments also shells out (through symfony/process). I don’t think it’s possible to run any non-PHP program if you don’t want to allow shelling out; how would you do that?

Thanks @Tacsipacsi.

My bad. I did not realize it was shelling out, too.

It looks like there's an API at https://pygments.org/docs/api/. Using the API does not look too difficult: https://pygments.org/docs/quickstart/#example.

Do you know if PHP can call the Python API? It seems like PHP should be able to do it. (I'm a C/C++/ObjC guy. I don't know PHP).

Thanks in advance.

I haven’t tried interacting with non-PHP code from PHP ever except for with proc_open(), so I don’t have any experience, but I wouldn’t hope it can directly interact with Python. Maybe PHP could interact with C and C with Python, as both Zend and CPython interpreters are written in C, but that’s quite a number of hops (and it’s complicated also by the fact that C should be compiled for each platform individually).

However, it just came into my mind that there’s a work in progress to containerize shell execution; maybe it’s acceptable for you security-wise to allow running arbitrary code in an isolated container. This is a priority for Wikimedia as well, so hopefully sooner or later it will be implemented for this extension, too.