Topic on Extension talk:SyntaxHighlight

Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted [Called from SyntaxHighlight::highlightInner

10
Lwangaman (talkcontribs)

With MediaWiki 1.35.3, I was able to get the SyntaxHighlight extension to work by using an external pygmentize:

$wgPygmentizePath = '/usr/local/bin/pygmentize';

However after upgrading to MediaWiki 1.36.2, pygmentize is no longer working correctly. If I turn on error reporting:

error_reporting(E_ALL);
ini_set("display_errors", 1);

$wgShowExceptionDetails = true;
$wgShowDBErrorBacktrace = false;
$wgShowSQLErrors = true;
$wgResourceLoaderDebug = false;
$wgDebugToolbar = true;
$wgDevelopmentWarnings = true;
$wgShowDebug = true;

I then get these errors:

Notice: Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted [Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316] in /path/to/mediawiki_1_36/w/includes/debug/MWDebug.php on line 499

Same or similiar problem for lilypond:

Unable to obtain LilyPond version:
Error clone: main.c:2608 main: Operation not permitted

In the debug data I see the following information:

[exec] Executing: '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
[exec] Error running '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\''' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': Error clone: main.c:2608 main: Operation not permitted
[objectcache] fetchOrRegenerate(global:highlight:f3eb16b7c32d4a36f595352942cb4e1b): miss, new value computed
[error] [YWLA1o0mAVG7arGqGzx1GAAAAA4] /wiki/User_talk:Johnrdorazio ErrorException: PHP Notice: Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted
[Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316]
[warning] Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted
[Called from SyntaxHighlight::highlightInner in /path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/includes/SyntaxHighlight.php at line 316]
Lwangaman (talkcontribs)

Does MediaWiki 1.36 require the usage of a local shellbox proxy? Or is that optional? I thought I would try adapting the instructions for setting up MediaWiki Shellbox to work in my Plesk setup, but I haven't been successful yet, so I've kept these two lines commented out:

//$wgShellboxUrl = 'http://shellbox.internal/shellbox';
//$wgShellboxSecretKey = 'random_generated_key_here';

But I still can't quite figure out if this is required by MediaWiki 1.36 or simply supported by MediaWiki 1.36.

Lwangaman (talkcontribs)

Running

/usr/local/bin/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8 < /dev/null

on the command line outputs:

<div class="mw-highlight"><pre><span></span>
</pre></div>

Same goes for

/path/to/mediawiki_1_36/w/extensions/SyntaxHighlight_GeSHi/pygments/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8 < /dev/null

. So they seem to work from the command line. Seems to me the Shell::Command is not working correctly to invoke pygmentize, or lilypond, or perhaps any shell command?

Lwangaman (talkcontribs)

I'm running MediaWiki/Shellbox 1.0.4, so I don't believe that Phabricator T274474 is the issue.

To further test this, I am attempting to run the same pygmentize under firejail from the command line. I'm explicitly whitelisting limit.sh when running from the command line, and this kind of seems to work:

firejail --profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile --blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php --whitelist=/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh --noroot --seccomp --private-dev --net=none -- /bin/bash /path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh '/usr/local/bin/pygmentize -l sql -f html -O cssclass=mw-highlight,encoding=utf-8,linenos=inline < /dev/null' SB_INCLUDE_STDERR=; SB_CPU_LIMIT=180; SB_CGROUP=''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes

This produces:

Reading profile /path/to/mediawiki_1_36/w/includes/shell/firejail.profile
Parent pid 3252769, child pid 3252770
Child process initialized in 24.94 ms
<div class="mw-highlight"><pre><span></span><span class="linenos">1</span>
</pre></div>

Parent is shutting down, bye...

However it only works if I add that " < /dev/null". Without the "< /dev/null" the process just hangs. Is it possible that this is missing from MediaWiki Shell Command?

Lwangaman (talkcontribs)

I tried editing wikimedia/shellbox/src/Command/BashWrapper.php like this:

        if ( $time > 0 || $mem > 0 || $filesize > 0 || $wallTime > 0 ) {
            $cmd = '/bin/bash ' . escapeshellarg( __DIR__ . '/limit.sh' ) . ' ' .
                escapeshellarg( $command->getCommandString() . ' < /dev/null' ) . ' ' .
                escapeshellarg(
                    "SB_INCLUDE_STDERR=" . ( $command->getIncludeStderr() ? '1' : '' ) . ';' .
                    "SB_CPU_LIMIT=$time; " .
                    'SB_CGROUP=' . escapeshellarg( $this->cgroup ) . '; ' .
                    "SB_MEM_LIMIT=$mem; " .
                    "SB_FILE_SIZE_LIMIT=$filesize; " .
                    "SB_WALL_CLOCK_LIMIT=$wallTime; " .
                    "SB_USE_LOG_PIPE=yes"
                );
            $command->unsafeCommand( $cmd )
                ->useLogPipe();
            if ( $command->getAllowedPaths() ) {
                // If specific paths have been allowed, make sure we explicitly
                // allow limit.sh. We don't do this unconditionally because it
                // doesn't work as expected in firejail, see T274474, T182486
                $command->allowPath( __DIR__ . '/limit.sh' );
            }
        }

Notice the extra . ' < /dev/null' after $command->getCommandString(). However this still doesn't help much, I'm getting the same error as before:

[exec] Executing: '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\'' < /dev/null' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes'
[exec] Error running '/usr/bin/firejail' '--quiet' '--profile=/path/to/mediawiki_1_36/w/includes/shell/firejail.profile' '--blacklist=/path/to/mediawiki_1_36/w/LocalSettings.php' '--noroot' '--seccomp' '--private-dev' '--net=none' -- /bin/bash '/path/to/mediawiki_1_36/w/vendor/wikimedia/shellbox/src/Command/limit.sh' ''\''/usr/local/bin/pygmentize'\'' '\''-l'\'' '\''js'\'' '\''-f'\'' '\''html'\'' '\''-O'\'' '\''cssclass=mw-highlight,encoding=utf-8,linenos=inline'\'' < /dev/null' 'SB_INCLUDE_STDERR=;SB_CPU_LIMIT=180; SB_CGROUP='\'''\''; SB_MEM_LIMIT=314572800; SB_FILE_SIZE_LIMIT=104857600; SB_WALL_CLOCK_LIMIT=180; SB_USE_LOG_PIPE=yes': Error clone: main.c:2608 main: Operation not permitted
Lwangaman (talkcontribs)

I see others have had a similar issue when running in a docker container (see https://github.com/netblue30/firejail/issues/1856), however I'm not running in a docker container. To make sure permissions are not an issue, I did a 'sudo su - mediawikiApacheUser' and then issued the same pygmentize under firejail from the command line, this time as the mediawiki apache user, and got the same result as above without any permission errors.

Lwangaman (talkcontribs)

I see someone else mentioned something about an old kernel being the issue, with missing NAMESPACE support.

My current kernel, on Ubuntu 20.04, is 5.4.0-73-generic. If I cd /usr/src/linux-headers-5.4.0-73-generic and egrep '_NS' .config, I seem to have all of the necessary NAMESPACE support:

CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NET_NSH=m
CONFIG_PATA_NS87415=m
CONFIG_PATA_NS87410=m
CONFIG_NS83820=m
CONFIG_GAMEPORT_NS558=m
CONFIG_TCG_NSC=m

So that doesn't seem to me to be the issue...

Tacsipacsi (talkcontribs)

(edit conflict)×3

If you don’t add < /dev/null, the process tries to read from the standard input, i.e. it waits for you to type something. Press Ctrl+D to tell it you’re done with typing.

I’m not really familiar with Firejail, but doesn’t its behavior depend on the user executing it? By using

sudo -u www-data firejail ...

instead of

firejail ...

you should become closer to what actually happens when MediaWiki tries to call Pygmentize (assuming you use Apache, different web servers may use different users).

Lwangaman (talkcontribs)

In fact I did give that a try, as mentioned here. Seems to be working ok, without any particular errors... So I'm not sure why the MediaWiki Shellbox::Command is producing Error clone: main.c:2608 main: Operation not permitted.

Lwangaman (talkcontribs)

For the time being I have just set $wgShellRestrictionMethod to false, rather than to 'firejail'. At least Lilypond and Pygments are working again. It would be nice for it to work with firejail...

Reply to "Failed to invoke Pygments: Error clone: main.c:2608 main: Operation not permitted [Called from SyntaxHighlight::highlightInner"