Topic on Extension talk:SyntaxHighlight

Olarp (talkcontribs)

Running MediaWiki 1.38.4 on

  • Windows Server 2019
  • IIS 10
  • PHP 8.1.13
  • Python 3.11
  • Pygmentize 2.13.0

For unknown reasons some lexers work while other don't and instead generate the "Pages with syntax highlighting errors"-category. The page below, as an example, generates the powershell and bash code correctly, but not the yaml.

If I swap the yaml for bash without changing the block of code below, it does not generate an "Pages with syntax highlighting" category of it, and of course it doesn't highlight it correctly either as it is trying to bash highlight a yaml snippet.

<syntaxhighlight lang="powershell">
# This is a PowerShell script
Write-Host "Hello world"
</syntaxhighlight>

<syntaxhighlight lang="bash">
# This is a shell script in bash
ls -l /var/lib
</syntaxhighlight>

<syntaxhighlight lang="yaml">
# This is a yaml block
- martin:
    job: Developer
</syntaxhighlight>

I did not try all lexers supported by pygmentize. But it seems some work, some not. powershell and bash works every time, yaml and text fails everytime. It doesn't matter what order the blocks are on the page, they can be alone and it still fails. I checked the file SyntaxHighlight.lexers.php and all the lexers that pygmentize support are there, marked as true.

This is what debug output looks for the above page, relevant section. Note it only runs pygmentize four times. First time with version check, second with lexers check, third and fourth is the syntax highlight blocks that work.

[DBQuery] MediaWiki\Storage\SqlBlobStore::fetchBlobs [0s] 127.0.0.1: SELECT old_id,old_text,old_flags FROM `text` WHERE old_id = 1822
[objectcache] fetchOrRegenerate(global:SqlBlobStore-blob:hkr_mediawiki:tt%3A1822): miss, new value computed
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b8cf480f6bf3322
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-V"" 2>&1
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b8cf480f6bf3322/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b8cf480f6bf3322/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b8cf480f6bf3322"
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-86fd2e5bba940d09
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-L" "lexer"" 2>&1
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-86fd2e5bba940d09/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-86fd2e5bba940d09/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-86fd2e5bba940d09"
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-237f871e31353545
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-l" "powershell" "-f" "html" "-O" "cssclass=mw-highlight,encoding=utf-8" "file""
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-237f871e31353545/file"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-237f871e31353545/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-237f871e31353545/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-237f871e31353545"
[objectcache] fetchOrRegenerate(global:highlight:1e04b7e8e2e671872099e468f594fa58): miss, new value computed
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-5a0641ea93a9910c
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-l" "bash" "-f" "html" "-O" "cssclass=mw-highlight,encoding=utf-8" "file""
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-5a0641ea93a9910c/file"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-5a0641ea93a9910c/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-5a0641ea93a9910c/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-5a0641ea93a9910c"
[objectcache] fetchOrRegenerate(global:highlight:79a0bec1f4c29b0a4e1f837a4b70f4a9): miss, new value computed
[DBConnection] Wikimedia\Rdbms\LoadBalancer::getLocalConnection: reused a connection for local/0
[DBQuery] LCStoreDB::get [0s] 127.0.0.1: SELECT lc_value FROM `l10n_cache` WHERE lc_lang = 'sv' AND lc_key = 'messages:syntaxhighlight-error-category' LIMIT 1																									
[DBConnection] Wikimedia\Rdbms\LoadBalancer::getLocalConnection: reused a connection for local/0
[DBQuery] LCStoreDB::get [0s] 127.0.0.1: SELECT lc_value FROM `l10n_cache` WHERE lc_lang = 'sv' AND lc_key = 'linkPrefixExtension' LIMIT 1
[DBConnection] Wikimedia\Rdbms\LoadBalancer::getLocalConnection: reused a connection for localAutoCommit/0

When swapping the yaml for bash it processes correctly and doesn't throw errors and doesn't get the "Pages with syntax highlighting errors". It now runs pygmentize five times, with the last working bash-block.

[DBQuery] MediaWiki\Storage\SqlBlobStore::fetchBlobs [0s] 127.0.0.1: SELECT old_id,old_text,old_flags FROM `text` WHERE old_id = 1823
[objectcache] fetchOrRegenerate(global:SqlBlobStore-blob:hkr_mediawiki:tt%3A1823): miss, new value computed
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-c3bb7d17062047d8
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-V"" 2>&1
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-c3bb7d17062047d8/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-c3bb7d17062047d8/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-c3bb7d17062047d8"
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0484336c3f645690
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-L" "lexer"" 2>&1
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0484336c3f645690/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0484336c3f645690/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0484336c3f645690"
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b05e1edd661e368
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-l" "powershell" "-f" "html" "-O" "cssclass=mw-highlight,encoding=utf-8" "file""
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b05e1edd661e368/file"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b05e1edd661e368/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b05e1edd661e368/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-0b05e1edd661e368"
[objectcache] fetchOrRegenerate(global:highlight:1e04b7e8e2e671872099e468f594fa58): miss, new value computed
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-d08c021bfe1b0311
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-l" "bash" "-f" "html" "-O" "cssclass=mw-highlight,encoding=utf-8" "file""
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-d08c021bfe1b0311/file"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-d08c021bfe1b0311/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-d08c021bfe1b0311/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-d08c021bfe1b0311"
[objectcache] fetchOrRegenerate(global:highlight:79a0bec1f4c29b0a4e1f837a4b70f4a9): miss, new value computed
[exec] Creating base path C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-90a3aca3d2df8f1f
[exec] Executing: cmd /s /c ""c:\Progra~1\Python311\Scripts\pygmentize.exe" "-l" "bash" "-f" "html" "-O" "cssclass=mw-highlight,encoding=utf-8,linenos=inline,hl_lines=4" "file""
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-90a3aca3d2df8f1f/file"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-90a3aca3d2df8f1f/sb-stderr"
[exec] Removed file "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-90a3aca3d2df8f1f/sb-stdout"
[exec] Removed directory "C:\WINDOWS\TEMP\mwtmp-IUSR/shellbox-90a3aca3d2df8f1f"
[objectcache] fetchOrRegenerate(global:highlight:3ad02a4ec124ea3009c02acf078da02e): miss, new value computed
[DBConnection] Wikimedia\Rdbms\LoadBalancer::getLocalConnection: reused a connection for local/0
[DBQuery] LCStoreDB::get [0s] 127.0.0.1: SELECT lc_value FROM `l10n_cache` WHERE lc_lang = 'sv' AND lc_key = 'linkPrefixExtension' LIMIT 1
[DBConnection] Wikimedia\Rdbms\LoadBalancer::getLocalConnection: reused a connection for localAutoCommit/0

I'm so out of ideas.

What do you think?

Olarp (talkcontribs)

Upgrading to MediaWiki 1.39.4 appears to have fixed this issue. All lexers work in this version.

The problem still persisted in MediaWiki 1.39.2 but an upgrade to 1.39.4 fixed it.