When I open very large pages, CodeMirror makes it very slow to open. Is there any way (maybe a js script?) to make sure that editor always opens with CodeMirror off and I can turn it on when I need it?
Extension talk:CodeMirror
Syntax coloring on very large files can be slow even outside wikis, e.g. in local text editors like Notepad++. The reason is that this coloring tries to apply everything at noce on the whole file even if, when opening the file, the coloring is just needed for the currently viewed part (usually the top portion), and could be made in the background, only on the visible part (and slightly above or below it by roughly one page). The most critical part however is not the syntax analysis, but applying the stylesheet and transforming the document into many spans: this requires lot of memory allocations and many uses of internal APIs (on a web page this involves using the DOM API).
But even without syntax coloring, this may occur just for splitting the document into lines and counting them (notably when automatic linewrap is enabled, forcing not just a parsing, but also computing display metrics), just to determine what to show or positioning scrollbars.
I don't think that loading the needed javascript is a real issue (it should not related to network delays with many requests, the scripts should be loaded once and cached in the browser), or loading the file content itself (unless it is very large) but to the fact that this requires lot of internal work.
If files are really large, one way would be to load it into an editable internal temporary cache and split it into separate segments (e.g. by page every 1000 lines or roughly 4KB), and generating an internal index that allows loading it and parsing it on demand, and saving lot of memory. This would allow fast scrolling, e.g. to the end of file without having to parse and colorize everything from the beginning, without needing many network requests to sync each segment. Only when the edited file will be saved/submitted, this splitted editing cache stored in some temporary file format (whose actual segment sizes would be variable, allowing fast insertions/deletions without moving lot of memory and using many reallocations) would be joined to a sequential stream. Such temporary "paged" cache would have a binary structure, and would be a bit larger than the actual file content, but editing would be much faster, and in fact would much less be stresssing the memory. The editor would also work within much lower memory constraints (notably if the browser uses a 32-bit implementation or runs in a constained environment).
However such improved design would require major redesign of the editor code, to use this "divide-and-conquer" optimization strategy. Basic text editors do not use it, but various wellknown IDEs used for code editing for working on large projects have implemented it.
I understand. I'm okay with it being slow. I just don't want it to start highlighting automatically because sometimes I don't need it. I was wondering if there is a way to make it disabled every time you open the editor but with the option to turn it on if you need it.
This is a valid request, in my opinion, to have CodeMirror enabled only on demand, and disabled by default, possibly with a button on the basic editor UI to activate it. But there will still be a similar issue with the basic editor, which also has size limits.
And CodeMirror should not turn on automatically for files larger than some reasonable threshold (which may be eventually tunable in preferences, or where syntax highlighting would use a simpler parser requiring much less style modification of edited files, by fragmenting it less into too small spans).
- There's a problem here: Can we make assumptions about reasonable thresholds, without performing some measurements, i.e. monotoring some timed metrics, both for successes and failures/timeouts/crashes on both on the server side and on the client side (with enough precision about the client-side platform or browser type and its capabilities ?" Ideally such metrics would also be useful for quality assurances on the CodeMirror extension, i.e. for A/B testing in cases of changes of implementation, or for tracking possible critical problems like detection of vulnerabilities or DDoS attacks against the server, or against some ranges of client type, or just to help developers track what they need to change and where improvements should be made. If we had some could tuning, we could avoid the need for users to use any user-specific settings in their preferences, or the server-side editor could display warnings banners if such edit would likely cause a crash of the client, or a blocking error if this can cause damages on the server side.
This also applies to "syntax highlight" tags within editable MediaWiki pages, which fails on the server for large files and does not always fallback automatically to the basic "plain text" language model after maximum sizes are reached.
This is also the same problem when enabling by default the VisualEditor on large pages, when they can only be safely edited using the simpler MediaWiki code editor: there are size limits for the MediaWiki online editor, already implemented into MediaWiki setting paximum page sizes, but another limit could also count the maximum number of tokens generated by the parser, untill it renounces using the VisualEditor and falls back to the basic code editor. Note that the maximum page size applies as well for pages edited with more capable external editors, and transfered by bulk uploads (but this could cause problems for rendering page histories if they can't compute differences, which also requires a complex parsing that may exhaust server resources).
That's also the reason why most media files are generally not parsed at all and just handled as blobs within edit histories, just tracking the existence of different versions (just detecting differences by computing a digital fingerprint like SHA1 on the whole blob), however limited parsers may just check if the file encoding is safe, and if it embeds some limited metadata which could be differentiated in histories, notably in photos, or for embedded licences. Their actual editing is made externally, and media files are transferred by full file downloads/uploads, MediaWiki having no tools by default to edit them more incrementally (except a few files formats like tabular data, requiring a custom extension).
May be there's a need for improvement for CodeMirror, so that stored pages using them would have some metadata attached and stored separately (just like existing metadata indicating the page content model and other file properties), indicating if the advanced CodeMirror editor should be enabled or disabled for specific large files, notably if they require frequent updates with more cooperation with more users.
MediaWiki is not very well tuned to load/save and track changes in the history for very large editable files. Such files should be limited in size in your project, try splitting these too large files if you can in your code design. This will also help other users cooperate on your project, when they have a less capable browser, with less memory or slower CPUs. This is in fact not specific to MediaWiki, this is a common issue as well for other cooperative projects, e.g within GitHub (even if their servers are very large and support very complex projects).
[There's a similar but unrelated problem with very large lists of user-tracked pages, but the issue is not in the client editor but in the data model on the server; the client implemented in the UI, loads these huge lists very slowly, then makes lots of modification in the DOM, exhausting the resources of the browser which may become unresponsive on load or can crash the session; correcting this by using more selective filters requires changes in MediaWiki on the server, so this is unrelated; for now all what users can do is to purge their tracking list completely or disable the automatic addition in this list; however this affects only users editing many pages on the same wiki over a long period of time, with the automatic tracking option enabled by default even for minor changes; or use an external editor to download and upload the list filtered/sorted in some external editor, using a more advanced MediaWiki feature to do that in their user profiles; there has been improvements for that, but the issue remains, however it does not affect a lot of users.]
The preference is 'sticky', so if you enable it whilst making one edit, the next time you open the editor it will be enabled (and vice versa); as a quick short-term solution, if you use it, you can try to remember to disable it again before publishing. Obviously this isn't great, but it's a start.
Adding a user preference to always disable CodeMirror is an understandable request, but should be weighed against the cost of doing so (mostly, the added complexity for other users). We could also add a runtime performance check to auto-disable on load based on some metric?
@Jdforrester (WMF) thanks. Is there any way to control it via a user script? If you give me the tail, I can write it myself. Any API and/or example?
Maybe override the click event on the Publish changes button and disable it with JS?
The preference key is usecodemirror and it appears to be off by default. There's advice on API:Options on how to set your preferences, but I'm in meetings right now, sorry!
On a MW 1.39 installation (1.39.5. PHP 7.4.33, CodeMirror 4.0) I see the follwing deprecation notice:
2023/12/18 14:48:08 [error] 22051#22051: *41089 FastCGI sent in stderr: "PHP message: PHP Deprecated: Use of User::getOption was deprecated in MediaWiki 1.35. [Called from CodeMirrorHooks::isCodeMirrorOnPage in extensions/CodeMirror/includes/CodeMirrorHooks.php at line 20] in includes/debug/MWDebug.php on line 381PHP message: PHP Stack trace:PHP message: PHP 1. {main}() index.php:0PHP message: PHP 2. wfIndexMain() index.php:46PHP message: PHP 3. MediaWiki->run() index.php:50PHP message: PHP 4. MediaWiki->main() includes/MediaWiki.php:562PHP message: PHP 5. OutputPage->output() includes/MediaWiki.php:922PHP message: PHP 6. MediaWiki\HookContainer\HookRunner->onBeforePageDisplay() includes/OutputPage.php:2871PHP message: PHP 7. MediaWiki\HookContainer\HookContainer->run() includes/HookContainer/HookRunner.php:946PHP message: PHP 8. MediaWiki\HookContainer\HookContainer->callLegacyHook() includes/HookContainer/HookContainer.php:137PHP message: PHP 9. CodeMirrorHooks::onBeforePageDisplay() includes/HookContainer/HookContainer.php:338PHP message: PHP 10. CodeMirrorHooks::isCodeMirrorOnPage() extensions/CodeMirror/includes/CodeMirrorHooks.php:38PHP message: PHP 11. User->getOption() extensions/CodeMirror/includes/CodeMirrorHooks.php:20PHP message: PHP 12. wfDeprecated() includes/user/User.php:2123PHP message: PHP 13. MWDebug::deprecated() includes/GlobalFunctions.php:765PHP message: PHP 14. MWDebug::deprecatedMsg() includes/debug/MWDebug.php:233PHP message: PHP 15. MWDebug::sendRawDeprecated() includes/debug/MWDebug.php:352PHP message: PHP 16. trigger_error() includes/This was updated 2 years ago via gerrit:742512. As far as I can see that became part of all releases since 1.38. It looks like you are running an older version of the extension.
If wrapped lines ends by spaces (so nearly always), End puts the cursor at the beginning of the next wrapped line - so it basically behaves like Home (except it puts the cursor in the wrapped line below).
You can't put the cursor behind the last space (in the same horizontal line), you can either put cursor before the last space, or "after" it (which puts cursor at the beginning of the wrapped line below).
Can it be set so it would behave like thread editor here: End puts the cursor after last space (in the same horizontal line). Cursor right places cursor at the beginning of wrapped line below)?
Example of a long wrapped line:
Góry Zielone (ang. Green Mountains, fr. Montagnes Vertes) – pasmo górskie w stanach Vermont i Massachusetts (Stany Zjednoczone) oraz prowincji Quebec (Kanada), wchodzące w skład Appalachów. Góry Zielone (ang. Green Mountains, fr. Montagnes Vertes) – pasmo górskie w stanach Vermont i Massachusetts (Stany Zjednoczone) oraz prowincji Quebec (Kanada), wchodzące w skład Appalachów.
As said before, I think these questions belong to https://codemirror.net.
TL: read the summary in my next post.
Not quite, because current version of CM wraps lines similar to what I proposed - except cursor right at the end of wrap (End) places the cursor after first char of the wrapped line below.
So this is either the question of CM implementation used by the extension (ie. this could be introduced in later version than 5.53.x), or some settings (or maybe mode/style influence the cursor?).
Edit: 5.53.3 can set the cursor at the end of line wrap, because it does so if the mode is loaded manually:
editor.setOption("lineWrapping",true);
var ek=editor.getOption("extraKeys");
if (!ek) ek={}
ek.End='goLineRight'
editor.setOption("extraKeys",ek);
So it's a matter of extension changes in implementation, its settings or mode (including styles).
I think that placing the cursor at the end of line by End is more natural than its current behavior in mediawiki mode.
(Does mediawiki mode is also covered by Extension:CodeMirror?).
Even later edit: I've pinpointed the setting that influences cursor behavior: it's inputStyle: 'contenteditable' (in ex.CodeMirror.js)
Full test code for clarity:
mw.loader.load("ext.CodeMirror") //execute separately
mw.loader.load("ext.CodeMirror.mode.mediawiki") //execute separately
var cmOptions={mode:"text/mediawiki",mwConfig:mw.config.get('extCodeMirrorConfig'),lineWrapping: true,inputStyle: 'contenteditable'}
var editor = CodeMirror.fromTextArea(document.getElementById("wpTextbox1"),cmOptions)
var ek=editor.getOption("extraKeys");
if (!ek) ek={}
ek.End='goLineRight'
editor.setOption("extraKeys",ek);
Summary of the above (sorry for multiple edits):
I guess it's a CM thing after all.
Since you can't change inputStyle at runtime (yet, according to comment in CM code), maybe there is a way to change it on Wikipedia on the browser client side (without the need of changing code in the extension)?
Edit: Also, on contenteditable in 2020: What are the pitfalls of using inputStyle ‘contenteditable’ in 2020?.
And in 2021: CM5: No cursor before bookmark.
As to the cursor at the end of wrapped lines (in contenteditable): it's probably possible, but would require implementing a custom cursor (CM5: inputStyle contenteditable, cursor display after goLineRight (wrapped line)).
If you do markText with option collapsed=true (with replacedWith), and place the cursor just before the arrows marking collapsed range, the cursor won't be showing.
Example code (part above the empty line run once, markText to test range, clear to remove mark):
var text = document.createTextNode('\u2194');
widget = document.createElement("span");
widget.appendChild(text);
widget.className = "CodeMirror-foldmarker";
var editor = $(".CodeMirror")
editor[0].CodeMirror.doc.markText({line:0,ch:1},{line:1,ch:10},{collapsed:true,replacedWith:widget})
editor[0].CodeMirror.getAllMarks()[0].clear()
pl.wikipedia.org
I'm afraid we aren't able to provide support for the upstream CodeMirror library.
I don't think it's entirely the CM fault, it could be something either in the changed CM library code, mediawiki mode, or in mediawiki css (or it's just a matter of config option?).
Because if you load mode manually, the cursor in front of the range is shown.
This works (in code editor without CodeMirror present - disable highlighting and refresh the editor by F5; use editor. in place of editor[0].CodeMirror. with markText line above):
mw.loader.load("ext.CodeMirror")
mw.loader.load("ext.CodeMirror.lib.mode.htmlmixed")
var editor = CodeMirror.fromTextArea(document.getElementById("wpTextbox1"),{mode:"htmlmixed"})
mw.loader.load("ext.CodeMirror")
mw.loader.load("ext.CodeMirror.mode.mediawiki")
var editor = CodeMirror.fromTextArea(document.getElementById("wpTextbox1"),{mode:"mediawiki",mwConfig:mw.config.get( 'extCodeMirrorConfig' )})
But if you switch normally to CM editor by the button, the cursor in front of the range won't be shown (move cursor by cursor keys, mouse isn't very reliable).
This is probably a CM bug (with inputStyle: "contenteditable"), like Thiemo said.
Any plans to make this available on Packagist or any other composer repos?
I'm afraid there is not much we can say without any context. Are the installation instructions not sufficient?
How to use this syntax highlighter's API on some userscript or out of MediaWiki?
There is not really an official API, as far as I'm aware of. What you can do is to re-use the original CodeMirror library the same way the extension does. In a user script this would probably be done via mw.loader.using( [ 'ext.CodeMirror.lib' ] ).then( … ) and then something like CodeMirror.fromTextArea( … ). See ResourceLoader/Developing with ResourceLoader as well as ext.CodeMirror.js for more information.
Thanks. I understood.
This syntax highlighter for wikitext is really good. It should be added into CodeMirror docs: https://codemirror.net/5/mode/index.html for other uses out of MediaWiki scope.
I didn't find much syntax highlighter for wikitext on Internet.
MW 1.35.6 / PHP 7.4.27 / CodeMirror 4.0.0
Despite the option being enabled, line numbering does not work.
$wgCodeMirrorLineNumberingNamespaces = [ NS_TEMPLATE ];
The feature didn't exist before MediaWiki 1.37.
MW 1.37.2 / PHP 7.4.27 / CodeMirror 4.0.0
The line numbering still does not work.
Try with $wgCodeMirrorLineNumberingNamespaces = null; → no more success.
The CodeMirror version number is not very meaningful, unfortunately. Are you sure you updated the extension? You can check if CodeMirror/extension.json contains "CodeMirrorLineNumberingNamespaces".
| MediaWiki | 1.35.2 |
| PHP | 7.4.16 |
| Page Forms | 5.3.4 (27db083) |
I followed these and I get the wikieditor to show but the highlighter is not there
You can't change mouse cursor through option selectionPointer (from CM selection-pointer addon).
It works if you load mediawiki mode manually (topic Collapsed range (markText) hides cursor just before the range).
mw.loader.load("//codemirror.net/addon/selection/selection-pointer.js")
editor.setOption("selectionPointer","grab")
pl.wikipedia.org
Oh, that because cm.display.selectionDiv doesn't have any children.
Also it doesn't use class CodeMirror-selected to mark the selection.
Correction/update: cm.display.selectionDiv eventually will have children, if you make more than one selection (with Ctrl). It stores all the selection boxes EXCEPT the current/last/active one (so if you only use one, it will be most likely always empty).
If you select text before switching to CodeMirror (CM) editor, the selection won't be visible - unless you click on the right scrollbar and press the Tab key.
pl.wikipedia.org, old code editor (with syntax highlighting/CM switch).
Edit: Now it seems that it also doesn't select the text if you turn off the CM. It did so couple days ago (at the time of writing topics below).