Topic on Extension talk:Lockdown

Hi All,

We have been running mediawiki for about 10 years and at some point started to notice issues with lockdown and visual editor for which there are many discussion pages and bugs for example: https://phabricator.wikimedia.org/T148582#4412138 https://phabricator.wikimedia.org/T148582

As our wiki is readable by all users but only editable by logged in users we hit the known issues. For several years now we have been applying a work around not to load lockdown for localhost, 127.0.0.1 and local server IP. With out this workaround protected namespaces cannot be edited. Workaround is similar to:

if ( !isset( $_SERVER['REMOTE_ADDR'] ) OR $_SERVER['REMOTE_ADDR'] == '127.0.0.1'        ) {
        $wgGroupPermissions['*']['read'] = true;
        $wgGroupPermissions['*']['edit'] = true;
} else {
wfLoadExtension("Lockdown");

        $wgGroupPermissions['*']['read'] = true;
        $wgGroupPermissions['*']['edit'] = false;
        $wgGroupPermissions['user']['read'] = true;
        $wgGroupPermissions['user']['edit'] = true;

        $wgNamespacePermissionLockdown[NS_L2]['*'] = [ 'tech-L2', 'tech-L3' ];
        $wgNamespacePermissionLockdown[NS_L3]['*'] = [ 'tech-L3' ];
}

However we have recently noticed that this intermittently breaks switching between visualeditor and source editor.

I started reading through all the aforementioned bugs, plus not a few more, and realised that this work around should no longer be necessary, however I can't get cookie forwarding to work for Parsoid JS, Restbase and VisualEditor in 1.34. Is there any chance anyone has as sample working config they are willing to share (restbase, parsoid and localsettings).

In particular I am interested to know if there are any flags that need to be added to restbase or parsoid config for cookie forwarding?

I have enabled (as well as the standard config) the following in LocalSettings:

$wgVirtualRestConfig['modules']['parsoid']['forwardCookies'] = true;
$wgVirtualRestConfig['modules']['global']['forwardCookies'] = true;
$wgVirtualRestConfig['modules']['restbase']['forwardCookies'] = true;
$wgVisualEditorParsoidForwardCookies = true;

Are there any other flags I need to set in restbase, parsoid or LocalSettings, currently the only configuration I have relevant to cookie forwarding is in LocalSettings.php?

In our examples we can't edit the L2 and L3 namespaces. Minimum configs i could reproduce on below if anyone is feeling helpful.

Thanks in advance,

Caleb

Error in parsoid logs

{"name":"parsoid","hostname":"testwiki.wiki.internal","pid":1173,"level":60,"logType":"fatal/request","wiki":"wiki$0","title":"L2:Test","oldId":null,"reqId":"8b724680-f202-11ea-ae20-13b87b0d14d7","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36","msg":"API response Error for TemplateRequest: request=; error={\"code\":\"accessdenied\",\"info\":\"You are not allowed to view L2:Test.\",\"*\":\"See http://testwiki.wiki.internal/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes.\"}","stack":"Error: API response Error for TemplateRequest: request=; error={\"code\":\"accessdenied\",\"info\":\"You are not allowed to view L2:Test.\",\"*\":\"See http://testwiki.wiki.internal/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes.\"}\n at TemplateRequest.ApiRequest._errorObj (/var/lib/parsoid/lib/mw/ApiRequest.js:342:9)\n at TemplateRequest._handleJSON (/var/lib/parsoid/lib/mw/ApiRequest.js:554:16)\n at TemplateRequest.ApiRequest._logWarningsAndHandleJSON (/var/lib/parsoid/lib/mw/ApiRequest.js:447:7)\n at TemplateRequest.ApiRequest._handleBody (/var/lib/parsoid/lib/mw/ApiRequest.js:483:7)\n at TemplateRequest.ApiRequest._requestCB (/var/lib/parsoid/lib/mw/ApiRequest.js:420:8)\n at Request._callback (/var/lib/parsoid/lib/mw/ApiRequest.js:332:35)\n at Request.self.callback (/var/lib/parsoid/node_modules/request/request.js:185:22)\n at Request.emit (events.js:315:20)\n at Request.<anonymous> (/var/lib/parsoid/node_modules/request/request.js:1154:10)\n at Request.emit (events.js:315:20)\n at IncomingMessage.<anonymous> (/var/lib/parsoid/node_modules/request/request.js:1076:12)\n at Object.onceWrapper (events.js:421:28)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (_stream_readable.js:1220:12)\n at processTicksAndRejections (internal/process/task_queues.js:84:21)","longMsg":"API response Error for TemplateRequest: request=; error={\"code\":\"accessdenied\",\"info\":\"You are not allowed to view L2:Test.\",\"*\":\"See http://testwiki.wiki.internal/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes.\"}","levelPath":"fatal/request","time":"2020-09-08T18:38:53.628Z","v":0}

Chrome console log showing 500 error

load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:141 JQMIGRATE: Migrate is installed with logging active, version 3.0.1 VM1016:150 This page is using the deprecated ResourceLoader module "jquery.tabIndex". (anonymous) @ VM1016:150 runScript @ load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:13 execute @ load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:15 doPropagation @ load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:7 requestIdleCallback (async) requestPropagation @ load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:8 setAndPropagate @ load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:8 implement @ load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:20 (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:1 load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:130 GET http://testwiki.wiki.internal/testwiki.wiki.internal/v1/page/html/L2%3ATest?redirect=false&stash=true 500 send @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:130 ajax @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:125 jQuery.ajax @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:144 requestParsoidData @ VM1016:148 requestPageData @ VM1016:145 (anonymous) @ VM1016:125 mightThrow @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:48 process @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 setTimeout (async) (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 fire @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:45 add @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:46 (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:50 jQuery.Deferred @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:152 then @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 activateTarget @ VM1016:125 activatePageTarget @ VM1016:127 activateVe @ VM1016:135 onEditTabClick @ VM1016:134 dispatch @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:69 elemData.handle @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:65 VM1016:148 RESTBase load failed: error (anonymous) @ VM1016:148 mightThrow @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:48 process @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 setTimeout (async) (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 fire @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:45 fireWith @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:47 fire @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:47 fire @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:45 fireWith @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:47 done @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:126 (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:130 load (async) send @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:130 ajax @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:125 jQuery.ajax @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:144 requestParsoidData @ VM1016:148 requestPageData @ VM1016:145 (anonymous) @ VM1016:125 mightThrow @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:48 process @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 setTimeout (async) (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 fire @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:45 add @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:46 (anonymous) @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:50 jQuery.Deferred @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:152 then @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:49 activateTarget @ VM1016:125 activatePageTarget @ VM1016:127 activateVe @ VM1016:135 onEditTabClick @ VM1016:134 dispatch @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:69 elemData.handle @ load.php?lang=en&modules=jquery&skin=vector&version=1xgm5:65

LocalSettings.php

<?php ###################################################################################################################################### #CORE ###################################################################################################################################### $wgSitename = "testwiki"; $wgSecretKey = "c5ded9cb55cc68669f706ad0fffbb9e7aba3e7b7f5664b2c7448259e274d9dd6"; $wgUpgradeKey = "11640ab816b1bb22"; #Short URLs and Search Config $wgScriptPath = ""; $wgArticlePath = "/$1"; $wgUsePathInfo = true; $wgScriptExtension = ".php"; $wgServer = "http://testwiki.wiki.internal"; $wgResourceBasePath = $wgScriptPath; ###################################################################################################################################### #DATABASE ###################################################################################################################################### $wgDBtype = "mysql"; $wgDBserver = "mariadb"; $wgDBname = "wikis"; $wgDBuser = "wikisql"; $wgDBpassword = "c493905d0e184ffd23e4c53420dc9cb6d557892d"; $wgDBprefix = ""; $wgDBTableOptions = "ENGINE=InnoDB, DEFAULT CHARSET=binary"; $wgDBmysql5 = false; ###################################################################################################################################### #GENERAL ###################################################################################################################################### // Skin wfLoadSkin( 'Vector' ); $wgDefaultSkin='vector'; ###################################################################################################################################### #Additional Mediawiki User Groups: tech-* ###################################################################################################################################### ###Tech/Engineering Groups (CREATE) $wgGroupPermissions['tech-L2'] = $wgGroupPermissions['user']; $wgGroupPermissions['tech-L3'] = $wgGroupPermissions['user']; ###################################################################################################################################### #Custom Namespaces ###################################################################################################################################### define('NS_L2' , 3002); define('NS_L2_TALK' , 3003); define('NS_L3' , 3004); define('NS_L3_TALK' , 3005); $wgExtraNamespaces[NS_L2] = 'L2'; $wgExtraNamespaces[NS_L2_TALK] = 'L2 Talk'; $wgExtraNamespaces[NS_L3] = 'L3'; $wgExtraNamespaces[NS_L3_TALK] = 'L3 Talk'; ###################################################################################################################################### #Lockdown ###################################################################################################################################### wfLoadExtension("Lockdown"); $wgGroupPermissions['*']['read'] = true; $wgGroupPermissions['*']['edit'] = false; $wgGroupPermissions['user']['read'] = true; $wgGroupPermissions['user']['edit'] = true; $wgNamespacePermissionLockdown[NS_L2]['*'] = [ 'tech-L2', 'tech-L3' ]; $wgNamespacePermissionLockdown[NS_L3]['*'] = [ 'tech-L3' ]; ##################################################################################################################################### #VisualEditor, Restbase and Parsoid ###################################################################################################################################### ##Global Rest Config $wgVirtualRestConfig = [ 'modules' => [], 'global' => [ 'domain' => 'testwiki.wiki.internal', 'timeout' => 360, 'forwardCookies' => true, 'HTTPProxy' => null, ] ]; # Rest config for Parsoid $wgVirtualRestConfig['modules']['parsoid'] = array( 'url' => 'http://localhost:8142', 'domain' => 'testwiki.wiki.internal', 'forwardCookies' => true, 'restbaseCompat' => false ); ## Rest Config for Restbase $wgVirtualRestConfig['modules']['restbase'] = array( 'url' => 'http://localhost:7231', 'domain' => 'testwiki.wiki.internal', 'forwardCookies' => true, 'parsoidCompat' => false ); #VisualEditor wfLoadExtension('VisualEditor'); $wgVisualEditorParsoidAutoConfig = false; $wgVisualEditorAllowLossySwitching = false; $wgVisualEditorFullRestbaseURL = 'http://testwiki.wiki.internal/testwiki.wiki.internal/'; $wgVisualEditorRestbaseURL = 'http://localhost:7231/testwiki.wiki.internal/v1/page/html/'; $wgVisualEditorParsoidForwardCookies = true; $wgDefaultUserOptions['visualeditor-editor'] = "visualeditor"; $wgDefaultUserOptions['visualeditor-enable'] = 1; $wgDefaultUserOptions['visualeditor-newwikitext'] = 1; $wgVisualEditorAutoAccountEnable = true; $wgVisualEditorShowBetaWelcome = false; $wgVisualEditorEnableDiffPage = true; $wgVisualEditorEnableWikitext = true; $wgVisualEditorNamespaces = array_merge($wgContentNamespaces, array( NS_TALK, NS_USER, NS_USER_TALK, NS_L2, NS_L2_TALK, NS_L3, NS_L3_TALK) ); $wgVisualEditorAvailableNamespaces = array_fill_keys($wgVisualEditorNamespaces, true);

config.yaml for restbase

# # Simple RESTBase config for Mediawiki Container # https://www.mediawiki.org/wiki/RESTBase/Installation # # - cassandra DB # - parsoid at http://localhost:8142 # - wiki at http://testwiki.wiki.internal/api.php # # - proxied via nginx, available via # - http://hostname/api/rest_v1/ # services: - name: restbase module: hyperswitch conf: port: 7231 salt: 988881adc9fc3655077dc2d4d757d480b5ea0e11 default_page_size: 125 user_agent: RESTBase ui_name: RESTBase ui_url: https://www.mediawiki.org/wiki/RESTBase ui_title: RESTBase docs spec: x-request-filters: - path: lib/security_response_header_filter.js - path: lib/normalize_headers_filter.js x-sub-request-filters: - type: default name: http options: allow: - pattern: http://localhost/api.php forward_headers: true - pattern: http://localhost:8142 forward_headers: true - pattern: http://testwiki.wiki.internal/api.php forward_headers: true - pattern: http://testwiki.wiki.internal:8142 forward_headers: true - pattern: /^https?:\/\// paths: /{domain:testwiki.wiki.internal}/{api:v1}: x-modules: - spec: info: version: 1.0.0 title: Wikimedia REST API description: Welcome to your RESTBase API. x-route-filters: - path: ./lib/normalize_title_filter.js options: redirect_cache_control: 's-maxage=0, max-age=86400' paths: /page: x-modules: - path: v1/content.yaml options: response_cache_control: 's-maxage=0, max-age=86400' - path: v1/common_schemas.yaml # Doesn't really matter where to mount it. /transform: x-modules: - path: v1/transform.yaml /media: x-modules: #- path: v1/mathoid.yaml # options: # host: http://localhost:10042 /{domain:testwiki.wiki.internal}/{api:sys}: x-modules: - path: projects/proxy.yaml options: backend_host_template: '{{"/{domain}/sys/legacy"}}' - spec: paths: /table: x-modules: - path: sys/table.js options: conf: version: 1 backend: cassandra hosts: - cassandradb pool_idle_timeout: 20000 retry_delay: 250 retry_limit: 10 show_sql: false keyspace: system defaultConsistency: localOne localDc: datacenter1 datacenters: - datacenter1 storage_groups: - name: local domains: /./ /legacy/key_value: x-modules: - path: sys/key_value.js /legacy/page_revisions: x-modules: - path: sys/page_revisions.js /post_data: x-modules: - path: sys/post_data.js /action: x-modules: - path: sys/action.js options: apiUriTemplate: "{{'http://localhost/api.php'}}" baseUriTemplate: "{{'http://localhost:7231/{domain}/v1'}}" /page_save: x-modules: - path: sys/page_save.js /events: x-modules: - path: sys/events.js /parsoid: x-modules: - path: sys/parsoid.js options: parsoidHost: http://localhost:8142 grace_ttl: 1000000 #/mathoid: # x-modules: # - path: sys/mathoid.js # options: # host: http://localhost:10042 # Finally, a standard service-runner config. info: name: restbase logging: name: restbase level: warn streams: - type: stdout num_workers: 0

config.yaml for parsoid
worker_heartbeat_timeout: 300000 num_workers: 2 logging: name: parsoid level: warn services: - module: lib/index.js entrypoint: apiServiceWorker conf: mwApis: - uri: 'http://localhost/api.php' domain: 'testwiki.wiki.internal' serverPort: 8142

nginx config

# nginx http config for Mediawiki server # # this config-file is updated during container-startup # 'testwiki.wiki.internal' is replaced globally with the FQDN of the Wiki # ############################## ### UPSTREAMS ############################## # restbase upstream restbase { server 127.0.0.1:7231; keepalive 32; } map $request_uri $restbasequeryapi { default "xx"; "~/api/rest_v1/(?<xrestbasequery>.*)$" "$xrestbasequery"; } map $request_uri $restbasequerylegacy { default "xx"; "~/testwiki.wiki.internal/v1/(?<xrestbasequery>.*)$" "$xrestbasequery"; } map $request_uri $imageDownloadAttachment { default ""; "~/images/.*(\?|&)download(=|&).*$" "attachment"; } ############################## ### MAIN HTTP SERVER ############################## server { ############################## ### HTTP Globals ############################## server_name _; listen 80; root /var/lib/mediawiki; client_max_body_size 100m; fastcgi_connect_timeout 10s; set $no_cache "0"; ############################## ### Proxy Restbase, Mathoid ############################## location /api/rest_v1/ { proxy_max_temp_file_size 0; proxy_buffer_size 64k; proxy_buffers 4 64k; proxy_http_version 1.1; proxy_pass http://restbase/testwiki.wiki.internal/v1/$restbasequeryapi; set $no_cache "1"; } location /testwiki.wiki.internal/v1/ { proxy_max_temp_file_size 0; proxy_buffer_size 64k; proxy_buffers 4 64k; proxy_http_version 1.1; proxy_pass http://restbase/testwiki.wiki.internal/v1/$restbasequerylegacy; set $no_cache "1"; } location /api/mathoid/ { proxy_max_temp_file_size 0; proxy_buffer_size 64k; proxy_buffers 4 64k; proxy_http_version 1.1; proxy_pass http://127.0.0.1:10042/; set $no_cache "1"; } location /rest.php/ { proxy_max_temp_file_size 0; proxy_buffer_size 64k; proxy_buffers 4 64k; proxy_http_version 1.1; try_files $uri $uri/ /rest.php?$query_string; set $no_cache "1"; } # Bypass cache if flag is set fastcgi_no_cache $no_cache; fastcgi_cache_bypass $no_cache; proxy_cache_bypass $no_cache; proxy_no_cache $no_cache; ############################## ### General Requests ############################## location ~ \.htaccess { deny all; } location ^~ /install-mw.sh { return 403; } location ^~ /update-mw.sh { return 403; } location ^~ /runjobs-mw.sh { return 403; } location ^~ /bootstrap/1_env.php { return 403; } location ^~ /bootstrap.php { # extended php-timeout ( e.g. for update.php ) fastcgi_read_timeout 200s; fastcgi_send_timeout 200s; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_pass unix:/run/php-fpm.sock; include fastcgi_params; } location /images { # enable CORS access to this dir from other origins add_header Access-Control-Allow-Origin "*"; add_header Access-Control-Expose-Headers "Age,Date,X-Cache,X-Varnish"; # allow downloads via Media Viewer add_header Content-Disposition $imageDownloadAttachment; # the hosting root-dir is updated/set during container startup root /wiki-shared/testwiki/storage; # set the 'Expires' header for 90days of image-caching to reverse-proxies # proxies often have their own expiry-lifetime for hot/cold cache items expires 90d; } location / { try_files $uri @rewrite; } location ^~ /mw-config/ { internal; } location @rewrite { rewrite ^/(.*)$ /index.php; } location ^~ /maintenance/ { internal; } location = /_.gif { expires max; empty_gif; } location ^~ /cache/ { internal; } ############################## ### PHP Config ############################## location ~ ^/system/nginxping$ { access_log off; return 200 'pong :-)'; } location ~ ^/system/phpstatus { fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_pass unix:/run/php-fpm.sock; include fastcgi_params; fastcgi_index index.php; } location ~ \.php$ { fastcgi_pass unix:/run/php-fpm.sock; fastcgi_split_path_info ^(.+\.php)(/.*)$; include fastcgi_params; fastcgi_param PATH_TRANSLATED $document_root$fastcgi_script_name; fastcgi_param HTTPS off; fastcgi_index index.php; } }

Since writing I've tested in versions 1.31, 1.32, 1.33, 1.34 and 1.35 of mediawiki, all versions have the same issue, except when using parsoid php implementation in 1.35, in the later case there is no longer a need to use the workaround not to load lockdown when the request comes from localhost.

But unless I am doing something wrong it is hard to believe this was fixed in T148582 but maybe this is a regression..