Talk:Parsoid

Jump to: navigation, search

About this board

By clicking "Add topic", you agree to our Terms of Use and agree to irrevocably release your text under the CC BY-SA 3.0 License and GFDL
Facerafter (talkcontribs)

Trying to setup parsoid on a new server, but keep running into this error when trying to start the parsoid service.

From the log file:

https://pastebin.com/mKY4N52s

Arlolra (talkcontribs)

How did you install Parsoid?

Facerafter (talkcontribs)

Followed the documentation provided to me.

https://gist.github.com/anonymous/8cffa7084d74b6f06e4b6af3a535bf77

This is for a Ubuntu 16.04 server.

We used the same for our live server and it works fine but now we are setting up a dev/test server and it suddenly doesn't work...

Arlolra (talkcontribs)

Is there any reason you aren't using the debian packages we provide? See Parsoid/Setup#Ubuntu .2F Debian

The instructions you provided are a little confusing. They point to a method that is trying to avoid using npm by cloning the deploy repo, which has the necessary dependencies already, and then invokes npm install anyways.

From the looks of it, one of your copy steps is putting things in the wrong place.

Facerafter (talkcontribs)

Trying to reinstall via the link you sent actually showed me the problem as it saw the config.yaml already existed and showed the differences.

Apparently in our old version we had: 'module: /lib/index.js' but it should be 'module: ../src/lib/index.js'

So I aborted the installation and fixed the file. But VE doesn't do anything. The last line on parsoid.log does say 'Startup finished' though.

Arlolra (talkcontribs)

Can you clarify what you mean by VE doesn't do anything? What does the VE section of your LocalSettings.php look like?

Facerafter (talkcontribs)

As in when clicking on the edit icon for VE the browser starts to load and finishes loading but nothing changes. I stay on the same page in 'viewing' mode.

Current config:

https://gist.github.com/anonymous/a1c652a4da3951a08ecc45628f2a305f

Arlolra (talkcontribs)

Is Parsoid being started on 8142 or 8000?

Facerafter (talkcontribs)

8142

Arlolra (talkcontribs)

And what does curl http://localhost:8142/version give you?

Facerafter (talkcontribs)

{"name":"parsoid","version":"0.7.1"}

Arlolra (talkcontribs)

So, Parsoid seems up at least.

Are there any errors in the browser developer console?

Given that the last line in your parsoid.log is the startup message, it doesn't seem like VE is making any requests to it.

Facerafter (talkcontribs)

Yeah two errors but those were already in the console before trying to edit a page with VE

Exception in module-execute in module site:

and

ReferenceError: Waves is not defined ReferenceError: Waves is not defined

No new errors appear when trying to use it.

Arlolra (talkcontribs)

Do you have any other extensions installed? Try disabling them all except VE and verify that doesn't work first.

Facerafter (talkcontribs)

After disabling them all except VE and then trying to use VE results in an error in the developer console.

GET <URL>/api.php?action=visualeditor&format=json&paction=parse&page=FAQ&uselang=en 500 (Internal Server Error)

After which it keeps giving me a browser msg/notifcation/popup thing with just 'http' which keeps repeating with the error mentioned above.

Arlolra (talkcontribs)

That seems like an improvement. Is there anything in the Parsoid logs now?

Facerafter (talkcontribs)

Btw, I edited my response above (damn you are fast)

But the parsoid logs says nothing new.

Arlolra (talkcontribs)

Since the server is returning 500, you should have server error logs you can look at to figure out what's going on.

Arlolra (talkcontribs)

To be clear, I mean whatever is serving mediawiki. Not parsoid.

Facerafter (talkcontribs)

Tried complete parsoid reinstallation. But encountered the exact same error as before the reinstallation

Nothing in the apache2 error log.

Arlolra (talkcontribs)

Is this a public or private wiki? If MediaWiki is returning a 500 error, you should have something in the apache error logs.

Facerafter (talkcontribs)

Public. The developers console is displaying a 500 error when trying to use VE, the site itself works fine.

Arlolra (talkcontribs)

Can you share the url?

Maybe try enabling some debugging flags in MediaWiki to generate more informative errors,

// error_reporting( -1 );

// ini_set( 'log_errors', 1 );

// ini_set( 'error_log', '/tmp/php-error.log' );

Facerafter (talkcontribs)

https://scwdev.czen.me

Ffs, I still think there is something wrong with my parsoid installation...

So I setup a local vm using a different install method (This one Parsoid/Setup#Ubuntu .2F Debian ) but than the /etc/mediawiki/parsoid directory is completely empty. Not config.yaml or anything...

It seems like om doing something wrong everytime... I just don't know what.

Arlolra (talkcontribs)

I don't think there's anything wrong with the Parsoid setup above.

Can you access, say, curl http://localhost:8142/localhost/v3/page/html/Magnus/21579? If so, Parsoid is fine. We established that above.

I'm noticing on your wiki that it's telling me, "You do not have permission to edit this page, for the following reason".

Can you try this https://www.mediawiki.org/wiki/Extension:VisualEditor#Linking_with_Parsoid_in_private_wikis ?

Also, there're still an exception, "Waves is not defined". You should disable all other extensions until this is resolved.

Reply to "Cannot find module"

Can Parsoid and my website share the same set of SSL certificates?

5
Summary by Arlolra

Use localhost for communication on the same machine.

MagaFun (talkcontribs)

Dear supporters,

I am getting curl error 35 when using VisualEditor under SSL connection, and I had my HSTS enabled. So I currently am investigating my stunnel configuration. Can Parsoid and my website use the same set of SSL certificate? Or do I have to generate a new set?

Below is my config file.

cert = /etc/letsencrypt/live/mydomian.com/cert.pem

key = /etc/letsencrypt/live/mydomian.com/privkey.pem

CAfile = /etc/letsencrypt/live/mydomian.com/chain.pem

[www]

accept  = 8143

connect = 8142

LocalSettings.php config:

$wgVirtualRestConfig['modules']['parsoid'] = array(

// URL to the Parsoid instance

// Use port 8142 if you use the Debian package

'url' => 'https://mydomian.com:8142',

// Parsoid "domain", see below (optional)

'domain' => 'mydomian.com',

// Parsoid "prefix", see below (optional)

'prefix' => 'mydomian.com'

);

Arlolra (talkcontribs)

Yes, they can share the same TLS cert, if needed.

MagaFun (talkcontribs)

Do you happened to know any possible reasons that might produce error 35? I am out of clues. Or is this error logged more detailly somewhere else?

Arlolra (talkcontribs)

Well, first things first. Did you get Parsoid working with VE before trying to enable TLS? Are Parsoid and MediaWiki hosted on the same machine?

MagaFun (talkcontribs)

I did get Visual Editor working without TLS. Problem starts with TLS. And they are hosted on the same machine.

Summary by Arlolra

Parsoid v0.7.x requires at least node.js v4.x

217.228.174.140 (talkcontribs)

Got an error:

The following packages have unmet dependencies:

 parsoid : Depends: nodejs (>= 4.0.0) but 0.10.29~dfsg-2 is to be installed

=> this was fixed in an version, published a day later...

Legoktm (talkcontribs)

I assume you're running Debian Jessie? If you enable Debian Backports (see https://backports.debian.org/Instructions/) you can get an updated nodejs version that is 4.7.2. (Might need a manual sudo apt-get install nodejs -t jessie-backports to work.)

Arlolra (talkcontribs)

That's correct though, Parsoid requires node.js >= v4.x

npm error with the last debian package of parsoid (0.7)

3
Summary by Arlolra

Parsoid v0.7.x requires at least node.js v4.x

130.223.9.34 (talkcontribs)

When I launch parsoid service, I have a nodejs error :

{"name":"parsoid","hostname":"myhostname","pid":32579,"level":60,"err":{"message":"Object function Object() { [native code] } has no method 'assign'","name":"TypeError","stack":"TypeError: Object function Object() { [native code] } has no method 'assign'\n    at /usr/lib/parsoid/node_modules/service-runner/lib/master.js:209:33\n    at Promise._execute (/usr/lib/parsoid/node_modules/bluebird/js/release/debuggability.js:300:9)\n    at Promise._resolveFromExecutor (/usr/lib/parsoid/node_modules/bluebird/js/release/promise.js:481:18)\n    at new Promise (/usr/lib/parsoid/node_modules/bluebird/js/release/promise.js:77:14)\n    at Master._startWorkers (/usr/lib/parsoid/node_modules/service-runner/lib/master.js:206:16)\n    at /usr/lib/parsoid/node_modules/service-runner/lib/master.js:107:21\n    at tryCatcher (/usr/lib/parsoid/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/usr/lib/parsoid/node_modules/bluebird/js/release/promise.js:510:31)\n    at Promise._settlePromise (/usr/lib/parsoid/node_modules/bluebird/js/release/promise.js:567:18)\n    at Promise._settlePromise0 (/usr/lib/parsoid/node_modules/bluebird/js/release/promise.js:612:10)\n    at Promise._settlePromises (/usr/lib/parsoid/node_modules/bluebird/js/release/promise.js:691:18)\n    at Async._drainQueue (/usr/lib/parsoid/node_modules/bluebird/js/release/async.js:133:16)\n    at Async._drainQueues (/usr/lib/parsoid/node_modules/bluebird/js/release/async.js:143:10)\n    at Async.drainQueues (/usr/lib/parsoid/node_modules/bluebird/js/release/async.js:17:14)\n    at process._tickCallback (node.js:419:13)","levelPath":"fatal/service-runner/unhandled"},"msg":"Object function Object() { [native code] } has no method 'assign'","time":"2017-04-05T12:16:05.577Z","v":0}

Unhandled rejection Error: Timeout waiting for config in worker 32585

    at null._onTimeout (/usr/lib/parsoid/node_modules/service-runner/lib/worker.js:33:20)

    at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

Arlolra (talkcontribs)

Looks like you're missing ES6 Object.assign()

Parsoid 0.6.1 was the last version to support node.js v0.10.x

Starting with 0.7.0, you'll need to upgrade node to v4.x

node v0.10.x is end-of-life, and no longer maintained upstream

See https://github.com/nodejs/LTS

130.223.9.34 (talkcontribs)

Many thanks, it was the problem: I update nodejs to v6, and it works now.

Update needed in the Windows setup guide to avoid damaging npm install

2
Summary by SSastry (WMF)

Updated docs

186.28.53.22 (talkcontribs)

As per https://github.com/npm/npm/issues/16037

Using npm@next will corrupt the npm install and cause it to become unable to install any package. The proper command is npm@latest

SSastry (WMF) (talkcontribs)

Done. Thanks for the heads up.

Parsoid Returns Invalid Domain

7
Summary by Sorvis

The Parsoid configuration file was incorrect because it had a commented line that needed to be uncommented:

# - # This is the only required parameter,

that should be,

- # This is the only required parameter,

Sorvis (talkcontribs)

Hello,

I am running an internal company wiki that allows read to anyone, but only write to users. I upgraded my MediaWiki instance from 1.24 to 1.28. Parsoid and VisualEditor used to work. After updating I saw that Parsoid didn't work anymore so I upgraded the Parsoid service from 0.1.8 to 0.6.1. Visual Editor still does nothing when I click on the edit link. The version of nodejs is 0.10.37

Going to http://mydomain:8142/ takes me to the welcome page for Parsoid. When I got to http://mydomain:8142/localhost/v3/page/html/Main_Page/ it says:

 Invalid domain: localhost

Doing curl -L http://localhost/api.php returns the api's page.

The parsoid.log file looks like this:

{
	"type": "log",
	"name": "parsoid",
	"hostname": "mydomain",
	"pid": 2643,
	"level": 60,
	"err": {
		"message": "Invalid domain: localhost",
		"name": "../src/lib/index.js",
		"stack": "../src/lib/index.js: Invalid domain: localhost\n    at errOut (/usr/lib/parsoid/src/lib/api/routes.js:30:13)\n    at routes.v3Middle (/usr/lib/parsoid/src/lib/api/routes.js:94:11)\n    at Layer.handle [as handle_request] (/usr/lib/parsoid/node_modules/express/lib/router/layer.js:95:5)\n    at next (/usr/lib/parsoid/node_modules/express/lib/router/route.js:131:13)\n    at Route.dispatch (/usr/lib/parsoid/node_modules/express/lib/router/route.js:112:3)\n    at Layer.handle [as handle_request] (/usr/lib/parsoid/node_modules/express/lib/router/layer.js:95:5)\n    at /usr/lib/parsoid/node_modules/express/lib/router/index.js:277:22\n    at param (/usr/lib/parsoid/node_modules/express/lib/router/index.js:349:14)\n    at param (/usr/lib/parsoid/node_modules/express/lib/router/index.js:365:14)\n    at param (/usr/lib/parsoid/node_modules/express/lib/router/index.js:365:14)\n    at param (/usr/lib/parsoid/node_modules/express/lib/router/index.js:365:14)\n    at param (/usr/lib/parsoid/node_modules/express/lib/router/index.js:365:14)\n    at Function.process_params (/usr/lib/parsoid/node_modules/express/lib/router/index.js:410:3)\n    at next (/usr/lib/parsoid/node_modules/express/lib/router/index.js:271:10)\n    at /usr/lib/parsoid/src/lib/api/ParsoidService.js:159:3\n    at Layer.handle [as handle_request] (/usr/lib/parsoid/node_modules/express/lib/router/layer.js:95:5)",
		"httpStatus": 404,
		"suppressLoggingStack": true,
		"levelPath": "fatal/request"
	},
	"msg": "Invalid domain: localhost",
	"time": "2017-03-09T13:28:39.611Z",
	"v": 0
}

Here is my pastebin for my config.yaml http://pastebin.com/T8CLbx5p

Here is the section of my LocalSettings.php where I setup Visual Editor

#visual editing
wfLoadExtension( 'Parsoid' );
wfLoadExtension( 'VisualEditor' );
#require_once("$IP/extensions/VisualEditor/VisualEditor.php");
$wgVirtualRestConfig['modules']['parsoid'] = array(
	// URL to the Parsoid instance
	// Use port 8142 if you use the Debian package
	'url' => 'http://localhost:8142',
	// Parsoid "domain", see below (optional)
	'domain' => 'localhost',
	// Parsoid "prefix", see below (optional)
	'prefix' => 'localhost'
);

// This feature requires a non-locking session store. The default session store will not work and
// will cause deadlocks (connection timeouts from Parsoid) when trying to use this feature.
$wgSessionsInObjectCache = true;

// Forward users' Cookie: headers to Parsoid. Required for private wikis (login required to read).
// If the wiki is not private (i.e. $wgGroupPermissions['*']['read'] is true) this configuration
// variable will be ignored.
//
// WARNING: ONLY enable this on private wikis and ONLY IF you understand the SECURITY IMPLICATIONS
// of sending Cookie headers to Parsoid over HTTP. For security reasons, it is strongly recommended
// that $wgVirtualRestConfig['modules']['parsoid']['url'] be pointed to localhost if this setting is enabled.
$wgVirtualRestConfig['modules']['parsoid']['forwardCookies'] = true;

// URL to the Parsoid instance
// MUST NOT end in a slash due to Parsoid bug
// Use port 8142 if you use the Debian package
#$wgVisualEditorParsoidURL = 'http://localhost:8142';

// Namespaces to enable VisualEditor in
#$wgVisualEditorNamespaces = array_merge( $wgContentNamespaces, array( NS_USER, NS_HELP, NS_TEMPLATE, NS_CATEGORY ) );

//Default user options

// Enable by default for everybody
$wgDefaultUserOptions['visualeditor-enable'] = 1;

Any suggestions of anything else to try? Thank you very much for the help.

Arlolra (talkcontribs)

The config.yaml looks ok. Based on the error though, it looks like it isn't actually loading that file. How did you install Parsoid? Did you restart Parsoid after editing the config? Try and confirm that it is indeed using that file, maybe by introducing some broken syntax there and looking for a new error.

Sorvis (talkcontribs)

Thank you for the suggestion Arioira. I put a garbage line in the config then restarted the service then I saw it place an error in the log file so I think it did at least try to parse the config file.

Parsoid was installed from the wikimedia package repository. This is the package it was installed from: https://releases.wikimedia.org/debian/pool/main/p/parsoid/parsoid_0.6.1all_all.deb

Arlolra (talkcontribs)

On second looks, I noticed you have this in the config file,

#- # This is the only required parameter,

that should be,

- # This is the only required parameter,
Sorvis (talkcontribs)

Good news that makes http://mydomain:8142/localhost/v3/page/html/Main_Page/ return data now!!!! Thank you very much. :-)

OK, problem number 2. Visual Editor just redirects to http://mydomain/index.php?title=Main_Page&veaction=edit when I click edit. It does not look like any new Parsoid logs are being created when this redirect happens. I checked out the wikimedia debug log and didn't see any errors in it. When I googled my problem most people seem to say they forgot to run the submodule update command on the VisualEditor repository, but I already ran that in this case.

Are there any other steps I could take to troubleshoot the problem?

Arlolra (talkcontribs)

Hmm, did you check for any errors in the your browser's development console? Otherwise, maybe temporarily disable any other extensions you have installed to see if that's preventing it from loading.

Lastly, you might get a better answer at https://www.mediawiki.org/wiki/Talk:VisualEditor

Sorvis (talkcontribs)

OK I'll give those suggestions are try. Thank you very much Arlolra. I documented the Parsoid configuration error you helped me find here https://www.mediawiki.org/wiki/Parsoid/Troubleshooting#Invalid_Domain_Error for others to use in the future.

Summary by Arlolra

Orthogonal to Parsoid

John Harry Lau (talkcontribs)

Cmd error:

C:\windows\system32>npm install parsoid

> contextify@0.1.15 install C:\windows\system32\node_modules\contextify

> node-gyp rebuild

C:\windows\system32\node_modules\contextify>if not defined npm_config_node_gyp (

node "C:\Users\Lau\AppData\Roaming\npm\node_modules\npm\bin\node-gyp-bin\\..\..\

node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "" rebuild )

gyp: binding.gyp not found (cwd: C:\windows\system32\node_modules\contextify) wh

ile trying to load binding.gyp

gyp ERR! configure error

gyp ERR! stack Error: `gyp` failed with exit code: 1

gyp ERR! stack     at ChildProcess.onCpExit (C:\Users\Lau\AppData\Roaming\npm\no

de_modules\npm\node_modules\node-gyp\lib\configure.js:305:16)

gyp ERR! stack     at emitTwo (events.js:106:13)

gyp ERR! stack     at ChildProcess.emit (events.js:191:7)

gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_proces

s.js:215:12)

gyp ERR! System Windows_NT 6.1.7601

gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Lau\\AppData\

\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "reb

uild"

gyp ERR! cwd C:\windows\system32\node_modules\contextify

gyp ERR! node -v v6.7.0

gyp ERR! node-gyp -v v3.4.0

gyp ERR! not ok

C:\windows\system32

`-- parsoid@0.5.2

`-- pegjs@0.8.0  (git+https://github.com/tstarling/pegjs.git#9162b1e114e41992d

d0fdafa24d2574a0b8a836a)

npm WARN enoent ENOENT: no such file or directory, open 'C:\windows\system32\pac

kage.json'

npm WARN system32 No description

npm WARN system32 No repository field.

npm WARN system32 No README data

npm WARN system32 No license field.

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: contextify@0.1.15 (node_modules\

contextify):

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: contextify@0.1.15 install: `node

-gyp rebuild`

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

Arlolra (talkcontribs)

Can you try doing this somewhere other than C:\windows\system32?

Like C:\Users\[USERNAME]

John Harry Lau (talkcontribs)

I still have error there

Arlolra (talkcontribs)

Can you try with npm install parsoid --no-optional?

Arlolra (talkcontribs)

We've removed the html5 library as a dependency, so contextify should no longer be an issue, even without --no-optional.

John Harry Lau (talkcontribs)

OMG. I think i seleted worng option installing git and i also didn't set up the username and email. LoL

John Harry Lau (talkcontribs)

And also reinstalling node.js

John Harry Lau (talkcontribs)

Also i switch to git cmd but i got this error: npm WARN addRemoteGit Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5) npm WARN addRemoteGit git+https://github.com/wikimedia/content-type.git#47b2632d0a2ee79a7d67268e2f6621becd95d05b resetting remote C:\Users\Lau\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-wikimedia-content-type-git-47b2632d0a2ee79a7d67268e2f6621becd95d05b-0dff22de because of error: { Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5) npm WARN addRemoteGit killed: false, npm WARN addRemoteGit code: 1, npm WARN addRemoteGit signal: null, npm WARN addRemoteGit cmd: 'git -c core.longpaths=true config --get remote.origin.url' } npm WARN addRemoteGit Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5) npm WARN addRemoteGit git+https://github.com/arlolra/negotiator.git#0418ab4e9a665772b7e233564a4525c9d9a8ec3a resetting remote C:\Users\Lau\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-arlolra-negotiator-git-0418ab4e9a665772b7e233564a4525c9d9a8ec3a-164c7910 because of error: { Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5) npm WARN addRemoteGit killed: false, npm WARN addRemoteGit code: 1, npm WARN addRemoteGit signal: null, npm WARN addRemoteGit cmd: 'git -c core.longpaths=true config --get remote.origin.url' } npm WARN addRemoteGit Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Socket.<anonymous> (internal/child_process.js:334:11) npm WARN addRemoteGit at emitOne (events.js:96:13) npm WARN addRemoteGit at Socket.emit (events.js:188:7) npm WARN addRemoteGit at Pipe._handle.close [as _onclose] (net.js:501:12) npm WARN addRemoteGit git+https://github.com/tstarling/pegjs.git#9162b1e114e41992dd0fdafa24d2574a0b8a836a resetting remote C:\Users\Lau\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-tstarling-pegjs-git-9162b1e114e41992dd0fdafa24d2574a0b8a836a-96a2073a because of error: { Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Socket.<anonymous> (internal/child_process.js:334:11) npm WARN addRemoteGit at emitOne (events.js:96:13) npm WARN addRemoteGit at Socket.emit (events.js:188:7) npm WARN addRemoteGit at Pipe._handle.close [as _onclose] (net.js:501:12) npm WARN addRemoteGit killed: false, npm WARN addRemoteGit code: 1, npm WARN addRemoteGit signal: null, npm WARN addRemoteGit cmd: 'git -c core.longpaths=true config --get remote.origin.url' } npm WARN addRemoteGit Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5) npm WARN addRemoteGit git+https://github.com/gwicke/kad.git#f35971036f43814043245da82b12d035b7bbfd16 resetting remote C:\Users\Lau\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gwicke-kad-git-f35971036f43814043245da82b12d035b7bbfd16-48a8d777 because of error: { Error: Command failed: git -c core.longpaths=true config --get remote.origin.url npm WARN addRemoteGit npm WARN addRemoteGit at ChildProcess.exithandler (child_process.js:211:12) npm WARN addRemoteGit at emitTwo (events.js:106:13) npm WARN addRemoteGit at ChildProcess.emit (events.js:191:7) npm WARN addRemoteGit at maybeClose (internal/child_process.js:885:16) npm WARN addRemoteGit at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5) npm WARN addRemoteGit killed: false, npm WARN addRemoteGit code: 1, npm WARN addRemoteGit signal: null, npm WARN addRemoteGit cmd: 'git -c core.longpaths=true config --get remote.origin.url' }

Arlolra (talkcontribs)

Are you behind a corporate firewall blocking connections to https://github.com?

John Harry Lau (talkcontribs)

no.

Arlolra (talkcontribs)

Ok, well, that issue seems unrelated to installing Parsoid in particular.

You should first confirm that your system is setup to git clone repositories and that you can npm install git urls.

John Harry Lau (talkcontribs)

how can i do that?

Arlolra (talkcontribs)

Well, first try,

git clone https://github.com/tstarling/pegjs

then try,

npm install git+https://github.com/tstarling/pegjs

Parsoid installation with Visual Editor / mwoffliner.js: Error loading data from server: 404: parsoidserver-http: HTTP 404

8
Summary by Arlolra

The private _wikitext and _html routes require devAPI: true to your config.yaml

Escalatr (talkcontribs)

Hello,

I am trying to install Parsoid in order to use the https://github.com/kiwix/mwoffliner script for dumping my MediaWiki for offline usage in http://www.kiwix.org/ first. Using the Visual Editor would be also nice later.

The mediawiki is running inside an intranet at http://172.18.200.40/himtwiki/, I have disabled short URLs for now, it is a private wiki.

$wgGroupPermissions['*'    ]['read']            = false;
  • The system is running on openSUSE 13.1 (Bottle) (x86_64).
  • MediaWiki: 1.25
  • Parsoid: 0.6.1 / 2016-11-14
  • NodeJS: v4.2.4
  • I followed the steps from Parsoid/Developer Setup
  • Parsoid resides in /root/parsoid/ for testing now
  • Created config.yaml as following:
        # Configure Parsoid to point to your MediaWiki instances.
        mwApis:
        - # This is the only required parameter,
          # the URL of you MediaWiki API endpoint.
          uri: 'http://172.18.200.40/himtwiki/api.php'
          # The "domain" is used for communication with Visual Editor
          # and RESTBase.  It defaults to the hostname portion of
          # the `uri` property below, but you can manually set it
          # to an arbitrary string.
          domain: 'localhost'  # optional
          prefix: 'localhost'
  • Service is running node bin/server.js
    • Update: Command line output added
{"name":"parsoid","hostname":"server223","pid":793,"level":30,"levelPath":"info/service-runner","msg":"master(793) initializing 4 workers","time":"2017-02-07T13:08:11.107Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":802,"level":30,"levelPath":"info","msg":"loading ...","time":"2017-02-07T13:08:11.858Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":802,"level":30,"levelPath":"info","msg":"ready on :8000","time":"2017-02-07T13:08:11.879Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":807,"level":30,"levelPath":"info","msg":"loading ...","time":"2017-02-07T13:08:12.637Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":807,"level":30,"levelPath":"info","msg":"ready on :8000","time":"2017-02-07T13:08:12.656Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":812,"level":30,"levelPath":"info","msg":"loading ...","time":"2017-02-07T13:08:13.403Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":812,"level":30,"levelPath":"info","msg":"ready on :8000","time":"2017-02-07T13:08:13.422Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":817,"level":30,"levelPath":"info","msg":"loading ...","time":"2017-02-07T13:08:14.171Z","v":0}
{"name":"lib/index.js","hostname":"server223","pid":817,"level":30,"levelPath":"info","msg":"ready on :8000","time":"2017-02-07T13:08:14.189Z","v":0}
{"name":"parsoid","hostname":"server223","pid":793,"level":40,"levelPath":"warn/service-runner","msg":"Startup finished","time":"2017-02-07T13:08:14.190Z","v":0}

Visual Editor

  • I have also went through the Visual Editor installation, which is not working, I get the button Edit source for editing only (Visual Editor is actually enabled in the user preferences).
    • Update: $wgVisualEditorNamespaces = array(NS_MAIN, NS_USER); was missing in my LocalSettings.php, now I can try to edit existing pages the Visual Editor, but it fails with Error loading data from server: 404: parsoidserver-http: HTTP 404, now I am looking at other users´ issues with the same error, but nothing has helped so far.

LocalSettings.php:

wfLoadExtension( 'VisualEditor' );

// Enable by default for everybody
// $wgDefaultUserOptions['visualeditor-enable'] = 1;

// Optional: Set VisualEditor as the default for anonymous users
// otherwise they will have to switch to VE
// $wgDefaultUserOptions['visualeditor-editor'] = "visualeditor";

// Don't allow users to disable it
// $wgHiddenPrefs[] = 'visualeditor-enable';

// OPTIONAL: Enable VisualEditor's experimental code features
#$wgDefaultUserOptions['visualeditor-enable-experimental'] = 1;

$wgVirtualRestConfig['modules']['parsoid'] = array(
	// URL to the Parsoid instance
	// Use port 8142 if you use the Debian package
	'url' => 'http://localhost:8000',
	// Parsoid "domain", see below (optional)
	'domain' => 'localhost',
	// Parsoid "prefix", see below (optional)
	'prefix' => 'localhost'
);
$wgSessionsInObjectCache = true;

Settings for private wiki from https://www.mediawiki.org/wiki/Extension:VisualEditor#Linking_with_Parsoid_in_private_wikis are added:

// This feature requires a non-locking session store. The default session store will not work and
// will cause deadlocks (connection timeouts from Parsoid) when trying to use this feature.
$wgSessionsInObjectCache = true;

// Forward users' Cookie: headers to Parsoid. Required for private wikis (login required to read).
// If the wiki is not private (i.e. $wgGroupPermissions['*']['read'] is true) this configuration
// variable will be ignored.
//
// WARNING: ONLY enable this on private wikis and ONLY IF you understand the SECURITY IMPLICATIONS
// of sending Cookie headers to Parsoid over HTTP. For security reasons, it is strongly recommended
// that $wgVirtualRestConfig['modules']['parsoid']['url'] be pointed to localhost if this setting is enabled.
$wgVirtualRestConfig['modules']['parsoid']['forwardCookies'] = true;

Thanks in advance

Arlolra (talkcontribs)

The _wikitext and _html routes are intended to be private, for development use only. To enable them, add devAPI: true to your config.yaml

Escalatr (talkcontribs)

Thanks for the remark, devAPI: true indeed enabled _wikitext output! This topic is at least partially solved.

However, Visual Editor and mwoffliner.js still fail with the error Error loading data from server: 404: parsoidserver-http: HTTP 404

Arlolra (talkcontribs)

Sounds like Parsoid itself is configured correctly now. Try setting the url in the $wgVirtualRestConfig in your LocalSetting.php to http://172.18.200.40:8000, since that's what you tested above.

Escalatr (talkcontribs)

Thanks for the reply, I tried several URLs already:

The page http://172.18.200.40:8000/localhost/v3/page/html/Main_Page/ is always working, however Visual Editor fails. The only difference with http://localhost:8000 and http://127.0.0.1:8000: the parsed page seems to be missing CSS files (I am seeing the default browser font and formatting).

I am quite unclear how domain and prefix should be configured, the documentation is rather contradictory, have tried several settings already.

I have started a thread with some more information at Visual Editor discussion: https://www.mediawiki.org/wiki/Topic:Tkngrm2kaak25q8o. Parsoid seems to work properly in general. Just the connection by VisualEditor and mwoffliner.js fails.

Escalatr (talkcontribs)

According to https://www.mediawiki.org/wiki/Offline_content_generator/Installation/en#Starting I called netstat -tuln, the output:

server223:/etc/init.d # netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:9200            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:9300            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:6556            0.0.0.0:*               LISTEN
tcp        0      0 :::8000                 :::*                    LISTEN
tcp        0      0 :::873                  :::*                    LISTEN
tcp        0      0 :::3306                 :::*                    LISTEN
tcp        0      0 :::22                   :::*                    LISTEN
udp        0      0 0.0.0.0:57053           0.0.0.0:*
udp        0      0 0.0.0.0:57231           0.0.0.0:*
udp        0      0 0.0.0.0:33014           0.0.0.0:*
udp        0      0 0.0.0.0:54328           0.0.0.0:*
udp        0      0 0.0.0.0:5353            0.0.0.0:*
udp        0      0 0.0.0.0:43376           0.0.0.0:*
udp        0      0 0.0.0.0:47603           0.0.0.0:*
udp        0      0 :::38093                :::*
udp        0      0 :::5353                 :::*
Arlolra (talkcontribs)

The colons indicate that it's listening on IPv6 addresses as well. That seems ok since you were able to reach the IPv4 address 172.18.200.40

No, Redis is not required in any way.

Escalatr (talkcontribs)

Thanks for the clarification.

Error when starting the server.js

16
Summary by Arlolra

Parsoid starts os.cpus().length workers by default, which may be a lot on shared hosting. Limit it with the -n flag

MagaFun (talkcontribs)

I ran the command node bin/server.js and it gives me such error:

{"name":"parsoid","hostname":"cp-hk-1.webhostbox.net","pid":279063,"level":60,"err":{"message":"spawn ENOMEM","name":"Error","stack":"Error: spawn ENOMEM\n    at exports._errnoException (util.js:1022:11)\n    at ChildProcess.spawn (internal/child_process.js:313:11)\n    at exports.spawn (child_process.js:385:9)\n    at exports.fork (child_process.js:63:10)\n    at createWorkerProcess (cluster.js:321:12)\n    at EventEmitter.cluster.fork (cluster.js:356:27)\n    at Master._startWorkers (/home/ahysprkj/parsoid/node_modules/service-runner/lib/master.js:200:30)\n    at Worker.<anonymous> (/home/ahysprkj/parsoid/node_modules/service-runner/lib/master.js:267:38)\n    at emitTwo (events.js:111:20)\n    at Worker.emit (events.js:191:7)\n    at ChildProcess.Worker.process.on (cluster.js:54:12)\n    at emitTwo (events.js:106:13)\n    at ChildProcess.emit (events.js:191:7)\n    at process.nextTick (internal/child_process.js:752:12)\n    at _combinedTickCallback (internal/process/next_tick.js:67:7)\n    at process._tickCallback (internal/process/next_tick.js:98:9)","code":"ENOMEM","errno":"ENOMEM","syscall":"spawn","levelPath":"fatal/service-runner/unhandled"},"msg":"spawn ENOMEM","time":"2016-12-01T11:53:28.886Z","v":0}

Is there a way to solve this? Thanks in advance.

This comment was hidden by MagaFun (history)
Arlolra (talkcontribs)

ENOMEM is the out of memory error. What does free -m say? Is swap enabled?

There's a way to restrict the heap in node.js, with the --max-old-space-size flag. You could try that.

MagaFun (talkcontribs)

total       used       free     shared    buffers     cached

Mem:         23929      22629       1299         65       2874       6427

-/+ buffers/cache:      13327      10602

Swap:         4095       2956       1139

The output is above.

Arlolra (talkcontribs)

I think node's default heap size ~2 GB and you've 1299 MB free. You might want to try to free up some RAM.

Alternatively, you can run with the above flag, --max-old-space-size

Or, reduce the number of workers Parsoid is launching. The default should be the same as the number of cpus on your machine. But try with 1 or 0, even, which will forgo the master (but that presents other challenges).

node --max-old-space-size=128 bin/server.js -n 1

This comment was hidden by MagaFun (history)
MagaFun (talkcontribs)

{"name":"parsoid","hostname":"cp-hk-1.webhostbox.net","pid":715569,"level":40,"levelPath":"warn/service-runner","msg":"Startup finished","time":"2016-12-02T03:29:34.794Z","v":0}

I tried the command you gave me but now it got stuck on this message and there is no more output.

Arlolra (talkcontribs)

EADDRINUSE means that you didn't stop a previously listening instance.

"Startup finished" means the service is up and running. Did you try querying it?

MagaFun (talkcontribs)

I tried localhost:8000 but my browser just stuck at "Connecting".

By the way my site has HSTS enabled.

Arlolra (talkcontribs)

Hmm, I'm not sure how HSTS would play in here.

You confirmed it's listening on 8000, right?

If you're using some sort of shared hosting / cloud infrastructure, maybe localhost isn't what you think it is. Do you have an assigned IP? Maybe try setting that as the host in Parsoid's config instead.

MagaFun (talkcontribs)

I could confirm it is listening on 8000.

I do am using shared hosting, and I changed the "domain" in the "mwApis" section to my IP address. I still cannot query it. The output hostname is still "cp-hk-1.webhostbox.net", which is not my domain.

The reason I mentioned HSTS is because I think the issue below mine by 59.163.27.11 could also play a role here.

Arlolra (talkcontribs)

Maybe verify that you browser is actually sending requests there? Run,

nc -l localhost 8000

and then try to visit it in the browser. You should see the http request.

Something else to try is eliminating the web frontend altogether. Parsoid comes with a command line tool.

node --max-old-space-size=128 bin/parse.js --config --domain localhost Main_Page < /dev/null

MagaFun (talkcontribs)

I ran nc -l localhost 8000 but it says command not found, I couldn't install that command either.

Any alternative to that? Thanks.

Arlolra (talkcontribs)

netcat

MagaFun (talkcontribs)

Netcat doesn't work either, I don't think my host installed that.

Tgr (talkcontribs)

I've run into a similar issue: I installed parsoid via the Debian package on a virtual host, and it tries to start 24(!) workers with 756M each. --max-old-space-size does not seem to have any effect; -n works but it's not easy to use with the init.d script which does not insert -- at the right places in the start-stop-daemon commands, and then I get a bunch of EADDRINUSE errors on restarting because the old service was not shut down. In the end I just replaced the contents of do_stop{} with killall nodejs as a terrible hack and that seems to work.

Can't find settings.js under /etc/mediawiki/parsoid in Debian

2
Summary by Zoglun

only config.yaml needed.

Zoglun (talkcontribs)

So I got Parsoid installed throguh apt-get install parsoid under Debian.

Then there is only a config.yaml under /etc/mediawiki/parsoid

Is this means I only need to config the config.yaml file for VisualEditor?

Arlolra (talkcontribs)

Yes, in Parsoid v0.6.x, config.yaml is all you need.