Talk:RESTBase

Jump to: navigation, search

About this board

We welcome your comments and technical feedback about the new RESTBase API. For bugs and issues, the recommended way to report issues is https://phabricator.wikimedia.org/

Previous discussion is at Talk:RESTBase/Archive.

By clicking "Add topic", you agree to our Terms of Use and agree to irrevocably release your text under the CC BY-SA 3.0 License and GFDL

Need clarification: high volume API timeframe

4
Fasnoosh (talkcontribs)

On the RESTbase docs page for rest_v1 it says:

High-volume access

  • Don't perform more than 500 requests/s to this API.

500 requests per what amount of time? Minute? Hour?

Here's how google specifies usage limits

Ltrlg (talkcontribs)

I guess the “/s” part means “per second”…

Mobrovac-WMF (talkcontribs)

Indeed, '/s' refers to 'seconds'

Fasnoosh (talkcontribs)

Good point. I think I speed-read that, and squished the s's together.

Might be helpful instead to show it like this: requests / sec

Reply to "Need clarification: high volume API timeframe"
Prh47bridge (talkcontribs)

I'm trying to get RESTBase set up on my wiki. I've downloaded it, run npm install and updated the configuration. However, about 3-4 seconds after startup I get the above error message from node_modules/service-runner/lib/worker.js. In case it is relevant I'm running on Ubuntu 14.04 using SQLite. I expect it is something silly I've missed but any pointers would be much appreciated.

Mobrovac-WMF (talkcontribs)

This means that the worker never received the config from the master process, which is rather unusual. Could you try starting your service with `no_workers: 0` set in your config YAML and see if the service works?

Prh47bridge (talkcontribs)

Thanks for your response. I've tried it and that doesn't seem to make any difference. However, if I put "-n 0" on the command line it exits immediately with the message:

const P = require('bluebird'); ^^^^^ {"name":"restbase","hostname":"XXXXX","pid":31218,"level":60,"err":{"message":"Use of const in strict mode.","name":"SyntaxError","stack":"SyntaxError: Use of const in strict mode.\n at Module._compile (module.js:439:25)\n at Object.Module._extensions..js (module.js:474:10)\n at Module.load (module.js:356:32)\n at Function.Module._load (module.js:312:12)\n at Module.require (module.js:364:17)\n at require (module.js:380:17)\n at Object.<anonymous> (/usr/local/restbase/node_modules/hyperswitch/index.js:3:18)\n at Module._compile (module.js:456:26)\n at Object.Module._extensions..js (module.js:474:10)\n at Module.load (module.js:356:32)\n at Function.Module._load (module.js:312:12)\n at Module.require (module.js:364:17)\n at require (module.js:380:17)\n at Master.BaseService._requireModule (/usr/local/restbase/node_modules/service-runner/lib/base_service.js:309:26)\n at P.map.then.self.serviceReturns (/usr/local/restbase/node_modules/service-runner/lib/worker.js:162:21)\n at tryCatcher (/usr/local/restbase/node_modules/bluebird/js/release/util.js:16:23)","moduleName":"hyperswitch","levelPath":"fatal/service-runner/worker"},"msg":"Use of const in strict mode.","time":"2016-12-24T10:34:40.865Z","v":0}

where XXXXX is the name of my server. There is no other error message.

Without the -n 0 I get:

{"name":"restbase","hostname":"XXXXX","pid":31224,"level":30,"levelPath":"info/service-runner","msg":"master(31224) initializing 4 workers","time":"2016-12-24T10:34:53.531Z","v":0} {"name":"restbase","hostname":"XXXXX","pid":30911,"level":60,"err":{"message":"Object function Object() { [native code] } has no method 'assign'","name":"TypeError","stack":"TypeError: Object function Object() { [native code] } has no method 'assign'\n at /usr/local/restbase/node_modules/service-runner/lib/master.js:207:33\n at Promise._execute (/usr/local/restbase/node_modules/bluebird/js/release/debuggability.js:299:9)\n at Promise._resolveFromExecutor (/usr/local/restbase/node_modules/bluebird/js/release/promise.js:481:18)\n at new Promise (/usr/local/restbase/node_modules/bluebird/js/release/promise.js:77:14)\n at Master._startWorkers (/usr/local/restbase/node_modules/service-runner/lib/master.js:204:16)\n at /usr/local/restbase/node_modules/service-runner/lib/master.js:107:21\n at tryCatcher (/usr/local/restbase/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/usr/local/restbase/node_modules/bluebird/js/release/promise.js:510:31)\n at Promise._settlePromise (/usr/local/restbase/node_modules/bluebird/js/release/promise.js:567:18)\n at Promise._settlePromise0 (/usr/local/restbase/node_modules/bluebird/js/release/promise.js:612:10)\n at Promise._settlePromises (/usr/local/restbase/node_modules/bluebird/js/release/promise.js:691:18)\n at Async._drainQueue (/usr/local/restbase/node_modules/bluebird/js/release/async.js:138:16)\n at Async._drainQueues (/usr/local/restbase/node_modules/bluebird/js/release/async.js:148:10)\n at Async.drainQueues (/usr/local/restbase/node_modules/bluebird/js/release/async.js:17:14)\n at process._tickCallback (node.js:448:13)","levelPath":"fatal/service-runner/unhandled"},"msg":"Object function Object() { [native code] } has no method 'assign'","time":"2016-12-24T10:29:09.261Z","v":0}

The above text is in white and is followed by the error message regarding a timeout in red. My bad - I hadn't looked properly at the white text before and should have reported that. Am I missing a dependency?

Mobrovac-WMF (talkcontribs)

Are you using Node >= 4 ? These messages indicate that you might be running it on an older version. Is that the case? Note that RESTBase needs Node >= 4 in order to run properly.

Prh47bridge (talkcontribs)

That would be the problem then! The README.md file says I need 0.10+ and, since I've got 0.10.37, I assumed that would be ok. Suggest someone updates the README! Thanks for your help.

Mobrovac-WMF (talkcontribs)

Oh, right! Will update, thank you for pointing that out.

Reply to "Timeout waiting for config in worker"
Wess (talkcontribs)

We are trying to install RESTbase to support the wikieditor-VisualEditor switch. It seems that REST base is working fine (on localhost) but the VisualEditor will not load, providing a http 0 server error.

Also the old Wikieditor will not load, and the console sends 'Uncaught TypeError: Cannot read property 'id' of undefined':

I guess it has to do with a missing configuration but any clue will be very helpful.

Wess (talkcontribs)

Apparently the editor issue was due to another extension that was not up to date. However the VisualEditor http 0 issue is still present...

Mobrovac-WMF (talkcontribs)

If RESTBase is working on localhost, then this is likely a VE-related issue. Try to get help on Extension:VisualEditor . You need to provide them with the versions of MW and VE you are running, along with your LocalSettings.php configuration. Also useful would be to get the stack trace of the error you see on the console.

Wess (talkcontribs)

Thanks. Apparently the issue was pointing VE RestbaseURL to localHost rather than to the external url.

MarcoSwart (talkcontribs)

When I try to use the all-days parameter I get the message

"The date(s) you used are valid, but we either do not have data for those date(s), or the project you asked for is not loaded yet.  Please check https://wikimedia.org/api/rest_v1/?doc for more information."

As the data for the month in question appear to be present: is there other data that is not loaded yet?

Milimetric (WMF) (talkcontribs)
One of the API devs here. The reason is that, unfortunately, we had performance problems when computing the monthly top articles. So we beed to change the algorithm and we haven't gotten to that yet. We have a tracking task in phabricator: https://phabricator.wikimedia.org/T120113
Mobrovac-WMF (talkcontribs)

@MarcoSwart could you provide the full URL you were using?

MarcoSwart (talkcontribs)

I tried several, but the one I really wanted was:

https://wikimedia.org/api/rest_v1/metrics/pageviews/top/nl.wiktionary/all-access/2015/12/all-days

Reply to "all-days parameter Pageviews"
Guitpicker07 (talkcontribs)

I don't see Swagger for the Search API in RESTbase:

API:Search

https://github.com/wikimedia/restbase

Mobrovac-WMF (talkcontribs)

Search is not yet available in RESTBase at this point, I'm afraid. Our goal is to first complete the API layout with stable-addressable content.

Reply to "Swagger for Search API?"
Arlolra (talkcontribs)

why https://rest.wikimedia.org/en.wikipedia.org/v1/?doc as opposed to https://rest.wikimedia.org/v1/en.wikipedia.org/?doc

ie. why isn't the domain included in the version?

Mobrovac-WMF (talkcontribs)

This allows us more flexibility in that each domain may run different versions of the API, which closely resembles the current state of WM projects - not all of them run the same MW version, nor do they have the same features enabled.

Arlolra (talkcontribs)

you're versioning the api though ... if a domain wants to use an older version, they would still have that option

you don't anticipate any future use case that isn't partitioned by domain?

Mobrovac-WMF (talkcontribs)

Ah, I see, the confusion is about the URI itself. We do not have any particular restrictions on the domain name itself, it can be "blah" as far as we are concerned, so you can have also stuff in there that are domain-agnostic. This layout allows us to activate/deactivate stuff per-domain easily.

A side note: the URL rest.wikimedia.org/en.wikipedia.org/v1/ should be though of as:

  • domain (host/port): rest.wikimedia.org/en.wikipedia.org
  • path: v1/

What this allows us is a (possible future) mapping to domains themselves directly, so that e.g. en.wikipedia.org/rest-api/ (or any other path) can be transparently mapped to rest.wikimedia.org/en.wikipedia.org/ .

Arlolra (talkcontribs)

if you can have stuff in there that's domain-agnostic, how do you distinguish the two?

you can still map

en.wikipedia.org/rest-api/v1/ to rest.wikimedia.org/v1/en.wikipedia.org/

GWicke (talkcontribs)

Arlo, all APIs are versioned per logical domain. We could have an additional global version like rest.wikimedia.org/v1/en.wikipedia.org/v1/, but didn't see a strong use case for this, as this would only version the domain part.

Reply to "versioning"
There are no older topics