Talk:Wikimedia REST API

About this board

Rate limiting of page/html API endpoint

5
Mitar (talkcontribs)

I am trying to get rendered HTML for a bunch of pages. I am pretty sure I am limiting myself to 200 requests per second to page/html API endpoint, and I have User-Agent set, but very quickly I am starting to get 429 responses. Have I misunderstood something about the rate limit for that API endpoint? I am making those 200 requests in parallel though. Could this be a problem? (I use stash=false as well.)

Mitar (talkcontribs)

It looks like limit of 100 requests works. (Hm, or not. It still starts returning 429 after some time.) So I tested now with 100 requests per second. I managed to make 45852 requests in 8 minutes and after that they started failing with 429. This is a bit less than 100 requests per second.

Mitar (talkcontribs)
Mutante (talkcontribs)

Please create a Phabricator ticket and paste the information above so people can check this and help you. Mutante (talk) 17:11, 4 May 2022 (UTC)

Mitar (talkcontribs)
Reply to "Rate limiting of page/html API endpoint"

MediaWiki vs Wikimedia REST APIs

2
Diegodlh (talkcontribs)

Hi, all! I'm not sure I understand the differences between MediaWiki and Wikimedia REST APIs. I understand that the Wikimedia REST API (not included with MediaWiki) is powered by the RESTbase caching/storing API proxy, making it more appropriate for high-volume use cases. I assume that this is not the case with the MediaWiki REST API.

Why is there a separate MediaWiki API? Is this because having a RESTbase API would be an overkill for most MediaWiki installations?

Thank you!

APaskulin (WMF) (talkcontribs)

Hi, Thanks for your question! You are correct: We created the MediaWiki REST API to provide a RESTful interface for anyone using MediaWiki, since running RESTBase would be a complex process for most MediaWiki installations.

Reply to "MediaWiki vs Wikimedia REST APIs"
Smith609 (talkcontribs)

I note that we are requested not to make more than 200 requests to the API, but a time interval is not specified. Is this intended to read "per day", or "per hour"? Is there someone to whom requests for a higher access rate might be directed? ~~~~

Skalman (talkcontribs)

It actually did say "200 requests/s". I've clarified it, and now it says "200 requests/sec".

Tomayac (talkcontribs)

I would love to learn about your plans for Service Worker support for the API.

GWicke (talkcontribs)

The REST API is primarily targeted at serving the data consumed by increasingly fatter clients, including those taking advantage of ServiceWorkers.

We are also looking into leveraging ServiceWorkers to improve performance and offline support. As part of this effort, we have done some research and prototyping into a) a node environment for running unmodified serviceworker code on behalf of older clients, and b) streaming HTML transformation / composition tooling. We do intend to develop a PWA targeted at mobile web over the next 12 months which might use some of this research, but expect this PWA to be limited to an opt-in beta feature until at least mid 2018.

Reply to "Service Workers"

Latency of hourly vs daily endpoints?

1
Aldebrn (talkcontribs)

Who could I ask about the latency of some of the REST endpoints? I see that I can get very recent pageviews data, e.g.

https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/en.wikipedia/all-access/all-agents/hourly/2018031600/2018031800

accessed now, at 2018/03/17 0230 UTC, gives me an hourly pageviews on the English Wikipedia at timestamp "2018031623", so about ~3.5 hours latency, very nice!

In contrast, asking for the daily number of edits via

https://wikimedia.org/api/rest_v1/metrics/edits/aggregate/en.wikipedia/all-editor-types/all-page-types/daily/20180225/20180317

only gives me data up to the end of February. This makes me think the daily datasets are generated only once a month? Who can I ask about this latency, and whether more recent "rest_v1/metrics/edits" data can be obtained? Thanks!

Reply to "Latency of hourly vs daily endpoints?"
Xover (talkcontribs)

Are there any plans to make the per-user Special:Watchlist available?

197.218.88.81 (talkcontribs)
Reply to "Watchlist access?"
GreenC (talkcontribs)
GWicke (talkcontribs)

When an end point is deprecated, "it means that it has not been useful to clients and will consequently be removed in the near future." These end points were experimental or unstable before, and have been deprecated as part of a larger clean-up effort in preparation for the 1.0 release.

So no, please do not use those end points any more. They are about to be removed in early May.

Reply to "Deprecated?"
There are no older topics