Wikimedia Enterprise/ja

From mediawiki.org
Jump to navigation Jump to search
This page is a translated version of the page Wikimedia Enterprise and the translation is 17% complete.
Other languages:

The Wikimedia Enterprise API is a new service focused on high-volume commercial reusers of Wikimedia content. It will provide a new funding stream for the Wikimedia movement; greater reliability for commercial reusers; and greater reach for Wikimedia content.

For general information, the relationship to the Wikimedia strategy, operating principles, and FAQ, see Wikimedia Enterprise on Meta. The project was formerly known as "Okapi".

See also the dedicated subpage for API documentation. Current development work is tracked on our Phabricator board. For information about Wikimedia community access to this service, please see Access on the project's Meta homepage.

Contact the team if you would like to arrange a conversation about this project with your community.
Monthly "Office hours": Third Friday of each month @ 15:00 UTC. Details on our Meta page

更新

This is the most recent months of technical updates. [All previous updates can be found at the archive]


2021-10: Website Launch and Wikimedia Dumps release!

  • Website Launch:
    • Our website is live! Check it out
    • Launched in this is our initial product offering details along with some pricing and sign up information.
  • Wikimedia Dumps release!
    • Wikimedia Dumps now has Wikimedia Enterprise dumps! Give it a download and please provide feedback to our team as you see relevant
    • Reminder: The Daily and Hourly Diffs are available on WMCS currently

2021-09: Launch! Building towards the next version and public access

  • V1 launched on 9/15/2021: This month we stepped out of beta and fully launched v1 of Wikimedia Enterprise APIs. V1 APIs include:
    • Real Time:
      • Firehose API: Three real time streams of all of the current events happening across our projects. You can hold this connection indefinitely and returns you the same data model as the others so that you can get all of the information in just one event object. The three streams are:
        • page-update: all revisions and changes to a page across the projects
        • page-delete: all page deletions to remove from records
        • page-visibility: highly urgent community driven events within the projects to reset
      • Hourly Diffs: An API that returns a zip file containing all of changes with in a day of all "text-based" Wikimedia projects
    • Bulk:
      • Daily Exports: An API that returns a zip file containing all of changes with in a day of all "text-based" Wikimedia projects
    • Pull:
      • Structured Content API: An API that allows you to lookup a single page in the same JSON structure as the Firehose, Hourly, and Daily endpoints.
  • Implementing new architecture:
    • We are starting to implement the architecture that we've been working on in past months to move towards a more flexible system that is built around streaming data. More information to be shared on our mediawiki page soon.
    • We are also working on rewriting some of our existing launch work into the new process - this is a lot of repurposing code but making for a stronger and more scalable system.
    • After this, we will begin the implementation of Wikidata, more credibility signals, and flexible filtering into the suite of APIs.
  • Public Access:
    • The Daily and Hourly Diffs are available on WMCS currently
    • We are planning to launch with Wikimedia Dumps soon as we launch hashing capabilities in the APIs in v1! Stay tuned.

2021-08: Roadmap Design and Building towards our September Launch!

  • Roadmapping the next six months:
    • Wikidata:
      • Wikidata is a heavily used project by Wikimedia Enterprise's persona of commercial content reusers. Looking into the future, it is important for us to include "text-based" projects as well as Wikidata in the feeds that we create.
      • Our goal is to add Wikidata to the Firehose streams, Hourly Diffs, and Daily Exports giving Enterprise users the ability to access all of the projects (except Commons) in one API suite.
    • Credibility Signals
      • As we work to solve the challenges of reliably ingesting in real time Wikimedia data at scale, there are two big problems that still come with our data: Content Integrity and Machine Readability.
      • Wikimedia data reusers are not necessarily savvy in the nuances of the communities efforts to keep the projects as credible as possible and miss much of the context that comes with revisions that might help inform whether or not a new revision is worth replacing in an external system. This is exacerbated as reusers aim to move towards real time data on projects that are always in flux.
      • We plan to draw out the landscape of what signals can be included alongside real time and bulk feeds of new revisions to help end users add more context to their systems. Stay tuned here.
    • Flexible APIs:
      • Customizable Payload: With the ever expanding data added to our schemas, we need more flexibility on the payloads that end users would like. This is not easy or possible for Hourly Diffs or Daily Exports since those files are pre-generated and static but we aim to work on this capability across the Firehose and Structured Content APIs.
      • Enhanced Filtering: Since there are so many different data points coming through the feeds, end users will start to build their comfortability of ingestion around a few feeds. It is imperative that we provide the ability to filter beyond client side so that we can limit the direct traffic on end user's systems. This also provides a much easier user experience for users o the APIs.
  • September Launch:
    • We are all hands on deck building and processing towards our launch of our initial launch product.

2021-07: Onboarding, Architecture, and Launch Schema

  • Added some new folks to our engineering team:
    • Welcome Prabhat Tiwary, Daniel Memije, and Tim Abdullin! They join us with each different perspectives and experiences adding substantial experience and capacity to our team.
    • With this came a lot of work stepping back and building onboarding documentation to make sure our team can grow and folks can join and contribute to our work.
  • New Architecture
    • As Wikimedia Enterprise APIs become more defined and complicated, we have started to draw out what a target architecture would look like. We are doing a lot of planning and taking time to think through what a streaming pipe should look like.
    • Our original architecture was centered around the solution of "Exports" and less around the real-time component, which in the long run will create flexibility issues with how we store and move data around our architecture.
  • Data Model / API Schema:
    • We have decided on a target schema, dataset, and set of APIs for our move out of beta in September. See more on our documentation page here


Past updates

For previous months' updates, see the the archive.


概要

背景

Due to the myriad of sources of information on the internet, compiling public and private data sets together has become a major proprietary asset (seen in customer knowledge graphs) for large tech companies when building their products. It is through this work that a company’s voice assistants and search engines can be more effective than those of their competitors. Wikimedia data is the largest public data source on the internet and is used as the "common knowledge" backbone of knowledge graphs. Not having Wikimedia data in a knowledge graph is detrimental to a product’s value, as we've proven through customer research.

In order for Wikimedia Enterprise API's customers to create effective user experiences, they require two core features from the Wikimedia dataset: completeness and timeliness.

Wikimedia content provides the largest corpus of information freely available on the web. It maps broad topics across hundreds of languages and endows consumer products with a feeling of “all-knowingness” and “completeness” that drives positive user experiences.

Wikimedia content originates from a community that authors content in real time, as history unfolds. Leveraging that community’s work provides customer products with the feeling of being “in-the-know” (i.e., “timeliness”) as events occur, thus generating positive user experiences.

There is currently no way for a data-consuming customer to make one or two API requests to retrieve a complete and recent document that contains all relevant and related information for the topic requested. This has resulted in customers building complex ad-hoc solutions that are difficult to maintain; expensive, due to a large internal investment; error prone, due to inconsistencies in Wikimedia data; and fragile, due to changes in Wikimedia responses.

Research Study

From June 2020 – October 2020, the Wikimedia Enterprise team conducted a series of interviews with third-party reusers [Users] of Wikimedia data to gain a better understanding of what companies are using our data, how they are using our data, in what products they are using it, and what challenges they face when working with our APIs. Our research showed that:

  1. Users cache our data externally rather than query our APIs for live data
  2. Each user approaches our current stack differently, with unique challenges and requests
  3. The Wikimedia APIs are not viewed as a reliable ingestion mechanism for gathering data and are prone to rate limits, uptime issues, and excessive use to achieve their goals
  4. All users have the same general problems when working with our content, and we have received similar asks from users of all size

The Enterprise API team has identified four pain points that cause large third-party reusers to struggle when using our public suite of APIs for commercial purposes. Note: Many of these concepts overlap with other initiatives currently underway within the Wikimedia movement, for example the API Gateway initiative.

  • High Frequency: Commercial reusers want to be able to ingest our content "off-the-press" so that they can have the most current worldview of common knowledge when presenting information to their users.
  • System Reliability: Commercial reusers want reliable uptime on critical APIs and file downloads so that they can build using our tools without maintenance or increased risk on their products.
  • Content Integrity: Commercial reusers inherit the same challenges that Wikimedia projects have in relation to vandalism and evolving stories. Commercial reusers desire more metadata with each revision update in order to inform their judgement calls on whether or not to publish a revision to their products.
  • Machine Readability: Commercial reusers want a clean and consistent schema for working with data across all of our projects. This is due to the challenges that come from parsing and making sense of the data they get from our current APIs.

Product Roadmap

In response to the research study, the Enterprise API team focuses on building tools for commercial reusers that will offer the advantages of a data service relationship while expanding the usability of the content that we provide.

The roadmap is split into two ordered phases focused on helping large third-party reusers with:

  1. Building a "commercial ingestion pipe"
  2. Creating more useful data to feed into the "commercial ingestion pipe"

Beta: Building a "Commercial Ingestion Pipe" (Current)

The goal of the first phase is to build infrastructure that ensures the Wikimedia Foundation can reasonably guarantee Service Level Agreements (SLAs) for 3rd-party reusers as well as create a "single product" where commercial reusers can confidently ingest our content in a clear manner. While the main goal of this is not explicitly to remove the load of the large reusers from Wikimedia Foundation infrastructure, it is a benefit, for we do not currently know the total capacity of these large reusers on donor-funded infrastructure.


The Q4 2021 release (May/June 2021) of the Enterprise APIs proposes the following:

Type Name Compare To What is it? What’s New?
Realtime Enterprise Activity "Firehose" API EventStream HTTP API A stable, push HTTP stream of real time activity across “text-based” Wikimedia Projects
  • Push changes to client with stable connection
  • Filter by Project and Page-Type
  • Be Notified of suspected vandalism in real time
  • Machine Readable and Consistent JSON schema
  • Guaranteed uptime, no rate-limiting
Enterprise Structured Content API Restbase APIs Recent, machine readable content from all “text-based” Wikimedia Projects
  • Machine Readable and Consistent JSON schema
  • Guaranteed uptime, no rate-limiting
Bulk Enterprise Bulk Content API Wikimedia Dumps Recent, compressed Wikimedia data exports for bulk content ingestion
  • Machine Readable and Consistent JSON schema
  • Daily “Entire Corpus” exports
  • Hourly “Activity” exports
  • Guaranteed delivery
  • Historical Downloads

Launch: Enhance Wikimedia Data for Reuse

The goal of the second phase of this project is to enhance the data that comes through the infrastructure provided by the Enterprise API. By doing this, we will create more opportunity for reusers ingesting our data feeds to efficiently use our content in their products. We have taken requests on ways that reusers would like to have our data evolve over time and created this list.

The elements listed are by no means in order of importance or commitments, just ideas and direct responses to requests that have been articulated to our team. We look forward to defining future directions in collaboration with other teams within the movement.

Future: Enterprise API Roadmap Considerations

Theme Feature Details
Machine Readability Parsed Wikipedia Content Break out the HTML and Wikitext content into clear sections that customers can use when processing our content into their external data structures
Optimized Wikidata Ontology Wikidata entries mapped into a commercially consistent ontology
Wikimedia-Wide Schema Combine Wikimedia project data together to create “single-view” for multiple projects around topics.
Topic Specific Exports Segment corpus into distinct groupings for more targeted consumption.
Content Integrity Anomaly Signals Update schema with information guiding customers to understand the context of an edit. Examples: page view / edit data
Credibility Signals Packaged data from the community useful to detect larger industry trends in disinfo, misinfo, or bad actors
Improved Wikimedia Commons license access More machine readable licensing on Commons media
Content Quality Scoring (Vandalism detection, “best last revision”) Packaged data used to understand the editorial decision-making of how communities catch vandalism.

Active Development (Beta)

See also: The API documentation subpage.

Note: We are still defining the exact nomenclature for API endpoints and documentation, but these are the main products that our team is currently building.

Structured Content API

High-volume reusers that use an infrastructure reliant on the EventStream platform depend on services like RESTBase to pull HTML from page titles and current revisions to update their products. High-volume reusers have requested a reliable means to gather this data, as well as structures other than HTML when incorporating our content into their KGs and products.

Wikimedia Enterprise Structured Content API, at release, will contain:

  • A commercial schema
  • SLA

In terms of schema, we are limiting our scope to optimize the reliability of these services. However, refer to our Product Roadmap to understand where this will expand. Please note, this schema is still in progress and is not final; it was built to articulate our ideas and is not fully named or coded. We will likely leverage ideas from schema.org when building out this schema in full.Welcome to the Wikimedia Enterprise APIs! Wikimedia Enterprise provides a suite of APIs designed for high-volume access to information from Wikipedia and other Wikimedia projects.

Here are some ways you can integrate with Wikimedia projects using Wikimedia Enterprise APIs.

Getting started with Wikimedia data?
  • Download Wikipedia articles in bulk using the exports API. Exports are updated daily across all supported projects.
Want to update your systems in real time? There two ways to do that:
  • Hook into the Firehose streams to automatically receive updates as they happen.
  • Use the hourly diffs API to get changes made within the last 24 hours.
Want to look up a single Wikipedia article?
  • Query the pull API to receive the latest version of an article.

Getting Started

Once you’ve received your Wikimedia Enterprise credentials, it’s time to make your first call. Start by running this curl command in your terminal to fetch the list of supported projects.


You should receive a list of all supported Wikimedia projects, including the project language and the size of the download. If not, check your credentials, and try again.


Next, download a project export. Run this curl command to download Afrikaans Wikipedia to your machine.


Open the downloaded file to see newline-delimited JSON with a line representing each article in the project. To learn about the data provided for each article, visit the data dictionary.


Now use the diffs endpoint to see what changed in Afrikaans Wikipedia today. Change the date to today’s date, and run this command to download a smaller file containing the pages that have changed in the last past day.


There you have it! You’re ready to download Wikimedia content and stay up to date with the latest changes. Try this workflow with another project, and explore the API reference to discover more APIs.

Authentication

Wikimedia Enterprise APIs require basic authentication passed via the Authorization header.

Your Wikimedia Enterprise credentials carry many privileges related to your account, so be careful to keep them secure. Do not share your credentials in publicly accessible areas.

All API requests must be made over HTTPS. Calls made over plain HTTP will be redirected to HTTPS. Calls without authentication in the header will fail.

Endpoints

API URL

Type Name Purpose Endpoint Responses Notes
Realtime API Activity "Firehose" API Returns stream of JSON containing new page revisions, new pages, and updates /v1/page-update
Returns stream of JSON containing recently deleted pages /v1/page-delete
Returns stream of JSON containing pages with changed visibility status. This can be used to determine when content is taken down or vandalism/illegal content is identified /v1/page-visibility
Hourly Exports Returns tarball of only the recent revisions done on a day in a single project, bundled at the end of each hour. These are smaller files with contents in JSON, see below data schema. /v1/diffs/download/{date}/{namespace}/{project}
  • 200: Success
  • 400: Bad Request
  • 404: Not Found
Namespace Index

0 = Article

6 = File

14 = Category

Returns list of all available hourly diffs for namespace /v1/diffs/meta/{date}/{namespace}
Returns single hourly diff metadata for specified namespace /v1/diffs/meta/{date}/{namespace}/{project}
On Demand API Returns structured JSON of the most recent revision for a title in any project. See below data schema. /v1/pages/meta/{project}/{name}
  • 200: Success
  • 400: Bad Request
  • 404: Not Found
Bulk API Daily Exports Returns tarball of all most recent revisions in a single project, bundled at the end of each day. These are large files with contents in JSON, see below data schema. /v1/exports/download/{namespace}/{project}
  • 200: Success
  • 400: Bad Request
  • 404: Not Found
Namespace Index

0 = Article

6 = File

14 = Category

Returns list of all available exports by namespace /v1/exports/meta/{namespace}
Returns export metadata by namespace /v1/exports/meta/{namespace}/{project}
Misc Projects Returns a list of relevant metadata for each of the projects included in Enterprise APIs /v1/projects
  • 200: Success
  • 404: Not Found
Namespaces Returns a list of available namespaces /v1/namespaces

Data Dictionary

Field Value Type Description Example
name string Name displayed at the top of the article "Squirrel"
identifier number Unique identifier of the article

Note: This is different from the revision identifier and maps directly to an article’s MediaWiki ID (primary key)

28492
date_modified string Timestamp of the last revision of the article in RFC3339 format "2021-08-31T04:51:39Z"
protection array List of community-specific protections and restrictions on the article. This is how you can tell which editor permissions are needed to work on this article.

There are three important details to track when it comes to a community created protection.

  • Type: The type of event that the protection is applied to
  • Level: Editor status needed to operate in the type of protection
  • Expiry: Length of time that this protection or restriction is active

Further reading can be found on mediawiki.org

[

   {

     "type": "edit",

     "level": "autoconfirmed",

     "expiry": "infinity"

   },

   {

     "type": "move",

     "level": "autoconfirmed",

     "expiry": "infinity"

   }

 ]

version object Metadata related to the latest specific revision of the article
version.identifier number Revision ID. Unique identifier of the revision, different from the article identifier 1041549311
version.comment string Comment attached by the editor to the latest version. This returns a parsed response.

Note: When a revision is saved, the editor has the ability to leave a comment describing why they created this revision.

"Sample comment describing reason for revision"
version.tags array Tags attached to the version

Note: Tags can be added to an article to signify that this revision was part of a campaign, specific project-run initiative, tracked edit type, or other Wikimedia event.

New events are constantly being added, and while there isn’t a consistent log, this list of tags on mediawiki.org can help

[

     "mobile edit",

     "mobile app edit",

     "android app edit"

   ]

version.is_minor_edit boolean Was this change considered minor by the editor?

Note: When a revision is saved, the editor has the ability to mark whether this is a minor revision. Although this option is set by editors themselves and can lack consistency, minor revisions are typically grammar corrections and are less critical to review.

false
version.is_flagged_stable boolean Was this revision marked stable by the project community?

Note: Depending on the project, there are different community approaches to having administrative approval on revisions as they happen. This will reflect whether or not the revision has been approved.

While false does not mean this is a vandalized article, true is typically a good indicator that this is a good revision.

false
version.scores object This object contains version scores calculated as part of Wikimedia’s ORES project. There are two models included:
  • damaging – predicts whether or not an edit causes damage
  • goodfaith – predicts whether an edit was saved in good faith

Included in these models are three data points:

  • Prediction
  • Probability true
  • Probability false
"scores": {

     "damaging": {

       "prediction": false,

       "probability": {

         "false": 0.959002615965355,

         "true": 0.040997384034645014

       }

     },

     "goodfaith": {

       "prediction": true,

       "probability": {

         "false": 0.01258411739167109,

         "true": 0.9874158826083289

       }

     }

   }

version.editor object Editor-specific signals that can help contextualize the revision:
  • Name
  • Edit count
  • Groups
  • Is bot
  • Is anonymous
  • Date started
{

     "identifier": 4904587,

     "name": "USERNAME",

     "groups": [

       "extendedconfirmed",

       "*",

       "user",

       "autoconfirmed"

     ],

     "date_started": "2010-10-20T05:41:16Z",

     "edit_count": 25123

   }

url string URL of the article "https://en.wikipedia.org/wiki/Squirrel"
namespace object Namespace that this article belongs to

Note: Within Wikimedia, namespaces are used to define the type of article that you are looking at. This indicates the difference between articles, discussion pages, category pages, and other article types.

Visit mediawiki.org to learn more.

"namespace": {

   "name": "Article",

   "identifier": 0

 }

in_language object Human language the article is written in

Use the projects API for a list of supported projects and languages.

"in_language": {

   "name": "English",

   "identifier": "en"

 }

main_entity object Wikidata QID that this article is related to "main_entity": {

   "identifier": "Q9482",

   "url": "https://www.wikidata.org/entity/Q9482"

 }

additional_entities array Object of entities with the following fields:
  • Identifier
  • URL
  • Aspects
"additional_entities": [

   {

     "identifier": "P1992",

     "url": "https://www.wikidata.org/entity/P1992",

     "aspects": [

       "C.P1630"

     ]

   },

   {

     "identifier": "P3031",

     "url": "https://www.wikidata.org/entity/P3031",

     "aspects": [

       "C.P1630"

     ]

   }...

}

]

categories array Project categories that this article belongs to

Note: Within Wikimedia, the Category namespace is used to aggregate articles around specific topics. At the article level, this helps conceptualize the landscape of topics an article belongs to.

[

   {

     "name": "Category:All articles to be expanded",

     "url": "https://en.wikipedia.org/wiki/Category:All_articles_to_be_expanded"

   },

   

 ]

templates array Wikitext templates used in this article

Note: You can reference these if parsing the wikitext content directly.

[

   {

     "name": "Squirrel",

     "url": "https://en.wikipedia.org/wiki/Squirrel"

   },

   {

     "name": "Template:;",

     "url": "https://en.wikipedia.org/wiki/Template:;"

   },

   {

     "name": "Template:About",

     "url": "https://en.wikipedia.org/wiki/Template:About"

   },

   {

     "name": "Template:Anglicise rank",

     "url": "https://en.wikipedia.org/wiki/Template:Anglicise_rank"

   }...

 ]

redirects array Wikimedia articles that redirect to this article

Note: Each name value is likely an alternate language spelling, similar topic item, or general consolidation of the content around this article.

[

   {

     "url": "https://en.wikipedia.org/wiki/Sciuridae",

     "name": "Sciuridae"

   },

   {

     "url": "https://en.wikipedia.org/wiki/Sciurid",

     "name": "Sciurid"

   },

   {

     "url": "https://en.wikipedia.org/wiki/Squirrels",

     "name": "Squirrels"

   },

   {

     "url": "https://en.wikipedia.org/wiki/Bushy_tailed_tree_rat",

     "name": "Bushy tailed tree rat"

   }...

 ]

is_part_of object Wikimedia project this article belongs to {

   "name": "Wikipedia",

   "identifier": "enwiki"

 }

article_body object Article content in HTML and wikitext {

    "html": "...html goes here...",

    "wiktext": "...wikitext goes here..."

 }

license array List of relevant licenses that affect this article and content reuse [

   {

     "name": "Creative Commons Attribution Share Alike 3.0 Unported",

     "identifier": "CC-BY-SA-3.0",

     "url": "https://creativecommons.org/licenses/by-sa/3.0/"

   }

 ]

Projects covered in Wikimedia Enterprise

Below are all the projects that we have in our tools. Use this as a reference for what is included in this project and the db_name property

Project (db_name) Project Home URL Avg Export Size (MB)
abwiki https://ab.wikipedia.org 6MB
acewiki https://ace.wikipedia.org 17MB
adywiki https://ady.wikipedia.org 1MB
afwiki https://af.wikipedia.org 533MB
afwikibooks https://af.wikibooks.org 1MB
afwikiquote https://af.wikiquote.org 1MB
afwiktionary https://af.wiktionary.org 34MB
akwiki https://ak.wikipedia.org 1MB
alswiki https://als.wikipedia.org 231MB
amwiki https://am.wikipedia.org 29MB
amwiktionary https://am.wiktionary.org 1MB
angwiki https://ang.wikipedia.org 8MB
angwiktionary https://ang.wiktionary.org 2MB
anwiki https://an.wikipedia.org 179MB
anwiktionary https://an.wiktionary.org 1MB
arcwiki https://arc.wikipedia.org 2MB
arwiki https://ar.wikipedia.org 13494MB
arwikibooks https://ar.wikibooks.org 8MB
arwikinews https://ar.wikinews.org 27MB
arwikiquote https://ar.wikiquote.org 9MB
arwikisource https://ar.wikisource.org 745MB
arwikiversity https://ar.wikiversity.org 4MB
arwiktionary https://ar.wiktionary.org 75MB
arywiki https://ary.wikipedia.org 11MB
arzwiki https://arz.wikipedia.org 7247MB
astwiki https://ast.wikipedia.org 1310MB
astwiktionary https://ast.wiktionary.org 38MB
aswiki https://as.wikipedia.org 98MB
aswikisource https://as.wikisource.org 4MB
atjwiki https://atj.wikipedia.org 1MB
avkwiki https://avk.wikipedia.org 114MB
avwiki https://av.wikipedia.org 11MB
awawiki https://awa.wikipedia.org 8MB
aywiki https://ay.wikipedia.org 6MB
aywiktionary https://ay.wiktionary.org 1MB
azbwiki https://azb.wikipedia.org 1241MB
azwiki https://az.wikipedia.org 1252MB
azwikibooks https://az.wikibooks.org 6MB
azwikiquote https://az.wikiquote.org 4MB
azwikisource https://az.wikisource.org 32MB
azwiktionary https://az.wiktionary.org 150MB
banwiki https://ban.wikipedia.org 25MB
barwiki https://bar.wikipedia.org 150MB
bat_smgwiki https://bat-smg.wikipedia.org 22MB
bawiki https://ba.wikipedia.org 459MB
bawikibooks https://ba.wikibooks.org 3MB
bclwiki https://bcl.wikipedia.org 70MB
bewiki https://be.wikipedia.org 1900MB
bewikibooks https://be.wikibooks.org 1MB
bewikiquote https://be.wikiquote.org 1MB
bewikisource https://be.wikisource.org 14MB
bewiktionary https://be.wiktionary.org 6MB
be_x_oldwiki https://be-tarask.wikipedia.org 523MB
bgwiki https://bg.wikipedia.org 1979MB
bgwikibooks https://bg.wikibooks.org 2MB
bgwikiquote https://bg.wikiquote.org 15MB
bgwikisource https://bg.wikisource.org 2MB
bgwiktionary https://bg.wiktionary.org 40MB
bhwiki https://bh.wikipedia.org 45MB
biwiki https://bi.wikipedia.org 1MB
bjnwiki https://bjn.wikipedia.org 12MB
bmwiki https://bm.wikipedia.org 1MB
bnwiki https://bn.wikipedia.org 1375MB
bnwikibooks https://bn.wikibooks.org 2MB
bnwikisource https://bn.wikisource.org 39MB
bnwikivoyage https://bn.wikivoyage.org 4MB
bnwiktionary https://bn.wiktionary.org 11MB
bowiki https://bo.wikipedia.org 50MB
bpywiki https://bpy.wikipedia.org 145MB
brwiki https://br.wikipedia.org 209MB
brwikiquote https://br.wikiquote.org 1MB
brwikisource https://br.wikisource.org 37MB
brwiktionary https://br.wiktionary.org 59MB
bswiki https://bs.wikipedia.org 1009MB
bswikibooks https://bs.wikibooks.org 1MB
bswikinews https://bs.wikinews.org 1MB
bswikiquote https://bs.wikiquote.org 7MB
bswikisource https://bs.wikisource.org 13MB
bswiktionary https://bs.wiktionary.org 13MB
bugwiki https://bug.wikipedia.org 145MB
bxrwiki https://bxr.wikipedia.org 10MB
cawiki https://ca.wikipedia.org 5971MB
cawikibooks https://ca.wikibooks.org 10MB
cawikinews https://ca.wikinews.org 37MB
cawikiquote https://ca.wikiquote.org 24MB
cawikisource https://ca.wikisource.org 27MB
cawiktionary https://ca.wiktionary.org 436MB
cbk_zamwiki https://cbk-zam.wikipedia.org 13MB
cdowiki https://cdo.wikipedia.org 29MB
cebwiki https://ceb.wikipedia.org 40478MB
cewiki https://ce.wikipedia.org 3741MB
chrwiki https://chr.wikipedia.org 2MB
chrwiktionary https://chr.wiktionary.org 1MB
chwiki https://ch.wikipedia.org 1MB
chywiki https://chy.wikipedia.org 1MB
ckbwiki https://ckb.wikipedia.org 208MB
cowiki https://co.wikipedia.org 14MB
cowiktionary https://co.wiktionary.org 4MB
crhwiki https://crh.wikipedia.org 31MB
crwiki https://cr.wikipedia.org 1MB
csbwiki https://csb.wikipedia.org 11MB
csbwiktionary https://csb.wiktionary.org 1MB
cswiki https://cs.wikipedia.org 4320MB
cswikibooks https://cs.wikibooks.org 7MB
cswikinews https://cs.wikinews.org 13MB
cswikiquote https://cs.wikiquote.org 19MB
cswikisource https://cs.wikisource.org 206MB
cswikiversity https://cs.wikiversity.org 21MB
cswiktionary https://cs.wiktionary.org 176MB
cuwiki https://cu.wikipedia.org 2MB
cvwiki https://cv.wikipedia.org 237MB
cvwikibooks https://cv.wikibooks.org 3MB
cywiki https://cy.wikipedia.org 729MB
cywikibooks https://cy.wikibooks.org 1MB
cywikiquote https://cy.wikiquote.org 1MB
cywikisource https://cy.wikisource.org 1MB
cywiktionary https://cy.wiktionary.org 13MB
dawiki https://da.wikipedia.org 1733MB
dawikibooks https://da.wikibooks.org 16MB
dawikiquote https://da.wikiquote.org 1MB
dawikisource https://da.wikisource.org 22MB
dawiktionary https://da.wiktionary.org 51MB
dewiki https://de.wikipedia.org 21544MB
dewikibooks https://de.wikibooks.org 320MB
dewikinews https://de.wikinews.org 22MB
dewikiquote https://de.wikiquote.org 8MB
dewikisource https://de.wikisource.org 1326MB
dewikiversity https://de.wikiversity.org 86MB
dewikivoyage https://de.wikivoyage.org 322MB
dewiktionary https://de.wiktionary.org 1745MB
dinwiki https://din.wikipedia.org 1MB
diqwiki https://diq.wikipedia.org 194MB
dsbwiki https://dsb.wikipedia.org 12MB
dtywiki https://dty.wikipedia.org 15MB
dvwiki https://dv.wikipedia.org 13MB
dvwiktionary https://dv.wiktionary.org 1MB
dzwiki https://dz.wikipedia.org 1MB
eewiki https://ee.wikipedia.org 1MB
elwiki https://el.wikipedia.org 2163MB
elwikibooks https://el.wikibooks.org 13MB
elwikinews https://el.wikinews.org 10MB
elwikiquote https://el.wikiquote.org 5MB
elwikisource https://el.wikisource.org 130MB
elwikiversity https://el.wikiversity.org 6MB
elwikivoyage https://el.wikivoyage.org 11MB
elwiktionary https://el.wiktionary.org 1041MB
emlwiki https://eml.wikipedia.org 36MB
enwiki https://en.wikipedia.org 85543MB
enwikibooks https://en.wikibooks.org 579MB
enwikinews https://en.wikinews.org 121MB
enwikiquote https://en.wikiquote.org 384MB
enwikisource https://en.wikisource.org 3520MB
enwikiversity https://en.wikiversity.org 296MB
enwikivoyage https://en.wikivoyage.org 392MB
enwiktionary https://en.wiktionary.org 8454MB
eowiki https://eo.wikipedia.org 1583MB
eowikibooks https://eo.wikibooks.org 3MB
eowikinews https://eo.wikinews.org 5MB
eowikiquote https://eo.wikiquote.org 9MB
eowikisource https://eo.wikisource.org 36MB
eowiktionary https://eo.wiktionary.org 127MB
eswiki https://es.wikipedia.org 15212MB
eswikibooks https://es.wikibooks.org 80MB
eswikinews https://es.wikinews.org 79MB
eswikiquote https://es.wikiquote.org 28MB
eswikisource https://es.wikisource.org 604MB
eswikiversity https://es.wikiversity.org 18MB
eswikivoyage https://es.wikivoyage.org 36MB
eswiktionary https://es.wiktionary.org 851MB
etwiki https://et.wikipedia.org 906MB
etwikibooks https://et.wikibooks.org 1MB
etwikiquote https://et.wikiquote.org 17MB
etwikisource https://et.wikisource.org 15MB
etwiktionary https://et.wiktionary.org 93MB
euwiki https://eu.wikipedia.org 2213MB
euwikibooks https://eu.wikibooks.org 9MB
euwikiquote https://eu.wikiquote.org 1MB
euwikisource https://eu.wikisource.org 3MB
euwiktionary https://eu.wiktionary.org 81MB
extwiki https://ext.wikipedia.org 6MB
fawiki https://fa.wikipedia.org 6940MB
fawikibooks https://fa.wikibooks.org 11MB
fawikinews https://fa.wikinews.org 5MB
fawikiquote https://fa.wikiquote.org 29MB
fawikisource https://fa.wikisource.org 94MB
fawikivoyage https://fa.wikivoyage.org 21MB
fawiktionary https://fa.wiktionary.org 77MB
ffwiki https://ff.wikipedia.org 1MB
fiu_vrowiki https://fiu-vro.wikipedia.org 10MB
fiwiki https://fi.wikipedia.org 3224MB
fiwikibooks https://fi.wikibooks.org 24MB
fiwikinews https://fi.wikinews.org 4MB
fiwikiquote https://fi.wikiquote.org 1MB
fiwikisource https://fi.wikisource.org 45MB
fiwikiversity https://fi.wikiversity.org 6MB
fiwikivoyage https://fi.wikivoyage.org 2MB
fiwiktionary https://fi.wiktionary.org 367MB
fjwiki https://fj.wikipedia.org 1MB
fjwiktionary https://fj.wiktionary.org 8MB
fowiki https://fo.wikipedia.org 44MB
fowikisource https://fo.wikisource.org 1MB
fowiktionary https://fo.wiktionary.org 2MB
frpwiki https://frp.wikipedia.org 13MB
frrwiki https://frr.wikipedia.org 31MB
frwiki https://fr.wikipedia.org 28915MB
frwikibooks https://fr.wikibooks.org 115MB
frwikinews https://fr.wikinews.org 142MB
frwikiquote https://fr.wikiquote.org 36MB
frwikisource https://fr.wikisource.org 2983MB
frwikiversity https://fr.wikiversity.org 119MB
frwikivoyage https://fr.wikivoyage.org 116MB
frwiktionary https://fr.wiktionary.org 4506MB
furwiki https://fur.wikipedia.org 8MB
fywiki https://fy.wikipedia.org 188MB
fywikibooks https://fy.wikibooks.org 1MB
fywiktionary https://fy.wiktionary.org 13MB
gagwiki https://gag.wikipedia.org 11MB
ganwiki https://gan.wikipedia.org 7MB
gawiki https://ga.wikipedia.org 172MB
gawiktionary https://ga.wiktionary.org 4MB
gcrwiki https://gcr.wikipedia.org 2MB
gdwiki https://gd.wikipedia.org 25MB
gdwiktionary https://gd.wiktionary.org 14MB
glkwiki https://glk.wikipedia.org 19MB
glwiki https://gl.wikipedia.org 1319MB
glwikibooks https://gl.wikibooks.org 2MB
glwikiquote https://gl.wikiquote.org 1MB
glwikisource https://gl.wikisource.org 2MB
glwiktionary https://gl.wiktionary.org 97MB
gnwiki https://gn.wikipedia.org 13MB
gnwiktionary https://gn.wiktionary.org 1MB
gomwiki https://gom.wikipedia.org 17MB
gomwiktionary https://gom.wiktionary.org 2MB
gorwiki https://gor.wikipedia.org 12MB
gotwiki https://got.wikipedia.org 2MB
guwiki https://gu.wikipedia.org 198MB
guwikiquote https://gu.wikiquote.org 1MB
guwikisource https://gu.wikisource.org 32MB
guwiktionary https://gu.wiktionary.org 2MB
gvwiki https://gv.wikipedia.org 22MB
gvwiktionary https://gv.wiktionary.org 1MB
hakwiki https://hak.wikipedia.org 29MB
hawiki https://ha.wikipedia.org 24MB
hawiktionary https://ha.wiktionary.org 1MB
hawwiki https://haw.wikipedia.org 3MB
hewiki https://he.wikipedia.org 3265MB
hewikibooks https://he.wikibooks.org 31MB
hewikinews https://he.wikinews.org 3MB
hewikiquote https://he.wikiquote.org 6MB
hewikisource https://he.wikisource.org 1066MB
hewikivoyage https://he.wikivoyage.org 52MB
hewiktionary https://he.wiktionary.org 58MB
hifwiki https://hif.wikipedia.org 22MB
hifwiktionary https://hif.wiktionary.org 1MB
hiwiki https://hi.wikipedia.org 1205MB
hiwikibooks https://hi.wikibooks.org 7MB
hiwikiquote https://hi.wikiquote.org 1MB
hiwikisource https://hi.wikisource.org 20MB
hiwikiversity https://hi.wikiversity.org 1MB
hiwikivoyage https://hi.wikivoyage.org 1MB
hiwiktionary https://hi.wiktionary.org 86MB
hrwiki https://hr.wikipedia.org 1108MB
hrwikibooks https://hr.wikibooks.org 2MB
hrwikiquote https://hr.wikiquote.org 5MB
hrwikisource https://hr.wikisource.org 49MB
hrwiktionary https://hr.wiktionary.org 3MB
hsbwiki https://hsb.wikipedia.org 37MB
hsbwiktionary https://hsb.wiktionary.org 19MB
htwiki https://ht.wikipedia.org 109MB
huwiki https://hu.wikipedia.org 5351MB
huwikibooks https://hu.wikibooks.org 190MB
huwikiquote https://hu.wikiquote.org 3MB
huwikisource https://hu.wikisource.org 89MB
huwiktionary https://hu.wiktionary.org 1368MB
hywiki https://hy.wikipedia.org 2957MB
hywikibooks https://hy.wikibooks.org 1MB
hywikiquote https://hy.wikiquote.org 6MB
hywikisource https://hy.wikisource.org 49MB
hywiktionary https://hy.wiktionary.org 376MB
hywwiki https://hyw.wikipedia.org 72MB
iawiki https://ia.wikipedia.org 73MB
iawikibooks https://ia.wikibooks.org 1MB
iawiktionary https://ia.wiktionary.org 2MB
idwiki https://id.wikipedia.org 4017MB
idwikibooks https://id.wikibooks.org 25MB
idwikiquote https://id.wikiquote.org 2MB
idwikisource https://id.wikisource.org 24MB
idwiktionary https://id.wiktionary.org 573MB
iewiki https://ie.wikipedia.org 14MB
iewiktionary https://ie.wiktionary.org 1MB
igwiki https://ig.wikipedia.org 6MB
ikwiki https://ik.wikipedia.org 1MB
ilowiki https://ilo.wikipedia.org 108MB
inhwiki https://inh.wikipedia.org 6MB
iowiki https://io.wikipedia.org 49MB
iowiktionary https://io.wiktionary.org 400MB
iswiki https://is.wikipedia.org 189MB
iswikibooks https://is.wikibooks.org 2MB
iswikiquote https://is.wikiquote.org 1MB
iswikisource https://is.wikisource.org 10MB
iswiktionary https://is.wiktionary.org 51MB
itwiki https://it.wikipedia.org 17209MB
itwikibooks https://it.wikibooks.org 105MB
itwikinews https://it.wikinews.org 56MB
itwikiquote https://it.wikiquote.org 189MB
itwikisource https://it.wikisource.org 642MB
itwikiversity https://it.wikiversity.org 50MB
itwikivoyage https://it.wikivoyage.org 86MB
itwiktionary https://it.wiktionary.org 520MB
iuwiki https://iu.wikipedia.org 1MB
iuwiktionary https://iu.wiktionary.org 1MB
jamwiki https://jam.wikipedia.org 2MB
jawiki https://ja.wikipedia.org 16597MB
jawikibooks https://ja.wikibooks.org 23MB
jawikinews https://ja.wikinews.org 11MB
jawikiquote https://ja.wikiquote.org 3MB
jawikisource https://ja.wikisource.org 139MB
jawikiversity https://ja.wikiversity.org 1MB
jawikivoyage https://ja.wikivoyage.org 2MB
jawiktionary https://ja.wiktionary.org 420MB
jbowiki https://jbo.wikipedia.org 1MB
jbowiktionary https://jbo.wiktionary.org 1MB
jvwiki https://jv.wikipedia.org 292MB
jvwiktionary https://jv.wiktionary.org 38MB
kaawiki https://kaa.wikipedia.org 3MB
kabwiki https://kab.wikipedia.org 6MB
kawiki https://ka.wikipedia.org 1013MB
kawikibooks https://ka.wikibooks.org 3MB
kawikiquote https://ka.wikiquote.org 3MB
kawiktionary https://ka.wiktionary.org 21MB
kbdwiki https://kbd.wikipedia.org 6MB
kbpwiki https://kbp.wikipedia.org 3MB
kgwiki https://kg.wikipedia.org 2MB
kiwiki https://ki.wikipedia.org 1MB
kkwiki https://kk.wikipedia.org 1395MB
kkwikibooks https://kk.wikibooks.org 2MB
kkwiktionary https://kk.wiktionary.org 1MB
klwiki https://kl.wikipedia.org 1MB
klwiktionary https://kl.wiktionary.org 1MB
kmwiki https://km.wikipedia.org 74MB
kmwikibooks https://km.wikibooks.org 3MB
kmwiktionary https://km.wiktionary.org 14MB
knwiki https://kn.wikipedia.org 304MB
knwikiquote https://kn.wikiquote.org 1MB
knwikisource https://kn.wikisource.org 86MB
knwiktionary https://kn.wiktionary.org 104MB
koiwiki https://koi.wikipedia.org 12MB
kowiki https://ko.wikipedia.org 4366MB
kowikibooks https://ko.wikibooks.org 10MB
kowikinews https://ko.wikinews.org 1MB
kowikiquote https://ko.wikiquote.org 2MB
kowikisource https://ko.wikisource.org 431MB
kowikiversity https://ko.wikiversity.org 1MB
kowiktionary https://ko.wiktionary.org 284MB
krcwiki https://krc.wikipedia.org 10MB
kshwiki https://ksh.wikipedia.org 6MB
kswiki https://ks.wikipedia.org 1MB
kswiktionary https://ks.wiktionary.org 1MB
kuwiki https://ku.wikipedia.org 110MB
kuwikibooks https://ku.wikibooks.org 2MB
kuwikiquote https://ku.wikiquote.org 1MB
kuwiktionary https://ku.wiktionary.org 492MB
kvwiki https://kv.wikipedia.org 19MB
kwwiki https://kw.wikipedia.org 8MB
kwwiktionary https://kw.wiktionary.org 1MB
kywiki https://ky.wikipedia.org 268MB
kywikibooks https://ky.wikibooks.org 3MB
kywikiquote https://ky.wikiquote.org 1MB
kywiktionary https://ky.wiktionary.org 37MB
ladwiki https://lad.wikipedia.org 9MB
lawiki https://la.wikipedia.org 435MB
lawikibooks https://la.wikibooks.org 1MB
lawikiquote https://la.wikiquote.org 2MB
lawikisource https://la.wikisource.org 201MB
lawiktionary https://la.wiktionary.org 39MB
lbewiki https://lbe.wikipedia.org 2MB
lbwiki https://lb.wikipedia.org 217MB
lbwiktionary https://lb.wiktionary.org 6MB
lezwiki https://lez.wikipedia.org 28MB
lfnwiki https://lfn.wikipedia.org 9MB
lgwiki https://lg.wikipedia.org 1MB
lijwiki https://lij.wikipedia.org 41MB
lijwikisource https://lij.wikisource.org 4MB
liwiki https://li.wikipedia.org 47MB
liwikibooks https://li.wikibooks.org 1MB
liwikinews https://li.wikinews.org 2MB
liwikiquote https://li.wikiquote.org 1MB
liwikisource https://li.wikisource.org 3MB
liwiktionary https://li.wiktionary.org 67MB
lldwiki https://lld.wikipedia.org 4MB
lmowiki https://lmo.wikipedia.org 221MB
lnwiki https://ln.wikipedia.org 5MB
lnwiktionary https://ln.wiktionary.org 1MB
lowiki https://lo.wikipedia.org 17MB
lowiktionary https://lo.wiktionary.org 34MB
ltgwiki https://ltg.wikipedia.org 2MB
ltwiki https://lt.wikipedia.org 1049MB
ltwikibooks https://lt.wikibooks.org 2MB
ltwikiquote https://lt.wikiquote.org 5MB
ltwikisource https://lt.wikisource.org 6MB
ltwiktionary https://lt.wiktionary.org 1089MB
lvwiki https://lv.wikipedia.org 748MB
lvwiktionary https://lv.wiktionary.org 11MB
maiwiki https://mai.wikipedia.org 101MB
map_bmswiki https://map-bms.wikipedia.org 15MB
mdfwiki https://mdf.wikipedia.org 2MB
mgwiki https://mg.wikipedia.org 161MB
mgwikibooks https://mg.wikibooks.org 1MB
mgwiktionary https://mg.wiktionary.org 973MB
mhrwiki https://mhr.wikipedia.org 61MB
minwiki https://min.wikipedia.org 358MB
minwiktionary https://min.wiktionary.org 2MB
miwiki https://mi.wikipedia.org 12MB
miwiktionary https://mi.wiktionary.org 1MB
mkwiki https://mk.wikipedia.org 1013MB
mkwikibooks https://mk.wikibooks.org 2MB
mkwikisource https://mk.wikisource.org 11MB
mkwiktionary https://mk.wiktionary.org 1MB
mlwiki https://ml.wikipedia.org 694MB
mlwikibooks https://ml.wikibooks.org 1MB
mlwikiquote https://ml.wikiquote.org 1MB
mlwikisource https://ml.wikisource.org 48MB
mlwiktionary https://ml.wiktionary.org 55MB
mnwiki https://mn.wikipedia.org 133MB
mnwiktionary https://mn.wiktionary.org 41MB
mnwwiki https://mnw.wikipedia.org 7MB
mrjwiki https://mrj.wikipedia.org 17MB
mrwiki https://mr.wikipedia.org 357MB
mrwikibooks https://mr.wikibooks.org 2MB
mrwikiquote https://mr.wikiquote.org 1MB
mrwikisource https://mr.wikisource.org 28MB
mrwiktionary https://mr.wiktionary.org 2MB
mswiki https://ms.wikipedia.org 2164MB
mswikibooks https://ms.wikibooks.org 1MB
mswiktionary https://ms.wiktionary.org 18MB
mtwiki https://mt.wikipedia.org 34MB
mtwiktionary https://mt.wiktionary.org 2MB
mwlwiki https://mwl.wikipedia.org 28MB
myvwiki https://myv.wikipedia.org 46MB
mywiki https://my.wikipedia.org 931MB
mywiktionary https://my.wiktionary.org 84MB
mznwiki https://mzn.wikipedia.org 67MB
nahwiki https://nah.wikipedia.org 8MB
nahwiktionary https://nah.wiktionary.org 9MB
napwiki https://nap.wikipedia.org 23MB
napwikisource https://nap.wikisource.org 1MB
nawiki https://na.wikipedia.org 2MB
nawiktionary https://na.wiktionary.org 1MB
nds_nlwiki https://nds-nl.wikipedia.org 12MB
ndswiki https://nds.wikipedia.org 168MB
ndswiktionary https://nds.wiktionary.org 15MB
newiki https://ne.wikipedia.org 195MB
newikibooks https://ne.wikibooks.org 1MB
newiktionary https://ne.wiktionary.org 12MB
newwiki https://new.wikipedia.org 247MB
nlwiki https://nl.wikipedia.org 8339MB
nlwikibooks https://nl.wikibooks.org 46MB
nlwikinews https://nl.wikinews.org 14MB
nlwikiquote https://nl.wikiquote.org 2MB
nlwikisource https://nl.wikisource.org 44MB
nlwikivoyage https://nl.wikivoyage.org 20MB
nlwiktionary https://nl.wiktionary.org 992MB
nnwiki https://nn.wikipedia.org 729MB
nnwikiquote https://nn.wikiquote.org 2MB
nnwiktionary https://nn.wiktionary.org 9MB
novwiki https://nov.wikipedia.org 3MB
nowiki https://no.wikipedia.org 3626MB
nowikibooks https://no.wikibooks.org 2MB
nowikinews https://no.wikinews.org 1MB
nowikiquote https://no.wikiquote.org 2MB
nowikisource https://no.wikisource.org 31MB
nowiktionary https://no.wiktionary.org 96MB
nqowiki https://nqo.wikipedia.org 2MB
nrmwiki https://nrm.wikipedia.org 10MB
nsowiki https://nso.wikipedia.org 23MB
nvwiki https://nv.wikipedia.org 82MB
nywiki https://ny.wikipedia.org 2MB
ocwiki https://oc.wikipedia.org 935MB
ocwikibooks https://oc.wikibooks.org 1MB
ocwiktionary https://oc.wiktionary.org 93MB
olowiki https://olo.wikipedia.org 6MB
omwiki https://om.wikipedia.org 1MB
omwiktionary https://om.wiktionary.org 3MB
orwiki https://or.wikipedia.org 120MB
orwikisource https://or.wikisource.org 11MB
orwiktionary https://or.wiktionary.org 77MB
oswiki https://os.wikipedia.org 44MB
pagwiki https://pag.wikipedia.org 18MB
pamwiki https://pam.wikipedia.org 45MB
papwiki https://pap.wikipedia.org 4MB
pawiki https://pa.wikipedia.org 228MB
pawikibooks https://pa.wikibooks.org 1MB
pawikisource https://pa.wikisource.org 5MB
pawiktionary https://pa.wiktionary.org 11MB
pcdwiki https://pcd.wikipedia.org 23MB
pdcwiki https://pdc.wikipedia.org 3MB
pflwiki https://pfl.wikipedia.org 12MB
pihwiki https://pih.wikipedia.org 2MB
piwiki https://pi.wikipedia.org 3MB
plwiki https://pl.wikipedia.org 11521MB
plwikibooks https://pl.wikibooks.org 65MB
plwikinews https://pl.wikinews.org 44MB
plwikiquote https://pl.wikiquote.org 61MB
plwikisource https://pl.wikisource.org 1081MB
plwikivoyage https://pl.wikivoyage.org 8MB
plwiktionary https://pl.wiktionary.org 1871MB
pmswiki https://pms.wikipedia.org 147MB
pmswikisource https://pms.wikisource.org 9MB
pnbwiki https://pnb.wikipedia.org 433MB
pnbwiktionary https://pnb.wiktionary.org 4MB
pntwiki https://pnt.wikipedia.org 2MB
pswiki https://ps.wikipedia.org 68MB
pswikivoyage https://ps.wikivoyage.org 1MB
pswiktionary https://ps.wiktionar.wikipedia.org 11172MB
ptwikibooks https://pt.wikibooks.org 39MB
ptwikinews https://pt.wikinews.org 167MB
ptwikiquote https://pt.wikiquote.org 15MB
ptwikisource https://pt.wikisource.org 134MB
ptwikiversity https://pt.wikiversity.org 24MB
ptwikivoyage https://pt.wikivoyage.org 11MB
ptwiktionary https://pt.wiktionary.org 456MB
quwiki https://qu.wikipedia.org 82MB
quwiktionary https://qu.wiktionary.org 1MB
rmwiki https://rm.wikipedia.org 21MB
rmywiki https://rmy.wikipedia.org 2MB
rnwiki https://rn.wikipedia.org 1MB
roa_rupwiki https://roa-rup.wikipedia.org 2MB
roa_rupwiktionary https://roa-rup.wiktionary.org 1MB
roa_tarawiki https://roa-tara.wikipedia.org 35MB
rowiki https://ro.wikipedia.org 4047MB
rowikibooks https://ro.wikibooks.org 3MB
rowikinews https://ro.wikinews.org 3MB
rowikiquote https://ro.wikiquote.org 1MB
rowikisource https://ro.wikisource.org 86MB
rowikivoyage https://ro.wikivoyage.org 2MB
rowiktionary https://ro.wiktionary.org 259MB
ruewiki https://rue.wikipedia.org 17MB
ruwiki https://ru.wikipedia.org 22711MB
ruwikibooks https://ru.wikibooks.org 54MB
ruwikinews https://ru.wikinews.org 4312MB
ruwikiquote https://ru.wikiquote.org 168MB
ruwikisource https://ru.wikisource.org 2346MB
ruwikiversity https://ru.wikiversity.org 30MB
ruwikivoyage https://ru.wikivoyage.org 176MB
ruwiktionary https://ru.wiktionary.org 4239MB
rwwiki https://rw.wikipedia.org 4MB
rwwiktionary https://rw.wiktionary.org 1MB
sahwiki https://sah.wikipedia.org 41MB
sahwikiquote https://sah.wikiquote.org 1MB
sahwikisource https://sah.wikisource.org 4MB
satwiki https://sat.wikipedia.org 22MB
sawiki https://sa.wikipedia.org 90MB
sawikibooks https://sa.wikibooks.org 10MB
sawikiquote https://sa.wikiquote.org 5MB
sawikisource https://sa.wikisource.org 468MB
sawiktionary https://sa.wiktionary.org 201MB
scnwiki https://scn.wikipedia.org 45MB
scnwiktionary https://scn.wiktionary.org 24MB
scowiki https://sco.wikipedia.org 241MB
scwiki https://sc.wikipedia.org 25MB
sdwiki https://sd.wikipedia.org 57MB
sdwiktionary https://sd.wiktionary.org 4MB
sewiki https://se.wikipedia.org 18MB
sgwiki https://sg.wikipedia.org 1MB
sgwiktionary https://sg.wiktionary.org 1MB
shnwiki https://shn.wikipedia.org 56MB
shnwiktionary https://shn.wiktionary.org 13MB
shwiki https://sh.wikipedia.org 5632MB
shwiktionary https://sh.wiktionary.org 640MB
simplewiki https://simple.wikipedia.org 1034MB
simplewiktionary https://simple.wiktionary.org 25MB
siwiki https://si.wikipedia.org 136MB
siwikibooks https://si.wikibooks.org 38MB
siwiktionary https://si.wiktionary.org 1MB
skwiki https://sk.wikipedia.org 1685MB
skwikibooks https://sk.wikibooks.org 1MB
skwikiquote https://sk.wikiquote.org 10MB
skwikisource https://sk.wikisource.org 1MB
skwiktionary https://sk.wiktionary.org 21MB
slwiki https://sl.wikipedia.org 1353MB
slwikibooks https://sl.wikibooks.org 2MB
slwikiquote https://sl.wikiquote.org 6MB
slwikisource https://sl.wikisource.org 288MB
slwikiversity https://sl.wikiversity.org 7MB
slwiktionary https://sl.wiktionary.org 18MB
smwiki https://sm.wikipedia.org 1MB
smwiktionary https://sm.wiktionary.org 2MB
snwiki https://sn.wikipedia.org 4MB
sowiki https://so.wikipedia.org 28MB
sowiktionary https://so.wiktionary.org 1MB
sqwiki https://sq.wikipedia.org 457MB
sqwikibooks https://sq.wikibooks.org 24MB
sqwikinews https://sq.wikinews.org 1MB
sqwikiquote https://sq.wikiquote.org 1MB
sqwiktionary https://sq.wiktionary.org 20MB
srnwiki https://srn.wikipedia.org 1MB
srwiki https://sr.wikipedia.org 11076MB
srwikibooks https://sr.wikibooks.org 1MB
srwikinews https://sr.wikinews.org 118MB
srwikiquote https://sr.wikiquote.org 1MB
srwikisource https://sr.wikisource.org 178MB
srwiktionary https://sr.wiktionary.org 1007MB
sswiki https://ss.wikipedia.org 1MB
sswiktionary https://ss.wiktionary.org 1MB
stqwiki https://stq.wikipedia.org 11MB
stwiki https://st.wikipedia.org 2MB
stwiktionary https://st.wiktionary.org 1MB
suwiki https://su.wikipedia.org 184MB
suwikiquote https://su.wikiquote.org 2MB
suwiktionary https://su.wiktionary.org 1MB
svwiki https://sv.wikipedia.org 21799MB
svwikibooks https://sv.wikibooks.org 6MB
svwikinews https://sv.wikinews.org 7MB
svwikiquote https://sv.wikiquote.org 2MB
svwikisource https://sv.wikisource.org 55MB
svwikiversity https://sv.wikiversity.org 1MB
svwikivoyage https://sv.wikivoyage.org 6MB
svwiktionary https://sv.wiktionary.org 430MB
swwiki https://sw.wikipedia.org 173MB
swwiktionary https://sw.wiktionary.org 9MB
szlwiki https://szl.wikipedia.org 113MB
szywiki https://szy.wikipedia.org 9MB
tawiki https://ta.wikipedia.org 1091MB
tawikibooks https://ta.wikibooks.org 4MB
tawikinews https://ta.wikinews.org 12MB
tawikiquote https://ta.wikiquote.org 4MB
tawikisource https://ta.wikisource.org 122MB
tawiktionary https://ta.wiktionary.org 319MB
tcywiki https://tcy.wikipedia.org 7MB
tetwiki https://tet.wikipedia.org 4MB
tewiki https://te.wikipedia.org 734MB
tewikibooks https://te.wikibooks.org 4MB
tewikiquote https://te.wikiquote.org 2MB
tewikisource https://te.wikisource.org 106MB
tewiktionary https://te.wiktionary.org 105MB
tgwiki https://tg.wikipedia.org 603MB
tgwikibooks https://tg.wikibooks.org 1MB
tgwiktionary https://tg.wiktionary.org 25MB
thwiki https://th.wikipedia.org 1672MB
thwikibooks https://th.wikibooks.org 11MB
thwikiquote https://th.wikiquote.org 5MB
thwikisource https://th.wikisource.org 114MB
thwiktionary https://th.wiktionary.org 392MB
tiwiki https://ti.wikipedia.org 1MB
tiwiktionary https://ti.wiktionary.org 1MB
tkwiki https://tk.wikipedia.org 6MB
tkwiktionary https://tk.wiktionary.org 2MB
tlwiki https://tl.wikipedia.org 300MB
tlwikibooks https://tl.wikibooks.org 1MB
tlwiktionary https://tl.wiktionary.org 13MB
tnwiki https://tn.wikipedia.org 1MB
tnwiktionary https://tn.wiktionary.org 1MB
towiki https://to.wikipedia.org 2MB
tpiwiki https://tpi.wikipedia.org 4MB
tpiwiktionary https://tpi.wiktionary.org 1MB
trwiki https://tr.wikipedia.org 3415MB
trwikibooks https://tr.wikibooks.org 4MB
trwikiquote https://tr.wikiquote.org 14MB
trwikisource https://tr.wikisource.org 49MB
trwiktionary https://tr.wiktionary.org 295MB
tswiki https://ts.wikipedia.org 3MB
tswiktionary https://ts.wiktionary.org 1MB
ttwiki https://tt.wikipedia.org 2056MB
ttwikibooks https://tt.wikibooks.org 1MB
ttwiktionary https://tt.wiktionary.org 5MB
tumwiki https://tum.wikipedia.org 1MB
twwiki https://tw.wikipedia.org 1MB
tyvwiki https://tyv.wikipedia.org 14MB
tywiki https://ty.wikipedia.org 1MB
udmwiki https://udm.wikipedia.org 14MB
ugwiki https://ug.wikipedia.org 28MB
ugwiktionary https://ug.wiktionary.org 1MB
ukwiki https://uk.wikipedia.org 12307MB
ukwikibooks https://uk.wikibooks.org 9MB
ukwikinews https://uk.wikinews.org 12MB
ukwikiquote https://uk.wikiquote.org 38MB
ukwikisource https://uk.wikisource.org 102MB
ukwikivoyage https://uk.wikivoyage.org 5MB
ukwiktionary https://uk.wiktionary.org 200MB
urwiki https://ur.wikipedia.org 1203MB
urwikibooks https://ur.wikibooks.org 1MB
urwikiquote https://ur.wikiquote.org 1MB
urwiktionary https://ur.wiktionary.org 9MB
uzwiki https://uz.wikipedia.org 490MB
uzwikiquote https://uz.wikiquote.org 1MB
uzwiktionary https://uz.wiktionary.org 133MB
vecwiki https://vec.wikipedia.org 449MB
vecwikisource https://vec.wikisource.org 16MB
vecwiktionary https://vec.wiktionary.org 7MB
vepwiki https://vep.wikipedia.org 15MB
vewiki https://ve.wikipedia.org 1MB
viwiki https://vi.wikipedia.org 7044MB
viwikibooks https://vi.wikibooks.org 17MB
viwikiquote https://vi.wikiquote.org 1MB
viwikisource https://vi.wikisource.org 78MB
viwikivoyage https://vi.wikivoyage.org 8MB
viwiktionary https://vi.wiktionary.org 291MB
vlswiki https://vls.wikipedia.org 12MB
vowiki https://vo.wikipedia.org 151MB
vowiktionary https://vo.wiktionary.org 23MB
warwiki https://war.wikipedia.org 3681MB
wawiki https://wa.wikipedia.org 18MB
wawiktionary https://wa.wiktionary.org 62MB
wowiki https://wo.wikipedia.org 4MB
wowikiquote https://wo.wikiquote.org 1MB
wowiktionary https://wo.wiktionary.org 2MB
wuuwiki https://wuu.wikipedia.org 56MB
xalwiki https://xal.wikipedia.org 7MB
xhwiki https://xh.wikipedia.org 3MB
xmfwiki https://xmf.wikipedia.org 68MB
yiwiki https://yi.wikipedia.org 30MB
yiwikisource https://yi.wikisource.org 15MB
yiwiktionary https://yi.wiktionary.org 1MB
yowiki https://yo.wikipedia.org 92MB
yuewiktionary https://yue.wiktionary.org 7MB
zawiki https://za.wikipedia.org 2MB
zeawiki https://zea.wikipedia.org 12MB
zh_classicalwiki https://zh-classical.wikipedia.org 27MB
zh_min_nanwiki https://zh-min-nan.wikipedia.org 1081MB
zh_min_nanwikisource https://zh-min-nan.wikisource.org 9MB
zh_min_nanwiktionary https://zh-min-nan.wiktionary.org 11MB
zhwiki https://zh.wikipedia.org 16412MB
zhwikibooks https://zh.wikibooks.org 39MB
zhwikinews https://zh.wikinews.org 101MB
zhwikiquote https://zh.wikiquote.org 17MB
zhwikisource https://zh.wikisource.org 8973MB
zhwikiversity https://zh.wikiversity.org 21MB
zhwikivoyage https://zh.wikivoyage.org 20MB
zhwiktionary https://zh.wiktionary.org 1417MB
zh_yuewiki https://zh-yue.wikipedia.org 408MB
zuwiki https://zu.wikipedia.org 16MB
zuwiktionary https://zu.wiktionary.org 1MB

Firehose API

High-volume reusers currently rely heavily on the changes that are pushed from our community to update their products in real time, using EventStream APIs to access such changes.High-volume reusers are interested in a service that will allow them to filter the changes they receive to limit their processing, guarantee stable HTTP connections to ensure no data loss, and supply a more useful schema to limit the number of api calls they need to make per event.

Enterprise Firehose API, at release, will contain:

  • Filtering of events by Project or Revision Namespace
  • Guaranteed connections
  • Commercially useful schema similar* to those that we are building in our Structured Content API and Bulk API
  • SLA

*We are still in the process of mapping out the technical specifications to determine the limitations of schema in event platforms and will post here when we have finalized our design.

Bulk API

For high volume reusers that currently rely on the Wikimedia Dumps to access our information, we have created a solution to ingest Wikimedia content in near real time without excessive API calls (Structured Content API) or maintaining hooks into our infrastructure (Firehose).

Enterprise Bulk API, at release, will contain:

  • 24-hour JSON*, Wikitext, or HTML compressed dumps of "text-based" Wikimedia projects
  • A hourly update file with revision changes of "text-based" Wikimedia projects
  • SLA

*JSON dumps will contain the same schema per page as the Structured Content API.

These dumps will be available for public use bi-weekly on Wikimedia Dumps and for WMCS users coming in June 2021

Past Development

Daily HTML Dumps

The Enterprise team's first product was building daily dump files of HTML for every "text-based" Wikimedia project. These dumps will help content re-users use a more familiar data type as they work with Wikimedia content.

Reusers have four immediate needs from a service that supports large-scale content reuse: system reliability, high frequency or real-time access, content integrity, and machine readability.

Web Interface

This is a screenshot from the alpha dashboard (when the project was codenamed "Okapi") where users can download and save daily exports of HTML from "text-based" Wikimedia projects

A downloader interface now in design stages allows for users to download a daily dump for each "text-based" project, search and download individual pages, and save their preferences for return visits. Currently the software is in Alpha and still in usage and quality testing. This dashboard is built in React with internal-facing client endpoints built on top of our infrastructure. The downloads are hosted and served through S3.

Rationale behind choosing this as the Enterprise API's first product

  • Already validated: Before the Enterprise team ran research to discover the needs of high-volume data reusers, this was the most historically requested feature. Large technology partners, researchers, and internal stakeholders within the Wikimedia Foundation have long sought a comprehensive way to access all of the Wikimedia "text-based" wikis in a form outside of Wikitext.
  • Take pressure off internal Wikimedia infrastructure: While not proven, anecdotally we can conclude there is a significant band of traffic to our APIs by high-volume reusers aiming to get the most up-to-date content cached on their systems for reuse. Building a tool where they can achieve this has been the first step to pulling high-volume reusers away from WMF infrastructure and onto a new service.
  • Standalone in nature: Of the projects already laid out for consideration by the Enterprise team, this is the most standalone. We can easily understand the specs without working with a specific partner. We were not forced to make technical decisions that would affect a later product or offering. In fact, in many ways, this flexibility forced us to build a data platform that produced many of the APIs that we are offering in the near future.
  • Strong business development case: This project gave the Enterprise team a lot of room to talk through solutions with reusers and open up business development conversations.
  • Strong introductory project for contractors: The Enterprise team started with a team of outside contractors. This forced the team to become reusers of Wikimedia in order to build this product. In the process, the team was able to identify and relate to the problems with the APIs that our customer base faces, giving them a broader understanding of the issues at hand.

Design Documents

Okapi architecture.png

Application Hosting

The engineering goal of this project is to rapidly prototype and build solutions that could scale to the needs of the Enterprise API's intended customers – high volume, high speed, commercial reusers. To do this, the product has been optimized for quick iteration, infrastructural separation from critical Wikimedia projects, and to utilize downstream Service Level Agreements (SLAs). To achieve these goals in the short term, we have built the Enterprise API upon a third-party cloud provider (specifically Amazon Web Services [AWS]). While there are many advantages of using external cloud for our use case, we acknowledge there are also fundamental tensions – given the culture and principles of how applications are built at the Foundation.

Consequently, the goal with the Enterprise API is to create an application that is "cloud-agnostic" and can be spun up on any provider's platform. We have taken reasonable steps to architect abstraction layers within our application to remove any overt dependencies on our current host, Amazon Web Services. This was also a pragmatic decision, due to the unclear nature of where this project will live long-term.

The following steps were taken to ensure that principle. We have:

  • Designed and built service interfaces to create abstractions from provider-specific tools. For instance, we have layers that tie to general File Storage capabilities, decoupling us from using exclusively "AWS S3" or creating undo dependency on other potential cloud options
  • Built the application using Terraform as Infrastructure as Code to manage our cloud services. [The Terraform code will be published in the near future and this documentation will be updated when it is]
  • Used Docker for containerization throughout the application
  • Implemented hard drive encryption to ensure that the data is protected (we are working to expand our data encryption and will continually as this project develops)

We have intentionally kept our technical stack as general, libre & open source, and lightweight as possible. There is a temptation to use a number of proprietary services that may provide easy solutions to hard problems (including EMR, DynamoDB, etc). However, we have restricted our reliance on Amazon services to what we can be found in most other cloud providers. Below is a list of services used by the Enterprise API within Amazon and its purpose in our infrastructure:

We are looking to provide Service Level Agreements (SLA) to customers similar to those guaranteed by Amazon's EC2. We don't have equivalent uptime information from the Wikimedia Foundation's existing infrastructure. However, this is something we are exploring with Wikimedia Site Reliability Engineering. Any alternative hosting in the future would require equivalent services or time to allow us to add more staff to our team in order to give us confidence to handle the SLA we are promising.

In the meantime, we are researching alternatives to AWS (and remain open to ideas that might fit our use case) when this project is more established and we are confident in knowing what the infrastructure needs are in reality.

Team

We are staffing our engineering team currently with Speed & Function. At this early stage in the project, we are not yet sure of the long-term engineering needs and wish to thoroughly assess the project’s ability to become self-sustaining. In this way, we hope not to disrupt other WMF projects or divert excessive resources.

See also

  • Wikitech: Data Services portal – A list of community-facing services that allow for direct access to databases and dumps, as well as web interfaces for querying and programmatic access to data stores.
  • Enterprise hub – a page for those interested in using the MediaWiki software in corporate contexts:
  • Wikimedia update feed service – A defunct paid data service that enabled third parties to maintain and update local databases of Wikimedia content.
この表: 閲覧 · トーク · 編集
API Availability URL base
MediaWiki-2020-small-icon.svg MediaWiki 操作 API MediaWiki に同梱されています

ウィキメディアのプロジェクト群で有効になっています

/api.php https://ja.wikipedia.org/w/api.php?action=query&prop=info&titles=地球
MediaWiki-2020-small-icon.svg MediaWiki REST API MediaWiki 1.35 以降に同梱されています

ウィキメディアのプロジェクト群で有効になっています

/rest.php https://ja.wikipedia.org/w/rest.php/v1/page/地球
Wikimedia-logo.svg ウィキメディア REST API MediaWiki に同梱されていません

ウィキメディアのプロジェクト群のみで利用できます

/api/rest https://ja.wikipedia.org/api/rest_v1/page/title/地球
For commercial-scale APIs for Wikimedia projects, see Wikimedia Enterprise