Wikimedia Services

We, the Wikimedia Services team, are aiming to shape the Wikimedia service infrastructure with platform services, APIs and tools. We develop scalable APIs with clearly defined Swagger / OpenAPI specs and versioning policies. We automate monitoring, metric reporting and logging. We keep the majority of services simple & stateless by offering general multi-datacenter storage (RESTBase) and change propagation solutions.

Overall, our goal is to make our infrastructure more flexible, robust and efficient by gradually migrating from a monolithic architecture to micro-services.

Services

 * RESTBase, the storage & API proxy service behind the Wikimedia REST API. Built using Node.js and Cassandra.
 * EventBus, a Kafka-based event distribution solution for asynchronous tasks & cross-service communication.
 * ChangeProp, a general change propagation service with support for templated HTTP requests and CDN purging.

Libraries

 * HyperSwitch, a Swagger / OpenAPI spec-based service framework used in RESTBase and ChangeProp.
 * Service-runner, a node.js cluster based service runner providing standardized facilities for logging, metrics & rate limiting.
 * limitation, an efficient node.js rate limiter library with a Kademlia DHT backend.

Services owned by other teams

 * Parsoid, providing a bidirectional conversion between wikitext and HTML. This service powers VisualEditor.
 * Mobile Content Service, a service extracting content and metadata tailored to the needs of our mobile apps.
 * Citoid, a service converting URLs, DOIs and ISBNs to full citation metadata. This powers the citation tool in VisualEditor.
 * Mathoid, offering SVG, MathML and PNG renders of mathematical formulae.
 * Offline Content Generator, rendering PDFs for collections of pages.
 * Content Translation Service, providing semi-automated content translation between projects.
 * Analytics Query Service, a HyperSwitch and Cassandra based service powering the pageview API.
 * ORES, web service that provides machine learning as a service for Wikimedia Projects.

Documentation & resources

 * Services Documentation Portal

Contact the team
Contact the team:
 * services@lists.wikimedia.org mailing list
 * #wikimedia-services IRC channel