Wikimedia Search Platform

English • Türkçe

Search Platform

We are responsible for maintaining and enhancing the various Search features and APIs for MediaWiki.

Group:	Technology
Team:	In teams: Engineering Erik Bernhardson, David Causse, Peter Fischer, Trey Jones, Ryan Kemper, Brian King Management Guillaume Lederrey, Engineering
Since:	December 2017

The Search Platform team (part of Wikimedia Technology) is responsible for maintaining and enhancing the various Search features and APIs for MediaWiki. This includes the CirrusSearch extension which relies on OpenSearch (formerly Elasticsearch), the search backend used at the Wikimedia Foundation to support Wikimedia projects.

Current work by this team is tracked on the Discovery-Search workboard in Phabricator (backlog board).

(Our team formerly maintained the Wikidata Query Service, the SPARQL endpoint used to query Wikidata. It has been taken over by the Wikidata Platform team.)

Mission

[edit]

Our mission is to help people easily discover knowledge on Wikipedia and its sister projects by providing tools and infrastructure for casual readers and expert users with precise needs, while maintaining a strong emphasis on privacy.

Overview

[edit]

We operate and maintain a disparate collection of production services related to content discovery, enabling the wiki community to find information that is not available through simply following links. We also provide a platform on which other people can create tools to support editing and other workflows.

We provide an open-source search engine, backed by an inverted index for non-structured on-wiki data. We work to develop more sophisticated searching with machine learning and natural language processing.

We endeavor to support underserved wiki communities, and we rely on those communities to help us understand their needs and evaluate potential solutions, especially with respect to underserved languages.

We prioritize privacy for logged-in users and anonymity for logged-out users over almost everything else, even when it slows down or complicates development or hinders our ability to collect or use data.

Goals

[edit]

The Search Platform team's goals are part of the Product & Technology department's goals (e.g., P&T FY 25-26 OKRs and other Essential Work related to sustaining the platform). You can see links to some of the Search Platform team's previous quarterly goals: 2017-2021, 2022-2025. (Note that each fiscal year's Q1 is July–September.)

Other Projects

[edit]

APIs

[edit]

Application Programming Interfaces (APIs) provide developers ways to interact with the MediaWiki software.

API:Search and discovery lists the search APIs available and in development.

The Team

[edit]

See the team page on Wikitech.

Communications

[edit]

Moved to wikitech:Search Platform/Contact

Process

[edit]

The Search Platform team uses a Scrumban process, which is a hybrid of Scrum and Kanban. It is described here: Search Platform Process.

Conferences, gatherings, and other events

[edit]

Past events

[edit]

All Hands—January 2018
Hackathon 2018—18–20 May 2018
Wikimania 2018—July 18–22, 2018
17th International Semantic Web Conference (ISWC 2018)—October 8–12, 2018
October 22–25, 2018, Wikimedia Technical Conference (WMTechConf, formerly known as DevSummit) in Portland, Oregon
Late January / early February 2019, All-Hands, San Francisco
May 2019, Hackathon, Prague
Late January / early February 2020, All-Hands, San Francisco
May 2020, Hackathon, Tirana, Albania (virtual due to COVID-19 global pandemic)

Docs and Other Links

[edit]

Moved to wikitech:Search Platform/Documentation

Deployers

[edit]

Useful reference for who can deploy code. It's nice to know whom to bug if you need something:

Person	MediaWiki Deployer	OpenSearch Deployer	Maps Deployer	Graphoid Deployer	Portals Deployer
dcausse
ebernhardsen
jan_drewniak
gehel

Code

[edit]

The Search Platform team supports the following code:

Repository	Phabricator/Diffusion	GitHub mirror
CirrusSearch extension	https://phabricator.wikimedia.org/diffusion/ECIR/	mediawiki-extensions-CirrusSearch
Elastica extension	https://phabricator.wikimedia.org/diffusion/EELA/	mediawiki-extensions-Elastica
GeoData extension	https://phabricator.wikimedia.org/diffusion/EGDA/	mediawiki-extensions-GeoData
PHP TextCat	https://phabricator.wikimedia.org/diffusion/WTEX/	wikimedia-textcat
Relevance Forge		wikimedia-discovery-relevanceForge
Discernatron		wikimedia-discovery-discernatron
Discovery Analytics	https://phabricator.wikimedia.org/diffusion/WDAN/	wikimedia-discovery-analytics
Lucene Explain Parser	https://phabricator.wikimedia.org/diffusion/WLEP/	wikimedia-lucene-explain-parser