Wikimedia Search Platform

From MediaWiki.org
Jump to navigation Jump to search

The Search Platform team (part of Wikimedia Technology) is responsible for maintaining and enhancing the various Search features and APIs for MediaWiki. This includes the CirrusSearch extension which relies on, the search backend used at the Wikimedia Foundation to support Wikimedia projects.

Current work by this team is tracked on this Phabricator workboard (backlog board is here) and on the public Search Analytics Dashboard that monitors and analyze the impact of our efforts, as well as the External Search Traffic dashboard that very broadly looks at where our requests are coming from.

The Search Platform team was formerly part of the Discovery Department in Audiences; but, as part of the re-organization (tune-up) of June 2017, the Search Platform team is now part of Technology.

Pages of historical note:


Goals[edit]

The Search Platform team's goals for FY 2018/19 can be found here, and they are part of the entire Technology Department's goals (FY2018/19 Q1).

Other projects[edit]

Wikidata Query Service (WDQS)[edit]

Searching structured data on Wikidata is an integral part of Discovery in building the Wikidata query service. It provides a API through which tools can access Wikidata. Learn more about the Wikidata query service. Our current work is tracked on this Phabricator workboard and weekly deployments of WDQS are documented on wikitech:Deployments; while a public WDQS Analytics Dashboard is used to monitor and analyze the impact of our efforts.

APIs[edit]

Application Programming Interfaces (APIs) provide developers ways to interact with the MediaWiki software.

API:Search and discovery lists the search APIs available and in development. View our public API Analytics Dashboard to monitor and analyze the impact of our efforts.


The team[edit]

This list was last updated on December 12th, 2017.


Communications[edit]

Weekly status updates[edit]

See Discovery weekly status updates for the archive of past team updates (Subscribe)

Mailing lists[edit]

Search Platform - A public mailing list about the Wikimedia Search Platform team and projects (formerly Discovery Department). Examples of topics would include:

  • Announcements, including major upcoming initiatives, completed major releases, quarterly or annual plans, requests for feedback or input
  • Technical discussions and brainstorming regarding our work:
    • Search, Elastic, Cirrus, the Relevance Forge, and other relevant subjects
    • Our dashboards or related analysis
  • Other team news, such as changes to team structure, significant changes to processes, changes in how we use phabricator or other tools like gerrit

IRC channels[edit]

#wikimedia-discoveryconnect

Meetup groups[edit]


Process[edit]

The Search Platform team uses a "scrumban" process, which is a hybrid of Scrum and Kanban. It is described here: Search Platform Process.

Conferences, gatherings, and other events[edit]

Upcoming events[edit]

  • October 22 - 25, 2018, Wikimedia Technical Conference (WMTechConf, formerly known as DevSummit) in Portland, Oregon
  • Late January / early February 2019, All-Hands, San Francisco
  • May 2019, Hackathon, Prague

Past events[edit]

Data Analysis[edit]

The data access and analysis guidelines used by the Search Platform team around data sources, or by other teams around Search Platform data sources, are documented on Meta.


Deployers[edit]

Useful reference for who can deploy code. It's nice to know whom to bug if you need something:

Person MediaWiki

Deployer

Elasticsearch

Deployer

Maps

Deployer

Graphoid

Deployer

Portals Deployer
dcausse Yes
ebernhardsen Yes Yes
jan_drewniak Yes
MaxSem Yes Yes
SMalyshev
gehel Yes Yes
^d Yes Yes


Code[edit]

Discovery team supports the following code:

Repository Phabricator/Diffusion Github mirror Active?
CirrusSearch extension https://phabricator.wikimedia.org/diffusion/ECIR/ mediawiki-extensions-CirrusSearch
Elastica extension https://phabricator.wikimedia.org/diffusion/EELA/ mediawiki-extensions-Elastica
GeoData extension https://phabricator.wikimedia.org/diffusion/EGDA/ mediawiki-extensions-GeoData
Wikidata Query Service https://phabricator.wikimedia.org/diffusion/WDQR/ wikidata-query-rdf
Wikidata Query Service GUI https://phabricator.wikimedia.org/diffusion/WDQG/ wikidata-query-gui
WDQS deployment https://phabricator.wikimedia.org/diffusion/WDQD/ wikidata-query-deploy
WDQS GUI deployment wikidata-query-gui-deploy
PHP textcat https://phabricator.wikimedia.org/diffusion/WTEX/ wikimedia-textcat
Relevance Forge wikimedia-discovery-relevanceForge
Discernatron wikimedia-discovery-discernatron
Discovery Analytics https://phabricator.wikimedia.org/diffusion/WDAN/ wikimedia-discovery-analytics
Lucene Explain Parser https://phabricator.wikimedia.org/diffusion/WLEP/ wikimedia-lucene-explain-parser