Discovery/Status updates/2017-06-12

This is the weekly update for the week starting 2017-06-12

==Highlights== Highlights
 * A recent update to the search results page on all wikis—sister project snippets—was deployed into production on June 15; see this email for more info.
 * Added a note to the Extension:Kartographer page about mapframe deployments
 * Sent out a communication about what the Discovery team's goals and future work will be.

==Discussions== Discussions

Search

 * Logstash scripts are now using curator, and some standard action files (enabling / disabling shard allocation) have been deployed
 * Deployed new versions of Wikimedia and other ElasticSearch plugins (epic task with lots of smaller subtasks
 * Various updates to getting the search clusters up to ElasticSearch 5.3.2
 * Fixed an issue where the sister project snippets were causing an weird display problem
 * We've updated Ukrainian-language wikis with a new Ukrainian language analyzer, which should provide better search results by recognizing related forms of a word. (An example in English would be that searching for "hope", "hoped", "hopes", or "hoping" can all find each other.) See T160106 and related Phabricator tickets.
 * We've updated Chinese-language wikis using a new Chinese language analyzer, which should provide better search results by doing a better job of breaking up Chinese text into words, and by automatically converting between Simplified and Traditional characters when searching. See T158203 and related Phabricator tickets.
 * We've updated Swedish-language wikis with a smarter configuration that recognizes å, ä, and ö as distinct letters (and not just variants of a and o). See Phabricator ticket T160562.
 * Setup testing, training and validation splits for learning to rank machine learning
 * Worked on calculating the NDCG of click data that feeds the machine learning rank pipeline

Wikidata Query Service

 * Enabled the Mediawiki Service API which allows interacting with Mediawiki API from SPARQL.
 * Added more federation endpoints.

Analysis

 * Finalized the migration from Vagrant to Puppet configuration for the dashboards
 * Investigated a drop in pageviews and clickthroughs on the Wikipedia.org portal - turns out summer is here
 * Fixed a minor issue with the desktop and mobile web graphs on the external search dashboard

Interactive

 * Achieved some clarity to the phabricator board with priorities and what is in progress, needs to be in the backlog or stalled.

--
 * View all open tickets related to Discovery.
 * Looking to get involved? See tasks marked as Easy or volunteer needed