Wikimedia Discovery/Meetings/Checkin/2017-10-10

= Topics from the past =
 * Wikimedia Developer Summit 2018 (Jan 22-23): Call for Position Statements, were due Oct 1
 * https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit

= Announcements, Information, Questions =
 * Quarterly Review for Search Platform (Technology) scheduled for 10/16
 * Quarterly Review for Audiences:Readers scheduled for 10/23
 * Deployment schedule (train and SWAT) for upcoming (end of year) holiday period:
 * No MW train the week of Thanksgiving (Nov 20th), SWATs open for high priority things
 * No deploys weeks of Dec 18th and 25th (last two weeks)
 * Normal week week of Jan 1st (minus no deploys that Monday)
 * The Dev Summit and WMF All Hands is the week of January 22nd, so that will be a "No Train but SWATs OK on Mon/Tues/Wed" week.
 * The following week (week of January 29th) the Release Engineering team will be on an offsite, so a week of "No Train, but SWATs and service deploys OK".
 * More information: https://lists.wikimedia.org/pipermail/engineering/2017-September/000475.html
 * Spark projects due on Oct 20
 * https://office.wikimedia.org/wiki/Spark_Project
 * Search Platform retrospective scheduled for Thursday (Oct. 12), Search FE folks are welcome to join
 * Search Platform offsite being scheduled for the weekend after all-hands
 * Audience:Readers offsite will be on Mon/Tues during the week of all-hands
 * Be sure to fill out travel request form for January 2018 All Hands week in SF office
 * Mapframe conversations:
 * Spanish and English Wikipedia enabling mapframe conversations wrapped up:
 * Spanish: https://es.wikipedia.org/wiki/Wikipedia:Caf%C3%A9/Archivo/T%C3%A9cnica/Actual#Mapas_din.C3.A1micos
 * English: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Should_we_ask_for_mapframe_to_be_turned_on.3F
 * Wikivoyage mapframe—possible vandalism—conversation:
 * https://en.wikivoyage.org/wiki/Wikivoyage:Travellers%27_pub#.7B.7BMapframe.7D.7D_vandalism
 * https://en.wikivoyage.org/wiki/Wikivoyage:Map
 * Had a conversation with Brave (Erika)

= Scrum of Scrums = Are we blocked? Are we blocking? Other dependencies (in either direction) which don’t need to be called out as “blocked” (e.g. are progressing smoothly, have no urgency, etc.)
 * None
 * None
 * None

= Discovery News =
 * See https://www.mediawiki.org/wiki/Discovery/Status_updates

= Quick Quarterly Goals/KPI Update (if needed) = Discovery Roadmap FY 2017/18: https://docs.google.com/a/wikimedia.org/presentation/d/1N41eNrz0vFHJamLkhQjSFCOSiDg-bKr5H3ROchqwVJU/edit?usp=sharing

FY 2017-18 Q2 (Oct-Dec) goals: https://www.mediawiki.org/wiki/Wikimedia_Discovery#Projects

This status was last updated 2017-10-10. Completed/dropped goals may not be shown.

Tech:Search Platform
https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2017-18_Q2#Program_1:_Make_knowledge_more_easily_discoverable

Search:

1. Implement advanced methodologies such as “learning to rank” machine learning techniques and signals to improve search result relevance across language Wikipedias. 2. Improve support for multiple languages by researching and deploying new language analyzers as they make sense to individual language wikis. 3. Investigate how to expand and scale Wikidata Query Service to improve its ability to power features on-wiki for readers 4. Address technical debt:
 * Begin to automate the machine learning pipeline, starting by targeting eight to ten languages, other than English, that match (at a minimum) current performance and then deploy those models. (IN PROGRESS)
 * Investigate open source language software that is available and see if it can be converted into ElasticSearch plugins. (IN PROGRESS)
 * Investigate usage of fall-back languages and fuzzy (phonetic) matching. (IN PROGRESS)
 * Continue general language support. (IN PROGRESS)
 * Work on sub-category filtering and searching within the Wikidata Query Service. (IN PROGRESS)
 * Convert existing Selenium tests to Node.js (IN PROGRESS)
 * Investigate ownership and maintenance of Logstash (NOT STARTED YET)

Structured Data on Commons:

1. Commons search will be extended via CirrusSearch and ElasticSearch and Wikidata Query Service, to support searching based on structured data elements describing media. 2. Advanced search capabilities (e.g., Wikidata Query Service, SPARQL queries) will be updated to support the more specific media search filters and the relationships to the topics they represent
 * Determine advanced search requirements and measures for structured data on commons. (NOT STARTED YET)
 * Begin work on prefix- and full-text search in ElasticSearch on Wikidata in preparation for the Structured Data on Commons project. (IN PROGRESS)

WDQS: Wikidata Query Service goal for this quarter will be to work on sub-category filtering and searching within the Wikidata Query Service; it will be maintained by Stas and Guillaume to support the continued growth and use of the service; the Analysis team will help with statistics.

Audiences:Readers:Discovery
https://www.mediawiki.org/wiki/Wikimedia_Audiences/2017-18_Q2_Goals#Readers

Portal: Update the Wikipedia.org portal codebase to be completely automated for ease of ongoing maintenance.
 * Automate portal project updates: statistics and translations (IN PROGRESS)

Maps: Support the move to be more operationally centralized and roll out a new map style that has numerous updates and enhancements.
 * Finalize and deploy new map style; replicate maps test cluster in Wikimedia Cloud Service; monitor for critical bugs (IN PROGRESS)

Analysis: The team will continue to work closely with the Search Platform team to analyze A/B tests and other assorted data; they will also begin working on determining a baseline set of metrics for Structured Data on Commons. (IN PROGRESS)

= FYI =
 * Oct 12: 11th DBpedia Community Meeting, Stas & Dario attending
 * Nov 11: US Holiday
 * Nov 23-24: US Holiday
 * Dec 25-29: WMF Holiday week
 * OΟО
 * Erika out 10/13 for Transformative Technology Conference (strange AND loopy)
 * Paul off October 19 - 22 + travel for State of the Map US in Boulder, CO
 * Deb off October 20 for State of the Map US in Boulder, CO
 * Chelsy off October 20 for vacation (instead of Columbus on Oct 9)
 * Chris at Readers Apps off-site Oct 16-20 in Philadelphia
 * Chris at Readers Web off-site Oct 23-27 in Queens, NY
 * Mikhail at Readers Apps off-site Oct 16-20 in Philadelphia
 * Stas at WikidataCon 28-29 Oct in Berlin (out on 26th, back on Oct 31)
 * Guillaume on vacation Nov 20-24