Wikimedia Search Team/Meeting minutes/2015-05-18

Checkins
Nik Stas James Max Yuri Erik Oliver Dan Wes (Didn’t attend, but sent these notes)
 * Did rolling restarts of elastic
 * Worked on query parser (has 50+ tests mimicking our integration tests)
 * Maybe 4 person-days away from being able _start_ replace the hated query_string
 * Trying to figure out when upstream releases Elasticsearch 1.6 - we wants it so badly!
 * Fixing some issues Elasticsearch has filed against features I wrote in Elasticsearch
 * Can now remove a bunch of dynamic groovy which will make ops happy.
 * We’ll need one more rolling restart after we merge and deploy some Cirrus changes to make turning it off really really real.
 * Released 0.0.1 to maven central
 * Need to set up semi-public endpoint for hackathon and others
 * Should we use blazegraph demo or create a front-end
 * Standard front end has a lot of features we intentionally don’t support (e.g. readonly)
 * Needs help. Can’t be alone. Especially around all this Java.
 * Has been working with S Page on API docs (focus on simplicity)
 * Want to create a portal, maybe including snippets, demos, etc.
 * Created quickie front-end for current WDQ front end
 * Could be a good starting point for WDQS front end
 * Meetings, reviews. Later in the week, continue on docs. Will create phab tasks
 * Postgres optimization. Needs to be faster before adding more features.
 * Then will work on missing necessary features, then polishing
 * Gave presentation of how to use new graph feature in wiki
 * Hungarian already uses over 200 graphs
 * Working w/Max on mapping features
 * Creating map presentation for hackathon (will also give graph presentation)
 * Plan to create vector graphics demo
 * Hope to create “wow” demo, maybe something with overlays
 * Wildcards now enabled in cirrus for other wikis, without performance problems
 * Improving multitasking in hhvm. Forking scripts causes some issues.
 * Added tests to some of Nik’s code
 * Will be making search index updates pause-able
 * Nik thinks they might be related to multi datacenter work
 * Will be somewhat unavailable intermittent over the next couple weeks
 * Still working on data visualisation dashboard
 * Still from static snapshot, until things settle out
 * “Find out how API users are using search”
 * Working on non-search project
 * Wrote a C++ tool to geolocate readers/editors - https://github.com/Ironholds/rgeolocate
 * Will continue on API and non-search work, as well as hiring
 * Was out some of last week
 * primarily planning for hackathon, wikidata conversations there and planning offsite
 * Met last week with Google discussing Wikidata Query Service
 * will follow up with Dan regarding that, and planning.

Workboard Walkthrough
https://phabricator.wikimedia.org/tag/search-team/
 * Not necessary yet; will probably become part of the agenda later

Scrum of Scrums

 * Are we blocked?
 * Still need OSM machine (in progress)
 * Analytics hardware is very overburdened, so slowing work
 * Kevin is working with Oliver to turn this into an actionable phab task
 * 3 different event logging schemes causes confusion--should unify
 * Dan has created a phab ticket: https://phabricator.wikimedia.org/T99762
 * Are we blocking?
 * https://phabricator.wikimedia.org/tag/blocked-on-search/

Info/To Discuss

 * Should James learn more about metrics/measurement?
 * Hiring is going slower than hoped (opening req took a while too)
 * Nik thinks we should build internal expertise rather than hiring
 * James is already sort of on that path; happy to do it; overburdened?
 * Oliver may have some sources of potential talent (less engineer-y)
 * Are we hiring a front-end developer?
 * We hope to open a req “soon”
 * Who can look at dynamic script removal?
 * James volunteered
 * Who can help Stas with Java stuff
 * James is interviewing a candidate w/Java experience
 * Proposal to keep feature-specific projects around
 * Yuri had some concerns, but they were resolved in offline discussions w/Kevin
 * Pushback on our team’s wiki page name (“Search and Discovery”)
 * Should it have “Team” at the end? “Department”
 * Is it reasonable that “Search” redirects to our team page?
 * “Editing” is in a similar position
 * Initial WDQS release (0.0.1) Status https://phabricator.wikimedia.org/T96579

FYI

 * Hackathon (May 23-25)
 * https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2015/Search_and_discovery
 * Wikimania (July 15-19)
 * (need to insert relevant team page here)
 * Review time has begun:
 * https://office.wikimedia.org/w/index.php?title=WMF_2014_-_2015_Annual_Review_and_Retrospective