Wikimedia Discovery/Meetings/Search retrospective 2016-06-01

Covering whatever has happened related to the team since the last retro (2016-04-18)
 * Team offsite
 * Tomasz leaving; Katie joining
 * Prep for Dan's vacation
 * TextCat A/B test
 * ElasticSearch upgrade
 * Discernatron
 * Search off-site
 * Data mapping (did this happen between the last retro and this one?) -> it seems to be in the "What went well", so I guess yes... but I was the one that put it in there too :-p

Review action items from before

 * Erik: Post action plans with rough timelines for finishing Q4 goals
 * Team has shared understanding; nothing was posted. We won't carry this action item forward.
 * Erik: Advance the conversation about improving browser tests
 * Had various conversations, so ✅. It had been running well until yesterday (machine config issue unrelated to earlier problems).
 * Chris: See if there are any past tests that weren't announced that should have been
 * Didn't find any. Calling this one ✅.
 * Chris: Update Discovery page on Mw.org to reflect q4 goals - ✅
 * Chris: Request feedback on search team draft page - ✅

What went well?

 * Tomasz has been gone two days and nothing exploded (yet) (well, it was a holiday weekend ;) )
 * The search offsite was productive and fun, and it was great to see everyone who could come +1
 * Elasticsearch upgrade is so far relatively painless
 * Guillaume is getting better at doing full elasticsearch cluster restarts
 * Implementing feature requests for Reading is going well *
 * Data mapping went well and was straightforward for us to do+1*

What could we improve?

 * Monitor github activity on our plugins: e.g. https://github.com/wikimedia/search-highlighter/issues/18 **
 * We can't properly test major production changes to search infrastructure because we can't duplicate LVS ***
 * Didn't notice breakage of elasticsearch logging in beta, noticed after initial production rollout
 * Not sure that I (Guillaume) give enough visibility on what I'm doing, since I'm split between different teams. Not always easy to jump from one project to the other.
 * Didn't notice that live updating for completion suggester was removed from Elasticsearch 2
 * Haven't done much analysis beyond a brief high level on the swap2and3 test *
 * Some issues with the data collected for the textcat test. We were able to work through it, but could use better processes to validate before shipping+1

What else should be noted?

 * Many issues with WDQS, we did a postmortem (Stas & Guillaume - Thanks for taking the time!). We have a few actions items on that
 * Discernatron functionality is good; "usability" is less good (soooo many results)—overall holds much promise ***
 * As we look toward Q1 how and where can the community liaison help? - selfish prod to think of me. :) ****

Discussion:

 * As we look toward Q1 how and where can the community liaison help? - selfish prod to think of me. :) ****
 * Draft goals: https://www.mediawiki.org/wiki/Wikimedia_Engineering/2016-17_Q1_Goals#Discovery
 * DG: For some internal tech issues like changing elastic settings, an announcement should be enough. For something like interwiki search, more communication makes sense. We can sync up offline about that.
 * EB: For the new search, it would be backward compatible so users wouldn't see a change at rollout, but it would open up new features in the future.
 * GL: Do you think you get enough information from tech folks like me?
 * CK: I get a lot of info from the weekly status reports. I'll send out a link to a video of my presentation of "what we do as CL's" (and don't)
 * (I'll send this out to the internal discovery list but if you're here now): https://www.mediawiki.org/wiki/EMWCon_Spring_2016/Technical_Collaboration_with_the_Wikimedia_Foundation
 * Discernatron functionality is good; "usability" is less good (soooo many results)—overall holds much promise ***
 * DG: Combination of being busy and lack of design support?
 * EB: I have been busy.
 * DG: Maybe host a lunch in SF where people can eat if they help with Discernatron?
 * We can't properly test major production changes to search infrastructure because we can't duplicate LVS ***
 * EB: We have discussed outside this meeting; things are moving; just an annoyance
 * GL: Not a major problem. It's not too bad testing in production.

Action Items

 * Chris: post the link to his "what technical collaboration team does" presentation
 * Chris: Chris and Erik should talk about implications of interwiki search indices
 * Trey & Deb: Chris needs to be aware of the ? at the end of queries
 * Erik: Figure out a plan to reliably monitor github (David and Guillaume have started to watch it)

Retro of retro:

 * needs to be longer than 1/2 an hour *****