Wikimedia Discovery/Meetings/Discovery retrospective 2016-05-16

Proposed topics which didn't end up being added to sections below

• Single opsen supporting multiple teams
• Single CL supporting multiple teams
• CREDIT showcase
• Office hours
• Publishing quarterly "action plan" for search (and portal?)
• Weekly departmental checkins

Issues raised in team retros

• (None that I could find in the last 2 months of retro notes)

Department-wide things that are working well

• Weekly status reports+++++
  • Communicaiton in general. I appreicate folks reaching out and being responsive in communicaiting with our communities. Makes me more effective in helping us all!+
• Few cross-team dependencies (Julien/Jan being the most notable remaining cross-team dependency)
• Unmeetings+++
• Team retrospectives were okay.
• Collaboration with Reading team (Deb and Kevin involved in their offsite) ++

Department-wide things that could be improved

• Documenting/communicating implementation details to analysis team***** (5)

Analysis team (Mikhail) has been quite busy helping out search and portal teams and doing a good job!+ (thanks!)

• will be great when we get a new hire in - allieviate the load of work to be done on one person
• no time for 10% time

Department-wide things that are a mixed bag

• 10% time: sometimes don't have time to do it; sometimes end up exploring what I'm already working on; sometimes don't have enough time to get something interesting done in a day. Sometimes it's awesome.
  • Hard to find/allocate time for something that is not directly related to current work.*** (3)
• One developer on Portal team - doing great, but lots of work to do (possible overburdening) * (1)
• One analyst on the Analysis team - also lots of work to do and doing great (possible overburdening)
• Collaboration with User Research team

Other Department-wide notes, issues, concerns, questions

• Dan will be out for a while
• Kevin has been out a lot this month
• Chris out May 24 -27 attending EMWCon

• General concerns on design, does this block any progress in the next few quarters?
  • Hiring is moving forward (Dan); hoping for Q1 start. "Sooner the better"
  • Not feeling design shortage yet on portal; Moiz left us in a good spot
  • Julien has been doing some design work for interactive. OK for now.
  • Talking to Volker about helping us out (e.g. standardization)
  • But there is a lot of design work for Maps * (1)
    • the sooner we get a designer, the better
    • a designer with experience with maps would be a plus
• Wikimania coming up
• State of the Map US conference
• State of the Map - Sep 23 - 25 in Brussels

Deeper Discussion

Documenting/communicating implementation details to analysis team

• There have been several cases recently where things weren't documented or communicated to analysts, so data was wrong; discovered later.
  • Catalyst was portal click-through rate, where a critical implementation decision wasn't documented. Several months before analysis team became aware.
• We came up with a couple possible solutions, but nothing perfect
  • Getting Mikhail involved in code review of AB tests
  • Getting engineering involved in review of reports before they are finalized
  • Portal team tries to make sure the plan is fully documented in a phab ticket
    • but having Mikhail seeing the actual code before it goes out would be good too
• Julien: We can improve the process, but that specific issue was documented in Nov 15 but got removedout of landscape with time. It will be hard to keep all reports and dashboards in sync with the code if you're relying on phab and written communications. Mikhail should be aware of the critical pieces of code that change, and look at it himself. I don't know of another efficient way.
• Jan and Mikhail already met to start to go through the code; Mikhail will do the same thing with the search team. Then will be able to code review event logging and testing code for both teams.
• David: This problem happened with the search API. I don't think we have a solution because it's very difficult. Would it make sense to try to put more analytics components inside beta cluster, and do fake analysis there? Maybe that would reveal discrepancies earlier. Maybe raise an exception if the results are unexpected.
  • Mikhail: Maybe look into that after we hire another analyst
• Trey: Would it make sense for there to be an "analysis liaison" on each team, who is responsible for making sure analysis is aware of relevant issues? Recently, it was not clear whether Erik or I should let Mikhail know about something.
  • Kevin: Some concerns about single point of failure; accumulate knowledge in one person that should be distributed
  • Tomasz: Maybe that should fall to the tech lead?

10% time: Hard to find/allocate time for something that is not directly related to current work.

• Julien: Mostly have been using the time as extra research time for work I'm already doing. Directly related to work. Can't find time to continue working on some prototypes I started a while ago. It's impossible to spend it outside of core work. The name "research time" is unclear whether it should tie to work or not. There is a lot to do, and there is a lot of research to do for current work.
• Yuri: We used to refer to it not as research time, but "project I think is important so I'll work on it during foundation time." Research seems limiting. (+ from Trey)
• Julien: Experimentation is a word I used. But hard to experiment on other parts of the app.
• Trey: I feel the same way. My time falls into 3 categories: 1) behind, so will skip. 2) interested in something related to work. so will spend a little time. 3) crazy idea to play around with.  One day every 2 (or 4) weeks is hard to get your head back into it.
• Tomasz Wikimedia Discovery/Experimentation Time It's OK if you don't use your experimentation time, or if you use it to fill in on what you're doing.

Action items

• Everyone: Review 10% time policy, and discuss possible changes on its talk page: https://www.mediawiki.org/wiki/Wikimedia_Discovery/Experimentation_Time
• Mikhail: Discuss beta analytics with David (and/or others)
• Mikhail: Consider what it might mean for a team lead to take on "analysis liaison" responsibilities
• Kevin: Schedule next DiscoRetro, and note in the etherpad that tl;dr from team retros are encouraged

Retro of this retro

• Miss the frequency of the old overall retro, but it's worth giving up for the more focused team retros.++
• Would we be able to bring in a few lines from each of the smaller retros to do a tl;dr here?+
  • Kevin: possibly delegate this to the individual teams
• Good opportunity for department-wide discussion of department-wide issues (e.g. the documentation thing)+
• Can be shorter, but should not be removed. Important to have a department-wide retro meeting +++
  • Not shorter!+
  • an hour seems good