Wikimedia Discovery/Meetings/Portal retrospective 2016-03-23

Discovery Portal Retrospective

2016-03-23

What has happened?
Covering whatever has happened related to the team since the last retro (2016-02-22)
 * Lila left; Katherine as interim ED
 * Julien transferred to Maps (aka "interactive team")
 * Pushed improved search box to production
 * Lots of good feedback from internal (WMF) and external users (community)
 * Quarterly goals planning
 * Lots of good and lively discussion on how and what we wanted to do
 * Pushed A/B test for language detection to production on March 22
 * Fixing a lot of minor bugs from the earlier search box production release
 * Updated portal stats
 * Decided on regular schedule to update the stats
 * Oliver left
 * MIkhail got a title change/promotion: Count Logula the Second
 * Oliver successfully trolled the foundation by announcing a title change/promotion that doesn't actually mean anything

Review action items from before

 * n/a

What went well?

 * Pushing the code to beta has been automated.
 * We had a few issues with keeping it in beta (was overwritten a few times), but it worked wonderfully to test out the new searchbox and the A/B test before they went live
 * Using the 'push to beta first' approach allowed us to catch caching issues before they reached production.
 * There was lots of community/team feedback & bug reports on the new search-box.
 * The phabricator 'discussion' column motivated lots of discussion on various topics.
 * Multiple reports analyzing users and sessions:
 * JS Support
 * 10% Referral Traffic
 * Post-deployment Assessment
 * Clickthrough rate by preferred language (coming soon!)
 * Code reviews seem to be working well even after Julien has moved to maps.
 * Chatting with Chris K about portal releases - he's always got the good connections and recommendations
 * Chatting with Design Research to plan our upcoming user-facing portal research
 * 3x a week meetings are good - they give enough of a catch-up for me (deb)
 * Moiz is rocking the new mocks for the A/B tests
 * Aha of creating a portal that doesn't lose any functionality

What could we improve?

 * Pushing the new search-box to production took considerably longer then expected. The design didn't take into account the no-Javascript/mobile use-case and thus the code had to be re-implemented for production.
 * Moving forward, devs are working closer with designer to prevent this kind of oversight
 * Seeing design WIP early on helps
 * This reflects a potential trade-off of testing rough ideas quickly--harder to move to production
 * Prefer to do things right the first time rather than writing "temporary" code, which all too often becomes permanent
 * Security won't allow quick dirty code even if it's only in tests
 * Jan: Moving forward: For prototypes (e.g. demos off user pages), quickie is fine, but for users, it should probably be production quality from the start
 * Jan: Beta site currently is master branch, destined for production, so should probably be production quality. Maybe should have more formalized demo site than user pages
 * There was some confusion as to what was going to be included in the A/B test, whether it contained the localized slogan or not. Showing more in-progress work would have made that clear.
 * Our new process of pushing to beta should help this; having earlier demo pages might help even more
 * Make sure tasks in workboard are in the correct column
 * A/B test results? (Actual results seemed different from what we expected based on the test)
 * Results actually were within report projections
 * Possibly report could emphasize the range more emphatically
 * WHAT IS GOING ON WITH QUICK SURVEYS... I THOUGHT IT WAS SUPPOSED TO BE QUICK (The ultimate irony.)
 * We lack visibility into the current status; we have hit various technical obstacles to implementing
 * Mikhail: QS exist, and teams can use it. Our desired use (for search satisfaction) is now blocked on legal.
 * Search satisfaction usages was blocked for a while before deciding that it was OK to be slow.
 * Needs a (cross-vertical) project manager, and product managers aren't currently filling that role
 * https://meta.wikimedia.org/wiki/QuickSurveys / https://www.mediawiki.org/wiki/Extension:QuickSurveys / https://www.mediawiki.org/wiki/Extension:QuickSurveys/Deploying_surveys

What else should be noted?

 * Some challenges working through the "groan zone" to decide on Q4 goals (and generally the roadmap)
 * Natural for groups of people; different perspectives, opinions, goals
 * Gets people involved and invested in the work (nice side effect)

Retro of retro?
This was the first-ever retrospective of the small Portal team (as opposed to the entire Discovery department). It was an experiment. How did it go?
 * Moiz: Good. Focused. I knew something about everything that was discussed, which was nice. Didn't make me feel guilty
 * Mikhail: Much like with the analysis retro, I liked the focused aspect. Going in depth with team-specific issues, rather than hearing just a little bit about all the things
 * Jan: I like it too
 * Deb: I liked it too, but was quieter than the whole team
 * An hour seems to be the right length
 * Maybe have a full-department retro every quarter? Inter-team stuff. (could be on Q boundary)

Action Items

 * Jan: Document "production quality", beta site, demo sites, etc.
 * Kevin: Try to unstick quick surveys