Wikimedia Discovery/Meetings/Portal retrospective 2016-03-23
Discovery Portal Retrospective
What has happened?
Covering whatever has happened related to the team since the last retro (2016-02-22)
- Lila left; Katherine as interim ED
- Julien transferred to Maps (aka "interactive team")
- Pushed improved search box to production
- Lots of good feedback from internal (WMF) and external users (community)
- Quarterly goals planning
- Lots of good and lively discussion on how and what we wanted to do
- Pushed A/B test for language detection to production on March 22
- Fixing a lot of minor bugs from the earlier search box production release
- Updated portal stats
- Decided on regular schedule to update the stats
- Oliver left
- MIkhail got a title change/promotion: Count Logula the Second
- Oliver successfully trolled the foundation by announcing a title change/promotion that doesn't actually mean anything
Review action items from before
What went well?
- Pushing the code to beta has been automated.
- We had a few issues with keeping it in beta (was overwritten a few times), but it worked wonderfully to test out the new searchbox and the A/B test before they went live
- Using the 'push to beta first' approach allowed us to catch caching issues before they reached production.
- There was lots of community/team feedback & bug reports on the new search-box.
- The phabricator 'discussion' column motivated lots of discussion on various topics.
- Multiple reports analyzing users and sessions:
- JS Support
- 10% Referral Traffic
- Post-deployment Assessment
- Clickthrough rate by preferred language (coming soon!)
- Code reviews seem to be working well even after Julien has moved to maps.
- Chatting with Chris K about portal releases - he's always got the good connections and recommendations
- Chatting with Design Research to plan our upcoming user-facing portal research
- 3x a week meetings are good - they give enough of a catch-up for me (deb)
- Moiz is rocking the new mocks for the A/B tests
- Aha of creating a portal that doesn't lose any functionality
What could we improve?
- Moving forward, devs are working closer with designer to prevent this kind of oversight
- Seeing design WIP early on helps
- This reflects a potential trade-off of testing rough ideas quickly--harder to move to production
- Prefer to do things right the first time rather than writing "temporary" code, which all too often becomes permanent
- Security won't allow quick dirty code even if it's only in tests
- Jan: Moving forward: For prototypes (e.g. demos off user pages), quickie is fine, but for users, it should probably be production quality from the start
- Jan: Beta site currently is master branch, destined for production, so should probably be production quality. Maybe should have more formalized demo site than user pages
- There was some confusion as to what was going to be included in the A/B test, whether it contained the localized slogan or not. Showing more in-progress work would have made that clear.
- Our new process of pushing to beta should help this; having earlier demo pages might help even more
- Make sure tasks in workboard are in the correct column
- A/B test results? (Actual results seemed different from what we expected based on the test)
- Results actually were within report projections
- Possibly report could emphasize the range more emphatically
- WHAT IS GOING ON WITH QUICK SURVEYS... I THOUGHT IT WAS SUPPOSED TO BE QUICK (The ultimate irony.)
- We lack visibility into the current status; we have hit various technical obstacles to implementing
- Mikhail: QS exist, and teams can use it. Our desired use (for search satisfaction) is now blocked on legal.
- Search satisfaction usages was blocked for a while before deciding that it was OK to be slow.
- Needs a (cross-vertical) project manager, and product managers aren't currently filling that role
- https://meta.wikimedia.org/wiki/QuickSurveys / https://www.mediawiki.org/wiki/Extension:QuickSurveys / https://www.mediawiki.org/wiki/Extension:QuickSurveys/Deploying_surveys
What else should be noted?
- Some challenges working through the "groan zone" to decide on Q4 goals (and generally the roadmap)
- Natural for groups of people; different perspectives, opinions, goals
- Gets people involved and invested in the work (nice side effect)
Retro of retro?
This was the first-ever retrospective of the small Portal team (as opposed to the entire Discovery department). It was an experiment. How did it go?
- Moiz: Good. Focused. I knew something about everything that was discussed, which was nice. Didn't make me feel guilty
- Mikhail: Much like with the analysis retro, I liked the focused aspect. Going in depth with team-specific issues, rather than hearing just a little bit about all the things
- Jan: I like it too
- Deb: I liked it too, but was quieter than the whole team
- An hour seems to be the right length
- Maybe have a full-department retro every quarter? Inter-team stuff. (could be on Q boundary)
- Jan: Document "production quality", beta site, demo sites, etc.
- Kevin: Try to unstick quick surveys