Wikimedia Engineering/WMF Tech Days 2013/Product and Data Workshop

From mediawiki.org
Jump to navigation Jump to search

Product & Data workshop[edit]

Notes: http://etherpad.wikimedia.org/p/data-and-product-workshop

see the brainstorming page
Leads
Dario Taraborelli, Aaron Halfaker, Toby Negrin
Schedule
1:30 pm - 60 minutes - 6th floor collab space
Audience
  • PMs and analysts: how do we incorporate data into product decisions?
  • Engineers who want to collect data on new features, changes to core etc.
Outcomes
  • how we incorporate data into product decisions
  • how to work with analysts to evaluate products
  • review features currently in design/under development and the most appropriate evaluation approach
Things to discuss
  • Evaluating new features/products
    • Product lifecycle
    • Using data to inform product design vs product optimization
    • Types of request: counting things vs testing hypotheses
  • Controlled tests (hypothesis testing)
    • When is it appropriate to run a controlled test?
    • A/B testing: low hanging fruits
    • A/B testing: state of the art. What can we do with the existing infrastructure?
    • What are the costs associated with a controlled test?
  • Descriptive usage data (counting things)
    • I want to use EventLogging to collect usage data, where do I start? How do I set up a dashboard?
    • Ad-hoc analysis: e.g. feature adoption by user segment
    • Cohort analysis: Wikimetrics
  • Other product evaluation methods
    • Usability testing
    • Qualitative analysis