Wikimedia Engineering/WMF Tech Days 2013/Product and Data Workshop

Product & Data workshop
Notes: http://etherpad.wikimedia.org/p/data-and-product-workshop


 * see the brainstorming page


 * Leads
 * Dario Taraborelli, Aaron Halfaker, Toby Negrin


 * Schedule
 * 1:30 pm - 60 minutes - 6th floor collab space


 * Audience
 * PMs and analysts: how do we incorporate data into product decisions?
 * Engineers who want to collect data on new features, changes to core etc.


 * Outcomes
 * how we incorporate data into product decisions
 * how to work with analysts to evaluate products
 * review features currently in design/under development and the most appropriate evaluation approach


 * Things to discuss
 * Evaluating new features/products
 * Product lifecycle
 * Using data to inform product design vs product optimization
 * Types of request: counting things vs testing hypotheses
 * Controlled tests (hypothesis testing)
 * When is it appropriate to run a controlled test?
 * A/B testing: low hanging fruits
 * A/B testing: state of the art. What can we do with the existing infrastructure?
 * What are the costs associated with a controlled test?
 * Descriptive usage data (counting things)
 * I want to use EventLogging to collect usage data, where do I start? How do I set up a dashboard?
 * Ad-hoc analysis: e.g. feature adoption by user segment
 * Cohort analysis: Wikimetrics
 * Other product evaluation methods
 * Usability testing
 * Qualitative analysis