Our departmental goals are captured on a quarterly basis for each fiscal year (Q1: July-September; Q2: October-December; Q3: January-March; Q4: April-June). This is an overview of our goals by quarter for each team. For our goals in the Wikimedia Foundation's annual plan, see this phabricator board.
Outcome 2: The adoption of Wikimedia technology can be reliably measured
Objective 1: Design a set of formal KPIs (key performance indicators) to measure the growth and diversity of our technology audience (task T171220)
Outcome 4: The collaboration with research in industry and academics is further scaled and supported.
Objective 1: Organize and host the annual Wiki Research Workshop. (task T171118)
Conduct structured interviews with team leads in Tech involved in building platforms with an externally-facing audience and get them to articulate who their core audience is, and how to characterize adoption of their platforms. (task T174733)
Write and submit a workshop proposal for the Web Conference (task T174741)
Write a call for papers for Wiki Workshop 2018 (task T174743)
Invite reviewers to review workshop submissions (task T174744)
Tech Management
Admin
Workshop co-organizers
Jonathan
Dario
Leila
EOQ
In progress
Program 9: Growing Wikipedia Across Languages.
Outcome 1: Surface relevant information about the articles to editors at the time of editing with the goal of helping editathon organizers
Objective 1: Build, improve, and expand algorithms that can provide more detailed recommendations to editors about how an article could be expanded. (task T171224)
Objective 2: Develop and gather design requirements for how the algorithms’ results should be exposed to the editors. (task T171225)
Build a prediction model to predict section ranks within category (task T174737)
Build a tool for article expansion recommendations and run a test for editors and editathon organizers (task T174739)
Submit a paper on article expansion recommendations, write a blog post, assess if any data release is helpful (task T174736)
Cross Departmental: Structured Data on Commons. (Segment 4: Programs)
Outcome 2: Develop a better understanding of existing needs for Structured Commons (task T152248)
Conduct survey of GLAM Commons contributors' workflows, wants and needs (task T175188)
Interview 6-10 volunteer contributors to Wikimedia Commons to understand current workflows and impact of structured data (task T175185)
Structured Data program management
Jonathan
EOQ
In progress
Cross Departmental: Community Health. (Segment 3: Research)
Outcome 1: We aim to understand and model the characteristics of harassment in Wikimedia projects.
Objective 1: Conduct research to characterize and model wikihounding. (task T171249)
Objective 2: Prototype new models to facilitate sockpuppet detection and the classification of toxic discussions. (task T171251)
Analyze editor-editor interactions for instances of wikihounding (task T175664)
Publish an annotated dataset of conversation structures on English Wikipedia talk pages and submit a paper on early indicators of toxic conversations (task T175666)
Prototype new models to facilitate sockpuppet detection (task T171635)
Outcome 1: We improve Wikipedia’s contributor diversity after designing and testing potential intervention(s).
Objective 1: Identify the underlying (potential) causes of lack of representative contribution from certain demographics task T166083
Objective 2: Design frameworks to change the current socio-technical infrastructure to address at least one of the underlying causes of lack of representativeness
Create one or more formal collaborations for this research (T166085)
Perform a literature review and potentially run survey(s) to identify the self-reported causes of imbalanced representation in contribution (T175215)
Initiate the design of a framework to address one of the identified/hypothesized causes of imbalance in contributor demographics, if time permits. (Stretch)
Run a couple of quick surveys to get a better sense of where in the pipeline we start loosing diversity. (Stretch)
External collaborators (Bob and Jerome)
Leila
EOQ
Done
The second stretch goal is under review and may move to the next quarter depending on its engineering needs. This latter goal is a nice-to-have that we can afford to drop if needed.
Program 9: Growing Wikipedia across languages via recommendations.
Outcome 1: Surface relevant information about the articles to editors at the time of editing with the goal of helping editathon organizers
Objective 1: Build, improve, and expand algorithms the can provide more detailed recommendations to editors and editathon organizers on how to expand articles/family-of-articles
Clean up the category system for machine consumption (this has been the focus for the past few months and we are very close to have a solution)
Start surfacing recommendations to collect feedback from editathon organizers (T174738)
External collaborators (Bob, Michele, Tiziano)
Leila
Done
Cross Departmental: Structured Data on Commons. Segment 4: Programs.
Outcome 2: Develop a better understanding of existing needs for Structured Commons (task T152248)
Develop interview protocol
interview 6-8 GLAM stakeholders
Share initial user stories and research themes at Structured Data offsite
Collaborate with New Readers core team on concept evaluation (against data from the field and with users we are building for) and iteration, focusing in on 1-3 concepts each for 2 of the 24 findings from research. T144016
Reading
Global Reach
Communications
Community Engagement
end of Q2
Done
FOCUS
New editor support
Provide concept development support for Edit Review Improvements project T137987
Perform and publish analysis of the quality of new editor support provided at Wikipedia Teahouse T132809
Perform and publish expanded analysis of the impact of Teahouse on new editor retention T127523
Research & Data
Editing (Collaboration)
end of Q2
In progress
STRENGTHEN
Design Research policy updates
Implement data audit process to ensure compliance with WMF data retention guidelines and privacy policy T143234
Draft general on-wiki privacy statement for design research user studies (generative and evaluative) T144641
Deliver design requirements, heuristic evaluations, and user study findings for key products across Reading, Editing, and Discovery.
Product teams: Mobile Apps, Desktop & Web, VE, Collaboration, Discovery
end of Q1
Done
FOCUS
Generative Research
Analyze and share results of Mexico, Nigeria, and India contextual inquiries T132799
New Readers project concept development and evaluation and collaboration on product iteration (in collaboration with Reading, Global Reach, Communications) based on contextual inquiries in South Africa, Mexico, Nigeria and India. Explore and evaluate 1-3 concepts coming out of the contextual inquiries with the goal of deciding on 1 concept to focus on by the end of the quarter. This work will involve researching what has been done in both wiki projects and in the market in general around the product area we decide to focus on as well as the possibility of new concepts and/or products. T129201T132800T132801
Reading, Global Reach, Communications (teams) and ReBoot (external contractor)
mid Q1
Done
STRENGTHEN
Research capacity building
Implement user testing tools to support moderated and unmoderated user testing on desktop and mobile (apps and web)
Experiment with recruiting research participants through Wikimedia/Wikipedia social media channels
funding for testing tool (annual subscription or a la carte)
Communications team
end of Q1
Done
EXPERIMENT
Research data mapping (stretch)
Document the sources of user data we manage, how we store it, and what potentially personally identifiable information it contains.
Ensure all archived user study data is retained and shared in compliance with WMF data retention policy and privacy policy.
Draft data access guidelines to inform our future data collection and dissemination practices.
Support the use of personas in product development.
Analysis on the collected Reader, New Editor and deep dive (New Readers) personas.
EOQ
not completed
Maintain evaluative design research projects with Editing, Reading, Search and Community Engagement teams
Continue to mature collaboration between design research and product teams (make our system more lean)
Continue to mentor and coach teams that do not have design support, to do design research (PC&L particularly this quarter)
Continue to provide evaluative research to official product teams (Reading, Editing, Search)
Functional collaboration and good planning with product (Reading, Editing and Search) teams.
TPG support to collaborate toward leaner and more effective collaboration with product teams.
EOQ
In progress
Deep dive / contextual inquiry in Nigeria and India
Forward our deep dive / contextual inquiry process and method to be a repeatable practice implementable by WMF.
Bring data and stories about New Readers in Nigeria and India back to WMF and the movement.
Start to look across deep dives (South Africa, Mexico and Nigeria and India) for patterns and for differences between various regions visited.
Collaboration with Communications, Partnerships and Reading team and Reboot (external consultant).
EOQ
Done
Continue, Complete Collaboration with UW on Survey
Deploy survey to 200 or more students at the University of Washington
Analyze and publish results on Wiki
External collaborators
EOQ
done
Create a project plan and understand dependencies and resource needs for benchmarking.
Pick up where we left off, using the work completed before to create a stable testing environment.
Make a plan to create a stable testing environment usable for iterative unmoderated, remote testing as well as benchmarking (mobile and desktop) and mobile field research.
Prepare to run pilots with UserZoom and Loop11 (and any other possible tools).
funding for tooling
tooling working on mediawiki software (necessitates a pilot)
Mentor people to do quality design research (those not on DR team).
One person outside the design research team does a complete design research project to DR team standards, expanding the number of people who do design research well.
Person outside the DR team having capacity to do a design research project.
EOQ
Done
STRENGTHEN
Design Research workshop to build collaboration capacity with product teams.
Workshop happens, and product teams learn more about collaboration with Design Research and vice versa.
Availability of workshop participants to attend (track leads, managers and individual contributors from each product track).
MidQ
Done
STRENGTHEN
Improve participant recruitment process
Implement changes to reduce steps in recruitment (of design research participants) process to make recruiting more efficient.
n/a
EOQ
Done
FOCUS
Collaborate with product teams to implement design research projects.
Design Research has done design research projects in collaboration with product teams to inform human centered product development.
All product teams we collaborate with
Research participants show up to research sessions (volunteers)
EOQ
Done
EXPERIMENT
Persona interviews
12 persona interviews completed
Design research participants show up to research sessions (volunteers)
Our focus in Q1 will be to coordinate and implement both evaluative and generative design research for the prioritized product track projects as well as iterating our pragmatic personas toward being fully researched Personas. Time permitting, we will mentor and collaborate with designers and prototypers toward doing usability testing for thier track teams, and we will grow our participant database.
Support prioritized user facing product projects with track teams by doing generative research (exploring and better understanding user needs, context and pain points) and evaluative research (evaluating concepts and functionality with specific users for usability and how they support needs.)
Research reports with recommendations and information useful for concept development and product iteration considering specific users.
Collaboration on timing, expectations, priorities and the research with product track teams.
end of Q1
Done
Iterate the set of pragmatic personas developed in Q4 FY15 toward fully researched personas.
12 persona interviews moving our pragmatic personas to more closely reflect the humans we are building for.
Community liaisons and others to connect us with the right people to interview
Participants being able to get on technology so we can interview them, as there is no budget to travel and meet them.
end of Q1
Partially done
Teach and mentor more designers and protoypers to do evaluative research (usability testing) to support user friendly, intuitive product development within the track teams. (Stretch)
Two designers or prototypers having done 2 or more usability tests for their teams with support from the design research team.
Collaboration and willingness from designers and prototypers on track teams.
Done
Grow our participant database. (Stretch)
Gain 1000 more volunteer participants (in our participant "database") to collaborate with in design research projects.
Fundraising and other teams to coordinate banners, and other outreach efforts (mass message, etc) with.