Wikimedia Technology/Annual Plans/ERF OKR: Impact Culture: Business Intelligence

Impact Culture: Business Intelligence

FY21/22 Organization Efficacy & Resilience OKR for Wikimedia Technology Department

Accountable: Sumeet Bodington

OKR Overview

Foundation leadership has access to organizational data, insights and dashboards for impactful decision making.

Key Result 1
Each quarter, 50% of departments work with the GDI team on the theory of change; develop org-level business metrics, including Social Return on Investment (SROI) measures by Q2

Key Result 2
By end of Jan 2022, design and roll out a data dashboard that can be used by at least 50% of Foundation departments for annual planning.

Objective Rationale

The Global Data & Insights Team gathers, analyzes, and shares data to inform practices across the organization. We assess the effectiveness, efficiency, and impact of Foundation programs and operations, the movement, and the world so we can continue to be an effective change agent in the movement for free knowledge.

To carry out that work, we need organization-wide frameworks and tools for a shared understanding of how Foundation activities are tied to outcomes and our strategic impact. By investing in developing these frameworks and tools, Foundation leadership can make more informed decisions for our current and future strategic priorities.

Definitions

• All Department Leads – TBD
• Annual Plan (AP) – The Wikimedia Foundation Annual Plan is the plan of record for the Wikimedia Foundation's work each year. These are scheduled for the Wikimedia Foundation's fiscal year, which starts at July 1 and ends at June 30 of the following year.
• Annual Planning Delegates - Foundation staff identified by their respective departments to act as delegates for the department in coordinating the implementation of the planning process.
• Annual Planning Delegates - Foundation staff identified by their respective departments to act as delegates for the department in coordinating the implementation of the planning process.
• Business Metric - A quantifiable measure that is used to track, monitor, and assess the status of a specific business process.
• Data Pipeline - For the purposes of this KR, a “data pipeline” does not necessarily refer to fully automated data pipeline, but rather the tooling and processes (manual or automated) that the GDI team will use to compile and manage metrics across disparate data collection systems.
• Evaluability - The extent to which an activity or project can be evaluated in a reliable and credible fashion. Is the evaluation feasible given our resources?
• Medium Term Plan (MTP) – A plan mapping out the major institutional and technical goals under the responsibility and mission of the Wikimedia Foundation. This plan is intended to support the vision laid out by Wikimedia 2030 and reflects the Foundation’s interpretation of the activities it can undertake to support the broader movement in becoming the essential infrastructure of free knowledge.
• Process Improvement Team - TBD
• Social Return on Investment – Social return on investment (SROI) is a principles-based method for measuring extra-financial value (such as environmental or social value not currently reflected or involved in conventional financial accounts). It can be used by any entity to evaluate impact on stakeholders, identify ways to improve performance, and enhance the performance of investments.
• Theory of Change – Theory of Change (ToC) is a methodology for planning, participation, and evaluation that is used in companies, philanthropy, not-for-profit and government sectors to promote social change. Theory of Change defines long-term goals and then maps backward to identify necessary preconditions.
• Quarterly Tuning Sessions - The Tuning Session is the Foundation’s quarterly process for assessing progress towards our MTP commitments and ongoing support for core work. The process is scheduled to commence as close as possible to the end of the previous quarter. The Foundation intends to use this process to communicate successes and misses, share what was learned from the previous quarter, and foster collaboration/alignment between teams.
• Value Chain - A value chain is a set of activities that an organization performs in order to deliver a valuable product or service for the market.

Key Result 1: Theory of Change (ToC) & Social Return on Investment (SROI)

Each quarter, 50% of departments work with the GDI team on the theory of change; develop org-level business metrics, including Social Return on Investment (SROI) measures by Q2

The GDI team will work with the Annual Planning team on theory of change; develop org-level business metrics, including Social Return on Investment (SROI) measures.

Intent and Desired Outcomes
This Key Result involves two relevant projects in FY21/22 within the ERF domain: Org-Level Theory of Change (ToC) and Social Return on Investment (SROI).

As the Foundation prepares to enter another phase of annual and medium-term planning in 2022, the Global Data & Insights team will facilitate the creation of an organization-wide Theory of Change framework to alignwith the Foundation’s overall strategic objectives. The Theory of Change will lay out the logical connection between the Foundation's programmatic investments, activities, outputs, and short- to medium-term outcomes, and will serve as a learning tool for the Foundation to deepen its long-term impact in future years.

In Q2 of FY 2021-22, the GDI team will organize workshops for the Annual Planning Delegates onthe Theory of Change methodology and how it can be leveraged to connect our planning activities to our strategic objectives. This will lay the groundwork for the Annual Planning team to leverage the Theory of Change framework during Annual Planning offsites in Q3 of FY 2021-22. During this time, the GDI team will also kick off conversations and workshops with the Board, C-Team and Foundation staff to co-create a multi-year Theory of Change. Both the development of the AP ToC and our multi-year strategy are intended to be collaborative and generative processes, and will require explicit buy-in from executive leadership, the Board, and the movement to be successful.

In FY 2021-22, the GDI will also work with specialists with expertise in Social Return on Investment (SROI) measurement and value chain analysis to craft a set of metrics specific to the Foundation’s long-term social impact objectives. SROI and value chain are frameworks used to help nonprofits quantify the impact of their programmatic activities. Our hypothesis is that by adopting a standardized set of metrics that can be socialized throughout the Foundation and the community, we’ll be able to encourage focus, clarity and intention during strategic planning and hold ourselves accountable to the outcomes and impact we’re ultimately trying to achieve through the work we prioritize. We will pilot this work in Q3 of FY 2021-22 alongside the Theory of Change mapping initiatives.

Related Quarterly OKRs

• Q1: Secure resources to facilitate Theory of Change training and workshops and the Social Return on Investment pilot project.
• Q2: Facilitate Theory of Change workshops for all Annual Planning Delegates and Global Data & Insights with XX% strongly agreeing they feel better prepared to help their department/team set impactful objectives and measurable key results.
• Q2: Clarify basic org-level Theory of Change and gather available metrics.
• Q3: Facilitate use of the Theory of Change framework during Annual Planning offsites with XX% strongly agreeing the revamped Annual Planning process helped their department/team to set impactful objectives and measurable key results.
• Q3: Initiate the adoption of the Theory of Change framework to inform MTP 2.0 through workshops conducted by outside facilitators for the Board, C-Team and Foundation staff.
• Q3: Pilot the Social Return on Investment framework with one cross-departmental initiative for FY 2022-23.
• Q4: Facilitate completion of the multi-year Theory of Change to inform MTP 2.0 and secure commitment to leverage the framework for future annual planning.

Activities, Deliverables & Resourcing

Q1: Secure resources for Theory of Change training and workshops and to Social Return on Investment pilot project.

Activity	Responsible	Accountable	Consulted	Informed
Initiate planning discussions with each department lead to build awareness about ToC and SROI projects.	CM + SBB	SBB	All department leads, F&A	C-Level
Secure resources to facilitate ToC workshops for AP Delegates and GDI in Q2.	CM + SBB	SBB	All department leads, AP Delegates, F&A	C-Level
Schedule ToC workshops for Q2.	CM	SBB	All department leads, AP Delegates, F&A	C-Level
Secure resources with expertise in SROI that will share best practices and assist with creating a project timeline for developing SROI metric(s) starting in Q2.	CM + SBB	SBB	All department leads, F&A	C-Level, Board
Secure resources to facilitate ToC workshops for MTP 2.0 planning.	SBB	SBB	All department leads, AP Delegates, F&A	C-Level
Coordinate with Movement Comms on communication plans for upcoming quarters.	CM	SBB	Movement Comms

Q2: Facilitate Theory of Change workshops for all Annual Planning Delegates and Global Data & Insights with XX% strongly agreeing they feel better prepared to help their department/team set impactful objectives and measurable key results.
Q2: Clarify basic org-level Theory of Change and gather available metrics.

Activity	Responsible	Accountable	Consulted	Informed
Facilitate ToC workshops for all AP Delegates and GDI.	CM	SBB	All department leads, AP Delegates, F&A	All WMF
Work across departments to clarify basic org-level ToC.	CM + TBD	SBB	All department leads, AP Delegates	All C-Level
Survey ToC workshop participants to evaluate workshop effectiveness.	CM + TBD	SBB	AP Delegates, GDI	All department leads, F&A, C-Level
Align ToC metrics to MTP/AP dashboard project (see Key Result 2 below).	CM + TBD	SBB	All department leads, GDI	C-Level
Identify at least one strategic cross-departmental initiative with high potential for social impact and good evaluability to pilot SROI metrics.	CM + TBD	SBB	All department leads, All initiative members, GDI	C-Level
Begin developing SROI metric(s) along with standardized definitions and data collection plan for at least one strategic cross-departmental initiative.	CM + TBD	SBB	ll department leads, All initiative members, GDI	All WMF
Develop a communication plan to explain the MTP 2.0 planning process to Foundation staff and community.	CM + SBB	SBB	Internal Comms, Movement Comms	All WMF, Movement

Q3: Facilitate use of the Theory of Change framework during Annual Planning offsites with XX% strongly agreeing the revamped Annual Planning process helped their department/team to set impactful objectives and measurable key results.
Q3: Initiate the adoption of the Theory of Change framework to inform MTP 2.0 through workshops conducted by outside facilitators for the Board, C-Team and Foundation staff.
Q3: Pilot the Social Return on Investment framework with 50% of departments committed to reporting SROI metrics quarterly for FY22/23.

Activity	Responsible	Accountable	Consulted	Informed
Facilitate use of ToC framework for AP planning.	CMM + SBB	SBB	All department leads, AP Delegates, F&A	All WMF
Develop baseline reporting for initial ToC metrics where data are readily available.	TBD	SBB	All department leads, GDI	C-Level
Communicate out the results of the ToC metrics report to Foundation leadership and staff with actionable recommendations.	TBD	SBB	All department leads, GDI	All WMF
Implement SROI (value chain) data collection plan for at least one strategic cross-departmental initiative.	TBD	SBB	All department leads, All initiative members	All WMF
Initiate workshops for multi-year ToC creation with Board, C-Team and Foundation staff to inform 2.0 version of MTP.	SBB	SBB	All department leads, GDI	All WMF, Movement
Initiate planning for ongoing strategy work.	SBB	SBB	GDI	C-Level

Q4: Facilitate completion of the multi-year Theory of Change to inform MTP 2.0 and secure commitment to leverage the framework for future annual planning.

Activity	Responsible	Accountable	Consulted	Informed
Collect usability feedback and revise ToC metric mapping and pipelines based on actionability of baseline reporting.	TBD	CM + SBB	All department leads, All initiative members, GDI	All WMF
Continue workshops for multi-year ToC creation with Board, C-Team and Foundation staff to inform 2.0 version of MTP.	SBB	SBB	All department leads, GDI	All WMF, Movement
Explore preliminary analyses and report on SROI metrics development progress and learning for at least one strategic cross-departmental initiative.	CM + SBB	All department leads, All initiative members, GDI	All WMF

Key Result 2: MTP / AP Dashboard

By end of Jan 2022, design and roll out a data dashboard that can be used by at least 50% of Foundation departments for annual planning.

Intent and Desired Outcomes
Each year, as Wikimedia Foundation leadership and staff enter the annual planning process, questions arise about whether the activities we’ve invested in in the previous year have achieved the intended outcomes and whether or not we could be using our Foundation resources more effectively to deliver on our highest impact objectives.

In coordination with the work described in Key Result 1 above (developing an organization-wide theory of change and social return on investment metrics), the GDI team plans to launch a dashboard that synthesizes progress on key AP and MTP priorities to leadership & Foundation staff. We aim to launch the dashboard by the end of January 2022 as a tool for the FY 2022-23 annual planning process.

If the pilot dashboard is well received as a tool for improving strategic decision-making, it would be updated on an ongoing basis as a single “source of truth” for regularly monitoring and evaluating the Foundation’s performance.

Related Quarterly OKRs

• Q1: Define scope of end-product in coordination with Annual Planning team and define a short list of initial metrics for inclusion.
• Q2: Finalize metrics and map out data access points and develop data pipelines for MTP/AP evaluation/monitoring.
• Q3: Develop and soft launch internal dashboard for use in the annual planning process with XX% of participants strongly agreeing the dashboard helped their department/team to set impactful objectives and measurable key results.
• Q4: Maintain data pipeline and establish plan for future iteration.

Activities, Deliverables & Resourcing

Q1: Define scope of end-product in coordination with Annual Planning team and define a short list of initial metrics for inclusion.

Activity	Responsible	Accountable	Consulted	Informed
Develop shared measurement strategy for quarterly Tuning Sessions and for Annual Planning process with AP Team.	CM	SBB	F&A	All C-Level
Complete APP survey analysis to understand pain points and opportunities for FY 2022-23 planning.	JA + TA	SBB	F&A	All C-Level
Assess dashboard capabilities in Adaptive and other candidate dashboard tools.	CM + TBD	SBB	F&A, GDI, Data Engineering
Complete retrospective ToC and define metrics short list for dashboard.	CM + Mission Measurement	SBB	GDI	C-Level, F&A

Q2: Map out data access points and develop data pipelines for MTP/AP evaluation/monitoring.

Activity	Responsible	Accountable	Consulted	Informed
Hire BI developer.	CM	SBB	GDI	C-Level
Coordinate with the Process Improvement Team to define business metrics for dashboard inclusion.	CM	CM + SBB	F&A, GDI, Process Improvement Team	All C-Level
Secure agreement on which metrics from Mission Measurement consultation are critical for quarterly Tuning Sessions and annual planning.	CM	CM + SBB	All Department Leads	All C-Level
Develop a product brief defining the design, systems, and security requirements for the initial dashboard.	CM	CM + SBB	GDI	All WMF
Create a mapping of all data sources necessary for the dashboard.	CM	CM + SBB	All department leads, All data owners, GDI	All department leads, All data owners, GDI
Design dashboard prototype.	CM + TBD	CM + SBB	All department leads, GDI	All C-Level
Develop data pipelines.	TBD	CM + SBB	All data owners, GDI	All C-level, All Department Leads
Test data pipelines.	TBD	CM + SBB	All data owners, GDI	All C-level, All Department Leads

Q3: Develop internal dashboard and launch for use in the annual planning process.

Activity	Responsible	Accountable	Consulted	Informed
Develop dashboard	TBD	CM + SBB	GDI + TBD	All WMF
Develop user guidance, documentation, and demo materials.	TBD	CM + SBB	GDI + TBD	All WMF
Soft launch of the AP dashboard for planning.	CM + TBD	CM + SBB	GDI	All WMF
Assess usability and effectiveness of the dashboard for annual planning.	TBD	CM + SBB	GDI, APP Delegates, C-Level	All WMF
Begin planning for version 2.0.	CM	SBB	GDI

Q4: Maintain data pipeline and develop assessment of dashboard use in annual planning.

Activity	Responsible	Accountable	Consulted	Informed
Maintain data pipelines.	TBD	CM + SBB	GDI + TBD	All WMF
Report out project outcomes.	CM + TBD	CM + SBB	GDI	All WMF