Fundraising tech

This document should explain or link to everything needed for fundraising tech development.

About Fundraising Tech
Fundraising Tech is responsible for the security, stability, and development of the Wikimedia Foundation’s online donation systems. Millions of relatively small donations make up the majority of the Wikimedia Foundation’s operating budget every year. The donation systems created and maintained by Fundraising Tech were built specifically to make the small donor model a reality, across as many localities as possible to further ensure the continued independence of our mission.

We do not write banners or run tests, we support the people and software that run them.

International payment processing

 * The majority of donations run through integrations with 6 payment processors. We are also integrated with several other processors for accessibility, location and fundraising event uses. These integrations enable us to support online fundraising campaigns in approximately 30 countries each year.

Retention of donor/donation data

 * We maintain an internal donor database and CRM (CiviCRM). This data is crucial in measuring the impact of our fundraising campaigns, developing our global strategy, and maintaining relationships with past donors.

Tools to create, manage, and deliver online fundraising campaigns

 * We are the primary developers and maintainers of the CentralNotice system, which delivers banner notices to the wikipedias and the sister projects. Historically, banners on the wikis have been the primary method by which we entice users to donate.
 * Note: We partner with the Advancement department, Major Gifts, Banner and Email teams, operations and Donor services. Fundraising tech provides stable platforms and tools to Advancement, and they create specific fundraising campaigns and other donor-facing messaging and content.

Learn More
Fundraising tech/Essential systems

Fundraising tech/Components

Team
Technical work supporting the Wikimedia Foundation's fundraising efforts.
 * Fundraising Tech Director: Erika Bjune
 * Fundraising Tech Engineering Manager: Maggie Epps
 * Fundraising Tech Lead: Elliott Eggleston
 * Fundraising Tech Product Manager: David Strine
 * Roles/Responsibilities: Fundraising tech/Roles and Responsibilities
 * Team training matrix: Fundraising tech/Training
 * Regular meetings and their agendas Fundraising tech/FR-tech Meetings
 * I need help!

Current Work
Phabricator task board: #fundraising-backlog

Roadmap
fr-tech's detailed Roadmap

Goals
Fiscal Year 2018-2019


 * Goals by quarter

Rhythm and code freeze
We have a special yearly window of not deploying major changes to some critical systems starting in the fall, in preparation for the Big English drive. Game on again in mid-January, assuming we haven't emitted a puff of smoke in early December.

This code freeze gives our development an annual cadence, with each season seeing similar types and intensities of work, year-over-year. Any long-running FR-tech project should take this into account.

To Be Documented

 * User requirements:
 * Specifications:
 * Test plan:
 * Documentation plan:
 * User interface design docs:
 * Schedule:
 * Release management plan:
 * Release management plan:

Payment Processors

 * Adyen
 * Amazon
 * AstroPay
 * Dlocal [missing]
 * Ingenico and see filesrv (
 * Developer manual: WebCollect_technical_guide_2013_Q4.pdf
 * Workflows (needs updating): smb://filesrv1/fundraising/Tech/Ingenico/Old/Tech%20specs%20and%20guides%20from%20PaymentProcessing )
 * Note: In the codebase, Ingenico can refer to the older GlobalCollect--Ingenico's former name--integration and the new integration with their updated API
 * PayPal / PayPal Express Checkout
 * New integration manual: Fundraising tech/New integration manual

Installing the software stack
Much of our toolchain is provided by MediaWiki-Vagrant, please follow setup instructions at the main MediaWiki-Vagrant page and then visit Fundraising_tech/vagrant.

If you wish to forsake Vagrant and try to do it all the hard way, see Fundraising Tech/Donation Pipeline Setup.

Once you're set up, and have a patch ready, learn to deploy code.

Communications
You can find our engineering team on the #wikimedia-fundraising irc channel. Include "fr-tech" in your message to make sure we see it.

Fundraising Tech Systems Dashboard
The Fundraising Tech dashboard is located here https://grafana.wikimedia.org/dashboard/db/fundraising-overview Prometheus is used to record metric data which is then displayed in Grafana.

Current items being monitored include:


 * Server load - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=1&fullscreen
 * Nginx requests - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=6&fullscreen
 * Syslog message - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=16&fullscreen
 * Syslog messages lost to rate-limit - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=17&fullscreen
 * MySQL replication lag - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=2&fullscreen
 * MySQL read binlog position - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=3&fullscreen
 * MySQL read binlog position - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=4&fullscreen
 * MySQL queries - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=19&fullscreen
 * Donations queued to civicrm - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=15&fullscreen
 * Donations queue - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=9&fullscreen
 * Jobs queue - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=10&fullscreen
 * Other queues - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=11&fullscreen
 * Eqiad payments memcache current items - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=5&fullscreen
 * Payments Nginx responses - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=8&fullscreen
 * Nginx avg request time - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=7&fullscreen
 * Nginx avg bytes sent - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=13&fullscreen
 * Mail queue - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=14&fullscreen
 * Civi mail - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=18&fullscreen
 * Process-control jobs - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=20&fullscreen
 * Queue delay - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=21&fullscreen
 * Donation to Civi Processing Rate - https://grafana.wikimedia.org/dashboard/db/fundraising-overview?refresh=1m&orgId=1&panelId=22&fullscreen