Fundraising tech/Contribution tracking



Contribution tracking, as of early 2020, is the last piece of our payments infrastructure that ties the payments-wiki frontend to the database server in the backend. We want to be able to take the backend db server down for maintenance without needing to shut down the payments frontend. To do this, we will stop creating drupal.contribution_tracking rows directly from the wikis and move to a queue-based insert system.

The sequence generator has an interface and is provided by a configurable class, all defined in SmashPig under the Core\SequenceGenerators namespace. Implementations for Redis and PDO backends are provided. We will use the Redis one in production and the PDO one with SQLite for tests. There is a convenience method Factory::getSequenceGenerator($sequenceName) that instantiates the generator defined in SmashPig's main.yaml under key sequence-generators/$seqenceName. Currently only one is defined, named contribution-tracking.

In DonationInterface, there is a new global $wgDonationInterfaceEnableContributionTrackingQueue which needs to be set to true to enable pulling new IDs from the sequence generator and sending c_t message to the queue. In the CRM codebase, instead of an enabling variable we have a patch that fully switches c_t table writes over to using the c_t sequence generator and queue.

The ContributionTrackingQueueConsumer class consumes messages from the c_t queue and inserts or updates messages in the contribution_tracking table. We should perhaps add a check on updates to ensure that only the expected type of update (adding a contribution ID) is happening. If messages are coming in with ct_ids that are already in the table and have more attributes than just a contribution ID, that could mean the sequence generator is malfunctioning.