ORES/Multiscore problem

Warning:

The ORES infrastructure is being deprecated by the Machine Learning team, please check wikitech:ORES for more info.

This page describes a problem we have in celery around dependent sub-tasks.

ORES multiscore pattern breakdown. ORES scoring activity broken down into 4 diagrams: (A) A simple request for a single score without worrying about celery can be split into IO and CPU work. (B) Requesting multiple scores for a single revision adds a minimal amount of IO and CPU. (C) This is decomposed by celery into two types of job -- one job for doing the real work and N jobs for return the result. (D) This decomposition lets parallel request de-duplicate requests for the same score. This is a common pattern.