Structured Data Across Wikimedia/Section-level Image Suggestions/Data Pipeline

General architecture
The general data workflow is the same both for ALIS and for SLIS, as illustrated in the following diagram:

SLIS
SLIS-specific data processing steps are shown in the following diagram: inputs go into the main component, namely a Spark job, which is then executed as one task of the image suggestions Airflow parent job:

How it works
TODO

Code base

 * Spark job at https://gitlab.wikimedia.org/repos/structured-data/image-suggestions/-/blob/main/image_suggestions/section_image_suggestions.py
 * Airflow DAG at https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/platform_eng/dags/image_suggestions_dag.py