Wikimedia Release Engineering Team/Deployment pipeline/2019-03-28

From MediaWiki.org
Jump to navigation Jump to search

Last Time[edit]

Next Quarter Goals[edit]

Q4 goals are posted

General[edit]

TODOs from last time[edit]

  • In progress In progress TODO various attack vectors document to start
    • antoine and I started to talk about it
    • thcipriani to more thoroughly noodle
  • TODO: Joe & James_F to work on eventual 2019-04-01 email

RelEng[edit]

  • Additional NPM packages during container image build
    • Left suggestions for alternative approaches
  • Implementing `.pipeline/config.yaml`
    • A basic execution graph (directed acyclic graph thingy) is done. Need to work on mapping it to Jenkins Pipeline stage definitions.
      • Is Form 3 in the Graph examples coherent?
      • Perhaps we don't need separate pipelines defined in `.pipeline/config.yaml` if we have DAG execution (allowing pipelines from multiple projects in the same repo to run parallel or intersect)
      • Jenkins has limitations on parallel execution but same graph may map to true DAG execution in [unknown future CI system]
  • Integration of dependent changes pre-merge
    • james: we currently run into this situation (A and B both have changes which mutually depend on each other) and we have a mess
      • We tell people to build forwards/backwards compatibility in both A and B and then merge but (a) that's expensive of developer time, and (b) potentially fragile/cache expensive/etc.
      • Sometimes the circular dependencies are only sanely fixable with forced-merges (which we don't want to encourage, or even allow?), which is bad.
      • The alternative (supporting circular dependencies and merging them together) means we'd then need a system to keep track of which versions of A and B are each compatible, which could also be a mess.
    • dan: the zuul merger creates possible futures via the dependent pipeline currently, but we don't use it that way
      • it currently applies to git futures, but it could be leveraged to create deployment futures? (seems rather complicated)

Serviceops[edit]

  • effie finished cxserver switching Yes Done

Services[edit]

As Always[edit]