Deployment tooling/Cabal/2016-01-23

Current Release
Prod: 3.4.2-1 (2016-12-14) Beta: 3.5.0-1~20170120173629.81 (2016-01-20)

Updates

 * 1st meeting since Dec 19th!

Next release
3.5.0 -- this week! Please check beta

MediaWiki

 * sync-dir and sync-file are now the same command
 * sync-dir is deprecated
 * l10n-purge is no longer a command
 * Flatten the deployment directory into a single git repository (prep for git sync)
 * /srv/mediawiki-staging is many git repos
 * /srv/medaiwiki is one git repo
 * Canaries now use explicit service for logstash_checker.py (checks fatals too)
 * old proxies and canaries (not in mw-installation) ignored
 * Fix for git-cache-info (sha1 regex)

Services/Scap3

 * (no message) -> (no justification)
 * Improved rollback behavior
 * all groups rolled back
 * per group and global failure limit
 * skip rollback where host failed due to ssh

General/Fun

 * optional fancy progress bar
 * improved cowsay compatibility for scap say

On the radar/Next/blocking

 * Scap deploy failed to sync git-fat artifacts
 * This is tracked in https://phabricator.wikimedia.org/T155856

Needs review

 * Make fancy progress bar optional
 * threepio branch: porting to python3 in progress, nearly finished

Soon™/Wishlist/On the radar/Updates

 * Targets via conftool (related: https://phabricator.wikimedia.org/T148992 )
 * Host list dumped on tin in /etc/dsh/groups (I think)
 * Check the list of nodes against conftool
 * TODO check conftool availability on deploy hosts/check with guiseppe to verify conftool is usable for outside projects
 * Keeping group in conftool
 * Fanout for scap3 (https://phabricator.wikimedia.org/T121276 )
 * This is primarily for MediaWiki scap3 deploys, could be helpful elsewhere
 * Few ideas floating around: clustershell, swift instead of fanout
 * Simple idea: scap3 deploy to proxy servers, scap2 deploy from proxies to remainder; monitor as we go

As Always

 * Phase 2
 * Phase 1
 * Workboard https://phabricator.wikimedia.org/tag/scap3/
 * Etherpad backed up to https://www.mediawiki.org/wiki/Deployment_tooling/Cabal
 * Future document https://www.mediawiki.org/wiki/Deployment_tooling/Future
 * Spreadsheet: https://docs.google.com/spreadsheets/d/1MlEsFxrLvdZdV_G82WEAIvBXr7ArO7nCEKaFClHhJEw/edit#gid=0