Deployment tooling/Cabal/2016-06-06

2016-06-06

• Config deploys
  • configs owned by deploy-service now on targets
  • need to settle on the puppet file on target and its location
    • which format? (yaml)
  • needs work on the config template

• Config deploy documentation: https://doc.wikimedia.org/mw-tools-scap/scap3/quickstart/setup.html#config-file-deploy

Submodule cache

• TODO task about submodule space on targets

List of Doom

No info:

abacist/abacist [ori]
analytics/kraken
cassandra/logstash-logback-encoder [godog urandom]
cassandra/metrics-collector [godog 
dropwizard/metrics
elasticsearch/plugins[gehel] 
??? eventlogging/eventlogging [ottomata]
fluoride/fluoride
graphoid/deploy [YuriK]
integration/config
integration/kss
integration/mediawiki-tools-codesniffer
integration/php-coveralls
integration/phpcs
integration/phpunit
integration/slave-scripts
jobrunner/jobrunner [__joe__ ori]
kibana/kibana [bd808]
logstash/plugins [bd808]
ocg/ocg [ugh]
parsoid/deploy [scheduled Soon™]
rcstream/rcstream [krinkle]
restbase/deploy
scholarships/scholarships
statsv/statsv
wdqs/wdqs [stats]
wikimedia/discovery [gehel]

Generated with:

find /srv/deployment -type d -maxdepth 2 | sort | nl (and some vim massage)

Services to migrate:

1. abacist/abacist
2.  Done analytics/aqs
3. analytics/kraken
4. analytics/refinery [in-progress]
5. cassandra/logstash-logback-encoder
6. cassandra/metrics-collector
7.  Done changeprop/deploy
8.  Done citoid/deploy
9.  Done cxserver/deploy
10. dropwizard/metrics
11.  Done dumps/dumps
12. elasticsearch/plugins
13.  Done eventlogging/analytics
14.  Done eventlogging/eventbus
15. ??? eventlogging/eventlogging
16. fluoride/fluoride
17. gdash/gdash (deprecated)
18. grafana/grafana (deprecated, using debs now on krypton)
19. graphoid/deploy
20. iegreview/iegreview [patch]
21. integration/config
22. integration/kss
23. integration/mediawiki-tools-codesniffer
24. integration/php-coveralls
25. integration/phpcs
26. integration/phpunit
27. integration/slave-scripts
28. jobrunner/jobrunner
29. kartotherian/deploy [in-progress]
30. kibana/kibana
31. librenms/librenms [ops]
32. logstash/plugins
33.  Done mathoid/deploy
34.  Done mobileapps/deploy
35. ocg/ocg
36. parsoid/deploy
37.  Done phabricator/deployment
38. rcstream/rcstream
39. restbase/deploy
40. scholarships/scholarships
41. sentry/sentry [possibly invalid]
42. servermon/servermon [ops]
43. statsv/statsv
44. tessera/tessera (deprecated)
45. tilerator/deploy [in-progress]
46. wdqs/wdqs
47. wikimedia/discovery
48.  Done zotero/translation-server
49.  Done zotero/translators

by last fetch date:

tin:/srv/deployment$ ls -latr */*/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  117 Jul 16  2013 fluoride/fluoride/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  119 Feb 27  2014 integration/php-coveralls/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  118 Mar 19  2014 gdash/gdash/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev 4152 May 12  2014 parsoid/Parsoid/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  111 Aug 18  2014 integration/phpcs/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  259 Oct 16  2014 mathoid/mathoid/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  128 Dec  2  2014 restbase/deploy.old/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  121 Dec 11  2014 integration/mediawiki-tools-codesniffer/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  124 Dec 27  2014 abacist/abacist/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  113 Feb 19  2015 integration/phpunit/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  132 Feb 24  2015 dropwizard/metrics/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  120 May  7  2015 kibana/kibana/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  121 May 30  2015 rcstream/rcstream/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  121 Jun 30  2015 tessera/tessera/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  130 Aug 11  2015 logstash/plugins/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  134 Sep 17  2015 elasticsearch/plugins/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  113 Oct  2  2015 integration/slave-scripts/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  252 Oct  2  2015 grafana/grafana/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  113 Oct 22  2015 iegreview/iegreview/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  127 Dec  3 14:52 restbase/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  135 Dec 15 01:59 jobrunner/jobrunner/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  126 Jan  1 19:27 scholarships/scholarships/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  138 Mar 22 16:31 cassandra/logstash-logback-encoder/.git/FETCH_HEAD
-rw-rw-r-- 1 twentyafterfour wikidev  531 Mar 24 00:27 phabricator/deployment/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  226 Mar 31 20:16 eventlogging/eventlogging/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  138 Apr 15 13:32 zotero/translation-server/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  276 Apr 15 13:34 zotero/translators/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  123 Apr 24 20:02 statsv/statsv/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  125 Apr 26 09:08 citoid/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  360 Apr 26 13:52 dumps/dumps/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  215 Apr 27 07:00 servermon/servermon/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  122 Apr 27 16:06 librenms/librenms/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  252 Apr 28 17:19 kartotherian/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 mforns          wikidev  228 May  2 11:08 eventlogging/analytics/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  226 May  3 13:14 eventlogging/eventbus/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  130 May 10 12:26 mobileapps/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  271 May 10 17:39 ocg/ocg/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  128 May 10 17:58 graphoid/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  267 May 16 17:34 analytics/refinery/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  129 May 19 18:24 parsoid/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  115 May 19 21:00 tilerator/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  127 May 20 08:08 mathoid/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  138 May 20 08:15 changeprop/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  128 May 20 11:32 cxserver/deploy/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  141 May 23 11:09 cassandra/metrics-collector/.git/FETCH_HEAD
-rw-rw-r-- 1 trebuchet       wikidev  887 May 23 17:05 wdqs/wdqs/.git/FETCH_HEAD

As Always

• Phase 2
• Phase 1
• Workboard https://phabricator.wikimedia.org/tag/scap3/
• Etherpad backed up to https://www.mediawiki.org/wiki/Deployment_tooling/Cabal
• Future document https://www.mediawiki.org/wiki/Deployment_tooling/Future
• Spreadsheet: https://docs.google.com/spreadsheets/d/1MlEsFxrLvdZdV_G82WEAIvBXr7ArO7nCEKaFClHhJEw/edit#gid=0