Phlogiston/Running

From MediaWiki.org
Jump to: navigation, search

Phlogiston has a stable labs server for WMF use, here.

In normal operation, Phlogiston should run automatically every day on both the production (phlogiston-1) and development (phlogiston-2) servers. It runs right after the Phabricator dump is made available. The normal sequence of operation is:

  1. Download new dump
  2. Load the dump into the database
  3. For each specified scope:
    1. Reconstruct data
      1. Normally, this runs incrementally compared to the last date processed, so it only runs on one day of data.
    2. Regenerate the report completely.

Manual Control[edit]

The current practice is for only the Phlogiston developer to work on production, and for the Phlogiston developer to be the primary user of the development server. One use case for shared development is supported: users of reports may reconfigure their reports and then re-run the reports on the development server to see results immediately instead of the next day.

Phlogiston has no run control or locking; multiple Phlogiston reports run at the same time will have bad results. We therefore have a manual convention on the development server that all Phlogiston runs should happen in a shared console session, to prevent two runs from happening at once.

To re-run a report on the development server[edit]

  1. Change the configuration files to make the desired changes, and commit to github.
  2. Log in to phlogiston-2.
    1. You must already have a wikitech labs (not Tool Labs) shell account. See Getting Started.
    2. Your account must be set up to access Phlogiston.
      1. Admin note: user must be in group project-phlogiston; not sure if this is set at server or at labs level.
  3. Change to be the phlogiston user: sudo su - phlogiston.
    1. Your phlogiston shell account on phlogiston-2 must be in the project-phlogiston group.
  4. Join the shared console: tmux a -t mission_control.
    1. If this fails with a message about "no sessions", then there is not already a mission_control session. Create it with tmux new -s mission_control.
  5. Re-run Phlogiston.
    1. ./batch_phlog.bash -m report -s your_scope_prefix
      1. replace your_scope_prefix with the code for your scope, for example, and for Android, or ve for VisualEditor. This is determined when the files for this scope report are originally created.
    2. This will automatically update files from git, and then rerun the report. It will not reprocess any of the data.

This convention is not followed on phlogiston-1, which should not have multiple users running reports.

Automation[edit]

Phlogiston is run on both servers automatically every day with the crontab entry

# m h  dom mon dow   command

15 4    *   *   *    bash ~/phlogiston/batch_phlog.bash -m incremental -s ana -s and -s col -s cot -s discir -s dismap -s dis -s diswik -s fr -s ja -s ios -s lan -s phl -s red -s rel -s tpg -s ve >>~/phlog.log 2>&1

Data integrity and idempotency[edit]

The data dump includes all historical data from Phabricator, so only the most current dump is required for operation. Each data dump load will provide Phlogiston with complete information, and the data dump does not need to be reloaded until a new dump is available. Loading the dump is independent of any specific scopes.

Reconstruction and reporting are partitioned by scope. Changes to one scope will not affect any other.

An incremental reconstruction will operate from the most recent date available in the already-processed data, so if it is run a second time on the same day, it will not corrupt data. A complete reconstruction will begin by wiping all data for that scope.

A report will wipe the existing report on the website prior to generating a new report, so it is possible to end up with a broken report if the new report fails.