Toolserver:Admin:Change control

In the past, there was very little control over maintenance and root tasks on the Toolserver. While we try to announce work in advance, in general there's no coordination or approval of tasks. To resolve this, we are now testing change control for root users. Change control is a way of regulating changes to production systems; before any change is made, it must be logged and approved. Only then can the work proceed. For now, we are only enforcing logging, not approval. We might introduce approval at a later time depending on how things go.

This page explains how to log your changes into the change control system. If you make any changes to the Toolserver, you must follow the procedure described here.

Changes with no user impact

 * For changes that have no impact on service to users, including software installation, follow this procedure. NB: for software installation with user-visible service impact, *do not* use this procedure.


 * 1) Open a request in the MNT project in JIRA, describing the change you will make, the effect it will have, and why it needs to be done.
 * 2) Assign the request to yourself.
 * 3) Transition the request to the In Progress state.
 * 4) Perform the work.
 * 5) If the work was completed, Transition the request to the Done state.  Otherwise, transition to the Aborted state.

Changes with user-visible service impact

 * For scheduled maintenance which will affect users, follow this procedure.


 * 1) Decide on a time to perform the work.  Ideally this will be during the off-peak period (approx. 1AM to 8AM UTC).  This should be at least 24 hours in advance; for large outage windows, more notice (up to 7 days) may be appropriate.
 * 2) Open a request in the MNT project in JIRA, describing the change you will make, the effect it will have, and why it needs to be done.  Clearly describe the user-visible service impact and estimate the total outage window.
 * 3) Mail toolserver-announce and toolserver-l to announce the outage, and include a link to the MNT ticket.
 * 4) Wait until the scheduled outage window.
 * 5) Transition the request to the In Progress state.
 * 6) Perform the work.
 * 7) If the work was completed, Transition the request to the Done state.  Otherwise, transition to the Aborted state.
 * 8) Mail toolserver-announce and toolserver-l with the result of the maintenance.

For emergency or unplanned work with user-visible service impact

 * When it is infeasible to schedule maintenance because the service is already offline or something is seriously broken, follow this procedure.


 * 1) Perform the work.
 * 2) Open a request in the MNT project in JIRA, describing the change, why it was done, what the cause of the problem was (if known), and the outage window (if any).
 * 3) Transition the request to the Done state.
 * 4) Mail toolserver-announce and toolserver-l with a description of the work, and include a link to the MNT ticket.