Quality Assurance/Levels

From mediawiki.org

Highest level[edit]

We run dedicated real-time test events. Such events are characterized by:

  • QA publishes a detailed test plan in advance
  • Event involves Product Managers, QA, Engineering Community directly
  • Scheduled meeting time on IRC and other communications channels published in advance
  • Actively solicit appropriate contributors in advance
  • Development team is prepared to act immediately on issues discovered by testing
  • Such events often (but not always) involve a separate prepared test environment, like Beta Cluster or test2wiki.

Examples: first test for weekly deployment, first test for AFTv5.

Middle level[edit]

We run asynchronous test events open for a period of time.

  • QA publishes test plan in advance
  • Testing involves QA personnel but others at their own discretion
  • No synchronous meeting time scheduled, but QA and other staff are prepared to support testers on known channels, IRC and mail lists and Bugzilla
  • Test environment is often in production, either dedicated pages, feature to be tested controlled by user preference, or other special arrangement
  • Issues discovered may or may not be given high priority

Example: Visual Editor test event.

Everyday level[edit]

QA and Community always investigate incoming issue reports for severity.

  • Weekly deployments get special scrutiny
  • New features and recently changed features get special scrutiny

Examples: UploadWizard, Math extension.