Quality Assurance/Levels

Highest level: We run dedicated real-time test events. Such events are characterized by:


 * QA publishes a detailed test plan in advance
 * Event involves Product Managers, QA, Engineering Community directly
 * Scheduled meeting time on IRC and other communications channels published in advance
 * Actively solicit appropriate contributors in advance
 * Development team is prepared to act immediately on issues discovered by testing
 * Such events often (but not always) involve a separate prepared test environment, like beta labs or test2wiki.

Examples: first test for weekly deployment, first test for AFTv5.

Middle level: We run asynchronous test events open for a period of time.


 * QA publishes test plan in advance
 * Testing involves QA personnel but others at their own discretion
 * No synchronous meeting time scheduled, but QA and other staff are prepared to support testers on known channels, IRC and mail lists and Bugzilla
 * Test environment is often in production, either dedicated pages, feature to be tested controlled by user preference, or other special arrangement
 * Issues discovered may or may not be given high priority

Example: Visual Editor test event.

Everyday level: QA and Community always investigate incoming issue reports for severity.


 * Weekly deployments get special scrutiny
 * New features and recently changed features get special scrutiny
 * Issues found in normal operations are often added to the automated test backlog.

Examples: UploadWizard, Math extension.