Quality Assurance/How we prioritize
This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date.
QA meeting Feb 26 -- how do we prioritize what we work on, re testing?
- originally: QA was Chris:
- Michelle Grover for mobile
- Zeljko for test automation
- Quim for community events
Chris: overview of QA activities
Projects they've helped with: (13 entries)
- AFTv5 - E2
- Echo - E2
- Page Triage - E2
- Guided Tour - E3
- Account Creation - E3
- PDBHandler - Community
- Environments: (11 entries)
- labs host (ee prototype)
- beta cluster (AFT, mobile)
- caching (Antoine says, "like 99% of our infrastructure is about caching so caching is the top root cause of issues :-]")
- search <--- Lucene could be setup in beta
- mediawiki.org (echo)
- enwiki/prod (ve, e3)
- Test2wiki (upload wizard, math, etc.)
- future: vagrant
- Pro volunteers
- Wiki volunteers
- needs subject matter experts (to write in plain english the testing needs)
QA Weekly goals:
- See "How do we decide what to prioritize? Stuff where it's useful to test before production." discussion
- also see "How do we decide what to focus exploratory testing energy on? How do we balance it on a per-area basis? How do we get those teams to accept the help?" section
- Outcome: things that would benefit from more software testing?
- needs to know what needs to be tested before projection?
E2 wants to be able to test stuff before it is merged -- beta cluster for non-master branches is being worked on.
matrix: Y - features, X- types of testing
X-axis - Types of testing:
- test environment creation
- unit testing (qunit & Jenkins) -- skip for today
- integration testing (Jenkins) -- skip for today
- automated browser testing (can stop because code is bad, or test environement broken, deployment broken). More for stable stuff?
- configuration testing (make sure it works on all browsers and operating systems)
- Exploratory testing
- developer/product manager exploratory testing - manual browser testing, acceptance testing
- volunteer exploratory testing
- QA staff exploratory testing (ex. AFT for Chris, Michelle for mobile)
Prioritization matrix: https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Agte_lJNpi-OdGpJSHdMZV83UjdTdHc5UFVrdXBwaWc#gid=0 Want from features:
- 1 exploratory testing project/month
- 1 automated testing project/month
- synchronize focuses with community/communications activities (blog posts, community events, etc.)
Current idea of process: build out this spreadsheet, see that it has all our engineering projects & what kinds of testing they're amenable to, and regularly check in to update it with Howie/Chris/Erik.