Jump to content

Wikimedia Quality Services

Issue tracker: #Quality-and-Test-Engineering-Team (QTE)
From mediawiki.org
Wikimedia Foundation projects
Quality Services
Providing services that ensures the quality of software developed by the Foundation through manual and automated testing while fostering a culture where Quality is everyone's responsibility.
Group: Technology → Developer Experience
Start: 2024-11-14
Team members: Anthony Borba, Dom Walden, Edward Tadros (contractor), Elena Tonkovidova, Emeka Chukwukere, Esther Akinloose, George Mikesell (contractor), Rummana Yasmeen, Vaughn Walters,
Backlog: #Quality-and-Test-Engineering-Team_(QTE)
Management: Sean Long

The Quality Services team is a sub-team of the Developer Experience group within the Wikimedia Foundation. Established in November 2024 after a split of the QTE team, we improve the overall quality of the software developed by the Foundation.

Join our IRC channel: #wikimedia-qte connect.

Check out our blog posts at phab:phame/blog/view/21/.

Key guiding principles

[edit]
  • Ensuring Quality is a part of everyone's work.
  • Ensuring Quality is in every part of the SDLC.

Things we do

[edit]
Test Engineering Support
Team Test Engineering Contacts OKR Alignment
Abstract-Wikipedia Elena Tonkovidova WE2 - Vital Knowledge
Trust and Safety Dom Walden WE4 - Safety and Security
Connection Vaughn Walters WE1.4 - Task prioritization
Community Tech Dom Walden/George Mikesell WE1 - Contributor Experiences
Editing Rummana Yasmeen/Esther Akinloose WE1 - Contributor Experiences
Contributor Growth Elena Tonkovidova WE1 - Contributor Experiences
Data Products Emeka Chukwukere
Mobile Apps Anthony Borba WE3 - Consumer Experiences
Reader Experience Edward Tadros / Elena Tonkovidova WE3 - Consumer Experiences
Reader Growth Edward Tadros / Elena Tonkovidova WE3 - Consumer Experiences
Language and Product Localization George Mikesell WE2 - Vital Knowledge

How we work

[edit]

As of , the Quality Services team follows an embedded model with a unified team under the Developer Experience organization.

The Quality Services team supports 11 developer teams - each developer team having their own processes and structure and unique challenges and expertise.

The Test Engineer embedded on each organization is currently responsible for ensuring the quality of their team's work. This involves both monitoring Phabricator, as well as reporting on the overall health of the features.

How we test

[edit]

Software testing happens on each team individually, therefore QS testing strategy has evolved team by team to serve the needs of the individual teams. This means that teams differ in numerous aspects:

  • Automation
  • Manual / Exploratory Testing
  • Tools used for testing
  • Documenting tests
  • Documenting bugs
  • Maintaining metrics

This section will detail how we currently test for each team supported.

Abstract-Wikipedia (Elena)

[edit]

Trust and Safety (Dominic)

[edit]
  • Test Strategy doc: No singular document describing approach for team
  • Bug Tracking: Trust and Safety Phabricator
  • Automation Strategy: Trust and Safety commits code directly to others' repos, so follows their strategy
  • QA Process: Tickets come in to QA column, a QA engineer tests it.
  • Test plan documentation: Per project, User:DWalden (WMF)/Test2wiki k8s migration
  • Metrics tracked: None QA related

Connection (George)

[edit]

Community Tech (Dominic/George)

[edit]
  • Test Strategy doc: No doc that I am aware of, and it would vary depending on the project anyways.
  • Bug Tracking: Community Tech Phabricator
  • Automation Strategy: Community Tech delivers code directly to the repo, and follows their testing protocols/directions from there
  • QA Process:Split into two teams that are time zone friendly. Testers pick up any tasks that are in the QA column. If an issue related to the task is found, it will be moved back to the 'In Development' column with a ping to the engineer on the Phab task and Slack for a heads up.
  1. USA- Sea Lion Squad
  2. World- Fox Squad
  • Test plan documentation: Test Plan in general, but will be different per project
  • Metrics tracked: None that I am aware of for QA

Editing (Rummana/Esther)

[edit]
  • Test Strategy doc:
  • Bug Tracking: Phabricator on-ticket
  • Automation Strategy: Unit + Integration testing
  • QA Process: Ticket by ticket testing, off-cycle to provide feedback on features.

Contributor Growth (Elena)

[edit]

Data Products (Emeka)

[edit]
  • Test Strategy doc: API Strategy
  • Bug Tracking: Per ticket, as notes on ticket
  • Automation Strategy: Fully automated against API
  • QA Process: Dedicated support per project
  • Test plan documentation: per project
  • Metrics tracked: Weekly reporting

Mobile Apps (Anthony)

[edit]
  • Test Strategy doc: Mobile Test Automation
  • Bug Tracking: Phabricator board - iOS and Android
  • Automation Strategy: Android: Smoke test (espresso) + XCTest (unit/integration)
  • QA Process: QA handles regression/smoke test suites per RC, records bugs and retests as necessary
  • Test plan documentation: Smoke test documented in google sheets with per-ticket testing tracked in Phabricator.
  • Metrics tracked: # of bugs found weekly, # of smoke tests run, # of consumer reported bugs
  • Smoke Test Suite: Smoke Test Documentation

Reader Experience (Edward / Elena)

[edit]
  • Test Strategy doc: Readers Test Strategy
  • Bug Tracking: Per ticket, as notes on ticket
  • Automation Strategy: Define smoke test suite per project, automate repeatable tests.
  • QA Process: Currently only testing what makes it to the QA column. Deployment validation testing has not been implemented.
  • Test plan documentation: Individual testing is documented in Phab tasks.
  • Metrics tracked: Weekly reporting to team + QE channel

Reader Growth (Edward / Elena)

[edit]
  • Test Strategy doc: Readers Test Strategy
  • Bug Tracking: Per ticket as notes on ticket
  • Automation Strategy: Define smoke test suite per project, automate repeatable tests. Light testing due to experimental model (functionality might be short-lived)
  • QA Process: Light, per project.
  • Test plan documentation:
  • Metrics tracked: Weekly reporting

Language and Localization Product (George)

[edit]
  • Test Strategy doc: No doc that I am aware of, and it would vary depending on the project anyways
  • Bug Tracking: LPL Phabricator
  • Automation Strategy: None
  • QA Process: Test what comes in to the QA column
  • Test plan documentation: Test Plan in general, but will be different per project
  • Metrics tracked: Weekly reporting

General testing terms

[edit]
Exploratory testing
A hands-on, freeform testing approach where testers actively explore the application to find defects, for the purposes of feature or functional testing.
Test case
A detailed set of conditions, steps, inputs, and expected results used to determine whether a software feature works as intended. Each test case is designed to validate a specific requirement or functionality.
Test script
A set of instructions, often written in code or a scripting language, used to automate the execution of test cases. Sometimes this will be the same “steps” as in a Test Case, but written for automated testing.
Test plan
A comprehensive document that outlines the scope, approach, resources, schedule, and activities intended for testing a certain scope. It includes objectives, deliverables, test environment, features to be tested, and risk mitigation strategies for work that will not be tested.
w:Test strategy
A higher level document that describes an overall approach to testing as adopted by a group. For instance, a team supporting Mobile Applications has a test strategy involving the tools and technologies involved in testing as well as standards to ensure quality goals are met.

Testing phases

[edit]
Test planning
The process of defining the objectives, scope, approach, and schedule of software testing activities. It involves identifying resources, estimating effort, determining test deliverables, and outlining risk mitigation strategies. The output is typically a Test Plan document.
Acceptance testing
A type of testing conducted to determine whether a system satisfies business requirements and is ready for delivery. It is often performed by the end user or client.
Smoke testing
A preliminary round of testing to verify that the basic and critical functionality of a software build is working. Often referred to as "build verification testing," it acts as a quick check before deeper testing is performed.
Regression testing
The process of re-running previously completed tests to ensure that recent code changes haven’t negatively affected existing functionality.
User testing
A technique in which real users interact with the software to evaluate its ease of use, interface design, and overall user experience.
Integration testing
This phase focuses on verifying the interfaces and interaction between different components or systems. The goal is to ensure that combined modules work together as expected, often by simulating data flow and dependencies.
Unit testing
Tests that verify individual components or functions of a program in isolation. It is usually automated and written by developers to ensure each "unit" behaves correctly with a given input and expected output.
Prospective testing

...testing precursors to a built product…may include discussing and questioning ideas for a product or feature; review of requirement and specification documents; performing testing on mock-ups, wireframes, prototypes, or simulators.

Primary testing

...testing now that we have a built product.

Automated testing terms

[edit]
End to end test automation (also called system testing)
Automated testing of a complete user workflow from start to finish to ensure the entire system behaves as expected. This includes frontend, backend, databases, and third-party integrations. E2E tests mimic real user behavior to validate critical application flows. This typically uses a “Black box” testing approach, where a system under test is acted upon by an outside agent (in this case, a browser agent).
Selenium automation
Open-source software that provides a suite of tools to automate web-based applications. Can be used to validate function-level testing (unit and integration tests) or end-to-end browser tests (using webdriver).
Selenium browser automation
Specifically refers to Selenium’s capability to automate browser actions—clicking, typing, navigating, and validating web pages—across multiple browsers like Chrome, Firefox, Safari, and Edge. It leverages browser drivers (e.g., ChromeDriver) to control browsers programmatically.
WebdriverIO
Browser and mobile testing framework that utilizes the Webdriver protocol + Puppeteer to automate browser tasks, used across WMF.
Cypress automation
Using the open-source Javascript-based cypress.io testing framework to automate frontend web tests.
Playwright automation
Uses Playwright—a Node.js-based open-source framework created by Microsoft—for automating web browser interactions. It supports Chromium, Firefox, and Safari and is known for its reliability, cross-browser testing, and modern architecture.
Critical User Journey (CUJ)
The most important user journeys within a given scope (for example: feature, full site).
User Acceptance Testing (UAT)
A type of testing done by users with open-ended questions to determine if it solves the proposed need. Typically not only finds bugs but also design improvement opportunities
Risk-based testing
An approach to testing that prioritizes what to validate based on the ultimate risk presented to the organization or consumers. For example, if you do not adhere to risk-based testing, you may take the approach to test the "code last touched" as a way to limit the amount of bugs introduced. In risk-based testing, you may prioritize testing primary consumer user journeys over the item last touched.
Risk based testing requires agreements between engineering, QA, product management and other leaders to ensure that risk is fully understood, as there is typically a de-prioritization of "non-risky" changes.

Automated tests available

[edit]

Last updated: 2026-04-06 00:01:04 UTC | 44 repos | 369 tests | Updated weekly by an automated cron job on Toolforge. To update which repos appear here, edit repos.txt.

Core

[edit]

Core (wdio)

[edit]
  • Special:SpecialPages > should not have Edit link
  • User > should be able to log in without page object
  • Main page > should have edit link when using expect
  • Main page > should have edit link when using mocha
  • API > Main Page should exist
  • API > Missing Page should not exist
  • Page > should be previewable
  • Page > should be creatable
  • Page > should be re-creatable
  • Page > should be editable
  • Page > should have history
  • Page > should be undoable
  • Page admin actions > should be deletable
  • Page admin actions > should be restorable
  • Page admin actions > should be protectable
  • Special:RecentChanges > shows page creation
  • Temporary user > should not see signup form fields relevant to named users
  • Temporary user account creation > should be able to create account
  • User > should be able to create account
  • User > should be able to log in
  • User > named user should see extra signup form fields when creating an account
  • User > should be able to block a user
  • BlankPage > should have its title

Extensions

[edit]

AbuseFilter (wdio)

[edit]
  • Special:AbuseFilter > page should exist on installation
  • Special:AbuseFilter > page should have the button for creating a new filter
  • Filter editing > The editing interface > is visible to logged-in admins
  • Filter editing > Trying to open a non-existing filter > I should receive an error
  • Filter editing > Creating a new filter > edit can be saved (1)
  • Filter editing > Creating a new filter > saved data is retained (1)
  • Filter editing > Editing an existing filter > edit can be saved (2)
  • Filter editing > Editing an existing filter > saved data is retained (2)
  • Filter editing > Restoring an old version of a filter > edit can be saved (3)
  • Filter editing > Restoring an old version of a filter > saved data is retained (3)
  • Filter editing > CSRF protection > a CSRF token is required to save the filter
  • Filter editing > CSRF protection > even if the token is invalid, the ongoing edit is not lost
  • Filter editing > Trying to save a filter with bad data > cannot save an empty filter
  • Filter editing > Trying to save a filter with bad data > cannot save a filter with rules but no name
  • Filter editing > editing interface is not visible to logged-out users
  • When importing a filter > the interface should be visible
  • When importing a filter > it should redirect to ViewEdit after submission
  • When importing a filter > bad data results in an error
  • When importing a filter > valid data shows the editing interface
  • When importing a filter > Data on the editing interface is correct > filter specs are copied
  • When importing a filter > Data on the editing interface is correct > filter flags are copied
  • When importing a filter > Data on the editing interface is correct > filter actions are copied
  • When importing a filter > Data on the editing interface is correct > the imported data can be saved

AchievementBadges (wdio)

[edit]
  • Special:Achievements > shows a logged-in user hint of long-user-page

AdvancedSearch (wdio)

[edit]
  • Advanced Search > inserts advanced search elements on search page
  • Advanced Search > allows logged-in users to remember the selection of namespaces for future searches
  • Advanced Search > selects the users default namespaces when logged in
  • Advanced Search > selects the namespaces from the URL
  • AdvancedSearch > namespace selection
  • AdvancedSearch > re-adds filetype namespace after search when file type option has been selected but namespace has been removed
  • AdvancedSearch > adds/removes the namespace tag when the namespace option is clicked
  • Advanced Search > adds search parameters to pagination links
  • Advanced Search > submits the search on enter when there is no text in "These Words" field
  • Advanced Search > submits the search with the specific chosen language

CampaignEvents (wdio)

[edit]
  • Edit Event Registration > can allow organizer to update event page and dates
  • Edit Event Registration > can allow organizer to change the event to be in person
  • Edit Event Registration > can allow organizer to change the event to be online and in-person
  • Edit Event Registration > can allow organizer to add an additional organizer
  • Enable Event Registration @daily > is configured correctly
  • Enable Event Registration @daily > requires event data
  • Enable Event Registration @daily > can be enabled
  • Event page > can have one user register publicly
  • Event page > can have one user register privately
  • Event page > can have a user cancel registration
  • MyEvents > can allow organizer to search events by name
  • MyEvents > can allow organizer to delete registration of first event in My Events

Cargo (wdio)

[edit]
  • Special:CargoTables > displays table information
  • Special:Drilldown > displays table name with proper page count
  • Special:Drilldown > displays proper data when drilling down

CentralNotice (wdio)

[edit]
  • CentralNotice > banner is displayed on Main Page

CirrusSearch (wdio)

[edit]
  • Smoke test for search > Search suggestions
  • Smoke test for search > Fill in search term and click search
  • Smoke test for search > Search with accent yields result page with accent

Cite (cypress)

[edit]
  • Cite popups integration > simple popup on hover and hide on leave
  • Cite popups integration > includes scrollbar and fadeout on long previews
  • Cite popups integration > hovering nested reference
  • Re-using refs in Visual Editor > should display re-used reference in article with correct footnote number and notification in context dialog
  • Re-using refs in Visual Editor > should display correct ref content and name attribute for re-used ref with existing name attribute
  • Re-using refs in Visual Editor using templates > should add a template reference and verify correct content in both saved and edit mode
  • Visual Editor Cite Integration > should edit and verify reference content in Visual Editor
  • Visual Editor Cite Integration > should display existing references in the Cite re-use dialog
  • Visual Editor Wt 2017 Cite Integration > should be able to create a basic reference
  • Visual Editor Wt 2017 Cite Integration > should be able to create a VE-Cite tool template

CodeMirror (wdio)

[edit]
  • CodeMirror code folding for the wikitext 2010 editor > folds the template parameters via the button
  • CodeMirror code folding for the wikitext 2010 editor > expands the template parameters via the button
  • CodeMirror code folding for the wikitext 2010 editor > folds the extension tag via the button
  • CodeMirror code folding for the wikitext 2010 editor > expands the extension tag via the button
  • CodeMirror code folding for the wikitext 2010 editor > folds all via keyboard shortcut
  • CodeMirror code folding for the wikitext 2010 editor > expands all via keyboard shortcut
  • CodeMirror code folding for the wikitext 2010 editor > folds all <ref> tags via keyboard shortcut
  • CodeMirror bracket match highlighting for the wikitext 2010 editor > highlights matching bracket
  • CodeMirror bracket match highlighting for the wikitext 2010 editor > matches according to cursor movement
  • CodeMirror (enabled) - VisualEditor 2017 wikitext editor > opens with the CodeMirror view displayed and focus set on the VE surface
  • CodeMirror (enabled) - VisualEditor 2017 wikitext editor > updates CodeMirror with VE document changes
  • CodeMirror (enabled) - VisualEditor 2017 wikitext editor > retains content when CodeMirror is disabled and maintains focus on VE surface
  • CodeMirror (enabled) - VisualEditor 2017 wikitext editor > retains content when CodeMirror is re-enabled
  • CodeMirror (enabled) - VisualEditor 2017 wikitext editor > adjusts gutter accordingly when pasting many lines of wrapping text
  • CodeMirror (enabled) - VisualEditor 2017 wikitext editor > should only load necessary modules when the CodeMirror preference is unset
  • CodeMirror textSelection for the wikitext 2010 editor > sets and gets the correct text when using setContents and getContents
  • CodeMirror textSelection for the wikitext 2010 editor > has usage of .val() routed to CodeMirror
  • CodeMirror textSelection for the wikitext 2010 editor > sets and gets the correct selection when using setSelection and getSelection
  • CodeMirror textSelection for the wikitext 2010 editor > correctly replaces the selected text when using replaceSelection
  • CodeMirror textSelection for the wikitext 2010 editor > returns the correct values for getCaretPosition
  • CodeMirror textSelection for the wikitext 2010 editor > correctly wraps the selected text when using encapsulateSelection
  • CodeMirror textSelection for the wikitext 2010 editor > correctly inserts the 'peri' option when using encapsulateSelection
  • CodeMirror textSelection for the wikitext 2010 editor > applies 'pre'/'post' to each line when 'splitlines' is used with encapsulateSelection
  • CodeMirror textSelection for the wikitext 2010 editor > scrolls to the correct place when using scrollToCaretPosition
  • CodeMirror textSelection for the wikitext 2010 editor > retains the contents after turning CodeMirror off

CommunityConfiguration (cypress)

[edit]
  • Community Configuration Example Page > should save configuration changes and verify them on the example page
  • Community Configuration Example Page > Form elements and basic functionality > should have all expected form elements and labels
  • Community Configuration Example Page > Form elements and basic functionality > should have a save button
  • Community Configuration Example Page > Form elements and basic functionality > should have a disabled save button for logged-out users
  • Community Configuration Example Page > Form elements and basic functionality > should update a simple string via API and verify the update on the form

CommunityRequests (wdio)

[edit]
  • WishlistIntake wish submission > should prompt logged out users to login
  • WishlistIntake wish submission > should show the form with VisualEditor when browsing to the intake form
  • WishlistIntake wish submission > should show errors when submitting an incomplete form
  • WishlistIntake wish submission > should not show an error if a title is over 100 chars because of translate tags
  • WishlistIntake wish submission > should hide errors if all required fields are filled in on submission
  • WishlistIntake wish submission > should show all the data entered in the form

ContactPage (wdio)

[edit]
  • ContactPage on Version page > ContactPage is listed in the version page under the special page category

ContentTranslation (wdio)

[edit]
  • Unified Dashboard - Desktop - New translation > should allow publishing a new translation started through search
  • Unified Dashboard - Suggestions > should show suggestions tab
  • Unified Dashboard - Suggestions > should allow user to save/unsave a suggestion for later
  • Unified Dashboard - Suggestions > should allow dismissing a suggestion
  • Unified Dashboard - Suggestions > should allow refreshing suggestions

Echo (wdio)

[edit]
  • Echo > alerts and notices are visible after logging in @daily
  • Echo > flyout for alert appears when clicked @daily
  • Echo > flyout for notices appears when clicked @daily
  • Notifications > checks for Notifications Page @daily

ElectronPdfService (wdio)

[edit]
  • ElectronPdfService > pdf download button is visible

EntitySchema (cypress)

[edit]
  • Schema Viewing Page > accepts statements of entity schema data value type
  • Schema Edit Page > given that a user is allowed > returns to schema view page on submit
  • Schema Edit Page > given that a user is allowed > detects an edit conflict when submitting the same form from two windows
  • Schema Edit Page > given that a user is allowed > properly limits the input length
  • Schema Edit Page > given the user is blocked > cannot be edited
  • NewEntitySchema:Page > is possible to create a new schema with full data
  • NewEntitySchema:Page > is possible to create a new schema with only a label
  • NewEntitySchema:Page > limits the name badge input length
  • NewEntitySchema:Page > limits the schema text input length
  • NewEntitySchema:Page > when blocked > cannot load form
  • NewEntitySchema:Page > when blocked > cannot submit form
  • SetEntitySchemaLabelDescriptionAliasesPage:Page > detects an edit conflict based on the baserev parameter
  • SetEntitySchemaLabelDescriptionAliasesPage:Page > limits the input length
  • SetEntitySchemaLabelDescriptionAliasesPage:Page > is possible to edit Schema in another language
  • SetEntitySchemaLabelDescriptionAliasesPage:Page > has existing data already prefilled
  • Schema Viewing Page > doesn\
  • wbui2025 entity schema datatype > mobile view - entity-schema datatype > displays item statement and supports full editing workflow

examples (wdio)

[edit]
  • api > can create a user
  • api > can read a page
  • Examples > is configured correctly

FacetedCategory (wdio)

[edit]
  • Special:CategoryIntersectionSearch > shows a page if valid subpage is given
  • Special:CategoryIntersectionSearch > shows a page if the category contains a space

GlobalWatchlist (wdio)

[edit]
  • Special:GlobalWatchlist > works with normal display

GrowthExperiments (cypress)

[edit]
  • Add a Link > link inspector can be used to accept/reject links and save an article.
  • Add Image Structured Task > desktop: user can view image info and image details
  • Add Image Structured Task > mobile: user can close the image suggestion UI
  • Special:EditGrowthConfig > redirects to Special:CommunityConfiguration
  • Special:Homepage > Shows a suggested edits card and allows navigation forwards and backwards through queue
  • Impact > shows the user\
  • Special:MentorDashboard > allows enrolling as a mentor
  • Revise Tone > On desktop > Shows the Revise Tone Edit Check
  • Revise Tone > On desktop > Closes the Editor when declining Edits and suggests a new task
  • Revise Tone > On desktop > Shows the Revise Tone Edit Check and tags edits
  • Revise Tone > On mobile > Shows the Revise Tone Edit Check
  • Revise Tone > On mobile > Closes the Editor when declining Edits and suggests a new task
  • Revise Tone > On mobile > Shows the Revise Tone Edit Check and tags edits
  • Template-based tasks > saves change tags for unstructured task edits made via VisualEditor
  • Special:Log > Shows the GrowthExperiments event Mentor assignment changes

GuidedTour (cypress)

[edit]
  • template spec > passes

Math (wdio)

[edit]
  • Math > should work for addition
  • Math polyfills skipFunction > returns true when all MathML features are supported
  • Math polyfills skipFunction > returns false when a required feature is missing
  • Math polyfills skipFunction > skips loading when the skipFunction returns true
  • Math polyfills skipFunction > loads when the skipFunction returns false

Newsletter (wdio)

[edit]
  • Newsletter > page should exist on installation

OpenIDConnect (wdio)

[edit]
  • OpenIDConnect > can login
  • OpenIDConnect > is configured correctly

PageTriage (wdio)

[edit]
  • Special:NewPagesFeed > is viewable
  • Special:NewPagesFeed > new article appears in feed
  • PageTriage Toolbar > should load
  • PageTriage Toolbar > should allow user to add a maintainence tag to a page
  • PageTriage Toolbar > should allow user to add a maintainence tag to a page and send a note to user

ParserMigration (wdio)

[edit]
  • ParserMigration > should use parsoid with useparsoid=1 param
  • ParserMigration > should use legacy parser with useparsoid=0 param

Popups (wdio)

[edit]
  • Dwelling on a valid page link > I should see a page preview
  • Dwelling on a valid page link > I should not see a page preview on hash fragment
  • Dwelling on a valid page link > Abandoning link hides page preview

ProofreadPage (wdio)

[edit]
  • For a page with the eis, the page status module > should initialize with the value of current page
  • For a page with the eis, the page status module > should update on page change
  • For a page with eis enabled > toolbar loads
  • For a page with eis enabled > prev to page 1
  • For a page with eis enabled > next to last page
[edit]
  • ReadMore > ReadMore is present in Minerva @daily

ReportIncident (wdio)

[edit]
  • ReportIncident dialog > Should open the dialog if the tools menu is used
  • ReportIncident dialog > Should be able to advance to step 2 and see a form with the expected fields
  • ReportIncident dialog > Should ask for additional details only when selecting "Something else"
  • ReportIncident dialog > Should display form errors when submit attempted with no form data
  • ReportIncident dialog > Should be able to submit a form with valid data
  • ReportIncident dialog > Should be able to fill a report from a thread

RevisionSlider (wdio)

[edit]
  • RevisionSlider expand > does not automatically expand by default
  • RevisionSlider expand > expands automatically when auto expand is on
  • RevisionSlider expand > does not expand automatically when auto expand is off
  • RevisionSlider expand > hides when collapsed manually
  • RevisionSlider diff links > older edit diff link can be clicked
  • RevisionSlider diff links > newer edit diff link can be clicked
  • RevisionSlider filter highlighting > highlights revisions by the same user when I use the user filter
  • RevisionSlider filter highlighting > stops highlighting revisions when the filter is clicked twice
  • RevisionSlider filter highlighting > highlights revisions that have the same tag when I use the tag filter
  • RevisionSlider filter highlighting > highlights revisions that have the same tag when I use the tag filter after I used the user filter
  • RevisionSlider filter highlighting > only highlights revisions that have the same tag when I selected a user but hover a tag filter
  • RevisionSlider filter highlighting > re-applies highlight when I selected a user but hover and on-hover a tag filter
  • RevisionSlider help > tutorial is present on first load
  • RevisionSlider help > tutorial is not present after it was dismissed once
  • RevisionSlider help > tutorial sequence works
  • RevisionSlider history > can be accessed using browser back and forward buttons after clicking
  • RevisionSlider history > can be accessed using browser back and forward buttons after dragging
  • RevisionSlider pointers > can be dragged
  • RevisionSlider timeline arrows > should be disabled with 3 revisions
  • RevisionSlider timeline arrows > should be enabled with adequate revisions
  • RevisionSlider revision tooltips > appears and remains on hovering it

Sanctions (wdio)

[edit]
  • Sanction > should be canceled by the author
  • Sanction > should be rejected if three users object
  • Sanction > should be passed if three users support before expired
  • Sanction > should block the target user of the passed sanction when logged in
  • Sanction > should not touch the summary of a expired handled sanction
  • Special:Sanctions > should hide and show the form as the conditions change
  • Special:Sanctions > should add voted tag on a sanction
  • Special:Sanctions > should show > an anonymous user not-logged-in warning
  • Special:Sanctions > should show > a newly registered user that you are too new

Score (cypress)

[edit]
  • add score statement > mobile view > loads the item view, allows statements to be edited

TemplateData (wdio)

[edit]
  • TemplateData users can favorite templates via VisualEditor > show a template search field, with focus
  • TemplateData users can favorite templates via VisualEditor > show anon users a message about needing to log in
  • TemplateData users can favorite templates via VisualEditor > logged-in users can favorite templates from the search results

TemplateWizard (wdio)

[edit]
  • TemplateWizard > has 1 (required) field visible
  • TemplateWizard > has 5 fields visible
  • TemplateWizard > has template inserted
  • TemplateWizard > can be closed from search

TwoColConflict (wdio)

[edit]
  • TwoColConflict EditUi > will switch to edit mode by clicking the column that is selected
  • TwoColConflict EditUi > edits of unchanged paragraphs should be saved
  • TwoColConflict EditUi > paragraph edits can be reverted
  • TwoColConflict EditUi > saving an editor with no changes will preserve the highlight portions
  • TwoColConflict without JavaScript > is showing the default version correctly
  • TwoColConflict without JavaScript > is showing the talk page version correctly
  • TwoColConflict without JavaScript > handles order selection on the talk page version correctly
  • TwoColConflict save and preview > should save a resolved conflict successfully including changes
  • TwoColConflict save and preview > should show a correct preview page including changes
  • TwoColConflict > shows the talk page screen on conflicts that also add new lines
  • TwoColConflict > on talk page conflicts > shows the talk page screen correctly
  • TwoColConflict > on talk page conflicts > swaps blocks when switch button is clicked
  • TwoColConflict > on talk page conflicts > shows correct preview when swapped
  • TwoColConflict > on talk page conflicts > stores correct merge when swapped and edited

VisualEditor (wdio)

[edit]
  • Content Editable > should load when an url is opened @daily
  • Content Editable > should be editable
  • Content Editable > should save an edit
  • Content Editable > should insert a table
  • Content Editable > should insert Bullet list
  • Content Editable > should insert Numbered list
  • Content Editable > should insert and indent Bullet list
  • Content Editable > should insert and indent Numbered list
  • Content Editable > should insert an internal link
  • Content Editable > should insert an external link
  • Content Editable > should insert a comment
  • Content Editable Formatting > should change text to Page title
  • Content Editable Formatting > should change text to Heading
  • Content Editable Formatting > should change text to Sub-heading 1
  • Content Editable Formatting > should change text to Sub-heading 2
  • Content Editable Formatting > should change text to Sub-heading 3
  • Content Editable Formatting > should change text to Sub-heading 4
  • Content Editable Formatting > should change text to Preformatted
  • Content Editable Formatting > should change text to Block quote
  • Content Editable Formatting > should change formatting to Bold
  • Content Editable Formatting > should change formatting to Italic
  • Content Editable Formatting > should change formatting to Superscript
  • Content Editable Formatting > should change formatting to Subscript
  • Content Editable Formatting > should change formatting to Computer code
  • Content Editable Formatting > should change formatting to Strikethrough
  • Content Editable Formatting > should change formatting to Underline
  • Toolbar > should open notices popup as soon as it loads
  • Toolbar > should open switch editor
  • Toolbar > should open page options
  • Toolbar > should open help popup
  • Toolbar > should open insert menu
  • Toolbar > should open structure options menu
  • Toolbar > should open style text options
  • Toolbar > should open format paragraph menu

Wikibase (cypress)

[edit]
  • wbui2025 language preferences and RTL functionality > mobile view > Checking RTL layout when editing statements
  • wbui2025 add qualifiers > mobile view > is possible to add and edit a qualifier
  • wbui2025 add qualifiers > mobile view (wbui2025) - tabular-data qualifier > can add a tabular-data qualifier with lookup
  • wbui2025 add qualifiers > mobile view (wbui2025) - geo-shape qualifier > can add a geo-shape qualifier with lookup
  • wbui2025 add qualifiers > mobile view (wbui2025) - geo-shape qualifier > shows dropdown menu when typing in geo-shape qualifier lookup
  • wbui2025 add reference > mobile view > is possible to add references for string and lookup datatypes
  • wbui2025 item view add statement > mobile view > loads the item view and shows property selector
  • wbui2025 item view add additional value to existing statement > full add-value workflow > opens the add-value modal, searches, selects, and confirms value
  • wbui2025 item view add additional value to existing statement > cancel behavior > opens the add-value modal and cancels cleanly without enabling the save button
  • wbui2025 entityId datatypes (item, property) > mobile view - > allows adding
  • wbui2025 edit references > mobile view > references are editable and deletable
  • wbui2025 item view edit statements > mobile view > loads the item view and shows a statement, which can be edited,
  • wbui2025 string datatypes (tabular-data, geo-shape, commonsMedia) > mobile view - tabular-data datatype > displays tabular-data statement and supports full editing workflow
  • wbui2025 string datatypes (tabular-data, geo-shape, commonsMedia) > mobile view - geo-shape datatype > displays geo-shape statement and supports full editing workflow
  • wbui2025 string datatypes (tabular-data, geo-shape, commonsMedia) > mobile view - commonsMedia datatype > displays commonsMedia statement and supports full editing workflow
  • wbui2025 time datatypes > mobile view - time datatype > allows adding time statement to empty item,
  • wbui2025 item view publish statement changes > mobile view > loads the item view and shows a statement, which can be edited
  • wbui2025 item view > mobile view > loads the item view

WikibaseLexeme (cypress)

[edit]
  • Lexeme:Forms > can be open the add Form, cancel the add, open it again and save.
  • Lexeme:Forms > can add, edit and remove representation. it
  • Lexeme:Forms > has statement list and can edit statements on a new Form
  • Lexeme:Forms > FormId generation > FormId counter is not decremented when addForm is undone
  • Lexeme:Forms > FormId generation > FormId counter is not decremented when old revision is restored
  • Lexeme:Forms > can be removed
  • Lexeme:Header > can edit the language and lexical category of a Lexeme
  • Lexeme:non-existing > says the entity does not exist
  • Lexeme:Senses > Sense header and container exist
  • Special:MergeLexemes > shows the form
  • Special:MergeLexemes > when blocked > is not possible to edit
  • NewLexeme:Page > shows the form and creating a lexeme is possible
  • Special:RecentChanges > shows lemmas in title links to lexemes on Special:RecentChanges
  • Special:Watchlist > shows lemmas in title links to lexemes on Special:Watchlist
  • Lexeme:Statements > can be added
  • Special Version Page > verifies that the WikibaseLexeme extension loads
  • wbui2025 Lexeme entityId datatypes (lexeme, form, sense) > mobile view - > displays item statement and supports full editing workflow

WikibaseQualityConstraints (cypress)

[edit]
  • show constraints > desktop view > loads the item view with a simple violation
  • show constraints > desktop view > loads the item view with an advanced violation
  • show constraints > wbui2025 mobile view > adds an indicator, and a popover containing all violations

WikiLambda (wdio)

[edit]
  • Installation checks > CreateObject > page should exist on installation but deny to logged-out user
  • Installation checks > CreateObject > page should exist on installation and work when logged in
  • Installation checks > RunFunction > page should exist on installation
  • Installation checks > ListObjectsByType > page should exist on installation
  • Dynamic Test for Implementation and Test Case Connection > Admin user > should dynamically connect or disconnect the implementation based on its initial state
  • Dynamic Test for Implementation and Test Case Connection > Admin user > should dynamically connect or disconnect the test case based on its initial state
  • Function > Function viewer (CUJ1) > should allow to evaluate a function
  • Function > Function editor (CUJ2) > should create a new function and display the function name
  • Function > Function editor (CUJ2) > should create a new function and display function aliases
  • Function > Function editor (CUJ2) > should create a new function and display function input labels and types
  • Function > Function editor (CUJ2) > should create a new function and display function output type
  • Implementation (CUJ 5) > Implementation via code > should create and publish a new implementation
  • Implementation (CUJ 5) > Implementation via composition > should create and publish a new implementation
  • Tester > Create a new test (CUJ 4) > should fill and submit the tester form
  • Tester > Edit the test > should edit the about block test

Skins

[edit]

MinervaNeue (wdio)

[edit]
  • Opening and closing the reference drawer > Opening a reference
  • Opening and closing the reference drawer > Opening a nested reference
  • Manage Watchlist > Add an article to the watchlist

Wikibase

[edit]

termbox (wdio)

[edit]
  • Termbox: AnonEditWarning > shows the warning overlay for anonymous users when clicking the edit button
  • Termbox: AnonEditWarning > can be dismissed
  • Termbox: AnonEditWarning > does not show the warning overlay again if the user opts out
  • Termbox: AnonEditWarning > never appears for logged in users
  • Termbox: LicenseOverlay > is shown when clicking publish
  • Termbox: LicenseOverlay > disappears when clicking cancel and goes back to edit mode
  • Termbox: LicenseOverlay > disappears and saves when clicking publish
  • Termbox: LicenseOverlay > does not reappear after saving by default
  • Termbox: LicenseOverlay > reappears after saving when unchecking the "remember my choice" checkbox
  • Termbox: editing > edit mode > is in edit mode after clicking the edit button
  • Termbox: editing > edit mode > switches back to reading mode when clicking the cancel button
  • Termbox: editing > editing > can edit labels, descriptions, and aliases
  • Termbox: editing > editing > shows an error when an edit fails to save when the entity was protected while editing
  • Termbox: reading > is in reading mode when opening the item page
  • Termbox: reading > primary language terms > contains the expected language with respective terms
  • Termbox: reading > "in more languages" section > has a collapse/expand button
  • Termbox: reading > "in more languages" section > is expanded by default
  • Termbox: reading > "in more languages" section > is collapsible, also hiding the "all entered languages" section
  • Termbox: reading > "in more languages" section > expands again when clicking the button twice
  • Termbox: reading > "in more languages" section > contains the expected languages with respective terms
  • Termbox: reading > "all entered languages" section > is collapsed by default
  • Termbox: reading > "all entered languages" section > has a collapse/expand button
  • Termbox: reading > "all entered languages" section > is expandable
  • Termbox: reading > "all entered languages" section > collapses again when clicking the button twice
  • Termbox: reading > "all entered languages" section > contains the expected languages with respective terms

Team Norms

[edit]

Working norms for the team: How to contact, core working hours.

Communication channels:

[edit]

Slack: #talk-to-quality

IRC: #wikimedia-qte

Primary Communication tool:

[edit]

Slack

[edit]
  • Shared channel: #quality-engineering
  • Talk-to channel: #talk-to-quality

https://slack.com/help/articles/360025054173-Set-up-Slack-for-work-hours

Suggestion: Set up Slack for no notifications after working hours so it’s easy to see when someone’s outside of typical working hours.

During working hours

  • Reply to DMs/tagged conversations ASAP (even if it’s with an 👀react or something)
    • Within your core availability, will be responsive within 4 hours

Outside of working hours

  • Do not expect response if the person has marked themselves Out of Office, Sick, or has paused notifications

Team Meeting

[edit]

Occurs weekly

If you can’t attend:

  • Let Sean or Quality Engineering channel know
  • If it’s a recurring conflict, let Sean know so we can possibly figure something else out
  • If there’s something you’d like shared in your place: post it to #quality-engineering or let Sean know

Out of Office

[edit]

If you are going to be out of office, please try to let the team know so we can make sure to appropriately document/follow up with information that might be missed.

Since sick leave is often unplanned, a quick message in #quality-engineering or directly to Sean (or a teammate who can share on your behalf) is enough to make sure everyone is aware.

For planned vacations, submit your time off through the web portal and communicate with relevant stakeholders, including QS and DevX leadership.

See also

[edit]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Quality_Services&oldid=8313543"