User:Etonkovidova (WMF)/GrowthExperiments

Documentation
General info/documentation is here - Growth team projects.

Growth team projects (GrowthExperiments) are run on target wikis and the projects aim to help new users to start editing. Below is the list of participating wikis that also exist in betalabs:


 * Betalabs: cswiki, kowiki, arwiki, viwiki


 * Production: cswiki, kowik i, arwiki, viwiki, testwiki

General testing notes
Start testing with creating a new user account on participating wikis (see the list above, or use the updated list on - Growth team projects.

For a new account on participating wikis, GrowthExperiments projects - Welcome survey/Homepage/Help panel - might be enabled  by default. However, due to variant testing (which gets adjusted based on the data analysis) the features might not be enabled by default, in this case they might be enabled manually (see below in related sections) which is also useful when testing is done for existing user accounts.

Refer to QA column on phabricator Growth Team Current sprint workboard to see which phabricator tasks need QA verification.

Pre-deployment testing
- the testing is done on betalabs env

- review QA column on Growth Team Current sprint workboard to finish QA verification before features would be deployed

Post-deployment testing
- check the GrowthExperiments features functionality on testwiki (posting Help panel questions, submitting edits can be done there)

- check logstash for errors in the following extensions- GrowthExperiments, Flow, Echo (these extensions are supported by Growth team).

Welcome survey
Welcome survey documentation

- create a new account

- for an existing account, add   to url to see Welcome survey UI

Homepage
Growth - Newcomer homepage

- new users should be redirected to automatically enabled Special:Homepage after Welcome Survey

- for existing accounts: enable Special:Homepage from Special:Preferences in User profile tab -> Newcomer homepage option

- check the presence and functionality of the following modules on Special:Homepag


 * Intro module
 * Mentorship module
 * Help module
 * Impact module
 * Suggested Edits (SE) module

Help panel
Growth- Help panel

- new users should have enabled Help panel when they click on Edit; Help panel is present in reading mode in some namespaces

- for existing accounts: enable the option “Enable the editor help panel”  on Special:Preferences-Editing tab


 * On participating wikis, visit User talk or “User  page - the Help panel should be present
 * Visit any article, click on Edit - the Help panel should be present

You can post test questions and check the whole functionality of Help panel on betalabs wikis and testwiki.

Guidance panel
Guidance panel appear for users who click on suggested article in Suggested edit module on Special:Homepage.

- Guidance panel is present in reading mode on suggested edit articles

- Guidance panel has post-edit dialog box for users who publish their edits.

Instrumentation testing
Several schemas are associated with Newcomer experience and Homepage features - HomepageModule, HelpPanel, EditorJourney, and EditAttemptStep. The following steps are applicable to all of the schemes checking (substitute the required schema name)

Before testing
- do some reading about event logging and possible updates/changed access etc - Analytics/Systems/EventLogging/TestingOnBetaCluster

- read the schemas description to understand what actions to perform to trigger eventlogging events

- create a test outline to make sure that all events would be tested for proper recording

In the browser
The testing with the console snippets can be done in beta cluster and in production.

- enable your browser's devtools and  enable "persistent log" in the Console

- use the following snippet (credit to User:KHarlan (WMF)

- perform the actions that trigger eventlogging and compare with the info documented in the relevant schema description

Server side
The instructions are for beta labs cluster only.

- log in and go to the directory with logs:

- to verify that events have been properly logged you may use  or any other means of your choice. For example, to see which editor_interface were recorded, you may use:

Feature flag testing
Growth team explores new approaches to engage new users in editing. Often features deployed either as A/B testing (or testing of groups of users in differently proportional segments) or the features have a feature flag to test them in production environment before turning on for everyone. Aside from regular QA workflow for feature testing, the following might be useful to keep in mind when using either javascript snippets or URL parameters:

- different variants of features (e.g. instead of creating new users every time you need to take a look at different UI layouts)

- re-test features that are presented only one one-time (or limites times) to new users (e.g. Welcome survey)

Please refer to Growth team projects page and to the Growth team for current projects testing needs.