Wikibase/Programmer's guide to Wikibase

This page describes the workflow for developing features for Wikibase.

General overview
We have considerably simplified the development process. Basically, whenever you develop something — develop a complete new feature or just make a small fix — you create a new branch. Whenever it makes sense, you squash the changes into one commit, rebase it on the current master, and submit that commit for review on Gerrit.

Once the changes are on gerrit, they need to be reviewed and approved for merging by another developer.

So, to keep things moving, you should review at least as many changes as you submit for review!

If possible, try to follow Git/Commit message guidelines.

Making Changes
As a general rule, we'll try to follow the workflow established by the Wikimedia Foundation, as documented on Git/Workflow.

For a smallish change (basically, anything that you would develop by yourself on a single day), use the following workflow:

Preparing your Patch
Make sure you are on "master" and create the branch from there git status
 * 1) On branch master

Create your feature branch locally git checkout -b feature_branch_xyzzy

Now you can work on that branch until your feature is ready for review. Note the gerrit likes to have a single commit on your feature branch and will complain that there are more. So, if you have already committed to the branch and want to change something later, use  to amend your original commit instead of creating a new one.

While you are working on your patch, make sure to add unit tests for any new classes or functions to introduce, and to update unit tests in places where you modify existing code. Run the Wikibase test group fequently while you are working on the code:

cd tests/phpunit php phpunit.php --group Wikibase

Before submitting your patch for review, test it thoroughly. It should apply cleanly to master, and it should pass all unit tests.

HINT: committed to the wrong branch?
It sometimes happens that you started to work on a change, and committed it, but forget to create a feature branch first. So you end up committing to master (or some other branch you were working on). This can easily be fixed.

First, write down the id (the hash) of your commit. It's shown when you commit, and you can see it using.

Then, reset the "wrong" branch, removing the commit from it:

git reset --hard HEAD^


 * Note: for two commits, use  or
 * Note: your commit is now "detached" from any branch. You can still find it using.

Now, create your feature branch as described above:

git checkout master git checkout -b feature-thingy

...and finally apply your commit to the new branch:

git cherry-pick 

that's it, your commit is now on the feature branch and no longer on the other branch, and you are ready to submit it for review!

Submitting your Patch
So, rebase to the latest master, and resolve any conflicts:

git rebase origin/master

Now, run all mediawiki unit test to make sure your changes didn't break anything (yes, even if your were only working on an extension). Make sure you have the extension in question enabled in your LocalSettings.php. Then run:

cd tests/phpunit php phpunit.php --group Wikibase

If this does not report any errors, you are now good to submit the test for review.


 * Caveat: if phpunit reports timeout errors, open tests/phpunit/suite.xml and change the "strict" setting to false. Make sure to not commit this change, though.

In order to submit your patch for review, first commit it to your branch, if you havn't already. Make sure to use a descriptive commit message. The first line of the commit message should by itself already describe the change, it will be used for the subject line of emails, etc. So, commit the change (using --amend if it's not your first commit on the branch). Make sure you first call  for all files you modified or added:

git add foo.php git add bar.php git commit


 * Caveat: commit -a will automatically include all modified files, but not new files!

Finally, send the change to gerrit for review:

git review

You need to have git-review installed for this (see Git/Workflow for instructions). And for every repository you want to use git-review with, run  once, to set it up for that repository.

Fixing your Patch
Often, reviewers will complain about some aspect of your patch, and ask you to improve it. To do that, go back to your feature branch:

git checkout feature_branch_xyzzy

or, if you don't have that around, fetch the change from gerrit

git review -d

This will automatically create the appropriate branch.

Now you can fix whatever the reviewers complained about. When you are done, amend your original commit (do not create a new commit). Again, don't forget to  anythign you changed:

git add foo.php git add bar.php git commit --amend

Please add a note about your changes at the bottom of your original commit message, but above the Change-id line. That line is magic automatically injected by git-review and must always remain the last line in the commit message!

Once you have done this, send the patch to gerrit again, but rember to use the -R option with git-review, so you don't clutter your original change with merges caused by a rebase:

git review -R

Repeat, until a reviewer has approved, verified and merged your change (see below).

Collaboration and Dependent Changes
In some cases, you may want to work on a feature that depends on some other change that was already submitted for review, but that has not been merged into master yet. You can do that by simply downloading that change (which will put you on the corresponding feature branch automatically), make your change and submit it as normal. Gerrit will detect that your change was done on a feature branch, and depends on the earlier change. That dependency will also be visible on Gerrits web interface.

This kind of dependency allows for multiple developers to collaborate on a feature, building changes on top of each others changes, without having to wait for each change to be reviewed and merged.

So, for example, let's say you have the following situation:


 * You want to add feature Y, which depends on feature X by Hans which was submitted as change 1234, but has not yet been merged into master.

So, you donwload that change:

git review -d 1234

This will put you on the respective feature branch, which will look something like:

review/hans/feature-X

Now, make your changes as normal, and finally commit and push them for review:

git commit git review

Git will create a new change, say 1245, and mark it as depending on 1234. It will not be possible to merge 1245 before 1234 is merged.

Note: git review will warn that you are about to submit multiple changes when you try to submit a dependent change. You can ignore this, as long as all of the changes in the list have already been submitted for review, or you actually want to submit all of them for review at once. Would be nice if gerrit would tell us which of the commits on the branch were already uploaded as changes...

Amending Changes Other Changes Depend On
Say you have two changes (let's call them A and B, with the commit hashes A1 and B1) where B depends on A, as described above (so, A1 is B1's parent commit). Now A needs to be amended for some reason, creating A2. In this situation, B becomes obsolete, because B1 depends on the obsolete commit A1. To fix this, do the following:


 * 1) download change B: , where B is B's change ID.
 * 2) rebase B1 on A2, creating B2: , where A2 is A2's commit hash.
 * 3) resolve any conflicts, use   to continue the rebase.
 * Hint: if git rebase tells you that there are no changes after you resolved some conflicts, use  to resolve this.
 * 1) use   to verify that the last commit in the log is still B, and the commit message still contains B's change ID.

Now the situation is resolved as far as git is concerned, but we still have to tell gerrit. So now we submit B2 to gerrit for review:

Gerrit should now figure out what you have done, showing a new change set B2 for B, and mark B as no longer obsolete, but still depending on A.

If you have more changes (say, C, D, and E) that depend on B, you have to repeat the procedure rebaseing C on B, D on C, and so on.

Workflow for Reviewing Changes
There are two parts to reviewing a change: approval and verification. Approval is conceptual and formal agreement, while verification is about technical compliance.

Approval
To approve a change, visit it's page on gerrit and look through the diffs (note the convenient "next" and "previous" links on the diff pages). Try to understand the intend and implementation of the change, and check the coding style.

The most important part of course is that the new code is sensible conceptually and that it is implemented correctly. But there are some more things that should be considered:


 * is the code internationalized? Are system messages used where they should be?
 * does the code follow the code conventions with respect to naming, indentation, spacing, etc?
 * is the code thoroughly documented on the class and method level?
 * does the code come with the appropriate unit (and/or integration) tests?
 * Is the code efficient, does it make use of caching where appropriate?

If you have questions or find anything objectionable, click inside the diff to leave an inline comment. Note that your inline comments will be saved as drafts but not directly published - this also applied to replies to comments! You need to actually post a review verdict (below) to publish your comments.

It's also useful to actually try the code you are reviewing. To do that, first download the change into a local branch:

git review -d

...where   is the (decimal) change number from the gerrit URL or the (hex) change id given on the gerrit page. This will create a local branch for the change set.

Now, try the new functionality (or whatever the change provides) at least superficially.

After you are finished your review, you should post your verdict by pressing the review button. Select the appropriate level of approval (between -2 and +2), add a comment if you like, and hit "Publish Comments". If you just want to publish your inline comments (or replies), leave the rating at 0.

Unless you have also done verification, leave the verification section untouched.

If you think the change can be merged, proceed to the merging section.

Verification
Verification makes sure that the change is technically complied, that is, it


 * 1) applies cleanly to the latest master
 * 2) passes all tests it provides
 * 3) causes no other tests to fail

This verification would ideally done automatically by Jenkins (see our request on Bugzilla). For the moment however, we are stuck doing this manually. So, here are the steps:

First, download the change into a local branch

git review -d

...where   is the (decimal) change number from the gerrit URL or the (hex) change id given on the gerrit page. This will create a local branch for the change set.

Next, try to merge the latest version of the master branch into the change's branch by doing:

git pull origin master


 * Note: technically, we want to check whether the change can be merged into master, not vice versa. But since 3-way-merge is a commutative operation, the resulting source code will be the same either way.

If there are conflicts, the change should be rejected (set to -1) as broken.

Testing
There are three different frameworks Also note the page at Requests for comment/Unit testing.
 * Selenium tests (Selenium Framework, old page)
 * QUnit tests (mw:Manual:JavaScript unit testing)
 * PHPunit tests (mw:Manual:PHP unit testing)

Wikidata Test Server
The Wikidata test server runs all the mentioned tests and takes care of other QA related jobs. It's jobs are documented here.

Prerequisites

 * Install Ruby on your local machine
 * For Linux, the easiest way is to use RVM, as described on https://rvm.io \curl -L https://get.rvm.io | bash -s stable --ruby Alternatively you can install Ruby by running sudo apt-get install ruby2.1
 * For Windows, use the installer from http://rubyinstaller.org/downloads. IMPORTANT: Use the latest 1.9 version, NOT the 2.0 version! 2.0 is not compatible with Cucumber under Windows yet!
 * On MacOS Ruby usually is included. You can also use homebrew to easily get Ruby by running brew install ruby
 * Further helpful information can be found on https://www.ruby-lang.org/en/downloads/
 * Update RubyGems
 * gem update --system
 * Install Bundler
 * gem install bundler

Install required packages

 * Checkout the Wikibase repository
 * cd tests/browser
 * bundle install
 * Make sure you have the latest version of Firefox installed

Setup test configuration
You can specify your configuration in two ways. You can either set the environment variables yourself or you use a configuration file. If you use a file you need to copy config/config.yml.sample to config/config.yml and adjust config.yml to your needs. Mandatory settings are: WIKIDATA_REPO_URL WIKIDATA_REPO_API WB_REPO_USERNAME WB_REPO_PASSWORD ITEM_NAMESPACE PROPERTY_NAMESPACE ITEM_ID_PREFIX PROPERTY_ID_PREFIX LANGUAGE LANGUAGE_CODE LANGUAGE_NAME BROWSER If you want to test using SauceLabs, you have to add the following settings: SAUCE_ONDEMAND_USERNAME SAUCE_ONDEMAND_ACCESS_KEY BROWSER VERSION PLATFORM

Running tests
Run all tests: bundle exec cucumber

You can also just run a specific feature by specifying the feature-file. e.g. run the label tests: bundle exec cucumber features/label.feature

You can also just run a specific scenario of a feature by specifying the line-number of that scenario inside the feature-file. e.g. run test which checks the label UI elements: bundle exec cucumber features/label.feature:14

You can also run scenarios with a specific tag. e.g. run tests which are tagged with @ui_only: bundle exec cucumber features/label.feature --tag @ui_only

Some scenarios need a valid login given in the environment variables WB_REPO_USERNAME and WB_REPO_PASSWORD:

WB_REPO_USERNAME=user WB_REPO_PASSWORD=password

Using a specific browser
By default your browser tests will use Firefox. If you want them to run in a different browser you can specify that by setting an environment variable. Linux/Unix: export BROWSER=chrome Windows: set BROWSER=chrome For browsers other than Firefox you would need to install an appropriate driver, e.g. chromedriver for Chrome. See the following links for details:
 * http://code.google.com/p/selenium/wiki/OperaDriver
 * http://code.google.com/p/selenium/wiki/InternetExplorerDriver
 * http://code.google.com/p/selenium/wiki/SafariDriver
 * http://code.google.com/p/selenium/wiki/ChromeDriver

Run the tests in the cloud
For testing different OS/Browser combinations it makes sense to run the tests on an external service. Saucelabs offers this possibility and you'll just need a few commands to run your tests in their cloud. You'll have to set your Saucelabs username and accesskey: Linux/Unix: export SAUCE_ONDEMAND_USERNAME=myusername export SAUCE_ONDEMAND_ACCESS_KEY=myaccesskey Windows: set SAUCE_ONDEMAND_USERNAME=myusername set SAUCE_ONDEMAND_ACCESS_KEY=myaccesskey Make sure you point to a public accessable Wikibase instance in your config/config.yml! Do NOT use the live site for testing! Now you can start your tests as described in the examples above and watch them running in your Saucelabs account.

Run tests in parallel
You can run features in parallel with the parallel_test gem. If you run bundle update the gem will be already installed. You can then run your features in parallel by doing: bundle exec parallel_cucumber features/ By default it will run as many features in parallel as your machine has CPUs. You can specify the number with -n. This will run e.g. 3 features in parallel: bundle exec parallel_cucumber features/ -n 3 If you want to pass options to underlying cucumber, you can pass them as string with -o. This will run the tests that have the @smoke tag and will skip tests that have the @skip tag: bundle exec parallel_cucumber features/ -o '--tags @smoke --tags ~@skip'

Run tests headless
If you're on a Linux machine you can run your tests on a headless browser. That makes them slightly faster and you wont have annoying browser windows popping up. The only prerequisite is that you have Xvfb installed: apt-get install xvfb If you want to run your tests headless, all you have to do, is setting the appropriate environment variable: export HEADLESS=true And then run your tests as usual.

QUnit tests
Before being able to run QUnit tests, they have to be enabled in the LocalSettings.php by adding: $wgEnableJavaScriptTest = true;

As of now, Wikibase specific QUnit tests only exist for the repo extension. To run all QUnit tests simply point your browser to the following URL: http://localhost//index.php/Special:JavaScriptTest/qunit

You might want to run the Wikibase tests exclusively which can be achieved by applying a filter: http://localhost//index.php/Special:JavaScriptTest/qunit?filter=wikibase

Run QUnit tests
As of now, we only have QUnit tests for the Wikibase repo extension. Let them run by pointing your browser to the following URL:

http://localhost//index.php/Special:JavaScriptTest/qunit?filter=wikibase

Having done changes to core, you may rather run all QUnit tests by not applying a filter:

http://localhost//index.php/Special:JavaScriptTest/qunit

Installing on Ubuntu
Install PEAR if its not already installed, that makes everything a lot easier. On an Ubuntu box this is done with a sudo apt-get install php-pear or you could do a wget http://pear.php.net/go-pear.phar php go-pear.phar

After its installed do a sudo pear upgrade pear

Then your at mw:Manual:PHP unit testing/Installing PHPUnit. Only difference is that you probably should use --alldeps to save some griefs sudo pear install --alldeps phpunit/PHPUnit

To be able to run tests I had to set absolute paths in LocalSettings.php, this could perhaps be better solved in suite.xml

Run PHPUnit tests
The the merge was successful, next try running the Wikibase unit tests:

cd tests/phpunit php phpunit.php --group Wikibase

This is just for finding any obvious breakage early on. If this passes, try running all unit tests:

php phpunit.php

If any tests fail, the change should be rejected (set to -1) as broken.

To get rid of failing tests outside our groups, for example all kinds of failures from testing with SqLite which probably are not common here, try --exclude-group sqlite

php phpunit.php --exclude-group sqlite

MediaWiki provides a custom wrapper for the standard phpunit command, which is located in tests/phpunit/phpunit.php. It supports all command line options and parameters the original phpunit command does (plus a few arcane ones we don't need).

To run all tests, including tests for any extension you have configured in LocalSettings.php:

cd tests/phpunit php phpunit.php

To run only tests for the Wikibase group (provided you have the Wikibase extensions configured in your LocalSettings.php):

php phpunit.php --group Wikibase

To run one specific test class:

php phpunit.php ../../extensions/Wikibase/repo/tests/phpunit/includes/api/ApiGetItemId.php

To run one specific test function: php phpunit.php --filter testMyThing

(note that ---filter actually takes a regular expression).

Advanced Options and Configuration
In order to catch more problems when running tests, enable strict error reporting in your LocalSettings.php:

error_reporting( E_ALL | E_STRICT ); ini_set( 'display_errors', 1 ); $wgShowExceptionDetails = true; $wgShowSQLErrors = true; $wgDebugDumpSql = true; $wgShowDBErrorBacktrace = true;

There are several things you may want to change in test/phpunit/suite.xml:

Turning off verbose output, removing all the annoying details about skipped and incomplete tests:

verbose="false"

Disabling test timeouts (and other strict checks):

strict="false"

Or increasing the timeout:

timeoutForSmallTests="8"

Note: make sure never to check in your modified suite.xml! you could also place your modified suit.xml in a different location and tell phpunit where to find it:

php phpunit.php --configuration /path/to/phpunit/suite.xml

For testing different setups (e.g. different database engines), you can tell phpunit to load a different LocalSettings.php file:

php phpunit.php --conf /path/to/server/LocalSettings.php

Writing PHPUnit Tests
Test case classes should use the following code skeleton:

/** * ...  *  * @ingroup Wikibase * @ingroup Test * * @group Wikibase * @group Stuff * * ...  */ class MyStuffTest extends MediaWikiTestCase { function testFoo { ...    } }

The important bits are:


 * extend MediaWikiTestCase
 * use a class name (and matching file name) that ends in "Test".
 * use method names that start with "test" for actual test functions. You can have other (helper) functions that don't use that prefix.
 * use @ingroup in the class comment to indicate to doxygen into which group documentation about this class should go. "Wikibase" and "Test" should be there.
 * use @group in the class comment to indicate to phpunit to which test group this test belongs. "Wikibase" should be there.

There are some special groups that trigger special behavior if you apply them:


 * @group Database causes phpunit to set up temporary database tables for use by the test, so that modifications performed by the test are not visible in the actual wiki database. This must be done for all tests that need the database, because Jenkins will run tests without @group Database without a valid database connection.
 * @group medium causes phpunit to consider the test to be "medium heavy" instead of the default "small". This will apply a greater timeout to the test when phpunit runs in strict mode.

Registering Extension Test Cases
Update the registerUnitTests method in the Wikibase.hooks.php file (resp. WikibaseLib.hooks.php or WikibaseClient.hooks.php) with file paths to the individual test files. This is all that is necessary to get it up and running. To test this from the command line use

php phpunit.php --group Wikibase

When everything is in place on a central test server it should be possible to run tests for the Wikidata extension just like any other tests.

See also mw:Manual:PHP unit testing/Writing unit tests for extensions

File structure
Test cases should be placed in the tests/phpunit directory under the extension's directory. Below that, follow the directory structure used by the php files under test: if the file to test is in includes/api, put the test case in tests/phpunit/includes/api, and so on.

Post your verdict
To post your verdict, press the review button on the change's page on gerrit. Select the appropriate level of approval (-1 if it failed or +1 if all is well), add a comment if you like, and hit "Publish Comments". Leave the approval rating as it is, unless you also went through the approval process.

If you think the change can be merged, proceed to the merging section.

Merging
Once the change is approved (+2) and verified (+1), it can be merged in to the main line's master branch. To do so, click the submit change button on the change's gerrit page. Gerrit will then post a message containing the result of the merge attempt. Either the change gets merged, or the merge failed for some reason.

If the merge failed, please use the Review button to change the verified level to -1 (broken).

Resources

 * Michael Hunter's You Are Not Done Yet (pdf), a comprehensive checklist of what can and should be checked. It seems to be aimed at Windows based desktop applications, but many points still apply to web applications.