XTools/Development

From mediawiki.org

Thanks for contributing to XTools! This guide gives a high-level walkthrough of everything you need to contribute to XTools in a local environment.

Prerequisites[edit]

Running a development server[edit]

  • Clone the repository: git clone https://github.com/x-tools/xtools.git && cd xtools
  • Install PHP dependencies: composer install
  • Run cp .env .env.local and fill in the missing values, using /Configuration as a guide.
  • npm install to install Node dependencies
  • Assets are generated with Symfony Encore. If you're working on JavaScript, CSS, or adding images, compile the assets with: npm run watch.
    • Before submitting your pull request, npm run build to build the production assets.[1]
  • Launch Symfony's built-in server: symfony serve
  • Assuming the configuration was done correctly and as desired, you should be up and running at http://localhost:8000

The Simple Counter is the simplest tool and should work as soon as you set up XTools. Test it by going to http://localhost:8000/sc and put in Jimbo Wales as the Username and en.wikipedia.org as the wiki. After submitting you should quickly get results.

The development server does not cache application data; any changes you make are visible after refreshing the page. However when you modify things in .env or otherwise aren't seeing changes you're making, you may need to clear the cache with symfony console cache:clear.

The logs are in var/logs/dev.log. If things are acting up unexpectedly, you might try clearing the cache or restarting the server.

Developing against Toolforge replicas[edit]

First make sure you have a Wikimedia developer account and access to Toolforge. Then you can set up the necessary tunnels and a shell session with:

ssh -L 4711:s1.web.db.svc.wikimedia.cloud:3306 -L 4712:s2.web.db.svc.wikimedia.cloud:3306 -L 4713:s3.web.db.svc.wikimedia.cloud:3306 -L 4714:s4.web.db.svc.wikimedia.cloud:3306 -L 4715:s5.web.db.svc.wikimedia.cloud:3306 -L 4716:s6.web.db.svc.wikimedia.cloud:3306 -L 4717:s7.web.db.svc.wikimedia.cloud:3306 -L 4718:s8.web.db.svc.wikimedia.cloud:3306 -L 4720:tools.db.svc.eqiad.wmflabs:3306 username@login.toolforge.org

Replace username with your Toolforge Unix shell username. Note our tunnel has to connect to each database slice. The ports used here should match up with the corresponding options in your .env, like so:

DATABASE_REPLICA_USER=<your-username>
DATABASE_REPLICA_PASSWORD=<your-password>
DATABASE_REPLICA_HOST_S1=127.0.0.1
DATABASE_REPLICA_PORT_S1=4711
DATABASE_REPLICA_HOST_S2=127.0.0.1
DATABASE_REPLICA_PORT_S2=4712
DATABASE_REPLICA_HOST_S3=127.0.0.1
DATABASE_REPLICA_PORT_S3=4713
DATABASE_REPLICA_HOST_S4=127.0.0.1
DATABASE_REPLICA_PORT_S4=4714
DATABASE_REPLICA_HOST_S5=127.0.0.1
DATABASE_REPLICA_PORT_S5=4715
DATABASE_REPLICA_HOST_S6=127.0.0.1
DATABASE_REPLICA_PORT_S6=4716
DATABASE_REPLICA_HOST_S7=127.0.0.1
DATABASE_REPLICA_PORT_S7=4717
DATABASE_REPLICA_HOST_S8=127.0.0.1
DATABASE_REPLICA_PORT_S8=4718
DATABASE_TOOLSDB_HOST=127.0.0.1
DATABASE_TOOLSDB_PORT=4720

Change the your-* bits to your own values, which you can find in your replica.my.cnf file in the home directory of your account on Toolforge.

Caching[edit]

XTools should probably take advantage of Doctrine's built-in caching mechanism, but it doesn't. Instead it is done in a somewhat manual fashion. This is only done in Repository classes, using this pattern:

public function doSomething($input)
{
    $cacheKey = $this->getCacheKey(func_get_args(), 'component_method');
    if ($this->cache->hasItem($cacheKey)) {
        return $this->cache->getItem($cacheKey)->get();
    }
    $results = 'results of big query';
    return $this->setCache($cacheKey, $results);
}

The cache key can be anything, so long as it is unique to the specific method. A third parameter can be passed to setCache to set the TTL, using the same syntax from the DateInterval class (e.g. P1D is one day, PT1H is one hour).

The above methods are just wrappers around a PSR-6 implementation, intended to reduce the repetition of similar lines of code.

Style guidelines[edit]

  • It's called "XTools", with two capital letters.
  • XTools conforms to PSR2 and Slevomat coding standards; use ./vendor/bin/phpcs -s -p . to check your code.
  • HTML routes must begin with the tool name, i.e. EditCounterYearCounts, while API routes begin with either Project, User, or Page.
  • Version numbers follow Semantic Versioning guidelines.

Tests[edit]

Tests are located in the tests/ directory, and match the src/ directory structure. They are built with PHPUnit. Repositories only handle fetching data and do not need to be tested. Controllers also interact with the database, and while tests are most welcomed for these, they will not run on the CI server (GitHub Actions or Scrutinizer) due to limitations. Instead, put all things that query the replicas behind a app.is_wmf check (example).

There are also tests for linting, phpDoc blocks, and file permissions.

Use composer test to run the full suite, or ./vendor/bin/phpunit tests/ to run just the unit tests.

Releases[edit]

Releases are made by tagging commits in the master branch. Before tagging a new release:

  • Update the APP_VERSION in the .env file
  • Check the copyright year in README.md
  • Update RELEASE_NOTES.md with any notable new information for the end user.

Then tag the release (follow the Semantic Versioning guidelines, and annotate the tag with the above release notes) and push it to GitHub.

Lastly, update the version and updated parameters at XTools.

Additional help[edit]

  • Email: tools.xtools @ toolforge.org
  • IRC: ##wikimedia-xtools connect
  • MediaWiki talk page: Talk:XTools

Notes[edit]

  1. We commit the assets to avoid having to build them on the production servers. Nothing in the public/build/ directory should be changed directly, including images. All such changes go in the assets/ directory then are compiled and/or copied to the build directory with Encore.