User:Mooeypoo/VisualEditor Development

Below are the full steps, as I've experienced them multiple times, in setting up a working development environment for MediaWiki and VisualEditor.

Please note that this isn't an installation guide, this article is specifically meant to help contributors set up their system for contributions to MediaWiki core and VisualEditor.

Motivation
Everything in this guide already appears in the various guides online, either in GitHub or in MediaWiki documentation.

However, from my experience, it tends to get a bit confusing to shift from one guide to the other, especially if you're just starting out with this type of development. So, I grouped everything together into one single newbie-friendly tutorial. If you have anything to add to this, feel free.

Assumptions
This guide assumes the user is working with a Linux-based system. I will concentrate on Debian / Ubuntu because that's the most beginner-friendly distribution out there, but each step also has information about other distributions.

If your system is Windows, I recommend either using MediaWiki-Vagrant or install Linux under a Virtual Machine. VirtualBox or VMWare Player both work well in my experience. If you go with Vagrant, ignore this guide and follow the vagrant guide instead, it will handle setting up a development environment for MediaWiki and any extensions you set it up with.

So, in this document I assume that:
 * You work with a Debian/Ubuntu linux distribution.
 * You already have LAMP installed on your system (Apache/MySQL/PHP) or similar configuration that allows for PHP and MySQL.

Request a developer account
The first step in getting into MediaWiki development is to ask for a developer access. This will allow you to make contributions through gerrit.
 * 1) Request wikitech account

Note: The 'instance shell account name' is used in the "username" field whenever you contact gerrit to pull, clone, push, or review code. Whenever you see this link "ssh:// @gerrit.wikimedia.org:29418" the 'username' is your instance shell account name you chose. It's also public, so choose wisely. You have to sign up to both sites. Use the same email details.
 * 1) Sign up to Gerrit

Set up git
The best information about installing git is on GitHub documents. To save you the trouble, here it is in sequence for Debian / Ubuntu systems.

Install
$ apt-get install git For other distributions, follow the instructions at the git website.

Set up your details
These details should match the details you signed up to gerrit and wikitech $ git config --global user.name "Your Name Here" $ git config --global user.email "your_email@example.com Optionally, you can set up colors in your terminal. This is useful to see errors or to spot trailing spaces. $ git config --global color.ui true

Public keys
To be able to use ssh with gerrit, you must set up a public key gerrit can use to identify you are you who say you are. This procedure was taken almost-verbatim from the GitHub tutorial. $ cd ~/.ssh $ ls# Create a new ssh key, using the provided email as a label $ ssh-keygen -t rsa -C "your_email@example.com" Generating public/private rsa key pair. Enter file in which to save the key (/home/you/.ssh/id_rsa): $ ssh-add id_rsa
 * 1) List the files in your .ssh directory:
 * 1) Add the key in the system

Now you need to add the public key to both wikitech and gerrit. $ sudo apt-get install xclip $ xclip -sel clip < ~/.ssh/id_rsa.pub
 * 1) Downloads and installs xclip. If you don't have `apt-get`, you might need to use another installer (like `yum`)
 * 1) Copies the contents of the id_rsa.pub file to your clipboard


 * 1) Add to wikitech settings
 * 2) Add to gerrit settings

Install git-review
Next, we need to work with git review. This will allow us to send our work to gerrit so other developers can review it and (eventually) merge it into the codebase.

The following instructions were taken from this page. You can see the full instructions there, as well as troubleshooting help.

$ sudo apt-get install python-pip $ sudo easy_install pip $ sudo pip install git-review

Note:  Do not install git-review via apt. It looks like it works, but doesn't.

If you do not have easy_install set up, try

sudo apt-get install python-setuptools

Setting up MediaWiki
The next step is to install and set up MediaWiki from the development repository. It's also a good idea to set it up so you can test your code on a running local development wiki.

Clone the repository
In your terminal, go to wherever you want your new mediawiki folder to be located and proceed with the following steps: $ git clone ssh:// @gerrit.wikimedia.org:29418/mediawiki/core.git Where is the 'instance shell account name' you chose when you signed up to wikitech and is an optional entry for you to specify a folder for the code. If you don't specify the folder, the code will be cloned into a folder with the repo name -- in this case, a folder called 'core'

After you've done that, you should initiailize git-review on this folder. $ git review -s The system will likely ask you for your gerrit username. Again, this is the same 'instance shell account name'. Get used to it, that's your ssh username from now on.

Install MediaWiki
Assuming your system already has some web service, php and mysql support, you should now install mediawiki so you can have a local working system you can use to test with.


 * Create a MySQL database for your development wiki.

$ ln -s  /var/www/ The symbolic name is usually what you see in your browser url, like 'wiki' in http://localhost/wiki
 * Optional: In case you don't know or didn't do this before, the mediawiki folder must be visible in the public web folder. In Apache, that folder is usually /var/www, and you can add a symbolic link from your MediaWiki installation to this folder so your wiki can be accessed through the browser:


 * Navigate to http://localhost/wiki (or whatever other name you chose for your link) and follow the installation instructions to install MediaWiki.

Setting up Parsoid
In order to work with VisualEditor we will need to set up the parsoid service. These instructions are also available in the Parsoid installation page.

Clone the service
Parsoid isn't an extension, it is a service, so you don't have to clone the folder into the mediawiki/extensions directory. In your terminal, go to the folder you wish to clone parsoid to and clone the repository: $ git clone ssh:// @gerrit.wikimedia.org:29418/mediawiki/services/parsoid

Install NodeJS
This step is a bit tricky. You need to work with NodeJS version 0.8 or 0.10. Usually, the regular installation will work in getting the latest version, however, in Ubuntu 12.04 and up, the version that's offered is 0.6, which is not what we need. We need the recent version of node, so if you're running Ubuntu 12.04 and up, follow these instructions:

$ sudo apt-get update $ sudo apt-get install -y python-software-properties python g++ make $ sudo add-apt-repository ppa:chris-lea/node.js $ sudo apt-get update $ sudo apt-get install nodejs

Verify you have the correct version of node:

$ node --version

Get into the repository folder and run the following command to install all dependencies:

$ npm install

Configure parsoid
Now, you will need to configure your parsoid service. The full configuration options are available in the official setup page.

Go to the  directory and create a   file based on.

go to the  directory and create a   file based on. Make sure this url points to your local wiki

parsoidConfig.setInterwiki( 'localhost', 'http://localhost/wiki/api.php' );

Run the parsoid service
Parsoid is now installed and set up. Start the service with

$ node api/server.js

This will start the Parsoid HTTP service on port 8000. To test it, point your browser to http://localhost:8000/. If you configured the 'localhost' prefix correctly, you should be able to parse pages via http://localhost:8000/localhost/.

Setting up VisualEditor
We're now at the stage where we can set up VisualEditor for both development and testing environment. You can see a fully detailed explanation on how to set VisualEditor up in the official setup page.

Clone the extension
In your terminal, go to your mediawiki folder and into /extensions. The VisualEditor repository must be cloned into the mediawiki/extensions folder. $ git clone ssh:// @gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor.git

Activate the submodules $ git submodule update --init

Add to LocalSettings.php
Open your mediawiki's LocalSettings.php file. Add the following to the end:

Notice that if you're contributing code to VisualEditor, it might be a good idea to enable experimental features by uncommenting the line with

We also have to link VisualEditor with the parsoid service. Again, at the end of the LocalSettings.php file, add the following

Congratulations! You have a working environment with MediaWiki, VisualEditor and parsoid running.

Setting up grunt tests
VisualEditor contains 'grunt', a javascript task runner that includes tasks to verify the code. This is crucial - it allows you to spot errors (to code and to style) before you commit your work. To set up grunt in your new VisualEditor folder, run the following commands: $ npm install $ cd lib/ve $ npm instal Now go back to the main VisualEditor folder and test your environment.

Testing
Test your installation by going into your wiki. You should have "Edit" and "Edit Source" tabs. Click on "Edit" and make sure VisualEditor is loading properly. If it doesn't, or if there is only one "Edit" tab, please see the VisualEditor installation page for troubleshooting. Make sure parsoid's settings are correct and that the service is running.

Make sure to test grunt by typing $ grunt test From your extensions/VisualEditor folder.

Useful Tools
Your system is ready for development now. There are several tools that can help you start (and continue) working more easily.

gerrit reference
There's a very good gerrit reference here. Here's the quickie on how to work with gerrit:

To start a new patch, make sure your repository is updated by checking out master and pulling: $ git checkout master $ git pull $ git submodule updat Then, start a new branch for your code: $ git checkout -b [BRANCH NAME] # See tips about branch name in the tutorial Change your files, do what you do, then when you are done, check the changes you've made $ git status You should see a list of changed files.

If you see 'lib/ve' in your list of changed files, do not do 'git commit --all'. Instead, add individual files to the commit manually: $ git add  And then commit without --all $ git commit

The commit note
Write a meaningful, helpful and explanatory commit note:
 * The first line in the commit message is the topic. Make it short and meaningful.
 * The commit message is separated by an empty line from the topic, and should explain the purpose of your fix and the point of your code. Think of other reviewers, explain to them what you did in your code and why so they have an easier time reviewing it.
 * If your fix is still under work and is not ready to be merged, start your topic with "[WIP]" (work in progress).

First commit in the branch
$ git commit --all NOTE: Be careful. Since VisualEditor uses a  submodule, it may sometimes be marked as changed even though you didn't change it.

Amending a previous commit
If the commit already exists -- either because you're fixing up some existing branch or reviewing one from gerrit, you should make sure to amend the commit $ git commit --all --amend Note again that  should only be there if there's no changes to lib/ve

Useful shortcuts
You'll work in the terminal quite a lot, so terminal shortcuts are very useful. Here are a couple that I am using. In order to set up shortcuts in the terminal, edit your  file. You can add your own custom aliases and shortcuts at the end of it.

Note: Some of these are shamelessly stolen and adapted from Krinkle's aliases, so thank him when you catch him online.

Shortcut: update and activate parsoid
You can choose your own alias name, of course. This will update the Parsoid repository and run the service. It saves a lot of time instead of going to the parsoid folder and running  every time. It also has the added benefit of updating parsoid to its latest 'master'.

Shortcut: update MediaWiki and VisualEditor
I use this every time I want to completely refresh my development system and make sure both MediaWiki and VisualEditor are up to date with the latest versions of the master branch. Note: The new aliases won't work until you reload

You can either close and reopen your terminal or run $ reload ~/.bashrc

Useful Links

 * MediaWiki Developer's Guide
 * Getting started with gerrit
 * VisualEditor extension
 * Parsoid