MediaWiki-Vagrant

From MediaWiki.org
(Redirected from Mediawiki-vagrant)
Jump to: navigation, search

Other languages:
Deutsch • ‎Ελληνικά • ‎English • ‎British English • ‎español • ‎français • ‎italiano • ‎日本語 • ‎polski • ‎português • ‎português do Brasil • ‎русский • ‎中文
MediaWiki Vagrant logo
Bryan Davis explains in an interview at Wikimania what MediaWiki Vagrant is

MediaWiki-Vagrant is a portable MediaWiki development environment. It consists of a set of configuration scripts for Vagrant and VirtualBox that automate the creation of a virtual machine that runs MediaWiki.

The virtual machine that MediaWiki-Vagrant creates makes it easy to learn about, modify, and improve MediaWiki's code: useful debugging information is displayed by default, and various developer tools are set up specifically for inspecting and interacting with MediaWiki code, including a powerful debugger and an interactive interpreter. Best of all, because the configuration is automated and contained in a virtual environment, mistakes are easy to undo.

Quick start[edit | edit source]

If you are installing MediaWiki-Vagrant from the USB distribution, follow the steps in the README rather than the first four steps.

  1. Get Git
  2. Get the latest VirtualBox[1]
  3. Get the latest Vagrant (the version must be 1.2.6 or higher; For NFS, use 1.5.4 and up)
  4. Get the code and create your machine:
    $ git clone https://gerrit.wikimedia.org/r/mediawiki/vagrant
    $ cd vagrant
    $ git submodule update --init --recursive
    $ ./setup.sh  # or setup.bat on Windows
    $ vagrant up
    

    Note Note: It may take quite some time before the first vagrant up completes; e.g., 1 hour.

    Note Note: MediaWiki-Vagrant will not run on a host that doesn't support VT-X because it specifies a 64bit guest. The 64bit guest is needed in part because it uses use deb packages from WMF production that are only built for amd64 architecture.

If prompted, enter your Gerrit user name (recommended), or just hit Enter. When Vagrant is done configuring your machine, browse to http://127.0.0.1:8080/ to find your MediaWiki instance. The admin password is vagrant.

Troubleshooting startup[edit | edit source]

Any host[edit | edit source]

  • Be sure to use the most recent versions of VirtualBox and Vagrant if you get errors with vagrant up. Linux repositories may provide older versions that are not compatible with MediaWiki-Vagrant. (If you're running Debian, try apt-get install virtualbox vagrant.)
  • If you got any puppet errors you might need to init puppet submodules, on vagrant directory run git submodule update --init
  • Use http://127.0.0.1:8080/info.php to check that Apache/PHP is up and running.
  • You might want to compare the output of the initial run of vagrant up in your terminal with this sample. The initial setup may take a long time; if it seems to hang somewhere but there are no errors, just give it a while.
  • Vagrant rarely loses pairing with your VM. This thread discusses some ways to work around it (e.g. attaching old VM harddrive to the new profile)
  • Make sure that vagrant/mediawiki repository is up to date:
$ cd vagrant/mediawiki
$ git pull

Windows-specific[edit | edit source]

  • If you are on Windows and you get "guest machine entered an invalid state" - "poweroff", try downloading a 4.3.15 build of VirtualBox (there is a known problem with 4.3.14 on Windows). If that does not help, make sure you enable Hardware Virtualization Technology (VT-x or AMD-V) in BIOS. Hardware Virtualization is required. It is not an optional performance enhancement. (Some laptops require you to remove the power cable and battery for 30 seconds [1])

NFS errors[edit | edit source]

  • Mac-specific To avoid NFS errors, while installing ensure the firewall will accept connections: Apple > System Preferences > Security & Privacy > Firewall > Firewall Options. You will need to UNcheck "Block all incoming connections" and probably also UNcheck "Enable stealth mode" in order to accept the following: netbiosd, nfsd, rpc.lockd, rpc.rquotad, rpcbind, VBoxHeadless. Note, you may need to restart your computer and change the status to "Allow incoming connections" during a couple vagrant ups. After installation, you may be able to re-check "Block all incoming connections and "Enable stealth mode" now that the firewall rules have been updated.
  • Ubutu/Debian MediaWiki-Vagrant uses NFS to share some folders with the host machine (your computer). You need to set up your computer as an "NFS server", see e.g. Ubuntu instructions. On Debian, apt-get install nfs-kernel-server will work; you may need to modprobe nfsv3 as well. Note that Debian's NFS server will not start without an entry in /etc/exports. If sudo rpcinfo -p doesn't show "nfs" services running, this is likely what's going on. Adding your home directory as the last line in /etc/exports and then /etc/init.d/nfs-kernel-server restart will usually be good enough to get you past this chicken-and-egg problem.

Ubuntu[edit | edit source]

  • Vagrant cannot be run from an encrypted directory especially if you are running on Ubuntu and setting for an encrypted home directory. To run vagrant you need to move the MediaWiki-Vagrant's directory on "/opt/" then simply launch vagrant up.

Basic usage[edit | edit source]

Screenshot

The vagrant command-line tool on the host machine provides several subcommands for controlling your virtual machine. You've already used one: vagrant up, which turns on the virtual machine. Like most vagrant subcommands, you need to run it from the MediaWiki-Vagrant directory or one of its children. When you first run it, Vagrant will fetch a system image and set up requisite software for running MediaWiki. This can take 1–2 hours of CPU and wall clock time on a broadband connection, but it only needs to happen once. When you run vagrant up in the future, it will simply boot up the machine.

vagrant ssh starts an interactive login shell on the virtual machine. It'll log you in as the user vagrant; root access is available to via sudo, which is passwordless. Because the virtual machine is entirely sandboxed within your computer, it is configured for convenience, not security. As a rule, whenever you encounter a password prompt, the password is vagrant.

When you log in, you should see a colorful MediaWiki banner and a few reminders of useful commands.

The command hhvmsh will start an interactive PHP interpreter with MediaWiki's codebase already loaded. You can type in some code, hit 'enter', and the code will be evaluated immediately. If you start a line with '=', its computed value will be pretty-printed. Type ? for quick help or help start for additional instructions.

The /vagrant folder corresponds to the MediaWiki-Vagrant folder on your host machine, and its contents is shared. MediaWiki's code is installed in /vagrant/mediawiki. This allows you to use your normal editor environment on your host machine to edit the MediaWiki code that runs on your virtual machine.

Use vagrant git-update to keep your git repositories and database schema up to date. This command is equivalent of performing git pull for core and all extension directories, followed by running the update.php script.

Log out of your virtual machine by typing logout or by pressing CTRL + D. Now that you're back in a standard command prompt, you can run vagrant halt to shut down the virtual machine and vagrant up to bring it back up. vagrant destroy will delete the virtual machine's files; this command is useful if you want to return your instance to a pristine state. (You'll need to follow up with vagrant up to provision a fresh instance.)

Using roles[edit | edit source]

MediaWiki-Vagrant sets up a basic MediaWiki instance by default, but it also knows how to configure a range of complementary software, including some popular MediaWiki extensions and their dependencies. These optional software stacks are collectively known as 'roles', and MediaWiki-Vagrant offers an easy and powerful command-line interface for managing them.

$ vagrant list-roles
# Display a list of available roles.

$ vagrant enable-role role
# Turn on role for this machine.

$ vagrant disable-role role
# Turn off role for this machine.

Watch a short screencast demonstrating how to use roles. Roles has more information about some roles.

If you add many roles, you may need to increase memory available to the Vagrant VM. In particular, setting up the "browsertests" role involves compiling the ffi ruby Gem which is a memory-hungry task; if it fails try freeing some memory in the VM or increasing its memory allocation (bug 53864).

See the section Authoring roles below if you're interested in adding roles to MediaWiki-Vagrant.

Additional Suggestions[edit | edit source]

Local MediaWiki core copy[edit | edit source]

Sometimes you may want to start over from scratch by removing the entire vagrant directory or cleaning out the vagrant/mediawiki directory. In order to speed up the vagrant provisioning process, you may want to consider keeping a local, updated clone of the MediaWiki core that you copy into vagrant/mediawiki.

for example, assuming you are cloning MediaWiki repos into ~/projects/mediawiki/:

# clone and store a clean copy of MediaWiki core in ~/projects/mediawiki/core
cd ~/projects/mediawiki/
git clone ssh://<your-gerrit-username>@gerrit.wikimedia.org:29418/mediawiki/core
 
# clone a clean copy of vagrant in ~/projects/mediawiki/vagrant
cd ~/projects/mediawiki
git clone ssh://<your-gerrit-username>@gerrit.wikimedia.org:29418/mediawiki/vagrant
 
# create the mediawiki sub-directory if it doesn't exist
cd ~/projects/mediawiki/vagrant
mkdir ~/projects/mediawiki/vagrant/mediawiki
 
# copy the clean MediaWiki core to the clean vagrant/mediawiki directory
cp -r ~/projects/mediawiki/core/ ~/projects/mediawiki/vagrant/mediawiki

Update cloned repos[edit | edit source]

Update the cloned repos as often as possible/necessary.

cd ~/projects/mediawiki/core
git pull
 
cd ~/projects/mediawiki/vagrant
git pull
git submodule update --init --recursive

Vagrant reload[edit | edit source]

Run vagrant reload after your initial vagrant up and after any new vagrant provision commands.

When to enable roles[edit | edit source]

Enable roles only once you've successfully run your first vagrant up.

Debugging[edit | edit source]

Provisioning[edit | edit source]

You can debug the provisioning process by running

PUPPET_DEBUG=1 vagrant provision

PHP[edit | edit source]

You can debug PHP with Xdebug if you vagrant enable-role zend. Debugging in PHP is different from other client-side debugging. Your IDE listens for incoming connections, and when you access the server with a browser, a special header instructs PHP to connect to your IDE. See MediaWiki-Vagrant/Advanced_usage#MediaWiki debugging using Xdebug and an IDE in your host for further information.

Chrome[edit | edit source]

  • For Chrome users, you should get XDebug Helper, and optionally Clear Cache, HTTP headers, and Mod Headers. Configure clear cache to automatically reload after clearing, and set up keyboard shortcuts (e.g. Ctrl+R for clear&reload, Ctrl+Shift+D to switch XDebugger on/off)

Firefox[edit | edit source]

  • Firefox users should check out easy Xdebug.
  • Install and configure an xdebug-compatible IDE on your machine (Eclipse, PhpStorm, Emacs, etc)
  • In IDE, start listening for the incoming debug connection
  • In IDE, set break point at the spot that interests you
  • Enable XDebug in the browser and navigate to your vagrant installation ( http://127.0.0.1:8080/... )

Debugging Unit Tests[edit | edit source]

This method is a bit hacky, but can be used until debugging remote interpreter improves (e.g. in phpStorm 8 EAP). This workaround lets you run MediaWiki unit tests from the browser.

  • Download phpunit.phar file to the root of you vagrant directory.
  • Create a php file unittest.php in the root of mediawiki directory (mapped as /w on apache). Do not commit this file to the repository.

Pushing commits[edit | edit source]

If you're using MediaWiki-Vagrant for development you'll probably want to push some commits to MediaWiki core or an extension's repository using git review. By default, all remotes point to the https:// URLs. To avoid overriding this on a case by case basis, run:

$ git config --global url."ssh://gerrit.wikimedia.org:29418/".insteadOf "https://gerrit.wikimedia.org/r/"

How do I...?[edit | edit source]

Check PHP version and settings
http://127.0.0.1:8080/info.php
Edit LocalSettings.php?
First, check that there is no role (vagrant list-roles) that already does what you need. If not, create a file in settings.d/ directory. See README and 00-debug.php-example file.
Update MediaWiki code?
The easiest is to use vagrant git-update from the host. Or you can use regular git fetch, pull, etc. commands in vagrant/mediawiki and vagrant/mediawiki/extensions/SomeExtension directories. You can run these commands on the virtual machine, but the file access will be faster on the host machine. MediaWiki-Vagrant pulls code from git master when you initially set up and/or add a role, but doesn't automatically update code after that.
Run MediaWiki PHP interpreter
ssh to vagrant and run mwscript eval.php. You might need to run it with sudo
Run MediaWiki SQL interpreter
ssh to vagrant and run mwscript sql.php. You might need to run it with sudo
Update virtual machine software packages?
vagrant provision does not update system packages in the VM. When you connect with vagrant ssh the login message will inform you
NN packages can be updated.
NN updates are security updates.

In vagrant ssh:
  • to update all packages, enter sudo apt-get update && apt-get upgrade
  • for "automatic installation of security (and other) upgrades", similar to labs instances, enter sudo unattended-upgrade
  • to update to the same packages that are on production WMF servers... TODO
Customize Vagrant
You should never need to change Vagrantfile directly. There are several aspects of vagrant you can customize:
  • Core settings (git user, ports, ram, ip, port forwarding) can be customized via .settings.yaml file. See vagrant config --help and vagrant forward-port --help for instructions.
  • Perform additional steps after Vagrantfile load by creating a file called Vagrantfile-extra.rb and placing it in the same folder as Vagrantfile - it will be automatically loaded. In case of conflict, values in the 'extra' file will supersede values in this file. See example in support/ directory.
Add custom Puppet code?
This is ideal if you want to work on your own MediaWiki site locally and let the MediaWiki-Vagrant install your dependencies for you. Its ideal if you have your own fork. There is a distinction between a role and this use case. Roles are meant to be installed in any order and without breaking. If your fork needs different calls and get in trouble with roles, create your own class and call what you need, including roles.
To do so, place your custom puppet code in puppet/modules/local/manifests/myown.pp with your own class, like so:
class local::myown {
    include ::role::svg
}

To apply your class, add it to the "classes" key in puppet/hieradata/local.yaml. You can create the file if it doesn't exist.

classes:
  - local::myown

Then run vagrant provision to apply the change via Puppet.

Update MediaWiki-Vagrant itself?
(For example, to use new roles.) In a terminal, change to the vagrant directory on the host computer and enter a regular git command such as git pull --ff-only. You will typically want to run vagrant provision after updating to apply any new puppet changes to your virtual machine.
Run GUI applications on the virtual machine?
If you have an X server installed, SSH into the virtual machine using ssh -- -X to enable X forwarding. (Mac users should update to the latest version of XQuartz.)
As an alternative, you can run the virtual machine in GUI mode, which allows you to interact with the VM as though it had a physical display. To enable GUI mode, create a file called Vagrantfile-extra.rb in the root repository folder, with this as its content:
Vagrant.configure('2') do |config|
    config.vm.provider :virtualbox do |vb|
        vb.gui = true
    end
end
Save the file and run vagrant halt followed by vagrant up. The virtual machine's display will appear in a window on your desktop.
Adjust the resources allocated to the VM?
If you'd like to allocate more or less CPU / RAM to the VM, see vagrant config --help for instructions.

Alternatively, you can do it by creating Vagrantfile-extra.rb (see support/ dir for an example):

Vagrant.configure('2') do |config|
    config.vm.provider :virtualbox do |vb|
        # See http://www.virtualbox.org/manual/ch08.html for additional options.
        vb.customize ['modifyvm', :id, '--memory', '768']
        vb.customize ['modifyvm', :id, '--cpus', '2']
    end
end
Change the editor used for git commit messages?
git config --global core.editor "vim"
Run a branch of MediaWiki other than master?
Set the "mediawiki::branch" key in puppet/hieradata/local.yaml. You can create the file if it doesn't exist.
mediawiki::branch: "wmf/1.24/wmf18"

Note Note: This change has to be made BEFORE running vagrant up for the first time. If you decide you want to do it later, make the change, destroy your current VM with vagrant destroy -f, delete your existing mediawiki checkout and finally build a new VM with vagrant up.


Advanced usage[edit | edit source]

As an alternative to managing all MediaWiki settings in a single, large LocalSettings.php file, consider grouping your configurations by component or theme, and creating a separate PHP file in settings.d/ for each group. This makes it quite easy to keep your settings organized, to temporarily disable specific configurations, and to share settings with others.

MediaWiki will automatically load any PHP files in settings.d/ in lexical order. You can control the order in which your configurations are set by adopting the habit of adding a two-digit prefix to each file name.

For example:

    settings.d/
    ├── 10-RunFirst.php
    ├── 20-SomeExtension.php
    └── 99-RunLast.php

Note that the settings files in settings.d/puppet-managed are automatically created and destroyed in response to your Puppet configuration. Don't put your custom settings there, because Puppet will erase or override them. Keep your custom settings files in settings.d/ instead.

Authoring roles[edit | edit source]

MediaWiki-Vagrant uses Puppet to configure MediaWiki on the virtual machine. Puppet is a configuration management tool that works by providing a domain-specific language for expressing software configurations in a declarative fashion. Files containing Puppet code are called 'manifests'. When Puppet runs, it interprets the manifests you feed it and configures the machine accordingly. MediaWiki-Vagrant's Puppet code is located in the puppet/ subdirectory.

MediaWiki-Vagrant's Puppet codebase contains abstractions that make it easy to automate the configuration of MediaWiki extensions and related software. If you are a developer working on a software project that relates to MediaWiki, you are encouraged to submit a patch with a Puppet role for your project. Adding a Vagrant role for your project makes it easy for other developers to check out your work. Using a managed virtual machine as a development sandbox for your project reduces the chance of "works-on-my-machine" errors that often result from geographically remote developers working in incompatible environments.

The virtual machine provisioned by MediaWiki-Vagrant resembles Wikimedia's production environment in key respects, and it uses the same tool—Puppet—that Wikimedia's technical operations team uses to manage production servers and Wikimedia Labs instances. This can help you write software that will meet the technical requirements for deployment on Wikimedia's production cluster.

The easiest way to get started with custom roles is to look at how existing roles are implemented in puppet/modules/role/manifests/*.pp. These roles depend on Puppet modules in puppet/modules. The Puppet code is generally well-documented and contains examples that demonstrate its proper usage.

Setting up labs instances[edit | edit source]

You can use Labsvagrant to configure Wikimedia Labs machines with a given MediaWiki-Vagrant role.

Bugs[edit | edit source]

If you spot a bug in MediaWiki-Vagrant, please report it. First, make sure the bug is not a known Vagrant or VirtualBox bug by searching the Vagrant issue tracker on GitHub and the VirtualBox bugtracker. If it is not, go ahead and submit the bug to Wikimedia Bugzilla. Clearly describe the issue and include steps to reproduce, whenever possible.

TLDR[edit | edit source]

Links[edit | edit source]

Notes[edit | edit source]

  1. If you are on Fedora, do not follow Oracle's instructions. Instead, enable the RPMfusion repositories (e.g. via Apper's configuration), then select VirtualBox and kmod-VirtualBox for installation (note: case-sensitive and sometimes version-sensitive! if you don't find the package, search in Apper instead). You will probably get an error about your kernel being too recent: copy the full kernel name with version and install it with sudo yum install (which removes your recent kernel), then repeat, then reboot. If your next kernel upgrade breaks VirtualBox, try and install dkms, then repeat what above.