Wikimedia Apps/Team/Android/App hacking

We welcome volunteer contributions!
The Wikipedia Android app is completely open source and welcomes contributions from all. Follow the steps below to get started, and feel free to ping any member of the WMF Android dev team (bearND, dbrant, mdholloway, niedzielski) on for assistance.

You can find our coding conventions here.

Open bugs and feature requests
Work items are tracked on our work board. First time app contributors are encouraged to consider an Easy task. Unassigned tasks in the "Bug Backlog", "Tech Debt Backlog", and "Product Backlog" columns are generally suitable for development and roughly organized by descending priority. (Please do not work on tasks in other columns like "Open Questions" and "Needs Triage".)

Create a developer account
See Developer access.

Add your SSH key to Gerrit
See Gerrit/Tutorial.

Install JDK 8
JDK 8 is needed to build the Android app. To find your current version:

$ javac -version javac 1.8.0_102

The open source OpenJDK may be installed on Debian / Ubuntu by executing. The closed source Oracle JDK may be installed from here.

Download and install Android Studio
https://developer.android.com/sdk/index.html

Run the setup wizard. Standard installation is fine. Installation will take a while.

Download the project source
With SSH credentials (requires Developer access):

Or anonymously:

Import into Android Studio
Open Android Studio > Import project > select build.gradle from the downloaded repository.

Important: If you are prompted about upgrades to components (e.g., Gradle or the Android Gradle plugin) during the initial build, please click "Don't remind me again for this project." We update these components in dedicated patches after a thorough review of the release notes, but they have a way of showing up in unrelated patches if automatic IDE prompting is enabled.

Checkstyle-IDEA
IntelliJ's Checkstyle-IDEA plugin is useful in ensuring that your code will meet the project style guidelines. Install from Android Studio > Preferences > Plugins, restart to complete the installation, then add the custom configuration file for the project:

Android Studio > Preferences > Other Settings > click on the '+' in the upper pane and select.

Make sure you activate the new configuration file by checking the checkbox next to it, so our checkstyle rules are enabled and deviations show up as errors in the code editor.

Install and configure git-review
Install git-review:  (Debian/Ubuntu) or   (OS X)

Set the default remote to origin:

git config --global gitreview.remote origin

Make changes and submit for review via Gerrit
After cloning the app repository, set it up for git review with  in the root directory.

Make and commit your changes, and provide a commit message describing your changes. (Please ensure that the commit message includes line breaks for lines running over 75 characters.) If your work is associated with a specific task in Phabricator, please include, as the last line of the commit message, the following line:

(where xxxxxx represents the number of the Phabricator task, e.g., T149500).

After creating your commit, submit it for review with.

You'll most likely receive requests to update and resubmit the patch. (It happens to all of us!) Make these changes locally and amend your patch with. Submit it again for review with, and your amended patch will show up as a new patch set in Gerrit provided you have not altered the   line inserted as the last line of the commit message by git-review when submitting the initial commit.

For more information, see Gerrit/Tutorial.

Setup
cd scripts virtualenv -p python2 .env . .env/bin/activate pip install -r requirements.txt Note: the lxml Python module seems to have system package dependencies on Debian and Ubuntu. If you're missing headers, try.

Troubleshooting Script Setup
On OS X a few of us ran into the issue that `libxml` couldn't get compiled. Running the following command helped: xcode-select --install If this still doesn't work, you might also want to try: pip install sh jinja2 unicodecsv

Update bundled CSS files
The various CSS files for this project are generated by the *.less files found in the MobileFrontend and MobileApp MediaWiki extensions.

You'll need a MediaWiki installation, MediaWiki-Vagrant recommended, to generate them.

In the ./vagrant directory:

vagrant enable-role mobileapp mobilefrontend vagrant provision

Change the url in scripts/make-css-assets.bash to http://127.0.0.1:8080/w instead of the bits url, then run the script and test.

Note: your system may be unable to run the Android emulator and a Vagrant / VirtualBox instance simultaneously.

Update bundled JavaScript
Portions of JavaScript code run inside the WebView component that displays articles.

This code is prepackaged using Grunt, which must be re-run every time the master .js files are edited before building.

Preparing:

First, install the Grunt CLI tool:

npm install -g grunt-cli

Install dependencies for packaging:

cd www npm install

Building: cd www grunt

This will produce output files under  which will be included in the apk file.

Run
cd scripts python make-templates.py mv *.java ../app/src/main/java/org/wikipedia/staticdata/

Automated Tests
The codebase supports two kinds of tests:
 * 1) JVM JUnit (preferred, off device, fast, less flaky)
 * 2) Android instrumentation (on device)

Command line usage

 * JVM JUnit
 * Android instrumentation
 * Both JVM JUnit and Android instrumentation (including screenshot tests)
 * Both JVM JUnit and Android instrumentation (including screenshot tests)
 * Both JVM JUnit and Android instrumentation (including screenshot tests)

Tips

 * When running, add the   flag to run the Android instrumentation tests even if the JVM JUnit tests failed.

Android Studio setup
Both kinds of tests are supported in Android Studio. You may toggle between them by changing the Test Artifact from View -> Tool Windows -> Build Variants. "Unit Tests" are JVM JUnit tests and "Android Instrumentation Tests" are Android instrumentation tests. A run configuration must be made to prior to executing the tests.

JVM JUnit

 * 1) Click Run -> Edit Configurations...
 * 2) Click Add New Configuration (a large plus icon) and select JUnit.
 * 3) Name the configuration.
 * 4) Change the Test kind to All in package.
 * 5) Set the package to org.wikipedia.
 * 6) Ensure the working directory is set to $MODULE_DIR$.
 * 7) Set Use classpath of module to  . (TODO: update screenshot)



Screenshot tests
Some Android instrumentation tests don't actually test much. Instead, they capture a screenshot to be diffed against an expected output. Since different platforms produce different results, tests must be executed on the Android N (API 24) Nexus S emulator. Tests are exercised and the screenshots downloaded by executing:

Or:

Both commands slowly run all Android instrumentation tests. To exercise specific tests, specify them. For example:

Then perform a diff against the previously committed screenshots:

If you're executing tests quickly via the GUI (recommended), a good workflow is: The turnaround should be about a minute.
 * 1) Right click on the test to execute and click run. If you see a permission error the first time, execute  , then start over
 * 2) If successful, execute

Once the screenshots are ready for versioning, move them from app/screenshots to app/screenshots-ref. git-diff-img is one way to compare versions before committing.

Conventions

 * Java code must run cleanly against Checkstyle
 * JavaScript must run cleanly against JSHint
 * We're working to be lint free! Currently Android Linting has many offenders but new contributions should help us progress towards zero.
 * Python changes should run cleanly against flake8.
 * All Android and JVM unit tests should pass.
 * Git commit messages should conform to the Gerrit/Commit message guidelines
 * Documentation should conform to the Wikipedia manual of style
 * When making changes to any strings.xml, corresponding explanatory text should be added to values-qq/strings.xml
 * Third-party libraries must meet acceptance criteria
 * Patches should conform to our coding conventions

Optimizing Gradle builds
The Gradle build system requires a notoriously large amount of system resources, but there are ways of speeding it up and making sure it works consistently. Add the following lines to your  file:

org.gradle.daemon=true org.gradle.parallel=true org.gradle.configureondemand=true org.gradle.jvmargs=-Xmx2048M

WebView debugging in Chrome
Wikipedia for Android makes extensive use of WebViews. To debug WebView activity, navigate Google Chrome to chrome://inspect/#devices, then click on the topmost “inspect” link under “WebView in org.wikipedia.” From there you can debug the WebView like any other web site in Chrome.

Useful Gradle commands
If you prefer command line use the wrapper script in the root of the repo: ./gradlew

To run a clean debug build: ./gradlew -q clean assembleDevDebug

You can skip the clean part usually, which makes it much faster (from 1m:05s to 7s on my box): ./gradlew -q assembleDevDebug

To install build on device/emulator: ./gradlew -q installDevDebug

To see ProGuard output: ./gradlew clean --info proguardDevRelease

To run checkstyle: ./gradlew checkstyle

To run Lint: ./gradlew lintDevDebug

To refresh dependencies (usually not needed): ./gradlew --refresh-dependencies

To list dependencies: ./gradlew app:dependencies --configuration compile

Making apt work in Ubuntu 64-bit
If you're using 64-bit versions of Ubuntu, you might see this error when building with gradle: "Cannot run program '~/.android-sdk/build-tools/22.0.1/aapt': error=2, No such file or directory". To fix this error, install the following packages:

sudo apt-get install lib32stdc++6 lib32z1

Working with Vagrant and the Content Service

 * To work with a local Vagrant instance, change the mediaWikiBaseUri developer setting and optionally disable mediaWikiBaseUriSupportsLangCode.
 * To work with a local RESTBase instance, update RESTBaseUriFormat and optionally enable useRestbase_setManually.
 * mediaWikiBaseUriSupportsLangCode controls whether the wiki language of mediaWikiBaseUri is prefixed.
 * mediaWikiBaseUri and mediaWikiBaseUriSupportsLangCode are the second argument of RESTBaseUriFormat.
 * useRestbase_setManually and useRestbase can affect whether a MediaWiki or RESTBase client is used internally. When a MediaWiki client is forced, RESTBaseUriFormat is unused.
 * The Wiktionary domain is currently hardcoded. Ex: http://localhost:6927/en.wiktionary.org/v1/page/definition/dog
 * It is recommend to fully terminate the app process after changing any of these settings.

Beta cluster config
Ex:
 * https://en.wikipedia.beta.wmflabs.org/api/rest_v1/feed/featured/2016/06/01
 * https://zh.wikipedia.beta.wmflabs.org/api/rest_v1/page/random/title

RESTBaseUriFormat (default): %1$s://%2$s/api/rest_v1/ mediaWikiBaseUriSupportsLangCode (default): enabled mediaWikiBaseUri (nondefault): https://wikipedia.beta.wmflabs.org

Local MediaWiki with production English Content Service config
Ex:
 * http://localhost:8080/w/api.php?action=query&format=json&titles=Main_Page&continue=&prop=pageimages&piprop=thumbnail&pithumbsize=320&pilimit=1

RESTBaseUriFormat (nondefault): %1$s://en.wikipedia.org/api/rest_v1/ mediaWikiBaseUriSupportsLangCode (nondefault): disabled mediaWikiBaseUri (nondefault): http://localhost:8080

Auth manager config
RESTBaseUriFormat (default): %1$s://%2$s/api/rest_v1/ mediaWikiBaseUriSupportsLangCode (nondefault): disabled mediaWikiBaseUri (nondefault): http://authmanager.wmflabs.org

Local Content Service config (broken)
Ex:
 * http://localhost:6927/en.wikipedia.org/v1/page/most-read/2015/12/01
 * http://192.168.1.10:6927/zh.wikipedia.org/v1/page/mobile-sections/%E4%B8%AD%E5%9C%8B

useRestbase_setManually (nondefault): enabled useRestbase (default): enabled RESTBaseUriFormat (nondefault): http://localhost:6927/%2$s/v1/ mediaWikiBaseUriSupportsLangCode (default): enabled mediaWikiBaseUri (default): https://wikipedia.org

This doesn't quite work at moment. The app attempts to use a RESTBase feed/ endpoint when page/ is needed for the Content Service.

Locale Vagrant with local Content Service config (broken)

 * http://192.168.1.11:6927/192.168.1.11:8080/v1/feed/featured/2016/10/11

useRestbase_setManually (nondefault): enabled useRestbase (default): enabled RESTBaseUriFormat (nondefault): http://localhost:6927/%2$s/v1/ mediaWikiBaseUriSupportsLangCode (nondefault): disabled mediaWikiBaseUri (default): http://localhost:8080

This is currently broken for same reason the local Content Service config is.

Help make it better!
Pick a task from Phabricator: Testing:
 * Easy / All tasks.
 * Alpha release
 * Beta release
 * Production release

See pending/recent code reviews:
 * Wikipedia Android app
 * Java MediaWiki-API
 * MediaWiki MobileApp extension

Publishing to Sonatype OSSRH
This is for our libraries; currently only java-mwapi but there could be more in the future.

Create GPG key
See hints at http://central.sonatype.org/pages/working-with-pgp-signatures.html#generating-a-key-pair.

Usually accept the default. But I would use the max value  for the requested keysize. Don't forget to add a passphrase.

Towards the end of the output you'll see the public key in a line similar to this: pub   4096R/F13983AB 2015-07-16

The highlighted portion would be your public key you want to enter in the signing config below as the.

Create a new Sonatype JIRA account
Sonatype account registration. Use these values for nexus* in the Signing configuration step below. This will be used for the  Gradle task. Hint: It's good to avoid special characters in the username and password that could be misinterpreted by Gradle or the shell.

Signing configuration
A GPG signature is necessary for signing but is independent of uploading. Add the following to your : signing.keyId= signing.password= signing.secretKeyRingFile= nexusUsername= nexusPassword=

Snapshots
If this was done for the java-mwapi library you should see a new SNAPSHOT version on OSSRH.
 * Update the version in lib/build.gradle.
 * Execute the following commands

Promotion

 * 1) Merge snapshot code.
 * 2) Promote artifacts from the web interface. After uploading a non-snapshot version, click staging repositories and search for "mediawiki". Select the repository and and at the top click "close". After refreshing, click release.
 * 3) Push and +2 the two version bump patches.
 * 4) Push the version tag.
 * 1) Push and +2 the two version bump patches.
 * 2) Push the version tag.