User:DWalden (WMF)/Permissions Testing

Introduction
This presents a strategy I often use when testing features that give users access to data which may be sensitive.

For example, information about who made a particular edit or performed a particular logged action.

The basic idea is to compare the information the feature you are testing gives the user access to with what information they can find out through existing means.

For example, the feature should not give the user information about a revision which they cannot see in the revision history of a page.

You also want to enumerate all the variables which affect whether or not a user has permission to see a particular bit of data and test all the combinations of these variables.

For example, revisions and logged actions can have various visibility settings which might mean a user is not allowed to see some or all of the information about it.

Enumerating variables
The relevant variables will depend on the data your feature gives access to. I give examples for revisions and logged actions.

Revisions

 * Visibility of revision (see Help:RevisionDelete and Manual:RevisionDelete)
 * Is the comment hidden?
 * Is the revision's wikitext hidden?
 * Is the editor hidden?
 * Is it suppressed (meaning it is visible only to users with  rights)? Bear in mind that this option only works if you also hide one or more of the above options.


 * Editor's suppressed status
 * Is the user who made the edit suppressed or not?


 * Editor type
 * Is the user who made the edit an anonymous (IP), temporary or named (logged in) user?


 * Your user's rights
 * Is there a specific right associated with using this feature and does the user have it?


 * Your user's blocked status
 * Should a user be able to see this information if they are blocked or suppressed?

You can then test each of the combinations of these such as in the below table, which shows only a small sample of the combinations:

You can also generate them directly by querying the database, which will also tell you which revisions to test. For example, User:DWalden_(WMF)/IPInfo/Permission_Testing_Data.

Logged actions
TODO

Generating data
You will often find that your test environment won't have revisions or logged actions which have all the combinations of relevant variables.

Here is a script to help you generate them.


 * 1) Follow the install instructions for pywikibot: Manual:Pywikibot/Installation.
 * 2) Copy the above file  into the pywikibot "scripts" directory.
 * 3) Find some revision IDs or log item IDs you want to change the visibility of. Ideally, you should find 15 of them. In the next step, we assume the revisions are numbered from 1 to 15.
 * 4) Run:

Revisions
There are lots of places to see revision information on a wiki, which you can compare to the information your feature is returning. For example, you can look at a pages revision history or Special:Diff.

If you want to do it programmatically, you can use prop=revisions. For example, to look up revision 123456 you can go to https://www.mediawiki.org/w/api.php?action=query&prop=revisions&revids=123456.

Example of a script doing this (except using the list=allrevisions).