Core Platform Team/Initiative/API Gateway/Documentation Plan

From mediawiki.org
Jump to navigation Jump to search

View the prototype Track this project on Phabricator Share feedback Subscribe to project updates

Overview[edit]

The API Gateway is a CPT initiative to create a rate-limited API proxy at api.wikimedia.org as part of the Platform Evolution program. The MVP scope includes the MediaWiki REST API (including page, media, history, and search endpoints) and the Feeds API, with the intention of adding APIs over time. See the initiative page and Phabricator tag.

Approximate list of endpoints for MVP
  • GET /feed/v1/{project}/{language}/announcements
  • GET /feed/v1/{project}/{language}/featured/{yyyy}/{mm}/{dd}
  • GET /feed/v1/{project}/{language}/onthisday/{type}/{mm}/{dd}
  • GET /core/v1/{project}/{language}/file/{title}
  • GET /core/v1/{project}/{language}/search/page
  • GET /core/v1/{project}/{language}/search/title
  • POST /core/v1/{project}/{language}/page
  • GET /core/v1/{project}/{language}/page/{title}
  • PUT /core/v1/{project}/{language}/page/{title}
  • GET /core/v1/{project}/{language}/page/{title}/bare
  • GET /core/v1/{project}/{language}/page/{title}/html
  • GET /core/v1/{project}/{language}/page/{title}/with_html
  • GET /core/v1/{project}/{language}/page/{title}/links/language
  • GET /core/v1/{project}/{language}/page/{title}/links/media
  • GET /core/v1/{project}/{language}/page/{title}/history
  • GET /core/v1/{project}/{language}/page/{title}/history/counts/{type}
  • GET /core/v1/{project}/{language}/revision/{id}/bare
  • GET /core/v1/{project}/{language}/revision/{from}/compare/{to}

The purpose of this document is to manage the planning and feedback process for the documentation for this project.

Components[edit]

Wikimedia API Gateway architecture diagram
MediaWiki
The API Portal will be a MediaWiki instance at api.wikimedia.org (beta: api.wikimedia.beta.wmflabs.org)
WikimediaApiPortal skin (Phabricator project)
A MediaWiki skin providing the look-and-feel of the API Portal
WikimediaApiPortalOAuth extension (Phabricator project)
A MediaWiki extension running on the API Portal that interfaces with the OAuth extension on Meta-Wiki
OAuth extension on Meta-Wiki
Responsible for storing and managing OAuth clients

Implementation plan[edit]

ID Description Owner Status

Phase 1: Planning and development

1 Finalize development contract Cindy Yes Done
2 Security approval for the Chameleon skin (phab:T246949) Cindy Cancelled (See phab:T252462.)
3 Collect feedback from Technical Engagement Alex Yes Done (Notes)
4 Create a wiki instance Cindy Yes Done
5 Submit request to create a beta wiki instance (phab:T254185) Cindy Yes Done
6 Create a MediaWiki skin ("WikimediaApiPortal") that matches the provided design, and create documentation for the extension on mediawiki.org Cindy Yes Done
7 Define technical specification for interactions between the API Portal and the OAuth extension Bill Yes Done
8 Update OAuth extension to support configuration via web API, including documentation updates
  • phab:T257982 (Update the OAuth extension to support the API Portal)
Bill Yes Done
9 Create a MediaWiki extension ("WikimediaApiPortalOAuth") that provides a simplified user interface for managing OAuth 2.0 consumers through the API Portal, matching the provided design, and update documentation Cindy: UI

Bill: API

Yes Done
10 Draft documentation for the API portal Alex Yes Done

Phase 2: Beta testing

Prerequisites:
  1. WikimediaApiPortal skinYes Done, WikimediaApiPortalOAuth extensionYes Done, and updates to OAuth extensionYes Done are code complete
11 Deploy WikimediaApiPortal (phab:T257959) and WikimediaApiPortalOAuth to api.wikimedia.beta.wmflabs.org Cindy Yes Done
12 Enable feature flag for new endpoints in the OAuth extension on beta Cindy Yes Done
13 Configure and test permissions on beta site (phab:T249834) Alex Yes Done
14 Test WikimediaApiPortal skin on beta site Alex Yes Done
15 Test WikimediaApiPortalOAuth extension and interactions with OAuth extension on beta site with beta-Meta Technical: Bill
Design: Alex
Yes Done

Phase 3: Private launch

Prerequisites:
16 Deploy WikimediaApiPortal and WikimediaApiPortalOAuth to api.wikimedia.org Cindy Yes Done
17 Configure permissions on api.wikimedia.org to enable private launch (Homepage and login page are publicly visible; other content is private) (phab:T249834) Cindy Yes Done
18 Add content to api.wikimedia.org, review, and iterate Alex Yes Done

Phase 4: Content testing

Prerequisites:
  • Completion of Phase 3
  • Completion of phab:T235276 (Client Developer uses MediaWiki REST API) and phab:T246265 (Client Developer uses Wikifeeds API)
19 Enable feature flag in the OAuth extension to enable new functionality Cindy Yes Done
20 Test tutorials and code samples Alex Yes Done

Phase 5: Public launch

Prerequisites:
  • Completion of Phase 4
  • Public launch of API Gateway (phab:T255032)
  • Performance reviews for WikimediaApiPortal skin (phab:T252462)Yes Done, WikimediaApiPortalOAuth extension (phab:T254950)Yes Done, and updates to OAuth extension (phab:T254951)Yes Done are complete
21 Update permissions to allow public read access to all content on api.wikimedia.org Cindy Yes Done
22 Work with engineers to plan, draft, and review documentation for implementers of the API Gateway (phab:T251440) Hugh Yes Done wikitech:API Gateway
23 Pass end-to-end testing for the Gateway, Portal, and OAuth extension in production (phab:T268257) Alex Yes Done
24 Announce launch to the wider movement (phab:T268257) Alex Postponed following product management handoff

Documentation for end-users of the API (API Portal)[edit]

Requirements[edit]

Documentation requirements[edit]

  • How to use the api.wikimedia.org API
  • How to use the OAuth 2.0 flow with the api.wikimedia.org API
  • Rate limiting policies
  • Links to other Wikimedia APIs and resources
  • Visual design similar to the prototype
  • Shared Wikimedia account login

OAuth 2.0 client management requirements[edit]

  • Developers can create, list, and disable OAuth 2.0 clients using a simplified interface
  • Visual and interaction design similar to the prototype

Scope[edit]

MVP scope[edit]

  • Manually created API reference docs and guides in English
  • Editing restricted to a docs-editors group while we stabilize the content and set up contribution flows (See phab:T249834)
  • User feedback via talk pages and “Was this page helpful?” gadget
  • OAuth 2.0 client management interface
  • Strategy for reducing duplication between the the API portal and the existing docs for the MediaWiki REST API and Feeds API.
  • Pageview data for the API portal

Future scope[edit]

  • API sandbox
  • Translate extension
  • Contribution flow for volunteers
  • Automated reference docs integrated with translatewiki
  • Asynchronous review process for contributors

Out of scope[edit]

  • Updates to the UI of the OAuth special pages on Meta

Implementation strategy[edit]

  • Use MediaWiki as the content platform
  • Implement the visual design using a new MediaWiki skin, configured to look similar to the prototype. Certain items such as the behaviors/flyouts may differ since they would use styles from Bootstrap. In the future, the visual style can be implemented using the improved Vector skin.
  • Create an extension that provides a simple UI to create simple OAuth 2.0 clients as shown in the prototype.
    • The new extensions will only be installed on the API Portal.
    • The new extension will interact with the OAuth extension installed on Meta through an API.
    • The new extension will be named WikimediaApiPortal and could also contain other UI elements needed for the API Portal.
  • Attempt to limit any additional libraries necessary for the UI.

Naming and location[edit]

Proposed location: MediaWiki instance at:

  • api.wikimedia.org (preferred)
  • Alternatives:
    • developer.wikimedia.org (could be confusing due to the fact that these docs won’t have the comprehensive scope of a Wikimedia Developer Portal)
    • restapi.wikimedia.org
    • apiportal.wikimedia.org
    • webapi.wikimedia.org
    • apidev.wikimedia.org
    • apidoc.wikimedia.org
    • dev.wikimedia.org (exists as a redirect to a page on mediawiki.org)

Considerations[edit]

Due to the maintenance effort required to sustain a new public wiki, we should seek to balance the design requirements of this project with the principles and processes of the Wikimedia movement.

Open questions[edit]

Prototype[edit]

View the API Portal prototype here, including:

Documentation for implementers of the API Gateway and rate limiting system[edit]

Documentation requirements[edit]

  • Overview of the implementation and configuration of the API Gateway and rate limiting system
  • How to manage rate limits
  • How to add and remove APIs from the Gateway

Implementation strategy[edit]

Proposed location: Wikitech TechOps docs

Objectives and metrics[edit]

Create a great developer experience for the Wikimedia API
Metric: Qualitative feedback from developer interviews
Make it easy for developers to get started
Metric: Percentage of visitors to the site who register an app
Metric: Percentage of registered apps with at least one API call
Make it easy for developers to build apps
Metric: Percentage of users with registered apps that have an app with more than five API calls