User:Brooke Vibber/Embed controls for iframes

From mediawiki.org

It'd be nice to expose a postMessage API for player control of cross-domain embedding of TimedMediaHandler video/audio assets and other rich media from the wiki into other sites. It'd be even nicer to use that same API for cross-wiki embedding via InstantCommons etc so local wikis don't need all the handler extensions installed to make use of rich players. And it'd be nicer still for that API to be something that can be standardish instead of MediaWiki-specific.

Basics[edit]

The basics of the HTMLMediaPlayer interface need to be doable via the iframe to allow for custom controls in the surrounding viewer or programmatic control by a demo harness or mashup sort of embedding use.

Need to be able to send commands:

  • play()
  • pause()
  • set currentTime / fastSeek()
  • set volume
  • set muted
  • etc

Need to be able to get values:

  • get currentTime
  • get volume
  • get muted
  • etc

Need to be able to subscribe to events:

  • loadedmetadata
  • loadeddata
  • timeupdate
  • playing
  • paused
  • ended
  • seeking
  • seeked
  • etc

Enhanced[edit]

  • querying for existence/compatibility of the API?
  • querying metadata about the resource?
  • exposing presence of the API through protocols like oEmbed, IIIF?
  • loading other files, a la switching videos in YouTube embed?
  • controls for non-a/v types?
    • 3d model -- set camera position/rotation/zoom
    • 2d panorama -- set position/zoom
    • spherical panorama -- set position/zoom
    • interactive diagram/widget/game -- ....????
    • etc
  • ???

Issues[edit]

By its nature, postMessage()->onmessage 2-way communication is always asynchronous.

This means emulating a synchronous API such as the currentTime property on HTMLMediaElement has to be approximated by subscribing to timeupdate events during playback.

Prior art[edit]

Look at the YouTube and Vimeo APIs for some examples. Vimeo's API internals are better documented (they have a library on github for the core of the message formatting behavior), while YouTube documents only the API of the JavaScript wrapper library.

I wrote up some notes in April 2016 at https://github.com/iframe-player/iframe-api-docs (see linked gdocs files from there) with the intention of fleshing out an ad-hoc demo and hoping to get folks interested in standardizing something, but haven't gotten back to it as of October 2016. Would need to search for and round up relevant folks.

Retrieved from "https://www.mediawiki.org/w/index.php?title=User:Brooke_Vibber/Embed_controls_for_iframes&oldid=6360060"