Extension:Embed Media

This page will document the development of the mediaWiki Embed Media Extension. To monitor its ongoing development you can check out the development server (may crash often)
 * See also: browser plugin-specs

Abstract
The Embed Media Extension consists of two major phases of implementation.


 * An extension of wiki image to allow the inline use of other media. This extension to include media will focus on theora encoded video and ogg encoded audio.
 * The development of a more complex system for tracking segmented reference of longer audio/video streams.

The first phase will extend the existing wikipedia image inclusion system into a “media” inclusion framework. The media framework will serve inline JavaScript to detect the client’s plug-ins and supported media types. The media framework will then write in embed code or link to download the file based on the available installed plug-ins. The media inclusion framework will initial support the in-line embedding of ogg theora encoded video but eventually could be useful for the inclusion of other media types such as browser rendered svg. Like this

Phase two will enable dynamic reference to segments from a wiki page. This will enable the highlighting of a piece of a longer video clip while providing a link back to the full original. This will be enabled through the usage of the mod_annodex extension. See the metavid site for examples of its usage. References to segments will be tracked and displayed on the media’s info wiki page. Also we will explore the practicality of converting uploaded video clips to ogg theora from ffmpeg supported codecs.

Plugin Options
The current ideal environment for various platforms is:


 * If you're using Windows and Firefox I recommend that you install annodex.net.
 * If you're using Windows and IE & have java, then the cortado player is used.
 * If using Linux and Firefox I recommend installing mozilla-plugin-vlc (available via apt-get)
 * If using Mac OSX and Firefox, for now use the java cortado player as problems have been reported for the OSX annodex-plugin. If you don’t have the java-plugin installed for firefox you may try safari)
 * If using Max OSX and Safari cortado player should kick in

The Future: Native Firefox ogg Support
Once the plugin is relatively stable we will petition mozilla foundation to integrate ogg theora video decoding into firefox. For the specs on the plugin see firefox Ogg Support

Native support for Ogg Theora in Opera
Opera has released an experimental version that has native support for Ogg Theora. Opera supports an (early version of) the proposed video element.

Annodex/VLC Extension/plug-in
users must: install annodex: annodex.net

Description: The vlc based annodex extension is similar to what video.google used before they migrated to flash. The annodex player includes a xul CMML annotation interface for video meta data. The ogg theora decoder plugin used in annodex is custom compile of the vlc mozilla plugin but restricted to decoding ogg theora streams.

pro: con:
 * Can load content from arbitrary servers. (this may make setting up mirrors slightly esier ie avoid iframes and javaapplet upgrades)
 * Has good javascript hooks for building playback interfaces in css/html/ajax.
 * has no interface must be built in css/html/ajax some js hooks have slight delay :(
 * Internet Explore support is still a bit scetchy But not very important for initial implementations.

Java applet include
Description: The cortado ogg theora decoder applet is used to play back ogg theora streams in a web browsers which support java applets.

pro: con
 * java loads in the browser (no need to install a separate extension)
 * small footprint, the java applet is only 130k or so but that may be counteracted by less client side applet caching vs. a plug-in being installed.
 * the applet and audio/video must reside on the same server.
 * would have to be implemented as an iframe
 * java paths are not url-rewrite friendly (still to fully research this)
 * java applets are not widely used and would need to be downloaded each time the browser loads and there are incompatibilities with various platforms

Flash embed include
This is certainly the current web standard

Given the flash install base it may a good thing to support it among other formats.

QuickTime
Could be supported by codec extension to decode ogg theora video. would have to use javascript to detect quicktime ogg theora component info:

Windows Media Player
Could potential be supported via codec extensions as well.. would have to research the ability to detect codec extensions for windows media player.

First phase
The default media will not play or download when the page is loaded. Wiki users will have to hit “play” for the media to play in their browser. Displayed along side the controls will be a link to the media’s info page which will include media license info and a link to download the clip disk.

The wiki syntax will be comparable to the image include syntax. The media extension will interpret the request and serve some inline JavaScript to detect the client’s plug-in support. The video plug-in-detection script will look for a vlc browser plugin, the annodex browser extension or a java run time environment (for cortado) and return the respective embed code. Once detected a session variable will be set to facilitate use of the embed code without the javascript detection in subsequent media display requests. The framework will eventually allow for the arbitrary selection of theora decoders from the user preferences.

Inline include
First test will be to alter the way the [[Media:foo_media.ogg]] is displayed.

upload processing
After a video clip is uploaded a thumbnail should be generated. This can be done a number of ways. mplayer -vo jpeg:z=6 -vf scale -zoom -xy 120 -ao null -ss '.$time.' -frames 2 \"'.$stream_file.'\" >/dev/null 2>&1
 * ffmpeg-php is a easy bridge between php and ffmpeg libraries. In my use of ffmpeg-php it works well for grabbing images in sequence but the grabbing of an arbitrary frame takes a long time as the indexing was not functional but this may not be a universal bug. ffmpeg-php is still sort of bleeding edge and usually needs to be compiled to be installed. (which can be tricky with all the ffmpeg libraries)
 * mplayer also uses the ffmpeg libraries but is easier to install. php could issue a shell command like:

to grab a frame $time There are likely some disadvantages to issuing shell commands but the performaces is probably similar.

The mplayer approach is probably the way to go, but both methods could be supported.

Inline include Syntax
The syntax will be similar to inline images: Basic include: Embed:my_theora.ogg Include with scaling + framing frame|right|320px| my theora clip

Second Phase
Still in development