Annodex Firefox Extension Viewer (AFE): Media Engine Documentation

Table of Contents

1.0 Installation of the Annodex Firefox Extension
    1.1 Installing the Media Engine Component
2.0 Introduction to the AFE Media Engine
    2.1 Building Concepts of the AFE Media Engine
        2.1.1 Precompiled Gecko SDKs for Windows and MacOS X
        2.1.2 Precompiled Gecko SDK Linking Issues on Linux
3.0 Compilation of the AFE Media Engine
    3.1 Building the AFE Media Engine in Cygwin
    3.2 Building the AFE Media Engine in Mac OS X
    3.3 Building the AFE Media Engine in GNU/Linux
4.0 AFE Media Engine Directory Structure

1.0 Installation of the Annodex Firefox Extension

Note: These installation steps are intended for developers; end-users will be supplied with a Mozilla .xpi package that they can simply download and double-click on to install. For Windows users, a batch file for Firefox Browser users called "WinSetup.bat" semi-automates the installation process, with minor configuration changes required as prompted at the end of the installation. The three main points of installation are as follows:

  1. Installing the Mozilla AFE Chrome:  Please refer to XUL Core documentation for installation of files.
    1. Copying the /Core/chrome/afeview/ directory to the Mozilla Firefox Chrome directory.
    2. Creating or modifying an installed-chrome.txt file to initialise the AFE Chrome with Mozilla Firefox, and deleting the existing chrome.rdf file.
  2. Installing the ANXLoader Mozilla Component: Please refer to XUL Core documentation for installation of files.
    1. Copying the components\ANXLoader.js file to the Mozilla Firefox Components directory.
    2. Deleting the compreg.dat auto-configuration file to be regenerated upon startup of Mozilla Firefox for initialising ANXLoader.
  3. Installing the Media Engine Component:
    1. Copying the platform dependent binary plug-in to the Mozilla Firefox plugins directory.

1.1 Installing the Media Engine Component

2.0 Introduction to the AFE Media Engine

The chrome and media engine are packaged as separate .XPIs. While the chrome is the same on all platforms, a different video plug-in is required for each operating system and CPU architecture. The two main components to AFE are as follows:

  1. The Chrome:  Refer to the XUL Core documentation for further details.
    Chrome is also known as the 'Core' XPI. This is a Mozilla XUL (XML User Interface Language) application providing the overall video browsing user interface. This shows the basic clip list, transport controls and media display. The chrome is platform-independent, and it registers itself with Mozilla as the handler for application/x-annodex, and it embeds the video plug-in. The AFE Chrome also handles the parsing of CMML metadata and clip information implementing JavaScript's XML to RDF parser functions.
     
  2. Media Engine
    The media engine is a Mozilla plug-in used to do the playback and rendering of the audio or video. Currently, we use the VLC Mozilla plug-in as the media engine on all platforms, though this may change in the future. The video plug-in is native code specific for each of the supported platforms on Windows, Linux (x86 and PowerPC) and MacOS. Each media engine registers with Mozilla as an application/x-annodex-player plug-in; e.g. the VLC media engine is registered as application/x-annodex-vlc-viewer-plugin.

2.1 Building Concepts of the AFE Media Engine

A "media engine" is a Mozilla plug-in which is responsible for the actual playback of the video or audio file. Currently, VLC is used on all three supported platforms (Windows, Linux, Mac OS X) as the media engine. You will need to compile a customised VLC Mozilla plug-in to get the Firefox extension to work correctly. Without a media engine, you can download and view the CMML content in the Table of Contents list for the video (as this is handled by the chrome XPI), but can't actually watch the video. To start building the VLC media engine, it requires three main components:

  1. The VLC sources, which must reside in a src/ directory here. This should automagically be checked out thanks to Subversion's svn:externals feature.
     
  2. The Gecko (Mozilla) SDK, which includes some header files and utility programs required for the building the VLC Mozilla plug-in. The Gecko SDK is common to all the media engines, which contain Mozilla header files and tools that the plug-ins will require to compile correctly:
     
  3. The libogg, libvorbis, libtheora and libspeex libraries installed in a location that VLC's ./configure script will pick them up. If you are building a one-click XPI installation package, you should ensure that these libraries are build statically only, and not shared (i.e. configure them with --enable-static --disable-shared). If you have all three requirements, the build process should be fairly simple by running the commands below.

    ./Patch
    cd build
    ./Build

2.1.1 Precompiled Gecko SDKs for Windows and Mac OS X

Obtaining the correct Gecko SDK for Windows and Mac OS X can be a bit of a pain (especially for Mac OS X, since no binary distribution of the Gecko SDK is available for that platform from mozilla.org). To make life a bit easier, we provide prebuilt Gecko SDK binaries for those platforms for you.

Follow the steps in Section 3 for your specific platform will patch and build the VLC media engine, depending on the host machine. The Installer/Make (For Linux, MacOS X and Windows platforms) VLCPluginXPI.sh scripts will take the files from /MediaEngine/VLC/bin-etc and package them into an .XPI, which is placed into the Installer directory. After making the XPIs using the scripts in the Installer directory, run Firefox and open the /Installer/installtest.html file to test out whether the XPI's are detected correctly by Mozilla as Annodex extensions.

cd MediaEngine/VLC
./Patch
cd src
./Build
cd ..
./PrepareInstall.sh

2.1.2 Precompiled Gecko SDK Linking Issues on Linux

There are a few extra complications trying to compile the VLC Mozilla plug-in on Linux:

  1. As mentioned above, ensure that you only have static libraries of lib (ogg, theora, vorbis, speex) available, otherwise the end-user will require them to be installed. For package maintainers, it is fine to simply link against your distribution's packages rather than statically compiling them in.
     
  2. You may need to adjust the mozilla-config script so that the Mozilla plug-in doesn't unnecessarily link against lib (plds4, plc4, nspr4, xpcom). These are Mozilla libraries that the VLC Mozilla plug-in doesn't need to link to at all, and causes problems because almost all end-users won't have those libraries installed in the locations you have them installed in.

    For example: on Debian GNU/Linux, Mozilla Firefox uses libplds4.so that is located at /usr/lib/mozilla-firefox/, but the mozilla-dev package installs a separate libplds4.so to /usr/lib/. When the VLC Mozilla plug-in is compiled, it will link against the library found at /usr/lib/. This means that end-users who don't have themozilla-dev package installed (this is not a default package on most systems) will not be able to load the plug-in on their system properly, due to the dynamic linker throwing an error. Package maintainers can choose of two options here:
  3. Be aware that if you are building an XPI intended to work across different distributions, you will want to statically link in libstdc++ (the C++ runtime library) into the Mozilla plug-in, since different distributions use different major versions of libstdc++. Package maintainers don't have to worry about this problem, since they can link against the standard libstdc++ required by their distribution.

    For example: Ubuntu 4.10 (Warty) uses libstdc++5, Fedora Core 3 uses libstdc++6, while Ubuntu 5.04 (Hoary) uses libstdc++5 and libstdc++6. This is taken care of in the Annodex patches to VLC; if you look at the VLC_Annodex.diff file, you'll notice that mozilla/Makefile.am has been modified to force libstdc++ to be linked in statically. While this is safe to do, the method in the diff file is the best known way of doing it from current development efforts. Attempts to do this command "-Wl,-Bstatic -lstdc++ -Wl,-Bdynamic" does not work at the moment. Any experts in this field who could give advice to these issues are welcome to contact the Annodex Development Team.

    Pending Issues: There is the problem of multiple video and audio output drivers for Linux. By default, the Mozilla plug-in will use only OSS for audio output and will try Xvideo then X11 (in that order) for video output. If you wish to support other types of audio or video plug-ins (such as jack, esd), you will need to modify some #defined constants near the start of mozilla/vlcshell.cpp file, which is beyond the scope of current developments.

3.0 Compilation of the AFE Media Engine

This section steps through in the process of compilation, to address issues covered in the relevant sections for each operating system when compiled using current system environments. It is impossible to cover all the possible scenarios of errors resulting in the compilation process, but rather serves as a general overview of the process to get you started in the VLC open source environment. The most likely issue you will encounter while compiling is the issue of ensuring the correct dependencies are setup correctly, to the compilation flags used in the makefile. In such circumstances, the first point of reference is to look at the Further Information section below. If the entire compilation process is successful, the Mozilla plug-in will be located in the /src directory under the following file types/linked libraries:

Development Advice: The subversion patch script was developed for VLC version 0.8.2. Any other versions of VLC may run into version conflicts in the patch process. These issues will be resolved in the upcoming development of the VLC Media Engine plug-in for version 0.8.5. The development was made on the following operating environments:

Further Information: For VLC Plug-in development, the following websites are recommended:

3.1 Building the AFE Media Engine in Cygwin

3.1.1 Cygwin Configuration Changes for VLC Version 0.8.2:

  1. Ensure the following categories are all selected when using Cygwin for basic system dependencies to be fulfilled:
    • Admin
    • Archive
    • Base
    • Devel
    • Libs
    • Mingw
    • System
    • Utils.
  2. Setup the latest win32 pre-assembled libraries for VLC by extracting the archive and copying it to the root cygwin directory.
    The contrib tarball archive can be found in the following location: http://download.videolan.org/pub/testing/win32/
     
  3. Checkout the 0.82 tag of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/tags/0.8.2
     
  4. Using the VLC 0.82 patch located at /Docs/Patches/VLC-0.82.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.82.patch
     
  5. Modify the file {Installation source of VLC}/Configure.cygwin as follows:

    The developer is required to changes these lines to make it the absolute path of the location of the dependencies and Gecko SDK:
    Line 9: export CPPFLAGS="-I/usr/win32/include -I/usr/win32/include/ebml -I/cygdrive/C/Cygwin/usr/win32/gecko-sdk/include"
    Line 10: "-L/usr/win32/lib -L/cygdrive/C/Cygwin/usr/win32/gecko-sdk/lib"
    Line 12: XPIDL=/cygdrive/C/Cygwin/usr/win32/gecko-sdk/bin/xpidl.exe
    Line 13: XPIDL_INCL="-Ic:\Cygwin/usr/win32/gecko-sdk/idl"
    Line 73: --enable-mozilla --with-mozilla-sdk-path=/cygdrive/C/Cygwin/usr/win32/gecko-sdk \
     
  6. Modify the file {Installation source of VLC}/mozilla/Makefile.am as follows to make it the absolute path of vlcintf.idl:
    Line 10: VLCIDL_FILE = "c:\{Path of VLC Source}/mozilla/vlcintf.idl"
     
  7. Copy the file {Installation source of VLC}/nsISupportsUtils.h into the Contribution dependency directory /usr/win32/gecko-sdk/include/
     
  8. In the directory where you have checked out the source, run the following script ./Build

3.1.2 Cygwin Configuration Changes for VLC Version 0.8.5:

  1. Setup the latest win32 pre-assembled libraries for VLC by extracting the archive and copying it to the root cygwin directory.
    The contrib tarball archive can be found in the following location: http://download.videolan.org/pub/testing/win32/
     
  2. Checkout the 0.85 tag of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/tags/0.8.5
     
  3. Using the VLC 0.85 patch located at /Docs/Patches/VLC-0.85.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.85.patch
     
  4. Make the following changes to the relevant files:
  5. In the directory where you have checked out the source, run the following script ./Build

3.1.3 Cygwin Configuration Changes for VLC Version 0.8.6:

  1. Setup the latest win32 pre-assembled libraries for VLC by extracting the archive and copying it to the root cygwin directory.
    The contrib tarball archive can be found in the following location: http://download.videolan.org/pub/testing/win32/
    Download the stable Gecko SDK (gecko-sdk-i586-pc-msvc.zip) from Mozilla.org: http://ftp.mozilla.org/pub/mozilla.org/mozilla/nightly/latest-trunk/; Copy over the contrib Gecko SDK in /usr/win32
     
  2. Checkout the 0.86 trunk of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/trunk
     
  3. Using the VLC 0.86 patch located at /Docs/Patches/VLC-0.86.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.86.patch
     
  4. Make the following changes to the relevant files:
  5. In the directory where you have checked out the source, run the following script ./Build

3.2 Building the AFE Media Engine in Mac OS X

3.2.1 MacOS Configuration Changes for VLC Version 0.8.2:

  1. Setup the contribution dependencies for MacOS. As there are no binary dependencies that can be downloaded, they will need to be compiled from source at the following location in the source directory:

    /MediaEngines/VLC/src/extras/contrib/
    Enter the command ./bootstrap and ./make src
     
  2. Checkout the 0.82 tag of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/tags/0.8.2
     
  3. Using the VLC 0.82 patch located at /Docs/Patches/VLC-0.82.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.82.patch
     
  4. Modify the file {Installation source of VLC}/Configure.darwin as follows:

    The developer is required to changes these lines to make it the absolute path of the location of the dependencies and Gecko SDK:
    Line 9: CPPFLAGS="-I`pwd`/contrib/include -I`pwd`/gecko-sdk/include"
    Line 10: LDFLAGS="-L`pwd`/contrib/lib -L`pwd`/gecko-sdk/lib"
    Line 11: PATH="/bin:/usr/bin:`pwd`/contrib/bin:`pwd`/gecko-sdk/bin"
    Line 12: PKG_CONFIG_PATH="`pwd`/src/contrib/lib/pkgconfig"
    Line 15: XPIDL=`pwd`/gecko-sdk/bin/xpidl
    Line 16: XPIDL_INCL="-I`pwd`/gecko-sdk/idl"
    Line 74: --enable-mozilla --with-mozilla-sdk-path=`pwd`/gecko-sdk \
     
  5. Modify the file {Installation source of VLC}/mozilla/Makefile.am as follows to make it the absolute path of the Mozilla SDK resources:
    Line 10: XPIDL=`pwd`/gecko-sdk/bin/xpidl
     
  6. In the directory where you have checked out the source, run the following script ./Build

3.2.2 MacOS Configuration Changes for VLC Version 0.8.5:

  1. Checkout the 0.85 tag of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/tags/0.8.5
     
  2. Using the VLC 0.85 patch located at /Docs/Patches/VLC-0.85.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.85.patch
     
  3. Compile the necessary dependencies from source in your VLC Source directory, which should take between 1 to 4 hours depending to complete: {Path of VLC Source}/extras/contrib
    Do not bootstrap, but modify the config.mak file to suit your system architecture (Universal x86 compilation can run into various issues, but this is beyond the scope of the documentation) and MacOS Developer SDK version.
    Run the command: make src
     
  4. Make the following changes to the relevant files:
  5. In the directory where you have checked out the source, run the following script ./Build
     
  6. Additionally, when it comes to packaging the plug-in, you must ensure it is saved in the default Firefox plugins directory, not the user profile's directory. The directory is located at /Library/Internet Plug-Ins

3.2.3 MacOS Configuration Changes for VLC Version 0.8.6:

  1. Checkout the 0.86 trunk of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/trunk
     
  2. Using the VLC 0.86 patch located at /Docs/Patches/VLC-0.86.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.86.patch
     
  3. Compile the necessary dependencies from source in your VLC Source directory, which should take between 1 to 4 hours depending to complete: {Path of VLC Source}/extras/contrib
    Enter the command ./bootstrap and ./make src
     
  4. In the directory where you have checked out the source, run the following script ./Build
     
  5. Additionally, when it comes to packaging the plug-in, you must ensure it is saved in the default Firefox plugins directory, not the user profile's directory. The directory is located at /Library/Internet Plug-Ins

3.2.4 MacOS XPI Packaging Configuration Changes:

In addition to compilation, to package a plug-in for MacOS systems, there is an additional requirement due to the nature of the MacOS HFS file system. More specifically, the resource forks need to be serialised after the archive is decompressed using a specific decompression utility. Using Mozilla's XPI installer functions will fail to serialise the resource fork, and thus the metadata will not be included to recognise the plug-in within Mozilla Firefox. The install script uses BOMArchiveHelper for the file decompression. Follow the instructions below to make the XPI package for MacOS:

Note: The file /Modified-MacOS/MacOS Installer.zip implements the MacOS packager as an alternative installation process to Mozilla's XPInstall functionality. Project files and installer scripts are self-explanatory if required by the developer.

  1. Create a temporary directory to create the XPI file (in this instruction, the temporary directory will be called /Users/root/tmp)
     
  2. Archive the /MediaEngines/VLC/src/mozilla/VLC Plugin.plugin into a zip file, and rename the zip file back to VLC Plugin.plugin.
    Note: This is to allow Mozilla to install the plug-in file into a temporary directory to extract the file using a separate utility.
     
  3. Create a new directory in the temporary directory called src (i.e. /Users/root/tmp/src/). Copy VLC Plugin.plugin into the src subdirectory.
     
  4. Extract install.js and moveMacPlugins.sh from /Modified-MacOS/MacOS XPInstall.zip into the temporary directory (i.e. /Users/root/tmp/). Make sure execution permissions are made for the script file.
     
  5. Archive the items of the temporary directory, not the temporary directory itself (i.e. archive install.js, moveMacPlugins.sh and the src directory together.) Rename the archive to the AnnodexViewer-MacOS.xpi

3.3 Building the AFE Media Engine in GNU/Linux

3.3.1 Linux Configuration Changes for VLC Version 0.8.2:

  1. Make sure the following dependencies are installed (Development using Ubuntu Dapper 0.6):
  2. Download the stable Gecko SDK from Mozilla.org at this location: http://ftp.mozilla.org/pub/mozilla.org/mozilla/releases/mozilla1.7rc3/gecko-sdk-i686-pc-linux-gnu-1.7rc3.tar.gz.
    It is recommended to extract the file within the same relative path of the source directory (/MediaEngines/VLC/).
     
  3. Checkout the 0.82 tag of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/tags/0.8.2
     
  4. Using the VLC 0.82 patch located at /Docs/Patches/VLC-0.82.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.82.patch
     
  5. Modify the file {Installation source of VLC}/Configure.linux as follows:

    The developer is required to changes these lines to make it the absolute path of the location of the dependencies and Gecko SDK:
    Line 19: CPPFLAGS="-I/usr/include -I/usr/include/ebml -I`pwd`/gecko-sdk/include"
    Line 20: LDFLAGS="-L/usr/lib -L`pwd`/gecko-sdk/lib"
    Line 21: XPIDL=`pwd`/gecko-sdk/bin/xpidl
    Line 22: XPIDL_INCL="-I`pwd`/gecko-sdk/idl"
    Line 81: --enable-mozilla --with-mozilla-sdk-path=`pwd`/gecko-sdk \
     
  6. In the directory where you have checked out the source, run the following script ./Build

3.3.2 Linux Configuration Changes for VLC Version 0.8.5:

  1. Download the stable Gecko SDK from Mozilla.org at this location: http://ftp.mozilla.org/pub/mozilla.org/mozilla/nightly/latest-trunk/
    It is recommended to extract the file within the same relative path of the source directory, but not essential.
     
  2. Checkout the 0.85 tag of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/tags/0.8.5
     
  3. Using the VLC 0.85 patch located at /Docs/Patches/VLC-0.85.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.85.patch
     
  4. Make the following changes to the relevant files:
  5. In the directory where you have checked out the source, run the following script ./Build
     
  6. Additionally, when it comes to packaging the plug-in, you must ensure it is saved in the default Firefox plugins directory, not the user profile's directory.
    By default, the directory is located at /usr/lib/firefox/plugins

3.3.3 Linux Configuration Changes for VLC Version 0.8.6:

  1. Checkout the 0.86 trunk of the VLC source on Subversion at the following location to a directory of your choice: svn://svn.videolan.org/vlc/trunk
     
  2. Download the stable Gecko SDK (gecko-sdk-i686-pc-linux-gnu.tar.gz) from Mozilla.org: http://ftp.mozilla.org/pub/mozilla.org/mozilla/nightly/latest-trunk/
    Decompress the Gecko SDK and install in the same directory as the source you have checked out. Ensure you have libIDL 0.6 in your /usr/lib directory.
     
  3. Using the VLC 0.86 patch located at /Docs/Patches/VLC-0.86.patch, copy the diff file to the VLC source you checked out and patch the source as follows:
    patch -p0 "$@" < VLC-0.86.patch
     
  4. In the directory where you have checked out the source, run the following script ./Build

4.0 AFE Media Engine Directory Structure

Directory Contents
/trunk/Core Core XPI Files: Refer to XUL Core development documentation for further information.
/trunk/Docs Documentation directory, in which this document and release notes reside.
/trunk/Installer Linux Shell Scripts for compilation of files into a XPI Firefox Plug-in Installer; and Windows Installer Batch file. Refer to section 1.0 for details.
/trunk/MediaEngines Contents of VLC Media Engine plug-in for Mozilla Firefox Browser.
/trunk/MediaEngines/GeckoSDK/gecko-sdk-win32 Pre-compiled Gecko SDK for Windows 32-bit systems. Refer to section 2.1.1 for information.
/trunk/MediaEngines/GeckoSDK/gecko-sdk-macosx Pre-compiled Gecko SDK for MacOS X systems. Refer to section 2.1.1 for information.
/trunk/MediaEngines/VLC/bin-linux-i386 Pre-compiled Linux plug-in of VLC Media Engine for Annodex media playback. (Intel x86 CPU architecture-based systems)
/trunk/MediaEngines/VLC/bin-linux-ppc Pre-compiled Linux plug-in of VLC Media Engine for Annodex media playback. (IBM PowerPC architecture-based systems)
/trunk/MediaEngines/VLC/bin-macosx Pre-compiled Mac OS version X and above plug-in of VLC Media Engine for Annodex media playback.
/trunk/MediaEnginesVLC/bin-win32 Pre-compiled Windows 32-bit plug-in of VLC Media Engine for Annodex media playback.
/trunk/MediaEngines/VLC/builc Cygwin Configuration files for setting up environmental paths for VLC media engine's supported platforms.
/trunk/MediaEngines/VLC/src Source code of VLC Media Engine. Refer to section 3.0 for information.