Accurate Player Documentation

Introduction

Accurate Player is a HTML5 media player, with frame accurate playback of assets using progressive download. It can be integrated into any web-based environment, through its rich JavaScript API on the front-end or its powerful REST API on the back-end. Accurate Player is designed to connect and integrate with any existing storage, such as databases (e.g PostgreSQL, MongoDB, DynamoDB), physical storage locations (e.g S3 buckets, network drives, Dropbox folders) or fully-fledged MAM systems.

Frame accuracy

Accurate Player is frame accurate, which means that it can address individual frames for an asset. It guarantees that the displayed SMPTE time code of the player always shows the exact frame you are viewing. It is possible to step forwards or backwards frame-by-frame or seek to an exact frame. Frame accuracy is paramount for a broadcast quality media player, such as Accurate Player, to enable management of metadata such as markers & subtitles on a frame level. Exporting markers for segmentation to a playout system, setting breakpoint markers for commercial breaks or marking in and out points for stitching all require the exact frame to be addressed.

Accurate Player provides frame accuracy across all the four major browsers, with the recommended proxy format, for all possible frame rates, both with or without drop frame time codes.

Player architecture

In the Accurate Player ecosystem there are more components than just the player, this section describes each more in detail. The Accurate Player ecosystem follows a very modular structure of micro services, using a container-based approach, with each component running inside a Docker container. Docker networks are used to restrict and provide communication through containers. Note that although this is the default, recommended deployment structure, it can be customized to each specific installation site.

This document describes a typical single installation, but it is possible to have multiple simultaneous systems installed, or any combination thereof. For example, one player can be installed and talk to two different systems through configuration. Multiple players can also be deployed, with different options, all talking to the same back-end system.

System overview

The above system diagram gives an overview of the system. The highlighted part are the components which make out Accurate Player. Each component is explained more in detail below.

Front-end application

The front-end application of Accurate Player is a package consisting of JavaScript, HTML and static content such as images, style sheets & fonts. This package is optimized and minimized in the build process. The package is ready to deploy using a web server that hosts static content, just as you would deploy a typical web page. There is no limitation on where the deployment is done, it can either be locally, publicly on your company's web server or in a cloud-solution like Amazon AWS or Microsoft Azure.

The application is developed using Angular 5, with code written in TypeScript (transpiled to JavaScript) and style sheets written in SASS. The Bootstrap framework is used to ensure a responsive design on all screen-sizes and devices.

Since Accurate Player uses some of the latest HTML5 API's, there are some limitations on the browser of choice. Recommended browsers are the latest stable version of Google Chrome, Mozilla Firefox, Apple Safari or Microsoft Edge.

JavaScript API

The front-end application exposes a JavaScript API that can be used when integrating Accurate Player into existing web applications, or if you want to build customized GUI components on top of the player. See below for details regarding the JavaScript API and how to use it.

Back-end application

The front-end application connects to a back-end component, which is a middleware that is responsible for connecting to the storage. The back-end application exposes a powerful REST API that is used by the front-end application and may be used by external services as well.

The back-end application is written in Java 8 and is typically deployed using WildFly Swarm as a fat-jar. It can also be deployed as a WAR/EAR file in any Java application server of choice. Communication between the front-end and back-end is JSON-based, but the REST API also provides XML support.

The back-end application can either run on the same server as the front-end, or on different servers. Hybrid solutions are possible with the front-end running in the cloud and the back-end locally, behind firewalls close to the internal storage. As long as there is a connection possible between the two components, any deployment structure is possible. The URL to the back-end component is configured as a property in the front-end application during installation.

REST API

Attached to the back-end application is a powerful REST API, which the front-end application uses for all communication. This API can also be used by external services, such as transcoders, ingest systems and QC engines. Custom applications can be built on top of the Accurate Player ecosystem and interact with the player through this REST API.

For example, the REST API can be used to create assets, ingest proxies for existing assets, adding metadata to assets, creating markers & subtitles and so on. For the complete REST API documentation see the link below.

Connector/adapter

The back-end component connects to a storage, which is usually a database or a MAM system, through a connector or adapter system. This adapter system is designed to allow flexibility for integration with any possible system. Although a powerful database like PostgreSQL or MongoDB is preferred, the storage can be a file, an Excel sheet or a Dropbox folder.

Accurate Player provides ready-to-use connectors for some of the most popular databases and MAM systems, providing plug-and-play deployment to these systems. For other systems, a customization phase is required before installation, where a connector is implemented for the system of choice.

Web server

Accurate Player streams assets through progressive download from a web server. Since HTML5 is used, there are limitations on what video codecs are usable. The browser of choice has to support decoding the video codec of the proxy file streamed to the player.

Transcoder

Accurate Player ships with its own transcoder, based on ffmpeg, that can be used to transcode video and audio files. Transcode profiles or presets can be created to define parameters for transcoding. There are limits to the available codecs for the transcoder. The transcoder is included for free and can be skipped, in favour for more powerful transcoders that you have access to.

Ingest system

Accurate Player also ships with its own ingest system which uses the player transcoder. Through this ingest system, you can configure drop and target folders, and define transcode presets for processing incoming files. Drop folders are file folders or network drives that listens for incoming files. Once a file has been detected, a transcode job will start based on the defined transcode preset (no transcode is also possible). The ingest system communicates with the back-end REST API and creates a new asset, adds a video file, adds still frames and populates asset metadata. Files are moved to target folders once ingested, which can then be mapped to web server locations, accessible by the player.

The ingest system is included for free with Accurate Player, and can also be skipped in favour for a more powerful system. You can also write your own, using the REST API of the back-end to add assets & populate metadata.

By default the ingest system uses a recommended proxy format for Accurate Player, which works well across all major browsers:

Data models

Accurate Player defines its own internal data models, or data entities which is used by all components. The REST API is designed around these models. When an adapter is implemented from existing data models, a conversion to the format required by Accurate Player is needed. This section gives an overview of each model, describing them more in detail.

Asset

An asset represents the top level of a collection of files. Each asset is uniquely represented by an id, and consists of a collection of files. Typical files for an asset could be the original, high-resolution video and a number of low-resolution video proxies. There could be subtitles in different formats, external discrete audio tracks and other associated miscellaneous files such as PDF-files or text documents.

Example JSON of asset structure without files included, taken from the REST API:

{
  "id": "1",
  "title":"Sintel",
  "creationDate": "2018-01-02T08:57:08.752",
  "updateDate": "2018-01-02T08:57:18.503",
  "version": 1,
  "videoFiles": [],
  "subtitleFiles": [],
  "markers": [],
  "metadata": [],
  "stillFrames": [],
  "representativeThumbnail": {
    "version": 0,
    "id": "0",
    "fileName": "thumbnails/sintel.jpg",
    "timeCode": {
      "frame": 60,
      "numerator": 25,
      "denominator": 1
    },
    "url": "https://apl.codmill.se/media/thumbnails/sintel.jpg",
    "type": "THUMBNAIL",
    "storageId": "0",
    "resolution": {
      "width": 320,
      "height": 136
    }
  }
}

File

A file is just that, a file associated with an asset. There are support for different types of files, such as video, audio, subtitle or other files. The file representation here does not need that the actual content of the file is stored inside the database. For example, it could be a just a file name mapped to a physical file on a network drive, for example a proxy video on a S3 bucket.

Storage

A storage is a representation of a folder or a network drive where files are kept. There needs to be at least one storage mapped in the system, to be able to save any files. If there is only one storage created, it will be used per default for all operations that require a storage. In case of multiple storages, the unique storage id needs to be given for these operations.

For example, there might be two storages, one S3 bucket in Europe and one in USA. In the database there exists two storage mappings, providing base URL's to both these buckets. Depending on what asset is loaded, it is either streamed through Europe or USA.

Example storage configuration:

{
  "id":"1",
  "name":"Default Storage",
  "uri":"https://my.domain/media/",
  "path":"/opt/storage/media",
}

The above configuration maps a local file path /opt/storage/media into a URL https://my.domain/media/ accessible by the player. This is done by a web server, such as nginx, Apache or Microsoft IIS.

Asset metadata

Asset metadata represents time coded, general metadata for the entire asset. This is metadata shared between all files for the asset.

File metadata

File metadata is specific to each file, and is of the form key:value, which is non-timecoded.

Marker

A marker represents a time coded event, or annotation. It is associated with an asset, so all files for an asset share the same markers. Markers have a start and end timecode, together with metadata fields depending on what type of data is annotated. Every marker has a name and a description.

Example marker structure:

{
  "id": "1",
  "timeSpan": {
    "start": {
      "frame": 2,
      "numerator": 25,
      "denominator": 1
    },
    "end": {
      "frame": 18,
      "numerator": 25,
      "denominator": 1
    }
  },
  "name": "Compliance Event",
  "description": "f5d569f2-f9dc-40f4-ad31-f304c783559c",
  "tags": [{
    "id": "1",
    "tag": "NTSC",
    "colour": "#1182a3"
  }, {
    "id": "2",
    "tag": "PAL",
    "colour": "#f0Ddeb"
  }],
  "colour": "#3529ce"
}

Marker tag

Marker tags are used as a category for markers, to group specific markers and allow filtering and search. A tag is simply a text field used to give a category for a marker, and is fully customizable by the user.

Still frame

A still frame of an asset represents a screen capture at a single frame. It has a resolution with dimensions, a type (low-resolution thumbnail, high-resolution poster), file name, and an associated storage where it is stored.

Quick start

To embed Accurate Player into your HTML application you will need to include a client API JavaScript and load the player through an iframe. The example below illustrates how to embed an instance of Accurate Player running at http://localhost:80.

index.html

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Accurate Player app</title>

  <!-- Load the Accurate Player JavaScript API file -->
  <script type="text/javascript" src='ap-js-api.js'></script>

  <!-- Your webapp file -->
  <script type="text/javascript" src='my-webapp.js'></script>
</head>
<body>
    <!-- Include iframe with a unique id "player" -->
    <iframe id="player" src="http://localhost:80/" class="player" allowfullscreen="true" frameBorder="0"></iframe>

    <!-- Buttons triggering player functions -->
    <button onclick="loadAsset('1')">loadAsset('1')</button>
    <button onclick="toggleFullscreen()">toggleFullscreen()</button>
    <button onclick="play()">play()</button>
    <button onclick="pause()">pause()</button>
    <button onclick="getCurrentTime()">getCurrentTime()</button>
    <button onclick="getCurrentFrame()">getCurrentFrame()</button>
</body>
</html>

The Accurate Player JavaScript API is loaded by including its js-file:

<script type="text/javascript" src='ap-js-api.js'></script>

The above code creates an iframe with the player, and creates some buttons for controlling it. These buttons are bound to methods by the onclick attribute. The code for my-webapp.js is seen below, which contains the JavaScript code for interacting with the player.

my-webapp.js

let player;

window.onload = () => {
  // Instantiate new player
  player = new ApJsApi("player");

  // Error callback
  player.onError = error => {
    console.error("Error", `${error.section}.${error.operation} failed! ${error.message}`);
  };

  // Register listener for asset loadstart
  player.registerStatusListener("asset", "loadstart", assetId => {
    console.log("Asset load start", assetId);
  });

  // Asset is loaded
  player.registerStatusListener("asset", "loaded", title => {
    console.log("Asset loaded", title);
  });

  // Loading asset failed
  player.registerStatusListener("asset", "loadfailed", message => {
    console.log("Asset load failed", message);
  });

  // Player volume changed
  player.registerStatusListener("playback", "volumechanged", volume => {
    console.log("Playback volume changed", volume);
  });

  // Player has loaded metadata
  player.registerStatusListener("playback", "loadedmetadata", assetId => {
    console.log("Playback loaded metadata", assetId);
  });

  player.registerStatusListener("playback", "playbackratechanged", rate => {
      console.log("Playback rate changed", rate);
  });
};

window.loadAsset = assetId => {
  player.loadAsset(assetId);
}

window.toggleFullscreen = () => {
  player.toggleFullscreen();
}

window.play = () => {
  player.play();
}

window.pause = () => {
  player.pause();
}

window.getCurrentTime = async () => {
  const currentTime = await player.getCurrentTimeAsync();
  console.log("Current Time", currentTime);
}

window.getCurrentFrame = async () => {
  const currentFrame = await player.getCurrentFrameAsync();
  console.log("Current Frame", currentFrame);
}

Player JavaScript API

NOTE: The JavaScript player API is still being developed and is subject to change. Not everything is yet documented.

Event listeners

Event listeners, or status listeners, are created to listen to specific events happening from the player. For example, reacting to when an asset is loaded, playback is started or the player is muted. These events follow a pattern of section and operation which unique identifies the event.

player.registerStatusListener("asset", "loadstart", assetId => {
  console.log("Asset Load Start", assetId);
});

The above example registers a status listener for section asset and operation loadstart. It returns the id of the asset loaded.

To unregister, simply call the matching unregisterStatusListener method.

player.unregisterStatusListener('asset', 'loadasset');

The following events are available as status listeners. The API is under development and more events are added continuously.

Playback methods

loadAsset(assetId: string, fileId?: string)

player.loadAsset('1', '2');

Loads an asset by entering the assetId. Optionally, a file id can be given as well, to load a specific video proxy for the given asset.

play()

player.play();

Starts playback, if not already started.

pause()

player.pause();

Pauses playback, if not already paused.

togglePlayPause()

player.togglePlayPause()

Toggles between play and pause. If playing, pauses. If paused, starts playing.

setVolume(volume: number)

player.setVolume('0.2');

Changes the volume, input is a number between 0 (muted) and 1 (full volume);

setMute(muted: boolean)

player.setMute(true);

Sets the mute flag on the player. The previous volume is restored when disabling mute in the player.

setPlaybackRate(rate: number)

player.setPlaybackRate(3);

Sets the playback rate of the player. Can also be negative.

setRelativePlaybackRate(rate: number)

player.setRelativePlaybackRate(5);

Increases or decreases the playback rate relative to current rate. Negative number will decrease the playback rate.

setFullscreen(fs: boolean)

player.setFullscreen(true);

Enables or disables full screen mode.

loadProxy(proxy: string)

player.loadProxy('mp4');

When an asset is loaded, changes to a new proxy for the same asset. Input is the unique identifier for the proxy. Only applicable if an asset is playing.

seekToFrame(frame: number, pause?: boolean)

player.seekToFrame(4621);

Seeks to a specific frame. Pause player after seeking if pause is true (optional parameter).

seekToMs(ms: number, pause?: boolean)

player.seekToMs(3000);

Seek to a specific millisecond. Pause player after seeking if pause is true (optional parameter).

seekToTime(second: number, pause?: boolean)

player.seekToTime(5, true);

Seek to a specific second. Pause player after seeking if pause is true (optional parameter).

seekToPercent(percent: number, pause?: boolean)

player.seekToPercent(52, false);

Seek to a specific percent of time for the current loaded asset. To seek to start use 0, to seek to end 100. Pause player after seeking if pause is true (optional parameter).

stepFrame(frame: number, pause?: boolean)

player.stepFrame(5);
player.stepFrame(-1, true);

Steps a number of frames relative to the current frame. Can be a negative number to step backwards. To step one frame at a time, use 1 or -1. Pause player after seeking if pause is true (optional parameter).

stepMs(ms: number, pause?: boolean)

player.stepMs(-3000, false);

Steps a number of milliseconds relative to the current time. Can be a negative number to step backwards. Pause player after seeking if pause is true (optional parameter).

stepTime(second: number, pause?: boolean)

player.stepTime(3, true);

Steps a number of seconds relative to the current time. Can be a negative number to step backwards. Pause player after seeking if pause is true (optional parameter).

stepPercent(percent: number, pause?: boolean)

player.stepPercent(10);

Steps a number of seconds matching the percent value, relative to the current time. Can be a negative number to step backwards. Pause player after seeking if pause is true (optional parameter).

Player state

Getter methods query the player API for current state, such as volume and full screen status. All these methods are asynchronous, meaning that a return value of T in the definition below is actually of the type Promise<T>. See MDN web docs for more information about promises.

Example for retrieving the current time, using promises:

getCurrentTime() {
  player.getCurrentTime().then(currentTime => {
    console.log('Current time', currentTime);
  });
}

This can be simplified using the async function:

async getCurrentTime() {
  const currentTime = await player.getCurrentTime();
  console.log("Current time", currentTime);
}

Currently available state methods:

getVolume(): number

player.getVolume();
-> 0.2

Retrieves the current volume. A number between 0 and 1.

getPlaybackRate(): number

player.getPlaybackRate();
-> -3

Retrieves the current playback rate.

getCurrentTime(): number

player.getCurrentTime();
-> 12.34567

Retrieves the current time of the playing file.

getCurrentFrame(): number

player.getCurrentFrame();
-> 345

Retrieves the current frame of the playing file.

getCurrentPercent(): number

player.getCurrentPercent();
-> 34.12

Retrieves the current percent played of the playing file.

getCurrentTimecode(): string

player.getCurrentTimeCode();
-> 00:02:03;23

Retrieves the SMTPE timecode of the current playing file.

getDurationTime(): number

player.getDurationTime();
-> 2345.23116

Retrieves the total duration of the current playing file in seconds.

getDurationFrame(): number

player.getDurationFrame();
-> 234

Retrieves the total duration of the current playing file in frames.

getDurationTimecode(): string

player.getDurationTimecode();
-> 02:01:34;22

Retrieves the total duration of the current playing file in SMPTE timecode.

getFrameRate(): number

player.getFrameRate();
-> 25

Retrieves the frame rate of the current playing file.

isMuted(): boolean

player.setMute(true);
player.isMuted();
-> true

Checks if the player is muted.

isFullscreen(): boolean

player.setFullScreen(true);
player.isFullscreen();
-> true

Checks if the player is in fullscreen mode.

isPlaying(): boolean

player.pause();
player.isPlaying();
-> false

Checks if the player is currently playing.

isPaused(): boolean

player.play();
player.isPaused();
-> false

Checks if the player is currently paused.

Points

setInFrame(frame: number, seekTo?: boolean, pause?: boolean)

player.setInFrame(12, true, false);

Sets the in point to given frame. Optionally seek to the given frame if seekTo is true. If pause is set to true, pause player after creating in point. If both seekTo and pause are set to true, player will first seek and then pause.

setOutFrame(frame: number, seekTo?: boolean, pause?: boolean)

player.setOutFrame(345, false);

Sets the out point to given frame. Optionally seek to the given frame if seekTo is true. If pause is set to true, pause player after creating out point. If both seekTo and pause are set to true, player will first seek and then pause.

setInPoint(pause?: boolean)

player.setInPoint(false);

Sets the in point to the current frame. If pause is set to true, pause player after creating in point.

setOutPoint(pause?: boolean)

player.setOutPoint();

Sets the out point to the current frame. If pause is set to true, pause player after creating out point.

seekToIn(pause?: boolean)

player.seekToIn(true);

Seeks to the current in point. If pause is set to true, pause player after seeking.

seekToOut(pause?: boolean)

player.seekToOut();

Seeks to the current out point. If pause is set to true, pause player after seeking.

clearInPoint()

player.clearInPoint();

Clears the current in point.

clearOutPoint()

player.clearOutPoint();

Clears the current out point.

clearPoints()

player.clearPoints();

Clears both the current in and out point.

Subtitles

enableSubtitle(identifier: string)

player.enableSubtitle('polish');

Enables the subtitle with given identifier for the current playing file.

disableSubtitles()

player.disableSubtitles();

Disables (clears) all current selected subtitles.

Markers

showNewMarkerForm()

player.showNewMarkerForm();

Shows the new marker form.

showMarkerTable()

player.showMarkerTable();

Shows the marker table.

showAssetPage()

player.showAssetPage();

Navigates to the asset page in the player.

Playlists

addToPlaylist(assetId: string)

player.addtoPlaylist('2');

Adds an asset to the current playlist.

removeFromPlaylist(assetId: string)

player.removeFromPlaylist('3');

Removes an asset from the current playlist.

clearPlaylist()

player.clearPlaylist();

Clears all assets from the current playlist.

nextInPlaylist()

player.nextInPlayList();

Plays the next asset in the current playlist.

previousInPlaylist()

player.previousInPlaylist();

Plays the previous asset in the current playlist.