Transcoders

When you import items the Vidispine transcoder will be used to detect the type of media that is being imported and, of course, to transcode the media to any formats that you have requested.

The common operations performed by the transcoder are:

  • Media shape deduction
  • Transcoding
  • Sequence rendering
  • Partial file extraction
  • XMP extraction and rewrite

The Vidispine transcoder has a REST API that Vidispine uses to perform the above operations. This API is not described in this document, as it typically should not be accessed directly.

Adding a transcoder

Add a transcoder by creating a new transcoder resource. The resource document should contain information on how to reach the transcoder and what storages the transcoder has direct access to.

POST /resource/
Content-Type: application/xml

<?xml version="1.0"?>
<ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <transcoder>
    <url>http://transcoder.example.com:8888/</url>
    <directAccess>
      <filter>file:/srv/media/.*</filter>
    </directAccess>
  </transcoder>
</ResourceDocument>

Vidispine checks the status of transcoders continuously in the background. As such, if the configuration is correct you will see that the transcoder shows up as online in a few seconds.

GET /resource/VX-7
<?xml version="1.0"?>
<ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <id>VX-7</id>
  <transcoder>
    <url>http://transcoder.example.com:8888/</url>
    <directAccess>
      <filter>file:/srv/media/.*</filter>
    </directAccess>
    <state>ONLINE</state>
  </transcoder>
</ResourceDocument>

The Vidispine installer will by default install and configure a transcoder in Vidispine for you, so this step is typically not needed.

Using multiple transcoders

Depending on your license, you may be allowed to use more than one transcoder. To do so, simply add additional transcoders as explained above. Vidispine will submit transcode jobs to the transcoder based on the current number of jobs being processed by the transcoder.

Vidispine will use the transcoder with the least amount of work. If a transcoder goes offline then any transcode job steps using that transcoder will fail and be retried using one of the online transcoders. If all transcoders are offline then jobs will wait for one to become available.

Warning

Do not connect multiple Vidispine installations to the same set of transcoders unless each installation has a distinct site name. If installations have the same site name then jobs from one installation may conflict with jobs from another.

How transcoders perform jobs

A transcoder will perform a job as soon as it is received, and will not schedule jobs for later execution. Vidispine, that is, the user of the transcoder is responsible for scheduling which jobs a transcoder should execute and when they should be executed.

The transcoder’s configuration file

The transcoder configuration file config.xml contains default settings for the transcoder and need typically not be modified, as the settings can instead by configured in Vidispine.

Modify the transcoder file

On a Linux system, copy the file /opt/vidispine/transcoder/config.xml to /etc/transcoder-config.xml. Then edit /etc/transcoder-config.xml. The file in /etc takes precedence over the file in /opt/vidispine.

Modify the transcoder resource

New in version 4.2.3.

On all operating systems, the transcoder configuration can be changed by adding configuration to the resource definition of the transcoder (Adding a transcoder).

Note that port of the transcoder cannot be changed in this fashion.

Modifying the transcoder configuration is this fashion takes precedence over the local configuration file and the global transcoder configuration, see below.

Modify all transcoders

It is also possible to change the configuration of all transcoders, by setting the configuration property transcoderDefaultConfiguration to the XML representation of the transcoder configuration.

Thumbnail settings

Note

The preferred way of changing the thumbnail and poster settings is by changing the appropriate values in the TranscodePresetDocument in a shape tag. For example, by changing the thumbnailResolution and thumbnailPeriod elements. The setting in shape tag have priority over the transcoder setting.

The thumbnailResolution element contains the default resolution of the thumbnails produced by the transcoder.

<a:thumbnailResolution>
   <a:width>320</a:width>
   <a:height>240</a:height>
</a:thumbnailResolution>

You can also change the thumbnailing frequency by changing thumbnailPeriod. For example, to thumbnail every 3 seconds:

<a:thumbnailPeriod>
   <a:samples>3</a:samples>
   <a:timeBase>
      <a:numerator>1</a:numerator>
      <a:denominator>1</a:denominator>
   </a:timeBase>
</a:thumbnailPeriod>

If the transcoder does not use SceneChangeDetectionPlugin, the frequency defaults to once every 10 seconds.

StatsD settings

New in version 4.2.3.

To have the transcoder send metrics to a StatsD server you can either:

  • Enable StatsD using the API, see StatsD
  • Update the transcoder configuration with the address and port of the StatsD server:
<a:statsd>
  <a:destination>
    <a:address>127.0.0.1</a:address>
    <a:port>8125</a:port>
  </a:destination>
  <a:prefix>t1</a:prefix>
</a:statsd>

The prefix element configures the prefix to use for each metric. By default this is the transcoder.

Transcoder resources settings

Path to temporary storage

Since:4.2.5

Controls where temporary files are stored. Defaults to /tmp on UNIX-like systems, or %TEMP% on Windows.

<a:tempPath>/mnt/largetemparea<a:tempPath>

Number of decoding threads

Controls the number of decoding threads. Defaults to 4 for I-frame-only formats. The actual number of threads used depends on codec. (New in 4.2.3.) Since version 4.2.3, this setting is used with more formats then before.

<a:decoderOfferThreads>8<a:decoderOfferThreads>

Number of encoding threads

Controls the number of encoding threads. Defaults to automatic setting. The actual number of threads used depends on codec.

<a:encoderThreads>8<a:encoderThreads>

Image processing

To control the memory and disk usage used by the transcoder for image processing, use the <imagemagick> element in the transcoder configuration. The most important settings are listed below, for a complete list, see http://www.imagemagick.org/script/resources.php (under environment variables, used without the MAGICK_ prefix in the transcoder configuration).

Maximum heap usage
<a:imagemagick>
  <a:key>MEMORY_LIMIT</a:key>
  <a:value>1GB</a:value>
</a:imagemagick>
Temporary work area
<a:imagemagick>
  <a:key>TEMPORARY_PATH</a:key>
  <a:value>/var/tmp</a:value>
</a:imagemagick>

(New in 4.2.5.) The default value is the value set by the general transcoder temporary path, see above. It is recommended that the tempPath setting is used, rather than the imagemagick one.

Operations overview

Zeroconf transcoders

The following is done to remove the need to configure the transcoders directly:

  • Vidispine pushes its own license to the transcoder, so that each transcoder does not need a license file of their own.
  • The transcoder returns the IP address from where the license was pushed, that is, the IP address of the application server, removing the need for explicitly configuring the reverse address, that is, where the transcoder can reach Vidispine, in most cases.
  • In addition, Vidispine generates temporary pre-authorized URIs that are used by the transcoder. This removes the need for entering any application server information in the transcoder configuration file.

Reverse address and NAT

The reverse address does not work if there is NAT or other port forwarding mechanisms between the application server and the transcoder. If so, the address to VS-EA can be overridden in the definition for the transcoder by setting the <reverseAddress> element.

<?xml version="1.0"?>
<ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <transcoder>
    <url>http://transcoder.example.com:8888/</url>
    <reverseAddress>vs.example.com</reverseAddress>
  </transcoder>
</ResourceDocument>

For GlassFish, a new HTTP listener (noauth-listener, default port is the standard API port + 9) is installed on GlassFish for communication from the transcoder to GlassFish. The rules for how the address to GlassFish is determined are as follows:

  1. If the configuration property apiNoauthUri is set, it is used for all transcoders.
  2. If the configuration property apiNoauthPort is set, it is used for together with the detected or manually set reverse address.
  3. If the port for noauth-listener can be determined (GlassFish only), it is used with the reverse address.
  4. If the port for http-listener-1 can be determined (GlassFish only), it is used with the reverse address.

For JBoss, it is currently required that at least apiNoauthPort is set.

Transcoder’s access to media

By default, the transcoder accesses non-file-schema media through the application server. This has several advantages:

  • The same user is used for all file access.
  • Possibility for support for extended file attributes and permissions.
  • Support for other file systems (URI schemes).

Streaming the media puts some extra load on the application server. Some tuning might be necessary.

The transcoder resource in Vidispine can be set up to access files directly. By adding a directAccess element to the transcoder resource, Vidispine will let the transcoder access the media directly. If no directAccess elements are present, an implicit

<directAccess>
   <filter>file:.*</filter>
</directAccess>

is added. In order to tell Vidispine that all files should go via the application server, add an

<directAccess>
   <filter>NO_MATCH</filter> <!-- dummy regular expression that does not match anything -->
</directAccess>

Growing files

For all file systems that supports read-while-write, and for container formats that are built for streaming (e.g. MXF), growing file is supported when streamed through the application server. If growing files is required to local files with the file scheme, a directAccess/NO_MATCH element as per above must be added to the resource configuration.