When you import items the Vidispine transcoder will be used to detect the type of media that is being imported and, of course, to transcode the media to any formats that you have requested.
The common operations performed by the transcoder are:
- Media shape deduction
- Sequence rendering
- Partial file extraction
- XMP extraction and rewrite
The Vidispine transcoder has a REST API that Vidispine uses to perform the above operations. This API is not described in this document, as it typically should not be accessed directly.
Add a transcoder by creating a new
transcoder resource. The resource
document should contain information on how to reach the transcoder and what
storages the transcoder has direct access to.
POST /resource/ Content-Type: application/xml <?xml version="1.0"?> <ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <transcoder> <url>http://transcoder.example.com:8888/</url> <directAccess> <filter>file:/srv/media/.*</filter> </directAccess> </transcoder> </ResourceDocument>
Vidispine checks the status of transcoders continuously in the background. As such, if the configuration is correct you will see that the transcoder shows up as online in a few seconds.
<?xml version="1.0"?> <ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <id>VX-7</id> <transcoder> <url>http://transcoder.example.com:8888/</url> <directAccess> <filter>file:/srv/media/.*</filter> </directAccess> <state>ONLINE</state> </transcoder> </ResourceDocument>
The Vidispine installer will by default install and configure a transcoder in Vidispine for you, so this step is typically not needed.
Depending on your license, you may be allowed to use more than one transcoder. To do so, simply add additional transcoders as explained above. Vidispine will submit transcode jobs to the transcoder based on the current number of jobs being processed by the transcoder.
Vidispine will use the transcoder with the least amount of work. If a transcoder goes offline then any transcode job steps using that transcoder will fail and be retried using one of the online transcoders. If all transcoders are offline then jobs will wait for one to become available.
clusterName property must be set if multiple Vidispine
installations are to share a transcoder. Each installation must have a
unique cluster name. This applies regardless if the installations have the
same site name or not.
A transcoder will perform a job as soon as it is received, and will not schedule jobs for later execution. Vidispine, that is, the user of the transcoder is responsible for scheduling which jobs a transcoder should execute and when they should be executed.
maxJob setting can be used to limit the number of Vidispine jobs that may use a specific transcoder
at the same time. If all transcoders are busy, jobs will be put on
WAITING state, with a
problem. The jobs will restart as soon as any qualified transcoder becomes available again.
<ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <id>VX-7</id> <transcoder> <url>http://transcoder.example.com:8888/</url> <maxJob>5<maxJob> </transcoder> </ResourceDocument>
A Vidispine job typically triggers multiple transcoder jobs, e.g. shape deductions and transcodes, so in the above case there may still be more than 5 running jobs on a transcoder. They will however all belong to at most 5 jobs in Vidispine.
The above setting works for VSA transcoders as well. VSA users can also add
transcoder.maxJob to one of
the agent configuration files. For example:
$ cat /etc/vidispine/agent.conf.d/transcoder.conf transcoder.maxJob=5
The transcoder configuration file
config.xml contains default settings
for the transcoder and need typically not be modified, as the settings
can instead by configured in Vidispine.
Modify the transcoder file¶
On a Linux system, copy the file
/etc/transcoder-config.xml. Then edit
/etc takes precedence over the file in
Modify the transcoder resource¶
On all operating systems, the transcoder configuration can be changed by adding configuration to the resource definition of the transcoder (Adding a transcoder).
Note that port of the transcoder cannot be changed in this fashion.
Modifying the transcoder configuration is this fashion takes precedence over the local configuration file and the global transcoder configuration, see below.
Modify all transcoders¶
It is also possible to change the configuration of all transcoders,
by setting the configuration property
to the XML representation of the transcoder configuration.
The preferred way of changing the thumbnail and poster settings is by
changing the appropriate values in the TranscodePresetDocument
in a shape tag. For example, by changing the
thumbnailPeriod elements. The setting in
shape tag have priority over the transcoder setting.
thumbnailResolution element contains the default resolution of the
thumbnails produced by the transcoder.
<a:thumbnailResolution> <a:width>320</a:width> <a:height>240</a:height> </a:thumbnailResolution>
You can also change the thumbnailing frequency by changing
For example, to thumbnail every 3 seconds:
<a:thumbnailPeriod> <a:samples>3</a:samples> <a:timeBase> <a:numerator>1</a:numerator> <a:denominator>1</a:denominator> </a:timeBase> </a:thumbnailPeriod>
If the transcoder does not use a scene change detection plugin, the frequency defaults to once every 10 seconds.
To have the transcoder send metrics to a StatsD server you can either:
- Enable StatsD using the API, see StatsD
- Update the transcoder configuration with the address and port of the StatsD server:
<a:statsd> <a:destination> <a:address>127.0.0.1</a:address> <a:port>8125</a:port> </a:destination> <a:prefix>t1</a:prefix> </a:statsd>
prefix element configures the prefix to use for each metric.
By default this is the
Transcoder resources settings¶
Path to temporary storage¶
Controls where temporary files are stored. Default is
on UNIX-like systems, or
%TEMP% on Windows.
Number of decoding threads¶
Controls the number of decoding threads. Defaults to
4 for I-frame-only formats.
The actual number of threads used depends on codec.
Number of encoding threads¶
Controls the number of encoding threads. Defaults to automatic setting. The actual number of threads used depends on codec.
HTTP buffer sizes¶
Controls the size of HTTP reads and writes of the transcoder.
dataBufferSize controls the maximum number of read bytes in memory. Default is 100 MB.
dataBufferWriteSize controls the maximum number of write bytes in memory. Default is 100 MB.
dataBufferFlushTime controls the number of seconds written bytes are stored in memory before it is flushed. Default is 4 seconds.
For system that uses segment files and where the transcoder has enough memory, it is recommended to increase these numbers, up to 10 times.
To control the memory and disk usage used by the transcoder for image
processing, use the
<imagemagick> element in the transcoder configuration.
The most important settings are listed below, for a complete list,
(under environment variables, used without the
in the transcoder configuration).
Maximum heap usage¶
<a:imagemagick> <a:key>MEMORY_LIMIT</a:key> <a:value>1GB</a:value> </a:imagemagick>
Temporary work area¶
<a:imagemagick> <a:key>TEMPORARY_PATH</a:key> <a:value>/var/tmp</a:value> </a:imagemagick>
The default value is the value set by the general transcoder temporary path,
see above. It is recommended that the
tempPath setting is used, rather
The following is done to remove the need to configure the transcoders directly:
- Vidispine pushes its own license to the transcoder, so that each transcoder does not need a license file of their own.
- The transcoder returns the IP address from where the license was pushed, that is, the IP address of the application server, removing the need for explicitly configuring the reverse address, that is, where the transcoder can reach Vidispine, in most cases.
- In addition, Vidispine generates temporary pre-authorized URIs that are used by the transcoder. This removes the need for entering any application server information in the transcoder configuration file.
Reverse address and NAT¶
The reverse address does not work if there is NAT or other port forwarding
mechanisms between the application server and the transcoder.
If so, the address to VS-EA can be
overridden in the definition for the transcoder by setting the
<?xml version="1.0"?> <ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine"> <transcoder> <url>http://transcoder.example.com:8888/</url> <reverseAddress>vs.example.com</reverseAddress> </transcoder> </ResourceDocument>
The rules for how the address to Vidispine is determined are as follows:
- If the configuration property
apiNoauthUriis set, it is used for all transcoders.
- If the configuration property
apiNoauthPortis set, it is used for together with the detected or manually set reverse address.
Transcoder’s access to media¶
By default, the transcoder accesses non-file-schema media through the application server. This has several advantages:
- The same user is used for all file access.
- Possibility for support for extended file attributes and permissions.
- Support for other file systems (URI schemes).
Streaming the media puts some extra load on the application server. Some tuning might be necessary.
The transcoder resource in Vidispine can be set up to access files directly. By
directAccess element to the transcoder resource, Vidispine
will let the transcoder access the media directly. If no
elements are present, an implicit
<directAccess> <filter>file:.*</filter> </directAccess>
is added. In order to tell Vidispine that all files should go via the application server, add an
<directAccess> <filter>NO_MATCH</filter> <!-- dummy regular expression that does not match anything --> </directAccess>
For all file systems that supports read-while-write, and for container
formats that are built for streaming (e.g. MXF), growing file is supported
when streamed through the application server. If growing files is required
to local files with the
file scheme, a
element as per above must be added to the resource configuration.