Version 4.2

The release notes will tell you what’s new in each version, and any changes that you must be aware of when upgrading. For reference, Vidispine ticket numbers are printed as (#1234).

4.2.15

2016-05-09

Security notice

  • Fix for CVE-2016-3714.

    Although it is not likely that Vidispine systems are affected, as Vidispine does its own file type checking, the updated transcoder contains the suggested fix to the ImageMagick policy.xml (#2331).

Bug fixes

  • PSQLException when deleting item with large metadata (#2330).
  • Slow mime-type check causing steps to disappear and retry (#2332).
  • NPE when retrieving empty access control list (#2288).
  • Collection metadata update with skipForbidden=True allows update of read-only fields (#2340).

Transcoder fixes

  • Fix for calculation of burnt-in timecode for QuickTime source sources contains EDLs (#2334).

  • Properly handle MPEG-2 with separate field encoding (#2333).

    For MPEG-2 interlaced video where fields are encoded separately, and stored in MPEG transport stream, use the following setting in the transcode preset document:

    <demuxerSetting>
      <key>pts_mode</key>
      <value>original</value>
    </demuxerSetting>
    

    This will be the default in Vidispine 4.6.

4.2.14

2016-03-21

MatrixStore

  • The MatrixStore SDK has been updated from 3.1.3.3 to 3.1.3.7. This fixes an exception when reading XML files from MatrixStore (#2246).

Bug fixes

  • Jobs not running on MySQL (SQLGrammarException) (#2245).
  • Slow update of user groups (#2272).
  • Missing object metadata in MatrixStore (MXFS_FILENAME_UPPER) (#2271).
  • SFTP file worker connection leak on file delete (#2249).
  • IllegalArgumentException on item list to Azure (#2254).
  • Moving a metadata timespan breaks references (#2218).
  • Slow access DOT graph for items in a large amount of collections (#2251).
  • No thumbnails created for certain files with an EDL (#2273).
  • No document metadata for PDF files when transcoding on import (#2257).

Transcoder fixes

  • Handle WAV file with metadata of odd number of bytes (#2258).

4.2.13

2016-02-16

Bug fixes

  • Slow listing of files on Azure (#2187).
  • Video misidentified as image/cgm (#2185).
  • Export transcode failure should fail export (#2183).
  • File removed from Solr on shape delete with keepFiles=true (#2212).
  • NPE on item reindex on thumbnail read error (#2221).
  • Don’t use IN subquery statements with MySQL (#2177).
  • Be able to skip indexing of timed metadata (#2212).
  • POST /storage/(storage-id)/file/data not creating file when using segment files (#2171).
  • NPE instead of 404 for API/storage/{external-id}/file/{id} requests (#2179).

Transcoder fixes

  • Insecure parsing of m3u4 files (#2222).
  • Shape deduction crash on invalid QuickTime file (#2180).
  • Make transcoder fail on transcode of truncated QuickTime files (#2220).
  • Transcoder crash on very short clip (#2223).

4.2.12

2015-12-07

Improvements

  • Support for marking a file as a duplicate of another (#2129).
  • The thumbnail format used by the transcoder is now exposed as a setting (#2128).

Bug fixes

  • Signiant transfers failing for large files (#2152).
  • Cannot delete group used in many ACLs on GlassFish (#2144).
  • Username parameter ignored for bulk item exports (#2137).

Transcoder fixes

  • Transcoder crash on aborted jobs (#2159).

4.2.11

2015-11-06

Bug fixes

  • Deadlock on job start causing job to stop running (#2067).
  • Rejected shape import requests causing empty jobs to be created (#2115).
  • Configurable exclusive job step wait time. See jobExclusiveStepMaxWait (#2116).
  • Export job failing to find and copy file created from transcode step (#2106).
  • List item jobs not writing directly to S3 (#2120).
  • List item jobs not finishing when listing by group (#2108).
  • FTP storage to empty directory does not work (#2062).

Transcoder fixes

  • Transcoder stuck on broken PDF (#2122).
  • Add option to fail if frame cannot be decoded (#2124).
  • Incorrect scaling of posters (#2123).

4.2.10

2015-10-01

Bug fixes

  • Storage rule supervisor repeatedly restarting due to a long running transaction, causing copy jobs not to start (#2050).
  • High JMS connection usage from storage supervisor (#2047).
  • Slow file system metadata updates causing growing imports to end early (Tunable growing file timeout) (#2053).
  • Slow library search due to rollbacks (#2028).
  • Cached search bigtext entries not always removed (#2041).
  • Solr searchers keeping GlassFish from shutting down (#2038).
  • Quote characters in a collection name generates broken access DOT graph (#2052).
  • Checksum not computed for file with %2B in name (#2056).

Transcoder fixes

  • Interlace flag not working for H.264 (#2036).
  • Incorrect subclipping of MXF OP1a with index in footer (#2057).
  • Incorrect bit depth for AES3 (#2059).
  • 32bit lpcm detected as 16bit (#2058).
  • Burnt in timecode text offset from text box (#2060).

4.2.9

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2015-08-27

This is a bug fix release with a large number of fixes.

Improvements

  • New Relic users will now see web requests with transaction names that match the path used in the request, instead of the automatic name assigned by New Relic which tends to group requests to different endpoints together.
  • Speed ups when faceting on many fields with many matching terms.

Bug fixes

  • Cached search documents not being removed from the big text table, causing it to grow over time.
  • The periodic update of a large library could cause library API requests and index updates to halt due to a locking issue.
  • When using SolrCloud the connection manager could abort long running requests, typically causing updates to Solr to fail.
  • Various bug fixes and improvements.

For the full list of changes, see the release notes.

4.2.8

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2015-07-03

This is a minor bug fix release that also adds two new job parameters.

Improvements

New job parameters:

Bug fixes

  • Storage not marked as offline if MatrixStore vault goes offline.
  • Poster format settings not working.
  • Installer ignoring required but failing HTTP requests.
  • Certain transcodes/render jobs crashing the transcoder.
  • Various bug fixes and improvements.

For the full list of changes, see the release notes.

4.2.7

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2015-06-02

This is a minor bug fix release. See the release notes for details.

4.2.6

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2015-05-13

Security notice

Apache POI has been updated from version 3.8 to 3.11 to mitigate XXE vulnerabilities (CVE-2014-3529 and CVE-2014-3574) when extracting metadata from Office documents. This metadata extraction is only done if enabled using parseFileMetadata.

XXE has also been disabled in the XML parser used by Vidispine, to address vulnerabilities when parsing XML API requests and sidecar files for example.

Performance improvements

This release contains a number of performance improvements.

For the API:

  • Faster collection update and delete.
  • Faster group update and delete.
  • Faster merged access retrieval.
  • Faster storage-rule creation.

For items:

  • Faster poster generation.

Changed defaults

Multi-site processing is now disabled by default. See disableSiteCrunching.

Other

  • Indexing of items with large metadata text fields could cause indexing to halt. This has now been fixed.
  • Field groups in the item metadata are now indexed less often.
  • Additional audio and video settings from the transcode preset are now supported when rendering or conforming.
  • More control over burnt-in subtitle placements. See Subtitle metadata fields and groups.
  • Various bug fixes and improvements.

4.2.5

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2015-02-26

Temporary transcoder path

It is now possible to use another directory for temporary files for the transcoder. This is controlled via the <tempPath> element in the transcoder. This element is controlled via local configuration file or transcoder resource definition or by changing the configuration for all transcoders.

Optional hit count

The hit count can now be omitted from the results when searching. This can be done to reduce the response time of search requests.

GET API/item?count=false

Other

  • Buckets with files in subfolders are now properly scanned and will now appear on your S3 storages.
  • Proxy services can be used for cloud licensing.
  • A number of issues with the multi-site sync have been fixed.
  • Various bug fixes and improvements.

4.2.4

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2015-01-29

FTP connection pooling

If you perform a large number of imports or exports over high latency FTP connections then you can now create a FTP connection pool to reduce the overhead of establishing a new connection each time.

PUT /configuration/ftp-pool
Content-Type: application/xml

<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <pool>
    <minSize>0</minSize>
    <maxSize>-1</maxSize>
    <evictionInterval>30000</evictionInterval>
    <minIdleTime>60000</minIdleTime>
  </pool>
<FtpPoolConfigurationDocument/>

Find collections with specific items

An item subquery can now be used when searching for collections to find collections based on the items that they contain.

PUT /collection
Content-Type: application/xml

<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="OR">
    <field>
      <name>title</name>
      <value>Peach</value>
    </field>
    <item>
      <field>
        <name>title</name>
        <value>Peach</value>
      </field>
    </item>
  </operator>
</ItemSearchDocument>

See Searching for collections with specific items.

Transcoder transfer and hash computation

Available transcoder with direct file access to media can be used to compute hash sums of files, or to do filesystem-to-filesystem transfers. This will offload the application server.

To enable this, use the configuration variables enableTranscoderHashing and enableTranscoderTransfer.

Database purging

Automatic routines for trimming two of the largest Vidispine tables are now included. For information about how to enable this, see Database purging.

4.2.3

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2014-12-18

StatsD metrics

Metrics related to operations performed by both Vidispine and the transcoders can now be exposed using the StatsD protocol. It is also possible to access the Vidispine metric statistics using JMX.

$ node stats.js config.js
17 Dec 12:23:31 - reading config file: config.js
17 Dec 12:23:31 - server is up
17 Dec 12:23:31 - DEBUG: Loading backend: ./backends/graphite
17 Dec 12:23:34 - DEBUG: vs.job.total.aborted:99|g
17 Dec 12:23:34 - DEBUG: vs.job.total.aborted_pending:0|g
17 Dec 12:23:34 - DEBUG: vs.job.total.failed_total:520|g
17 Dec 12:23:34 - DEBUG: vs.job.total.finished:50491|g
17 Dec 12:23:34 - DEBUG: vs.job.total.finished_warning:3|g
17 Dec 12:23:34 - DEBUG: vs.job.total.ready:0|g
17 Dec 12:23:34 - DEBUG: vs.job.total.started:1|g
17 Dec 12:23:34 - DEBUG: vs.job.total.waiting:0|g
17 Dec 12:23:34 - DEBUG: vs.service.load.5:0.02|g
17 Dec 12:23:34 - DEBUG: vs.service.load.60:0.02|g
...

Multithreaded transcoder pipeline

The decoding part of the transcoder is now multithreaded for I-frame-only content. This means that for content such as ProRes, D10/IMX, etc, you will see a speed-up, especially if the input material is in high resolution.

FileCatalyst transfers

FileCatalyst is now available as a transfer method between storage locations. The Vidispine application acts as a FileCatalyst client which can communicate to FileCatalyst servers for transferring files.

In order to register a FileCatalyst server for a storage, add a new transfer method to the storage.

<method>
  <uri>filecatalyst://fc:s3cret@localhost:2100/incoming/</uri>
  <type>TRANSFER</type>
</method>

For more information, see FileCatalyst Integration.

StorNext file information

For files residing on a Quantum StorNext file system, Vidispine can now show metadata on the file. In order to use this, enable web services on StorNext and add a HSM method to the storage.

<method>
  <uri>stornext://webservice:webservice@localhost:81/stornext/snfs/</uri>
  <type>HSM</type>
</method>

For more information, see StorNext Integration.

Standalone metadata

It is now possible to create standalone documents with arbitrary metadata. If you are using global metadata but need to store a large amount of data, leading to a large metadata documents, then consider splitting it up into smaller documents, for example by entity or group of entities.

PUT API/document/company_a
<MetadataDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <timespan start="-INF" end="+INF">
    ...
  </timespan>
</MetadataDocument>

Filters and facets

Search filters have been added to replace the facet filters. They support arbitrary queries compared to facet filters that only allow a single field to be queried.

Filters can also be excluded from certain facets. This can be used to reduce the number of search requests need to display search pages that uses multiple facets and multiple drill down options.

PUT API/item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <filter name="typeFilter">
    <field>
      <name>mediaType</name>
      <value>audio</value>
    </field>
  </filter>
  <facet count="true">
    <field>mediaType</field>
    <exclude>typeFilter</exclude>
  </facet>
</ItemSearchDocument>
<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <hits>1</hits>
  <item end="+INF" id="VX-361763" start="-INF">
    <timespan end="+INF" start="-INF"/>
  </item>
  <facet>
    <field>mediaType</field>
    <count fieldValue="none">1867</count>
    <count fieldValue="image">33</count>
    <count fieldValue="video">10</count>
    <count fieldValue="audio">1</count>
    <count fieldValue="data">1</count>
  </facet>
</ItemListDocument>

Other

  • It is no longer necessary to use the application server’s /tmp directory to store output files for Azure, S3, FTP destinations. Instead, this can be handled using segment files on the destination storage.
  • The timestamp handling for generating MP4/H.264 files have been rewritten. This means that proxy files are frame accurate without without the previous tweaks (useDTSmode et al).

For the full list of changes, see the release notes.

4.2.2

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2014-11-03

Thumbnails on cloud storages

Thumbnails can now be stored on cloud storages such as Amazon S3 and Azure. Thumbnails will be stored using one file per thumbnail.

<ResourceDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <thumbnail>
    <path>direct+azure://:+kLZrqckLZrqckLZrqckLZrqc==@mystorage/my-container/</path>
  </thumbnail>
</ResourceDocument>

Job pools

Use job pools to ensure that long running low priority jobs don’t block high priority jobs from running.

PUT /configuration/job-pool
Content-Type: application/xml

<JobPoolListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <pool>
    <priorityThreshold>HIGH</priorityThreshold>
    <size>2</size>
  </pool>
  <pool>
    <priorityThreshold>LOWEST</priorityThreshold>
    <size>3</size>
  </pool>
</JobPoolListDocument>

Joins

Support for joins has also been added to allow for cross-entity search between items, shapes and files.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <text>peach</text>
  <shape>
    <field>
      <name>language</name>
      <value>en</value>
    </field>
  </shape>
</ItemSearchDocument>

Platform

This release brings support for:

  • PostgreSQL 9.3.
  • Java 7 update 67.

Other

  • Signiant can now be used to transfer files between storages.

For the full list of changes, see the release notes.

4.2.1

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2014-09-16

WADL improvements

A number of improvements have been made to the WADL file. Missing parameters have been added and duplicate parameters have been removed for example. It has been updated to also include:

  • Parameter options.
  • Markers for repeating parameters.

The WADL file can be obtained using GET API/application.wadl.

Other

  • Support for copying of DTV 608/708 Closed Captions.
  • Files can now also be sorted by file extension.

For the full list of changes, see the release notes.

4.2

These are release highlights. For a full listing of the features, bug fixes and upgrade notes, please see the release notes in the Vidispine Partner Portal.

2014-07-04

API documentation

The API documentation has been moved from the Vidispine wiki into this documentation that you are now reading.

It is available online at http://apidoc.vidispine.com/latest/ and also on your local installation at /APIdoc.

Java 7

The 4.2 series now requires Java 7, specifically Java 7 update 25 as later versions have known bugs with GlassFish 3.x.

Efficient file I/O

The new Java 7 File API is used to reduce the number of file system operations that are used when scanning local storages.

Indexing

The Solr configuration now exists in the indexing configuration, but note that the Solr configuration properties are still supported.

This configuration can also be used to specify which fields should be included in the full text index, unless specified explicitly for a specific field.

PUT /configuration/indexing
Content-Type: application/xml

<IndexingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <solrPath>http://localhost:8088/solr</solrPath>
  <fieldDefault>
    <name>xmp_*</name>
    <fullText>false</fullText>
  </fieldDefault>
</IndexingConfigurationDocument>

Detect renamed files

Renamed files can be detected and and re-associate based on the file checksum. Enable it using the detectRenamedFiles storage property.

Platform

This release adds support for Ubuntu 14.04, Windows 2012 R2 and MySQL 5.6. At the same time, support for PostgreSQL 8.x, MySQL 5.1 and Java 1.6 has been discontinued.

Other

For the full list of changes, see the release notes.