Version 5.3

The release notes will tell you what’s new in each version, and any changes that you must be aware of when upgrading. For reference, Vidispine ticket numbers are printed as (#1234).

5.3.1

2020-08-30

New features

  • Support returning additional user/group information in ACL (#4505).
  • Allow core metadata fields to be modified (#4482).

Improvements

  • Send file delete notification if a lowres file is removed due to transcoding failure (#4499).
  • Be able to configure the global spring async pool (#4495).

Bug fixes

  • Change the owner of an entity will generate a duplicate “indexLog” entry (#4450).

    Note: db migrate is needed to remove the already existing duplicate entries in t_indexlog. The migration is considered as optional since this issue doesn’t cause any problem functionally. It only affects the performance of ACL reindex. You can choose when to run db migrate, and the duplicate entries won’t be removed until then.

  • Prevent possible JGroups “split-brain” issue during VidiCore startup (#4518).

  • Mimetype missing on new shape versions (#4514).

  • The output file of a “m4a” shape tag should have “m4a” as file extension (#4504).

  • Reindex/index progress stuck on large collections (#4498).

  • VidiCore fails to update VidiCoder License in some cases (#4494).

  • Incorrect search hit on the /search endpoint in some cases (#4488).

  • Possible OOM when updating metadata referenced in many places (#4484).

  • Glacier file with special character in name stops storage scan (#4472).

  • Export between VSA shares takes long time and results in incorrect file size (#4460).

  • Too many parameter error in SQL query from DB cleanup (#4438).

  • Placeholder/Raw import jobs missing the configured CC extraction (#4421).

  • Incorrect timecode if fetching metadata containing “inherited timespan” using the “interval” query parameter (#4368).

  • Export with both tag and interval ignores the tag (#4156).

  • Removed child collection is still available from parent collection search (#4404)

  • Glacier Vault Archive restore job created by storage rule is aborted due to timeout (#4523)

Transcoder fixes

  • Creating video in 60 fps produces invalid length (#4481).
  • Incorrect system metadata fields on item level for NTSC drop-frame videos (#4480).
  • Fix invalid audio in MXF created by AVID (#4418).

5.3

2020-07-15

New names

VidiCore

Vidispine Server, Vidispine API, VaaS are now all called VidiCore. The packages and names in documentation will change during the second half of 2020. Consequently, VSA and VMA now stands for VidiCore Server Agent and VidiCore Management Agent.

VidiCoder

The transcoder’s new name is VidiCoder. Packages and documentation will change to reflect the new name during 2020.

Breaking changes

  • The priority attribute in the MergedAccessDocument has been changed to rank (#4430). Check doc for more information.
  • If you’re using external identifiers, database migration may require manual intervention. Check the “Upgrading from 5.2” section below for details.

Features and improvements

New file analyze endpoints

Shape deduction can now be performed on non-imported files or IMF packages in VidiCore (#4039).

POST /storage/{storage-id}/file/{file-id}/analyze
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<JobDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <jobId>VX-10525</jobId>
  <user>admin</user>
  <started>2020-07-14T09:04:51.768Z</started>
  <status>READY</status>
  <type>FILE_ANALYZE</type>
  <priority>MEDIUM</priority>
</JobDocument>

The resulting shape can be found in the job metadata and retrieved using the new shape endpoint on a file.

Bulky metadata storage

By default, the values of bulky metadata are stored in the database. In a large system, this can occupy a large portion of the database. Now, it is possible to store bulky metadata on the file system (or cloud storage) (#4440). See bulky metadata storage.

Configurable site prefix, identifier format, and logging level via the API

Traditionally, the VidiCore site prefix and identifier format can be configured using JVM properties. And the log levels are configured in the server.yaml file.

In 5.3, they can be configured using the API as well (#4417). See doc. This is especial useful to VidiCore users on VidiNet.

Entity ownership transfer

The ownership of an entity can be transferred to a different user when the original owner is being deleted. Check the usage of the transferAccess parameter (#4327).

VSA Thread configuration

It is now possible to configure the number of worker threads and selector runner threads in agent properties (#4226). This could improve VSA performance when under high load.

VidiCoder (Transcoder) library update

The transcoder uses ffmpeg for some codecs. The ffmpeg libraries are bundled with the transcoder. In version 5.3, a major upgrade of the ffmpeg library is done. This means some ffmpeg-specific parameters may have be renamed. If you are using these parameters, let Vidispine now. We have also taken the opportunity to clean up some of the edge-case handling of slightly incorrect media files in VidiCoder, as the new ffmpeg library is better at handling these. Hence, you should test that your type of media works with the new VidiCoder version. However, in the meantime, rest assured that you can still use VidiCoder 5.2 with VidiCore 5.3, they are compatible.

Other improvements

  • Add support for Bearer token validation using a public key (#4478).
  • Implement support for import of MPEG-DASH package (#4379).
  • Implement support for export of MPEG-DASH package (#4381).

Bug fixes

  • There can be duplicate external identifiers for the same entity type (#4350).
  • Sidecar file cannot be imported for the second time (#4235).
  • Glacier Vault storage can only get credentials from AwsCredentials.properties (#4219).

Upgrading from 5.2

External identifier migration

External identifiers in VidiCore should be unique among the same entity type. But if there are multiple “overlapping” external identifier namespaces defined, duplicate identifiers could exist.

This has been fixed in 5.3 (#4350). A stronger constraint has been added to the table to prevent duplicates. However, if there are already duplicate entries in the table, manual intervention is required.

Below are the steps:

  1. Use this SQL statement to identify duplicated entries:
SELECT i2.c_entity_type, i2.c_entity_id, i2.c_identifier_value, i2.c_identifier_name
FROM
(SELECT c_entity_type, c_identifier_value FROM t_externalidentry
    GROUP BY c_entity_type, c_identifier_value HAVING COUNT(*) > 1) i1
INNER JOIN
(SELECT * from t_externalidentry ) i2
ON i2.c_entity_type = i1.c_entity_type AND i2.c_identifier_value = i1.c_identifier_value
ORDER BY c_identifier_value;
  1. If the above query doesn’t return anything, meaning no duplicates, you can skip the rest and processed with automatic database migration.

  2. If there are duplicate entries, you need to decide which one(s) to remove.

    Below is a sample data produced by the SQL query. The result should help you determine which entities have duplicate external-ids, and their external-id values and namespaces.

    And the external identifier can be removed using: DELETE {entity-type}/{entity-id}/{identifier-value}.

     c_entity_type     | c_entity_id | c_identifier_value | c_identifier_name
-----------------------+-------------+--------------------+-------------------
 com.vidispine.db.Item | VX-11       | my_test_id         | namespace_1
 com.vidispine.db.Item | VX-2704     | my_test_id         | namespace_2
(2 rows)

Automatic database migration can be performed after all duplicates have been removed.