Upgrade notes

General

  • Your system must be running Vidispine version 4.0.x or later in order to upgrade to Vidispine 4.x.
  • You may upgrade directly from release 4.0.x or later to e.g. 4.5, without first installing each intermediate release.
  • When upgrading a major or minor (4.x to 4.y) version you should always migrate the database, for a maintenance release (4.13.x to 4.13.y) it is typically not needed.
  • To be able to upgrade to version 5.0 of Vidispine you must first upgrade to Vidispine 4.17 and complete the database migration process.

Upgrading from 23.3 to 23.4

  • Adjustments made to database schema. Run db migrate to apply changes.

Upgrading from 23.2 to 23.3

  • Search: No changes to the Metadata documents stored in OpenSearch between version 1.2 and 2.7. Re-indexing is not required.

    For information on upgrading the OpenSearch cluster see the OpenSearch and AWS documentation.

  • Adjustments made to database schema. Run db migrate to apply changes.

Upgrading from 23.1 to 23.2

  • Search: No changes to the documents. Re-indexing is not required.
  • Adjustments made to database schema. Run db migrate to apply changes.

Upgrading from 22.4 to 23.1

  • Search: No changes to the documents. Re-indexing is not required.
  • Adjustments made to database schema. Run db migrate to apply changes.

Upgrading from 22.3 to 22.4

  • Setting new metadataGroupSearchStyle to mergeToTopLevelGroup, re-index will be required.
  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 22.1 to 22.2

  • OpenSearch: Switching from Elasticsearch to OpenSearch reindex is required

Upgrading from 5.4 to 5.5

  • APIinit is needed due to a new job step for analyze jobs.
  • Solr: No changes to the documents. Re-indexing is not required.
  • This release contains database migrations for both metadata and collections that may take time on larger systems. Please plan upgrades accordingly.

Breaking changes

  • Searching with cursors: Previously, when reaching the end of a result set, the next cursor returned was null. With 5.5 a valid cursor is returned instead. This enables tailing searches. To check for end of search, either check hits being less than the requested number, or if the returned cursor is the same as the previous cursor.
  • When enabling indexCollectionItemOrder, the order is in ascending order by the first entry of the item in the respective collection. (It is still recommended that this property is set to false for performance reasons.)

Upgrading from 5.3 to 5.4

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 5.2 to 5.3

External identifier migration

External identifiers in VidiCore should be unique among the same entity type. But if there are multiple “overlapping” external identifier namespaces defined, duplicate identifiers could exist.

This has been fixed in 5.3 (#4350). A stronger constraint has been added to the table to prevent duplicates. However, if there are already duplicate entries in the table, manual intervention is required.

Below are the steps:

  1. Use this SQL statement to identify duplicated entries:
SELECT i2.c_entity_type, i2.c_entity_id, i2.c_identifier_value, i2.c_identifier_name
FROM
(SELECT c_entity_type, c_identifier_value FROM t_externalidentry
    GROUP BY c_entity_type, c_identifier_value HAVING COUNT(*) > 1) i1
INNER JOIN
(SELECT * from t_externalidentry ) i2
ON i2.c_entity_type = i1.c_entity_type AND i2.c_identifier_value = i1.c_identifier_value
ORDER BY c_identifier_value;
  1. If the above query doesn’t return anything, meaning no duplicates, you can skip the rest and processed with automatic database migration.

  2. If there are duplicate entries, you need to decide which one(s) to remove.

    Below is a sample data produced by the SQL query. The result should help you determine which entities have duplicate external-ids, and their external-id values and namespaces.

    And the external identifier can be removed using: DELETE {entity-type}/{entity-id}/{identifier-value}.

     c_entity_type     | c_entity_id | c_identifier_value | c_identifier_name
-----------------------+-------------+--------------------+-------------------
 com.vidispine.db.Item | VX-11       | my_test_id         | namespace_1
 com.vidispine.db.Item | VX-2704     | my_test_id         | namespace_2
(2 rows)

Automatic database migration can be performed after all duplicates have been removed.

Upgrading from 5.1 to 5.2

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 5.0 to 5.1

  • APIinit is needed due to a new job step for raw and essence imports.
  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.17 to 5.0

Warning

Because of a change in our internal system for the handling of database migrations, all users MUST upgrade to version 4.17 of Vidispine server and complete the database migration process before attempting to upgrade to Vidispine Server 5.0.

  • Reindex needed due to Solr/Elasticsearch upgrade and changes to how dataset field values are indexed.
  • APIinit is needed due to new roles being added.
  • Solr: Supported version has been changed to 8.1.0.
  • Elasticsearch: Supported version has been changed to 6.8.1.
  • ActiveMQ: Supported version has been changed to 5.15.9.

For more information on upgrading to 5.0, see Upgrading to Vidispine 5.0.

Upgrading from 4.16 to 4.17

  • APIinit is needed due to new metadata fields added for TTML parsing.
  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.15 to 4.16

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.14 to 4.15

  • Solr: No changes to the documents. Re-indexing is not required.
  • The type of job that is created for the placeholder raw import request has changed from THUMBNAIL/TRANSCODE to PLACEHOLDER_IMPORT.

Upgrading from 4.13 to 4.14

  • Solr: No changes to the documents. Re-indexing is not required.

  • Elasticsearch: Supported versions have been changed to 5.6.x, 6.0.x, 6.1.x, or 6.2.x.

  • The item re-index process will no longer also rebuild the thumbnail index that is maintained internally by VS. The thumbnail index can now instead be rebuilt separately using a re-index thumbnail request. To restore the old behaviour, set disableThumbnailReindexing to false.

  • A number of new job steps have been added to support checksum validation on transfer.

    • COPY_FILE/MOVE_FILE, step 90 - Waiting for source file hash.
    • COPY_FILE/MOVE_FILE, step 92 - Retrieving source file hash.
    • COPY_FILE/MOVE_FILE, step 140 - Waiting for destination file hash.
    • COPY_FILE/MOVE_FILE, step 150 - Verifying file hashes.
  • Previous versions had a bug where collections containing libraries did not always have recursive ACL’s properly applied items in those libraries. This is fixed for newly created ACL’s, but any old ones with this problem will need to be re-indexed with:

    PUT /reindex/acl
    
  • A role have been added for reading Vidispine Agents: _vxa_read. To make sure admin users receives the role properly, please run APIinit:

    POST /APIinit
    

Upgrading from 4.12 to 4.13

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.11 to 4.12

  • Solr: No changes to the documents. Re-indexing is not required.
  • All media shape deductions from jobs are now performed using asynchronous job steps. A number of new job steps have been added to support this.
    • TRANSCODE, step 400 - Finalizing media check.
    • CONFORM, step 400 - Finalizing media check.
    • AUTO_IMPORT, step 550 - Finalizing media check.
    • AUTO_IMPORT, step 1100 - Finalizing media check.
    • ESSENCE_VERSION, step 800 - Finalizing media check.
    • TIMELINE, step 400 - Creating the entities.
  • To support removal of old essence files, a new job step has been added:
    • SHAPE_IMPORT, step 1000 - Remove old essence files.
  • Make sure to run APIinit when upgrading to create the above mentioned job steps. Any existing custom job steps using these step numbers will be overwritten, so make sure to adjust the step number of any custom job step definitions using these numbers and recreate the custom steps.

Upgrading from 4.10 to 4.11

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.9 to 4.10

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.8 to 4.9

  • Solr: No changes to the documents. Re-indexing is not required.
  • Support for Ubuntu 12.04 has been discontinued.

Upgrading from 4.7 to 4.8

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.6 to 4.7

  • Solr: The Solr schema must be updated to use the new long integer datatype.
  • Solr: No changes to the documents. Re-indexing is not required.
  • Version 3.2 of the MatrixStore SDK is now installed by default. The vidispine-server-matrixstore3.1 package must be installed to connect to MatrixStore 3.1.

Upgrading from 4.5 to 4.6

  • Solr: No changes to the documents. Re-indexing is not required.
  • Support for GlassFish has been discontinued. Use Installation instead.
  • Support for Java 7 has been discontinued. Use Java 8 instead.
  • Support for Microsoft Windows Server has been discontinued.

Upgrading from 4.4 to 4.5

  • Solr: No changes to the documents. Re-indexing is not required.
  • Support for JBoss has been discontinued. Use Installation instead.

Upgrading from 4.3 to 4.4

  • Solr: No changes to the documents. Re-indexing is not required.
  • The property indexCollectionItemOrder is now set to false by default. Set it to true before upgrading if you rely on the old behaviour.
  • The transcode preset script is now also evaluated for conform jobs. The difference compared to transcodes is that the shape is empty. If you use the same preset for both transcodes and conforms, then make sure that the script verifies the existence of the components of the shape before using them to avoid null pointer exceptions from the script.

Upgrading from 4.2 to 4.3

  • Solr: No changes to the documents. Re-indexing is not required.

Upgrading from 4.1 to 4.2

  • Solr: The indexing of items, collections and files has changed. Re-indexing is required.
  • Essence version resource representation has changed from URIListDocument to EssenceVersionListDocument, and now returns additional information about the version.

Platform:

  • Java 7u67 is now supported with GlassFish. The installer must be used to install this update onto GlassFish, as configuration and libraries in GlassFish must be patched to work with Java 7u67.
  • The installer will now install Solr 4.10.0 instead of Solr 4.5.1, but Solr 4.5.1 is still supported. Large systems may benefit of upgrading to Solr 4.10.0 as it brings additional search performance improvements.

Upgrading from 4.0 to 4.1

  • The ObjectMatrix MatrixStore client SDK has been updated from 2.7.2.6 to 3.1.3.3. This version is not compatible with older MatrixStore server versions, so verify that you are running MatrixStore server 3.1.x before upgrading to Vidispine 4.1.5 or later.
  • Support for Ubuntu 10.04 LTS has been discontinued, use Ubuntu 12.04 LTS instead.
  • On Linux the number of allowed open files per process (File Descriptor Limit) must be raised from the default (1024) to at least 50000 to avoid Java IO errors java.io.IOException: Too many open files. See How to increase File Descriptor Limit on Linux.