System configuration¶
Indexing configuration¶
The indexing configuration contains the parameters that relate to search and indexing.
- Where VidiCore can reach OpenSearch
- The default field settings.
This configuration replaces some configuration properties listed under Search and indexing.
Example¶
Full text indexing could be disabled for all fields, unless explicitly specified for a field, using:
<IndexingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<elasticsearchPath>http://localhost:9200/</elasticsearchPath>
<fieldDefault>
<name>*</name>
<fullText>false</fullText>
</fieldDefault>
</IndexingConfigurationDocument>
Metrics configuration¶
See StatsD on how to configure how metrics are sent to StatsD. The configuration resource is described at Metrics settings.
FTP pool configuration¶
By default jobs that need to read or write to an FTP server will establish, use and end separate connections to the server. By configuring a FTP connection pool you can change so that the jobs share and reuse FTP connections. This can reduce the time it takes to transfer files over high latency connections.
For example, to create a connection pool with the default settings:
PUT /configuration/ftp-pool
Content-Type: application/xml
<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<pool/>
<FtpPoolConfigurationDocument/>
If no pool is specified then pooling will be disabled. Unless overridden, the pool will be unbounded, and connections will expire after 1 minute. That is, the above configuration is identical to:
PUT /configuration/ftp-pool
Content-Type: application/xml
<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<pool>
<minSize>0</minSize>
<maxSize>-1</maxSize>
<evictionInterval>30000</evictionInterval>
<minIdleTime>60000</minIdleTime>
</pool>
</FtpPoolConfigurationDocument>
The FTP pool configuration resource is described at FTP pool configuration.
Database purging¶
VidiCore supports mechanisms for purging old information in database tables. Especially four tables can grow quite large without purging enabled.
Note
When running VidiCore on a PostgreSQL database please ensure that PostgreSQL vacuuming is in place. Vacuuming makes the space freed by VidiCore database purging available to further write operations. Without proper vacuuming in place, the database will consume more and more disc space even if database purging is configured properly.
For more details on PostgreSQL vacuuming please refer to the knowledge base.
Updated method for purging configuration¶
The preferred way of updating the database purging configuration is by create a DatabasePurgingConfigurationDocument, see Database purging configuration.
Change-log table¶
The change-log table holds information about data that should be sent to
other sites.
If multi-site is disabled (disableSiteCrunching
), this table grows
forever.
To enable purging of the table, two configuration properties are used:
changeLogPurgingTime
and changeLogForcePurgingTime
.
The first one controls deletion of entries that have been processed,
the other one controls deletion of entries regardless of state.
Sensible values are 43200 and 86400, corresponding to one and two months, respectively.
Configuration can also be controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<changeLog>
<age>43200</age>
<forceAge>86400</forceAge>
</changeLog>
</DatabasePurgingConfigurationDocument>
Audit trail table¶
The audit trail table contains all API requests, see Audit trails.
To enable purging of the table, two configuration properties are used:
auditTrailPurgingTime
and auditTrailPurgingDirectory
.
Both must be set in order for purging to take place.
When purging is enabled, entries that are older than
auditTrailPurgingTime
minutes will be removed and put in a file inside the
auditTrailPurgingDirectory
folder.
A sensible value is 43200 or higher, corresponding to one month.
Configuration can also be controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<auditTrail>
<age>43200</age>
<uri>s3://key:secret@archival-bucket/requests/</uri>
</auditTrail>
</DatabasePurgingConfigurationDocument>
For discarding the purged entries without exporting them use empty:/
as uri
value.
By setting the compress
element in the configuration document, entries
will be stored using gzip compression. The default value is true.
The entries are stored in batches. The number of entries per batch is
controlled by the batch
element in the configuration document. The
default value is 10000 entries. If there are less than 10000 entries
that fulfill the time criteria, the purging of the audit trail table
will pause.
By setting the body
element in the configuration document, entries
will also include the request bodies and response codes. The default value is
false.
The directory can be a full URI, such as a S3 or FTP location.
Job table¶
To enable purging of the table, two configuration properties are used: jobPurgingTime and jobPurgingDirectory. Both must be set in order for purging to take place.
When purging is enabled, entries that are older than
jobPurgingTime
minutes will be removed and put in a file inside the
jobPurgingDirectory
folder.
Configuration can also be controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<job>
<age>43200</age>
<uri>s3://key:secret@archival-bucket/job/</uri>
</job>
</DatabasePurgingConfigurationDocument>
For discarding the purged entries without exporting them use empty:/
as uri
value.
By setting the compress
element in the configuration document, entries
will be stored using gzip compression. The default value is true.
The directory can be a full URI, such as a S3 or FTP location.
Transfer log table¶
Configuration is controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<transferLog>
<age>1440</age>
<uri>s3://key:secret@archival-bucket/transfers/</uri>
</transferLog>
</DatabasePurgingConfigurationDocument>
Both the age
and uri
element has to be set.
Entries than represents a finished transfer older than age
minutes are exported to the destination.
For discarding the purged entries without exporting them use empty:/
as uri
value.
The forceAge
element controls exports of non-finished transfers - transfers that have disappeared for
some reason. The default value of forceAge
is the value of the age
element.
By setting the compress
element in the configuration document, entries
will be stored using gzip compression. The default value is true.
The entries are stored in batches. The number of entries per batch is
controlled by the batch
element in the configuration document. The
default value is 10000 entries. If there are less than 10000 entries
that fulfill the time criteria, the purging of the transfer log table
will pause.
Default job priority¶
New in version 5.2.1.
The default job priorities are configurable by type. For example, the following configuration
document will make IMPORT
jobs default to MEDIUM
priority and EXPORT
jobs default to
HIGH
priority:
<JobPriorityConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<job type="IMPORT">MEDIUM</job>
<job type="EXPORT">HIGH</job>
</JobPriorityConfigurationDocument>
If no default priority has been specified for a given job type, the job will default to MEDIUM
priority.
CORS configuration¶
New in version 4.15.
VidiCore can be configured to emit Cross-Origin Resource Sharing (CORS) headers.
The CORS configuration is set using the CORS configuration resource. The configuration document consists of a number of entries, each of them are checked for CORS evaluation. If an entry condition matches the incoming request, the CORS headers set in the entry are outputted, and no other entries are matched. For example:
PUT /configuration/cors
<CORSConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<entry>
<request/>
<response>
<allowOrigin>*</allowOrigin>
</response>
</entry>
</CORSConfigurationDocument>
The conditions in each entry can match HTTP method, the request path, the CORS origin, or any other header. For example:
<CORSConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<entry>
<request>
<pathRegex>API.*/item/.*</pathRegex>
<headerRegex>
<key>connection</key>
<value>.*aliv.*</value>
</headerRegex>
</request>
<response>
<allowOrigin>*</allowOrigin>
<allowMaxAge>86400</allowMaxAge>
</response>
</entry>
</CORSConfigurationDocument>
Semantics are as follows.
- Multiple methods can be set per entry. If one entry method matches the request method, the entry matches as far as for method. If no methods are set on the entry, the entry matches.
- Origins can be specified both exact or via regular expressions. If one entry origin matches the request origin, the entry matches as far as for origin. If no origins are set on the entry, the entry matches.
- Request paths can be specified via regular expressions. If one entry path matches the request path, the entry matches as far as for path. If no paths are set on the entry, the entry matches.
- Request headers can be specified via regular expressions. All specified headers for the entry must match the request. If the HTTP request does not contain a value for a specified header, the entry does not match. If the HTTP request contains multiple values for a specified header, it is sufficient that only one value matched the entry header condition.
When the entry matches, the entry may contain which origin that is allowed, the max age for the access, which methods that are allowed, which HTTP headers. The entry can also contain other, arbitrary, headers. If methods are not specified, the methods are automatically deduced from the API methods.
Configuration properties¶
Configuration properties are used in VidiCore to control system-wide parameters.
Since 5.3, configuration properties can be used to control some system property settings, see below.
See also
See Create/modify configuration properties for more information about how to configure properties.
Some configuration properties are cached locally, and it may take up to 5 minutes until the new value is observed. These configuration properties are marked with “Cached: yes” below.
General¶
-
apiUri
¶ URI to Application Server. Used by transcoder(s), so need to be a proper host if transcoder(s) run on another machine.
Mandatory: true
Data Type: string
Example: http://localhost:8080/API/
Environment Dependent: true
-
apiNoauthUri
¶ URI to Application Server, to use to access the no-auth API. Used by transcoder(s), so need to be a proper host if transcoder(s) run on another machine.
Example: http://localhost:8080
Data Type: string
Environment Dependent: true
-
clusterName
¶ Optional alphanumerical identifier for the VidiCore installation/cluster. Must be set (to a unique identifier) if multiple VidiCore installations are to share a common set of transcoders.
Data Type: string
Example: ABPRD
,ABDEV
,BCPRD
Environment Dependent: true
-
disableSiteCrunching
¶ Do not build site replication packages. Recommended to be set to
true
for systems not running site replication. Since 4.2.6 this property is set totrue
by default.Data Type: bool
Default: true
Environment Dependent: true
-
validatexml
¶ Enable schema validation of the incoming and outgoing xml document.
Data Type: bool
Default: false
Environment Dependent: false
-
slaveLicenseProxy
¶ Use a proxy for Connection to Vidispine Online Licensing System. Format is
http://IP:port/
orsocks://IP:port/
Proxy authentication is not supported.
Data Type: string
Default: (none) Environment Dependent: true
-
defaultTranscoder
¶ Default transcoder resource to use. Valid values are:
vidinet
- To use the first available transcoder from VidiNet.- A resource id - To use the transcoder with that id.
Data Type: string
Example: VX-1
Default: (none) Environment Dependent: true
-
stackTraceInterval
¶ (New in 22.4.1.)
When set, output a stack trace periodically as per the interval set (in seconds). The value of
0
means no stack trace is output.Data Type: int
Example: 60
Default: 0
Environment Dependent: true
Search and indexing¶
Deprecated since version 4.2: The Solr and ZooKeeper properties are deprecated. Use Indexing configuration instead.
-
solrPath
¶ URI ( not path! ) to Solr.
Data Type: string
Mandatory: true
(false for SolrCloud)Example: http://localhost:8081/solr/
Environment Dependent: true
-
elasticsearchPath
¶ URI to OpenSearch’s RESTful interface.
Mandatory: true
(if using Elasticsearch/OpenSearch). Optional ifurl
has already been configured in search backendData Type: string
Example: http://localhost:9200/
Environment Dependent: true
-
opensearchPath
¶ URI to OpenSearch’s RESTful interface.
Mandatory: true
(if using OpenSearch). Optional ifurl
has already been configured in search backendData Type: string
Example: http://localhost:9200/
Environment Dependent: true
-
zkHost
¶ For SolrCloud: A comma separated list of host:port pairs to the servers in the ZooKeeper ensemble.
Data Type: string
Mandatory: false
(Yes for SolrCloud)Example: example.com:3001
Environment Dependent: true
-
reindexFixedDelay
¶ Normally,
ReindexCruncher
will be waked up immediately if an entity is marked for reindex, and work continuously until all pending entities has been processed. This is to make sure that changes to entities are indexed as soon as possible.However, for use cases like migrating data from other system into VidiCore, many updates could be performed to an entity in a short amount of time, causing multiple reindex request and thus puts additional load on the search backend.
If the entities are not expected to be searchable ASAP,
reindexFixedDelay
can be used to set a fixed working delay (in seconds) inReindexCruncher
, allowing entity updates to be buffered and reducing load on the search backend.New in version 21.3.1.
Data Type: int
Default: (none) Example: 60
Environment Dependent: false
-
solrCollection
¶ For SolrCloud: The collection in Solr to be used by VidiCore.
Data Type: string
Mandatory: false
(Yes for SolrCloud)Example: collection1
Environment Dependent: true
-
solrQueryTimeout
¶ The request timeout in milliseconds to use when querying Solr.
Data Type: int
Default: 60000
Environment Dependent: true
-
solrGroupLimit
¶ The maximum number of timespans to return per item or collection.
Mandatory: false
Data Type: int
Example: 10
Environment Dependent: false
-
solrPingAttempts
¶ The number of times to ping a Solr node before aborting an active request.
Data Type: int
Default: 5
Environment Dependent: true
-
solrPingTimeout
¶ The request timeout in milliseconds to use when checking if a Solr node. is alive
Data Type: int
Default: 5000
Environment Dependent: true
-
solrCommitInterval
¶ The interval (in milliseconds) of VidiCore sending hard commit to Solr.
Data Type: int
Default: 10000
Environment Dependent: true
-
solrSoftCommitInterval
¶ The interval (in milliseconds) of VidiCore sending soft commit to Solr.
Data Type: int
Default: -1
(disable)Environment Dependent: true
-
solrAutoSoftCommit
¶ If VidiCore should send soft commit to Solr automatically.
Data Type: bool
Default: true
Environment Dependent: true
-
solrUpdateQueueSize
¶ Number of documents VidiCore will send in batch to Solr.
Data Type: int
Default: 100
Environment Dependent: true
-
solrDeleteMergeSize
¶ The number of delete queries to merge into one before sending to Solr.
Changed in version 5.6: The default was changed from
-1
(do not merge any delete queries) to100
.Data Type: int
Default: 100
Environment Dependent: true
-
solrMaxBatchLength
¶ Limit of the size of merged updates sent to Solr in number of bytes.
Data Type: int
Default: 100000000
(100 Mi), before 22.3.4 no limitSince: 22.3.4 Environment Dependent: true
-
elasticsearchWorkerCount
¶ Number of worker threads to use when sending documents to OpenSearch.
Data Type: int
Default: 1
Since: 5.6 Environment Dependent: true
-
opensearchWorkerCount
¶ Number of worker threads to use when sending documents to OpenSearch.
Data Type: int
Default: 1
Since: 22.2 Environment Dependent: true
-
searchIndexSettings
¶ Defines whether VidiCore uses search version 1 or search version 2 or both. See Migration to search version 2 for details.
Data Type: String
Default: OnlyOld
AllowedValues: [ OnlyOld
,NewAndOld_UsingOldSearch
,NewAndOld_UsingNewSearch
,ONLY_NEW
]Since: 25.2 Environment Dependent: true
-
elasticsearchBulkBuffer
¶ The document buffer size (in bytes) in the OpenSearch worker thread.
Data Type: int
Default: 10000
Since: 5.6 Environment Dependent: true
-
opensearchBulkBuffer
¶ The document buffer size (in bytes) in the OpenSearch worker thread.
Data Type: int
Default: 10000
Since: 22.2 Environment Dependent: true
-
indexFieldGroups
¶ If metadata field groups should be indexed in Solr. Setting this to
false
can reduce the load and the size of the index if items have a large number of groups in the metadata, but will mean that no results will be available when searching for field groups.Data Type: bool
Default: true
Environment Dependent: false
-
indexCollectionItemOrder
¶ If the order of an item in a collection should be indexed in Solr. Settings this to
false
can greatly reduce the number of fields created in Solr and improve performance on systems with a lot of collections. This affects collection item retrieval. See also Retrieve the child-collections of a collection.Recommended to be set to
false
for applications not relying on that feature. Requires a clean Solr index and a full re-index to take effect.Data Type: bool
Default: false
Environment Dependent: false
-
indexTimespans
¶ If time coded metadata should be indexed in Solr. Setting this to
false
can reduce the load and the size of the index if items/collections have a large number of timespans in the metadata, but will mean that no time coded metadata can be found.Data Type: bool
Default: true
Environment Dependent: false
-
maxSearchResults
¶ Maximum number of search results allowed to be returned (see Search items).
Data Type: int
Default: 100
Environment Dependent: true
-
legacyTransientFieldTypes
¶ This setting controls the datatype of the transient metadata fields. If
true
then all transient fields will be of typestring
. Iffalse
the*_size
and*_count
fields will be of typeinteger
, and the rest will have typestring
.Data Type: bool
Default: true
Environment Dependent: false
-
skipLibraryIndexUpdates
¶ If set to
true
, the auto-refreshing libraries won’t be updated after item metadata changes.Data Type: bool
Default: false
Environment Dependent: false
-
indexDocumentMetadata
¶ If document metadata should be indexed. Setting this to
false
can reduce the load and size of the index. Document metadata is not searchable if this property is is set tofalse
.Data Type: bool
Default: false
Since: 5.0 Environment Dependent: false
-
indexQueueLimit
¶ This setting throttles indexing to a maximum number of messages in the ActiveMQ queue. In a system where indexing (Solr/OpenSearch) is slower than the database, this setting avoids that too many messages are stored on the queue. Set this value to
0
to specify no limit. See alsoactiveMQAdminUrl
.Data Type: int
Default: 0
Since: 5.7.1 Cached: yes Environment Dependent: true
-
filterInheritedTimespans
¶ If set to true, inherited metadata where the ancestor is a collection with absoluteTime=true and the adjusted inherited timespan would lie entirely outside of the item startTime..startTime+duration is not returned.
The metadata is also not indexed for the item. In order to accurately reflect this in search results, a re-index is necessary.
Data Type: bool
Default: false
Since: 22.1 Environment Dependent: false
-
metadataGroupIndexStyle
¶ Set the value to groupPath to enable the new group field search feature feature. Set the value to fieldNameOnly if you want the group search behavior prior to 22.3.
Data Type: string
Default: fieldNameOnly
Since: 22.3 Environment Dependent: false
-
metadataGroupSearchStyle
¶ By default, parent and child metadata groups are indexed as separate search documents. Setting this value to
mergeToTopLevelGroup
so that only the parent groups are indexed, and all the child group values will be flattened into the the parent document. Note that a reindex is needed after changing this property.Data Type: string
Default: empty
Since: 22.4 Environment Dependent: false
-
opensearchParallelFacetSearch
¶ Search operations with multiple facets against OpenSearch can be time consuming, executing these searches in parallel can speed this up. Set this property to
true
to let VidiCore execute the OpenSearch queries for all facets in parallel.Please be aware that parallel facet searches increase the CPU load on the OpenSearch server.
Data Type: bool
Default: false
Since: 24.1.1 Environment Dependent: false
-
opensearchConnectionRequestTimeout
¶ Timeout in seconds for connecting to the OpenSearch server. Note that changed values are only picked up during startup of VidiCore.
Data Type: int
Default: 3
Since: 24.2 Environment Dependent: true
-
opensearchSocketTimeout
¶ Socket timeout in seconds for http connections to OpenSearch server. Note that changed values are only picked up during startup of VidiCore.
Data Type: int
Default: 60
Since: 24.1.1 Environment Dependent: true
Metadata¶
-
disableMetadataSchema
¶ If a metadata schema has been defined (see Metadata schema), allows metadata that does not comply to the schema.
Data Type: bool
Default: false
Environment Dependent: false
-
useAbsoluteSccTimeCode
¶ If set to true Vidicore will not try to adjust the time codes in the SCC sidecar file to the usual relative (zero-based) timecodes for each item when imported.
Data Type: bool
Default: false
Since: 5.6 Environment Dependent: false
-
skipIdenticalMetadataValues
¶ If set to true Vidicore will not create a change set if a metadata update operation does not result in a change of the entity’s metadata document.
Data Type: bool
Default: false
Environment Dependent: false
Bulky Metadata¶
-
bulkyMetadataMigrationThreads
¶ Number of threads to use for bulky metadata migration.
Data Type: int
Default: 1
Since: 5.4.5 Environment Dependent: true
-
bulkyMetadataPackingKeyPrefixes
¶ The prefixes for the bully metadata keys which should be packed during analyze of the data.
If this entry does not exist, the packer does a fallback to
audio_amplitude
andloudness
as prefix filter.If the entry exists and its value is an empty string, the packer will skip filtering (legacy behaviour).
If the entry exists, the value will be split by
,
or;
and uses these values for filtering.Data Type: string
Default: audio_amplitude,loudness
Since: 24.1.2 Environment Dependent: false
-
bulkyMetadataItemDeletionBatchSize
¶ The number of attached bulky metadata to delete in a single batch when deleting items with bulky metadata.
Default: 5000
Since: 24.2.10 Data Type: int
Environment Dependent: false
Authentication¶
-
passwordHashAlgorithm
¶ The hash algorithm used to hash all user passwords. Note that changing this will make it impossible to authenticate with any existing user.
Data Type: string
Default: MD5
Environment Dependent: false
-
ldapAuthentication
¶ If set to true, LDAP authenticated will be enabled.
Data Type: bool
Default: false
Environment Dependent: false
-
userTokenMaxInterval
¶ Maximum token time for token created by regular user, in seconds.
Data Type: int
Default: 60
Environment Dependent: false
-
userTokenDefaultInterval
¶ Default token expiration time, in seconds.
Data Type: int
Default: 60
Environment Dependent: false
-
userTokenRefreshInterval
¶ Minimum time between token refreshments, in seconds.
Data Type: int
Default: 10
Environment Dependent: false
Jobs and imports¶
-
concurrentJobs
¶ Number of jobs that are allowed to be started.
Data Type: int
Default: 3
Environment Dependent: true
-
concurrentJobThreads
¶ Number of jobs threads that are allowed to be executed in parallel. See job concurrency for details.
Data Type: int
Default: 10
Since: 25.2 Environment Dependent: true
-
dedicatedJobPool
¶ Enable/disable dedicated job pool configuration.
Data Type: bool
Default: false
Environment Dependent: true
-
jobRetryCount
¶ Number of retries for a job step before job continues with next step.
Data Type: int
Default: 5
Environment Dependent: false
-
jobExclusiveStepMaxWait
¶ The maximum number of seconds that a job step will wait before executing if there’s a job step running from another job for the same item or file. This exists to reduce the number of optimistic locking exceptions for job steps that are known to conflict.
Only applies to steps with the exclusive flag (0x0100000) set.
Data Type: int
Default: 1
Environment Dependent: false
-
jobFeedbackInterval
¶ Number of seconds between progress updates of file transfers and transcoder jobs.
Prior to version 25.2 the job feedback interval was 5 seconds and not configurable.
Data Type: int
Default: 5
Since: 25.2 Environment Dependent: false
-
jobFeedbackDeadlineMargin
¶ Number of seconds to wait for feedback on a job step before the step expires and is restarted.
Data Type: int
Default: 300
Environment Dependent: false
-
jobFeedbackTTL
¶ Time-to-live in milliseconds of job feedback messages sent to ActiveMQ for file transfers and transcoder jobs.
Data Type: long
Default: 30000
Environment Dependent: false
-
defaultIngestStorage
¶ The default destination storage for imports and transcodes. Note that storages selected by storage rules will take priority over this.
Data Type: string
Example: VX-1
Environment Dependent: true
-
parseFileMetadata
¶ If set to true, file metadata will be metadata parsed and inserted as Item metadata. Supported formats for this type of metadata include Office formats and PDF files.
Data Type: bool
Default: false
Environment Dependent: false
-
maxFileMetadataLength
¶ The max length of file content (in characters) that will be extracted as Item metadata.
-1
means unlimited.Data Type: int
Default: 100000
Environment Dependent: false
-
parseXMP
¶ If set to true, XMP metadata will be parsed and inserted as Item metadata.
Data Type: bool
Default: false
Environment Dependent: false
-
xmpIgnoreElements
¶ Contains comma-separated list of elements that are not read when parsing XMP data.
Data Type: string
Default: DocumentAncestors,Pantry,History
Environment Dependent: false
-
simpleImageProcessor
¶ If false, use ImageMagick (must be installed, see Using ImageMagick for image handling). Otherwise, use built-in image handling.
Data Type: bool
Default: true
Environment Dependent: false
-
disableThumbnailGeneration
¶ Will disable thumbnail generation by default. Can be overridden on a per job basis.
Data Type: bool
Default: false
Environment Dependent: false
-
alwaysGenerateThumbnails
¶ When
true
, thumbnails will be generated on import even if no transcoding takes place.Data Type: bool
Default: false
Environment Dependent: false
-
disableThumbnailReindexing
¶ When
false
, the thumbnail index will be rebuild when items are reindexed.Data Type: bool
Default: true
Environment Dependent: false
-
representativeThumbnailPosition
¶ Defines how the representative thumbnail is selected for videos and gifs. Valid values are:
- a valid VidiCore time code textual representation - Use the thumbnail at or after this timeline position. If not available, use the last thumbnail before this position.
middle
- Use middle entry in list of all available thumbnails.
Non-gif images and PDFs always use the first thumbnail as representative thumbnail.
Data Type: string
Default: middle
Since: 24.1.1 Environment Dependent: false
-
mediaCheckInterval
¶ The retry interval of media check (seconds).
Data Type: int
Default: 3
Environment Dependent: false
-
shapeDeductionThreshold
¶ The threshold at which a shape is ready for transcoding (points).
- The steps of the heuristic to determine if the shape is ready are:
- containerComponent exists: 80 points
- audioComponent and videoComponent exists and audioComponent has not changed since last check: 40 points
- for every second waited since shape has either audio or video component: 1 point
- for every second waited since a detected file change: 1 point
- file state is not OPEN or TO_APPEAR: 138 points
- document or archive file extension like doc, pdf or zip: 80 points
If the value is set to 125, transcoding could, for example, start when container, video and audio components have been detected on the shape and at least 5 seconds have passed since then.
Data Type: int
Default: 120
Environment Dependent: false
-
useLegacyScaling
¶ (New in 21.4.)
When true, use the legacy behavior when scaling segments during sequence rendering. When it is not set, or set to false use the new behavior when scaling segments.
- Legacy behavior:
- Segments without scaling effect, scale to fit the canvas resolution.
- Segments with scaling effect, apply to source clips resolution.
- New behavior:
- Segments are scaled to fit the canvas resolution, without cropping, while keeping aspect ratio. Then any scaling effects are applied.
Data Type: bool
Example: true
Environment Dependent: false
-
cloudConvertVersion
¶ The version of cloudconvert resource to use.
Data Type: int
Default: (none) Example: 2
Since: 5.7.4 Environment Dependent: true
-
cloudConvertSandbox
¶ When set to
true
, VidiCore will try to select a cloudconvert sandbox resource to use.Data Type: bool
Example: false
Default: (none) Since: 5.7.4 Environment Dependent: true
Storage and file¶
-
groupImportableFiles
¶ When true, auto-import will only import one file for each file prefix as an item. Other non-sidecar files with the same prefix will be ignored. Set to false to import all files.
Data Type: bool
Default: true
Environment Dependent: false
-
keepMissingFiles
¶ If set to false then missing files that do not belong to any items will be removed from the database instead of being marked as lost.
Can be overridden on a per storage basis using the
keepMissingFiles
storage metadata property.Data Type: bool
Default: false
Environment Dependent: false
-
keepEmptyDirectories
¶ Do not delete empty parent directories when deleting the last file in a directory, see Parent directory management.
Can be overridden on a per storage basis using the
keepEmptyDirectories
storage metadata property.Data Type: bool
Default: false
Environment Dependent: false
-
scanMethodAlgorithm
¶ Set default scan method algorithm, see Storage scanning algorithm.
Can be overridden on a per storage basis using the
scanMethodAlgorithm
storage metadata property.Data Type: string
Default: (none) Since: 5.5.2 Environment Dependent: false
-
storageActivationFile
¶ Require a .storage file to be present in the storage method’s URI for storage to register as online.
Data Type: bool
Default: false
Since: 4.17 Environment Dependent: true
-
fileHashAlgorithm
¶ Hashing algorithm used. If changed, the
c_hash
column of thet_file
table should probably be set toNULL
.Data Type: string
Example: SHA-1
Environment Dependent: false
-
enableTranscoderHash
¶ Off-load file hash calculation available VSA.
Data Type: bool
Default: false
Environment Dependent: false
-
fileTempKeyDuration
¶ Number of minutes a no-auth URI is valid (Auto method types).
New in version 4.16: This property also controls the valid duration of a VSA noauth URI.
Data Type: int
Example: 10
Environment Dependent: false
-
useS3Proxy
¶ When
true
, VidiCore will create S3 pre-signed URLs for reading during job.Data Type: bool
Example: false
Environment Dependent: false
-
s3ProxyValidTime
¶ The validate time (in minutes) of S3 pre-signed URL.
Data Type: int
Example: 60
Environment Dependent: false
-
s3ConcurrentParts
¶ The number of threads used for each S3 file upload.
Data Type: int
Default: 1
Environment Dependent: true
-
s3PartSize
¶ The S3 chunk/part size. Note that multipart uploads are always performed regardless of file size. Each part that is uploaded will be larger than the previous part. To control the increase of the part size, use
s3PartSizeIncrease
.Data Type: int
Default: 5242880
Environment Dependent: false
-
s3PartSizeIncrease
¶ The size increase of each S3 part that is uploaded.
The default is chosen so that the maximum part size is 2 GB. With S3 supporting a maximum of 10000 parts, and the default part size being 5 MB, this gives a maximum object size of 3.5 TB. The default value is automatically selected based on the part size.
Data Type: int
Example: 10000
Default: (auto) Environment Dependent: false
-
s3ConnectionTimeout
¶ The timeout (in milliseconds) when establishing a connection to S3.
Data Type: int
Default: 50000
Environment Dependent: true
-
s3SocketTimeout
¶ The timeout (in milliseconds) when reading from a connection to S3.
Data Type: int
Default: 50000
Environment Dependent: true
-
s3MaxErrorRetry
¶ The maximum number of times to retry a failed S3 request.
Data Type: int
Default: 3
Environment Dependent: true
-
useAzureProxy
¶ When
true
, VidiCore will create AZURE-SAS URLs for reading during job.Data Type: bool
Example: false
Environment Dependent: false
-
azureSasValidTime
¶ Specifies for how many minutes an AZURE-SAS URI will be valid. See Retrieve a file.
Data Type: int
Example: 60
Environment Dependent: false
-
maxRequestedSignedURLTime
¶ Specifies the maximum allowed expiration time when creating a signed URI. If not set there is no limit.
This limit have only effect if expiration is specified. See Method metadata
Data Type: int
Example: 60
Environment Dependent: false
-
stornextFileMetadata
¶ Specifies which fields that should be stored on the VidiCore file entity from StorNext metadata. See StorNext Metadata.
Data Type: string
Default: location,class,existingCopies,targetCopies
Environment Dependent: false
-
useSegmentFiles
¶ If
true
, files generated by the transcoder on storages that do not support partial modification are written as segment files on the storage, instead of local files on the application server. See Temporary storages for transcoder output.Data Type: bool
Default: false
Environment Dependent: false
-
useMutableRangeWrites
¶ If
true
, use a writing pattern that is more efficient for S3 writes. The only reason for not have this set totrue
is in a clustered set-up with transcoders directly connected to VidiCore, i.e., not via VSA.Data Type: bool
Default: true
Environment Dependent: false
-
s3CredentialType
¶ Controls the type of S3 credentials being sent to a VSA. Allowed values are
none
,temporary
andsecretkey
. For more explanation, please check here.Data Type: string
Default: temporary
Environment Dependent: true
-
stsCredentialDuration
¶ The duration (in seconds) of any temporary AWS credentials generated for agents. The allowed range of values is
[900, 129600]
Data Type: int
Default: 129600
Environment Dependent: false
-
stsRegion
¶ To generate temporary credentials VidiCore will use the AWS Security Token Service (STS). Set this parameter to the region where you want VidiCore to call the STS API. A good choice is the same region as your VidiCore is running in.
Data Type: string
Default: us-west-2
Environment Dependent: true
-
stsAssumeRole
¶ This is an optional configuration to use when generating credentials using the
s3
scheme. The name of the role to use when generating assume role credentials to give direct access to a file using Generate temporary credentials.Data Type: string
Since: 4.15 Environment Dependent: true
-
defaultStorageRuleJobPriority
¶ Controls which priority that should be assigned to jobs started as a result of storage rules, such as copy and delete jobs.
Data Type: string
Default: MEDIUM
Since: 5.1 Environment Dependent: false
-
fileDeletionBatchSize
¶ Controls the batch size, in which VidiCore tries to delete files with the status TO_BE_DELETED.
Data Type: int
Default: 10000
Environment Dependent: false
Since: 24.2
Archival¶
-
trustArchivedFiles
¶ A file needs to have a replica (another file with the same hash) before it can be removed by the storage rules.
If set to true, then archived files will be treated as valid replicas.
Data Type: bool
Default: false
Environment Dependent: false
-
defaultRetrievalTier
¶ The default retrieval tier to use when restoring archived files.
Allowed values:
Standard
,Bulk
,Expedited
.Data Type: string
Default: Standard
Since: 24.2.9 Environment Dependent: false
-
glacierArchiveDescription
¶ Format the archive description according to the defined pattern:
{itemId}
- Replaced by the item id.{fileId}
- Replaced by the id of the archived file.{metadata-field:name}
- Replaced by the value of the metadata field with the given name.{sourceId}
- Replaced by the id of the source file.{sourcePath}
- Replaced by the path of the source file.{sourceUri}
- Replaced by the URI of the source file.{date}
- Replaced by the archive date in ISO 8601 format.{dateString}
- Replaced by the archive date, in formatdow mon dd hh:mm:ss zzz yyyy
.
Example: Item:{itemId}, file:{fileId}, path:{sourcePath}, Archive date:{date}, Title:{metadata-field:title}
Data Type: string
Default: my archive {dateString}
Environment Dependent: false
File system¶
Tip
Since 4.1.1, several of the stat
system calls that was made by
the JRE has been migrated into call in the JNI code. This can be
enabled using the localFSTimeData
option. On systems where local
file systems are sensitive to stat loads, it is recommended to enable
this option, and possibly the statsPerSecond
option.
-
fileHierarchy
¶ See Using a tree structure for files.
Data Type: int
Example: 0
Environment Dependent: false
-
fileSequenceStart
¶ The starting number for file sequences.
Data Type: int
Example: 0
,0001
Default: 1
Environment Dependent: false
-
thumbnailHierarchy
¶ See Using a tree structure for thumbnails.
Data Type: int
Example: 0
Environment Dependent: false
Warning
Changing this property will cause old thumbnails to be lost. If you need to change the value on a system in production, please contact Vidispine.
-
statsPerSecond
¶ Limit the total number of stats done on local file system. See also per-storage metadata ( Storages).
Data Type: int
Environment Dependent: true
-
localFSTimeData
¶ Use JNI methods for retrieving file modification time. See below.
Data Type: bool
Default: false
Environment Dependent: false
-
firstLastModifiedAsCreationTime
¶ Use the first reading of modification time as the creation time. Can be used on file systems which do not have the notion of creation time.
Data Type: bool
Default: false
Environment Dependent: false
-
disableATime
¶ Do not record atime. Used in conjunction with
localFSTimeData
.Data Type: bool
Default: false
Environment Dependent: false
Transfers¶
-
signiantManagerHost
¶ Hostname of Signiant manager. See Signiant Integration
Data Type: string
Environment Dependent: true
-
signiantManagerUser
¶ Username for Signiant manager. See Signiant Integration
Data Type: string
Environment Dependent: true
-
signiantManagerPassword
¶ Password for Signiant manager. See Signiant Integration
Data Type: string
Environment Dependent: true
-
enableTranscoderTransfer
¶ Off-load file-to-file transfers of non-growing files to an available VSA.
Changed in version 5.3.1: Default value was changed to true
Data Type: bool
Default: true
Environment Dependent: false
-
storageRuleDisableArchiveSources
¶ If
true
, archived files will not be used as sources for storage rule transfers.Data Type: bool
Default: false
Environment Dependent: false
Library¶
-
libraryUpdateInterval
¶ Default library update interval in the system (seconds).
Data Type: int
Default: 60
Environment Dependent: false
-
libraryExpireTime
¶ Default library expire time in the system (seconds).
Data Type: int
Default: 86400
Environment Dependent: false
-
useLucene
¶ If Lucene should be used directly when updating auto-refreshing libraries. This is faster than using Solr when there are a large amount of auto-refreshing libraries, but only works with the default Solr configuration that is shipped with VidiCore.
Data Type: bool
Default: false
Environment Dependent: false
Growing files¶
-
fileGrowingTimeout
¶ The max time a file can keep growing (seconds).
Data Type: int
Default: 36000
Environment Dependent: false
-
fileNotGrowingTimeout
¶ A file is considered as not growing if it has not been changed during this period (seconds). Can be overridden for a storage by setting the key-value metadata
probeFileBySizeChangeInterval
on the storage.Data Type: int
Default: 600
Environment Dependent: false
JavaScript¶
-
javascriptInterpreter
¶ The default JavaScript engine to use for scripts that don’t explicitly target a specific engine. Valid values are:
graalvm
- Use GraalVM JavaScript.rhino
- Use Mozilla Rhino.
Changed in version 5.0: GraalVM JavaScript was made the default.
Data Type: string
Default: graalvm
Since: 5.0 Environment Dependent: false
-
x509Certificates
¶ Concatenated list of X.509 certificates (in PEM format) that are added to the list of root certificates used in https connections with Javascript.
Data Type: string
Default: (none) Since: 22.2 Environment Dependent: true
Example: -----BEGIN CERTIFICATE----- MII.... -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MII....(second certificate) -----END CERTIFICATE-----
Services¶
-
itemDeleteInterval
¶ The running interval (seconds) of
ItemDeleteCruncher
during the idle period (no item to delete).Data Type: int
Default: 60
Environment Dependent: false
-
itemDeleteIntervalShort
¶ The running interval (seconds) of
ItemDeleteCruncher
during the busy period (there are items to be deleted).Data Type: int
Default: 5
Environment Dependent: false
-
itemDeleteExecutionTime
¶ Max running time (seconds) of
ItemDeleteCruncher
thread, after that it goes to sleep.Data Type: int
Default: 5
Environment Dependent: false
-
fileHashExecutionTime
¶ Max running time (seconds) of a file hashing thread, after that it goes to sleep.
Data Type: int
Default: 10
Environment Dependent: false
Broker¶
-
compressDocumentMessages
¶ If JMS messages containing XML should be compressed or not. If
true
then the JMS_SUN_COMPRESS property will be set on JMS messages so that compression/decompression is performed by the OpenMQ client.Works only with OpenMQ.
Data Type: bool
Default: true
Environment Dependent: false
-
activeMQAdminUrl
¶ Specifies the admin URI of ActiveMQ, for example
http://myactivemq:8761
. Normally it is not necessary specify it, but some functionality requires it, for exampleindexQueueLimit
. For embedded broker, this does not have to be set.Data Type: string
Default: (none) Since: 5.7.1 Cached: yes Environment Dependent: true
-
activeMQBrokerName
¶ Specifies the ActiveMQ broker name used. Used in conjunction with
activeMQAdminUrl
.Data Type: string
Default: localhost
Since: 5.7.1 Cached: yes Environment Dependent: true
Database management¶
-
auditTrailPurgingTime
¶ Remove all audit trail entries older than the specified time (in minutes) and put them in XML format in files inside the directory described by
auditTrailPurgingDirectory
. See Audit trail table.Data Type: int
Default: (none) Environment Dependent: false
-
auditTrailPurgingDirectory
¶ Data Type: string
Default: (none) Environment Dependent: true
-
auditTrailPurgingCompress
¶ If
true
, purged logs are stored using gzip compression. Iffalse
, they are stored as plain XML documents. Equivalent to thecompress
element in the database purging configuration documentData Type: bool
Default: true
Environment Dependent: false
-
auditTrailPurgingBatch
¶ Sets batch size of audit log entries when puring. If there are fewer entries, purging is paused. Equivalent to the
batch
element database purging configuration documentData Type: int
Default: 10000 Environment Dependent: false
-
auditTrailIncludeBody
¶ When
true
, the audit trail will include the body of requests, as well as the response code returned by the requests. These will always be included in purged documents, and can be seen in GET /log requests using thebody
query parameter. Equivalent to thebody
element in database purging configuration documentData Type: bool
Default: false
Since: 5.1 Environment Dependent: true
-
changeLogPurgingTime
¶ Remove all processed change-log entries older than the specified time (in minutes). See Change-log table.
Data Type: int
Default: (none) Environment Dependent: false
-
changeLogForcePurgingTime
¶ Remove all change-log entries (processed or unprocessed) older than the specified time (in minutes).
Data Type: int
Default: (none) Environment Dependent: false
-
jobPurgingTime
¶ Remove all job entries older than the specified time (in minutes) and put them in XML format in files inside the directory described by
jobPurgingDirectory
. See Job table.Data Type: int
Default: (none) Environment Dependent: false
-
jobPurgingDirectory
¶ Data Type: string
Default: (none) Environment Dependent: true
-
jobPurgingBatch
¶ Number of jobs processed per iteration.
Data Type: int
Default: 100
Since: 22.3.3 Environment Dependent: false
-
purgeLoopDelay
¶ Add delay in milliseconds to purging in order to throttle.
Data Type: int
Default: 100
Since: 22.3.3 Environment Dependent: false
-
purgeTransactionMaxTime
¶ Maximum running time of job purging loop in milliseconds.
Data Type: int
Default: 10000
Since: 22.3.3 Environment Dependent: false
-
disableSequenceChecker
¶ On start-up, VidiCore checks the sequences for all tables, which can be a lengthy process. On a database which is known to be in a consistent state, setting
disableSequenceChecker
totrue
will cause this step to be skipped.Data Type: bool
Default: false
Environment Dependent: true
Transcoding¶
When a transcoding job has started, and transcoder connection becomes available, wait for this time (seconds) for connection to be restored until job fails.
Data Type: int
Default: 60
Environment Dependent: false
-
bulkyMetadataKeysToIgnore
¶ Comma-separated list of bulky metadata keys to ignore from analysis results, e.g.
crop
,Data Type: string
Default: (none) Environment Dependent: false
-
transcoderNonblockingStatusInterval
¶ How frequently the transcode progress of a job will be updated, in milliseconds. A lower number may give a better user experience, but also a higher number of writes to the database.
Data Type: int
Default: 5000
Environment Dependent: false
VidiCore Server Agent¶
-
syncVxaFileChanges
¶ This determines whether items created from the VSA or VDA are re-created when the corresponding original file is changed. If the original file is deleted, the item would be removed.
Data Type: bool
Default: false
Environment Dependent: false
-
syncVxaDeletes
¶ Similar to
syncVxaFileChanges
but this one only affects file deletions. When the original file on an agent storage is removed, the corresponding item in VidiCore would also be deleted.Data Type: bool
Default: true
Environment Dependent: false
-
useVxaHash
¶ If set to
true
, VidiCore server agent will be used to compute the hash of the files.Data Type: bool
Default: false
Environment Dependent: false
-
useVxaMimeType
¶ If set to
true
, VidiCore server agent will be used to detect the mime type of the files.Data Type: bool
Default: false
Environment Dependent: false
Deletion lock¶
-
autoRemoveExpiredDeletionLocks
¶ It set to
true
, expired deletion locks on collections and items will be removed automaticallySince: 4.15 Data Type: bool
Default: false
Environment Dependent: false
-
deletionLockCleanUpBatchSzie
¶ Number of
to be removed
orexpired
deletion locks that will cleaned up in one batch.Data Type: int
Since: 4.17.7 Default: 100
Environment Dependent: false
System properties¶
System properties are set as argument to the JVM. See Setting JVM options.
The following properties are used in VidiCore:
-
com.vidispine.site
¶ The site id prefix for the current site.
Data Type: string
Default: VX
-
com.vidispine.license.dir
¶ The directory containing the VidiCore license or slave license file.
Data Type: string
Default: ${com.sun.aas.instanceRoot}
-
com.vidispine.license.tmpdir
¶ The directory where temporary license files may be stored.
Data Type: string
Default: ${com.sun.aas.instanceRoot}
-
com.vidispine.credentials.dir
¶ The directory containing credentials files such as the
AwsCredentials.properties
file used with Amazon S3 and Glacier.Data Type: string
Default: ${com.sun.aas.instanceRoot}
-
com.vidispine.log.dir
¶ The directory containing the server log files.
Data Type: string
Default: ${com.sun.aas.instanceRoot}/logs
-
vidispine.identifier.format
¶ If
full
, output Long identifiers.Data Type: string
Default: Normal, short identifiers
-
com.vidispine.xml.prefix
¶ Controls namespace prefix mapping of XML written by VidiCore that is _not_ part of API output, e.g. XML written to files. Comma-separated list of
{namespace}={prefix}
assignments. Namespaces without assignment get ans#
prefix. The default value is onlyns#
prefixes.Data Type: string
Example: http://www.smpte-ra.org/schemas/2067-2/2016=cc
-
infi.sleep_after_start
¶ The number of seconds to sleep after starting Infinispan, before VidiCore going on starting other services.
This should prevent split-brain issue during startup in some cases.
Data Type: int
Default: -1
(No delay)
-
com.vidispine.asyncpool.coresize
¶ Controls the size of the internal async pool in VidiCore (New in 5.3.1.). It’s also available as a configuration property.
Data Type: int
Default: 5
-
com.vidispine.service.quorum
¶ Prevents non-clustered services from being started on multiple nodes in a cluster with a split-brain issue. See Clustering for details.
The quorum value should be set to
> N/2
, whereN
is the total instance count in the cluster.For Example: The quorum should be set to
2
for a two-node-cluster as well as a three-node-cluster.(New in 5.3.5.).
Data Type: int
Default: 1
Since 5.3, some system properties can be overridden by configuration properties. The properties that can be overridden are:
com.vidispine.site
vidispine.identifier.format
com.vidispine.xml.prefix
Also in 5.3, is is possible to change the log levels of components using configuration properties. The configuration properties used for this is:
loglevel.
followed by the component name. (e.g.loglevel.com.vidispine.filemgmt.storagesupervisor
)
New in version 5.4.4.
It is now also possible to override the com.vidispine.service.quorum
system property using a configuration property as above.
While it is possible to change these values on a system in production, it is adviced that changing these values are done on an idle system, as the update in a cluster is asynchronous.
Bulky metadata storage¶
New in version 5.3.
By default, the values of bulky metadata are stored in the database. In a large system, this can occupy a large portion of the database. In version 5.3, there is the possibility to store bulky metadata on file system (or cloud storage).
To change the configuration the bulky metadata configuration resource is used. The configuration document contains two parameters, a base URI for where the bulky metadata should be stored, and an option to disable that storage. An explanation on how these to parameters interact follows below.
Transferring bulky metadata from database to file system¶
To move bulky metadata entries from database to the file system pointed out by the URI, use a configuration document like this:
PUT /configuration/bulkymetadata
Content-Type: application/xml
<BulkyMetadataConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<uri>file:///mnt/srv/bulkystorage/</uri>
</BulkyMetadataConfigurationDocument>
When this configuration is set, all bulky metadata is written to the file storage. Reading metadata is also done from file storage, but if the file storage does not contain the metadata, the database is read instead.
The path that is used to store the bulky metadata looks like this:
- Item id, with the id split in thousands
- Shape id, with the id split in thousands
- Key, channel, stream, and timecode, delimited by
-
.
In order to migrate all metadata to the storage, a reindex command is used:
PUT /reindex/bulkymetadata
When the reindex process has finished, all metadata is on the storage. This can be verified by retrieving the bulky metadata configuration, which also returns status information.
GET /configuration/bulkymetadata
Accept: application/xml
<BulkyMetadataConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<uri>file:///mnt/srv/bulkystorage/</uri>
<status>
<metadataInDatabase>0</metadataInDatabase>
<metadataOnStorage>45033</metadataOnStorage>
<storageStatus>OK</storageStatus>
</status>
</BulkyMetadataConfigurationDocument>
New in version 5.4.5: The bulky metadata migration can be parallelized by setting the bulkyMetadataMigrationThreads
property.
Transferring bulky metadata from file system to database¶
To move bulky metadata entries back from the file system to database, use a configuration document like this:
PUT /configuration/bulkymetadata
Content-Type: application/xml
<BulkyMetadataConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<uri>file:///mnt/srv/bulkystorage/</uri>
<storageDisabled>true</storageDisabled>
</BulkyMetadataConfigurationDocument>
Note the URI is still present, but the storageDisabled is set to true. It is vital that the URI remains set, and unchanged, as metadata will be read from the storage if it is not present in the database.
To migrate all metadata, use again:
PUT /reindex/bulkymetadata
GET /configuration/bulkymetadata
Accept: application/xml
<BulkyMetadataConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<uri>file:///mnt/srv/bulkystorage/</uri>
<storageDisabled>true</storageDisabled>
<status>
<metadataInDatabase>45033</metadataInDatabase>
<metadataOnStorage>0</metadataOnStorage>
<storageStatus>OK</storageStatus>
</status>
</BulkyMetadataConfigurationDocument>
Once everything has been migrated, the URI can be removed from the configuration, if preferred.
Transferring bulky metadata from one storage to another¶
There are two ways of moving bulky metadata from one storage to another.
- Move metadata from storage to database, ensure every metadata is moved, then move metadata to new storage.
- Stop/pause VidiCore, copy or move files from the old storage to the new storage. Start VidiCore and change URI.
If the second option is chosen, tools like rsync or aws s3 sync can be used to do a first synchronization while the system is in use.
Important notes¶
Note
The number of bulky metadata entries on storage returned by GET /configuration/bulky-metadata
is based on database information, and not explicit file system scanning.
Note
When the bulky metadata is moved from database to file system, note that database backups no longer contain the bulky metadata values. Make sure appropriate backup procedures is in place for the bulky metadata storage. Good tools are rsync and aws s3 sync (mentioned above) or incremental tar balls.
Note
Before migrating bulky metadata, a full database backup is recommended.
Usage reporting¶
Systems utilizing a VidiNet issued license automatically send version and usage statistics to VidiNet, in order to better serve the system.
Usage reporting as a whole can be disabled by disabling the UsageObservabilityRunner
service.
It is also possible to enable and disable specific metrics.
Note that no metadata or media is sent, only statistics and versions.
The following usage collectors exists today:
Name | Enabled by default | Default interval | Contains |
---|---|---|---|
vidicore-version |
yes | 4 hours | VidiCore, VidiCoder, VSA versions |
kubernetes-version |
yes | 4 hours | Information from Kubernetes cluster, e.g. MediaPortal, VidiEditor versions. Ignored if not running in K8s. |
cluster |
yes | 24 hours | VidiCore JVM information. |
database |
yes | 24 hours | Database table sizes, and vacuum information. No actual database content is sent. |
storage |
yes | 24 hours | Storage schema and files count. No actual files or any credentials are sent. |
job |
yes | 4 hours | Number of jobs of specific type and success rate. No job metadata is sent. |
For changing the configuration of the usage collectors, see the API Reference.
Advanced configuration/tweaking¶
Methods in this section should only be used after recommendation by Vidispine Support.
SQL Query rewriting¶
New in version 5.0.6.
SQL queries sent by VidiCore can be modified by creating a file
sqltranslations.txt
in the /etc/vidispine/
directory. The format of the file
are pairs of lines, where the first line of each pair is a
regular expression,
and the second line of the pair is a replace pattern, e.g.:
(?i)^(select .*)$
/* comment inserted */ $1
(?i)^(delete .*)$
/* another comment inserted */ $1