System configuration¶
Indexing configuration¶
The indexing configuration contains the parameters that relate to search and indexing.
- Where Vidispine can reach Solr or ZooKeeper.
- When to commit or soft commit.
- The Solr query request parameters.
- The default field settings.
This configuration replaces the configuration properties listed under Search and indexing.
Example¶
Full text indexing could be disabled for all fields, unless explicitly specified for a field, using:
<IndexingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<solrPath>http://localhost:8088/solr</solrPath>
<fieldDefault>
<name>*</name>
<fullText>false</fullText>
</fieldDefault>
</IndexingConfigurationDocument>
Metrics configuration¶
See StatsD on how to configure how metrics are sent to StatsD. The configuration resource is described at Metrics settings.
FTP pool configuration¶
New in version 4.2.4.
By default jobs that need to read or write to an FTP server will establish, use and end separate connections to the server. By configuring a FTP connection pool you can change so that the jobs share and reuse FTP connections. This can reduce the time it takes to transfer files over high latency connections.
For example, to create a connection pool with the default settings:
PUT /configuration/ftp-pool
Content-Type: application/xml
<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<pool/>
<FtpPoolConfigurationDocument/>
If no pool is specified then pooling will be disabled. Unless overridden, the pool will be unbounded, and connections will expire after 1 minute. That is, the above configuration is identical to:
PUT /configuration/ftp-pool
Content-Type: application/xml
<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<pool>
<minSize>0</minSize>
<maxSize>-1</maxSize>
<evictionInterval>30000</evictionInterval>
<minIdleTime>60000</minIdleTime>
</pool>
<FtpPoolConfigurationDocument/>
The FTP pool configuration resource is described at FTP pool configuration.
Database purging¶
Vidispine supports mechanisms for purging old information in database tables. Especially three tables can grow quite large without purging enabled.
Change-log table¶
New in version 4.2.4.
The change-log table holds information about data that should be sent to
other sites.
If multi-site is disabled (disableSiteCrunching
), this table grows
forever.
To enable purging of the table, two configuration properties are used:
changeLogPurgingTime
and changeLogForcePurgingTime
.
The first one controls deletion of entries that have been processed,
the other one controls deletion of entries regardless of state.
Sensible values are 43200 and 86400, corresponding to one and two months, respectively.
Audit trail table¶
New in version 4.2.4.
The audit trail table contains all API requests, see Audit trails.
To enable purging of the table, two configuration properties are used:
auditTrailPurgingTime
and auditTrailPurgingDirectory
.
Both must be set in order for purging to take place.
When purging is enabled, entries that are older than
auditTrailPurgingTime
will be removed and put in a file inside the
auditTrailPurgingDirectory
folder.
A sensible value is 43200 or higher, corresponding to one month.
Job table¶
New in version 4.3.
To enable purging of the table, two configuration properties are used: jobPurgingTime and jobPurgingDirectory. Both must be set in order for purging to take place.
When purging is enabled, entries that are older than
jobPurgingTime
will be removed and put in a file inside the
jobPurgingDirectory
folder.
Configuration properties¶
Configuration properties are used in Vidispine to control system-wide parameters.
General¶
-
apiUri
¶ URI to Application Server. Used by transcoder(s), so need to be proper host if transcoder(s) run on another machine.
Mandatory: Yes Example: http://localhost:8080/API/
-
apiNoauthUri
¶ URI to Application Server, to use to access the no-auth API. Used by transcoder(s), so need to be proper host if transcoder(s) run on another machine.
Example: http://localhost:8080
-
clusterName
¶ Optional alphanumerical identifier for the Vidispine installation/cluster. Must be set (to a unique identifier) if multiple Vidispine installations are to share a common set of transcoders.
Example: ABPRD
,ABDEV
,BCPRD
Since: 4.4
-
disableSiteCrunching
¶ Do not build site replication packages. Recommended to be set to
true
for systems not running site replication.Default: true
since 4.2.6. Previous versions:false
Since: 4.0.4
-
validatexml
¶ Enable schema validation of the incoming and outgoing xml document.
Default: false
Since: 4.2.1
-
slaveLicenseProxy
¶ Use a proxy for Connection to Vidispine master license service. Format is
http://IP:port/
orsocks://IP:port/
Proxy authentication is not supported.
Default: none Since: 4.2.5
Search and indexing¶
Deprecated since version 4.2: The Solr and ZooKeeper properties are deprecated. Use Indexing configuration instead.
-
solrPath
¶ URI ( not path! ) to Solr.
Mandatory: Yes (No for SolrCloud) Example: http://localhost:8081/solr/
-
zkHost
¶ For SolrCloud: A comma separated list of host:port pairs to the servers in the ZooKeeper ensemble.
Mandatory: No (Yes for SolrCloud) Example: localhost:3000,example.com:3001
Since: 4.1
-
solrCollection
¶ For SolrCloud: The collection in Solr to be used by Vidispine.
Mandatory: No (Yes for SolrCloud) Example: collection1
Since: 4.1
-
solrQueryTimeout
¶ The request timeout in milliseconds to use when querying Solr.
Default: 60000
Since: 4.1.1
-
solrPingAttempts
¶ The number of times to ping a Solr node before aborting an active request.
Default: 5
Since: 4.1.1
-
solrPingTimeout
¶ The request timeout in milliseconds to use when checking if a Solr node. is alive
Default: 5000
Since: 4.1.1
-
solrCommitInterval
¶ The interval (in milliseconds) of Vidispine sending hard commit to Solr.
Default: 10000
-
solrSoftCommitInterval
¶ The interval (in milliseconds) of Vidispine sending soft commit to Solr.
Default: -1
(disable)
-
solrAutoSoftCommit
¶ If Vidispine should sending soft commit to Solr automatically.
Default: true
-
solrUpdateQueueSize
¶ Number of documents Vidispine will send in batch to Solr.
Default: 100
-
indexFieldGroups
¶ If metadata field groups should be indexed in Solr. Setting this to
false
can reduce the load and the size of the index if items have a large number of groups in the metadata, but will mean that no results will be available when searching for field groups.Default: true
-
indexCollectionItemOrder
¶ If the order of an item in a collection should be indexed in Solr. Settings this to
false
can greatly reduce the number of fields created in Solr and improve performance on systems with a lot of collections. This affects collection item retrieval. See also Ordering collections.Recommended to be set to
false
for applications not relying on that feature. Requires a clean Solr index and a full re-index to take effect.Changed in version 4.4: This is now disabled by default.
Default: false
Since: 4.2.8
-
indexTimespans
¶ If time coded metadata should be indexed in Solr. Setting this to
false
can reduce the load and the size of the index if items/collections have a large number of timespans in the metadata, but will mean that no time coded metadata can be found.Default: true
Since: 4.2.13
-
maxSearchResults
¶ Maximum number of search results allowed to be returned (see Search items).
Default: 100
-
legacyTransientFieldTypes
¶ This setting controls the datatype of the transient metadata fields. If
true
then all transient fields will be of typestring
. Iffalse
the*_size
and*_count
fields will be of typeinteger
, and the rest will have typestring
.Default: true
Metadata¶
-
disableMetadataSchema
¶ If a metadata schema has been defined (see Metadata schema), allows metadata that does not comply to the schema.
Default: false
Authentication¶
-
passwordHashAlgorithm
¶ The hash algorithm used to hash all user passwords. Note that changing this will make it impossible to authenticate with any existing user.
Default: MD5
-
userTokenMaxInterval
¶ Maximum token time for token created by regular user, in seconds.
Default: 60
Since: 4.2.2
-
userTokenDefaultInterval
¶ Default token expiration time, in seconds.
Default: 60
Since: 4.2.2
-
userTokenRefreshInterval
¶ Minimum time between token refreshments, in seconds.
Default: 10
Since: 4.2.2
Jobs and imports¶
-
jobRetryCount
¶ Number of retries for a job step before job continues with next step.
Default: 5
-
jobExclusiveStepMaxWait
¶ The maximum number of seconds that a job step will wait before executing if there’s a job step running from another job for the same item or file. This exists to reduce the number of optimistic locking exceptions for job steps that are known to conflict.
Only applies to steps with the exclusive flag (0x0100000) set.
Default: 1
Since: 4.2.11
-
defaultIngestStorage
¶ The default destination storage for imports and transcodes. Note that storages selected by storage rules will take priority over this.
Example: VX-1
-
parseFileMetadata
¶ If set to true, file metadata will be metadata parsed and inserted as Item metadata. Supported formats for this type of metadata include Office formats and PDF files.
Default: false
-
parseXMP
¶ If set to true, XMP metadata will be parsed and inserted as Item metadata.
Default: false
-
xmpIgnoreElements
¶ Contains comma-separated list of elements that are not read when parsing XMP data.
Default: DocumentAncestors,Pantry,History
Since: 4.0.10
-
simpleImageProcessor
¶ If false, use ImageMagick (must be installed, see Using ImageMagick for image handling). Otherwise, use built-in image handling.
Default: true
-
disableThumbnailGeneration
¶ Will disable thumbnail generation by default. Can be overridden on a per job basis.
Default: false
Since: 4.0.3
-
alwaysGenerateThumbnails
¶ When
true
, thumbnails will be generated on import even if no transcoding takes place.Default: false
Since: 4.0.3
-
mediaCheckInterval
¶ The retry interval of media check (seconds).
Default: 3
Since: 4.1
Storage and file¶
-
keepMissingFiles
¶ If set to false then missing files that do not belong to any items will be removed from the database instead of being marked as lost.
Can be overridden on a per storage basis using the
keepMissingFiles
storage metadata property.Default: false
Since: 4.1
-
keepEmptyDirectories
¶ Do not delete empty parent directories when deleting the last file in a directory, see Parent directory management.
Can be overridden on a per storage basis using the
keepEmptyDirectories
storage metadata property.Default: false
Since: 4.2.5
-
fileHashAlgorithm
¶ Hashing algorithm used. If changed, the
c_hash
column of thet_file
table should probably be set toNULL
.Example: SHA-1
-
enableTranscoderHashing
¶ Off-load file hash calculation available transcoder.
Default: false
Since: 4.2.4
-
fileTempKeyDuration
¶ Number of minutes a no-auth URI is valid (Auto method types).
Example: 10
-
useS3Proxy
¶ When
true
, Vidispine will create S3 pre-signed URLs for reading during job.Example: false
Since: 4.1
-
s3ProxyValidTime
¶ The validate time (in minutes) of S3 pre-signed URL.
Example: 60
Since: 4.1
-
s3ConcurrentParts
¶ The number of threads used for each S3 file upload.
Default: 1
-
s3PartSize
¶ The S3 chunk/part size. Note that multipart uploads are always performed regardless of file size.
Default: 5242880
-
s3ConnectionTimeout
¶ The timeout (in milliseconds) when establishing a connection to S3.
Default: 50000
-
s3SocketTimeout
¶ The timeout (in milliseconds) when reading from a connection to S3.
Default: 50000
-
s3MaxErrorRetry
¶ The maximum number of times to retry a failed S3 request.
Default: 3
-
useAzureProxy
¶ When
true
, Vidispine will create AZURE-SAS URLs for reading during job.Example: false
Since: 4.1
-
azureSasValidTime
¶ Specifies for how many minutes an AZURE-SAS URI will be valid. See Get status of file in storage.
Example: 60
Since: 4.0.1
-
stornextFileMetadata
¶ Specifies which fields that should be stored on the Vidispine file entity from StorNext metadata. See StorNext Metadata.
Default: location,class,existingCopies,targetCopies
Since: 4.2.3
-
useSegmentFiles
¶ If
true
, files generated by the transcoder on storages that do not support partial modification are written as segment files on the storage, instead of local files on the application server. See Temporary storages for transcoder output.Default: false
Since: 4.2.3
Archival¶
-
trustArchivedFiles
¶ A file needs to have a replica (another file with the same hash) before it can be removed by the storage rules.
If set to true, then archived files will be treated as valid replicas.
Default: false
-
glacierArchiveDescription
¶ Format the archive description according to the defined pattern:
{itemId}
- Replaced by the item id.{fileId}
- Replaced by the id of the archived file.{metadata-field:name}
- Replaced by the value of the metadata field with the given name.{sourceId}
- Replaced by the id of the source file.{sourcePath}
- Replaced by the path of the source file.{sourceUri}
- Replaced by the URI of the source file.{date}
- Replaced by the archive date in ISO 8601 format.{dateString}
- Replaced by the archive date, in formatdow mon dd hh:mm:ss zzz yyyy
.
Example:
Item:{itemId}, file:{fileId}, path:{sourcePath}, Archive date:{date}, Title:{metadata-field:title}
Since: 4.8 Default: my archive {dateString}
File system¶
Tip
Since 4.1.1, several of the stat
system calls that was made by
the JRE has been migrated into call in the JNI code. This can be
enabled using the localFSTimeData
option. On systems where local
file systems are sensitive to stat loads, it is recommended to enable
this option, and possibly the statsPerSecond
option.
-
fileHierarchy
¶ See Using a tree structure for files.
Example: 0
-
thumbnailHierarchy
¶ See Using a tree structure for thumbnails.
Example: 0
Warning
Changing this property will cause old thumbnails to be lost. If you need to change the value on a system in production, please contact Vidispine.
-
statsPerSecond
¶ Limit the total number of stats done on local file system. See also per-storage metadata ( Storages).
Since: 4.1.1
-
localFSTimeData
¶ Use JNI methods for retrieving file modification time. See below.
Default: false
-
firstLastModifiedAsCreationTime
¶ Use the first reading of modification time as the creation time. Can be used on file systems which do not have the notion of creation time.
Default: false
-
disableATime
¶ Do not record atime. Used in conjunction with localFSTimeData.
Default: false
Since: 4.2.5
Transfers¶
-
signiantManagerHost
¶ Hostname of Signiant manager. See Signiant Integration
-
signiantManagerUser
¶ Username for Signiant manager. See Signiant Integration
-
signiantManagerPassword
¶ Password for Signiant manager. See Signiant Integration
-
enableTranscoderTransfer
¶ Off-load file-to-file transfers of non-growing files to available transcoder.
Default: false
Since: 4.2.4
Library¶
-
libraryUpdateInterval
¶ Default library update interval in the system (seconds).
Default: 60
Since: 4.1
-
libraryExpireTime
¶ Default library expire time in the system (seconds).
Default: 86400
Since: 4.1
-
useLucene
¶ If Lucene should be used directly when updating auto-refreshing libraries. This is faster than using Solr when there are a large amount of auto-refreshing libraries, but only works with the default Solr configuration that is shipped with Vidispine.
Default: false
Since: 4.0
Growing files¶
-
fileGrowingTimeout
¶ The max time a file can keep growing (seconds).
Default: 36000
Since: 4.1
-
fileNotGrowingTimeout
¶ A file is considered as not growing if it has not been changed during this period (seconds).
Default: 600
Since: 4.1
Services¶
-
itemDeleteInterval
¶ The running interval (seconds) of
ItemDeleteCruncher
during the “idle” period (no item to delete).Default: 60
Since: 4.1
-
itemDeleteIntervalShort
¶ The running interval (seconds) of
ItemDeleteCruncher
during the “busy” period (there are items to be deleted).Default: 5
Since: 4.1
-
itemDeleteExecutionTime
¶ Max running time (seconds) of
ItemDeleteCruncher
thread, after that it goes to sleep.Default: 5
Since: 4.1
-
fileHashExecutionTime
¶ Max running time (seconds) of a file hashing thread, after that it goes to sleep.
Default: 10
Since: 4.1
Broker¶
-
compressDocumentMessages
¶ If JMS messages containing XML should be compressed or not. If
true
then the JMS_SUN_COMPRESS property will be set on JMS messages so that compression/decompression is performed by the OpenMQ client.Works only with OpenMQ.
Since: 4.2.2 Default: true
Database management¶
-
auditTrailPurgingTime
¶ Remove all audit trail entries older than the specified time (in minutes) and put them in XML format in files inside the directory described by
auditTrailPurgingDirectory
. See Audit trail table.Since: 4.2.4 Default: not set
-
auditTrailPurgingDirectory
¶ Since: 4.2.4 Default: not set
-
changeLogPurgingTime
¶ Remove all processed change-log entries older than the specified time (in minutes). See Change-log table.
Since: 4.2.4 Default: not set
-
changeLogForcePurgingTime
¶ Remove all change-log entries (processed or unprocessed) older than the specified time (in minutes).
Since: 4.2.4 Default: not set
-
jobPurgingTime
¶ Remove all job entries older than the specified time (in minutes) and put them in XML format in files inside the directory described by
jobPurgingDirectory
. See Job table.Since: 4.3 Default: not set
-
jobPurgingDirectory
¶ Since: 4.3 Default: not set
-
disableSequenceChecker
¶ Since: 4.6.1 Default: false On start-up, Vidispine checks the sequences for all tables, which can be a lengthy process. On a database which is known to be in a consistent state, setting
disableSequenceChecker
totrue
will cause this step to be skipped.
Transcoding¶
When a transcoding job has started, and transcoder connection becomes available, wait for this time (seconds) for connection to be restored until job fails.
Since: 4.4.1 Default: 60 seconds
-
bulkyMetadataKeysToIgnore
¶ Comma-separated list of bulky metadata keys to ignore from analysis results, e.g.
crop
,Since: 4.4.1 Default: (none)
-
transcoderNonblockingStatusInterval
¶ How frequently the transcode progress of a job will be updated, in milliseconds. A lower number may give a better user experience, but also a higher number of writes to the database.
Since: 4.4.1 Default: 5000
Vidispine Server Agent¶
-
syncVxaFileChanges
¶ This determines whether items created from the VSA or VDA are re-created when the corresponding original file is changed. If the original file is deleted, the item would be removed.
Since: 4.6 Default: false
-
syncVxaDeletes
¶ Similar to
syncVxaFileChanges
but this one only affects file deletions. When the original file on an agent storage is removed, the corresponding item in Vidispine would also be deleted.Since: 4.6 Default: true
System properties¶
System properties are set as argument to the JVM. See Setting JVM options.
The following properties are used in Vidispine:
-
com.vidispine.site
¶ The site id prefix for the current site.
Default: VX
-
com.vidispine.license.dir
¶ The directory containing the Vidispine license or slave license file.
Default: ${com.sun.aas.instanceRoot}
Since: 4.3
-
com.vidispine.license.tmpdir
¶ The directory where temporary license files may be stored.
Default: ${com.sun.aas.instanceRoot}
Since: 4.3
-
com.vidispine.credentials.dir
¶ The directory containing credentials files such as the
AwsCredentials.properties
file used with Amazon S3 and Glacier.Default: ${com.sun.aas.instanceRoot}
Since: 4.3
-
com.vidispine.log.dir
¶ The directory containing the server log files.
Default: ${com.sun.aas.instanceRoot}/logs
Since: 4.3
-
vidispine.identifier.format
¶ If
full
, output Long identifiers.Default: Normal, short identifiers.