Storages¶
Managing storages¶
Retrieve list of storages¶
- GET /storage¶
Query Parameters: - size – Optional range of bytes, in format s1-s2. Returns storages with nominal size that is in that range. Either number can be omitted to not specify lower/upper limit. Size units can be used. Default is -, all storages.
- freebytes – Optional range of bytes, in format s1-s2. Returns storages with free space that is in that range. Either number can be omitted to not specify lower/upper limit. Size units can be used. Default is -, all storages.
- usedbytes – Optional range of bytes, in format s1-s2. Returns storages with used space that is in that range. Either number can be omitted to not specify lower/upper limit. Size units can be used. Default is -, all storages.
- freeamount – Optional range of percent as integers, in format s1-s2. Returns storages with used space that is in that range. Either number can be omitted to not specify lower/upper limit. Default is -, all storages.
- files – Optional range of files as integers, in format s1-s2. Returns storages with number of files that is in that range. Either number can be omitted to not specify lower/upper limit. Default is -, all storages.
- storagegroup –
Optional list of storage groups.
- storage-group - Returned storage is member of specified storage group.
- -storage-group Returned storage is not member of specified storage group.
- default Return all storages.
- status –
Optional list of storage status, see Storage states
- status - Returned storage has this status.
- -status - Returned storage does not have this status.
- default - Return all storages.
- url – Returns storages with a method matching this URL. May include wildcards * and ?.
Produces: - application/xml, application/json – StorageListDocument
- text/plain – CRLF-delimited list of storage ids
Role: _storage_read
Example¶
GET /storage
<StorageListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<storage>
<id>VX-1</id>
<state>NONE</state>
<type>LOCAL</type>
<capacity>1600000000000</capacity>
<freeCapacity>19993896424</freeCapacity>
<method>
<id>VX-6</id>
<uri>file:///mnt/main/</uri>
<bandwidth>0</bandwidth>
<read>true</read>
<write>true</write>
<browse>true</browse>
<lastSuccess>2014-07-04T08:39:00.739+02:00</lastSuccess>
<type>NONE</type>
</method>
<metadata/>
<lowWatermark>0</lowWatermark>
<highWatermark>0</highWatermark>
<showImportables>true</showImportables>
</storage>
<storage>
<id>VX-2</id>
<state>NONE</state>
<type>LOCAL</type>
<capacity>15000000000</capacity>
<freeCapacity>19548305962</freeCapacity>
<method>
<id>VX-4</id>
<uri>file:///mnt/ingest/</uri>
<read>true</read>
<write>true</write>
<browse>true</browse>
<lastSuccess>2014-07-04T08:38:56.773+02:00</lastSuccess>
<type>NONE</type>
</method>
<metadata/>
<lowWatermark>0</lowWatermark>
<highWatermark>200000000000</highWatermark>
<showImportables>true</showImportables>
</storage>
</StorageListDocument>
Create storage¶
- POST /storage¶
Creates a new storage.
Accepts: - application/xml, application/json – StorageDocument
Produces: - application/xml, application/json – StorageDocument
- text/plain – Storage id
Role: _storage_write
Note
If your storage is to be used for autoimport, the showImportables property in the StorageDocument must be set to true. The files on the storage will then be discovered and available for use with GET /storage/importable or GET /storage/auto-import/.
Example¶
POST /storage
Content-Type: application/xml
<StorageDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<type>LOCAL</type>
<capacity>150000000000</capacity>
<method>
<uri>file:///mnt/ingest/</uri>
<read>true</read>
<write>true</write>
<browse>true</browse>
</method>
<lowWatermarkPercent>90</lowWatermarkPercent>
<highWatermarkPercent>75</highWatermarkPercent>
<showImportables>true</showImportables>
</StorageDocument>
<StorageDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<id>VX-2</id>
<state>NONE</state>
<type>LOCAL</type>
<capacity>150000000000</capacity>
<freeCapacity>150000000000</freeCapacity>
<method>
<id>VX-4</id>
<uri>file:///mnt/ingest/</uri>
<read>true</read>
<write>true</write>
<browse>true</browse>
<type>NONE</type>
</method>
<metadata/>
<lowWatermark>112500000000</lowWatermark>
<highWatermark>135000000000</highWatermark>
<lowWatermarkPercentage>75</lowWatermarkPercentage>
<highWatermarkPercentage>90</highWatermarkPercentage>
<showImportables>true</showImportables>
</StorageDocument>
Get storage¶
- GET /storage/(storage-id)¶
Produces: - application/xml, application/json – StorageDocument
Role: _storage_read
Example¶
GET /storage/VX-2
<StorageDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<id>VX-2</id>
<state>NONE</state>
<type>LOCAL</type>
<capacity>150000000000</capacity>
<freeCapacity>150000000000</freeCapacity>
<method>
<id>VX-4</id>
<uri>file:///mnt/ingest/</uri>
<read>true</read>
<write>true</write>
<browse>true</browse>
<type>NONE</type>
</method>
<metadata/>
<lowWatermark>112500000000</lowWatermark>
<highWatermark>135000000000</highWatermark>
<lowWatermarkPercentage>75</lowWatermarkPercentage>
<highWatermarkPercentage>90</highWatermarkPercentage>
<showImportables>true</showImportables>
</StorageDocument>
Modify storage¶
- PUT /storage/(storage-id)¶
Updates an existing storage.
Accepts: - application/xml, application/json – StorageDocument
Produces: - application/xml, application/json – StorageDocument
- text/plain – Storage id
Example¶
PUT /storage/VX-2
Content-Type: application/xml
<StorageDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<type>LOCAL</type>
<capacity>150000000000</capacity>
<lowWatermarkPercentage>80</lowWatermarkPercentage>
<highWatermarkPercentage>95</highWatermarkPercentage>
</StorageDocument>
<StorageDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<id>VX-2</id>
<state>NONE</state>
<type>LOCAL</type>
<capacity>150000000000</capacity>
<freeCapacity>150000000000</freeCapacity>
<method>
<id>VX-4</id>
<uri>file:///mnt/ingest/</uri>
<read>true</read>
<write>true</write>
<browse>true</browse>
<type>NONE</type>
</method>
<metadata/>
<lowWatermark>120000000000</lowWatermark>
<highWatermark>142500000000</highWatermark>
<lowWatermarkPercentage>80</lowWatermarkPercentage>
<highWatermarkPercentage>95</highWatermarkPercentage>
<showImportables>true</showImportables>
</StorageDocument>
Delete storage¶
- DELETE /storage/(storage-id)¶
Deletes the storage. All files in storage will remain after call, but the Vidispine system will no longer manage them.
Query Parameters: - safe –
- true - Storage will only be deleted if there are no files connected to items on the storage.
- false (default) - Storage will be deleted, and any file entities will be disconnected from items and deleted.
Status Codes: - 409 Conflict – If safe parameter is true this error code will be given if there are files connected to items on the storage.
Role: _storage_write
- safe –
Storage information¶
Set the storage type¶
- PUT /storage/(storage-id)/type/(type)¶
Sets the type of the storage.
Role: _storage_write
Get storage status¶
- GET /storage/(storage-id)/status¶
Produces: - text/plain – Status string
Role: _storage_read
Get amount of free space on storage¶
- GET /storage/(storage-id)/freespace¶
Produces: - text/plain – Amount of free space as decimal number, between 0 and 100, inclusive
Role: _storage_read
Rescanning¶
The scanInterval property can be used to control how often (in seconds) a storage is scanned. Default is 60 seconds. By calling /rescan, the system is forced to rescan a storage without delay.
Rescan a storage¶
- POST /storage/(storage-id)/rescan¶
Triggers a rescan of a single storage.
Storage methods¶
New in version 4.1: Credentials are encrypted. This means that passwords cannot be viewed through the API/server logs.
Get storage methods¶
- GET /storage/(storage-id)/method¶
Query Parameters: - url – Optional URL. Only return methods with this URL. Wildcards (?, *) can be used, for example http:*.
Matrix Parameters: - read –
- true - Only return methods which have file read capability.
- false (default) - Return methods regardless of file read capability.
- write –
- true - Only return methods which have file write capability.
- false (default) - Return methods regardless of file write capability.
- browse –
- true - Only return methods which have file browse capability.
- false (default) - Return methods regardless of file browse capability.
Produces: - application/xml, application/json – StorageMethodListDocument
Role: _storage_read
Example¶
GET /storage/VX-2/method
<StorageMethodListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<method>
<id>VX-4</id>
<uri>file:///mnt/ingest/</uri>
<bandwidth>0</bandwidth>
<read>true</read>
<write>true</write>
<browse>true</browse>
<lastSuccess>2014-07-04T09:55:09.779+02:00</lastSuccess>
<type>NONE</type>
</method>
</StorageMethodListDocument>
Add new/update storage method¶
- PUT /storage/(storage-id)/method¶
Adds a new access method to the storage. If URL matches an existing method, a new method is not created, instead the existing one is updated.
Query Parameters: - url – Mandatory URL, method is access through this URL
- read –
- true (default) - Method has file read capability
- false - Method does not have file read capability
- write –
- true (default) - Method has file write capability
- false - Method does not have file write capability
- browse –
- true (default) - Method has file browse capability
- false - Method does not have file browse capability
- bandwidth – The bandwidth of this method in bytes per second. Default 0.
- type – Optional method type, see Storage methods. Default is empty.
Produces: - text/plain – Tabbed tuples : id, URL, status string of method
Role: _storage_write
Example¶
PUT /storage/VX-2/method?url=http://10.12.0.3/ingest/&read=true&write=false&browse=false
VX-5 http://10.12.0.3/ingest/ NONE
Get storage method¶
- GET /storage/(storage-id)/method/(method-id)¶
Retrieves a specific access method to storage.
Produces: - application/xml, application/json – StorageMethodListDocument
Role: _storage_read
Add new/update storage method¶
- PUT /storage/(storage-id)/method/(method-id)¶
Updates access method to the storage.
Query Parameters: - url – Mandatory URL, method is access through this URL
- read –
- true - Method has file read capability.
- false - Method does not have file read capability.
- write –
- true - Method has file write capability.
- false - Method does not have file write capability.
- browse –
- true - Method has file browse capability.
- false - Method does not have file browse capability.
- bandwidth – The bandwidth of this method in bytes per second. Default 0.
- type – Optional method type, see Storage methods.
Produces: - text/plain – Tabbed tuples : id, URL, status string of method
Role: _storage_write
Remove storage method¶
- DELETE /storage/(storage-id)/method/(method-id)¶
- DELETE /storage/(storage-id)/method;url=(url)¶
Delete an access method to storage.
Role: _storage_write
Importing/exporting a storage definition¶
Vidispine can export a definition document describing all the files within a storage, which can later be imported to a new storage on a different Vidispine instance.
Export a storage definition¶
- GET /storage/(storage-id)/export¶
Creates a StorageImportDocument that describes every file on the storage. This should be saved to a file which can later be used to import the storage definition.
Produces: - application/xml, application/json – StorageImportDocument
Role: _storage_read
Import a storage definition¶
- POST /storage/import¶
Creates a new storage based on the StorageImportDocument. A file entity will be created for each entry in the document, if a file with that ID does not already exist. Finally, a storage method will be added, with the path supplied in the call.
Query Parameters: - path – The file system path to where the files are located.
Accepts: - application/xml, application/json – A StorageImportDocument
Role: _storage_write
Evacuating storages¶
If you would like to delete a storage, but you still have files there which are connected to items, you can first trigger an evacuation of the storage. This will cause Vidispine to attempt to delete redundant files, or move files to other storages. Once the evacuation is complete, the storage will get the state EVACUATED.
Evacuate storage¶
- PUT /storage/(storage-id)/evacuate¶
Trigger evacuation of a storage.
Role: _storage_write
Cancel evacuation of a storage¶
- DELETE /storage/(storage-id)/evacuate¶
Cancel the evacuation process on a storage.
Role: _storage_write
Key-value metadata¶
Storages support key-value metadata.
Reserved keys¶
Certain keys are used to control the behavior of a storage, they must not be used to store generic metadata.
- closeLimit¶
An integer specifying the number of minutes until the system automatically considers an open file to be closed.
Default: 5
- lostLimit¶
An integer specifying the number of minutes until the system automatically considers a missing file to be lost.
Default: 10
- toAppearLimit¶
An integer specifying the number of minutes the system waits for a file to appear before considering it to be missing.
Default: 10
- refreshInterval¶
The interval (in seconds) that Vidispine will compare the list of file seen on disk with the list of files in the database. This to fix any possible mismatch.
Default: 900
- hashMode¶
Set to false to disable hash computation for this storage.
Default: true
- additionalHash¶
A comma-separated list of additional hashing algorithms that should be applied to files. E.g.: MD5,SHA-256
Default: (none)
- fileListBatchSize¶
Maximum size of list of files that are updated in the database at the same time.
Default: 100000 Since: 4.0.3
- keepMissingFiles¶
If set to false then missing files that do not belong to any items will be removed from the database instead of being marked as lost.
Default: To the value of the configuration property keepMissingFiles. Since: 4.1
- keepEmptyDirectories¶
Do not delete empty parent directories when deleting the last file in a directory, see Parent directory management.
Default: To the value of the configuration property keepEmptyDirectories. Since: 4.2.5
- statsPerSecond¶
An integer specifying the maximum number of stat system calls that are made to the storage (only file:// URIs). See also the configuration property statsPerSecond.
Default: (none) Since: 4.1.1
- refreshOnStart¶
By default, Vidispine synchronizes the database with the file system on start-up. On very large systems, this can take a very long time. Set this parameter to false, and use the rescan functionality (see below) instead.
Default: true Since: 4.1.1
- deleteFileIfNotFound¶
If set to true: if file that is marked for deletion cannot be deleted, but is not found on the file system, it is assumed to be deleted.
Default: false Since: 4.1.2
- excludeFilter¶
A regular expression. Any file with a path (relative to the storage’s root folder) that matches the expression will be ignored. Note that the expression must match the entire path, not only a part of the path.
Default: (none) Since: 4.2
- detectRenamedFiles¶
If this metadata is set to true, whenever a file belonging to an item is marked as LOST, and there is another file on the same storage with the same hash value, the new file is associated with the item instead.
Other possible values are all, where the old (lost) file resided on any storage, or a comma-separated list of identifiers of storages on which the old (lost) file resided on.
Default: (none) Since: 4.2
- probeFileBeforeClosing¶
If this metadata is set to true, Vidispine will attempt to read OPEN files before marking them as CLOSED. This is useful for Windows storages where the metadata is constant while a file is copied to the storage, but the file cannot be read (file is busy).
Default: false Since: 4.2.1
- vxaId¶
If this metadata field is set, the storage will be automatically deleted if the share is removed from the Vidispine Server Agent.
Default: (none) for regular storages, automatically set by VSA when a share is added and a storage is created. Since: 4.4
Key-value method metadata¶
Storage methods support key-value metadata.
Reserved keys¶
- sqsName¶
The name of Amazon SQS queue to poll for S3 file events. See S3 Event Notifications for more information.
This key is only read for S3 methods.
- sqsEndpoint¶
The endpoint of Amazon SQS queue.
This key is only read for S3 methods.
Example: sqs.eu-west-1.amazonaws.com
Size units¶
In the query parameters above, size can be specified in number of bytes as integer [ [ whitespace ] unit ] where the multiplier unit can be one of (case insensitive):
Unit | Factor |
---|---|
KB | 10001 |
K, KiB | 210 |
MB | 10002 |
M, MiB | 220 |
GB | 10003 |
G, GiB | 230 |
TB | 10004 |
T, TiB | 240 |
PB | 10005 |
P, PiB | 250 |
EB | 10006 |
E, EiB | 260 |
ZB | 10007 |
Z, ZiB | 270 |
YB | 10008 |
Y, YiB | 280 |
It should be noted that currently, Vidispine has a 64-bit limit on the size of a storage, which means that multipliers larger than ZB are of limited use.