Jobs¶
Jobs make up the long running tasks in Vidispine. They are created in response to requests that would otherwise not be able to respond in time, such as import, export and transcode requests.
Managing jobs¶
Create a job¶
See Creating jobs.
List all jobs¶
-
GET
/job
¶ Return jobs matching the criteria given.
Changed in version 5.1: The priority parameter was added.
Query Parameters: - jobmetadata (string[]) –
Multiple query parameters can be specified. If no query parameters are specified, all jobs are returned.
- key
=
value - Filter out only the jobs that has job metadata according to the filter criteria.
Note: the metadata field
item
is generated and cannot be filtered on, instead the fielditemId
should be used.Note:
=
is part of the query parameter, and has to be encoded (%3d
). - key
- metadata (boolean) –
true
- Include job metadata with all jobs.false
(default) - Do not include job metadata with all jobs.
- idonly (boolean) –
true
- Only return a list of idsfalse
(default) - Return job information such as job status
- starttime-from (string) – ISO 8601 timestamp. Return only jobs started after and at the given timestamp.
- starttime-to (string) – ISO 8601 timestamp. Return only jobs started before and at the given timestamp.
- finishtime-from (string) – ISO 8601 timestamp. Return only jobs finished after and at the given timestamp.
- finishtime-to (string) – ISO 8601 timestamp. Return only jobs finished before and at the given timestamp.
- step (boolean) –
true
- Include step information in the job listing.false
(default) - Do not include step information in the job listing.
- type (string) – Comma-separated list of job types. Default is
all
, return all jobs. - state (string) – Comma-separated list of job states. Default is
all
, return all jobs. - priority (string) – Comma-separated list of job priorities. Default is
all
, return all jobs. - first (integer) – Return jobs from that number in the list of sorted jobs. Default is
1
, the first jobs. - number (integer) – Return at most that number of jobs. Default returns the first 100 jobs.
- sort (string) –
List of form
field (asc|desc)[, ...]
. Sort by specific fields.jobId
type
state
user
startTime
priority
- user (boolean) –
true
(default) - Include only jobs created by current userfalse
- Include all jobs
- field (string) – Comma-separated list of fields to include in the result metadata.
Produces: - application/xml, application/json – JobListDocument
- text/plain – CRLF-delimited list of job ids.
Role: _job_read
- jobmetadata (string[]) –
Retrieve a job¶
-
GET
/job/
(job-id)¶ Return information about specified job.
When returning in format
text/plain
, only a string representation of the state is returned.Query Parameters: - metadata (boolean) –
true
- Include job metadata with all jobs.false
(default) - Do not include job metadata with all jobs.
- field (string) – Comma-separated list of fields to include in the result metadata.
Status Codes: - 404 Not found – Invalid id
Produces: - application/xml, application/json – JobDocument
- text/plain – State of job
Role: _job_read
- metadata (boolean) –
Search and count jobs¶
-
PUT
/job/search
¶ New in version 5.2.
Return jobs matching the given criteria and chosen facets for the result-set. There are three types of facets:
type
Counts occurrences of each job typestate
Counts occurrences of each job stateuser
Counts occurrences of each user
Query Parameters: - jobmetadata (string[]) –
Multiple query parameters can be specified. If no query parameters are specified, all jobs are returned.
- key
=
value - Filter out only the jobs that has job metadata according to the filter criteria.
Note: the metadata field
item
is generated and cannot be filtered on, instead the fielditemId
should be used.Note:
=
is part of the query parameter, and has to be encoded (%3d
). - key
- metadata (boolean) –
true
- Include job metadata with all jobs.false
(default) - Do not include job metadata with all jobs.
- idonly (boolean) –
true
- Only return a list of idsfalse
(default) - Return job information such as job status
- starttime-from (string) – ISO 8601 timestamp. Return only jobs started after and at the given timestamp.
- starttime-to (string) – ISO 8601 timestamp. Return only jobs started before and at the given timestamp.
- finishtime-from (string) – ISO 8601 timestamp. Return only jobs finished after and at the given timestamp.
- finishtime-to (string) – ISO 8601 timestamp. Return only jobs finished before and at the given timestamp.
- step (boolean) –
true
- Include step information in the job listing.false
(default) - Do not include step information in the job listing.
- type (string) – Comma-separated list of job types. Default is
all
, return all jobs. - state (string) – Comma-separated list of job states. Default is
all
, return all jobs. - priority (string) – Comma-separated list of job priorities. Default is
all
, return all jobs. - first (integer) – Return jobs from that number in the list of sorted jobs. Default is
1
, the first jobs. - number (integer) – Return at most that number of jobs. Default returns the first 100 jobs.
- sort (string) –
List of form
field (asc|desc)[, ...]
. Sort by specific fields.jobId
type
state
user
startTime
priority
- user (boolean) –
true
(default) - Include only jobs created by current userfalse
- Include all jobs
- field (string) – Comma-separated list of fields to include in the result metadata.
Status Codes: - 404 Not found – Field not valid
Produces: - application/xml, application/json – JobListDocument
- text/plain – CRLF-delimited list of job ids.
Accepts: - application/xml, application/json – JobSearchDocument
Role: _job_search
Example¶
A faceted search should count the occurrences of each
type
, state
and user
for all existing jobs.
It is possible to supply attributes name
, minCount
, maxCount
and maxResults
.
Facets can be named to make it easier to distinguish between different facets.
By supplying attributes minCount
, maxCount
any fields that has a count lower/higher
than the specified values will be excluded. By default a minimum count of 0
and a maximum count of integer max size is returned. By default, all facet counts will be returned.
By using the maxResults
attribute, this behaviour can be changed.
PUT /job/search
Content-Type: application/xml
<JobSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<facet count="true" name="JobTypes">
<field>type</field>
</facet>
<facet count="true" minCount="1" maxCount="5" maxResults="10">
<field>state</field>
</facet>
<facet count="true">
<field>user</field>
</facet>
</JobSearchDocument>
<JobListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<hits>2</hits>
<job>
<jobId>VX-1</jobId>
<user>admin</user>
<started>2020-02-19T13:53:21.110Z</started>
<finished>2020-02-19T13:53:21.242Z</finished>
<status>FINISHED</status>
<type>RAW_IMPORT</type>
<priority>MEDIUM</priority>
</job>
<job>
<jobId>VX-2</jobId>
<user>admin</user>
<started>2020-02-19T13:53:21.110Z</started>
<finished>2020-02-19T13:53:21.242Z</finished>
<status>STARTED</status>
<type>AUTO_IMPORT</type>
<priority>MEDIUM</priority>
</job>
<facet name="JobTypes">
<field>type</field>
<count fieldValue="RAW_IMPORT">1</count>
<count fieldValue="AUTO_IMPORT">1</count>
</facet>
<facet>
<field>state</field>
<count fieldValue="FINISHED">1</count>
<count fieldValue="STARTED">1</count>
</facet>
<facet>
<field>user</field>
<count fieldValue="admin">2</count>
</facet>
</JobListDocument>
Run a job step¶
-
POST
/job/
(job-id)/step/
(step)/run
¶ Start a job step. The job must be in HOLD state.
Status Codes: - 404 Not found – Invalid id
Role: _job_write
Modify a job¶
-
PUT
/job/
(job-id)¶ Updates the job by setting job priority
Query Parameters: - priority (string) – Change the job priority. One of the valid job priorities <job_priority>: Default is
MEDIUM
Status Codes: - 404 Not found – Invalid id
Role: _job_write
- priority (string) – Change the job priority. One of the valid job priorities <job_priority>: Default is
Abort a job¶
-
DELETE
/job/
(job-id)¶ Does not delete the job, but aborts it.
To delete one or more jobs, use
DELETE /job
.The job is marked for abortion, but the call may return before all tasks have been killed. Hence, the status return by this call is likely to be ABORTED_PENDING rather than ABORTED. Caller should poll the status of the job or use job notifications to find out when job has been fully aborted.
Query Parameters: - reason (string) – Reason for cancellation.
- cleanup (boolean) –
true
(default) - Run cleanup steps before aborting.false
- Skip the cleanup steps, unless the job is already running. For READY jobs this means that the job will immediately be marked as ABORTED.
Status Codes: - 404 Not found – Invalid id
Role: _job_write
Create a duplicate job¶
-
POST
/job/
(job-id)/re-run
¶ Retrieves an existing job, duplicates it and starts the duplicated version.
Query Parameters: - priority (string) – The priority of the new job. If no priority is specified then the priority of the existing job will be used.
Status Codes: - 404 Not found – Invalid id
Produces: - application/xml, application/json – JobDocument
Role: _job_write
Start a job with custom type¶
-
POST
/job
¶ Starts a new job, of the type specified in the
type
parameter.Changed in version 5.0.
Additional job metadata can also be added using an optional SimpleMetadataDocument. If any
jobmetadata
keys would collide between the query parameters and the SimpleMetadataDocument, the key and value from the SimpleMetadataDocument would have precedence over the query parameter.Query Parameters: - type (string) – Required. The job type name.
- notification (string) – The placeholder job notification to use for this job.
- notificationData (string) – Any additional data to include for notifications on this job.
- priority (string) – The priority to assign to the job. Default is
MEDIUM
. - jobmetadata (string[]) – Additional information for the job task.
- holdJob (boolean) – If set to
true
, the job will be created in aHOLD
state. Default isfalse
.
Status Codes: - 400 – Invalid job type name
Accepts: - application/xml, application/json – SimpleMetadataDocument
Produces: - application/xml, application/json – JobDocument
Role: _job_write
Job problem conditions¶
Jobs can enter the state WAITING if a recoverable problem has occurred. Depending on the problem the system might resolve itself or require manual assistance, e.g. out of storage space.
List all job problems¶
-
GET
/job/problem
¶ Returns a list of unresolved problems, together with what jobs are waiting for them to be resolved.
Produces: - application/xml, application/json – JobProblemListDocument.
Role: _job_read
List all problems for a job¶
-
GET
/job/
(job-id)/problem
¶ Retrieves a list of problems that affects the specified job.
Status Codes: - 404 Not found – Invalid id
Produces: - application/xml, application/json – JobProblemListDocument
Role: _job_read
Job states¶
Job states¶
The following states are defined for a job:
- READY
- Job can be run, in queue.
- STARTED
- One or several job steps are running.
- VIDINET_JOB
- One or several job steps are running on Vidinet.
- FINISHED
- The job has finished with success.
- FINISHED_WARNING
- The job has finished, but a non-critical step failed.
- FAILED_TOTAL
- The job has finished, but a critical Job step has failed.
- WAITING
- The job is waiting for a condition.
- ABORTED_PENDING
- A request to abort the job has been made.
- ABORTED
- The job is aborted, and all job steps have finished.
- HOLD
- The job and its step(s) will not be started by VidiCore, instead the Run a job step endpoint has to used to start each individual job step.
Step states¶
The following states are defined for a job step (task):
- NONE
- Job step has just been initialized. Normally this should not be returned.
- READY
- Job step about to start.
- STARTED
- Job step started, running in a transaction. (Short tasks.)
- STARTED_ASYNCHRONOUS
- Job step started, running outside of an transaction. (Longer tasks, or tasks that cannot execute in a Java class.)
- STARTED_PARALLEL
- Job step has started, and signals that other parallel tasks can start.
- STARTED_PARALLEL_ASYNCHRONOUS
- Job step has started, and signals that other parallel tasks can start.
- STARTED_SUBTASKS
- Job step has started, and will be performed in multiple subtasks.
- FINISHED
- Job step has finished successfully.
- FAILED_RETRY
- Job step has failed, but will be retried.
- FAILED_FATAL
- Job step has failed, and will not be retried.
- WAITING
- Job step is waiting for a condition, see List all problems for a job for cause.
- DISAPPEARED
- The job worker was missing (possible cause is a restart of the application server). The job step will be re-run.
Job priority¶
Jobs have a priority setting that determines their order of execution. The following priority levels exists, from lowest to
highest: LOWEST
, LOW
, MEDIUM
, HIGH
, HIGHEST
, and IMMEDIATE
.
Warning
Jobs with priority IMMEDIATE
are always started, even if the max concurrent jobs
limit is reached. This could impact system performance. To execute the job with IMMEDIATE
priority the user must be a super user, that is, have role _super_access_user
.
Job types¶
The following job types exists. They can also be retrieved
using GET /jobtype
.
- NONE
- Not used.
- IMPORT
- Not used.
- PLACEHOLDER_IMPORT
- Regular import (using URI or file id to existing or new item).
- RAW_IMPORT
- Import where essence is in request body.
- AUTO_IMPORT
- Import using auto import rules.
- SHAPE_IMPORT
- Import using URI or file id to a shape.
- SIDECAR_IMPORT
- Import a sidecar file to an existing item.
- ESSENCE_VERSION
- Import a new essence version.
- TRANSCODE
- Transcode of item.
- TRANSCODE_RANGE
- Transcode of part of item.
- CONFORM
- Conform an item sequence.
- TIMELINE
- Conform a timeline.
- THUMBNAIL
- Create thumbnails or posters of item.
- ANALYZE.
- Do analysis of item.
- SHAPE_UPDATE
- Update shape information of item.
- RAW_TRANSCODE.
- Send transcode job directly to transcoder.
- EXPORT
- Export item to remote location.
- COPY_FILE
- Copy file (and keep track of new copy).
- MOVE_FILE
- Move file.
- DELETE_FILE
- Delete file.
- LIST_ITEMS
- Generate item report.
- FILE_ANALYZE
- Analyze a file to deduce its shape.
- IMF_ANALYZE
- Analyze an IMF package to deduce its shape.
List all job types¶
-
GET
/jobtype
¶ Get list of job types
Produces: - application/xml, application/json – URIListDocument
- text/plain – list of job types
Job metadata¶
Additional job metadata can be specified using the jobmetadata
parameter, when a job is created. Note that the equals sign is part
of the value of the query parameter, so it has to be URL encoded (%3d).
Hint
Prefer to always prefix any custom job metadata by the name of your
application, for example, myApp_customSetting
, to avoid
conflict with any existing or future job metadata used by Vidispine.
Reserved keys¶
-
smpteTimeCode
¶ The first frame to be included in transcoded output.
Type: String (SMPTE timecode)
-
lastSmpteTimeCode
¶ The last frame to be included in transcoded output.
Type: String (SMPTE timecode)
-
checksumMode
¶ Can be set to
transfer
to have the checksum computed during the transfer step of the import job.Note: This will not work if the files are transferred by the transcoder.
-
cerifyPriority
¶ The priority to assign jobs created in Cerify. One of
LOW
,MEDIUM
orHIGH
.Default: LOW