Migration to search version 2¶
This page describes how to enable search 2 in your system.
Important
Search version 2 still is in beta phase.
Configure search version¶
The configuration property searchIndexSettings defines which search version to use. This has an effect on
the index to be filled by VidiCore, which search APIs are usable, and which search version is used for internal searches.
See searchIndexSettings for available options.
Note
- Indexing will only fill the index versions enabled by
searchIndexSettings. Manual reindex operation on disable indices will fail. - When changing this property a reindex is required (except for changing between
NewAndOld_UsingOldSearchandNewAndOld_UsingNewSearch). - Both search versions come with a different feature set. Refer to Difference between both searches for a comparison of the two search versions.
Usage of both indices in parallel¶
It is possible to fill both indices in parallel. If the property searchIndexSettings is set to either
NewAndOld_UsingOldSearch or NewAndOld_UsingNewSearch both indices will be indexed.
The reindex for the chosen indices happens automatically for adding, updating or removing metadata for all indexed entities. To trigger
a manual reindex, it is necessary to specify the kind of document, which should be indexed (item, collection,shape ...) and
which index model you use (search version 1 index = PARENT_CHILD, search version 2 index = NESTED). A manual reindex will trigger
an index for all chosen entities, see Re-indexing metadata.
Note
- For other value of this property only one index is used in all indexing operations.
- For internal searches it has to be decided which index model is to use. If the older index model is used for internal searches, but the direct search is performed with the search2 API resources results may differ.
- If both indices must be up to date, the property must not be
OnlyNeworOnlyOld.
Usage of both search API versions in parallel¶
It is still possible to use both search API versions in parallel.
The search version 1 API can be reached with the already existing API endpoints, see Search version 1. For search version 2 new API-calls were implemented, see Search version 2.
For a step-by-step migration between the search versions, the property searchIndexSettings should be set to
NewAndOld_UsingOldSearch or NewAndOld_UsingNewSearch.
Libraries¶
To migrate libraries the search version 2 it is required to recreate all libraries. It is not possible to transfer the existing libraries search version 2 due to changes in the library creation process. To recreate and use the libraries, please use the existing API, see Libraries.
Difference between both searches¶
- This chapter explains the main differences between both searches.
- The search version 2 automatically filters all filterable input. The actual search is now smaller and faster.
- The index in search version 2 has one document per item or collection. The index in search version 1 has one document per timespan. This way the index for the search version 2 is smaller and the performance is increased.
- Events like concrete timespans or direct event or group search is now more efficient and more detailed.
- The search version 2 can only be used via the new API-endpoints Search version 2.
- It is necessary to clearly separate search criteria in generic elements (field, operator) and timed elements (timespans).
- These API calls only work for searching via the OpenSearch index. All other operations not requiring a search index continue working as before.
Interval¶
The interval element of the ItemSearchDocument is not used anymore in search version 2. To achieve the same results, it is necessary to enter all search related fields in the correct element of the SearchDocument.
- Interval =
allcan be achieved if the search is performed with the field or operator AND the timespans elements of the SearchDocument.- Interval =
timedcan be achieved if the search is performed with the timespans element of the SearchDocument.- Interval =
genericcan be achieved if the search is performed with the field or operator elements of the SearchDocument.
Filter¶
The SearchDocument has no direct filter value. In the search version 2, it is sufficient to search with a simple
field for receiving the correct result. Search version 2 automatically filters for values which qualify for filtering
e.g. keywords, integer or boolean. Following examples produce the same result:
search version 1:
<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<filter operation="OR" name="productType">
<field>
<name>type</name>
<value>pc</value>
</field>
<field>
<name>type</name>
<value>phone</value>
</field>
</filter>
</ItemSearchDocument>
search version 2:
<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<field>
<name>type</name>
<value>pc</value>
<value>phone</value>
</field>
</SearchDocument>
New in version 25.2.4.
The SearchFieldType has a new element called filterName. This is used to specify an element
which can be used in the exclude element of the SearchFacetType to remove search criteria for the facet count.
Groups¶
The search element group of the ItemSearchDocument is not used in the search version 2. If a group is used in version 2,
it has to be searched with all subgroups and fields as on single field. If a search is performed with only the group name, but not with
the field name, the search will fail.
<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<field>
<name>Group/subgroup/field_name</name>
<value>example</value>
</field>
<field>
<name>Group/subgroup/another_field_name</name>
<value>example</value>
</field>
<field>
<name>Group/another_subgroup/field_name</name>
<value>example</value>
</field>
</SearchDocument>
The search within groups can not be supported with the new index structure of the search version 2. Meaning if you have a MetadataDocument with multiple entries of the same group in the same timespan, the search version 2 can not differ between values of those groups. Consider a collection with groups like this:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MetadataDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<revision>VX-2,VX-1</revision>
<timespan start="-INF" end="+INF">
<group inheritance="true" uuid="4b377d31-8f5b-4d82-af17-cd049db0b028" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_example_group</name>
<group uuid="4bc5f648-0e2b-413b-aeb3-13c58f269639" user="admin" timestamp="2025-11-04T12:03:50.773+01:00"
change="VX-2">
<name>VX_example_subgroup1</name>
<group uuid="fe8fcfcd-847c-40b0-bb18-f47d2feeb809" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_example_subgroup2</name>
<field uuid="55f22990-0b08-4737-a68a-5692802f13b7" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_number_field</name>
<value uuid="8deeca9b-7c07-4d9b-a38e-e28f5a4e4cb9" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">1234567890
</value>
</field>
<field uuid="1518348f-d5bc-40e6-8601-d9717b8965f2" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_string_field</name>
<value uuid="bb5f8542-d2cd-4d83-8e50-6081dbe2c01d" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">This is one of both groups for this example
</value>
</field>
<field uuid="42ddb9b5-3b78-4eb6-937a-5ddb2c4a1d74" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_string_exact_field</name>
<value uuid="ab81fc65-16e5-4c67-93ea-456acd1e2504" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">First example value
</value>
</field>
<field uuid="f3a6d905-f427-40cd-b495-5c165edcd285" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_boolean_field</name>
<value uuid="20743e8c-8a89-4875-9040-6320f3c746cc" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">true
</value>
</field>
</group>
<group uuid="c3493277-55bf-4381-919e-231238d26e29" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_example_subgroup2</name>
<field uuid="5c8ce7b8-0aab-45ae-b647-1725eff461b4" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_string_field</name>
<value uuid="3e1a5baf-5a70-4b36-95dc-3c20be78aad0" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">This is the second of both groups for this example
</value>
</field>
<field uuid="e86ad9b8-c44d-4f9a-b641-af5ae94d8cde" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_string_exact_field</name>
<value uuid="36837235-82f9-4d6b-b6db-0e8e1dc34654" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">Second example value
</value>
</field>
<field uuid="4a820efb-9662-4d8f-b14f-0fcb460b30b0" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_boolean_field</name>
<value uuid="f3e67f7d-2a15-414b-9f8d-111fe754f465" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">false
</value>
</field>
<field uuid="10ddab16-7231-4eef-aaa9-f858cd5d5dc5" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">
<name>VX_number_field</name>
<value uuid="eb2ba0d5-c6d5-453f-8e61-bd70c3593825" user="admin"
timestamp="2025-11-04T12:03:50.773+01:00" change="VX-2">123456
</value>
</field>
</group>
</group>
</group>
</timespan>
</MetadataDocument>
Now a search is performed with the search version 2:
<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<operator operation="AND">
<field>
<name>VX_example_group/VX_example_subgroup1/VX_example_subgroup2/VX_boolean_field</name>
<value>false</value>
</field>
<field>
<name>VX_example_group/VX_example_subgroup1/VX_example_subgroup2/VX_string_exact_field</name>
<value>First example value</value>
</field>
</operator>
</SearchDocument>
The search 2 is looking for the string exact value First example value of the first VX_example_subgroup2 and the boolean value false
in the second VX_example_subgroup2. Because all groups and fields are named equally and saved in one timespan document, search version 2
only searches if those fields in the format Example_group/Example_subgroup1/Example_subgroup2/... containing the searched values in one
timespan not one group.
MetadataGroups¶
The direct search for only metadata groups is not supported directly anymore in search version 2.
To obtain corresponding results a search containing highlighting and text or field search has to be performed.
Consider a group called V3_Football is searched with the value in the text search, the highlighting fields
have to contain all subgroups and fields of this group to determine, which field (and if fitting which timespan)
contains this value.
Note
- Each field which has to be highlighted, has to be send with all groups it belongs to. The highlighting will be performed on those exact fields. Any other field might not be highlighted.
<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<field>
<name>V3_Season</name>
<value>2021/2022</value>
</field>
<timespans operation="OR">
<timespans name="Location_Pitch">
<fields>
<name>V3_Event_Football/V3_Event_Location_Pitch</name>
<value>PASS</value>
</fields>
</timespans>
</timespans>
<highlight>
<prefix><em></prefix>
<suffix></em></suffix>
<field>V3_Season</field>
<timespansField>V3_Event_Football/V3_Event_Location_Pitch</field>
</highlight>
</SearchDocument>
Comparison for search calls of version 1 and version 2¶
| Search version 1 | Request body search 1 | Search version 2 | Request body search 2 |
|---|---|---|---|
| GET API/collection/{id}/item | no search body | PUT API/search2/item | {
"field": [
{
"name": "__collection",
"value": [
{
"value": "VX-21"
}
]
}
]
}
|
| PUT API/collection/{id}/item | no/empty ItemSearchDocument | PUT API/search2/item | {
"field": [
{
"name": "__collection",
"value": [
{
"value": "VX-21"
}
]
}
]
}
|
| GET API/collection/{id}/collection | no search body | PUT API/search2/collection | {
"field": [
{
"name": "__child_collection",
"value": [
{
"value": "VX-21"
}
]
}
]
}
|
| PUT API/collection/{id}/collection | empty ItemSearchDocument | PUT API/search2/collection | {
"field": [
{
"name": "__child_collection",
"value": [
{
"value": "VX-21"
}
]
}
]
}
|
| PUT API/search/autocomplete | {
"field": "V3_FirstName",
"text": "J",
"maximumSuggestions": 10
}
|
PUT API/search2/autocomplete | {
"field": "V3_Event_Football/V3_PrimaryParticipant/V3_FirstName",
"text": "J",
"maximumSuggestions": 10,
"nested": true
}
|
| PUT API/collection/metadata-group | MetadataGroupSearchDocument | MetadataGroups | |
| PUT API/search/shape | ShapeSearchDocument | PUT API/search2/shape | SearchDocument |
| GET API/search/shape | no search body | PUT API/search2/shape | no/empty SearchDocument |
| PUT API/search/file | FileSearchDocument | PUT API/search2/file | SearchDocument |
| GET API/search/file | no search body | PUT API/search2/file | no/empty SearchDocument |
| GET API/search | no search body | PUT API/search2 | no/empty SearchDocument |
| PUT API/collection | ItemSearchDocument | PUT API/search2/collection | SearchDocument |
| GET API/collection | no search body | PUT API/search2/collection | no/empty SearchDocument |
| PUT API/item | ItemSearchDocument | PUT API/search2/item | SearchDocument |
| GET API/item | no search body | PUT API/search2/item | no/empty SearchDocument |