Search version 2 (beta)

Search version 2 is based on the same principles as search version 1. The SearchDocument has no intervals, joins, groups, metadataGroups, and filter elements as in ItemSearchDocument. Information on how functionality is made accessible in search version 2 can be found in Difference between both searches.

Applicable exceptions and new usage concepts are explained on Search version 2 and Migration to search version 2.

Note

To use the new search implementation we recommend to review and revise your metadata structure and clearly separate metadata in generic and timed data:

  • generic: Data that applies to the whole entity (item or collection), i.e. the generic -INF..+INF timespan, and only is set once per entity, for example: created, owner, accessGroups, admin. This corresponds to intervals=generic in search version 1.
  • timed: Data that is used in non-generic timespans and may set on multiple timespans. When searching for timed data, the criteria are defined in the timespans element of the SearchDocument. This corresponds to intervals=timed in search version 1.
  • If a search is performed with timed and generic data, the search operation corresponds a search with the element intervals=all in the search version 1.

New in version 25.4: VidiCore now supports multilingual search. It is enabled by setting useMultipleSearchIndices to true. When enabled, multiple language-specific indices can be created which analyze fields of type string for the index’ language.

On search operations the new query parameter searchLang specifies which language index should be used for search operations. The searchLang query parameter must equal one one the lang attribute values in MetadataValueType for VidiCore to determinate the correct index.

Please refer to Multilingual search migration before activating this feature. Indices for all languages need to be created before performing a search on one of them. See Search2 index creation (beta) for index creation.

Important

When setting useMultipleSearchIndices to true be sure to run POST /APIinit/indices or POST /APIinit to synchronize existing default indices of VidiCore for this feature. When initializing or recreating the default indices of VidiCore and the useMultipleSearchIndices is already set to true, the call POST /APIinit/indices or POST /APIinit is necessary as well.

SearchDocument explained

The following documentation explains the SearchDocument and all search options available. Please note that this only applies to search version 2.

  • text: With the text part VidiCore performs an overall search and returns all entities which contains the text value anywhere in their metadata documents. If the text value located is in an timed event, the result document contains specific timestamps and fields. When specifying multiple text values, an implicit OR is added. For example, if you search for text1 and text2, the result will contain all entities which contain either text1 or text2 in their metadata documents.

  • field: With the field part you search only for generic fields in the index. If a field is embedded in one or multiple group(s) please specify the group path to this field with this pattern: Group/field, Group/subgroup/field. Otherwise you simply search for the field. For examples for a field search with timespans please see Field elements.

  • operator: The operator allows combining multiple search criteria. Each operator contains a single operation. Supported operations are AND, OR, and NOT. With AND all criteria must be met. With OR at least one of the criteria must be met. With NOT the criteria must not be met. An operator may contain a list of field elements to which the operator is applied. See Operator for examples.

  • timespans: The outer timespans element may contain one or multiple inner timespans and a single operation.

    • timespans: Each timespans contains multiple fields, an operator and a name. The purpose of a timespans object is to search in one index field within all timespans of an entity. By using multiple timespans you can search multiple index fields within different timespans within an entity.

      • name: A unique specification for the timespans element for identification; this is essential for highlightíng. If multiple timespans are used in the search, it is necessary to specify a name for each timespans`. Otherwise the search will result in an exception and fail.
      • fields: Add all fields as explained. For examples for a field search with timespans please see Field elements.
      • operator: For more complex searches, one operator contains one operation and multiple fields. Additionally each operator contains multiple inner operators. Please be aware that an operator in timespans is not the same operator as in the ItemSearchDocument and contains only the previously called elements. See Operator for examples.

    • operation: The operation of the outer timespans element is applied to all inner timespans elements. Supported operations are AND, OR, and NOT.

  • sort: Sorts all matching entities s by the given criteria.

    • field: Field name as explained previously.
    • order: ascending or descending
    • nested: Specifies if the field is nested (timed data, fields in shapes or fields in files connected to shapes). This is a boolean value which default is false.

  • highlight:

    • prefix: the html start tag, e.g. for xml requests: &lt;em&gt; or for json requests: <em>
    • suffix: the html end tag, e.g. for xml requests: &lt;/em&gt; or for json requests: </em>
    • field: A list of field names only for generic data.
    • timespansField: A list of field names only for timed data.
    • matchingOnly: Specifies whether to highlight only fields that contain a search query match.

  • suggestion:

    • maximumSuggestions: Integer of how many suggestion should be returned.
    • accuracy: Double value of how accurate the suggestions should be.

  • autocomplete: Each autocomplete request must contain only one field name, a prefix of the word, and the boolean statement if the request is nested. Search 2 autocompletion requires a prefix.

    • field: Field name as explained previously.
    • text: The beginning of the wanted word.
    • nested: See sort explanation.
    • maximumSuggestions: The maximum of returned suggestions.

  • cursor: The cursor element is a marker to return entities after the given cursor. If the value of the cursor is *, the search returns entities from the beginning and returns a new cursor, which points to the end of the given entities. Use the received cursor to continue the search where you stopped.

  • facet: The facet element is used to get a count of matching items and collections. The result can contain a count or a range of the results. Search version 2 can search within timed data as well. For more details look at Faceting.

Text elements

With the text element of the SearchDocument you can perform an overall search with on or multiple values. The text element only performs a search on values of metadata of type string or string-exact.

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <text>text-example-value</text>
   <text>text-example-value2</text>
</SearchDocument>

Field elements

If a field is embedded in one or multiple group(s), it has to be searched as one field as following pattern: Group/field, Group/subgroup/field. Otherwise you simply search for the field name.

Note

For a search with groups, please note following information: Groups.

A search can be performed for only generic, only timed or for both kind of metadata. To combine both generic and timed metadata, the search input has to be sorted in the correct search elements.

Operator

The main operator and the operator in the timespans element are handled similarly. The main operator in the SearchDocument is expected once, while the timespans element contains a list of operator.

Timespans operator

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <timespans operation="AND">
        <timespans name="operator-example">
            <operator operation="AND">
                <field>
                    <name>V3_Event_Football/V3_PrimaryParticipant/V3_FirstName</name>
                    <value>Bob</value>
                </field>
            </operator>
            <operator operation="AND">
                <field>
                    <name>V3_Event_Football/V3_PrimaryParticipant/V3_Position</name>
                    <value>Midfielder</value>
                </field>
            </operator>
        </timespans>
    </timespans>
</SearchDocument>

Generic operator

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <operator operation="AND">
        <operator operation="OR">
            <operator operation="NOT">
                <field>
                    <name>V3_HomeTeam/V3_Player/V3_FirstName</name>
                    <value>Luke</value>
                </field>
                <field>
                    <name>V3_HomeTeam/V3_Player/V3_Position</name>
                    <value>Forward</value>
                </field>
            </operator>
            <operator operation="NOT">
                <field>
                    <name>V3_HomeTeam/V3_Player/V3_FirstName</name>
                    <value>Samuel</value>
                </field>
                <field>
                    <name>V3_HomeTeam/V3_Player/V3_Position</name>
                    <value>Forward</value>
                </field>
                </operator>
            </operator>
        </operator>
    </operator>
</SearchDocument>

Note

When searching for field values inside field groups you can search for different fields that are defined within the same group like shown in the above examples.

If items (or collections) have multiple instances of this field group, VidiCore will return all items matching the requested field within any instance of the group. However, it is not guaranteed that the match is within the same field group instance.

Example:

  • Item VX-1:

    • Group V3_Event_Football/V3_PrimaryParticipant instance 1:

      • V3_FirstName="Luke"
      • V3_Position="Forward"

    • Group V3_Event_Football/V3_PrimaryParticipant Instance 2:

      • V3_FirstName="Samuel"
      • V3_Position="Midfielder"

  • Item VX-2:

    • Group V3_Event_Football/V3_PrimaryParticipant instance 1:

      • V3_FirstName="Luke"
      • V3_Position="Midfielder"

    • Group V3_Event_Football/V3_PrimaryParticipant instance 2:

      • V3_FirstName="James"
      • V3_Position="Forward"

When searching for items with V3_Event_Football/V3_PrimaryParticipant/V3_FirstName="Luke" and V3_Event_Football/V3_PrimaryParticipant/V3_Position="Midfielder", VidiCore will return item VX-2 because instance 1 of the group is matching. VidiCore will also return item VX-1 because V3_Event_Football/V3_PrimaryParticipant/V3_FirstName="Luke" is present in group instance 1 and V3_Event_Football/V3_PrimaryParticipant="Midfielder" is present in Person group instance 2.

It is not possible to restrict search to having a match of both fields within the same group instance.

Highlighting

For highlighting you need to specify the prefix, the suffix, the wanted fields and if matching fields in timespans should be highlighted with specific timestamps. If you add a specific text value, the highlighting will return which of the given fields contains the value.

New in version 25.2.7.

To highlight, fields have to be sorted in either field for generic data or in timespansField for timed data. A combined highlight with generic and timed fields is possible.

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <text>text-example-value</text>
    <highlight>
        <prefix>&lt;em&gt;</prefix>
        <suffix>&lt;/em&gt;</suffix>
        <timespansField>V3_Event_Football/V3_Event_Location_Pitch</field>
        <field>V3_Season</field>
    </highlight>
</SearchDocument>

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <timespans operation="AND">
        <timespans name="Location_Pitch">
            <fields>
                <name>V3_Event_Football/V3_Event_Location_Pitch</name>
                <value>PASS</value>
            </fields>
        </timespans>
    </timespans>
    <highlight>
        <prefix>&lt;em&gt;</prefix>
        <suffix>&lt;/em&gt;</suffix>
        <timespansField>V3_Event_Football/V3_Event_Location_Pitch</field>
    </highlight>
</SearchDocument>

Sorting

If the nested element is true, then the given field is defined as timed data, in shapes or in files connected to shapes. If it is set to false, then the given field is generic in items or collections. The default value for sorting is the field created.

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <sort>
        <field>created</field>
        <order>ascending</order>
        <nested>false</nested>
    </sort>
</SearchDocument>

Autocomplete

If the nested element is true, then the given field is defined as timed data, in shapes or in files connected to shapes. If it is set to false, then the given field is generic in items or collections. The text element must be given in the autocomplete request. Without the text element, the request will fail. If the autocomplete request is send without field, the response will contain matching values which begin with the text element.

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <autocomplete>
        <text>S</text>
        <field>V3_HomeTeam</field>
        <nested>true</nested>
        <maximumSuggestions>10</maximumSuggestions>
    </autocomplete>
</SearchDocument>

Faceting

For faceting set the nested element to true if the count has to be performed for timespans fields, fields in shapes or fields in files connected to shapes. If set to false, the count will be performed on generic fields of items and collections.

<SearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <facet count="true" nested="false" name="V3_Event_Football/V3_PrimaryParticipant/V3_FirstName">
        <field>V3_Event_Football/V3_PrimaryParticipant/V3_FirstName</field>
        <exclude>V3_Event_Football/V3_PrimaryParticipant/V3_FirstName</exclude>
    </facet>
    <facet count="true" nested="false" name="V3_CollectionType">
        <field>V3_CollectionType</field>
        <exclude>V3_CollectionType</exclude>
    </facet>
</SearchDocument>

For more details and examples regarding the facet element see search version 1 - faceting.

New in version 25.2.4.

To use the exclude element from the SearchFacetType for search version 2, it is necessary to specify the filterName of the SearchFieldType. All exclude entries must match at least one filterName, otherwise the field won’t be excluded and added as a filter criteria for the facet count.