Accommodating Hybrid Retrieval in a Comprehensive Video Data(5)

objects are linked together in the same video segments, and the semantic features contain spatio-

temporal features]

EDICS

There are two paths to search for the low-level image features. Entry point one is simply to search from theimage feature object collection. Entry point two is to search from (d) to get the visual objects. Then, usingthe link between the visual object and the image feature object, low-level image feature can be obtained. Entry point 1: Image-Feature

Entry point 2: (d) à Visual-Object à Image-Feature(1) To search for the low-level image feature of a semantic feature object, there are two search paths that

can achieve. Entry point one is to search the image feature from the collection of image feature objects.Then, it is to get the visual object that is pointed by the image feature object. Using (d), semantic featureobject can be retrieved. Entry point two is similar to entry point one but it starts from the end of theentry point one.

Entry point 1: Image-Feature à Visual-Object à (d) à Semantic-Feature

Entry point 2: Semantic-Feature à (d) à Visual-Object à Image-Feature

(2) To search for the low-level image feature of a visual object, the search paths are shown as follows.

Entry 1. Image-Feature à Visual-Object

Entry 2. Visual-Object à Image-Feature

(3) To search for activities occurred in a video program, it is first to use the low-level image features to

search for the semantic feature objects and the visual objects (refer to the above point (1) and (2) of theHybrid Search). Then follow the Entry point: Semantic-Feature & Segment List à Semantic-Feature /Segment-List à Segment.

3.2.3 Additional retrieval method

With respect to CBR, some additional video object comparison operators are defined and introduced intoour query language, as summarized below.

(1) Keyframe similar_to Keyframe

By color, shape, texture, or any combination;

By visual objects (i.e. the number of similar visual objects).

(2) Segment similar_to Segment

By the number of Keyframes deemed as similar (e.g. >50%);

By the temporal ordering of Keyframes.

(3) Scene similar_to Scene

By the number of Segments;

By the temporal ordering of Segments;

By the number of Activity/Event/Motion/Object from the Activity Model [CL99b].

(4) Video similar_to Video

By the number of Scenes;

By the temporal ordering of Scenes;

EDICS

Those method are based on a content-based video similarity model, whose implementation details aregiven in and can be referred to [WZ00b].

3.3 Language Syntax and Query Refinement

3.3.1 Syntax of CAROL/ST with CBR

The complete syntax of VideoMAP+ query language (i.e. CAROL/ST with CBR) is summarized below,with the shaded clauses being the ones not yet supported in the current version, but planned to be devised: SELECT <object | attribute of object> [{,<object | attribute of object>}]

FROM <object class> [{,<object class>}]

[ WHERE <search condition>

[{AND <search condition>}] ]

[GROUP BY <object | attribute of an object> [HAVING <search condition>]]

[ORDER BY <object | attribute of an object> [ASC | DESC] ] ;

For the WHERE clause, there are four kinds of search conditions:

1. Composite condition:

<object variable 1> HAS <object class> <object variable 2> [BY OID=<object id> | BY

ONAME=<object name>]

If object variable 1 is linked with another object (either through the inheritance or composite

relationships), object variable 1 can be retrieved by the composite condition.

2. Comparative condition:

<attribute of object> <comparison operator> <value>

If an object contains an attribute that is a simple data type, the attribute can be compared with the samesimple data type.

3. Spatio-Temporal condition:

<object variable> AT <location operator> [FOR <comparison operator> <frame number>]

If an object is annotated with some spatio-temporal features, it can be retrieved by specifying its locationand time duration.

4. Similarity condition:

<object variable 1> SIMILAR_TO <object class> <object variable 2> [BY OID=<object id> | BYONAME=<object name>] [BY [COLOR | TEXTURE]]

If an object is associated with some visual feature objects, the object can be processed with the similaritymeasurement of images. Therefore, the <object class> above should be restricted to Keyframe, Segment,Scene, and Video objects.

EDICS

3.3.2 Query refinement and feedback

Supported by CAROL/ST with CBR, queries in VideoMAP+ are flexible and able to accommodatevarious requirements. In particular,

a single feature query will be simple to handle, since it uses the separated search schema: text-based

retrieval or content-based retrieval. Text-based retrieval requires exact match; however, content-basedretrieval adopts inexact ("fuzzy") match.

a range query is a query that explicitly specifies a range of values for the feature weight, for example:

50% color, 30% texture, 20% shape. The weight for feature vector can be specified too.

a heterogeneous feature query such as “find video segments similar to the sample segment1, and in

addition satisfy the annotation restrictions” may involve convolution. Since text-based and content-based are both considered, different search paths and match methods should be incorporated. Thisproblem can be tackled as follows:

1) Intersection operation: The query can be any video object, maybe Keyframe, Segment, Scene,Video or Feature. Text-based s …… 此处隐藏：5899字，全部文档内容请下载后查看。喜欢就下载吧 ……