Sample queries were collected by interviewing a variety of users. These queries guided the design of indexes that can be used to answer the types of queries users want to ask. Three types of indexes were identified: 1) bibliographic (e.g., title, abstract, subject keywords, genre, director, cast, etc.), 2) structural (e.g., segments, scenes, shots, transitions, sound effects, etc.), and 3) content (e.g., keyframe sequences, scripts, objects and actors in scenes, etc.).
A mixed-mode query interface was built that allows a user to select a set of videos and/or frame sequences and incrementally modify the answer set. The interface includes relational, hierarchical browsing, and keyword search operations.