Vizard
This project proposes the manual creation of high-level metadata that will be applied to audiovisual clips. The metadata allows the director to create a work based on narrative association. Associated clips are presented in varying spatial arrangements dependent on the user’s previous choices.
High-level metadata is different from the low-level data created by automatic data generation systems. One such system is the EU-IST (Information Society Technologies) funded Vizard. Vizard uses ‘shot detection’ to analyse the physics of clips and produce distinct data that can be linked to each clip. This metadata is made up of ‘content elements’. These primarily functional ‘building blocks’ represent features of the video clips such as clip colour, contrast, motion and edit points.
Vizard has been created in response to the increasing problem of logging large amounts of digital video footage. Vizard will batch log many hours of footage and present a table of clips that can be automatically edited according to a script. The script is developed using an existing template. The template presents a similar story e.g. a wedding and lists the order of clips such as bride arrives at church, bride and groom leave church etc. The Vizard system imports the new footage and with some operator assistance it identifies the features of the clips and orders them in a similar pattern.
This system is too blunt an instrument with which to create a complex interactive narrative. The parameters it uses to define clips lack distinctive qualities that would lead to an intelligent and thoughtful presentation of the footage. The quality of the metadata created is too simple to clearly identify the main elements of each clip. However, the producers of the package have developed a system that could automate the post-production of simple, formulaic audiovisual presentations that are not intended to be anything more than the equivalent of a wedding video.
SMIL
Synchronised Multimedia Interaction Language (SMIL) is used to create metadata from video clips. Designed to be used with XML, SMIL creates metadata automatically so that it can be presented and referenced in much the same way as a web page. Future search engines will allow searches for video clips with specific content. This content can be words, visuals, music depending on the level of metadata. SMIL was originally released with RealPlayer technologies but its usage has been slow. Unlike HTML and graphic editors, XML is a complex programming language requiring significant technical skills and abilities. Despite its potential, most web designers have not used it.
High-Level Metadata
High-level metadata identifies the elements of the clip in complex concepts such as emotion levels and character development. With high-level metadata the filmmaker can reinterpret the clips to create new sub-narratives and associations. Metadata can also be used to apply new clip associations depending on the user’s previous choices. For example, a film clip shows a young male character reading the paper at a bus stop. Beside him waits an elderly woman. She looks vulnerable. We will interpret this scene based on previous clip elements. Thus, if the user previously viewed a scene where they saw the young man looking for work and being unable to find a job, our interpretation of his character would be sympathetic. Our interpretation of the same young man waiting at the bus stop would be quite different if we had previously watched him stealing in a local shop. In the former the character and the older person would have similar problems, no longer masters of their own destiny. In the latter we would fear for the safety of the older person while she shares space with this less lovable character. Both scenes are viable. In a linear presentation the young man may have been driven to steal due to his inability to secure a job. In the interactive presentation we are, as in real life, trying to fill in the blanks based on the bits of information we derive from the simultaneous presentation of associated clips.
The following flowchart lays out the presentation of clips. Clips A and B or optional. The user can view these clips if they click on them. Either way, the base clip plays out as is and this is our primary point of focus. This is the anchor to the storyline. Clips A and B provide some degree of character exposition which will differentiate our experience from that of the next user.
By incorporating a database array we can track the choices of the user as they progress through the clips. In later clips the user will only be given the option of viewing specific extra audiovisual materials if they have previously selected and viewed selected clips.
The following table is a simple database of clips. Parameters have been assigned to each clip to identify their content. Therefore clips can be selected ‘on the fly’ and presented as part of an ongoing narrative.
Clip |
Duration |
Image size |
Character |
Dialogue |
Audio |
Camera movement |
Location |
Clip location |
Timeline |
1 |
15 |
small |
1 |
Yes |
0 |
1 |
1 |
c:/clip/clip1small |
1 |
2 |
15 |
large |
2 |
Yes |
0 |
1 |
1 |
c:/clip/clip1large |
1 |
3 |
30 |
small |
1 |
No |
1 |
1 |
1 |
c:/clip/clip2small |
1 |
4 |
30 |
large |
1 |
No |
1 |
1 |
1 |
c:/clip/clip2large |
1 |
5 |
30 |
large |
1 |
Yes |
2 |
2 |
2 |
c:clip/clip3large |
2 |
Lev Manovich’s ‘SoftCinema’ uses such a system. This interactive film project uses a voiceover that tells a story. Visuals, audio clips, text and photographs are presented on screen in spatial montage depending on the users’ basic interaction. At various points of the voiceover an ‘anchor’ is applied and the clips are refreshed with a set presentation order. The user can then begin interacting with them again and changing the clips onscreen. This system was developed in Macromedia Director and uses and extensive database of over 700 clips shot in many countries. The clips are often only conceptually connected to the story and do not have any clear relevance. However, Manovich has employed Eisenstein’s theory of montage to present juxtaposed clips on screen with other ‘plastic’ elements that will be work together to create an individual response for each viewer.
I believe the narrative in interactive film needs to be engaging and character driven. This will promote interactive searching for more information about the characters. In her essay on ‘Scripting the User’ (from ‘Six Principles: Toward a Theory of Interactive Narrative’) Jessica Hammer has noted the importance of creating a ‘dramatic context for the user’. The user must have some idea about what they want to achieve by interacting with the story. Do they want to solve a problem or simply learn more about an interesting character and their predicament? The level of engagement will depend on the relevance of the offered clips to their train of thought.
Jessica Hammer also notes that the tracking of interactivity to influence future choices made available to the user has been used in games design. ‘Crash Bandicoot’ calculates the difficulty of each level depending on the users’ success in previous levels. This makes the game more enjoyable for the player as they will be presented with a less difficult version of the game if they are unable to play it well. Similarly, in interactive film it is possible to present the viewer with additional applicable scenes that run concurrently with the main narrative depending on their previous inputs.Other games such as ‘Grand Theft Auto: Vice City’ have created a scenario with locations, vehicles and characters. The player has a series of goals to achieve and can choose to undertake these mission or ignore them and simply navigate around the narrative. However, the structure of the game means that the player feels they are only in this scene to undertake the set task and therefore just get on with the game. This is a very good example of a narrative structure that completely engrosses the viewer so that they do not want to make decisions or do things outside the context of the narrative. They stay within the storyline.
Interactive adventure games such as ‘Traveller’ and ‘Dungeons and Dragons’ required the players to imagine a virtual world based on descriptive passages. They were then asked to navigate this world under the guidance of a ‘dungeon master’ or ‘referee’. The dungeon master ensured that the elements of the scenario were properly explained and kept the players on track within the context of the narrative. Without a board representing a tangible version of this virtual world it was a challenge for all players to ‘see’ the same things. With other board games such as ‘Car Wars’ the players could see what was happening and physically interact with their characters. ‘Car Wars’ required more strict set of rules to contain the narrative. Firstly, the board represented an enclosed space or arena where cars could drive about at speed. There was no world outside this arena and no possibility of the gameplay continuing once they left it. In comparison, the referee of adventure games that did not use a physical board could create an endless world of interactivity that could be tailored to match the game playing style of the players.
