27 Letters; Trajectories and Multimodality in Interactive Storytelling

This paper presents the initial findings from the making and a pilot study of “Letters to José,” an interactive, non-linear narrative. The events of this narrative were presented in a series of multimodal and unfolding story worlds that combine unique paper mechanisms with embodied, visual, and auditory modes. The interplay between these modes and the dual role of the interactor-performer have different implications on the performative experience of the playable story itself due to its multimodal nature where the performer is both a spectator and protagonist. Based on this, the paper presents three different “trajectories” –as unique user journeys that retain the same overall coherence – a concept proposed by Benford et al. and discussed further in Benford & Giannachi, concerning the user’s narrative experiences (712; 11; 38). These trajectories represent ephemeral relations between physical and digital elements. First, the “narrative trajectory” unfolds narrative events using spoken and printed words, environmental sound effects, posable avatars, and various paper mechanisms. Second, the “interface trajectory”, shows visual and cognitive cues through directional light paths, kinetic typography and graphics. And last, the “performative trajectory” made by the performer involves physical displacement through and between the story worlds, and object manipulation as the engine of episodic engagement. The discussion will pay particular attention to movement and motion as the bridge between the narrative and the intention of the interactor-performer. The findings will contribute to the characterization of the performative experience of interactive narratives with multimodal design. This paper concludes by using the case of “Letters to José” to illustrate designers and writers, in regards to the creation and design of playable stories and other narrative games.