Rasmus Juhlin
A narrative might be adapted to suit the presentation of any one medium. Be it movies, a painting, or a printed book, the narrative is presented through the modalities of the medium at hand. This study explored how a medium composite of text and sound was perceived and defined how one might regard the sound design in relation to the text and the narrative in such a medium. To contextualize, sound refers to sound effects, ambiences, and music and not a speaking voice akin to a narrator in audiobooks.
The presence of sound alongside the written indicated the sonic environment affected how the narrative was visualized by a reader of the medium, though not without inciting concern. Namely, participants expressed the sound’s intrusion and impact upon their imagination and visualization when reading as both immersion enhancing and as forcibly guiding their imagination.
Through the study, a notation system for coding sound events in relation to written text and the overall narrative structure was developed. This was done using several concepts from sound design and the theory of remediation as the overall framework. Alongside this, a total of eight sound design guidelines was defined when approaching this particular narrative content (fiction novels) of the medium.
In a society where multimodal presentations are available through phones, tablets, and other devices, exploration of potential mediums, their application, presentation, and reception is a promising subject for further studies.