This paper looks at a few experiments on comics-storytelling in digital comics. The paper starts with introducing aspects from media psychology and research on technical documentation to look into the narrative and graphic structure of comics and touches on the characteristics of digital media before focusing on specific examples in more detail.

It can be said that a lot of digital and analogue comics constantly experiment on formal and narrative options. This is most obvious where elements of other narrative media get included (see Dittmar 2012 for a more thorough discussion of digital comics). The growing spectrum of forms offers more and more areas to use comics for: not only fictional but also non-fictional issues are communicated increasingly often in comics. For instance, maintenance manuals and assembly instructions for all kinds of artefacts are provided in sequential images more and more (see Schwender 2007, also: Jüngst 2010) – they are much easier to read than descriptive texts, as no translation of text into visual information is done, but the artefact in question and its parts are depicted and can be recognised easily.

Digital Comics Experiments

Obviously, not only digital comics but also analogue forms constantly experiment on and expand formal and narrative options. But in digital comics elements of other narrative media can be included quite easily as long as they are available as digital information as well. Also, distinct presentation media can be employed, like smartphones with their specific screen formats and standard image resolutions.

Two screens taken from Tom Wallgren’s “Urgent Delivery”

Tom Wallgren wrote and produced “Urgent Delivery” during a comics-course at Malmö University specifically to be read on smart-phones. The format of the images is adjusted to make the most of these screens – and accordingly, their size limits options for juxtaposed pictorial sequence. The dramaturgical development is built around this peculiarity: Almost all images are placed one per screen and only a few double images are used in the story. These combine our experience of split-screen from film with the diagonal frames between images in mainstream action comics. The story makes the most of each individual stage – images use the options of light and colour-intensity that only a screen can guarantee. To achieve the same colour-qualities in print would be quite expensive (starting with the need for high quality glossy paper).

The example below by Oscar Lagerström Carlsson and Felix Strandberg, also from one of our comics-courses, allows the reader to choose the narrative perspective onto the story. In this example, it is not the visual plane that is changed but the textual content – the reader can select whether to read the story with accompanying narrator’s comments or with direct speech of the depicted figures included. A blend of the two is also possible, adding a more text-centred version to the two image-centred narrations. In the result, three different modes are offered: the first showing all dialogue and conversations of the figures with each other, the second being an internal monologue, in which the main figure’s perspective is used as the narrator’s voice for the story, and a third in which the previous two forms are added onto each other. Each results in quite a different experience of the narration, without using different images at all.

Oscar L. Carlsson & Felix Strandberg allow for different narrative settings for the same visual material – here: [selectable narrator:dialogue]

The second text-perspective offered by Oscar L. Carlsson & Felix Strandberg in their comic: [monologue: first-person narrator]

The combination of the two previous text-layers by Oscar L. Carlsson & Felix Strandberg: [first-person narrator & dialogue combined]

Beyond Comics Definitions

Experiments on narrative forms partly challenge and even change established definitions of what comics are and what might easiest be described as mixed media storytelling. It has to be asked whether the established definitions of comics are fitting for the various forms of digital and web-comics or whether we are witnessing the establishment of a new literary form, which is neither film nor comic nor audio storytelling. Computer-based forms of comics that allow for readers’ choices in the development of narration (i.e. interactivity) usually switch easily (and often) between push and pull aspects of the medium (readers have to choose actively to be able to read further).

If we describe comics in the abstract for a moment, we can describe images as a blend of para-social and socio-culturally coded information, while written texts consist only of socio-culturally coded visual information. Some signs are understood without having to learn their meaning; basic emotions (fear, happiness, anger and despair) for example are understood by all humans in all the world, while other codes and messages depend on cultural learning – languages and writing for example: The difference between pre-social and socio-cultural recognition (for details see e.g. Berghaus 1986 and Boehm 1994, esp. 325 ff.). The visual rhetoric of comics can use elements from all kinds of codes and other visual depictions, even empty spaces. For the reader these need to be de-codable, otherwise the story can get misunderstood – or is not understood at all.

The structural basis for comics-storytelling is rather simple, while the difficulties are given in the interweaving (“tressage”, Groensteen 1999). It turns out to be slightly more difficult to describe the way in which images relate to other images. The way relations are constructed between individual places in the narration (“la spatio-topie”) and between narrative elements and themes (“l’arthrologie”) (Groensteen 1999). As Helena Magnusson summarised: ‘The first is about spatial relations, the second about semantic relation’ (Magnusson 2005: 42) – within each specific comic. The linearity or non-linearity of relations between events is crucial: it gives structure to the story itself, this in turn causes decisions on how to interrelate scenes and figures’ appearances within the narration.

This might sound more complicated than it is, but keep in mind that every comic is constructed from elements that are placed on various structural layers that are overlaying each other. These are comparable to “cells” – transparent sheets – in analogue animation film, but do not separate into different stages within a movement (e.g., the different positions of the leg in movement). Instead, the different layers are separated according to their internal visibility, i.e. within the narration: the images themselves, images inserted into images, texts written into images (e.g. sound-words), texts in frames (comments by narrator) and as a different group which is visually related quite closely: texts in bubbles. These can be separated into representations of speech and into thoughts. And if we understand the elements of comics storytelling to be placed on separate layers, we can more easily understand the potential of each of these elements within the construction of a story.

The smallest unit in comics-storytelling is the individual image that is limited by its frame (see Dittmar 2011 for a more detailed discussion of units in comics storytelling). There always are frames, only some of them are not decorated – but each image stops somewhere in some specific manner or style. The narrative development – the dramaturgy – of each comic is built from sequencing frames: They allow for narrative punctuation of the story, for visualising rhythm and structure of events.

Accordingly, the composition of each image has to be analysed or planned. Also, the style of drawing (or other graphical production methods) is crucial for the narrative and the construction of atmosphere: The depicted lighting conditions, the colouring, the choice of tools and reproduction media. And of course the point of view (incl. tilted and other framings) chosen for each individual image and sequence. Each page is an image containing several images – all the different names for full pages relate to concepts of comics-structure: the page as “hyperframe” (or “hypercadre” in Peeters 2003), “meta panel” or “super panel” (Eisner 2004), etc.

The depiction of physical environment does of course offer all the pictorial information on the placement of figures in whatever surrounding, while sounds can become physical (figures are hurt by sound, sometimes), but usually are on a separate layer, close to the speech balloons that represent direct speech. Thoughts are separate from these again, readers understand them to be visualised for their benefit, they are usually not imagined to be readable by the other figures within the comic, just like narrator’s comments, which are rather textual information that is framing the image in question and relating it to other aspects of the narration (Dittmar 2011: 179-182). Each of these layers is representing specific aspects of the situation and all of them do have their specific properties – as can be seen in comics which leave out one of these layers to tell their story with a distinct reduction of information.

On a more media-specific level, it has become obvious that the boundaries between comics and multimedia or even games are blurring (see e.g. Goodbrey 2014 for a discussion of this development). Comics’ definitions so far (e.g. in Groensteen, Carrier, McCloud; see Dittmar 2011 for a detailed discussion of comics definitions) do allow for, but do not discuss the consequences of floating or flexible page layouts for the dramaturgy of stories. Digital comics can follow these conventions or break them by introducing a different pacing of story-arcs that would not fit on printed formats. Decisions about the number of images and their placing and style are crucial for the storytelling style of each comic as each new page works as a meta-panel (or meta-image) that consists of all its individual images and combination of their designs (“mise-en-page”). Digital comics can allow for floating images on the page and variability in image-sizes. As a result, content of a page no longer sets the narrative structure of comics for each reader, but each gets to see a different depiction of the page according to the individual settings of the browser. The discussion about the applicability of classes of page styles cannot be repeated here, but remember the basic forms of page styles as either regulated or constant, as decorative, as rhetoric, or as productive (cf. Peeters 2003). Defining styles helps to discuss the frame-structure not only of each page, but also of the full comic (strip or album alike). These can be seen as a multiframe (“multicadre”, “multicadre feuilleté”: van Lier 1988) which is the comic‘s skeleton (Magnusson 2005).

It might be necessary to point out that printed and similar comics do cause different comics definitions than digital comics do. Definitions for digital comics are deducted from and related to the “classic” definitions, of course, but as medial demands and options are distinctly different with digital media, definitions have to adjust. The ongoing debate on digital comics is doing just that: it is testing established theories (comics definitions amongst other issues) to adjust them to describe current practices in digital comics.

To include hidden text that is shown when the mouse-pointer is dragged over it (the “alt-function”) seems not to be a challenge to comics-definitions. But including sounds, spoken texts, and music as accoustic and not as visual information causes a problem, as non-visual information is added. Is a comic still a comic if it includes moving images or sound-bites? Can it be a film, if the reader determines her/his individual speed of reading – especially if the images are placed juxtaposed? Karl-Johan Thole suggested on the current course on “Digital Comics” at Malmö University the following condition to mark the boundary between digital comics (which possibly include animated images) and animation film: ‘The reader should be able to look at the picture at any time of the animation and take in the whole meaning of the picture. So that the reader can read the comic without having to stop to watch a longer animation play’.

This condition limits the extent to which animated content can get included into a comic without turning it into film. From looking at many of the comics that are published on the internet, it is obvious that animated sequences of up to a minute are quite popular with the makers of these stories. But does this turn them into comics with short film sequences – or rather into stories told in multimedia? From a film perspective, most of the information and action in these stories is given in the comics-sections, while the animated bits mostly add atmosphere to the story, but only limited story-development.

Advanced interactivity might turn digital comics into games

Not only digital but also analogue comic-formats constantly experiment on and expand formal and narrative options. New options arrive with our changing uses of all kinds of media. They all result in options to tell stories in new or different ways. Especially with digital comics, elements of other narrative media get included; moving images or sound effects are interlinked with the dramaturgical development of the story. Also, distinct presentation-media are employed, for example mobile phones that are used as computer-and-screen-units in connection with the internet (not as phones, obviously). Often, interactivity is a crucial quality, e.g. visual planes can be selectable, or perspectives onto the story and its development – this area is closely linked to developments of games and might be rather more than multi-medial storytelling or digital comics. With growing interactivity the intended narrative sequence and dramaturgy gets communicated less safely, as the reader turns into a user that decides on the sequence of events and even on what might happen and what not. And when that stage is reached, we no longer talk about literature, but about games.

Sources

Berghaus, Margot (1986): “Zur Theorie der Bildrezeption. Ein anthropologischer Erklärungsversuch für die Faszination des Fernsehens.” in: Publizistik Jg.31, Heft 3-4: 278-295.

Boehm, Gottfried (1994): Was ist ein Bild? München: Fink.

Carrier, David (2000): The Aesthetics of Comics. Pennsylvania: Pennsylvania State University Press.

Dittmar, Jakob F. (2011): Comic-Analyse. Konstanz: UVK.

Dittmar, Jakob F. (2012): “Digital Comics” in: Scandinavian Journal of Comic Art (SJoCA), Winter 2012; 82–91.

Eisner, Will (2004): Comics and Sequential Art. Tamarac, FL: Poorhouse Press.

Goodbrey, Daniel Merlin (2014): “Game Comics: An Analysis of an Emergent Hybrid Form” in: Journal of Graphic Novels and Comics, Vol 5, Issue 4.

Groensteen, Thierry (1999): Système de la Bande Dessinée. Paris: Presses Universitaires de France.

Jüngst, Heike Elisabeth (2010): Information Comics. Frankfurt et al.: Peter Lang.

Magnusson, Helena (2005): Berättande Bilder. Göteborg & Stockholm: Makadam.

McCloud, Scott (1993): Understanding Comics. New York: Harper Collins.

Peeters, Benoît (2003): Lire la Bande Dessinnée. Paris: Flammarion.

Schwender, Clemens; Ulrich Bühring (2007): Lust auf Lesen. Die lesemotivierende Gestaltung von Technischer Dokumentation. Lübeck: Schmidt-Römhild.

Van Lier, Henri (1988): “La Bande Dessinée, une Cosmogénie Dure” in: Bande Dessinée, Récits et Modernité. Colloque de Cerisy. URL (16.09.2014): http://www.anthropogenie.com/anthropogenie_locale/semiotique/bande_dessinee.pdf

Jakob F. Dittmar studied British Studies, Religion, et al. in Oldenburg and Exeter. PhD in science of arts in Essen. Venia legendi and facultas docendi in media science on comics-analysis and on en-passant-media at TU Berlin. Assoc. prof. and senior lecturer at the School of Arts and Communication at Malmö University.