Klaus Speidel

Visual Narrative: Cultural Diversity – Cognitive Unity?

New Tools and Perspectives for Narratology and Picture Science

Neil Cohn (Ed.): The Visual Narrative Reader. New York: Bloomsbury 2016. 349 pp. USD 39.95 (paperback). ISBN 978-1-4725-8559-2

Introduction

As a proponent of the idea that there is a (purely visual) language of comics, Neil Cohn has emerged as a new voice in the domain of visual narrative studies over the past ten years. In his Visual Narrative Reader, the linguistically-minded author of The Visual Language of Comics presents a choice of texts from different disciplines. Diversity is a major strength of a collection in which even readers who are well-versed in the theory or history of visual narrative, will discover unfamiliar methods and material. Including the introduction, the Reader contains twelve articles divided into three sections. Part I deals with “Theoretical Approaches to Sequential Images”. Part II is concerned with the “Psychology and Development of Visual Narrative”. Part III is entitled “Visual Narratives across Cultures”. The articles fall broadly into four categories: theory papers, overview articles on theories, descriptions of experimental research, and analyses of specific cultural practices. Six of the twelve contributions focus mainly on either comics or manga corpora.

I will first introduce each of the articles, some very briefly. I will then take up themes from several contributions in more detail to point out how they could influence theorizing in narratology and picture theory, and this review will end with a few more general remarks on the collection and the underlying convictions it is based on.

Overview of the Articles

Janina Wildfeuer and John Bateman sum up major publications in “Linguistically Oriented Comics Research in Germany” (Chapter 2) from the 1970s to the present, discussing sixteen publications in more detail, and explaining the major ideas and methodological foundations of the different authors. Pascal Lefèvre analyzes graphic style in comics (Chapter 3), arguing that on evolutionary biological grounds line drawing can be considered a particularly efficient way to communicate narratives quickly. Graphic style, in contrast, can be considered “a matter of cultural choices and personal idiosyncrasies” (p. 72). Based on both the literature and his own research, Lefèvre then offers a taxonomy differentiating between graphic styles. More specifically, he proposes a list of seven factors that determine graphic style: [1] “the amount of details versus the degree of simplification […], [2] the amount and degree of deformation versus respecting normal proportions”(p. 75), [3] the kind of lines, “rectangular or rather rounded lines, clear, crisp lines or rather vague, ‘hesitant’ lines […]”, [4] “the amount and type of traces and the way they are distributed” (ibid.), [5] if and how depth cues are provided, [6] how shades and light are used (ibid.), and, finally, [7] different ways of coloring (p. 76). While more differentiation within categories is certainly needed, Lefèvre’s proposal is a far cry from the piecemeal treatments of style as ‘expressive’ or ‘Baroque’, or intuitive assessments such as ‘energetic’, and his work can enrich the theory and analysis of pictures beyond the realm of narrative drawing. It would be interesting to test the taxonomy with other kinds of drawings, in order to evaluate the degree to which the categories are universal and which of them need to be modified.

In the next contribution, Charles Forceville discusses the applicability of conceptual metaphor theory to visual narrative (Chapter 4). This is also something Neil Cohn takes up in the final article of the collection. I will come back to their central ideas below. In Chapter 5, Mario Saraceni distinguishes different ways in which panels can relate to each other and to textual elements. In Chapter 6, Eric Stainbrook explores the similarities between the “connexity” (p. 131) of words and pictures, i.e. the ways in which sentences and panels that follow one another (can) connect with each other. Concluding that there is sufficient similarity between visual and verbal connections, he seeks to show that essential textual information in comics can be replaced by pictures without affecting comprehension. He concludes from his examples: “Not only are the picture and the dialogue coherent with each other, but the nature of their connection is just as plain as the connection would have been between two lines of written text” (p. 146f.). The intuition of the reader is his evidence. There is however a problem in the way he sets up his ‘experiment’: in each of the three examples, we first see the version with two panels that combine both text and pictures. We then see the same narrative with a picture-only version of the second panel. Hence we are already familiar with the story when we look at the second version. It is hard to tell if we would understand this version so easily if it was not preceded by the verbally explicit first version. The article also lacks some basic reflections on how the reader (re)constructs the scenarios that are evoked. Intelligibility is presupposed rather than demonstrated and its mechanisms are not sufficiently explained.

Part II of the collection consists of two articles: an overview of psychological research on reading manga in Japan by Jun Nakazawa (Chapter 7), and a text by Brent Wilson which sums up some of the major results of his studies of narrative sequences elicited from children with different cultural and social backgrounds (Chapter 8). Wilson argues that children tend toward storytelling not only when they speak but also when they spontaneously draw. According to Wilson, the narrative strain in drawings by children was long obscured by educators who tended to “disdain” and “discourage” drawings that resembled comics. Such works “disrupted the modernist narrative of a cultural [sic] unmediated creative development” (p. 186). To analyze the drawing of such narratives, Wilson developed a simple experiment, which he maintained throughout the years: he gave children a paper with six empty panels and “asked them to draw stories, showing in six frames what happened and what happened next, and how things finally turned out” (p. 194). When he first solicited six-frame drawings in Japan in the early 1980s, he discovered that Japanese children often drew more complex visual narratives (with more changes of point of view etc.) than children of the same age in other parts of the world, a phenomenon which he was able to link to the exposure to manga. While the Japanese children’s drawings were more standardized, both in terms of style (big eyes, childlike features) and content (razor blade haircuts, recurrence of cyborg figures, etc.), the drawings were nonetheless bolder, more diverse and more accurate than drawings of children not exposed to this form of sequential art. Wilson’s article is very interesting on sociological grounds, but lacks precise distinctions of drawing styles, as well as close reading. A downside of the experimental design is that it imposes clear restrictions on the children (the six empty panels and the guiding question suggest sequential narration). If we want to find out how strongly children tend toward narrative when drawing spontaneously, and what kinds of narrative structures they choose, we need to give them more freedom. Wilson’s experimental design also makes historical comparisons difficult, because studies like those of Levinstein (1905), Luquet (1927), and many others did not impose such specific constraints. But this objection does not make his work less insightful on its own ground. I also believe that an open-access databank of his collection of thousands of drawings would help other researchers to follow up on his work and hope that it will eventually exist.

Part III begins with an article by Nancy Munn that introduces some of the major conventions in narrative sand drawings of the Walbiri people in Australia. In Chapter 10, David P. Wilkins furthers our understanding of Aboriginal sand drawing, focusing on the Arrernte, an indigenous group neighboring the Walbiri, and isolating different drawing conventions. I will come back to Munn and Wilkins below.

In Chapter 11, Soren Wichmann and Jesper Nielsen introduce the readers to ancient Maya narrative vase paintings (Chapter 11). They point out similarities in the ways the Maya and modern-day comic book artists represent movement and create image sequences, suggesting that these visual forms might be determined by cross-culturally shared cognitive features. The last article, by Neil Cohn himself, discusses different theories about how language might influence conceptualization and perception (Chapter 12) and suggests ways in which this influence might reveal itself in drawings.

Zooming in on Three Issues: Gapping, Indexical Narrative, and Cross-cultural Tropes

The Reader has a lot to offer to narratological audiences and image theorists alike. I will illustrate this by presenting four essays that enriched my views on reading, on visual narrative, and on the existence of cross-cultural visual tropes respectively.

The first is based on an experiment related by Jun Nakazawa. While experiments are often conceived as ways to test theories, this is an interesting example of how ideas for theory can emerge on the basis of experiments. Nakazawa provides an overview of twenty-five years of experimental research on reading manga in Japan. One interesting study he presents used eye-tracking to compare two seventh-grade girls’ readings of the same Doraemon story. One girl read this specific kind of manga every day. Her friend only read it once a week. The results are quite stunning: not only was the frequent reader much faster, used more systematic scanning patterns, skipped more panels and fixated less on word-balloons, her fast reading also provided her with better recall and better understanding of the story. As Nakazawa suggests, the fast reader filled in the blanks left by her reading based on her knowledge of the typical script of Doraemon stories. While the low number of subjects makes the evidence too anecdotal for general conclusions, the study design shows the power of combining eye-tracking with questionnaires and can encourage future research in the same direction, using similar methodologies. It would be particularly interesting to test to what degree the reading abilities of frequent readers can be transferred from their preferred kind of texts to other corpora. If Nakazawa’s results could be confirmed with more subjects and other media, there would be theoretical implications. One way it might impact theory is to encourage a shift of focus from gaps in works – a crucial concept in the theories of Wolfgang Iser, Umberto Eco, Hans-Robert Jauss, Michel Charles, or David Herman, for instance – to gaps created in the reading process. ‘Undisciplined’ readings seem particularly interesting when they leave comprehension intact like in the case discussed by Nakazawa.

The first two essays in the final section, by Nancy Munn and David P. Wilkins respectively, also open surprising new perspectives for Western image science. As Wilkins explains, landmarks such as a waterhole, a tree, a hill etc. are represented in Aboriginal sand-drawing through an “iconic outline of the object as it would appear ‘from above’” (Wilkins, p. 260), and the forms of smaller mobile objects are often derived from traces that objects and actors leave in the sand. Thus a point can represent a fighting stick that has been stuck into the ground, a ‘U’-shape can stand for a person sitting down, and a long straight line can, among other things, refer to a spear, a digging stick, a person, or an animal lying on the ground (Munn, p. 234). The meaning of a trace is fixed through the accompanying verbal narrative. What is interesting here is how strongly this practice intertwines iconicity (i.e. resemblance with an object) and indexicality (i.e. representation through traces).

One of the principles identified by Wilkins further exploits the indexical character of the depictions: “The longer a mobile entity is statically located, the deeper its ‘track’ […]. A seated person already drawn as a ‘U’ figure may later be retraced several times, creating an increasingly deeper impression, in order to emphasize that a significant amount of time has passed while the person has remained seated in that place” (p. 259). In other words, the Arrernte sometimes foreground and indeed use what I would like to call the pseudo-indexicality of their signifiers in order to make meaning.1 This is also apparent when movement is shown. To illustrate this point, Wilkins cites an account by the anthropologist T. G. H. Strehlow, who had observed how a storyteller walked her fingers across the sand, leaving traces behind (ibid.). What is interesting here is that the temporal dimension of the activity of drawing suddenly becomes referential. Iconicity is thus extended: here the salient resemblance is not only visual and spatial, but also temporal. Iconicity even extends to the body of the storyteller, as in sign language: her fingers move forward like the running boy whose story is being told. And like the boy in the story they also leave traces. When members of the tribe arrive after the telling has already begun, the marks in the sand may help them understand what happened before, at least in the same scene. Wilkins here draws a connection to a cultural practice essential for survival in the outback: tracking, aimed not only at understanding what animals are in the neighborhood and what they were doing before, but also as a way of figuring out what they might be doing at the moment when the traces are observed (p. 258). The importance of trace-reading in Aboriginal life seems to me to indicate a possible cultural function of the sand-drawings, which are mostly performed by women for children: by learning to connect pseudo-traces to fictional events in stories, children are taught to develop their own factual narratives based on real traces. One way to test this hypothesis would be to look for trace-based narrative practices in other communities in which the deciphering of traces is similarly important. On a conceptual level, the link between storytelling and reading traces also suggests an as yet little explored possibility for visual narrative: i.e. trace-based (or indexical) narrative.2 As this example shows, looking into narrative practices beyond the realm of Western cultures can engender more comprehensive theories.

An issue deemed important by several of the contributors to the Reader is how culture and language might influence the creation and perception of pictures, and whether or not there are cross-cultural tropes. Some of the authors look for principles that reappear in different media and cultures, while others are more concerned to point out cultural influences. Thus Wilkins, Wilson, and Nunn focus more on differences, while Cohn, Forceville, and Wichmann / Nielsen are more interested in continuity, comparing certain icons and symbols used in Maya vase painting with those of contemporary comics (pp. 299-302). An explanation for a certain cross-cultural recurrence of tropes of visual narrative and meaning-making is provided by conceptual metaphor theory (CMT). In the Reader, this link is first developed by Charles Forceville in Chapter 4 (“Conceptual Metaphor Theory, Blending Theory and Other Cognitivist Perspectives on Comics”). Like Cohn (Chapter 12), Forceville argues that metaphors in comics can to a certain degree be universal. CMT was made popular by George Lakoff and Mark Johnson in their seminal study Metaphors We Live By (1980). According to their argument, metaphors are not a purely literary phenomenon, but have deep roots in human cognition. According to CMT, a transfer like “anger is a hot fluid in a pressurized container” (Forceville, p. 103) could be based on quite physical sensations. If anger is like a container under pressure, this would, for example, explain why we speak of someone ‘exploding’. Hence CMT is also referred to as a ‘theory of embodied metaphor’. Drawing on this basic idea, Cohn and Forceville argue that embodied metaphors might have a visual equivalent, which could be recurrent across cultures. In the case of anger, images showing smoke coming out of an angry character’s ears or fire bursting out of her mouth might be seen as confirmation of the theory that metaphors occur across media. Forceville looked at 103 angry characters in the 398 panels of Asterix and the Roman Agent and observed that many of them were “at least commensurate with the angry container metaphor” (p. 104, emphasis in original).

As Cohn points out, “this metaphor – and many others – seems to be transcending cultural boundaries” (p. 325): its visual equivalents appear in Japanese manga as well as European comics (ibid.). Wichmann and Nielsen reproduce an image of a Maya vase painting, on which “two jaguar-like felines seem to be roaring or exclaiming fierce words, since speech-balloons with a flame-like outline emanate from their mouths” (p. 299) ‒ an example they explicitly connect with Forceville’s research. In his introduction to the reader, Cohn himself goes so far as to propose a “Principle of Equivalence’” according to which “we should expect that the mind / brain treats all expressive capacities in similar ways, given modality-specific constraints. In other words, the brain uses common cognitive resources across all of these expressive modalities” (p. 4, emphasis in original). Although it cannot entirely explain the phenomenon, such an equivalence might at least be part of an explanation for the recurrence of the same metaphors across media and cultures. However, while I find the approach interesting, what makes me skeptical is the regular repetition of the same examples across publications. Shouldn’t it be quite easy to discover new examples if we really lived (and drew) by metaphors?

Conclusion

The Reader rewards an open-minded narratological audience with a truly inspirational wealth of methods and examples from a wide array of cultural and historical contexts. I felt positively challenged by the presentations of cultural practices and experiments which have not, to my knowledge, been discussed in narratology and picture theory before, but perhaps should be in future. The combination of speech, gesture and ephemeral sand-drawings in Aboriginal storytelling is a particularly rich example of multimodality. The visual conventions alone are so different from most Asian, Middle-Eastern, and European traditions that they could act as test-cases for theories of visual narrative. Moreover, the ways in which indexicality and iconicity interact in this practice call for further research.

One somewhat ambivalent aspect of the Reader is that the authors of many of the more experimental, ethnological, or historical articles (in Parts II and III, Chapters 7-11) tend to confine themselves to presenting and roughly categorizing their material. Apart from Wichmann / Nielsen, the authors concerned with culture-specific corpora do not suggest how their results might influence theory beyond the scope of the cultures or cases they are discussing. There is also a certain disequilibrium between the theory-laden articles in the first part and the often low level of theorizing when specific corpora and practices are discussed in the second and third parts.

A more general problem is that definitions of narrative or narrativity are nowhere to be found, and that the authors of the reader largely presuppose both the narrativity and the narrative reception of the artifacts in their corpora. Visual narrative is only understood in terms of sequences of static images, but this choice remains largely implicit. More minimal examples of visual narrative ‒ such as those in single still images ‒ are never discussed. This gap could be linked to Cohn’s own theory of narrative, according to which narrative is constituted by an initial-peak-release sequence (Cohn 2013, 421), which leads him to doubt the potential of single pictures to tell full-blown stories (see e.g. Steiner 1988, Wolf 2003, Speidel 2013 and 2017 for a different position).

While the lack of more general theorizing in the second and third parts of the publication may be seen as a weakness, it is also what leaves the door open for complementary perspectives and future collaborations. It may, then, be hoped that the collection will contribute to the “full research program” which Cohn invokes in the introduction (p. 11), where theory, experimentation and corpus analysis (to which I would add ‘close reading’) are performed collectively and across disciplinary boundaries.

Bibliography

Cohn, Neil (2013): “Visual Narrative Structure”. In: Cognitive Science. A Multidisciplinary Journal 37 (No. 3), pp. 413-452. URL: http://onlinelibrary.wiley.com/doi/10.1111/cogs.12016/full (29.09.2016).

Ginzburg, Carlo (1983): “Morelli, Freud, and Sherlock Holmes: Clues and the Scientific Method”. In: Umberto Eco / T. A. Sebeok (Eds.), The Sign of Three: Dupin, Holmes, Peirce Advances in Semiotics. Indiana, pp. 81-118.

Luquet, Georges-Henri (1927): Le Dessin Enfantin. Paris.

Levinstein, Siegfried (1905): Das Kind als Künstler. Kinderzeichnungen bis zum 14. Lebensjahr. Leipzig.

Speidel, Klaus (2013): “Can a Single Still Picture Tell a Story? Definitions of Narrative and the Alleged Problem of Time with Single Still Pictures”. In: DIEGESIS 2 (No. 1). URN: urn:nbn:de:hbz:468-20130604-083458-0 (29.09.2016).

Speidel, Klaus (2017): “Jak pojedyncze obrazy opowiadają historie. Krytyczne wprowadzenie do problematyki narracji ikonicznej w narratologii” [“How Single Pictures Tell Stories. A Critical Introduction to the Problem of Iconic Narrative in Narratology”]. In: Katarzyna Kaczmarczyk (Ed.), Narratologia transmedialna. Wyzwania, teorie, praktyki [Transmedial Narratology. Challenges, Theories, Practices]. Krakow [forthcoming].

Steiner, Wendy (1988): Pictures of Romance. Form against Context in Painting and Literature. Chicago.

Wolf, Werner (2003): “Narrative and Narrativity: A Narratological Reconceptualization and Its Applicability to the Visual Arts”. In: Word & Image 19 (No. 3), pp. 180-197.



Dr. Klaus Speidel
FWF Lise Meitner Postdoctoral Fellow
Lab for Cognitive Research in Art History at the Department of Art History and the Cognitive Science Research Platform
University of Vienna
Spitalgasse 2, Hof 9
1090 Wien / Austria
E-mail: klaus-peter.speidel@univie.ac.at
URL: https://univie.academia.edu/KlausSpeidel

Please do not cite the HTML version but only the PDF file / Bitte zitieren Sie nicht die HTML-Version, sondern ausschließlich die PDF-Datei:

URN: urn:nbn:de:hbz:468-20170606-144430-0

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

1 I speak of ‘pseudo-indexicality’ as one would speak of a ‘pseudo scorpion’, because the signs in the sand are traces, but not traces of the things they refer to in the narrative. However, their trace-character still plays an important role for their meaning.

2 See, however, a brief remark in a collection of articles on abduction where the Italian historian Carlo Ginzburg (1983, 89) suggests that narrative could have originated in hunters’ tracking practices.