Multimodality
Description
This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today's increasingly complex combinations of multimodal artefacts and performances of all kinds.
Reviews / Votes
"Bateman, Wildfeuer und Hiippala legen mit diesem Buch eine Einführung in die Multimodalitätsforschung vor, die Neulingen einen umfassenden, praxisbezogenen Einblick in das Feld ermöglicht und gleichzeitig vielfältige Anregungen und Anstösse für Forschung bereithält. Für Überlegungen zur Multimodalität im DaF/DaZ-Bereich steht hier somit eine hilfreiche Vergleichsfolie für weitere Diskussionen zur Verfügung [...] Besonders hervorzuheben sind auch die umfangreiche Bibliographie und das Stichwortregister, die den Zugang zum Feld der Multimodalität ungemein erleichtern und vermutlich so schon alleine für eine Vielzahl von Studierenden ein gewinnbringender Ausgangspunkt für eigene Arbeiten werden dürften."
Arne Krause in: Zielsprache Deutsch 45, 1 (2018); 73-75
"The breadth of topics and detailed writing are finely balanced with an accessible, engaging style that is annotated with color figures which model concepts, encapsulate frameworks and demonstrate analyses. The book's tripartite structure, from foundation to methods to case studies, offers students and practitioners alike firm ground and a clear path forward for conducting detailed multimodal analyses -a welcomed and needed foundational textbook."
Chris Featherman in: Discourse Studies 2018, Vol. 20(3) 443-457
"Les outils présentés au sein de cet ouvrage sont ainsi le point de départ fondamental pour explorer le champ de la multimodalité dans son ensemble et, plus largement, pour s'ouvrir à de nouvelles perspectives d'analyse."
Alida M. Silletti in: publifarum,
http://farum.it/lectures/ezine_articles.php?art_id=540
"[...] [the title] can make a worthwhile addition to the field and may encourage LL scholars to engage more explicitly with multimodality research."
Jeffrey L. Kallen in: Linguistic Landscape 4, (2018), 96-99
More details
Other editions
Additional editions
Persons
Content
- Intro
- Contents
- How to use this book
- Part I: Working your way into 'multimodality'
- 1 Introduction: the challenge of multimodality
- 1.1 First steps ... a multimodal turn?
- 1.2 The journey ahead
- 1.3 What this chapter was about: the 'take-home message'
- 2 Recognising multimodality: origins and inspirations
- 2.1 The 'problem space' of multimodality as such
- 2.2 Materiality and the senses: sound
- 2.3 Materiality and the senses: vision and visuality
- 2.4 Language
- 2.5 Systems that signify: semiotics
- 2.6 Society, culture and media
- 2.7 What this chapter was about: the 'take-home message'
- 3 Where is multimodality? Communicative situations and their media
- 3.1 Stepping beyond the terminological jungle
- 3.2 Communicative situations
- 3.3 Medium (and media)
- 3.4 What this chapter was about: the 'take-home message'
- 4 What is multimodality? Semiotic modes and a new 'textuality'
- 4.1 Semiotic mode
- 4.2 Modes and media
- 4.3 Genre, text, discourse and multimodality
- 4.4 What this chapter was about: the 'take-home message'
- Part II: Methods and analysis
- 5 The scope and diversity of empirical research methods for multimodality
- 5.1 What are methods? What methods are there?
- 5.2 Getting data?
- 5.3 Corpus-based methods to multimodality
- 5.4 Eye-tracking methods for multimodality
- 5.5 Computational methods in multimodality research
- 5.6 Summary and conclusions: selecting tools for the job
- 5.7 A word on good scientific practice: quoting multimodal artefacts
- 6 Are your results saying anything? Some basics
- 6.1 Why statistics?-and how does it work?
- 6.2 What is normal?
- 6.3 Assessing differences
- 6.4 Assessing similarities
- 6.5 How much is enough?
- 6.6 Refinements and restrictions
- 6.7 Inter-coder consistency and reliability
- 6.8 What affects what? Looking for dependencies
- 6.9 Summary: many types of tests and possibilities
- 7 Multimodal navigator: how to plan your multimodal research
- 7.1 Starting analysis
- 7.2 Undertaking multimodal investigations of phenomena
- 7.3 The basic steps in multimodal analysis reviewed
- 7.4 Conclusions and lessons for effective multimodal research
- Part III: Use cases
- Use case area 1: temporal, unscripted
- 8 Gesture and face-to-face interaction
- 8.1 Previous studies
- 8.2 Describing gesture and its functions
- 8.3 Conclusions
- Use case area 2: temporal, scripted
- 9 Performances and the performing arts
- 9.1 Performance and scripted behaviour
- 9.2 Previous studies
- 9.3 Example analysis: theatre and its canvases
- 9.4 Example analysis: Berlin Philharmonic concerts 'live'
- 9.5 Conclusions
- Use case area 3: spatial, static
- 10 Layout space
- 10.1 Perspectives from graphic design
- 10.2 Example analysis: school textbooks
- 10.3 Example analysis: posters
- 10.4 Summary
- 11 Diagrams and infographics
- 11.1 Aspects of the diagrammatic mode
- 11.2 Example analysis: assembly instructions
- 11.3 Example analysis: information graphics
- 11.4 Summary
- 12 Comics and graphic novels
- 12.1 Comics: basic ingredients
- 12.2 An aside on the notion of 'narrative'
- 12.3 Beyond narrative: comics as non-fiction and metacomics
- 12.4 Issues of literacy
- 12.5 Moving onwards: empirical multimodal comics research
- 12.6 Summary
- Use case area 4: spatial, dynamic
- 13 Film and the moving (audio-)visual image
- 13.1 The technical details of the virtual canvas of film
- 13.2 Multimodal film analysis: an example
- 13.3 Films and comics: adaptation and convergence
- 13.4 Summary
- 14 Audiovisual presentations
- 14.1 Characterising the medium
- 14.2 Exploring the canvases
- 14.3 Summary
- Use case area 5: spatiotemporal, interactive
- 15 Webpages and dynamic visualisations
- 15.1 Challenges and difficulties: determining objects of analysis
- 15.2 Example analysis: dynamic data visualisations
- 15.3 Summary
- 16 Social media
- 16.1 Previous studies
- 16.2 Communicative situations in social media
- 16.3 Social media analyses and possible methods: Instagram
- 16.4 Summary
- 17 Computer and video games
- 17.1 Example analysis: turn-based strategy games
- 17.2 Example analysis: first-person, real-time games
- 17.3 Summary
- 18 Final words
- 18.1 Lessons learned: the take-home messages
- 18.2 Our goals in the book
- 18.3 Be a multimodalist: have fun and explore!
- Bibliography
- Index
