I’ll admit it: I’m a nerd. I love to think about semiotics, especially multimodality. Multimodality (also known as social semiotics), at its most basic level, is descriptive—it describes how people use language to create meanings and explores potential meanings that people could have created if they had used different meanings or if they had been in different social settings. In other words, multimodality researchers and educators are concerned both with what was taken up as possible and what could have been taken up as possible but wasn’t in any given social interaction. Take a look at this mural I saw this summer at the University of Antioch in Medellin, Colombia, for instance. Why depict their heads as TVs? Why are TV heads wearing suits? What are the other symbols representing? Why the other symbols placed above the heads? Why is the radio tower literally towering over the TV heads? Why is one of them reading a newspaper? Why are the TV heads looking directly at us, the viewers?
This type of analysis is incredibly useful in accomplishing the two main goals in my research. The first is to find a way to describe what is occurring with all the different modes (visual, oral, written) in youth produced media. There have been some attempts to describe these processes in youth video, many of them quite good (see Burn & Parker’s Analysing Texts (2003), for example), but we can go further in our understanding of how to describe just what is going on in youth videos and in the pedagogy that is used to teach it.
Answering these questions is just one part of what multimodal analysis would hope to accomplish. But, there are more questions: Why paint this at a university? How does it factor in that this university is known in Colombia for activism and that this is just one of many murals on their campus? Understanding how signs are put together is one thing. Understanding how people are using them to express thoughts, identities, and their places in the world, well, that’s another thing entirely. That’s what multimodality is to me.
Multimodality is a collection of theories that helps me to accomplish both goals. In multimodality, it is language itself that creates learning. With multimodality, though, the analysis goes beyond linguistic analysis: language use not only constructs learning, it also constructs how relationships are created and maintained. This is vital because often marginalization is classified under the general categories of race, class, gender, and disability, and learning is classified under those categories as well. This is understandably difficult to trace. However, with multimodality, I can understand how relationships are created and how learning is happening by tracing how language is used (in all its modes) by the adults and the children as the youth learn media literacy. Multimodality helps me to answer some of the difficult questions around marginalization and empowerment by seeing what voice is allowed and what voice is expressed.
All of these are things I think about all the time. So, in this blog, I will post some of my musings about multimodality and ethics. Some of it will relate to my work. Some of it really will be musings. I welcome conversations, so feel free to comment on anything written: to add, to wrestle with the ideas, to laugh. Think of this as another set of signs put out by someone trying to make some sense of the world and the people in it. Now, let’s get started….