4.1 Multimedia learning

We begin talking about Multimedia Learning by starting with a definition, the one developed by Mayer and Moreno (2003), two of the most relevant authors in the field, who define, then, multimedia learning as learning mediated by both words and images and involving multimedia instruction as a vehicle for delivering content. Specifically, words can be either written on a screen or conveyed by speech as a narrative. The image, too, can be either static or moving, and in this case it is possible to call it animation. Static images can be charts, graphs, diagrams, and illustrations. Animations can be both interactive animations and videos.

Into this perspective comes the theme of meaningful learning, which indicates, therefore, a deep understanding of the material. Learners take, therefore, part in this learning process when they are able to make connections between information that is processed in both the visual and verbal channels. In this sense, learners also turn out to be able to identify what are the key contents and mentally organize them by integrating them with prior knowledge.

The challenge from the designers’ standpoint – instructional multimedia designers – is to introduce new concepts that are engaging and at the same time do not create cognitive load. Finally, briefly, the phenomenon of cognitive load is caused when the content to be cognitively processed exceeds the quantity and effort demanded from the cognitive capacity available to the student. With this brief introduction we introduced some of the core topics of multimedia design:

• the multimedia materials,
• the dual channel assumption,
• cognitive load and
• meaningful learning.

Thus, to get specifically into the heart of the matter, we speak of multimedia learning in cases where information is presented not only through one channel, the verbal channel of words – the traditional one – but materials are built on both images and words (later, we will have the chance to discuss this aspect also.

Before going into the scope, however, it may be necessary to begin by asking why it may be better to present images along with words. Why prefer it to the verbal-only method of instruction?

On the one hand, on the positive side, the verbal mode of instruction has a deeply rooted past in the history of education, and words are clearly the most widely used vehicle within the training-school setting. On the other hand, what can be considered a negative aspect is that verbal modes of instruction are often based on an inadequate conception of how students learn – what we can call the information delivery view. According to this defined ”inadequate” view, learning is defined as the process of adding more and more information to memory, so that, again in this way, teaching involves sharing information to learners through words.

This kind of approach, in the face of the current state of the art of learning reflection appears flimsy. For example, exactly contrary to this theory is that of constructivism, which sees students as the protagonists of their own learning process. Not only that, there is also empirical evidence to support that the verbal mode alone does not always work so well. Some research such as (Mayer, 2003) shows that on average students who hear (or read) learning content that is only presented as words are not able to remember the central elements of what they have read and, moreover, find it difficult to use, to put properly into practice what had been presented to solve new problems.

Precisely given these critical issues, it is possible to begin by mentioning a number of situations that can represent a bit of what we mean by multimedia learning: thus, starting from the media: not only books and texts but learners can also learn from an online encyclopedia, within which it is possible to enjoy animations that can tell in a decidedly more interactive way some dense content of complexity. Or, not considering the online medium, it is also possible to refer to the use of explanatory images that can support the verbal element. Both of these two examples are, therefore, relatable to the realm of multimedia learning since they present instruction presented to learners in two formats: words (whether at the level of orality and/or writing) and images (online animations and/or illustrations). Of course, it is definitely not just the condition of adding one format to the other that guarantees the possibility of improving the learning process, making it more effective and efficient.

What is a multimedia message?

Multimedia messages can be defined according to several characteristics:

• according to the transmission device (for example a computer screen or speakers to amplify sound);
• the mode of presentation (for example pictures and words);
• and sensory modalities (for example, acoustic and visual).

The process of learning through multimedia can be viewed as information acquisition (where, precisely, multimedia messages are the vehicle for conveying information) or as knowledge construction (where, on the other hand, multimedia messages are supported for the process of sense-making.

Possible outcomes from a learning perspective can be:

• no learning (as indeed may occur by poor retention and poor transfer performance);
• rote learning (as indicated by good retention and poor retention performance),
• and meaningful learning (as indicated by good retention and transfer performance), where meaningful learning is the condition described at the outset.

Thus, meaningful learning processes depend on the learner’s cognitive activity during the learning process rather than on the learner’s own behaviors; precisely because of this, multimedia messages that are designed and imagined in light of the functioning of the human mind are more likely to have a meaningful effect than those that are not designed with the functioning of the human mind in mind. To get into the merits of this, it is possible to consider some elements related to human cognition.

Thus, again citing Meyer, the cognitive theory of multimedia learning – Cognitive Theory of Multimedia Learning – assumes that the processing system is organized into two channels: one that takes in visual/image information and one that takes in auditory/verbal information, and is derived from the research of Paivio (1986; Clark & Paivio, 1991) and Baddeley (1992, 1998).

Also according to the above theory–and we will now proceed to elaborate on the other two assumptions of CTML, limited capacity, and active processing – according to the first, each channel is not considered to have an unlimited processing pool but rather limited. According to the second, learning can be defined as active learning when learners are in a condition to coordinate a set of cognitive processes during learning, linking new content with prior learning.

Five steps within multimedia learning can be:

• select relevant words within the presented text or narrative;
• select relevant pictures from all the illustrations presented;
• organize the selected words within a coherent verbal representation;
• organize the selected pictures within a coherent visual representation;
• connect the visual and verbal elements with learners’ prior knowledge.

Clearly, the processing of images occurs primarily in the visual channel while that of (heard) words occurs in the auditory channel; at the same time, written words are conveyed initially in the visual channel while they are then processed in the auditory channel.

Still considering the work of Richard E. Mayer, Professor of Psychology at the University of California, it is possible to enucleate 12 principles of multimedia design:

• Multimedia principle,
• Spatial contiguity principle,
• Temporal contiguity principle,
• Coherence principle,
• Modality principle,
• Redundancy principle,
• Individual differences principle (also known as personalization principle),
• Signaling principle,
• Segmenting principle,
• Pre-training principle,
• Voice principle,
• Image principle,
• Redundancy,
• Pre-training,
• Signaling,
• Personalization.

Multimedia: Learners learn better from words and pictures than from words alone. When words and pictures are both presented, learners have the opportunity to construct verbal and figurative (illustrated) mental models and, more importantly, are able to make connections between the two. When words are presented alone, students only have the opportunity to build a verbal mental model but are less likely to build a figurative mental model and connect the two models. Therefore, instructional designers should make sure to include at least two modes in their content: video, graphics, animation, and narration.

Spatial Contiguity Principle: Students learn best when words and images connected to each other are placed close together (Spatial Contiguity). When items are spatially placed close together (on the page or screen), learners do not have to deploy cognitive resources to search for the specific page or part on the screen, and more importantly, they are more likely to retain the whole thing in their working memory at the same time. Thus, summarizing, multimedia content is much more effective and well designed when words and images are placed next to each other. (Sorden, 2012).

Temporal Contiguity Principle: Students learn better if words and pictures are presented at the same time (temporal contiguity) instead of at two separate times. How much the corresponding texts and illustrations are presented at the same time, learners are more likely to create mental representations of both at the same time, and more importantly, again, they are more likely to make mental connections between verbal and visual representations.

Coherence Principle: Students learn best when their attention is directed toward material that has internal coherence, and which, therefore, does not include content that is not related to the main object of investigation. 

In addition, the principle of coherence can be divided into three complementary versions:

• Student learning is impaired when interesting but irrelevant words and images are added to the multimedia presentation;
• Student learning is affected when sounds and music that are irrelevant to the main object are added to the multimedia presentation;
• Student learning is enhanced when unnecessary words are removed from a multimedia presentation.

Materials that are inconsistent with the main object of reflection can divert attention from the more relevant and coherent materials and can invalidate the process of organizing the material. Specifically, the multimedia product should not contain within the same slide/page content that is unrelated to each other (and, in this sense, it should also not include many different images in the same slide so that students may not be in the condition of cognitive overload).

Modality Principle: Students learn better from animation and narration than from animation and written text. When words and pictures are both presented visually (precisely, like animations and written text), the visual channel is in the overload condition. Therefore, graphics and narration are more effective in instructional multimedia pieces than text and graphics.

Redundancy Principle: Students learn better if they enjoy animation and narration instead of narration, animation and written text. When words and images are both presented visually, the visual channel is overloaded. Therefore, each designer should not include interactive animations if the video being used can distract learners’ attention.

Pre-training Principle: Students learn better when training on components precedes rather than follows a message. When learners receive pre-training they are able to build mental models of each part that makes up the whole, reducing the cognitive load that might arise when building a mental model of the whole system.

Signaling Principle: Students learn best when learning is clear and shared, when the importance of any content is indicated. In this way, students are better able to give attention to relevant materials, reducing cognitive load. Therefore, from a practical and design point of view, students learn best when they are able to recognize and learn information that is highlighted as important, so the use of arrows, underlining, highlighting for central aspects is preferred.

Personalization Principle (Individual differences principle): Students learn best when words are conveyed through language that is not overly formal.

Segmenting principle – Learners understand the instructional multimedia piece better when the lesson is broken into user-paced chunks rather than all in one multimedia piece (Sorden, 2012).

Voice principle – Learners are more engaged in the learning process when the voice in a multimedia presentation is human vs. a computer generated voice. 

Image principle – The instructor’s image on the screen does not generate more meaningful learning than if the image was not present.