Editors’ Introduction

Abstract

Over the last four decades, the pace of human life has surrendered to an ever-increasing haste, as if our planet had begun to spin at an accelerated rate. The rapid development of technology has made an everyday reality of what, until recently, was an unimaginable situation of connecting people and accessing information, new media with new possibilities of expression, simultaneously, within multiple semiotic systems. It also brings artificial intelligence and its integration into modern communication processes, which can generate feelings of discomfort and uncertainty to humanities scholars. From a technological point of view, communication has proliferated, revealing the lifestyle of modern society. In modern communication, mostly multimodal texts are created, built from different semiotic sources (Kress, van Leeuwen, [1996] 2004). These are not new in the history of mankind, since the text has always multimodal, e.g. spoken texts are composed of language and paralanguage, as Ngo et al. (2022) call the Voice Quality, Facial Affect and body language, which are used to realise meanings associated with language. Also in the written text e.g. both modern texts and ancient manuscripts use colour, size font and initials to create multimodality. In modern times, due to the already mentioned technological development with the possibilities of different printing, multimodal texts are becoming dominant in printed media, in which the verbal message is either replaced by a pictorial one, or the meanings of the pictorial and the verbal are complemented or upgraded. In order to express meaning, film, TV and digital media, (by their very technological design), employ the simultaneous use of several semiotic systems. In this communicative landscape, the individual spontaneously adopts ways of encoding and decoding meanings in complexmultimodal texts, just as he spontaneously adopts his complex mastery of language itself. From this parallelism, the conclusion emerges that, in addition to the development and enhancement mother-tongue language skills, pupils and students

in schools should learn and be aware of the characteristics of meaning in multimodal texts, critical evaluation of the selection of appropriate semiotic sources, reading and creative construction of such texts.

Text is a phenomenon that arises in the process of communication (Halliday [1977] 2002), multimodal texts offer a greater breadth of communication: more resources, with a greater choice of means of expression, supposedly faster and simpler, but in its own right more demanding and responsible precisely because of the greater openness of meaning. Social and cultural conditions change texts and semiotic systems and texts change social, cultural relations, as many text researchers have claimed (Halliday, ibid., Kress, van Leeuwen, 2004, van Leeuwen, 2005, Bateman, 2008, Martin, White, 2005, Ventola, M. Guijarro, 2009, etc.).

As noted by the members of the New London Group (1996) and Kress (2000), the modern dynamic society requires a different kind of education than is mostly practiced today, namely education for the future, open, engaged, taking into account different cultural and social dimensions, collaborative and creative. The participants of the education should acquire multiliteracy (this certainly also includes multimodal textual literacy), so that they can creatively integrate themselves into the society of the future, which is supposed to be even less stable than the present one.

The current thematic issue of the Journal of Elementary Education fits into this perspective by discussing multimodal texts in modern communication with the aim of showing how we perceive and deal with multimodal texts in our (Slovene, Italian, Spanish, Finnish, South African) cultural environment and how and how much we include them in the school system. At the same time, we want to stimulate a critical discussion within the aforementioned discourse. Thematically, the articles are linked to the theoretical research of multimodal texts and the connection with school learning, as well as case studies of analyses of the use of semiotic means in picture books and textbook texts to express a message (socially sensitive topics - gender stereotypes - and humour). Discussions are based on the Systemic Functional and visual grammar and other theoretical aspects.

Specifically Jesús Moya-Guijarro and Eija Ventola focus on transitivity strategies in six picture books that challenge gender stereotypes. The findings show that the meaning load carried by embedded images, together with verbal and mental processes of perception, provides essential cues for fostering progressive gender discourses. Their study also looks at how metonymies are essentially used to highlight important aspects of the plot that challenge gender stereotypes.

Luna Bergh and Tanya Beelders examine eye gaze in relation to multimodal texts. They investigate the effect of the Stroop test on eye gaze. Results indicate images are used when unfamiliar objects are referred to and faces and characters attract attention. The use of incongruent colours definitely causes cognitive dissonance and negatively affect reading.

Dragica Haramija and Janja Batič present a sample of original Slovene rebus stories, which are widely read in the pre-school period, but there has been no in-depth analysis so far. In the research, they focus on the literary genres and themes that appear in rebus stories, the position of illustrations in them and their characteristics. They establish the dominant didactic function of the rebus stories in the process of the child's literacy, the function of illustration in its various substitutions with words, and through the analysis they also highlight the hybrid form of the rebus stories, in which the elements of the picture book and the rebus stories interlock.

Ana E. Kerman De Luisa and Andreja Žele discuss Slovene sign language and the basic word formations and sign-forming elements as resources for the formation of words and gestures. They represent the system of gesture language in the process of converting a vocal sign into gesture, facial expressions, gaze and body position. They also present an example of the use of gesture language in school and the role of an interpreter in lessons.

Davide Taibi identifies looks at characteristics of the OpenMWS platform within video corpus construction and analysis. These relate to student tasks such as the creation of video corpora ex novo when completing dissertations, traineeships, group project work and with remodelling existing video corpora to meet the needs of new audiences such as primary and secondary schoolchildren. It also considers how analytics records student interactions with the platform, an approach that invites students to reflect on their own learning trajectories.

Mariavita Cambria also uses the OpenMWS platform with students. She reports on the progress made by second-year language degree students in English linguistics regarding their use of online corpus construction, annotation and search tools when exploring video genres.  The Online Video Project proved beneficial for the students both in terms of acquiring textual competences and as regards creating new interactive communities.

In the discussion, Sonja Starc focuses on learning visual grammar in a university program for primary teachers. It points out the theoretical premises of SFJ and visual

grammar as a basis for dealing with multimodal texts in schools, along with a case study of a student analysis of a multimodal text, it presents the more challenging aspects of student acquisition of theory and the use of metalanguage. It justifies the need and shows the possibilities of dealing with multicode texts in school education.

Nick Komninos presents tools to measure multimodal literacy and metasemiotic awareness. These are interesting for multiliteracy development both for the individual student over time and in comparison to other students, as well as when comparing data from larger cohorts to see patterns or strengths and weaknesses in multimodal literacy within a wider perspective.

Nuša Ščuka and Simona Kranjc discuss teaching material for Slovene from the perspective of critical discourse and visual grammar in the conception of gender and the relationships between them. The analysis focuses on the concept of gender and the relationship between them. They find that both visually and verbally they establish equalities and differences between the sexes, which often creates an unequal relationship in the area of stereotypical gender roles (e.g. in the family and between professions).

Martina Rodela deals with the concept of humor and its role in textbooks. She analyses how humor is created with verbal and pictorial signs in a sample of primary school textbooks for Slovene as a first language, which signs are used more often for this purpose, and how humor in these textbooks is perceived by 3rd, 6th and 9th grade students.

References

Halliday, M. ([1977] 2002). Text as Semantic Choice in Social Contexts. V Webster J. (ur.). M. A. K. Halliday: Linguistic Studies of Text and Discourse, London, New York: Continuum, 23–84.

Kress, G., van Leeuwen, T. ([1996] 2004). Reading Images. The Grammar of Visual Design. London, New York: Routledge.

Ngo,T., Hood, S., Martin, J.R., Painter, C., Smith, B.A., Zappavigna, M. (2022). Modelling Paralanguage Using Systemic Functional Semiotics. Theory and Application. London: Bloomsbury.

Van Leeuwen, T., 2005. Introducing Social Semiotics: An Introductory Textbook. Oxon: Routledge.

Bateman, J., 2008. Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. London: Palgrave Macmillan.

Ventola, M., Moya Guijarro, J. (2009). The World Told and the World Shown: Multisemiotic Issues. Hampshire: Palgrave Macmillan.

Kress, G. (2000). A Curriculum for the Future. Cambridge Journal of Eduaction. 30 (1), 133–145.

Martin, J., White, P. ([2005]2007). The Language of Evaluation. Appraisal in English. London: Palgrave Macmillan.

New London Group. (1996.) A Pedagogy of Multiliteracies. Harvard Educational Review. 60 (1), 66–92.

Lektorski pregled tematske številke/Proofreading of Special Issue:

Dr. Sonja Starc & Nikolas Komninos