The ‘‘Reading the Mind in Films’’ Task: A Pilot Study on Complex Emotions and Mental States Recognition for the Italian Adaptation in Adults with and Without Autism Spectrum Conditions

1. Introduction

To the best of our knowledge, Autism Spectrum Condition (ASC) is neurodevelopmental condition marked by challenges in communication and social interaction [1,2]. These challenges persist throughout life, and despite ongoing learning, difficulties in social and communication skills remain. This is evident even among individuals without cognitive impairment who were diagnosed with Asperger Syndrome (AS) or High Functioning Autism (HFA) [3,4,5,6]. The conceptualization of autism as a spectrum has revolutionized our understanding of this complex neurodevelopmental condition. This spectrum perspective emerged in the mid-20th century as clinicians and researchers recognized the heterogeneous nature of autism. It has driven research into the genetics, neuroscience, and etiology of autism, aiming to provide more targeted therapies and effective support. A significant shift occurred with the DSM-V (2013), which replaced distinct subtypes with a unified diagnosis of Autism Spectrum Disorder (ASD). This change challenges the traditional view of autism as a singular entity, emphasizing personalized approaches to assessment and intervention, and reshaping societal attitudes toward autism. Autism encompasses individuals with a wide range of symptoms, abilities, and challenges under the umbrella term ASC. Despite this diversity, all individuals with ASC share cognitive and behavioral difficulties in communication and social interaction, often accompanied by restricted or repetitive behaviors and interests. People with ASC exhibit behaviors that differ from those of most others, showing limited interest in social stimuli and, depending on the severity of their condition, facing challenges in social situations due to reduced social skills (e.g., language development, gaze perception, emotion recognition). Moreover, social communication and interaction pose significant challenges, given their avoidance of eye contact, lack of response to name calling, limited facial expression recognition, and difficulty in matching facial expressions with the emotional content of spoken language.

Humans are inherently social creatures; from birth, we learn to perceive, reproduce, and use social signals to interact with others. Individuals with ASC, however, struggle with social interaction, reflecting a lack of use of social cues—such as others' gaze and emotional expressions—which are essential for navigating the social world. The conceptualization of social difficulties in ASC has been widely debated in recent decades. One prominent cognitive theory of autism suggests that these difficulties stem from a deficit in understanding others' minds, often referred to as 'Theory of Mind' (ToM) [7], 'Mind-reading' [8], or 'Mentalizing' [9]. This ability to "mentalize" fully emerges around age five, enabling children to understand others' beliefs and desires and to empathize with others [10]. Individuals with ASC, with limitations in this ability, may find themselves confused by others' behavior, struggling to grasp the intentions, emotions, and thoughts behind human actions. This may result in varying degrees of 'Mind-blindness' [11] or deficits in 'Empathizing' [12].

Neuroscientific studies have identified a network underlying these abilities, often referred to as "the social brain" [13]. This network involves the medial, inferior frontal, and superior temporal cortices, along with the amygdala, and processes social stimuli. Perceptual information, in part, arises from regions in the fusiform gyrus and the adjacent inferior occipital gyrus, which are activated in response to faces. Neuroimaging studies of ToM in ASC have shown reduced activation in these "social brain" areas compared to controls. For instance, an fMRI study revealed that individuals with ASC exhibit less extensive frontal activation and no activation of the amygdala when tasked with inferring others' mental states and emotions from pictures of eyes (e.g., the "Eyes Task") [14].

Another prominent cognitive theory of autism is the 'Weak Central Coherence' theory (WCC) [15]. WCC posits that individuals with ASC have an enhanced focus on details and difficulties integrating information into a coherent whole—they may "miss the forest for the trees" [9]. This theory suggests that individuals with autism struggle to derive meaning from the social world because they perceive it in a fragmented manner, making it difficult to integrate and contextualize details. Social functioning, which requires the rapid integration of context-dependent information, may be significantly impaired under such conditions. Brain function models in autism suggest that WCC reflects a cognitive outcome of altered connectivity between local, "low-level" neural networks, affecting the processing and integration of information. Research indicates that this disruption in low-level visual information processing, such as reduced connectivity between brain regions, may underlie the social challenges faced by individuals with ASC in using social cues to interpret socioemotional contexts [16,17,18]. Furthermore, a meta-analysis suggests that individuals with ASC show reduced attention to the social aspects of visual scenes compared to control groups [19].

Research suggests that effective emotion recognition relies on integrating multimodal information, such as visual and auditory cues. Both the Theory of Mind (ToM) and Weak Central Coherence (WCC) theories predict difficulties in this domain for individuals with ASC [20]. Studies investigating emotion recognition in computer models designed to mimic neurotypical perception patterns have further demonstrated that employing multimodal information—such as simultaneous presentation of facial expressions and acoustic cues—enhances the accuracy and robustness of emotion recognition [21]. This evidence underscores the importance of considering multimodal cues for a comprehensive understanding of emotion recognition, particularly for neurotypical individuals who benefit from the integration of multiple sensory inputs to accurately interpret emotional contexts.

However, many traditional assessment tools used to evaluate emotion recognition in individuals with ASC rely primarily on a single modality, often using static images or presenting stimuli through a single sensory channel. A notable example of this is the Reading the Mind in the Eyes Test (RMET), developed by Baron-Cohen in 1997 [22]. This task, designed to assess the ability to infer emotions and mental states through eye gaze, utilizes only static black-and-white photographs of the eye regions. While the RMET is a widely used tool to evaluate ToM and has been proven to be valuable in distinguishing individuals with ASC from controls, its limited multimodal presentation makes it less ecologically valid and less reflective of real-life social interactions.

To address this gap, researchers have developed more ecologically valid tools like the "Movie for the Assessment of Social Cognition" (MASC), which incorporates short film clips depicting realistic social interactions, requiring participants to infer the characters' mental states [23]. However, despite its strengths, the MASC has been criticized for its length and the potential difficulty in interpreting some of its items. Other attempts to move beyond traditional tests include the "Awkward Moments Test" (AMT), which uses short video clips taken from advertisements to depict scenarios that might cause embarrassment or social discomfort [24]. Participants are asked to interpret the social dynamics, emotional states, and intentions of the individuals depicted. However, the AMT has been criticized for its lack of ecological validity. The scripted nature of the videos often fails to capture the spontaneity and complexity of real-life social interactions. Thus, while the AMT represents an advancement in integrating social information through video, it does not fully replicate the nuances of everyday social situations. This limitation highlights the need for tools that offer a more realistic and dynamic assessment of emotion recognition, particularly in the context of autism.

Golan and his collaborators [20] developed the "Reading the Mind in Films" (RMF) task, which provided a more comprehensive assessment of emotion recognition compared to traditional methods. The task utilizes short film clips that present diverse social cues within rich contextual scenarios, offering a more realistic and ecologically valid assessment of emotional understanding in daily life. Participants view these clips and are asked to indicate the complex emotions and mental states of the characters depicted, considering not only their facial expressions and body language but also their vocal intonations and the overall context of the scene. This multimodal approach aligns more closely with how emotions are experienced and recognized in everyday interactions.

Despite the RMF's potential as a valuable tool for assessing complex emotion and mental state recognition, no Italian version is available, in particular there is no availability of video clips dubbed in Italian. This is a particularly important point since the Italian population is accustomed to enjoying film and video content in Italian since all the foreign films are translated and dubbed into Italian. The film clips used in the RMF task, however, are sourced from movies that do not have Italian dubbing, which presents a considerable challenge for using this task in Italy.

This pilot study aims to address the need for an Italian adaptation of the RMF to enhance the ecological validity of emotion and mental state recognition assessments for individuals with and without ASC in Italy. By translating the original scripts and dubbing the original film clips, we aimed at creating an assessment tool that reflects the complexities of social interactions and fills-in this critical gap.

2. Materials and Methods

2.1. Apparatus

After acquiring all necessary materials from the Autism Research Centre – University of Cambridge website (www.autismresearchcentre.com), the initial task was to inspect the 22 film clips used in the task. These clips are from English films released between the late 1990s and early 2000s, with only one film ("L’ospite d’inverno") also released in Italy. Due to the lack of Italian dubbing for most clips, with permission a script was created, capturing all actor dialogues, emotions, and contexts, followed by an accurate Italian translation, subsequently reviewed by two English native speakers fluent in Italian.

Professional voice actors and an audio technician were then engaged to assist with dubbing. An expert in audio and video editing was instrumental in recreating the original background sounds and synchronizing the new dubbing with the actors' lip movements and timing. Voice actors were sourced from the Fiverr platform (https://it.fiverr.com), where voice actor profiles were meticulously reviewed to find voices matching the original actors' characteristics and performance requirements. The selected voice actors were briefed and provided with scripts for their performances. The final dubbed audios were sent to the audio-video editing expert who, after extensive collaboration and effort, produced clips that maintained the original timing, emotion, and overall mood.

At the conclusion of the dubbing and editing process, we proceeded to create an Italian version of the video clips to be used to conduct the same experiment designed by Golan et al. [20], administered via the INQUISIT software [25]. The dubbed video clips were meticulously revised by the same English-Italian speakers to ensure semantic accuracy. Additionally, minor modifications were made to the original English script to incorporate changes necessitated by the translation. Utilizing the original experiment script, available for free on the Autism Research Centre website, we translated all the remaining sections and material of the experiment (i.e., instructions) into Italian. This Italian translated version of the experiment script was presented to a sample of 20 Italian participants to test its correctness.

2.2. Participants

Based on the sample analyzed by Golan and colleagues [20] and resulting from a power analysis conducted using G*power software [26], considering an alpha=.05; effect size= .5, r= .41, and power= .367, the required number of participants was determined to be 44 (20 males and 24 females), with an age range between 19 and 39 years (M= 23.4; SD= 3.77).

Participants were divided into two groups (22 participants each) and were asked to fill-in the Autism Spectrum Quotient (AQ) questionnaire (see Materials) [27]. The neurotypical sample consisted exclusively of students from Milano - Bicocca University, recruited through the university’s research platform (Sona System; https://www.sona-systems.com/) and had an AQ score lower than 24. The autistic sample consisted of individuals from Scuola Futuro Lavoro (SFL - a post-diploma vocational training school: https://scuolafuturolavoro.it/), all of whom had been diagnosed with high-functioning autism, with the exception of two participants. As a manipulation check, these individuals completed the AQ questionnaire as well, reporting significantly higher scores than the control group (see Table 2). The ASC group was then labelled as high autistic traits group. Diagnoses were provided by qualified clinical professionals. Only adults with typical cognitive functioning and without significant comorbidities that could affect task performance were included in the study.

2.3. Materials

In accordance with Golan et al. [20], the Autism Spectrum Quotient (AQ) [27] and the Reading the Mind in the Eyes Test (RMET) [28] were administered, along with the newly developed Italian version of the Reading the Mind in Films (RMF) task.

Autism Spectrum Quotient (AQ; Baron-Cohen et al., 2001b)

The AQ is a self-report questionnaire designed to assess the degree to which an adult from the general population with a normal IQ exhibits traits associated with the autism spectrum. Scores range from 0 to 50, with higher scores indicating a greater presence of autistic traits (≥ 24). The validated Italian version of the original AQ, developed by Simon Baron-Cohen and colleagues in 2001, was adapted by Liliana Ruta, a specialist in Child Neuropsychiatry at the University of Catania. Ruta's adaptation (2012) ensured that the questionnaire was both linguistically and culturally appropriate also for Italian speakers (2012; https://docs.autismresearchcentre.com/tests/ AQ_Adult_Italian.pdf).

Reading the Mind in The Eyes (RMET; Baron-Cohen et al., 1997)

The RMET is a widely recognized psychological assessment tool designed to evaluate the ability to infer emotional and mental states by examining photographs of individuals’ eye regions. The Italian version of "Reading the Mind in the Eyes Test" was developed by Marcello Vellante and colleagues in 2013.

Reading the Mind in Films (RMF; Golan et al., 2006) - Italian Version

Following Golan and colleagues [20], we presented our participants with 22 short film scenes (5-30 secs long, M=14.8, SD=9.2) characterized for their dramatic content and frequent emotional interactions. These scenes featured 14 characters displaying complex emotions and mental states, such as smugness, awkwardness, and concern. Each scene focused on a main character, and participants were required to label the emotional and mental state of the character at the end of each video clip, selecting from a list of four adjectives. Of the four options, only one was correct, while the other three were distractor "foils".

In Golan and colleagues’ study [20] the foils were chosen to be comparable to the correct answers in terms of verbal complexity. The authors employed an emotion taxonomy developed by Baron-Cohen and colleagues [29]. This taxonomy included 412 emotions and mental states categorized into six developmental levels. Foils were chosen to match or be simpler than the target word in terms of developmental level. While foils corresponded to certain aspects of the emotional content, such as language, they did not match others, such as intonation or context. As in the original study, our participants were provided with a handout containing definitions of all the target and foil words for reference both before and during the task, translated into Italian.

2.4. Procedure

The study was conducted over two experimental sessions (T1 and T2), with a minimum interval of one day between them. The study was conducted in accordance with the guidelines of the Declaration of Helsinki and its later amendments or comparable ethical standards. In T1, participants provided written informed consent before completing a computerized version of the Italian AQ and the RMET. In T2, participants performed the RMF task, followed by a debriefing session where they had the opportunity to ask questions about the tasks. To ensure optimal conditions, participants were seated 60 cm from a computer monitor and wore headphones to minimize external distractions, consistent with the methodology of Golan and colleagues [20]. The video clips were presented to 44 Italian adults (20 males, 24 females; mean age 24,4) on a computer using INQUISIT milliseconds experimental software [25]. Examples of the task and items are shown in Figure 1 and Figure 2, respectively.

Example (A): The clip portrays a young woman complimenting an older woman on the way she has raised the children. The older woman calmly thanks her several times, then runs towards her with tears in her eyes, saying, “Oh miss, I’m so glad you’re here”. Participants were asked: “How is the older woman feeling?” (Italian translation: Come si sente la donna più anziana?) and had to choose the right one among the four options available: Sociable (Socievole), Admiring (Ammirata), Overcome (Sopraffatta), and Liked (Apprezzata).

Example (B): The clip was labeled as "awkward," this scene depicts a man walking into a living room full of women engaged in conversation. He was searching for the land lady and as the woman, busy talking with the other ladies, notices him, he remarks: “I seem to have picked the wrong time.”

3. Results

Data Analysis

The statistical analyses were carried out using JAMOVI software [30]. Through a descriptive analysis, it was possible to analyze the participants’ responses to the RMF (Table 2) task and subsequently compare them with the results obtained by Golan and colleagues [20] (Table 2). Task scores were calculated by counting the number of correct answers for each participant. As a manipulation check, participants completed the AQ questionnaire, with the ASC group scoring significantly higher than the control group. Specifically, the ASC group had a mean AQ score of 29.2 (SD = 5.04), while the control group scored a mean of 13.4 (SD = 5.81). An independent sample t-test confirmed a statistically significant difference in AQ scores between the groups (t(42) = -9.67, p < .001), indicating that participants in the ASC group scored significantly higher on the AQ than those in the control group.

In Table 2 and Table 2, we visually compare the participants' percentages of accuracy for each RMF item with those found by Golan et al. [20]. Focusing on Table 2, which reports our results, we observe differences in the percentages of correct responses between participants with high and low levels of autistic traits for various emotions. These results indicate that participants in our groups differ, sometimes only slightly, in their ability to recognize and read specific emotions.

Comparing our results with those of Golan et al. [20], intriguing differences in accuracy percentages emerge. For example, in our study, participants in the ASC group show a slightly lower accuracy in recognizing "Imbarazzato" (Embarrassed) compared to the control group (36% versus 32%), similar to Golan et al.'s findings (64% versus 68%). Both studies also show that ASC individuals have similar difficulties recognizing certain emotions like "Amareggiato" (Bitter), with an accuracy of 18% in both groups across both studies. However, in our study the ASC group exhibited 100% accuracy in recognizing "Preoccupato"(Concerned), "Commosso" (Overcome), and "Rassegnato" (Resigned), unlike in Golan et al.'s study, where they performed worse than the control group for these emotions. For "Goffo" (Awkward), our accuracy percentages (18% versus 23%) contrast with Golan's findings (64% versus 86%), yet still indicate better performance by the control group. For "Preoccupato" (Concerned), the accuracy percentages were similar between our study (77% versus 73%) and Golan and colleagues (73% versus 73%). These similarities suggest a consistent pattern in the challenges faced by ASC individuals in recognizing specific emotions, as observed in both our research and Golan and colleagues' study. Additionally, the Italian version of the task used in our study showed generally a wide score range, aligning with the original study's methodology.

The total number of correct responses in the RMF was counted and depicted in the graph below (Figure 3). As evidenced by the graph, the trend of responses is similar across all participants. There is no observable difference between the ASC group (numbers 23 to 44 on the graph) and the control group (1-22) in performance.

Subsequently, to further analyze our results, we calculated the percentage of times participants in the control group and ASC groups chose the correct emotion word for each RMF item (see Figure 4). The performance of participants in both the control and ASC groups showed interesting patterns of results across the RMF items. In the initial item, no significant differences were observed between the two groups, with both predominantly selecting the emotion “Sorpreso” (Surprised) over the correct response “Imbarazzato” (Embarrassed). In the following items distinct differences in response accuracy and emotion selection became evident. For instance, in the second item, the control group had a higher selection rate for the correct emotion “Compiaciuto” (Smug) (44.4%) than the ASC group (31.8%), who more frequently chose “Subdolo” (Devious) (45.5%). For some items, both groups showed similar patterns in emotion selection, albeit with slight variations in accuracy. In the seventh item, the ASC group achieved a perfect accuracy rate (100%) in selecting the correct emotion “Preoccupato” (Concerned), compared to 90.9% in the control group. Conversely, items like the twelfth displayed similar performance between the two groups, with the correct response “Pungente” (Prickly) chosen by approximately half of the participants in both groups. However, certain items showed a higher accuracy rate in the ASC group, such as the ninth item, where the correct emotion “Commosso” (Overcome) was consistently chosen by the ASC group (100%) compared to the control group (81.8%). In summary, this variability underscores the nuanced differences in emotional and mental state recognition abilities between the two groups (see Figure 4). Graphs illustrating all the choices made by each group for every task item can be found in the appendix (See Appendix).

Moreover, a comparison of performance in the Reading the Mind Tasks between our two groups was conducted. Specifically, two independent sample t-tests were performed to compare the proportions of correct responses between ASC and control groups on the RMF task and the RMET. For the RMF task, the analysis revealed no statistically significant difference in performance between individuals ASC and control groups (t(42) = 0.0800, p = 0.937). Similarly, for the Italian version of the "Reading the Mind in the Eyes" task, no statistically significant difference in performance was found between the two groups (t(70) = 0.654, p = 0.515).

4. Discussion

In this pilot study, which aimed to translate and adapt the RMF task, we found minimal differences in response accuracy between our participants and those in the original study by Golan et al. [20], suggesting that our dubbing and translation efforts were both effective and successful. This alignment highlights the robustness of our careful translation process.

Our findings underscore specific difficulties in recognizing certain emotions among individuals with higher autistic traits, particularly with emotions like Goffo (Awkward), Lieto (Pleased), and Compiaciuto (Smug), consistent with observations by Golan et al. [20]. These difficulties may stem from the cinematic clips not fully conveying the intended emotions, leading participants to rely on alternative emotional cues. These cues could be influenced by the complex interplay of contextual, performative, verbal, and prosodic elements within the scenes.

Although some RMF task scenes may not fully reflect everyday life in Italy, particularly in 2023 (data collection), these cultural and temporal differences did not significantly affect the insights gained from the study. While the rapid pace and certain cultural nuances of the scenes may have added complexity, this did not hinder the test ability to assess emotion recognition in our sample. Nevertheless, to further enhance the ecological relevance of social cognitive tests, future research should consider adapting test materials to better align with contemporary sociocultural contexts. Incorporating scenes that resonate more closely with participants’ daily experiences could improve both the accuracy and applicability of these assessments.

Furthermore, our results indicate no statistically significant differences in performance on the RMF task and the RMET between ASC and the control groups. A possible explanation for this unexpected finding may relate to the unique characteristics of our ASC group, which comprised individuals with substantial exposure to filmmaking, video-making, and script-writing courses at Scuola Futuro Lavoro. This specialized training likely enhanced their familiarity with photographs, emotional expressions, and cinematic portrayals, thereby positively influencing their performance on both emotion recognition tasks. Indeed, the absence of significant differences in emotion recognition between the ASC and the control groups may reflect the benefits of consistent exposure to emotional stimuli and training. According to Galván and colleagues [31], the human brain exhibits notable structural and functional plasticity throughout life in response to learning and exposure. In the present study, individuals with ASC engaged in vocational courses that emphasized emotional understanding, likely facilitating improvements in their emotion and mental states recognition skills. Consequently, their increased experience in fields requiring nuanced interpretations of human emotions may have allowed them to perform comparably to the control group, which contrasts with the generally reported lower performance associated with ASC in the literature. This finding highlights the potential for targeted training and exposure to enhance emotion recognition abilities in individuals with ASC, suggesting a valuable avenue for future research and intervention strategies.

Overall patterns of response accuracy were similar between the two groups. While ASC group generally showed slightly lower accuracy in some emotions, their performance on certain items, such as for example recognizing "Preoccupato" (Concerned), "Commosso" (Overcome) and "Rassegnato" (Resigned), was notably high and even better than that surpassed that of the control group. The similarity in performance between the two groups suggests that the processes underlying emotion comprehension may operate comparably, regardless of differences in their levels of autistic traits. Additionally, the consistency of results from our control group with those of the original study reinforces the reliability of our findings and supports our translation and dubbing process of the RMF task.

While we acknowledge certain limitations in our methodology compared to Golan and colleagues [20], such as the lack of formal language comprehension testing, it is worth noting that all students at Scuola Futuro Lavoro successfully completed high school, came from Italian families, and were fluent in Italian. Furthermore, we did not pre-test whether the original selection procedure for the 22 clips used in the RMF task would also apply to our sample. Future research should consider these aspects to enhance the adaptation of the RMF task for use in the Italian context.

Nevertheless, our findings reveal an intriguing similarity in performance between both groups (ASC, Control) in the administered tests (RMET, RMF). This unexpected result suggests that differences in academic vs. non-academic education are unlikely to account for it. Notably, the non-university group, comprising individuals with ASC, actively participated in film-related vocational courses that involved filmmaking, scriptwriting, and film analysis. This consistent exposure to activities requiring engagement with expressive emotions, facial analysis, and storytelling likely contributed to their proficient performance in emotion and mental state recognition tests. Moreover, these findings imply that active involvement in filmmaking courses may enhance emotional and mental state recognition abilities, providing valuable insights for potential interventions for individuals on the autism spectrum.

A consideration for future research is the absence of a formal IQ measurement in our study. Unlike Golan et al. [20], who used the Wechsler Abbreviated Scale of Intelligence (WAIS, Wechsler, 1999) to assess IQ, we did not administer this task, resulting in a lack of objective IQ measures. Our assumption that participants fell within the normal IQ range was based on the premise that all individuals with autism spectrum conditions (ASC) in our sample were high-functioning and enrolled in a post-diploma school, indicating they likely possess typical cognitive abilities. The alignment of behavioral responses across groups alleviates concerns regarding potential cognitive disparities. Our decision to forgo additional testing was pragmatic, aiming to minimize participant burden. While acknowledging this limitation, we emphasize that the primary focus of the present pilot study was to test the Italian adaptation we made of the RMF task. Future research should systematically explore the role of IQ and the cognitive profiles of individuals with ASC in relation to performance in the RMF task, incorporating formal IQ measurements to provide a more nuanced understanding of cognitive factors influencing emotional and mental state recognition.

5. Conclusions

In conclusion, this pilot study effectively validated the reliability and accuracy of the dubbing process, which successfully mirrors the original clips while preserving their emotional content and intentionality. Findings suggest that participation in professional courses requiring extensive exposure to film and video—along with the emotional narratives they depict—may enhance performance in complex emotions and mental states recognition tasks. This potential benefit may stem from the immersive nature of these courses, which engage students within complex emotional scenarios and foster a nuanced understanding of emotional expression.

Future research should explore this hypothesis further, ideally through longitudinal studies that examine performance changes among individuals with autism spectrum conditions (ASC) engaged in similar training programs. Such studies could identify specific elements of film and video exposure that contribute to enhanced emotional recognition skills.

Moreover, expanding the participant pool to include neurotypical individuals and those with difficulties in understanding and expressing emotions (e.g., individuals with alexithymia) could provide valuable insights into the broader applicability of the RMF task. This comparative approach may reveal how different cognitive and emotional profiles affect performance on emotion recognition tasks, potentially informing more targeted interventions and educational strategies. By systematically investigating these directions, future research could make meaningful contributions to our understanding of emotional recognition, improving both diagnostic tools and support strategies for individuals with ASC and related conditions.