SED-GPT: A Non-Invasive Method for Long-Sequence Fine-Grained Semantics and Emotions Decoding

1. Introduction

Human emotional experience is complex and involves inner speech activities and self-dialogue during semantic perception [1]. These inner speech processes encode subjective emotional states and elicit neural activation in regions such as the anterior cingulate cortex, insula, and ventromedial prefrontal cortex. These patterns partially overlap with those observed during emotional expression [2,3,4]. Therefore, emotion constitutes not merely a physiological or behavioral response but also involves cognitive-semantic processing. Emotional states activate their associated semantic networks, while semantic representations contribute to the refined construction of emotional experience [7,8,9]. Long-sequence semantic decoding can effectively extract context-dependent information during emotional construction, thereby providing the foundation for fine-grained emotional representation [10,11].

Common neuroimaging modalities for emotion decoding primarily include electroencephalography (EEG), magnetoencephalography (MEG), functional near-infrared spectroscopy (fNIRS), and functional magnetic resonance imaging (fMRI) [12]. EEG and MEG offer millisecond temporal resolution but poor spatial specificity due to volume conduction and field spread, making source localization inherently uncertain [13]. For fNIRS, although it can provide relatively high spatial resolution, its measurement depth is only 2-3 cm [14]. In contrast, fMRI offers high spatial resolution and whole-brain coverage. This technique can not only enable precise localization of emotion related brain regions but also supports simultaneous detection of co-activation patterns between language processing networks and emotion regulation systems via blood oxygenation level dependent (BOLD) signals [15]. These features provide a clear advantage for studying the neural mechanisms of emotion and semantic interaction in the Chinese context.

Traditional emotion classification methods typically rely on short-sequence emotion-induction paradigms and machine learning (ML) algorithms. Kassam et al. elicited nine discrete emotions using word-cued spontaneous emotion induction tasks, achieving a rank accuracy of 0.84 in fMRI-based emotion discrimination [16]. Saarimäki et al. employed multivariate pattern analysis (MVPA) to distinguish six basic emotions, demonstrating good cross-subject generalizability [17]. However, conventional algorithms show significantly reduced decoding accuracy when the number of target emotion categories increases, and these algorithms face constraints from short stimulus sequences, non-naturalistic experimental paradigms, and coarse-grained emotion classification [18].

In recent years, large language models (LLMs) exemplified by GPT have demonstrated outstanding performance across diverse domains. These models can effectively capture rich contextual semantic representations and exhibit deep-level semantic reasoning capabilities, offering a novel approach to long-sequence emotion decoding [19]. Tang et al. developed an fMRI based generative decoding framework that maps brain activity patterns into the latent semantic space of LLMs, achieving neural reconstruction of continuous English speech (BERTScore = 0.82) and revealing the distributed encoding characteristics of the brain’s language network [20]. However, current research has two critical limitations. First, existing methods have not been validated on high-context language systems such as Chinese [21]. Second, prior studies have primarily focused on decoding basic semantic content, failing to further examine the subtle emotional components of language processing [22]. These limitations constrain our understanding of the neural mechanisms that underlie language and emotion interaction in real-world scenarios.

Accordingly, this study aims to develop and validate a fine-grained semantic and emotion decoding framework for extended Chinese narratives, bridging the current gap in understanding the neural mechanisms underlying language-emotion interactions. The innovations of this study include:

• Extended emotion decoding in high-context language: This study investigates the feasibility of long-sequence emotion decoding in Chinese, extending emotion decoding to high-context language systems.
• Fine-grained emotion decoding with SED-GPT: We propose a novel fine-grained decoding framework (SED-GPT) for Chinese narratives, which aligns brain activity with LLM-based semantic vector representations to reconstruct inner speech semantics.
• Dynamic neural interactions in emotion-semantic processing: By systematically examining the dynamic interplay between the language network and emotional systems during Chinese semantic processing, this work provides neural evidence for cognition-emotion coupling.

Overall, this approach is expected to promote the development of affective brain computer interfaces (BCI), demonstrating potential applications in depression treatment and cognitive behavioral therapy.

2. Materials and Methods

2.1. Dataset

This study employed the publicly available dataset SMN4Lang to evaluate our model, which comprises structural MRI and functional MRI data from 12 participants, with each participant contributing 6 hours of fMRI data [23].

At trial onset, a screen displayed the instruction "Waiting for scan", followed by an 8-second blank screen. The instruction then changed to "The audio will begin shortly. Please listen carefully" for 2.65 seconds before the auditory stimulus was presented.

The stimulus set consisted of 60 audio clips (4–7 minutes each) from People's Daily news stories, covering diverse topics including education and culture. All recordings were narrated by the same male speaker. Manual timestamp alignment was performed to ensure precise synchronization between the audio and corresponding textual transcripts.

Structural MRI and fMRI were acquired using a Siemens Prisma 3T scanner equipped with a 64-channel receive coil. T1-weighted images were obtained with a 3D MPRAGE sequence at an isotropic spatial resolution of 0.8 mm³ with the following parameters: TR = 2.4 s, TI = 1 s, TE = 2.22 ms, flip angle = 8°, FOV = 256 × 256 mm.

fMRI data were collected using a BOLD-sensitive T2*-weighted GE-EPI sequence with the following parameters: TR = 710 ms, TE = 30 ms, flip angle = 54°, in-plane resolution = 2 mm, FOV = 212 × 212 mm.

2.2. Data Preprocessing

In this research, fMRIPrep was used for batch preprocessing of the structural MRI and fMRI data in the Chinese semantic perception dataset. 4 participants were excluded after preprocessing due to extensive occipital lobe damage in their structural scans, leaving eight participants for analysis.

30 long-sequence task fMRI files were randomly sampled to identify brain regions showing significant activation during Chinese semantic perception and in response to twenty-one emotion word categories using second-level general linear model (GLM) [24,25]. Psychophysiological interaction analysis (PPI) then probed the neural pathways underlying semantic perception [26].

In comparing activation between task and resting states, at the individual analysis stage we used a first level GLM based on the hemodynamic response function and an autoregressive noise model. In the group analysis we constructed group level activation statistical maps using a second level GLM. After applying a voxel level correction (p < 0.001, Z > 3.09) and filtering by a minimum cluster size of 50 voxels, we identified regions showing significant activation [27].

In the study of neural pathways underlying Chinese semantic processing, BOLD signal time series were extracted from predefined brain seed regions. Physiological regressors, psychological regressors, and the original interaction regressor were constructed and then convolved with the hemodynamic response function. A first level GLM was fitted, followed by a second-level GLM group comparison to generate Z-statistic parametric maps. False Discovery Rate (FDR) correction (p < 0.001) was applied to identify brain regions exhibiting significant functional coupling with the semantic perception hubs [26].

In this study, we used the Affective Lexicon Ontology to mark timestamps for 21 emotions (joy, calm, respect, praise, trust, love, well wishing, anger, sadness, disappointment, guilt, longing, panic, fear, shame, frustration, disgust, blame, jealousy, doubt, surprise) and constructed an event matrix to compare BOLD signal differences evoked by emotion words and neutral words [28]. After multiple comparison correction and filtering by a minimum cluster size of 10 voxels, we identified the ROIs showing significant activation for each emotion category [27].

Based on the significant activated brain regions and the functionally coupled regions identified above, along with prior knowledge of brain areas involved in Chinese semantic perception [29,30], MNI coordinates were extracted to generate ROI masks. Using these ROI masks, the fMRI response time series of voxels within each mask were extracted. The 3D coordinates of the voxels were then re-encoded into 1D indices, forming a 2D neural response matrix.

2.3. SED-GPT

In this study, we propose a non-invasive method for long-sequence fine-grained semantics and emotions decoding, termed Semantic and Emotion Decoding Generative Pretrained Transformer (SED-GPT), as shown in Figure 1.

In the encoding stage, we employed a Semantic to Brain Response Conversion Module (SBRCM) to minimize the inter-modal distance between linguistic stimuli and brain activity patterns. This module established a mapping between stimulus features and corresponding neural responses, incorporating a word rate model while estimating noise covariance matrices to improve model generalizability.

In the decoding stage, new semantic sequences were generated using linguistic priors from the language model and candidates. These sequences were then projected into neural response space through the SBRCM. The predicted simulated responses were compared with empirically observed brain activity patterns to compute likelihood distributions. Through iterative Bayesian integration of prior probabilities and likelihood estimates, the optimal semantic sequence was reconstructed. This iterative process enabled the generation of extended semantic vectors, whose emotional valence was subsequently classified into multiple categories using the GoEmotions framework [31].

2.4. Fine-Tuning of LLMs

Due to the differences in writing systems and word boundary markers, English text contains clear separators (spaces) between words and exhibits rich morphological variation, making word-level tokenization effective for capturing semantic and syntactic information. In contrast, Chinese lacks explicit word boundaries, and each character carries its own meaning, so character-level tokenization allows flexible combination into complete lexical units [32]. Consequently, English LLMs are typically trained with words as tokens, whereas Chinese LLMs use characters as tokens.

Character-encoded Chinese LLMs successfully avoid the complexity of Chinese word segmentation and require less computation for training [32]. However, this design choice also means that high-performance, open-source Chinese GPT based on word-level tokens remain unavailable [33,34]. Character-level tokenization in Chinese GPT models does not align with word-based semantic encoding mechanisms: neuroimaging studies show that Mandarin users rely on explicit word boundaries, encoding meaning at the word rather than character level [34,35].

Unlike the character-level tokenization commonly used in Chinese LLMs, human language comprehension integrates lexical, syntactic, and contextual information via a distributed semantic network, activating and mapping the corresponding conceptual representations to enable word-level semantic processing and reasoning. In this process, newly encountered symbols are matched, retrieved and compared against existing network nodes, activating relevant concepts or semantic domains [36]. This semantic information processing mechanism parallels the token-based encoding used in LLMs, where each input token is projected into a high-dimensional semantic space to capture and combine conceptual features [37].

In this study, we indexed semantic feature vectors at the word level. We used GPT-4 to translate the time aligned Chinese transcripts into English in batch and used a GPT-2 model—fine-tuned on the DeepMind Q&A news corpus—to construct the stimulus matrix [38]. The prompt for LLM-based timestamp conversion was: Convert the Chinese text into the most appropriate English. You may split intervals and their corresponding times; ensure that each interval corresponds to exactly one English word and that the xmax value at the end of each sentence remains consistent.

For each word-time pair at every timestamp, the word sequence was fed into the GPT language model, and the semantic feature vector of the target word was extracted from the 9th layer of the model, where the semantic feature vector represents a 768-dimensional semantic embedding [39]. After obtaining the target words and their corresponding semantic feature vectors, these embedding vectors were temporally resampled to match the fMRI data acquisition time points using a three-lobe Lanczos filter [40].

2.5. Semantic to Brain Response Conversion Moduler

In the encoding stage, the SBRCM receives input from two modalities: 768-dimensional semantic embeddings extracted from GPT-2 and BOLD signals recorded by fMRI. To account for the temporal-scale difference between these two modalities, the semantic stimulus vectors from 5-10th TRs prior to neural response onset were concatenated to construct a joint feature space. A linear mapping from this feature space to neural signals was then established through L2-regularized regression, with a word rate model being derived by estimating semantic occurrence frequency within individual TRs. To enhance model generalizability, a bootstrapping approach was incorporated in the encoding phase [41].

In the decoding stage, candidate semantic sequences were initially generated based on GPT-2 prior probabilities. These candidate sequences were subsequently projected into neural response space through the SBRCM to produce predicted brain responses. Each candidate's likelihood was computed by evaluating the correspondence between its predicted response and the empirically observed neural signals. Candidate sequences exhibiting significant discrepancies with the actual brain response patterns were iteratively filtered out. The language model priors were then integrated with the neural likelihoods, and beam search was employed to select a new set of candidate tokens, with this process being repeated until multiple complete text passages were generated. Finally, the sequence exhibiting the highest likelihood was selected, and the GoEmotions model was applied to extract an emotion probability distribution, thereby enabling the decoding of emotional content throughout the semantic perception process [31].

2.6. Evaluation Metrics

2.6.1. Semantic Similarity

BERTScore, word error rate (WER) and Euclidean distance (ED) were employed to quantify how closely the decoded text matches the original stimulus.

BERTScore measures semantic overlap by aligning contextual embeddings of the candidate and reference texts to compute precision, recall and F1 score [42]. Precision reflects the matching quality between the two texts. Recall reflects the extent of matching coverage. The F1 score balances both quality and coverage. The metrics are defined as follows:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{1}{M}}\sum _{i=1}^M \underset{j\in \{1,\dots ,N\}}{max} \mathrm{c}\mathrm{o}\mathrm{s}(a_i,b_j),$$

(1)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{1}{N}}\sum _{j=1}^N \underset{i\in \{1,\dots ,M\}}{max} \mathrm{c}\mathrm{o}\mathrm{s}(b_j,a_i),$$

(2)

$$F1={\displaystyle \frac{2\cdot Precision\cdot Recall}{Precision+Recall}},$$

(3)

where $A=\{a_1,a_2,\dots ,a_M\}$ denotes the set of candidate text vectors, and $B=\{b_1,b_2,\dots ,b_M\}$ denotes the set of reference text vectors.

ED computes the Euclidean distance between corresponding semantic vectors, where smaller values indicate greater vector proximity and consequently lower semantic reconstruction error [43]. The metric is defined as follows:

$$\mathrm{E}\mathrm{D}=\sqrt{\sum _{i=1}^N{\left(a_i-b_i\right)}^2},$$

(4)

where $A=\{a_1,a_2,\dots ,a_M\}$ denotes the set of candidate text vectors, and $B=\{b_1,b_2,\dots ,b_M\}$ denotes the set of reference text vectors.

Word error rate (WER) measures, at the word level, the proportion of insertion, deletion, and substitution errors between the predicted text and the reference text relative to the total number of words in the reference [44]. The metric is defined as follows:

$$\mathrm{W}\mathrm{E}\mathrm{R}={\displaystyle \frac{S+D+I}{N}},$$

(5)

where $S$ is the number of substitutions in the candidate text, $D$ the number of deletions (words present in the reference but missing in the candidate), $I$ the number of insertions (extra words in the candidate), $N$ the total number of words in the reference text.

2.6.2. Emotional Similarity

For emotion decoding, the GoEmotions framework was applied to extract normalized emotion probability distributions from the decoded text [31].

Cosine Similarity (CS) and Jensen-Shannon (JSS) similarities were then computed between two sets of comparisons: (a) 30 randomly sampled decoded emotion distributions and the true distributions, and (b) 30 randomly generated emotion distributions and true distributions. The emotion-decoding performance of SED-GPT was quantified through direct comparison of these similarity scores.

CS is defined as the cosine of the angle between the predicted and true probability vectors, reflecting the overall alignment of the distributions [45]. To evaluate emotion similarity, we employed the following formula:

$$CS\left(P,Q\right)={\displaystyle \frac{P^{\mathrm{\top }}Q}{\parallel P\parallel \parallel Q\parallel }},$$

(6)

where the normalized emotion probability distribution of the decoded text is $P=\left(p_1,\dots ,p_n\right)$, and the normalized emotion probability distribution of the random text is $Q=\left(q_1,\dots ,q_n\right)$.

JSS is defined as one minus the Jensen–Shannon divergence between the two distributions. JSS measures the similarity between the predicted emotion distribution and the true distribution [46]. To evaluate semantic similarity, we employed the following formula:

$$JSS\left(P,Q\right)=1-\sqrt{{\displaystyle \frac{1}{2}}\sum _{i=1}^n \left(p_i{\log }_2{\displaystyle \frac{2p_i}{p_i+q_i}}+q_i{\log }_2{\displaystyle \frac{2q_i}{p_i+q_i}}\right)},$$

(7)

where the normalized emotion probability distribution of the decoded text is $P=\left(p_1,\dots ,p_n\right)$, and the normalized emotion probability distribution of the random text is $Q=\left(q_1,\dots ,q_n\right)$.

The similarity between the generated text and the reference text was quantified for each emotion category using GoEmotions with multi-emotion classifications. A similarity metric was defined through computation of the normalized ratio for each emotion category. The closer this ratio is to 1, the more consistent the predicted probabilities for that emotion between the two texts [47]. To evaluate semantic similarity, we employed the following formula:

$$\mathrm{S}\mathrm{i}\mathrm{m}\left(p_i,q_i\right)={\displaystyle \frac{\min \left(p_i,q_i\right)}{\max \left(p_i,q_i\right)}},$$

(8)

where the normalized emotion probability distribution of the decoded text is $P=\left(p_1,\dots ,p_n\right)$, and the normalized emotion probability distribution of the random text is $Q=\left(q_1,\dots ,q_n\right)$.

3. Results

3.1. Brain Activation and Functional Connectivity of Chinese Semantic Perception

3.1.1. Brain Activation of Chinese Semantic Perception

Comparative results between task-state and resting-state activation during the Chinese semantic perception task are shown in Figure 2 and Table 1.

During Chinese semantic processing task, significant activation was observed in the classical language network and semantic processing brain regions, accompanied by deactivation patterns in the default mode network (DMN) and primary sensorimotor cortices. Specifically, enhanced neural activity was identified in the bilateral frontal poles and posterior superior temporal gyri.

3.1.2. Functional Connectivity of Chinese Semantic Perception

Based on the above activation results, we selected bilateral frontal poles and posterior superior temporal gyri as seed ROIs and performed PPI analysis to investigate the functional connectivity mechanisms underlying semantic processing, as shown in Figure 3 and Table 2.

The PPI analysis revealed that core semantic processing regions exhibited significant task-state functional coupling with distributed brain areas. These semantic processing regions showed significant functional coupling with the 79 distinct connectivity clusters (FDR, p<0.001) including paracingulate gyrus, anterior cingulate cortex and insular cortex.

These connections support processes such as semantic retrieval, contextual and narrative maintenance, motor simulation, attentional control, visual imagery and spatial scene construction, and affective and self-referential processing [48,49,50,51,52,53,54].

3.2. Brain Regions Activated by Emotional Words

The comparative activation results between each category of emotion words and neutral words during Chinese semantic perception tasks are shown in Table 3.

Under conditions of continuous natural language stimulation, all emotion categories elicit widely distributed neural activation patterns. Specifically, the processing of emotional words engaged not only classic limbic regions associated with affective processing, but also significantly activated: primary visual cortex, sensorimotor cortices, facial expression-modulation regions and high-level cognitive cortices [55,56,57].

3.3. Semantic Decoding Performance

To assess semantic decoding performance, the text outputs generated by the decoder were quantitatively compared with both the original stimulus texts (ground truth) and randomly generated control texts, as shown in Table 4.

EXP denotes the experimental group. RM denotes the random group. BERT denotes BERTScore F1.

Across all three metrics, the differences between the experimental and random groups were highly significant (p < 0.001). This indicates that the text produced by our semantic decoder significantly outperformed the random baseline at capturing and reconstructing semantic information in long Chinese narratives.

3.4. Emotion Decoding Performance

The emotion recognition performance was quantitatively assessed by computing both CS and JSS between the decoded emotion distributions and the corresponding true distributions. These metric values were compared against a random baseline condition, as shown in Table 5.

EXP denotes the experimental group. RM denotes the random group.

For emotion recognition evaluation, we conducted multidimensional affective analysis of the decoded results and compared them with random baseline. Overall, the experimental group demonstrated significantly higher scores in both CS and JSS compared to the random group (p < 0.05).

The normalized ratios between the emotion distributions of decoded texts and the true emotion distributions were calculated. These ratios were then compared against random baseline distributions, as detailed in Table 6.

The decoding accuracy rates for anger, disgust, embarrassment, fear, grief, joy, nervousness, neutral, remorse, sadness, caring, confusion, desire and love in the experimental group was significantly above the random baseline (p < 0.05). These results underscore the decoder’s robust sensitivity to a wide spectrum of emotional states under naturalistic language conditions.

4. Discussion

Conventional emotion decoding methods are constrained by factors such as short-sequence stimuli, coarse-grained categories, and low-context language systems, making it difficult to capture the interaction mechanisms between language and emotion in real-world scenarios [16]. Our study demonstrates fine-grained emotion decoding in long-sequence narratives and provides a methodological basis for investigating the neural mechanisms underlying the interaction between emotion and semantic processing. It may address the bottleneck in dynamic emotion monitoring for depression treatment and cognitive behavioral therapy (CBT) [58].

In this research, we introduce SED-GPT, a fine-grained emotion decoding framework designed for long-sequence Chinese language processing, which establishes a neural alignment between brain activity and LLM-based semantic vector representations to enable inner speech semantic reconstruction. Moreover, our findings reveal the dynamic interplay between language-related cortical networks and affective neural systems during Chinese emotional semantic processing, providing novel neurocognitive evidence for thought-emotion integration mechanisms.

For semantic decoding, SED-GPT achieved a BERTScore F1 of 0.65, an ED of 12.432 and a WER of 0.924. For emotion decoding, it attained a CS of 0.504 and a JSS of 0.469. All decoding performances significantly surpass random baseline.

The GLM results from both task and resting states demonstrated that during Chinese semantic processing, the coordinated activation of the left frontopolar cortex and right medial frontopolar cortex likely involves top-down attentional modulation and cross-modal semantic integration [59]. Bilateral posterior superior temporal gyri were engaged in acoustic feature analysis and semantic primitive extraction, and bilateral temporal poles participated in abstract semantic representation and integration of social context [60,61]. Concurrently, the suppression of the DMN and other non-task regions (e.g., the default mode network) indicates directed allocation of cognitive resources to core language networks [62]. Activity along the left central sulcus during Chinese lexical ambiguity resolution suggests enhanced phonological working memory [63]. The co-activation of the working memory network (left superior frontal gyrus to right superior parietal lobule) with the attentional network (right precuneus) supports context integration and interference suppression [64]. Cross-modal activation of the left middle occipital gyrus implies that orthography–phonology associations may facilitate automatic mapping from speech to visual representations [65]. In higher level comprehension (e.g., discourse level processing), bilateral frontal poles and middle frontal gyri support context maintenance and are implicated in controlled semantic retrieval [66,67].

PPI analyses showed that the right-hemisphere target seed region exhibited significant functional coupling with the paracingulate gyrus, anterior cingulate cortex (ACC), and insular cortex. From a network perspective, this model can be reasonably explained theoretically. The paracingulate gyrus is primarily involved in self-monitoring and reality monitoring processes [68], the anterior cingulate integrates negative emotions, pain and cognitive control [69], and the insula enriches semantic understanding by integrating abstract semantic information with interoceptive bodily states and socio-emotional experience [70]. This task-dependent functional coupling mechanism may collectively support the multi-level synergistic integration of semantic content with contextual and emotional cues during narrative comprehension [71,72].

The GLM contrast between emotional and neutral words suggests:

• Network reorganization and resource redistribution. Emotional words were associated with widespread changes across cortical networks. High social value words (e.g., praise) activate an integrated network of empathic, motor simulation and evaluation, while activity in general executive control regions is significantly reduced [57].
• Embodied simulation mechanism. In the joy condition, significant activation was observed in the left and right precentral gyrus, which may involve the recruitment of oral and facial motor representations. In the praise condition, notable activation was detected in the postcentral gyrus and the anterior supramarginal gyrus, reflecting the engagement of somatosensory and speech-related pathways. These findings support the notion that abstract emotions participate in sensory-motor representation mapping [56].
• The emotion-visual imagery coupling mechanism. Positive emotions (e.g., joy, admiration) and negative emotions (e.g., fear) were associated with activation in the inferior occipital cortex, cuneus and right higher-order visual areas, revealing the multi-level integration of vivid mental imagery with emotional processing [73].
• The self/others reference and value assessment mechanism. Emotional words (e.g., admiration) activated the dorsomedial prefrontal cortex and superior frontal gyrus, reflecting metacognitive simulation of self and others in complex social emotions [74].
• Suppression patterns of specific emotions. Emotional words (e.g., criticism) elicited large-scale deactivation in visual-semantic, sensorimotor and executive/metacognitive networks (e.g., middle frontal gyrus, frontal pole). This pattern may reflect down-regulation of DMN processing to concentrate cognitive resources on affective synesthesia and social reasoning network [75].

Fine-grained emotion decoding results indicate that the proposed method can distinguish 14 emotional states from brain activity. Notably, negative emotions (anger, disgust, fear, grief, sadness) exhibit greater decoding accuracy, which may be attributed to humans’ preferential processing of negative stimuli. During disgust-related words processing, participants experienced embodied interoceptive imagery (e.g., nausea), activating the right somatosensory cortex (postcentral gyrus) [76]. Concurrently, stronger engagement of higher-order cognitive regions (e.g., right frontal pole) was required to evaluate and regulate this negative emotional response (e.g., suppressing the "gagging" impulse) [76]. Sadness-related words can cause significant suppression in right lateral occipital cortex, right precuneus and right frontal pole. It indicates that the subjects exhibited weakened imagery, spatial association, and metacognitive processing under sad emotions [77]. Humans allocate more attentional, perceptual, learning and memory resources to negative stimuli, resulting in stronger and more stable neural responses with a higher signal-to-noise ratio, which facilitates emotion decoding [78].

5. Conclusions

In this study, we achieved the textual reconstruction of semantic content by aligning fMRI signals induced by Chinese auditory narratives with semantic representations and further extracted multidimensional emotional features for classification. Our findings not only demonstrate the neural decoding of complex emotional components in the Chinese context but also highlight the immense potential of integrating large-scale language models with neuroimaging technology. Overall, this approach could advance the development of affective brain-computer interfaces (BCIs) and show promising applications in the treatment of depression and cognitive behavioral therapy.