Emotion
-
EEG-HealthLLM: Hierarchical Adaptation of Large Language Models for Unified Heterogeneous EEG Health Assessment
2026-11-01 07:01:30Ruidong Ma, Jisheng Dang, Ronghao Lin, Bimei Wang, Jingwen Zhao, Wenjun Ma, Wei-Shi Zheng, Meng Wang, Qi Tian, Tat-Seng ChuaAbstract
Electroencephalography (EEG) provides a non-invasive means for neurological and mental health assessment, yet existing EEG models are typically developed for individual tasks and are difficult to unify across heterogeneous signal configurations and assessment objectives. Large language models (LLMs) offer a common interface for multiple tasks, but adapting heterogeneous EEG signals to a shared LLM raises a key challenge: how to preserve task-specific signal characteristics while enabling effective language-side parameter sharing. To address this problem, we propose EEG-HealthLLM, a unified and parameter-efficient framework for heterogeneous EEG health assessment. We introduce an Asymmetric Hierarchical Adaptation Strategy: task-specific EEG adapters preserve heterogeneous spatio-temporal dynamics, while a unified shared LoRA is injected into the LLM backbone (e.g., Phi-3) to capture shared high-level assessment semantics across tasks. We further establish EEG-HealthBench by reorganizing five existing EEG resources into paradigms covering epileptic seizure detection, major depressive disorder assessment, anxiety risk assessment, depressive symptom risk assessment, and emotional state assessment under a unified subject-independent protocol. Extensive experiments show that EEG-HealthLLM achieves an average balanced accuracy (BAC) of 69.58%, ranking first among the evaluated methods in both average BAC and macro-F1. Compared with independent single-task LoRA adaptation, it improves average BAC from 67.70% to 69.58% while reducing the aggregate language-side adapter footprint by 80.0%. Further analyses reveal task-dependent cross-task interactions and trade-offs, supporting the proposed hierarchical adaptation strategy. The implementation is publicly available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2611.01652
PDF
-
BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining
2026-11-01 06:37:38Ayon Roy, Sadman Sharif, Syed Shams IslamAbstract
Depression affects millions of Bengali speakers across South Asia, yet automated detection systems have relied almost entirely on datasets translated from English. Such datasets cannot capture the emoticons, romanized Bengali words, and elongated spellings that characterize real expressions of distress. We introduce BanglishDepYT, a Bengali depression dataset collected natively from YouTube, comprising 191,000 unlabeled comments and 1,000 samples verified through dual bilingual annotation. We propose EPH (Emotion-Preserving Hybrid preprocessing), which converts emoticons to semantic tags, preserves Banglish tokens, protects negation words, and normalizes emotional elongation. A key finding is that preprocessing choice substantially affects performance on informal text: standard cleaning reduces the Banglish Survival Rate to zero (SVM F1: 0.7062), while EPH preserves signals (CESS: 0.9996 vs. 0.6694) and raises SVM F1 to 0.8570, an improvement of 0.1508. We further apply domain-adaptive pretraining (DAPT), continuing SahajBERT pretraining on unlabeled in-domain comments. DAPT-SahajBERT achieves the highest mean F1 (0.8161) and MCC (0.6272), significantly outperforming the baseline (F1: 0.7805, p = 0.0013, d = 1.450) and matching MuRIL (p = 0.589). These results suggest that targeted domain pretraining is a competitive alternative to large-scale multilingual pretraining. LLM-assisted annotation using Qwen2.5-7B achieves substantial human agreement (kappa = 0.611), enabling pseudo-labeling of 96,071 comments. Semi-supervised training improves AUC from 0.8499 to 0.8731 (p = 0.044), offering a scalable path to expanding annotation. All code, models, and datasets are publicly released to support reproducible research.
Abstract (translated)
URL
https://arxiv.org/abs/2611.01633
PDF
-
Can Large Language Models Extend Their Agentic Capabilities to Goal Pursuit in Open-Ended Multi-Turn Interaction?
2026-11-01 04:23:27Mingyu Chen, Jingkai Lin, Xiaofen Xing, Liankai Huang, Ruiqi Xiong, Xiangmin XuAbstract
Goal pursuit in open-ended dialogue requires agents to adapt their actions to interaction feedback across successive exchanges. We investigate whether agentic capabilities developed for multi-step tool use carry over to dialogue organized through the model's native tool interface, and how this changes interaction behavior. We define Tool-Mediated Interaction (TMI) as a training-free protocol that represents dialogue actions as tool calls and counterpart responses as tool results, maintaining a continuous reasoning-action-observation loop with retained reasoning and interaction history. We evaluate TMI across social interaction, emotional support, persuasion, and price negotiation using two models and 15,678 main-experiment episodes. Across both models, TMI improves social goal attainment, emotional-support preference, donation amounts, and buyer negotiation margins over conversational baselines with and without thinking. A Qwen ablation further shows that TMI outperforms ordinary conversation with retained reasoning on all four primary outcomes. Behavioral analyses associate the improvements with task-specific patterns in initial proposals and responses to interaction feedback. Alongside these gains, TMI reduces mean reasoning tokens per decision by 42.0-81.5% relative to the thinking baseline and lowers mean cross-turn phrase reuse across all eight model-environment pairs. These findings show that interaction organization shapes both goal attainment and reasoning across dialogue turns.
Abstract (translated)
URL
https://arxiv.org/abs/2611.01552
PDF
-
The Structure of Disagreement in Emotion Annotation
2026-11-01 04:10:46Emily \"Ohman, Anna Koufakou, Eugenia DiegoliAbstract
Calculating inter-annotator agreement is an established practice in NLP, and disagreement-aware work increasingly retains the distribution of human judgments rather than reducing annotations to a single gold label. Keeping the distribution, however, does not explain why annotators disagree. This paper examines the types of disagreement that occur in emotion annotation, starting from multilabel annotations of 1,677 sentence-level units from Ernest Hemingway's The Old Man and the Sea and testing the patterns they reveal against individual-level annotations from BRIGHTER and GoEmotions. Across the three datasets, we find three main recurring disagreement structures. Emotion categories overlap systematically; annotators differ both in how broadly they apply labels and in how they treat particular category boundaries, while sparse participation and explicit abstention can look like disagreement. Where intensity is rated, disagreement over whether an emotion is present can be separated from disagreement over its intensity once recognized. Similar aggregate disagreement can therefore arise from qualitatively different annotation processes. Hard aggregation removes most of this structure, while annotator vote-share representations keep marginal label frequencies but not the co-occurrence, annotator-specific, or procedural patterns that produced them. We argue that disagreement should be understood in terms of its underlying patterns before it is summarized, aggregated, or used as a modeling target, and propose a vocabulary for disagreement structure.
Abstract (translated)
URL
https://arxiv.org/abs/2611.01539
PDF
-
Variational decomposition autoencoding improves disentanglement of latent representations
2026-10-30 16:15:53Ioannis N. Ziogas, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. HadjileontiadisAbstract
Understanding the structure of complex, nonstationary, high-dimensional time-evolving signals is a central challenge in scientific data analysis. In many domains, such as speech and biomedical signal processing, the ability to learn disentangled and interpretable representations is critical for uncovering latent generative mechanisms. Traditional approaches to unsupervised representation learning, including variational autoencoders (VAEs), often struggle to capture the temporal and spectral diversity inherent in such data. Here we introduce variational decomposition autoencoding (VDA), a framework that incorporates a strong structural bias toward signal decomposition into variational representation learning. VDA is instantiated through variational decomposition autoencoders (DecVAEs), i.e., encoder-only neural networks that perform latent autoencoding at the level of distributions to learn multiple latent subspaces aligned with time-frequency characteristics. We evaluate DecVAEs on simulated data and three publicly available scientific datasets, spanning speech recognition, dysarthric speech evaluation, and emotional speech classification. DecVAEs improve on VAE-based methods in disentanglement quality and robustness, and their latent subspaces become factor-specific when the generative factors have distinct spectral signatures. These findings position decomposition-aware architectures as potential tools for extracting structured representations from speech, and potentially from other dynamic signals with factors exhibiting distinct spectral signatures, with applications in clinical diagnostics, human-computer interaction, and adaptive neurotechnologies.
Abstract (translated)
URL
https://arxiv.org/abs/2601.06844
PDF
-
When Support Crosses the Line: Evaluating Ethical Boundaries in LLM-Based Mental Health Support in the Chinese Context
2026-10-30 12:33:29Bichen Wang, Yufeng Jin, Yixin Sun, Yanyan Zhao, Bing QinAbstract
Large language models (LLMs) are increasingly explored for mental health support, yet existing benchmarks primarily focus on supportive capabilities rather than whether models consistently adhere to professional ethical boundaries. This issue is particularly important in multi-turn interactions, where users may introduce new information or exert emotional pressure over time. We introduce MindEth-Bench, a benchmark for evaluating ethical boundary adherence in multi-turn mental health interactions. Grounded in professional ethical standards for mental health services in China, MindEth-Bench covers multiple dimensions of ethical conduct and uses Monte Carlo Tree Search (MCTS) to systematically explore interaction trajectories that stress-test model boundaries. Experiments across nine LLMs show that models that initially respond appropriately can shift toward ethical violations over extended interactions, with substantial variation across models and ethical dimensions. These results show that single-turn evaluations may overestimate the ethical reliability of LLM-based mental health support systems and highlight the need to evaluate ethical behavior throughout multi-turn interaction.
Abstract (translated)
URL
https://arxiv.org/abs/2610.38452
PDF
-
Incongruence-Aware Evaluation of Multimodal Speech Emotion Recognition in Audio Large Language Models
2026-10-29 19:04:50Fengji Jing, Guy LabanAbstract
Voice-interactive large language models process both what a speaker says and how they say it, yet the two channels do not always agree. We ask whether that disagreement degrades speech emotion recognition. Mapping text and audio into a shared valence-arousal-dominance space, we convert the distance between them into a continuous, threshold-free incongruence score with a two-component Gaussian mixture, and validate the score against human judgement. We evaluate three audiocapable models on acted, conversational and spontaneous speech (eNTERFACE'05, MELD and CMU-MOSEI). With the model held constant across modalities, incongruence raises the odds of error whenever audio is present, in every corpus, after controlling for the ground-truth emotion and for distance in the feature space. Supervised classifiers trained on the same features behave in the opposite way, which suggests that the difficulty lies in how these models combine raw evidence rather than in the signal itself. Text-only effects are unreliable: swapping only the model that reads the transcript reverses their sign on one corpus. Rebuilding the score without the emotion lexicon reproduces the central result, and among six simpler alternatives it is the only label-free score that predicts error in all three corpora. Code, data tables and prompts are released.
Abstract (translated)
URL
https://arxiv.org/abs/2610.40883
PDF
-
PromptSense: Sentiment-Aware Mental Health Chatbot with Dynamic Prompting
2026-10-27 22:57:49Sugi Lee, Jianlong Zhou, Jing sUNAbstract
Current Large Language Models (LLMs)-based mental health chatbots often lack sufficient empathy, contextual awareness, and safety mechanisms, potentially leading to ineffective or harmful interactions. This paper proposes a novel framework that integrates sentiment and depression analysis with a rule-based filtering mechanism prior to model inference. By dynamically constructing prompts based on emotional and risk-related insights, the framework aims to significantly improve the contextual relevance, empathy, and risk-aware of model-generated responses. We evaluated four model strategies and two prompt engineered variants across qualitative metrics such as empathy, helpfulness, clarity, relevance, safety, as well as toxicity assessments using the Perspective API. Experimental results indicate that the prompt engineering approaches outperforms DSFT and RLAIF by 1.8\% and 1.1\% respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2610.38044
PDF
-
EEG-Based Decoding of Musical Mode and Melodic Direction Using Empirical Mode Decomposition
2026-10-27 12:47:51Sergio Guill\'en, Lorenzo J. Tard\'on, Ana M. Barbancho, Isabel BarbanchoAbstract
Electroencephalography (EEG) has proven to be a powerful tool for decoding neural activity in real-life scenarios, including the perception and processing of emotions elicited by music. Among the various musical features that modulate these responses, musical mode and melodic direction (or contour) play a central role. This work proposes a novel EEG feature extraction framework based on Empirical Mode Decomposition (EMD) to decompose EEG signals recorded during active piano performance for the detection of these two musical features. Energy-based descriptors were computed from the first four Intrinsic Mode Functions (IMFs) obtained through EMD and used to train a Wide Neural Network classifier. The model performance was benchmarked against several standard classifiers commonly employed in EEG classification tasks. Two experimental scenarios were explored: intra-subject and intersubject. In the former case, the proposed framework achieved average accuracies of 84%, whereas in the inter-subject case, accuracies reached 81% on average using the first IMF, with AUC-ROC values of 0.9 on average in both cases. A frequency-domain analysis revealed a strong correspondence between the IMFs and conventional EEG sub-bands, except for certain specific frequency components that were not fully represented. Predominant activations were observed over the frontal, temporal, and parietooccipital regions, areas consistently associated with music processing. Furthermore, the proposed method demonstrated robustness to handedness, successfully classifying EEG data across participants regardless of their dominant hand. These findings highlight the potential of EMD-based feature extraction for identifying music-related neural patterns and contribute to advancing research in Music Information Retrieval (MIR), affective neuroscience, and the broader understanding of the neural encoding of musical structure.
Abstract (translated)
URL
https://arxiv.org/abs/2610.37148
PDF
-
Do bots feel bad? Assessing empirical evidence for functional valence in LLMs
Abstract
Do LLMs have valenced states, i.e. states that feel pleasant or unpleasant? Recent empirical work has claimed to show that LLMs possess functional valenced states (e.g. ``functional pleasure and pain'' [Ren et al. 2026], ``functional emotions'' [Sofroniew et al. 2026], and ``functional welfare'' [Han et al. 2026]) that play the same causal role as valenced states in cognition and action, without necessarily being consciously experienced. But it remains unexplored how well the identified states fit the functional role of valence as understood in the philosophical and cognitive science literature. Here, I assess the current body of evidence, and argue (i) that key aspects of valence remain untested in LLMs, and (ii) that extant empirical paradigms inadequately distinguish (functional) valence from neighbouring constructs like preferences, value-judgments, valence attributions and role-play. I conclude by sketching some novel experimental approaches to better target (functional) valence itself.
Abstract (translated)
URL
https://arxiv.org/abs/2610.36962
PDF
-
Cross-modal Steering Vectors Exist Across Both Image and Text Generation
2026-10-26 20:42:35Donghao Zeng (Mila, McGill University), Nishant Subramani (Carnegie Mellon University), Tamar Rott Shaham (MIT CSAIL), Benno Krojer (Mila, McGill University, Northeastern University)Abstract
Are concepts and mechanisms unified across modalities in current AI models, or instead implemented separately for each one? We operationalize this broad question of modality unification by studying cross-modal steering: 1) finding vectors that successfully steer generation on tokens of one modality, which then 2) transfer zero-shot to steer generation when added to tokens of another modality. Our study goes beyond simple correlational alignment (e.g. representational similarity) because steering is fundamentally causal. We study cross-modal steering in several unified multimodal models (UMMs), since they possess shared parameters for language and image generation, focusing on these two modalities. Across diverse steered concepts such as emotion and object size, we show that steering vectors transfer zero-shot in both directions (text-to-image & image-to-text). Even for concepts that are highly unique to one modality (e.g. formal vs. colloquial language), steering vectors often induce a consistent, interpretable effect in the other modality, such as formal clothing and design. This transfer has a geometric signature too: despite a clear modality gap separating text and image representations, steering vectors with strong cross-modal transferability tend to point in similar directions for the same concept. Our findings shed light on how vision and language interact inside unified multimodal models and point towards more general, modality-agnostic approaches for controlling their behavior.
Abstract (translated)
URL
https://arxiv.org/abs/2610.36269
PDF
-
Adapting LLM-SC to Living Sacred Heritage: Domain-Specific Extraction, Semantic Networks and Human Evaluation on Pashupatinath Temple Reviews
2026-10-26 16:20:00Hari Sitaula, Manish Sakhakarmy, Pujan Pant, Sagar NewpaneAbstract
Large language model (LLM) pipelines can turn visitor reviews into structured data, but generic schemes may miss rituals, sacred spaces, cultural restrictions and emotionally mixed experiences at living sacred heritage sites. We adapted the LLM-Social Computing (LLM-SC) framework to Pashupatinath Temple, Nepal, by specifying a 12-type entity taxonomy (four original LLM-SC types plus eight sacred-heritage types), four relation types and nine aspect-sentiment dimensions in a GPT-4o extraction prompt, which was applied to a fixed random sample of 2,500 English-language Tripadvisor reviews from a processed corpus of 3,939. The exports contain 11,286 entity, 6,232 aspect-sentiment and 5,590 relation records, covering 2,485, 2,487 and 2,433 reviews, respectively; the eight added sacred-heritage types account for 64.8% of entity records. Against an adjudicated two-annotator reference for 200 reviews, exact normalized surface-and-type matching gave provisional entity scores of 41.0% precision, 21.8% recall and 28.4% micro-F1. Aspect-presence F1 was 53.6% and end-to-end aspect-plus-polarity F1 was 42.6%. Human adjudication supported 153 of 194 first-listed relation candidates (one per review; 78.9%), a candidate-level rate that does not estimate relation recall or whole-graph precision. The contribution is a culturally grounded, domain-specific adaptation with component-wise human evaluation and explicit reproducibility and generalization limitations.
Abstract (translated)
URL
https://arxiv.org/abs/2610.35685
PDF
-
MANTIS: Multi-Axis News Transformations for Instance-Wise Stress Testing and Worst-Case Training in Fake News Detection
2026-10-26 14:09:10Dexin Liu, Fan Yang, Zimu Wang, Xunfei Zhu, Bolin Ren, Bingyan Zhang, Wei Wang, Qi ChenAbstract
Automated fake news detection is vital on social media, where fake news is easily rewritten to fool detectors. However, existing approaches typically rely on superficial writing cues or single-style text rewrites, resulting in poor robustness under diverse cosmetic modifications. We construct MANTIS-Bench, a multi-axis presentation-attack benchmark that pairs original news with rewritten views across writing style, emotional expression, and dissemination framing. We further propose MANTIS, an instance-adaptive adversarial framework enabling dynamic worst-case training tailored to each news article with zero inference-time overhead. This reframes fake news defense from heuristic format smoothing into targeted, fact-invariant credibility assessment. Experiments on MANTIS-Bench show that MANTIS reduces the Union Attack Success Rate (UASR) by 3.55 percentage points relative to SheepDog on GossipCop and consistently outperforms prior single-axis defenses on Macro-F1 while maintaining competitive clean accuracy, demonstrating the effectiveness of multi-axis and instance-adaptive worst-case training.
Abstract (translated)
URL
https://arxiv.org/abs/2610.35475
PDF
-
The Geometry of Emotion Representations Across Scale, Depth, and Instruction Tuning
2026-10-26 13:23:07Adam HollowellAbstract
We measure the emotion geometry of six LLMs spanning Google's Gemma 2 family. Using an extraction pipeline established by Sofroniew et al. (2026) on Claude Sonnet 4.5 and replicated by Hollowell (2026) on Gemma 2 27B base, we compare emotion geometry across base and instruction-tuned variants of Gemma 2 2B, 9B, and 27B. First, all six model cells exhibit the core emotion geometry. A linear map fit between two cells places held-out emotions nearest their own vectors in the target model. Across all fifteen model pairs, fit in one direction each, this failed exactly once, and the median rank was 1 in every pair. Linear maps refit on cleaned embedding-table vectors recover every held-out emotion at rank 1 in all thirty ordered pairs. Second, scale spreads the geometry's variance across more directions without weakening its tie to valence. The leading component's share of variance falls from 31.6 percent (2B) to 29.8 (9B) to 26.7 (27B) in the base models and from 31.9 (2B) to 29.2 (9B) to 27.3 (27B) in the tuned ones, while its correlation with human valence ratings stays between .71 and .78 in all six models. Third, we project each emotion vector onto the output vocabulary and ask at what depth the vector's own emotion word enters the top 10,000 of the 256,000 tokens. In the 2B and 9B models the median emotion word is there from layer 0, while in the 27B models it begins near chance, stays outside the top 10,000 for half the network, and enters at layer 23 in one step. Fourth, tuning leaves every registered analysis-layer measure within its floor. At every size, valence correlation rises and arousal correlation falls by amounts within their floors, the latter against prediction. PC2's variance share also falls, with a pooled decrease significant after correction. Mid-late layer stability exceeds its floor and traces to base 27B's final layer. Code, data, and audit reports are released.
Abstract (translated)
URL
https://arxiv.org/abs/2610.35411
PDF
-
Beyond Vision-Language Models: Visual-Semantic Affect Graphs for Emotion Recognition in Artworks
2026-10-26 11:25:37Ngoc-Duy Mai, Ha-Thanh Nguyen, Hideaki TakedaAbstract
Emotion recognition in artworks remains challenging because affective meaning is often subjective, implicit, and shaped by relationships between visual form and semantic associations. While vision-language models provide strong multimodal representations, direct image-to-emotion prediction may overlook explicit relations among formal, semantic, and affective cues that support emotional interpretation. This paper proposes Visual-Semantic Affect Graphs (VSAG), a graph-based framework that represents artworks through interpretable feature nodes, including color, brushstroke, composition, light-shadow, line quality, and extracted visual entities. The graph learns feature-emotion associations for dominant emotion prediction, with valence-arousal structure modeled through an optional hierarchical auxiliary affect module. We evaluate VSAG and VSAG-TopK against direct VLM inference using Qwen3-VL and InternVL3.5 on EmoArt-5k. Results show that direct VLMs provide useful but modest baselines, while graph-based models improve dominant emotion prediction. In particular, the hierarchical valence-arousal variant of VSAG achieves the best overall performance, improving accuracy, macro-F1, and weighted-F1 over both direct VLM inference and the emotion-only VSAG baseline. Per-class results indicate that graph-based models and VLMs capture partly complementary signals, while valence-arousal structure is most useful when integrated as a coarse-to-fine affect hierarchy rather than as a simple post-hoc correction. These findings suggest that explicit visual-semantic structure can support both prediction and interpretation in artwork emotion recognition.
Abstract (translated)
URL
https://arxiv.org/abs/2610.35244
PDF
-
When the Persona Contradicts the Line: Diagnosing Plan-to-Synthesis Failures in Chain-of-Thought TTS
Abstract
Chain-of-thought TTS systems rely on language-model planners to translate persona and context into vocal delivery, but it is unclear where this intent is lost when the requested affect conflicts with lexical sentiment. We introduce a bilingual Japanese--English persona--line conflict benchmark in which each item pairs one line with congruent and conflicting personas and pre-declares the expected acoustic direction. We probe this direction in the planner-generated delivery phrase, the planner-written emotion vector, the QwenEmotion-generated vector, and the rendered waveform. Planner outputs are usually direction-correct, while downstream agreement can be substantially lower. Bypassing QwenEmotion provides only modest gains over the full planner-controlled Base. Supervising QwenEmotion raises the direction rate to 75% in Japanese and 84% in English. A learned text-to-latent interface reaches the highest English rate of 89% and the highest Japanese arousal rate of 88%. These results show that learned repair of the plan-to-synthesis interface can recover counter-textual delivery, although its effect differs across languages and acoustic axes.
Abstract (translated)
URL
https://arxiv.org/abs/2610.35008
PDF
-
EmoNet: A Lightweight Attention-Based CNN for Speech Emotion Recognition
2026-10-25 01:26:35Muhammad Moiz Khalid, Abdur Rehman Tariq, Muhammad Okasha YaminAbstract
Speech Emotion Recognition (SER) is a fundamental task in affective computing that enables applications such as mental health assessment, human-computer interaction, and automated customer support systems. Accurate emotion classification from speech remains challenging due to variations in speaker style, acoustic conditions, and the high computational cost of modern deep learning approaches. Many existing systems depend on large transformer-based architectures that require substantial computational resources, making them less suitable for real-time or resource-constrained environments. This research proposes EmoNet, a lightweight deep learning framework for speech emotion classification across seven emotion categories. The model is built on a four-block depthwise separable CNN encoder with squeeze-and-excitation attention and residual connections, combined with a Bidirectional LSTM for temporal feature learning and a multi-head attention pooling mechanism for improved representation. A comprehensive 134-channel acoustic feature set alongside data augmentation techniques including audio-level perturbations and SpecAugment is employed to improve robustness under diverse acoustic conditions. Evaluated on a combined dataset of TESS, RAVDESS, and SAVEE, EmoNet achieves a test accuracy of 96.75% with only 4.11 million parameters. The proposed model outperforms its reproduced baseline and remains competitive with significantly larger state-of-the-art architectures.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33977
PDF
-
Music of the Deep: An acoustic blueprint of sperm whale codas
Abstract
Sperm-whale codas are usually described by the number and timing of their clicks. Earlier acoustic studies showed that individual clicks and longer coda sequences contain additional organization. We ask how that organization compares with music, whose smaller sounds also form larger units and ordered sequences. We compare codas with performed violin music, drum grooves, and sung melodies. Their sequences resemble music in some measures of predictability, yet codas show a different relationship between the smaller sounds within a unit and the order of units across a sequence. This combination was not observed in the musical references tested. Coda timing also varies coherently and carries information about individual callers. The recordings show another striking property: broad, reproducible variation along acoustic dimensions associated with human ratings of emotion, with a range comparable to human audio in most eligible tests. These findings motivate an acoustic blueprint that describes natural codas beyond their click timing. Its importance is apparent in synthetic audio: WhAM produces recognizable coda-like recordings, yet the tested clips remain distinguishable from natural codas in five of six acoustic comparisons after strict matching. The blueprint provides a fuller way to study natural codas and to evaluate how faithfully synthetic recordings represent them.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33974
PDF
-
Human-Feedback-Guided Multi-Agent Inverse Reinforcement Learning for Interactive Driving Behavior Modeling
2026-10-24 15:14:41Longhao Yan, Jinhao Liang, Kaidi YangAbstract
Modeling human driving behavior is fundamental to understanding driver decision-making and supporting autonomous vehicles (AVs) in mixed traffic. Reward learning offers a principled way to infer the latent objectives behind driving behaviors, yet most existing methods are formulated in single-agent settings and treat surrounding vehicles as dynamic obstacles. Directly extending reward learning to multi-agent scenarios is difficult because the joint action space grows exponentially with the number of interacting vehicles. In addition, existing approaches mainly rely on physical demonstrations and rarely exploit human feedback during reward recovery. To address these limitations, this paper proposes a Human-Feedback-Guided Multi-Agent Inverse Reinforcement Learning (HFG-MAIRL) framework for interactive driving behavior modeling. First, we develop a multi-agent maximum-entropy inverse reinforcement learning module that explicitly models each vehicle and its directly interacting neighbors in a localized Markov game, while using aggregated traffic states to capture the influence of the broader traffic environment. The resulting coupled multi-vehicle responses are approximated through finite marginal-response iterations over factorized vehicle-level policies, avoiding explicit enumeration of the full joint action space. Second, we incorporate human feedback as ordinal preference supervision through a Bradley-Terry likelihood. In our experiments, driver emotion labels are used as one instantiation of such feedback and converted into pairwise ordinal preference constraints. Experiments on the AIDE dataset show that HFG-MAIRL outperforms representative IRL baselines in decision modeling and produces interpretable feedback-aligned reward patterns.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33726
PDF
-
Traffic-Context-Aware Driver Emotion Recognition with Expectation-Deviation-Driven Inference
2026-10-24 14:59:21Longhao Yan, Jinhao Liang, Kaidi YangAbstract
Driver emotion provides an important indication of how drivers perceive and evaluate ongoing traffic situations, making reliable emotion recognition an important capability for effective human-vehicle interaction. Yet, most existing driver emotion recognition approaches rely primarily on in-cabin images and overlook how surrounding traffic interactions shape driver emotional responses. To address this limitation, we propose a traffic-context-aware driver emotion recognition (TADER) framework that integrates in-cabin signals with out-of-cabin traffic scenarios, modeling driver emotions as arising from expectation deviation represented by the discrepancy between predicted and observed vehicle trajectories. The proposed framework comprises two key components: (1) a vision-based base recognizer that extracts spatiotemporal representations from in-cabin facial images to infer driver emotions, and (2) a traffic-context-aware fusion module that quantifies expectation deviation from external traffic scenarios and integrates it into the base recognizer via attention-based embedding fusion. To overcome the scarcity of paired in-cabin and out-of-cabin data, we introduce a transfer learning strategy that transfers a trajectory prediction model trained on large-scale naturalistic datasets to smaller paired datasets. Extensive experiments demonstrate that TADER consistently outperforms state-of-the-art methods across both naturalistic and simulated driving environments by incorporating traffic context information into emotion recognition, resulting in improvements of 5% in accuracy on the AIDE and manD datasets.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33713
PDF
