Relation
-
Benchmarking LLM Next-Step Recommendations with Knowledge Space Theory
2026-11-10 07:15:25Peter Steiner, Jan HochweberAbstract
Selecting an LLM for next-step recommendation requires benchmarks that reflect the intended educational task and domain. We present a benchmarking method that uses Knowledge Space Theory (KST) as a formal evaluation reference. Given a task catalogue and an existing prerequisite relation, the method derives admissible knowledge states and the complete set of immediate next steps for each state. Recommendations generated by a model under a specified prompt are compared with these reference sets. A sample evaluation study uses 2,400 responses from six models across four mathematics domains, four prompt conditions, and two catalogue orders. Exact-set accuracy was 61.3% with descriptions only, 62.8% when models were asked to reconstruct prerequisites, 96.3% with supplied prerequisite relations, and 98.2% with the complete knowledge space. Performance varied across models and domains, and recommendation sets agreed across the two presentations in 85.25% of pairs. The benchmark supports comparisons of model-prompt combinations against a context-specific reference. Its scores describe structural recommendation accuracy under the evaluated conditions; they do not establish learning benefit or the necessity of using an LLM for selection.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13667
PDF
-
Predicting Single-Cell Spatial Gene Expression from H&E using Pathology Foundation Models
2026-11-10 06:44:33Ashley P. Tsang, Evan Liu, Conrad Foo, Bob Chen, Hassaan Maan, Donny Chan, Elisa Penna, Veronica Ibarra Lopez, Hector Corrada Bravo, Aicha BenTaieb, Lisa M. McGinnis, Runmin Wei, Kenneth T. GaoAbstract
Predicting gene expression directly from hematoxylin and eosin (H&E) images offers a scalable and cost-effective approach to deriving spatial molecular insights from routine histology. Pathology foundation models, typically pretrained using tile-level objectives, may not adequately capture the fine-grained cellular features required for single-cell molecular prediction. In this work, we systematically evaluate and establish domain-informed and task-specific strategies for adapting pathology foundation models to single-cell spatial gene expression prediction. Using paired H&E and Xenium 5K data, our approach improves mean gene-wise Pearson correlation by 2.67x over the standard zero-shot baseline and achieves state-of-the-art performance on this task. We demonstrate strong gene-detection performance across the full gene panel (AUROC = 0.800) and importantly identify that predictive performance varies substantially across biological contexts, with our model achieving its highest Pearson correlation for spatially variable genes (r = 0.643). Our findings demonstrate that, despite the challenges of sparse single-cell measurements, H&E-derived representations can recover biologically meaningful spatial expression patterns, highlighting both the opportunities and limitations of morphology-based molecular inference at cellular resolution.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13649
PDF
-
DepthGS: Depth-Guided Gaussian Splatting with Adaptive Regularization for High-Quality Surface Reconstruction
2026-11-10 06:16:16Xiao Ren, Yu Liu, Ning An, Jian Cheng, Xin Qiao, You-Fu Li, He KongAbstract
Recently, 3D Gaussian Splatting (3DGS) has emerged as an efficient representation for novel view synthesis, offering fast differentiable rendering and high visual fidelity, and has increasingly been extended to surface reconstruction. However, the relationship between depth information and reconstruction error remains insufficiently explored, limiting the principled design of geometric optimization strategies. To fill the above gap, we propose DepthGS, a depth-guided Gaussian splatting framework. We derive theoretical relations between depth discrepancies and reconstruction error under single-view and multi-view settings, and rigorously establish corresponding upper bounds on reconstruction error. The above analysis informs the design of adaptive regularization strategies in both single-view and multi-view optimization. More specifically, for single-view optimization, we propose a depth-difference-guided adaptive normal regularization strategy that modulates the normal constraint strength using a geometric reliability weight computed from local depth differences. For multi-view optimization, we introduce a cross-view depth consistency regularization strategy that promotes consistency between neighboring-view depth maps and the corresponding depth predictions obtained by transforming the current view's depth predictions into those neighboring views. Extensive experiments on DTU, Mip-NeRF360, and Tanks and Temples demonstrate that DepthGS achieves competitive performance in both surface reconstruction and novel view synthesis.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13628
PDF
-
When to Start Denoising? Adaptive Denoising Entry for Generative Video Compression
2026-11-10 06:02:13Zifu Zhang, Yuqiao Li, Ding Ding, Shengxi Li, Mai Xu, Daowen Li, Kai Li, Ying ChenAbstract
Generative video compression offers a promising solution for compressing and reconstructing videos under extremely low bitrate constraints, among which the diffusion-based solutions still achieve the best performance by providing the powerful generative latent prior. However, their effectiveness critically depends on when the compressed latent is introduced into the diffusion process, which directly governs the trade-off between reconstruction fidelity and perceptual realism. To address this limitation, we identify denoising start-timestep selection as a key factor governing this trade-off and formulate it as a geometric optimization problem. Based on this formulation, we propose a novel adaptive diffusion entry method for generative video compression (ADE-GVC), the first timestep-aware generative video compression framework, featuring a lightweight predictor that estimates an adaptive denoising start point from the compressed latent. A temporal adapter further extracts compression-aware conditions and injects them into a pretrained video diffusion model, enabling the reconstruction of videos with both high fidelity and realism. More importantly, we observe a strong quantitative relationship between the predicted timesteps and the distortion between the original and compressed latents, providing empirical support for our optimization formulation. Extensive experiments demonstrate that ADE-GVC achieves the state-of-the-art performance on perceptual metrics and substantially improves reconstruction realism over existing generative video compression methods. Notably, our ADE-GVC achieves an average bitrate savings of 40.99\% (on DISTS) and 95.24\% (on PSNR) compared to the latest open-source baseline GLC-Video.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13620
PDF
-
Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System
Abstract
Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but which facts that layer should supply is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, posed closed-book, with the reference table, and with computed facts at increasing levels. In eleven open-weight models answering directly, errors concentrate where attributes must be combined: none gets more than 4\% of sexagenary days or 1\% of collisions right. Static attributes have no consistent effect, and the complete relation list leaves most direct-answering models near zero although it contains the answer. A query-filtered layer (exploratory) raises Qwen3-8B and Qwen3-32B to 78--92\% and four further models to 96--100\%. Three proprietary reasoning models compute 68--95\% of collisions unaided. On chess positions, under predictions locked in advance, attributes again do not substitute for relations and listed distractors are misattributed. The fact layer must supply computed interactions; how they are presented depends on the model and the domain.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13597
PDF
-
Metamorphic testing of learned AC optimal power flow surrogates reveals representation dependence
Abstract
Evaluating a learned AC optimal power flow (AC-OPF) surrogate on one grid serialization cannot show whether its accuracy depends on that encoding. We test this dependence with thirteen metamorphic relations, each specifying when a transformation is valid and how to compare its outputs. Our audit covers 17 model and pipeline configurations: two released trained checkpoints and 15 random-weight architecture or mechanism controls. Every configuration violates at least six relations, even at reporting tolerances ten times tighter or looser. For GridSFM-Open v1.1, source inspection and intermediate-tensor interventions point to an indexing bug and three noncanonical or numerically unstable preprocessing choices; a repaired checkpoint remains untested. In a separate case500 training experiment, augmentation lowers transformed-view error in three of four models and median discrepancy and constraint-violation P95 in all four. These benefits depend on the training protocol: some relations worsen, and every evaluated prediction remains AC infeasible. A GPU/CPU eigensolver mismatch accounts for most of the large raw-Laplacian-baseline gain. We recommend reporting representation consistency alongside accuracy, feasibility, and cost. These results establish neither deployment readiness nor universal invariance.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13566
PDF
-
CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation
2026-11-10 04:23:11Sanghyuk Chun, William Yang, Amaya Dharmasiri, Olga RussakovskyAbstract
Uncertainty estimation has been a long-standing challenge in AI models; it amounts to "knowing what you don't know," and metacognition is notoriously difficult even for humans (cf. the Dunning-Kruger effect). Although it is still far from solved even in simpler classification systems, tackling it in multimodal large language models (MLLMs) is becoming increasingly important. Within MLLMs, uncertainty can stem from any of the diverse sources as well as from their relationships, and further can stem from the unbounded answers in the open-ended setting. To tackle the issues, we propose CoMet, an MLLM uncertainty estimation method by decomposing uncertainty into a context-specific term and a multiplicity-specific term. The former captures ambiguity induced by the given context (e.g., task or prompt), while the latter captures how many plausible answers determined by the context remain compatible with the given input. We train a lightweight post-hoc uncertainty module to estimate these quantities, which enables efficient uncertainty estimation without autoregressive answer generation or repeated sampling. Experiments on various open-ended multimodal benchmarks, hallucination detection, and multiple-choice visual question answering benchmarks show that CoMet consistently improves uncertainty estimation over existing baselines while remaining efficient in practice. Code is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2606.32012
PDF
-
Navigating relational agency in AI feedback musicianship: a diffractive dialogue
2026-11-10 04:20:02Yichen Wang, Charles Patrick Martin, David Gunawan, David CooperAbstract
Musical AI systems behave in non-linear, dynamic ways that exceed the tool-oriented framings inherited from earlier instrument design. Existing frames of control and ensemble interaction do not articulate the entangled, relational agency that emerges when performers work with AI feedback instruments over time. We examine a duet practice with two AI-enabled feedback instruments, Sù and Agentier, in which feedback operates at the inner level of the AI's recurrent processes and the outer loop of musical interaction between performer, instrument, and co-performer. Our work draws on agential realism and a diffractive dialogue methodology to read our independent reflections through one another across five performances, examining how agency is enacted between performers and AI instruments. We find that control and agency are relational and shift across performances; musical textures enact feedback moments of control with performers; reaching a productive feedback environment is itself a shared exploratory act; and performers maintain their instruments through funnelling and injection, developing the skill of navigating agency along a continuous spectrum. Feedback musicianship with AI instruments demands navigation of this relational agency, displacing the leading-versus-responding binary that initially framed our reflections. This work contributes a materially-grounded vocabulary for AI feedback musicianship and a model of human-AI creative collaboration in which mutual agency is worked for through practice rather than designed in.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13555
PDF
-
Mass Scaling Laws for Energy-Optimal Horizontal Multirotor Flight
2026-11-10 04:03:48Ayush Patnaik, Nicolas Michel, Leonard Baursfeld, Davide Scaramuzza, Xinfan LinAbstract
Multicopters are a type of aerial robot seeing extensive real-world applications but with range and endurance as critical bottlenecks. Optimization of its energy efficiency through energy-aware flight planning (e.g. cruise speed-scheduling) has hence been an important topic. The mass of the multicopter is an important factor to consider, as it has a significant impact on efficiency, and many common applications are subject to frequent mass variation due to payload change. This work derives closed-form scaling relationships between the energy-optimal steady horizontal flight behavior of a multicopter and its vehicle mass, and validates them experimentally on two multirotor platforms. Specifically, for mass-only variation, i.e., when payload changes the vehicle mass while the airframe geometry, aerodynamic characteristics, and rotor configuration remain unchanged, we show through a combination of numerical simulation, theoretical derivation, and experimental validation that (1) the energy-optimal velocity is proportional to the square root of vehicle mass; (2) the energy-optimal pitch angle is constant with respect to mass; and (3) the optimal energy per meter is proportional to mass. The analysis shows that these three relationships arise from a common mass-similarity property of the steady-state flight and rotor-aerodynamic equations. Furthermore, the obtained scaling relationships enable simple yet effective cruise-speed planning to optimize energy efficiency and extend the range and endurance of the multicopter.
Abstract (translated)
URL
https://arxiv.org/abs/2611.04748
PDF
-
FaiSonic MMD: A popular music dataset with temporally aligned multidimensional annotations for multitask music analysis
2026-11-10 03:16:27Tieyao Zhang, Yuke Liu, Jiaxing Yu, Xinda Wu, Genfang ChenAbstract
Temporal analysis in music information retrieval concerns the positions of musical events and structures within recordings. Joint modeling has demonstrated the value of combining musical descriptions, but existing resources differ in joint coverage and expert verification across dimensions, constraining detailed joint supervision and the analysis of relationships on the same recordings. To support multitask modeling and the study of relationships among musical dimensions in popular music, we introduce FaiSonic Multidimensional Music Dataset (FaiSonic MMD), containing 500 songs totaling approximately 32.86 hours, comprising 250 Chinese-language and 250 English-language songs. Professional musicians annotate phrases, sections, chords, and rhythm in real recordings, followed by independent review. Events and intervals across dimensions share a recording-based time reference while retaining their respective musical granularity and boundaries, allowing direct comparison across musical descriptions. We describe the annotations, collection procedure, and statistics, and establish a comprehensive multitask baseline for phrase boundaries, section boundaries, section functions, chords, beats, and downbeats. We report reference performance for all prediction targets. FaiSonic MMD provides shared expert annotations and accompanying benchmarks for temporal music analysis.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13503
PDF
-
Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation
2026-11-10 03:08:22Jiawei Xiao, Boya Wang, Longtian Qiu, Shan Ning, Xuming HeAbstract
Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13496
PDF
-
Off-target posterior mass: A training-free diagnostic for fine-tuning spoken language diarization probes
2026-11-10 01:15:31Zhuoyan Tao, Aditya Kommineni, Shrikanth NarayananAbstract
Spoken Language Diarization (SLD) models provide frame-level language information of a speech recording. SLD modeling pipeline includes feature extraction from pre-trained speech models followed by clustering. Fine-tuning pre-trained speech models has shown performance gains for SLD with its magnitude being dataset dependent. Off-target Posterior Mass (OTPM), computed based on posterior language identification probabilities is proposed as a diagnostic metric to forecast performance gains from SLD fine-tuning. OTPM obtains positive correlation with probe gains, 0.56 on unseen synthetic bilingual data and 0.23 on real-world corpora. These results indicate promising potential of OTPM as a diagnostic metric for SLD.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13430
PDF
-
The AI Era's Lines of Code? A Case Study on What an Engineer's Token Spend Measures
2026-11-10 00:17:26Shyam Agarwal, Olga Lvova, Agnia Sergeyuk, Anastasiia Serova, Timofey Bryksin, Thomas Zimmermann, Bogdan VasilescuAbstract
Organizations use engineers' token spend to monitor AI adoption, set budgets, and measure productivity, but the metric's relationship with engineering output is unclear. We link five weeks of token usage in coding agents to the commits of 404 engineers at JetBrains, and analyze a company survey of 412 employees. We find that token spend varied mostly between engineers, and which engineers spent the most stayed largely the same over the five weeks. Among weeks with similar commit counts, the 90th percentile of spend was 39 to 240 times the 10th. Within an engineer, doubling spend was associated with about 10% more commits. Available work metrics explained 38% of the variation in spend between engineers and 13% of the week-to-week variation within an engineer. In the survey, respondents in technical roles reported in nearly nine of ten agent evaluations using agents for tasks beyond coding, such as analysis, research, and planning. Token spend therefore measures how much engineers use agents across many kinds of work and not the outcome and value of that work. We recommend using token spend at most for resource accounting, and evaluating agent-assisted work with evidence about tasks and outcomes.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13396
PDF
-
Agent-Guided Discovery of Exchange-Correlation Functionals with Physics-based Validation
2026-11-09 22:56:02Achuth Chandrasekhar, Amir Barati FarimaniAbstract
Accurate, compact exchange--correlation functionals are central to predictive density functional theory. Symbolic search and language-model agents can propose new functional forms, but it remains unclear whether their gains survive self-consistency and external evaluation, whether the new terms carry physical content, whether agents outperform evolutionary search, and how language-model backbones compare. We connect frontier LLM agents to a GPU evaluator through the Model Context Protocol, with training-only fitting, persistent memory, and self-consistent and external evaluation. Opus-5.5 agents passed the screening gate with about 40-fold fewer tokens than Codex (GPT-6-Astra) agents and found a candidate with lower validation and held-out errors; only this candidate advanced to self-consistent evaluation. The resulting 13-parameter functional, O159, multiplies same-spin correlation by a smooth density window. Refitted controls favor a window that scales the entire same-spin factor, reducing it by 15--30\% at its peak, and that acts on reaction energies mainly in near-iso-orbital regions; against a refitted no-window control, it lowers heavy-atom total-energy errors on GSCDB137 by 29\%. With about 400 fits each, SyFES evolution did not improve on its GAS22 seed, whereas the agents found forms 0.10--0.13 \kcal\ lower in validation error by adding an explicit density scale. Self-consistent O159 lowers the held-out MGCDB84 test error of $\omega$B97M-V by 11.7\%, and its mean normalized error ratio on 36 GSCDB137 datasets (1.139) is comparable to $\omega$B97M-V (1.144) and below GAS22 (1.190). Frontier-LLM agents thus produced O159, a compact functional whose improvement over $\omega$B97M-V survives self-consistent, physics-based validation.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13329
PDF
-
Does Haptic Connection to a Healthy Partner Improve Motor Learning in Chronic Stroke? A Pilot Study of Robot-Based Dyadic Training
2026-11-09 22:20:25Erica L. Waters, Natalia Stroutinsky-Sobotka, Reem M. Azar, Frances S. Shofer, Pamela Z. Cacchione, Rochelle J. Mendonca, Michelle J. JohnsonAbstract
Purpose: Robot-based haptic dyads, where two users exchange forces through robots, have been proposed for robot-assisted neurorehabilitation. This intervention could increase motor learning by promoting motor variability, providing an implicit communication channel, and facilitating social interaction. Despite the potential benefits, few studies have investigated robot-based haptic dyads for neurological populations, and none have included a comprehensive clinical assessment. Design/Methodology/Approach: We present a pilot study investigation of individuals with chronic stroke learning a complex tracking task in solo and dyad conditions. Prior to training, participants completed a battery of cognitive, motor, and sensory assessments. We assessed changes in key kinematic outcomes including tracking error, distance traveled, mean absolute power, and sample entropy. We also performed a correlation analysis of clinical scores with baseline tracking errors. Findings: Baseline tracking error was strongly correlated (r > 0.6) with lower cognitive scores and higher motor scores. Only the solo group demonstrated lower tracking error at assessment ({\Delta} = -5.33, 95%CI [-10.25, -0.40], p = 0.027), but this result was tempered by the removal of an outlier participant. Neither group demonstrated clear retention. The dyad group showed significantly elevated performance, effort, and movement complexity during training. Research Limitations/Implications: The selected task was too difficult to reveal learning for either group, which may be related to its high cognitive demand. Sample size and stroke heterogeneity were also limitation. Originality/Value: Dyadic training increased movement complexity and effort, which are associated with better motor learning. Clinical characteristics such as lesion location, cognition, sensation, and chronicity may influence potential benefits of dyadic training.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13307
PDF
-
ExpertCrowd-SWE: Measuring Frontier Coding Agents on Expert-Authored Engineering Tasks in Unfamiliar Codebases
2026-11-09 21:20:57Josip Vrdoljak, Akash Singh, Bas Steunebrink, Alex Yarotsky, John Lorenzo, Achille Nicoletti, Sam PigottAbstract
ExpertCrowd-SWE (EC-SWE) is a benchmark of 100 expert-authored, multi-file software engineering tasks for evaluating autonomous coding agents in large, unfamiliar codebases. Each task is a change request written by a practising engineer against a pinned commit of a real repository, paired with a sealed reference solution and a task-specific grader whose weighted, mostly executable criteria award partial credit; must-pass blockers and pass/fail gates encode what a maintainer would refuse to merge. The tasks span 87 open-source projects and 11 languages. The median ticket is 148 words, while the median reference solution changes 471 lines. Every task image is calibrated before release so that the reference solution scores 1.0 and both the untouched repository and an inert change score 0; nearly half of the tasks also ship alternative, lazy and adversarial control patches that the grader must classify correctly. The final 100 tasks were selected for difficulty against current frontier models. We evaluate seven frontier models, five runs each (3,495 scored runs), in a single platform harness that gives each model its provider-native tool interface. Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) lead without a statistically significant gap, followed by Claude Fable 5.1 (46.5%), Gemini 3.8 Flash (40.0%), and a statistically indistinguishable group of Qwen 3.8 Max, DeepSeek V4.1 Flash and Grok 4.6 (32-34%). No task is fully solved by every model, and on 80 tasks no model earns full credit on all five runs. Scores span 35 points. Depending on the model, agents take a median of 33 to 166 steps per run, with little relation to score. We release 24 tasks with their environments, graders and results, and hold out 76 to limit contamination.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13246
PDF
-
Characterising divergence between heuristic and language-model ratings of mobile user interface quality
2026-11-09 21:15:50Md Rejaul Korim Sadi, Golam Mostofa Naeem, Toufiqur Rahman Tasin, Syed Mostofa Moosa, Mahmudul Hasan Emon, Mahmudur Rashid, Ferdus AhmedAbstract
Automated assessment of mobile user interface quality is attractive because manual review does not scale, and large language models are increasingly proposed for the task. Whether their ratings correspond to those produced by established rule-based heuristics has not been characterised at scale. We construct a 15,000-screen corpus from the RICO dataset using six documented filters, score every screen with a literature-grounded heuristic scorer, and collect ratings from three open-weight language models across two input representations, yielding 270,000 ratings. We report linearly weighted Cohen's kappa with bootstrap confidence intervals, offset-corrected kappa, intraclass correlation, and Bland-Altman limits of agreement, together with a chance baseline for the tolerance-band agreement rule. Agreement between the two rating sources is consistently weak: raw kappa falls below 0.15 in seventeen of eighteen model-representation-dimension cells. However, fifteen of eighteen cells are statistically distinguishable from zero, and correcting for a constant offset raises five cells above 0.15, so the divergence is better described as a combination of systematic bias and weak covariation than as absence of relationship. Model ratings track structural properties of the interface, but substantially less strongly than the heuristic does, and more strongly from an untruncated structural element list than from a screenshot. We further show that dimensions elicited in a single generated response are correlated by construction, reaching rho = 0.97 in one condition, which constrains what such designs can measure. We report these results as a characterisation of divergence between two automated raters rather than as evidence that either is correct, and we set out the validation a stronger claim would require.
Abstract (translated)
URL
https://arxiv.org/abs/2609.05423
PDF
-
Reliability of Claude Code as an Agentic AI Grading System for Handwritten University Exams
2026-11-09 20:39:02Rayed AlGhamdiAbstract
This study examines the reliability of Claude Code, an agentic deployment of the Claude Opus 4.6 large language model, as a grading system for handwritten university exams. Four research questions addressed agreement with expert human grading on total scores (RQ1), agreement across five question types from multiple choice to evaluative judgement (RQ2), consistency across two independent grading sessions one week apart (RQ3), and the accuracy of AI-generated feedback (RQ4). Claude Code graded the 20-mark mid-term papers of 60 consenting students in a Saudi computer graphics course in a single agentic session, guided by three rubric files. Reliability was estimated on disjoint samples: 20 papers for RQ1, RQ2 and RQ4, and 15 papers for RQ3. Analyses used Pearson correlation, mean absolute error, Bland-Altman analysis, weighted Cohen kappa, intraclass correlation, and a four-category coding scheme separating diagnostic from confirmation feedback. The author, as the sole subject-matter expert for the course, provided the human benchmark, so the design assesses agreement with a domain-expert reference rather than inter-rater reliability. AI grading agreed strongly with human grading on totals (r = 0.97; MAE = 0.85 marks) with a small but significant AI-positive bias of 0.68 marks, showed excellent test-retest consistency (ICC = 0.96), and produced accurate feedback (92.8% diagnostic; 97.8% confirmation). Drawing items were the outlier on three of four measures. Three failure modes recurred: keyword matching without conceptual depth, selection-state ambiguity on corrected multiple-choice answers, and region-of-attention drift on hand-drawn diagrams. The findings suggest that AI grading at this capability level can assist instructors on objective and short-form items under structural safeguards but is not yet ready to replace instructor judgement at boundary cases or on drawing items.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13217
PDF
-
Structured Kernel Surrogate Models for Multimodal Attribution
2026-11-09 20:21:39Ziniu Zhang, Zhenshuo Zhang, Jianglin Lu, Ruoxuan Xiong, Yun Fu, Hongyang R. ZhangAbstract
We study the problem of cross-sample modality attribution, which involves measuring how one modality in one sample affects the learning outcome in other multimodal samples. This problem is crucial in a multimodal system because a data modality (such as a text description) may be redundant for one sample (a self-explanatory image), but the same data modality would otherwise be uniquely informative for another sample, or informative when combined with other modalities. Prior work focuses on influence estimation at the level of one sample, and it remains an open question how one should capture the interactions across both samples and modalities. To capture scenarios like the above, we propose structured kernel surrogate models, which are surrogate mappings from both samples and modalities to the prediction outcome when they are trained together. We design an algorithm, SKS, which applies kernel ridge regression with a structured, compositional kernel on the selection states of individual samples and modalities. Specifically, the kernel function multiplies kernel-based similarity scores between every pair of data modalities for one sample and a pair of two samples. Additionally, we design a gradient-based estimation algorithm to efficiently estimate the structured kernel surrogate model. We evaluate our approach on two multimodal benchmarks, spanning a range of data modalities including satellite images, road networks, text, audio, and tabular data. We find that SKS achieves $57.8$% higher correlation with the true prediction outcomes than existing methods. In downstream data selection, SKS improves the average accuracy by $1.93$% over baselines. Models with data selected by SKS have lower Hessian traces, indicating a flatter loss surface. Using SKS to select satellite images identifies road segments, where accident scenes commonly occur, as opposed to background images.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13198
PDF
-
From Process Models to Composable Agentic Workflows: Combining BPMN, Agents and Enterprise Knowledge
2026-11-09 20:18:32Christian Fillies, Jan Fillies, Frauke Weichhardt, Gay Anne Wood-AlbrechtAbstract
LLM-based agents can reason, use tools and collaborate. However, coordinating increasingly complex agentic applications becomes difficult to understand, monitor and improve. Decisions distributed across prompts, tool calls and application logic make workflows implicit. We introduce **Recursive Capability Composition**, enabling agents to execute business processes and expose them as reusable Agentic Tools. This paper presents a BPMN-based approach to Agentic Workflows, combining explicit orchestration with autonomous reasoning. SemTalk BPMN models are transformed into executable workflows using a LangGraph/Pregel runtime. Agentic Tasks introduce LLM reasoning at selected activities, while conventional BPMN controls process flows, deterministic decisions, system interactions and human approvals. Agentic Participants provide reusable interfaces for agent behavior or other executable processes. BPMN orchestration makes multi-agent systems more understandable and manageable. Explicit process models provide developers and operators with a shared view of workflow structures and execution progress, facilitating debugging, monitoring and optimization. Enterprise agents also need organization-specific context. Process Models and Knowledge Graphs describe activities, participants, information objects, decisions and relationships between processes. They provide structured knowledge of how an organization operates, supporting informed agent decisions and actions. Our contributions are threefold: 1. **Model-Driven Execution:** Transforming BPMN models into executable Agentic Workflows. 2. **Recursive Capability Composition:** Exposing agents and executable business processes as reusable tools for other agents. 3. **Enterprise Context:** Using Process Models and Ontologies to provide organization-specific knowledge for agent reasoning.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13196
PDF
