arXiv_CV
-
Continuous Angular Control in Text-to-Image Generation via Geometric Token Encoding
2026-11-09 19:55:34Furkan G\"uzelant, Basavaraj Sunagad, Aysegul Dundar, Adam KortylewskiAbstract
Text-to-image diffusion models can render a photorealistic analog clock but cannot control the time it displays. Across four pretrained models, fewer than 1% of generated clocks place both hands within 6 degrees of the requested time; the models instead collapse onto visually dominant configurations such as the canonical 10:10 pose. The same failure appears when a specific fraction of a circular object is requested, where models return complete or conventionally sliced objects. We observe that these two failures share a geometric structure: both target quantities can be parameterized by angular variables. We introduce a geometric token encoding framework that conditions generation directly on these angles. Each angular component is encoded as a periodic sine-cosine feature and projected into a role-specific token embedding that is injected into the prompt before a frozen text encoder. To spatially ground these tokens without requiring localization input at inference, we introduce Stochastic Localization Relaxation (SLR), which stochastically relaxes attention supervision from component-level to object-level to unconstrained localization during training. For clocks, we additionally introduce a hand-swap preference objective that discourages assigning the correct angles to the wrong hand roles. On analog clocks our method reaches 63-78% Acc@6 degrees across evaluation methods, compared with below 1% for the baselines. On circular-fraction generation, it achieves 89.5% Acc@0.10, while maintaining 88.8% on unseen object categories and 84.9% on unseen target values, compared with roughly 20% for pretrained baselines.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13172
PDF
-
VIGOR: Benchmarking Visual Rationale Correctness in Multimodal Large Language Models
2026-11-09 19:30:03Yanlin Chen, Tang Li, Xi PengAbstract
Multimodal large language models (MLLMs) are typically evaluated by whether they produce correct answers, but correctness alone does not reveal whether their outputs are supported by the right visual evidence. We introduce VIGOR, a benchmark for evaluating VIsually GrOunded Rationales in MLLMs. VIGOR represents visual rationales as explicit concepts paired with pixel-level groundings, covering multiple abstraction levels including low-level visual properties, intermediate structures, objects, parts and high-level motion, relations. To construct VIGOR at scale, we develop an automatic annotation framework that reuses existing dense supervision when available and generates missing concept groundings through proposal-based annotation followed by conservative multimodal verification. Using VIGOR, we evaluate whether MLLMs "look at what they say" by comparing model-derived visual evidence for each expressed concept against annotated ground-truth rationales. Our results show that current MLLMs exhibit limited visual rationale correctness and reveal distinct grounding failures across concept levels. VIGOR provides a benchmark and scalable data construction pipeline for studying whether MLLM outputs are supported by appropriate visual evidence. Code is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2611.13150
PDF
-
Stylization as Geodesic Navigation on Diffusion Manifolds
2026-11-09 15:36:42Yan Li, Zehao Li, Xinlong Sun, Fuchen Long, Xi Chen, Wenhan Luo, Yike GuoAbstract
Facial stylization asks for a delicate transformation: the image should look unmistakably artistic, yet the person should remain unmistakably the same. Existing training-free and instruction-driven methods often fail to expose such control, either drifting away from identity or producing only weak stylization. We introduce Stylus, a training-free framework that treats facial style transfer as navigation on the curved latent manifold of a pretrained diffusion model. Rather than interpolating in flat Euclidean space, \method refines the interpolated content--style point by lowering a ratio-weighted discrete geodesic energy under the pullback metric, made tractable by a rank-one power-iteration approximation of the Jacobian. This principle drives two complementary controls: content--style latent interpolation for smooth semantic fusion, and self-attention query optimization for spatially adaptive style injection. The resulting dual-control mechanism provides continuous, fine-grained control over stylization strength while preserving identity-critical facial structure, all without additional training. Experiments show that Stylus achieves the strongest identity and content preservation at competitive stylization strength.
Abstract (translated)
URL
https://arxiv.org/abs/2611.14097
PDF
-
SpatialSR: Spatial Resolution-Centric Pipeline for Arbitrary-Scale Remote Sensing Super-Resolution
2026-11-09 12:47:34Yan Zhang, Runfeng Luo, Liangke Gao, Lin Yuan, Bin Xiao, Xinbo GaoAbstract
Although remote sensing image arbitrary-scale super-resolution (RSI-ASSR) methods support reconstruction at arbitrary or continuous scales, they predominantly condition on the relative scale factor and have not sufficiently investigated the absolute spatial resolution itself as a reconstruction prior, thereby disregarding the inherent spatial context of objects, which frequently leads to spatial-related distortions and compromised performance in downstream remote sensing applications. To address these challenges, we introduce SpatialSR, a SRe-centric arbitrary-scale RSISR method that leverages the spatial resolution (SRe) of remote sensing images (RSI) as an essential prior for super-resolution reconstruction. Distinct from existing SR methods, SpatialSR aims to restore high-resolution remote sensing images with a target SRe (typically set to 0.1m/pixel to align with the metadata of commonly used downstream task datasets). Specifically, we design a Spatial Resolution Estimation Module (SREM) to estimate the SRe of low-resolution (LR) images, thereby dynamically determining the required arbitrary scaling factor. Next, we employ a pretrained SRe-to-image diffusion model to extract SRe-specified texture-style codes, which are further incorporated into a carefully designed Resolution-aware Style Consistency Module (RSCM) to constrain the SRe-aware texture consistency of reconstructed images. Furthermore, to overcome the high computational expense inherent in diffusion-based SR methods, we wrap the iterative diffusion process as a Trajectory Consistency (TC) loss, which enables comparable reconstructed texture quality of the generative model in SpatialSR within a single step. Extensive experiments demonstrate that the proposed SpatialSR achieves state-of-the-art (SOTA) performance on commonly used RSISR and downstream (image classification and segmentation tasks) datasets.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13046
PDF
-
MICRO: A Surgery-Phase Retinal Image Rigid Registration Benchmark and One-Step Frequency-Structure Similarity Registration Learning
2026-11-09 12:45:53Mingdao Gong, Yan Hu, Mingzhen Yuan, Jialu Zhang, Junming Feng, Yiming Jiang, Jing Ma, Jinghua Liu, Huazhu Fu, Hai Lu, Xiaoqing Zhang, Jiang LiuAbstract
Precise multimodal retinal image rigid registration (MRIRG) is essential for aligning cross-modal vessel structures, offering critical guidance for ophthalmic surgery. Despite its significance, the advancement of MRIRG is hindered by the lack of publicly available multimodal retinal image rigid registration datasets spanning various ophthalmic surgery phases (e.g., preoperative, intraoperative, and postoperative). To address this issue, we introduce the MICRO dataset, a large-scale MRIRG benchmark, comprising 1,365 cross-modal retinal image pairs with standardized annotations across all ophthalmic surgery phases. Moreover, existing deep registration methods achieve competitive MRIRG results via a two-step segmentation-then-detection framework, but segmentation quality heavily constrains their performance. We argue that this dependency stems from an implicit need for keypoint topology constraints, which have not been exploited for MRIRG through a one-step registration framework. Inspired by the above analysis, we propose a novel One-step Frequency-Structure Similarity Registration (OFS2Reg) learning framework for MRIRG that leverages the potential of cross-modal vessel structure similarities to construct keypoint topology constraints under a shared detection network. To be specific, OFS2Reg applies a well-designed frequency-guided structure preservation descriptor to construct coarse keypoint topology constraints in descriptors by highlighting significant vessel structures. Additionally, we develop a new cross-modal vessel structure similarity learning method that refines cross-modal keypoint topology constraints to precisely align pixel-level cross-modal vessel structures by leveraging cross-modal vessel structure similarity priors.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13045
PDF
-
TypedInk: Knowing When Not To Draw
Abstract
Can a scientific figure generator know when it does not have enough evidence to draw? Complete-source illustrators map a finished methodology section to a schematic, or a table to plotting code, and are never asked this question because their input is assumed complete. The research record is rarely that complete: an outline, a draft paragraph, a table, or a one-sentence intent, in any combination, and painting measured numbers with an image model when the table is missing is a hallucinated result. TypedInk infers a figure type from {diagram, plot, plate} from a possibly incomplete source and dispatches to the renderer that can honor that type's evidence contract, refusing dispatch when the contract cannot be met. On the six gold plots in TypedFigBench (5 views, n=30), TypedInk's code path inks every gold table token when a table is observed (12/12), refuses the 12 hidden-table views instead of inventing numbers, and marks 6 intent-only views illustrative, with no execution failures. An always-draw coder with no such gate inks 30/30 charts, including all 18 with no table, and recovers zero gold numbers on any of them; two vision-language raters (not humans; human replication is future work) still judge those always-draw charts submittable 17/36 times, against TypedInk's refusal 0/36 times. A diagnostic field ablation of the type router (16 illustrations, 5 views each, n=80) locates where this failure would originate if unchecked: an outline alone yields type accuracy 0.31 and sends every gold plot into an integrity violation, and a chat-model planner narrows but does not close the gap (overall accuracy 0.85, still integrity-failing on 25% of outlines). Diagram-label recall is 0.47 on the full record and 0.19 on an outline. Gold plates are typed at dispatch (oracle integrity-fail 0.00) and rendered as stills.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13027
PDF
-
Is a CNN-Transformer Ensemble Worth It When a Chest X-Ray Foundation Model Is Available?
Abstract
Ensembles of ImageNet-pretrained convolutional networks and Vision Transformers are a common way to improve multi-label chest X-ray classification. We ask whether such an ensemble is still worth its cost when a model pretrained on chest radiographs is available. On the official patient-disjoint split of NIH ChestX-ray14 (25,596 test images, 2,797 patients), we train five ImageNet-pretrained backbones and the EVA-X-S foundation model with validation-only model selection, compare five fusion rules, and estimate every difference with a patient-level paired bootstrap. EVA-X-S alone reached a test macro-AUROC of 0.828; the five-model ensemble reached 0.814 (difference -0.0136, 95% CI -0.0178 to -0.0098) at about 7.8 times the summed backbone latency and ten times the parameters. Averaging the five ImageNet models with EVA-X-S changed AUROC by -0.0056 (-0.0095 to -0.0021). With all five ImageNet backbones trained with the EVA-X-S recipe, their ensemble scored 0.817, still -0.0109 (-0.0142 to -0.0076) below EVA-X-S, and across three seeds EVA-X-S varied by at most 0.001 AUROC. Under the ImageNet models' recipe, however, EVA-X-S fell to 0.798. Our full-stacking configuration never outperformed calibrated averaging. In this setting, the tested ImageNet ensembles did not improve on a single chest X-ray foundation model fine-tuned with its own recipe.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13014
PDF
-
Preserving Cultural Records: A Large-Scale Benchmark and Visual-Prior Routing Network for Ancient Book Image Restoration
2026-11-09 03:10:29Shumin Fan, Tianyu Song, Hongming Chen, Xiaodong Zhou, Xiang ChenAbstract
Ancient books preserve invaluable cultural heritage, yet their images often suffer from complex degradations caused by long-term ageing, environmental exposure, and physical damage. Existing ancient book image restoration benchmarks are limited in scale and rely on simplified degradation simulations, which cannot fully represent real-world degradation patterns and restrict the generalization ability of restoration models. To address these challenges, we introduce AncBench, a large-scale synthetic paired benchmark containing 100K aligned clean-degraded image pairs. AncBench models realistic composite degradations through a comprehensive degradation pipeline that considers material ageing, paper texture variation, blur, staining, water-induced geometric distortion, and local occlusion. Furthermore, we propose AncRestore, a degradation-aware visual-prior routing network for ancient book image restoration. AncRestore first learns degradation representations from degraded images and then dynamically selects and integrates suitable visual priors according to different degradation characteristics. Guided by these degradation cues, a dynamic routing mechanism combines expert-wise and region-wise routing strategies adaptively adjusts the contribution of these priors across different regions, enabling the model to recover both global appearance and local text details. Extensive experiments show that AncRestore effectively suppresses composite degradations while preserving text structures and enhancing restoration quality against other methods, highlighting its application potential for intelligent ancient book image restoration in cultural heritage preservation.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13010
PDF
-
Instance-preserving graded semantic reporting for surgical action triplets
2026-11-09 01:39:39Yanchong Xue, Wanting Zhang, Lingtao Yu, Guochao ZhangAbstract
Complete-or-reject reporting of surgical action triplets can discard a correctly identified instrument and action when the target is uncertain. We introduce instance-preserving graded semantic reporting, which separates instance selection from the choice of semantic detail. Each selected instance retains its predicted mask and labels while hierarchical correctness scores guide instrument (I), instrument-verb (IV), or complete instrument-verb-target (IVT) reporting. The utility-increment rule DELTA weighs the expected gain from finer reporting against its error cost. We compare three decision rules on 5,641 test frames from eight surgical videos using fixed predictions and scores. At the 10% development calibration target, DELTA retains correct partial reports for 80.0% of candidates rejected by a matched complete-or-reject baseline (5,329/6,662). Adding IV increases source-normalized weighted correct-report yield by 19.13 to 23.13 per 100 frames across all three rules relative to separately recalibrated I/IVT controls. Weighted reporting risk changes by -0.77 to +1.01 percentage points, with complete correct-triplet counts unchanged or reduced. Graded reporting makes partial surgical predictions available as localized descriptions, with an explicit trade-off between report utility and completeness.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13009
PDF
-
G-STAR: Group-Scaled Token Capacity for Deeply Compressed Autoregressive Generation
2026-11-08 16:59:18Xu Huang, Ye Huang, Xiaotong Li, Zhengrong Yue, Jialv Zou, Minhao Jing, Haodong Li, Wenhao Yan, Yaxin Luo, Duomin Wang, Daquan ZhouAbstract
Discrete autoregressive models offer a scalable framework for image generation. However, they face a fundamental trade-off: long token sequences preserve representation quality but slow generation, while aggressive spatial compression reduces representation capacity. Based on product quantization, we show that, under a fixed nominal information budget, group size should scale quadratically with the per-axis spatial compression factor. Building on this principle, we propose GSTAR, a spatial--group co-scaling framework that enables \(32\times\) and even effective \(64\times\) compression while preserving strong reconstruction, understanding, and generation performance. Since larger groups introduce intra-group dependencies that must be modeled efficiently, we analyze their impact on inference latency and generation quality and introduce a semi-autoregressive decoder that combines parallel feature computation with lightweight causal modeling. With these designs, our highly compressed models perform competitively with leading models across five representative text-to-image benchmarks (GenEval, DPG-Bench, PRISM, LongText-Bench, and Qwen-Image-Bench). At \(1024\times1024\) resolution, our model delivers a \(20\times\) speedup over prior discrete autoregressive models, generating each image in 2.5 seconds on a single NVIDIA H200. Together, these results show that highly compressed discrete representations can enable high-quality autoregressive generation at low latency.
Abstract (translated)
URL
https://arxiv.org/abs/2611.13004
PDF
-
Evidence-Grounded Iterative Reasoning for the Diagnosis of Diffuse Cystic Lung Diseases
2026-11-08 07:48:04Haoqing Li, Jun Shi, Tou Li, Xianmeng Chen, Rui Wang, Yating Peng, Qiwei Jia, Li Gao, Qianyi Chen, Hong An, Ruoyun Ouyang, Xiaowen HuAbstract
Overlapping imaging manifestations complicate the differential diagnosis of Diffuse Cystic Lung Diseases (DCLDs), while limited annotated data constrain supervised model development. Multimodal Large Language Models (MLLMs) can integrate imaging and clinical information, but their diagnostic reasoning remains limited by incomplete volumetric perception and insufficient grounding in disease-specific evidence. We propose DCLDs-EIR, an evidence-grounded iterative reasoning framework coupling diagnostic hypothesis refinement with multimodal evidence acquisition and verification. Multiview renderings of predicted lung and cyst segmentations characterize global cyst distributions and initialize the differential diagnosis. Competing hypotheses guide axial CT inspection to identify local findings that, together with clinical context, inform retrieval from a disease-specific knowledge graph. Accumulated patient observations and retrieved knowledge are jointly assessed for diagnostic support and refutation to revise the diagnostic state. Updated hypotheses and unresolved distinctions guide subsequent inspection, forming a feedback loop between evidence acquisition and hypothesis verification. Multicenter evaluation demonstrates that DCLDs-EIR consistently improves diagnostic accuracy over baselines and enables traceable differential diagnosis through explicit links between diagnostic hypotheses, patient observations, and disease-specific knowledge.
Abstract (translated)
URL
https://arxiv.org/abs/2611.12981
PDF
-
Selective Surgical Action-Center Localization:Retaining Localization Gains through Supervised Scoring
2026-11-08 04:58:41Yanchong Xue, Wanting Zhang, Lingtao Yu, Guochao ZhangAbstract
Surgical action-center localization requires both an accurate reference point and a decision about whether to report it. When frames with and without a reportable center are mixed, the benefit of improved localization depends on which correct candidates survive selection. We present a selective localization pipeline combining heatmap and coordinate supervision with a logistic-regression scorer. The scorer learns to identify predictions with a reportable center and a localization error within a specified tolerance. The internal evaluation uses 5,000 frames from 50 surgical sources, five source-grouped folds and two random seeds. Matched locator and scoring comparisons track correct candidates and their rejection. At an 80% acceptance quota per source and a tolerance of 0.10 times the image diagonal, our method retains 46.08 correct outputs per 100 inputs. The baseline locator with the same scoring approach retains 44.20. The gain is 1.88 (descriptive 95% source-paired bootstrap interval, 0.94 to 2.79). Under entropy-based selection, increased rejection of correct predictions offsets the higher supply of correct predictions on average. For the coordinate-supervised locator, supervised scoring reduces these rejections from 6.15 to 2.99 per 100 inputs. Evaluating retained correct outputs reveals whether a selection rule preserves or erodes an improvement in localization.
Abstract (translated)
URL
https://arxiv.org/abs/2611.12977
PDF
-
Partition, Refine, and Detect: Preserving Novel Category Capacity in Open-Vocabulary Object Detection
2026-11-08 04:13:25Wei Zhang, Zhu Teng, Zhixiang Chi, Junnan Zou, Yi Liu, Jianping FanAbstract
Open-vocabulary object detection (OVD) expects the model trained on base classes to recognize both base and novel classes at inference. Typical methods rely on CLIP-style vision-language embeddings, yet the learned space is often biased toward base categories, with limited effective capacity preserved for novel concepts. Such a base-dominated representation can compress novel semantics, leading to semantic collapse and mis-calibrated confidence at inference. To address this issue, we present PR-OVD, a partition-and-refine framework that explicitly reserves capacity for novel categories and strengthens category separation, while being seamlessly integrated into different detector architectures. Specifically, PR-OVD first performs coarse partitioning to build global divisions for base, novel, and an explicit background partition in both visual and textual modalities, preventing object-irrelevant embeddings from leaking into category-specific representations. It then applies category-level refinement within each partition to sharpen decision boundaries and alleviate the lack of fine-grained region-level pretraining in CLIP. To further provide reliable visual anchors for novel categories, we design a cognitively inspired Structure Prompt (SP) that encodes both egocentric attributes and allocentric context, facilitating stable partition-and-refine learning. Extensive experiments on two OVD benchmarks show that PR-OVD delivers an average 6.5\% proportional improvement on novel categories over the state-of-the-art methods. Notably, PR-OVD also benefits other open-vocabulary tasks, including segmentation and classification, further confirming its effectiveness and generality.
Abstract (translated)
URL
https://arxiv.org/abs/2611.12976
PDF
-
Attribute-Evidence Learning with Multimodal Prompt for Generalized Category Discovery
2026-11-08 04:01:26Wei Zhang, Zhu Teng, Zhixiang Chi, Junnan Zou, Wenxin Luo, Linfeng Ye, Jianping Fan, Konstantions N. PlataniotisAbstract
Generalized category discovery (GCD) aims to discover novel categories from unlabeled data by leveraging limited labeled samples from old classes. Existing methods that rely solely on visual cues often neglect the multimodal nature of human perception, prompting recent approaches to incorporate textual information such as category tags and attributes. However, these approaches typically entangle global tags and local attributes within a shared global image representation, rendering attributes unreliable and potentially providing misleading cues that can drive the model toward incorrect predictions. To this end, we propose AptM-GCD, a novel multimodal framework that exploits apt textual guidance through attribute evidence learning and multimodal prompt reasoning. Specifically, we design an Attribute-Evidence Learning (AEL) mechanism that treats attributes as localized visual evidence rather than global category descriptors. By interacting attribute embeddings with patch-level visual tokens, AEL identifies reliable attribute-related regions and extracts discriminative local evidence, thereby reducing the impact of noisy or irrelevant attribute cues. Building upon the extracted evidence, we further develop a Multimodal Prompt Learning (MPL) mechanism to bridge local attribute semantics and global visual understanding. MPL converts fine-grained textual evidence into visual prompts and injects them into a multimodal prompt transformer, enabling attribute information to enhance category discrimination through contextual reasoning without enforcing direct global alignment between modalities. Extensive experiments on six datasets demonstrate that AptM-GCD consistently achieves SoTA performance across both fine-grained and generic benchmarks. Moreover, the proposed framework exhibits strong generalization ability, delivering consistent gains under unimodal and class-imbalanced GCD settings.
Abstract (translated)
URL
https://arxiv.org/abs/2611.12975
PDF
-
Evaluation protocols for historical script classification: the cost of manuscript-level leakage and the role of soft labels
2026-11-07 16:37:46Daniil I. Naryzhnyi (ITMO University), Roman A. Popkov (ITMO University), Daniil A. Bakalin (ITMO University)Abstract
A paleographer applies a script type classifier to unseen manuscripts. We measure how much testing on other pages of training manuscripts inflates accuracy. On VML-HP-ext (715 pages, 171 Hebrew manuscripts, 14 script classes) we compare page-level and manuscript-level splits. Page-level accuracy drops from 97% to 54% (59% at manuscript level), and is 39% on the official blind test set. The gap of 40 to 47 percentage points (pp) at patch, page and manuscript level is statistically significant and reproduces with ResNet18; on binarized patches (ink and background only) it is 34 to 42 pp, also significant. On unseen manuscripts the model is overconfident: mean confidence 70% against 54% correct pages; patch-level expected calibration error (ECE, mean gap between confidence and accuracy) rises from 0.065 to 0.444. Binarization leaves patch-level overconfidence but cuts it markedly at page level. Confidence still helps triage: in the more confident half of pages 67% are correct, 41% in the rest. Training on expert soft labels showed no evidence of an accuracy difference, though a small page-level loss cannot be ruled out. The predicted distributions are closer to the expert's assessment. ECE is lower at patch level (0.21 to 0.31) and in most runs at page level, but not significantly at manuscript level. Label smoothing (blurring the label, no expert input) also lowers ECE but comes no closer to the expert. Temperature scaling (post-training confidence correction) helps significantly only at patch level. On 97 pages of 16 manuscripts with mixed expert assessment, soft labels cut divergence from the expert by 22 to 30% (significant in four of five runs); the point estimate of accuracy is 16.5 to 23.7 pp below hard labels, but with 16 manuscripts the loss can neither be confirmed nor ruled out. We propose recommendations for the evaluation protocol; code will be released with the paper
Abstract (translated)
URL
https://arxiv.org/abs/2611.12969
PDF
-
Video-Coder: Reinforcing Agentic Coding for Video Generation
2026-11-06 18:59:57Chaoyang Wang, Sicheng Gao, Junan Hu, Kunyang Li, Dongyang Chen, Yiyang Zhou, Huaxiu Yao, Yong Jae Lee, Radu Timofte, Yuzhang ShangAbstract
Recent years have witnessed remarkable progress in video generation in terms of visual quality and temporal consistency. However, generating videos with precise spatial structure, coherent motion dynamics, and controllable scene composition remains challenging. In this paper, we introduce Video-Coder, the first attempt to train a code-driven video generation agent, which autonomously writes and executes code to construct a dynamic 3D scene (Draft Video) that provides structure and motion signals to downstream video generators. To support this paradigm, we design a dedicated data pipeline and curate two high-quality datasets, Video-Coder-SFT-6k and Video-Coder-RL-2k, which contain diverse prompts and ground-truth reference videos. We further develop the Video- Coder Harness, a unified interactive environment integrating code execution, 3D scene rendering, and execution feedback for agent training and inference. We then train Video-Coder with SFT fol- lowed by harness-native agentic reinforcement learning with dual reward feedback, which combines execution-based and video-based rewards. Additionally, we design an annealing training strategy that gradually reduces the execution reward over time, allowing the model to rely more on the video reward in the later training stages. Experiments show that Video-Coder achieves significant gains, improving Minimax-H3 by approximately 7 points on PhyGenBench and 24 points on VBench-2.0. We hope this work can serve as an open foundation for coding agents in video generation, and we fully open-source our data, models, and code
Abstract (translated)
URL
https://arxiv.org/abs/2611.09894
PDF
-
On Agentic Manipulation Policies
2026-11-06 18:59:34Chen Chen, Omid Bagherieh, Naveen Nikamanth Appiah Balaji, Jesse T. Gonzalez, Negin Heravi, Fahad Islam, Xueting Li, Jean-Baptiste Regli, Kejia Ren, Petr \v{S}vec, David J. Yoon, Matthias M\"uller, Vladlen KoltunAbstract
Recent work on robot manipulation has focused on specialist policies trained on demonstrations. We show that generalist AI models can control manipulators more reliably than specialist policies such as vision-language-action models (VLAs) and world-action models (WAMs). We systematically analyze agentic manipulation policies (AMPs), in which a generalist AI model commands the manipulator directly, adapting its actions as the task unfolds. We present AMP1, a reference harness that runs unchanged across single-arm, bimanual, and mobile manipulation. AMP1 achieves state-of-the-art results on the RoboLab-120 and CaP-Gym benchmarks, with no demonstrations or task-specific training. On RoboLab-120, AMP1 reaches an 85.6\% success rate zero-shot, compared to 49.2\% and 42.9\% for leading VLA and WAM policies. The agent succeeds zero-shot in 104 of 108 real-world trials spanning twelve tasks and three robot instances, reducing failure rates by 20x and 15x relative to strong VLA and WAM baselines. The basic harness supplies no grasping, pushing, or placing skills; the agent develops its own strategies, such as searching for hidden objects, using objects as tools, and calibrating its cameras. With the addition of a persistent memory store, the agent can also learn from experience: an augmented harness with a memory representation that is carried across episodes lowers the failure rate on RoboDojo bimanual manipulation tasks by 4.6x relative to zero-shot deployment. Our analysis suggests that AMPs are substantially more general and robust than specialist manipulation policies, providing a promising foundation for robot manipulation.
Abstract (translated)
URL
https://arxiv.org/abs/2611.09892
PDF
-
FlowMoP: Stochastic Multi-Person Motion Prediction
2026-11-06 18:56:41Aadya Agrawal, Ho Kei Cheng, Alexander SchwingAbstract
Predicting the future motion of multiple people is inherently stochastic: the same observed history admits many plausible continuations depending on intent, coordination, and social context. Yet the dominant paradigm in multi-person 3D motion prediction remains deterministic, with existing methods producing a fixed number of outputs per agent. In contrast, we present a conditional flow matching framework that jointly generates diverse future motions for multiple agents, modeling both global trajectories and root-relative poses. A B-spline reparameterization compresses the generative state space while enforcing temporal smoothness, and a dual-stream transformer conditions each agent's predictions on observed neighbor motion for interaction-aware forecasting. Our method achieves state-of-the-art performance on CMU-Mocap (UMPM) and 3DPW, outperforming deterministic baselines on joint position error, pose error, and final displacement error despite producing a full predictive distribution. We additionally establish the first motion prediction baseline on WorldPose, a large-scale professional soccer dataset, where social conditioning demonstrably concentrates predictive uncertainty along interaction-constrained directions.
Abstract (translated)
URL
https://arxiv.org/abs/2611.09882
PDF
-
ZeoBench: A Benchmark for Self-Supervised Learning on 3D Zeolite Representations
2026-11-06 18:55:34Aaron Sun, Yachan Liu, Ping Yang, Peng Bai, Subhransu MajiAbstract
Zeolites are an important class of porous crystalline materials with broad applications in gas storage, separations, and catalysis. Although hundreds of thousands of experimentally realized and hypothetical zeolite structures are known, identifying materials optimized for a target application remains challenging because property labels are expensive to obtain and available only for limited structure-molecule pairs. While prior work has demonstrated that 3D neural representations can predict zeolite properties effectively, it remains unclear which representation-learning strategies are most effective in the low-data regime relevant to materials discovery. We present ZeoBench, a benchmark for self-supervised learning on dense 3D representations of all-silica zeolites, and evaluate the resulting features on six existing downstream adsorption-property prediction tasks, across a range of training set sizes. Our study compares self-supervised 3D pretraining on volumetric zeolite data, adaptations of image-pretrained 2D ConvNets and vision transformers to 3D via multiview and channel-inflation strategies, and 3D ConvNets pretrained on out-of-domain volumetric datasets, alongside hand-crafted descriptors and supervised baselines. Using our pretraining and evaluation framework, we find self-supervised 3D representations outperform existing methods and achieve state-of-the-art performance across both in-domain label regimes and for the unseen domain of metal organic frameworks. Learned representations consistently outperform hand-crafted descriptors, and convolutional architectures are generally more effective than vision transformers in this setting. These results establish ZeoBench as a practical benchmark for 3D representation learning in zeolites and provide guidance for data-limited adsorption property prediction.
Abstract (translated)
URL
https://arxiv.org/abs/2611.09880
PDF
-
Graded convolutional networks: scale invariance without the scale and robustness to unknown blur
Abstract
The Gaussian jet of an image is a graded vector space: a dilation by $\lambda$ multiplies derivatives of order $k$ by $\lambda^{-k}$. Local graded convolutions are exactly homogeneous differential operators, and a local feature is dilation-invariant if and only if it factors through the scale-normalized jet or, without the scale, through the retraction onto the weighted sphere of the grading. Under unknown Gaussian blur the retraction changes only by a change of scale, whereas the scale-normalized jet acquires a grading factor that no radial normalization removes. At the zero jet the retraction separates image families that radial normalizations and continuous classifiers cannot. Networks built on the retraction are universal for invariant features. On MNIST such a network keeps 88.4% accuracy under a two-pixel blur, against 66.2% for the same network on scale-normalized derivatives; in Lindeberg's Gaussian derivative network the retraction raises accuracy at that blur from 91.5% to 95.1%.
Abstract (translated)
URL
https://arxiv.org/abs/2611.09878
PDF
