Tracking
-
AdaDexTrack: Dynamic Modulation for Adaptive and Generalizable Dexterous Manipulation Tracking
2026-10-24 18:46:10Jianibieke Adalibieke, Qianwei Han, Xueyi Liu, Yuzhe Qin, Li YiAbstract
Language is a natural way to command robots, but converting a single instruction into a long-horizon, contact-rich hand-object interaction remains challenging: synthesized references are noisy, human-to-robot retargeting introduces embodiment bias, and fixed-reference tracking lets small errors snowball. We address this with AdaDexTrack, a modulator-in-the-loop framework for language-conditioned manipulation tracking. A distilled generalist tracker serves as the skill carrier, while a tightly aligned modulator performs three feedback corrections: reference modulation (continual adjustment of what to track), object-latent modulation (online adaptation of the object representation to recruit suitable skills), and positional-target modulation (small state-dependent refinements for execution). The tracker is learned via large-scale specialist to generalist distillation on a corpus of language-conditioned hand-object trajectories; the modulator is trained with RL under the same task objective, ensuring tight coupling. Across large-scale evaluations, AdaDexTrack consistently outperforms prior SOTA on unseen-trajectory and unseen-object sets in both average tracking error and success rate, demonstrating robustness and generalization. We further show zero-shot sim-to-real transfer on real hardware, where adding the modulator yields substantial gains over a tracker-only variant. AdaDexTrack reframes language-conditioned dexterous manipulation as modulated tracking, replacing the open-loop, fixed-reference tracking with in-loop modulation that adjusts the reference, object latent, and positional target, yielding drift-resistant execution from noisy text references.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33852
PDF
-
PACE: Research Proving Agent for Adaptive Conjecture Exploration
2026-10-24 14:13:06Shu Miao, Feiming Wang, Wentao Long, Chenyi Li, Kun Yuan, Zaiwen WenAbstract
Many mathematical research agents focus on solving problems with fixed statements. However, conjecture research, especially in applied mathematics, may require revising candidate questions or conclusions as investigation proceeds, coupling problem formulation with proof development. We present PACE, a research-proof agent that supports this process through a structured research loop that selects mathematical objectives and develops or challenges candidate mechanisms. The whole prover constructs proofs from these mechanisms, and an independent verifier checks them against the original target. Evidence-tracking memory stores assumptions and dependencies arising during research and marks evidence for revalidation when a candidate or condition changes. This memory supports task-specific context composition that keeps checked premises, exploratory material and unresolved obligations distinct. To evaluate proof construction and conjecture research, we introduce ResearchProofBench, an applied-mathematics benchmark focused on optimization, comprising 51 known-proof tasks with 331 hidden checkpoints under historical abstract access and six open-problem cases. The known-proof tasks assess reconstruction of published results, while the open-problem cases assess mathematical investigation and proof development. On the 51 known-proof tasks, PACE achieves an aggregate Proof Score of 75.16 and completes 20 proofs. In conjecture research, PACE develops conditional results, with its submissions receiving higher review scores than the compared baselines.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33690
PDF
-
Energy-Based Fault Detection and Isolation Framework for Flat Port-Hamiltonian Mechanical Systems
2026-10-24 13:50:17Johnny Garcia, Nicolas Salguero, Naveed Iqbal, Brahim Brahmi, Ramy RashadAbstract
Fault diagnosis during regulated motion requires informative signals even when feedback attenuates tracking deviations. This paper develops an energy-based diagnostic framework that constructs mechanical port-power residuals without a dedicated dynamic observer. Spectral Hamiltonian Algebraic Power Evaluation (SHAPE) combines port-Hamiltonian variables and differential flatness to generate nominal power references from planned motion for fully actuated mechanical systems with collocated actuation. Subtracting these references from commanded effort multiplied by measured flow yields residuals whose final evaluation requires neither recursive observer updates nor measured-state dynamics calculations. Continuous wavelet transform (CWT) scalograms and a shared-encoder convolutional neural network are used to distinguish nominal operation, actuator loss of effectiveness, anomalous friction, and vibration from complete trajectories. Evaluation uses a simulated six degree of freedom Kinova manipulator under model predictive control and fixed plant-model mismatch, with single faults at a specified joint. Across five training seeds and 3000 held-out test runs, SHAPE-CWT achieves a mean accuracy of 84.69 percent, versus 81.67 percent for torque residuals and 97.17 percent for the generalized momentum observer under common CWT processing. The respective sample standard deviations are 0.70, 0.70, and 0.37 percentage points. These results demonstrate useful fault-type discrimination from algebraic power residuals, while severity-excitation profiles and offline velocity perturbations reveal limits to their diagnostic capability.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33679
PDF
-
Tracking Is Not Sensing: Artifact-Aware Thermal Video Respiration Estimation in Head-Free Marmosets
2026-10-24 13:18:58Penghao Wang, Sen Yang, Fenghua Xie, He Wang, Xindong SongAbstract
Continuous physiological monitoring is essential for linking behavior, internal state, and neural activity in neuroscience and translational research. In common marmosets, a key nonhuman primate model for social neuroscience, respiratory rate provides a basic readout of arousal, stress, and physiological regulation. Contact sensors can provide accurate measurements, but they add preparation burden and may interfere with head-free behavioral protocols. Thermal video offers a noncontact alternative by capturing breathing-related temperature changes around the nose and mouth. However, in head-free marmosets, these cues are weak, localized, and easily corrupted by head motion, pose changes, and fur. Here we show that this setting exposes a broader challenge in video-based physiological sensing: visually reliable tracking does not necessarily imply physiologically valid measurement. Using synchronized thermal video and contact respiration recordings, we show that marmoset respiration is recoverable from thermal video but strongly ROI-sensitive. Under head-free motion, visually plausible nasal tracking can still produce traces inconsistent with contact respiration, demonstrating that tracking is not sensing. We therefore estimate respiratory rate from learned temporal signals extracted from thermal clips and introduce Selective Temporal Attenuation Regularization (STAR). STAR selectively attenuates high-motion thermal discontinuities through local temporal imputation during training, regularizing models toward respiration-consistent dynamics while leaving inference unchanged on raw video. Across multiple deep-learning backbones, STAR improves head-free estimation, reaching 2.23 bpm MAE compared with 20.76 bpm for tracked ROI analysis.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33661
PDF
-
HARP: Handoff-Aware Recovery Planning for Active Target Acquisition in Multi-UAV Handoff
2026-10-24 12:47:38Heegyeong Kim, Alice James, Avishkar Seth, Endrowednes Kuantama, Richard HanAbstract
Continuous aerial tracking often requires multiple drones to maintain target coverage over long periods. During handoff, the receiving UAV may lack prior target observations and must actively acquire a usable view without disrupting the sender's ongoing coverage. This work introduces HARP, a handoff-aware recovery planner that registers bounded sender predictions in the receiver's local map, distinguishes field-of-view failure from physical occlusion, and generates failure-specific yaw, translation, and parallax viewpoints. Candidate motions are checked for receiver clearance, sender-fiducial observability, protected sender--target visibility, temporal validity, and deadline compliance. When a complete trajectory cannot be admitted, HARP may execute a sampled admissible nonzero prefix and replan from updated evidence. In simulation, HARP acquired the target in all 49 standard trials and in 15 of 16 stress trials. Field trials validated HARP's recovery geometry under field-of-view limitation and environmental occlusions.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33640
PDF
-
HA-DiPO: Diffusion Policy Optimization for Heterogeneous Cooperative Transport
2026-10-24 10:24:37Xuebin Li, Jinpeng Xiao, Zong Chen, Pierluigi Arpenti, Bruno Siciliano, Jiebao Sun, Yiqun LiAbstract
Cooperative transport with heterogeneous legged robots requires coordinating different motion capabilities while supporting an unfixed shared payload. We present HA-DiPO (Heterogeneous-Agent Diffusion Policy Optimization), a hierarchical multi-agent reinforcement learning framework under centralized training with decentralized execution (CTDE). The framework decouples team-level coordination from embodiment-specific motion control: at the high level, a centralized critic uses team-value estimates to jointly train independent coordination policies, aligning robot-specific action responses with a shared transport objective; at the low level, embodiment-specific robust motion policies track the coordination commands. The high-level diffusion policies offer more flexible action distributions than single-Gaussian parameterizations, allowing potentially multimodal coordination responses to be represented. The coordination policies learn from scratch through reward-driven simulator interaction without demonstrations. At execution, each robot generates coordination commands using only local robot and payload observations, without partner-state inputs or inter-robot messages. Simulation comparisons show that HA-DiPO achieves higher final mean returns and velocity-tracking success rates, as well as lower velocity errors, than the evaluated baselines. Real-world experiments further demonstrate quadrupedal-wheeled-legged and human-robot cooperative transport.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33584
PDF
-
Cross-View and Cross-Time Tracking of Crack Evolution in Concrete
2026-10-24 09:06:04Pengru Deng, Jiapeng Yao, Su Wang, Xuhui He, Zhao Wang, Takanori Emaru, Takashi Matsumoto, Xinrun LiAbstract
Crack propagation provides richer information than static crack observations for evaluating structural conditions of concrete infrastructure and planning maintenance strategies. Unlike static crack detection, which can now be largely automated using computer vision, vision-based crack tracking is a coupled cross-view and cross-time problem: the same physical crack must retain its identity as camera viewpoints change, while newly developed growth must be distinguished from existing crack segments in a unified reference view. Existing segmentation and 3D reconstruction methods can recover crack masks or scene geometry, but they do not preserve crack identity across views or quantify temporal extension. To address this gap, this study formulates crack growth monitoring as a cross-view and cross-time crack-instance tracking task and proposes a geometry-assisted, morphology-aware framework. The framework first uses image-based geometric registration to align observations from different viewpoints into a shared structural reference, then applies morphology-aware instantiation to assign physically interpretable crack identities, and finally performs temporal mapping in the unified view to distinguish newly developed growth from existing crack segments. To enable evaluation, a temporally annotated dataset was established and used, covering three representative cracking scenarios: single dominant crack propagation, mixed-mode branching, and multiple-crack interaction, with pixel-level masks and cross-time instance labels. Experimental results show that the proposed framework can establish loose cross-view identity correspondence under changing viewpoints, while strict spatial overlap and growth localization remain limited by early micro-crack segmentability and projection drift, highlighting directions for future crack-monitoring research.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33550
PDF
-
MimicAll: Structured Embodiment Conditioning for Cross-Humanoid Whole-Body Motion Tracking
Abstract
Humanoid motion trackers reproduce diverse behaviors, but sharing a policy across bodies requires accounting for differences in structure, morphology and dynamics. We present MimicAll, a reinforcement-learning framework for shared whole-body motion tracking across heterogeneous humanoids. A structured embodiment descriptor is encoded into global and joint-aligned conditions, which are combined with observation history and body-matched motion references to predict joint-target offsets. Physics-consistent embodiment randomization, body-calibrated retargeting and heterogeneous simulation support training across 4,096 variants from 16 humanoid families. We introduce a benchmark of 464 bodies across five distribution strata to evaluate motion generalization, within-family embodiment shifts and unseen-family transfer. Controlled ablations show that embodiment information improves motion completion with simple concatenation, while self-attention yields the lowest global position errors. With equal training transitions, shared training achieves 99.4% 30-second success on randomized G1 bodies in 100STYLE, versus 42.9% for a single-body specialist. With Flat-MLP encoding and concatenation, continued training on a motion subset excluding obstacle categories achieves 98.3% and 93.5% 30-second success on 100STYLE for sampled training bodies and out-of-distribution variants within seen families, respectively. On an eight-motion LAFAN1 subset excluding ground and get-up sequences across 464 bodies, mismatched body conditions reduce self-attention's 30-second success from 61.7% to 47.0%, demonstrating a closed-loop benefit from embodiment information.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33508
PDF
-
Explicit Frictional Contact Modeling in Neural Garment Simulation
2026-10-24 06:41:28Thanh-Nguyen Truong, Vu-Hoang Tran, Ching-Chun HuangAbstract
Graph neural networks (GNNs) simulate garments efficiently through physics-based self-supervision, but leave body-garment friction implicit or inside training energies, with no friction coefficient $\mu$ to set at inference; retraining the tested GNNs with a coefficient-specific friction term barely responds to $\mu$. Optimization-based solvers decide each contact's stick-slip state from the reaction that governs it, but only by re-optimizing the garment state at every step; a friction term in a one-pass training objective acts as a fixed-weight penalty on sliding, and the trained weights apply no per-contact Coulomb condition at inference. We therefore resolve friction explicitly after prediction: a Modular Contact Operator, attached to a frozen predictor without retraining or an iterative solve, estimates the normal load, applies a closed-form Coulomb correction along a path across body triangles, and propagates the induced motion through the cloth as a linearized membrane wave. On HOOD and ContourCraft, it raises sticking and lowers slip monotonically in $\mu$, tracking a reference solver's response while coefficient-specific retraining stays flat, with low penetration and reference-consistent stretching at an eighth of the solver's runtime.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33492
PDF
-
Fast Instinct, Slow Reason: A Dual-Speed Agentic Framework for Self-Configuring Multi-Object Tracking in Satellite Video
2026-10-24 02:06:09Fan He, Baining Wu, Yan Li, Xiaoyang ZengAbstract
Multi-object tracking in satellite video underpins persistent monitoring, yet offline-tuned trackers lack the onboard intelligence and efficiency to adapt within an observation of tens of seconds. The root is that adaptation still means relabelling and retraining the detector, which cannot run on board in time. Tuning the tracker configuration strongly controls accuracy without touching the network, yet search needs labelled or proxy evaluations, and a language agent costs seconds and tokens per decision. We propose a dual-speed agentic framework pairing a fast instinct compiled offline with a slow reason serving the mission. Its algorithm, Arbitration of Unlabelled Shortlists from Prior Experience, or AUSPEX, learns a covering shortlist and a label-free ranker from labelled source tasks and commits one configuration per sequence. Its system, a Cascaded Onboard Runtime with Verified Uplinked Skills, or CORVUS, cascades reflex, skill, and agent layers, so that a verified skill commits 2.5 s into each observation while a language agent serves natural- language missions. On SatMTB and SatVideoDT with three recent detectors, AUSPEX raises IDF1 by 2.84 to 2.94 points and MOTA by 3.21 to 3.29 points on average without target labels, balances accuracy and labels best among sixteen methods, and loses at most 0.28 IDF1 points at 1.25 tracking passes. CORVUS executes four missions with execution accuracy up to 0.99 and target recall at the perception ceiling, spends no token on the routine decision, runs on Jetson Xavier NX, and hosts fifteen language models from six families without altering any commitment, demonstrating its extensibility and generalization.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33396
PDF
-
Tight Learned Event-Inertial Odometry : a Transformer-Based Architecture for End-to-End Motion from Event Streams
2026-10-23 22:54:00Matteo Cascone, David Jensen, Alessandro Pirini, Angelo Stefanini, Pietro Ven\`e, Marta Tintor\'e Gazulla, Javier Hidalgo Carri\`oAbstract
Event cameras provide microsecond temporal resolution, high dynamic range, and sparse asynchronous measurements, making them compelling for odometry under fast motion and challenging illumination. Yet, accurate event-based state estimation remains difficult: existing systems often use events as a complement to frames or rely on feature tracking, correspondence search, and optimization-heavy backends, limiting the efficiency promised by event sensing. We introduce Tight Learned Event-Inertial Odometry (TLEIO), a compact filter-based pipeline that learns short-window relative motion directly from voxelized event streams and tightly fuses the resulting constraints with IMU propagation. Its learned front-end, EventsFormer, predicts local displacements and per-axis uncertainty from event voxels in a single forward pass, while a stochastic-cloning EKF fuses these overlapping predictions with IMU propagation, producing metric trajectories without explicit feature tracks and pose optimization at inference time. The network is trained directly from pose displacements, without requiring event-flow labels. We evaluate TLEIO on the TartanAir/TartanEvent benchmark and on real DAVIS240c sequences. On TartanAir, TLEIO reduces average ATE from 2.26 m to 1.10 m over the IMU-only TLIO baseline, while achieving up to 48.9% lower runtime than optimization-based event odometry methods.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33356
PDF
-
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development
2026-10-23 22:51:29Aditya Sivakumar, Peter Ciaccia, Hassaan Qadir, Ian Hsu, Ashu SinghalAbstract
Molecular cloning is one of the most routine wet-lab workflows, yet existing evaluations of language models in biology mostly test knowledge recall or isolated steps rather than end-to-end experimental planning. We introduce BenchBench-Cloning, a benchmark for large language models comprising 65 molecular cloning tasks grounded in sequence records, laboratory notebooks, and published experimental studies. Planning a molecular cloning experiment requires defining a set of constructs needed for a scientific objective and a construction route to produce them. BenchBench-Cloning contains 50 protocol-development tasks, in which agents produce an executable protocol for a specified construct, and 15 construct-design tasks, in which agents determine the constructs a scientific objective calls for. Protocol-development tasks are evaluated by execution in a cloning simulator; construct-design tasks are evaluated with task-specific deterministic checks and, where needed, rubric-based LLM-as-a-judge evaluation. Across eight systems with repeated attempts per task, GPT-6 Astra scores highest at 30.6%, with the others between 8.6% and 22.2%, and the benchmark remains unsaturated under best-of-ten sampling. As integrating individual cloning decisions into complete, valid plans remains challenging even for frontier models, we expect BenchBench-Cloning to serve as a useful benchmark for tracking progress in experimental planning.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33355
PDF
-
Estimation of Propagated Risk in LLM-based Multi-Agent Systems
2026-10-23 20:46:33Shreyan Ganguly, Souradeep Chattopadhyay, Qisai Liu, Soumik SarkarAbstract
Large Language Model (LLM) based multi-agent systems introduce a distinctive reliability challenge in which actions taken by upstream agents can affect the correctness of downstream agents. While prior work has investigated error propagation and developed methods for estimating uncertainty inherited from upstream agents, existing approaches do not explicitly model how realized sender-receiver interactions contribute to downstream error risk under different sender correctness states. In this work, we study agent-level error-risk estimation from multi-agent execution traces. We propose Interaction-Effect Propagation (IE-Prop), an interaction-conditioned model that learns separate effects for correct and incorrect upstream agents. For each sender-receiver interaction, the model combines these conditional effects using the sender's estimated error risk, and recursively propagates the resulting risk estimates through the execution graph. We evaluate our approach across three multi-agent architectures and three benchmarks. Across all evaluated architectures, our approach raises the final-node AUROC over the established baseline by approximately 0.05 points on average. Furthermore, tracking risk across multiple agent steps contributes a distinct +0.03 AUROC gain in sequential systems. These results show that realized inter-agent interactions provide predictive information beyond receiver-local evidence, while the value of multi-step risk tracking depends on the system's information-flow structure.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33282
PDF
-
Bracketing Shows the Way: Adaptive Multi-Exposure Policy as Input for Robust Visual SLAM
2026-10-23 18:16:24Olivier Gamache, Anthony Lavertu, David-Alexandre Duclos, Jean-Michel Fortin, Fran\c{c}ois Pomerleau, Philippe Gigu\`ereAbstract
Rapid illumination transitions can disturb Visual SLAM (VSLAM) by saturating images before the Automatic Exposure (AE) policy adapts, removing the visual correspondences needed to maintain tracking. Our proposed methods mitigate this disturbance at two levels: B-Adapt acquires adaptive exposure brackets to cover the scene dynamic range while promoting photometric overlap, and BVSLAM directly uses the multi-exposure sequence as constraints in a factor graph state estimator without relying on High Dynamic Range (HDR) merging. We evaluate both methods against state-of-the-art approaches in nominal environments and during a repeated tunnel-exit transition at an emulated vehicle speed of 32 m s-1. In the tunnel, combining our B-Adapt with BVSLAM reduces the state estimation error by 78 % compared with AE images. These results suggest that bracketing efficiently improves visual pose estimation robustness to rapid illumination disturbances while retaining low error under nominal conditions. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33174
PDF
-
ViT-H: Generative Video-To-Humanoid Control with Motion Tracking Feedback
2026-10-23 17:54:09Zimo He, Tianyi Bian, Zizhong Yang, Hongjie Li, Nan Jiang, Jieming Cui, Baoxiong Jia, Yizhou Wang, Siyuan Huang, Yixin ZhuAbstract
Learning humanoid control from monocular human videos offers a scalable way to acquire diverse whole-body behaviors. Yet, existing reconstruction-retargeting-tracking pipelines are limited in system-level scalability by separately optimized intermediate stages and cannot directly optimize reference motion executability using downstream tracking feedback. We present ViT-H, a video-to-humanoid control framework following a generation-and-tracking paradigm. ViT-H directly generates humanoid reference motions from videos with a diffusion model, bypassing human motion reconstruction and retargeting, and uses motion-tracking outcomes to post-train the generator with direct preference optimization. We further introduce an autoregressive variant that enables real-time humanoid video teleoperation with sub-400 ms latency. Extensive experiments on the Unitree G1 demonstrate that ViT-H outperforms state-of-the-art reconstruction-retargeting-tracking pipelines in both motion quality and executability. We further show that ViT-H scales favorably with model size and with both pre-training and post-training data scale, while motion-tracking feedback consistently improves executability across different motion trackers.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32966
PDF
-
Sounding Surgical Instrument Localisation through Audio-Visual Correspondence
2026-10-23 15:51:48Dominic Behling, Jonas Hein, Matthias SeiboldAbstract
Surgical scene understanding is a prerequisite for context-aware and intelligent systems in the operating room. Current methods rely mostly on visual data, while acoustic sensing has shown potential to provide complementary information on surgical activity. First approaches for the localisation of surgical activity relied on microphone arrays and acoustic beamforming. This work investigates whether the active instrument can be localized from a single microphone and a camera by learning the correspondence between the appearance and the sound of surgical instruments, without annotating its position. We adapt weakly supervised audio-visual localisation, which learns from the co-occurrence of image and sound, to the surgical setting. We record simulated surgical actions, sawing, drilling, and chiseling, and train the model on these recordings without any spatial annotation to produce a heatmap over the sounding tool. We extend the training objective by two terms: a silence term, which suppresses the output for silence and background noise, and a composite term, which trains the model to select the sounding one of two visible instruments. The model localizes the active instrument on an independent corpus recorded with a different camera and microphone, in which instrument positions were obtained by optical tracking. It separates tool activity from silence and background noise and can attribute sound correctly when two tools are present. Audio-visual correspondence learned from in-domain recordings connects the sound of a surgical action to the instrument that produces it, using a single microphone and a camera. This proof of concept introduces a novel paradigm that enables the detection and localisation of surgical activity without manual annotations.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32816
PDF
-
Investigating the trackability of landmarks in NBI and WLI
2026-10-23 11:29:12Jonas Below, Piermarco Gerli, Anna J\"otten, Dennis Eggert, Axelle Felicio-Briegel, Veronika Volgger, Christian Betz, Alexander SchlaeferAbstract
Reliable visual tracking of anatomical landmarks in endoscopic video is a key prerequisite for computer-assisted interventions in laryngeal surgery. While Narrow Band Imaging (NBI) is clinically established for enhancing vascular contrast, its effect on automated tracking relative to conventional White Light Imaging (WLI) remains unexplored. We present a quantitative comparison of landmark trackability in NBI and WLI using a forward-backward error framework applied to 61 endoscopic videos from 39 patients. Two regions of interest are evaluated: the vocal folds (VF) and mucosa with a prominent vascular pattern (MPVP). We compare a classical feature-based tracker (CSRT) against learning-based point trackers (CoTracker3-Online and TrackOnR) across both modalities. NBI consistently reduced the forward-backward error in both regions, reaching statistical significance for the vocal folds, while failure rates stayed low. These findings indicate that NBI can improve tracking robustness in laryngeal endoscopy, with implications for motion analysis, navigation, and robotic assistance.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33125
PDF
-
Delay Randomization Widens the Delay Margin of Learned Humanoid Motion Trackers
Abstract
Learned humanoid controllers are usually trained in one simulator and then checked in a second one before they go near hardware. We ask what that check says about command delay. We train motion-tracking policies for a 24-joint humanoid on five motions taken from monocular video and test them in an independent CPU MuJoCo harness, which we first verify against the training model. Every policy completes its motion in at least 26 of 30 episodes under the training randomization. With 40 ms of constant command delay, however, survival drops to 15% on average (0% - 45%), and the delay margin is only 20 - 30 ms. Randomizing the delay during training (0 - 50 ms) lifts survival at 40 ms to 95% on average (75% - 100%) and moves the margin to 50 - 70 ms. A control arm trained for the same number of extra iterations without delay does not improve, so the gain comes from the randomization itself. This holds for all five motions with three training seeds each, and a policy trained in PhysX behaves the same way. The price is slightly looser tracking (0.02 - 0.06 rad on four motions) and, for the most dynamic motion, less robustness to other perturbations. We also show that the first sim-to-sim gap we saw came from the harness, a missing floor and a wrong timestep, and not from the physics engines. The 0 - 20 ms delay ranges common in locomotion training lie inside the margin an unrandomized tracker already has. The benefit comes from randomizing beyond it, so the range should be set from the robot's measured latency, and delay margin belongs next to other sim-to-sim metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32277
PDF
-
SyncPro: Synchronous Reasoning and Responding for Proactive Streaming Video Understanding
2026-10-23 03:58:17Xueyun Tian, Bingbing Xu, Jiaming Liu, Jinpeng Yu, Yang Shi, Yu Zhao, Ruihua Huang, Yuanzhuo Wang, Huawei ShenAbstract
Proactive streaming video understanding requires assistants to track unfolding events and respond when a user's requested condition is met. To make these decisions, assistants must accumulate relevant evidence and reassess response conditions as new observations arrive. Existing work offers complementary capabilities for this process: proactive video assistants learn when to respond, while streaming video reasoning methods generate scheduled intermediate text to organize observations and support subsequent answers. Together, these capabilities provide a foundation for proactive reasoning, but how to coordinate ongoing reasoning with timely responses remains underexplored. In a direct adaptation, fixed reasoning intervals can trigger updates when nothing relevant has changed or postpone useful updates until the next scheduled step. Sharing one generation path also couples two distinct requirements: preserving evidence for future decisions and producing only the response requested by the user. Moreover, responses generated after reasoning must wait for it to finish, even when sufficient evidence is already available. To address these challenges, we introduce SyncPro, a synchronous dual-stream framework with specialized generation paths for evidence tracking and user-facing responses. Its Inner and Response branches share causal history and activate independently, allowing relevant evidence changes to trigger internal updates and responses to proceed concurrently with ongoing reasoning. We construct 189K dual-stream examples and train SyncPro through supervised fine-tuning followed by trajectory-level reinforcement learning. Experiments across three proactive benchmarks show improved response timing and more accurate answers to user requests.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32132
PDF
-
DriveMind: Mind-Evolving Belief Tracking for Closed-Loop Autonomous Driving
2026-10-23 02:13:50Yan Shi, Kehua Chen, Jiarui Qi, Yihong Tang, Haichen Liao, Yinhai WangAbstract
Recent multimodal large language models (MLLMs) can perceive a traffic scene, reason about it, and propose a driving action within a single model. However, existing MLLM-based drivers reason from scratch at each frame and commit to a single explanation, discarding earlier hypotheses about the scene. When the scene is ambiguous, the agent locks in on the wrong interpretation and acts on it before evidence accumulates. In this paper, we propose DriveMind, an MLLM-based driving framework that keeps several hypotheses about the scene alive at the same time and updates their weights as new frames arrive. Inspired by particle filtering, DriveMind propagates each hypothesis through a VLM, scores it against the current observation, and resamples to maintain diversity. On Bench2Drive, adding belief tracking to the same driving pipeline improves the Driving Score (DS) from 68.50 to 81.37 and the Success Rate (SR) from 41.36% to 54.09%. The largest gains occur on ambiguous scenarios such as unsignalized intersections (+20.0 DS) and oncoming-traffic interactions (+20.5 DS). Beyond the overall metrics, DriveMind also improves fine-grained interactive driving abilities such as merging, emergency braking, and give-way.
Abstract (translated)
URL
https://arxiv.org/abs/2610.32069
PDF
