Abstract
Panoramic multi-object tracking is important for industrial safety monitoring, wide-area robotic perception, and infrastructure-light deployment in large workspaces. In these settings, the sensing system must provide full-surround coverage, metric geometric cues, and stable target association under wide field-of-view distortion and occlusion. Existing image-plane trackers are tightly coupled to the camera projection and become unreliable in panoramic imagery, while conventional Euclidean 3D formulations introduce redundant directional parameters and do not naturally unify angular, scale, and depth estimation. In this paper, we present $\mathbf{S^3KF}$, a panoramic 3D multi-object tracking framework built on a motorized rotating LiDAR and a quad-fisheye camera rig. The key idea is a geometry-consistent state representation on the unit sphere $\mathbb{S}^2$, where object bearing is modeled by a two-degree-of-freedom tangent-plane parameterization and jointly estimated with box scale and depth dynamics. Based on this state, we derive an extended spherical Kalman filtering pipeline that fuses panoramic camera detections with LiDAR depth observations for multimodal tracking. We further establish a map-based ground-truth generation pipeline using wearable localization devices registered to a shared global LiDAR map, enabling quantitative evaluation without motion-capture infrastructure. Experiments on self-collected real-world sequences show decimeter-level planar tracking accuracy, improved identity continuity over a 2D panoramic baseline in dynamic scenes, and real-time onboard operation on a Jetson AGX Orin platform. These results indicate that the proposed framework is a practical solution for panoramic perception and industrial-scale multi-object this http URL project page can be found at this https URL.
Abstract (translated)
全景多目标跟踪对于工业安全监控、广域机器人感知以及大型工作空间中的轻量化基础设施部署具有重要意义。在此类场景中，感知系统必须提供全向覆盖、度量几何线索，并在大视场畸变与遮挡条件下保持稳定的目标关联。现有图像平面跟踪器与相机投影紧密耦合，在全景图像中可靠性不足；而传统的欧几里得三维表述会引入冗余的方向参数，且无法自然统一角度、尺度与深度估计。本文提出 $\mathbf{S^3KF}$，一种基于电机驱动旋转激光雷达与四鱼眼相机阵列的全景三维多目标跟踪框架。核心思想是在单位球面 $\mathbb{S}^2$ 上建立几何一致的状态表示，其中目标方位通过两自由度切平面参数化建模，并与边界框尺度及深度动态联合估计。基于该状态表示，我们推导出扩展球面卡尔曼滤波流程，实现全景相机检测与激光雷达深度观测的多模态融合跟踪。 further 建立基于地图的真值生成流程，通过佩戴式定位设备与共享全局激光雷达地图配准，无需动作捕捉设施即可实现定量评估。在自采集真实场景序列上的实验表明，该方法达到分米级平面跟踪精度，在动态场景中相比二维全景基线提升了身份连续性，并可在Jetson AGX Orin平台实现实时机载运行。这些结果证明，所提框架为全景感知与工业级多目标跟踪提供了实用解决方案。项目页面可通过此 http URL 访问，详情见此 https URL。
URL
https://arxiv.org/abs/2603.27534
