Abstract
In this paper, we propose H-MoRe, a novel pipeline for learning precise human-centric motion representation. Our approach dynamically preserves relevant human motion while filtering out background movement. Notably, unlike previous methods relying on fully supervised learning from synthetic data, H-MoRe learns directly from real-world scenarios in a self-supervised manner, incorporating both human pose and body shape information. Inspired by kinematics, H-MoRe represents absolute and relative movements of each body point in a matrix format that captures nuanced motion details, termed world-local flows. H-MoRe offers refined insights into human motion, which can be integrated seamlessly into various action-related applications. Experimental results demonstrate that H-MoRe brings substantial improvements across various downstream tasks, including gait recognition(CL@R1: +16.01%), action recognition(Acc@1: +8.92%), and video generation(FVD: -67.07%). Additionally, H-MoRe exhibits high inference efficiency (34 fps), making it suitable for most real-time scenarios. Models and code will be released upon publication.
Abstract (translated)
在这篇论文中，我们提出了一种新颖的管道系统H-MoRe（Human-centric Motion Representation），用于学习精确的人体运动表示。我们的方法能够动态地保留相关的人类动作同时过滤掉背景移动。值得注意的是，与之前依赖于合成数据进行完全监督学习的方法不同，H-MoRe直接从现实场景中以自监督方式学习，结合了人体姿态和体型信息。 受动力学的启发，H-MoRe 采用矩阵格式来表示每个身体部位的绝对运动和相对运动，并捕捉细微的动作细节，称为世界局部流（world-local flows）。H-MoRe 提供了对人类动作的深入见解，可以无缝地集成到各种与行动相关的应用中。实验结果表明，H-MoRe 在多种下游任务中带来了显著改进，包括步态识别(CL@R1: +16.01%)、行为识别(Acc@1: +8.92%) 和视频生成(FVD: -67.07%)。此外，H-MoRe 还表现出高推断效率（34 fps），适合大多数实时场景应用。 论文发布后将公开模型和代码。
URL
https://arxiv.org/abs/2504.10676
