Abstract
Gait recognition, a fundamental biometric technology, leverages unique walking patterns for individual identification, typically using 2D representations such as silhouettes or skeletons. However, these methods often struggle with viewpoint variations, occlusions, and noise. Multi-modal approaches that incorporate 3D body shape information offer improved robustness but are computationally expensive, limiting their feasibility for real-time applications. To address these challenges, we introduce Mesh-Gait, a novel end-to-end multi-modal gait recognition framework that directly reconstructs 3D representations from 2D silhouettes, effectively combining the strengths of both modalities. Compared to existing methods, directly learning 3D features from 3D joints or meshes is complex and difficult to fuse with silhouette-based gait features. To overcome this, Mesh-Gait reconstructs 3D heatmaps as an intermediate representation, enabling the model to effectively capture 3D geometric information while maintaining simplicity and computational efficiency. During training, the intermediate 3D heatmaps are gradually reconstructed and become increasingly accurate under supervised learning, where the loss is calculated between the reconstructed 3D joints, virtual markers, and 3D meshes and their corresponding ground truth, ensuring precise spatial alignment and consistent 3D structure. Mesh-Gait extracts discriminative features from both silhouettes and reconstructed 3D heatmaps in a computationally efficient manner. This design enables the model to capture spatial and structural gait characteristics while avoiding the heavy overhead of direct 3D reconstruction from RGB videos, allowing the network to focus on motion dynamics rather than irrelevant visual details. Extensive experiments demonstrate that Mesh-Gait achieves state-of-the-art accuracy. The code will be released upon acceptance of the paper.
Abstract (translated)
步态识别是一种基本的生物识别技术，利用独特的行走模式进行个体身份识别，通常采用二维表示方法，如轮廓或骨架。然而，这些方法在面对视角变化、遮挡和噪声时往往表现不佳。多模态方法结合3D身体形状信息可以提高鲁棒性，但计算成本较高，限制了其在实时应用中的可行性。为了解决这些问题，我们提出了一种新颖的端到端多模态步态识别框架Mesh-Gait，该框架直接从2D轮廓重建三维表示，有效地融合了两种模式的优点。 与现有的方法相比，直接从3D关节或网格中学习3D特征既复杂又难以与基于轮廓的步态特征相融合。为了克服这一难题，Mesh-Gait 重构了3D热图作为中间表示形式，使模型能够有效捕捉三维几何信息，同时保持简洁性和计算效率。 在训练过程中，在监督学习下，通过计算重建的3D关节、虚拟标记和3D网格与其对应的地面真实值之间的损失来逐步重构中间的3D热图，并且这些热图会变得越来越准确。这确保了精确的空间对齐和一致的三维结构。 Mesh-Gait 能够以计算高效的方式从轮廓和重建的3D热图中提取判别性特征，这种设计使模型能够捕捉到步态的空间和结构性质，同时避免直接从RGB视频中进行复杂的三维重构带来的沉重开销。这样网络可以专注于运动动态而非无关视觉细节。 广泛的实验表明，Mesh-Gait 达到了最先进的精度水平。论文一经接受，代码将公开发布。
URL
https://arxiv.org/abs/2510.10406
