ACM Multimedia 2026

GenPrior: Unleashing Text-to-Motion Generative Priors
for Zero-Shot Skeleton-based Action Recognition

Jidong Kuang Hongsong Wang* Jie Gui*

Southeast University

*Corresponding authors

Paper arXiv ACM MM 2026 Code
TL;DR

GenPrior distills kinematic structure from generated motions, adaptively injects it into textual semantics, and refines unseen-class prototypes.

Abstract

Zero-shot skeleton-based action recognition (ZSAR) aims to recognize unseen action categories by aligning skeleton features with textual semantics. However, existing methods rely on text-derived prototypes that inherently lack geometric structure and physical constraints, resulting in a pronounced semantic-kinematic gap. To bridge this gap, we propose GenPrior, the first framework to exploit generative priors from pre-trained Text-to-Motion models for ZSAR. Specifically, we introduce dispersion-gated feature fusion, which distills kinematic prototypes and intra-class dispersion from generative motion sequences and employs a learned gating network to adaptively inject reliable structural cues into textual embeddings while suppressing synthetic artifacts. Furthermore, we propose generative prototype refinement, which leverages generation-enhanced prototypes as anchors to mine high-confidence unseen samples, calibrating class prototypes toward the true distribution. Extensive experiments on NTU-60, NTU-120, and PKU-MMD demonstrate state-of-the-art performance under both zero-shot and generalized zero-shot settings.

Comparison of prior text-only zero-shot recognition and GenPrior with generative motion priors.
Motivation. Text-derived prototypes lack physical constraints and can deviate from real skeleton features. GenPrior enriches them with structurally grounded generative motion priors.

Framework

The GenPrior framework, covering motion generation priors, adaptive semantic fusion, latent alignment, and generative prototype refinement.

Experiments

GenPrior is evaluated on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD under fixed-split ZSL, random-split ZSL, and generalized ZSL protocols.

Fixed-split zero-shot recognition

Accuracy (%). Gains are against the strongest inductive baseline.

Fixed-split zero-shot recognition results on NTU-60 and NTU-120.
Fixed-split ZSL. GenPrior reaches 73.43% on NTU-60 48 / 12 and 90.44% / 81.39% on NTU-120, with its margin increasing on the more challenging unseen-class splits.

Random-split zero-shot recognition

Mean accuracy (%) over three random splits.

Random-split zero-shot recognition results on NTU-60, NTU-120, and PKU-MMD.
Random-split ZSL. Averaged over three splits, GenPrior improves the strongest prior results by 6.4, 13.5, and 3.1 points.

Generalized zero-shot recognition

Seen accuracy (S), unseen accuracy (U), and harmonic mean (H), all in %.

Generalized zero-shot recognition results on NTU-60 and NTU-120.
Generalized ZSL. GenPrior improves unseen accuracy and establishes strong harmonic means across the four evaluation splits.

Visual Results

Text-to-motion models provide kinematically grounded cues for action classes that have no labeled skeleton samples. The examples below are generated motions used as priors, not real NTU samples.

Six uniformly sampled skeleton frames for ten text-to-motion generated action sequences.
Generated motion sequences. Six frames are sampled from each of ten representative action sequences. Color indicates temporal progression, revealing physically plausible poses and coherent trajectories.
Kicking something Full generated sequence

BibTeX

If you find GenPrior useful, please cite our work.

@article{kuang2026genprior,
  title={GenPrior: Unleashing Text-to-Motion Generative Priors for Zero-Shot Skeleton-based Action Recognition},
  author={Kuang, Jidong and Wang, Hongsong and Gui, Jie},
  journal={arXiv preprint arXiv:2608.02236},
  year={2026}
}