MoST: Motion Style Transformer between Diverse Action Contents

Boeun Kim1,2,3, Jungho Kim1, Hyung Jin Chang3, Jin Young Choi2,
1Korea Electronics Technology Institute, 2Seoul National University, 3University of Birmingham
Paper Video Code

MoST effectively transfers style between two motions, even when they have different action contents.

Abstract

While existing motion style transfer methods are effective between two motions with identical content, their performance significantly diminishes when transferring style between motions with different contents. This challenge lies in the lack of clear separation between content and style of a motion. To tackle this challenge, we propose a novel motion style transformer that effectively disentangles style from content and generates a plausible motion with transferred style from a source motion. Our distinctive approach to achieving the goal of disentanglement is twofold: (1) a new architecture for motion style transformer with 'partattentive style modulator across body parts' and 'Siamese encoders that encode style and content features separately'; (2) style disentanglement loss. Our method outperforms existing methods and demonstrates exceptionally high quality, particularly in motion pairs with different contents, without the need for heuristic post-processing. Furthermore, our method can generate stylized global translation, unmatched by any existing method.

Video

Framework

(a) Overall framework of MoST comprising Siamese motion encoders $\mathcal{E}$, motion generator $\mathcal{G}$, and part-attentive style modulator (PSM). PSM modulates style feature $S^S$ under the condition of both contents of content motion and style motion, $i.e.$, $C^C$ and $C^S$. $\mathcal{G}$ generates final output motion with content dynamics feature $Y^C$ and the modulated style feature $\tilde{S}^S$.

(b) Detailed operations in PSM.

Results of MoST

Samples transferring different styles. (Xia dataset)
Samples transferring different styles. (BFA dataset)

Comparisons

BibTeX

@misc{kim2024most,
      title={MoST: Motion Style Transformer between Diverse Action Contents}, 
      author={Boeun Kim and Jungho Kim and Hyung Jin Chang and Jin Young Choi},
      year={2024},
      eprint={2403.06225},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}