SIGGRAPH Posters 2026 · Selected for Technical Papers Sessions
Poster Session: Animation & Simulation · Technical Paper Session: Humans & Hands — Reconstruction

SeatSwap: Learning-Assisted Differentiable Kinematics for Human-Object Interaction Adaptation in Production-Scale Character Animation

Jasmine Liang , Stephen Mylabathula, Shai Segev, Priyamvad Ravindra Deshmukh, Ying Liu, and Amaury Aubel

Meta Reality Labs

🔗 DOI 📅 SIGGRAPH Schedule
⌄
SeatSwap teaser — adapting sitting motions to diverse furniture

SeatSwap automatically adapts motion-captured sitting/lying animations to arbitrary 3D furniture geometry.

Abstract

Training robust motion tracking models requires massive datasets of human-object interactions across diverse body shapes, poses, and furniture geometries. Collecting such data in the real world is prohibitively expensive by using motion capture (MoCap) system: each furniture piece must be 3D-scanned, motion capture sessions require significant studio time, and the combinatorial explosion of body shapes, sitting poses, and furniture variations makes exhaustive capture impossible. We present SeatSwap, a differentiable optimization framework that solves this data scarcity problem by automatically adapting a small set of motion-captured sitting animations to arbitrary 3D furniture geometry. Given a captured sitting motion and a target furniture mesh, SeatSwap produces a physically plausible motion sequence of sitting pose where the character maintains proper contact with the seat surface—effectively enabling unlimited permutations of body-furniture combinations from limited source data. Our key technical contribution is a novel biomechanical optimization pipeline that operates directly on skeletal parameters, enabling gradient-based refinement through forward kinematics and linear blend skinning. SeatSwap retargets existing motion-captured animations while preserving their natural motion characteristics. We integrate neural signed distance fields for differentiable intersection computation and extend the system to support multi-person scenarios through sequential optimization with person-to-person collision avoidance, as well as lying down poses for bed and lounge furniture. Our system processes thousands of furniture pieces in parallel via a distributed job orchestration system, generating the diverse synthetic training data needed to train motion tracking models at Meta scale.

0
Furniture Pieces Processed
0
Cost Reduction vs MoCap
0
Body Shapes Supported
∞
Pose-Furniture Combinations

Method Pipeline

1. Asset Preprocessing
3D Furniture Mesh
↓
8×8×8 Voxel Grid
↓
GoalNet CVAE
↓
10 Candidate Positions
—
Mesh Surface
↓
SDFNet 6-Layer MLP
↓
Continuous SDF Field
2. Motion Processing
Source MoCap Animation
↓
Selective Parameter Optimization
↓
SDF-Aware Kinematic Params θ
3. Differentiable Optimization
Rigid Root Alignment
↓
Joint-Level Refinement
↓
Forward Kinematics
↓
Linear Blend Skinning
↓
Mesh Vertices
↓
Unified Loss Evaluation
L_contact
L_intersection
L_person
L_regularization
4. Production Output
Physically Plausible Motion
↓
Verified Contact & Zero Intersection
↓
Meta-Scale Synthetic Data
4,000+ Pairs · 97.3% Validity

Results

Diverse furniture adaptation
Adaptation across diverse furniture geometries.
Body shape variation
Motion variation on the same furniture.
Single character in room scene
Single character placed in a full room scene.
Multi-person scenario
Multiple characters with varied sitting motions in a full room scene via parallel optimization.
Lying down poses
Lying down poses for beds and lounges.
Lying down across diverse beds
Lying down adaptation across diverse bed geometries.
Multi-person colocation
Multi-person colocation in shared spaces.

BibTeX

@inproceedings{liang2026seatswap,
  title={SeatSwap: Learning-Assisted Differentiable Kinematics for Human-Object Interaction Adaptation in Production-Scale Character Animation},
  author={Liang, Jasmine and Mylabathula, Stephen and Segev, Shai and Deshmukh, Priyamvad Ravindra and Liu, Ying and Aubel, Amaury},
  booktitle={Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters (SIGGRAPH Posters '26)},
  year={2026},
  doi={10.1145/3799825.3818703},
  isbn={979-8-4007-2548-7},
  publisher={ACM},
  address={Los Angeles, CA, USA}
}

Acknowledgement

We thank Marcelo Simoes Wolf for his direction, guidance, support, and vision; Boris Schauerte for mentorship; Chris Twigg and Carsten Stoll for rigs and model infrastructure support; and Filippo Arcadu for coordinating data sourcing—along with all the XFNs who supported this work. We are grateful to our leadership and Reality Labs for the resources that made this work possible.