1. No category

The Stitched Puppet: A Graphical Model of 3D Human Shape and Pose
1MPI-Intelligent
http://stitch.is.tue.mpg.de/
1,2,3Silvia
2. Contribution
A 3D part-based statistical
shape model
Popular models used in CG are mesh-based
and learned from 3D scans of people [3]
+: realistic representation of human shape
-: hard to fit to data
3. Representation
4. Graphical model
A puppet is obtained as a vertex-based deformation of a template in T-pose
x18
x0 x13 x16
x2
x9 x11 x15
x1
p(x|S) /
x3
mi
pi
+Bs,i si
+Bp,i di + µp,i
Stitching
˜i
p
xi =
x8 x7
x10 x4
Deformation are represented with low-dim linear model learned with PCA.
We model intrinsic shape deformations and pose-dependent deformations
Bs,torso
Bs,head
Bp,torso
Bp,url
PC1
i
Iij (k)
j
Iji (k)
Y
i (S|xi )
i
Y
ij (xi , xj )
T
T
T
T T
[oi , ri , di , si ]
Pose training data
[1] P.F. Felzenswalb, D.P. Huttenlocker, Pictorial Structures for Object
Recognition, IJCV 2005
[2] L. Sigal, S. Bhatia, S. Roth, M.J.Black, Tracking Loose-limbed People,
CVPR 2004
[3] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Davis, SCAPE: Shape
Completion and Animation of People, SIGGRAPH 2005
[4] F. Bogo, J. Romero, M. Loper, M.J. Black, FAUST: Dataset and
Evaluation for 3D Mesh Registration, CVPR 2014
Samples
pi (si , di ) = Bp,i di + µp,i + Bs,i si + mi
˜ i (si , di , ri , oi ) = Ri (ri )pi (si , di ) + oi
p
x12
Stitching term
ij (xi , xj )
= exp(
X
k=1..Nij
k˜
p iI
ij
(x
)
i
(k)
˜ jI
p
2
ji
(x
)k
)
j
(k)
Alignment of SP (light blue) to the 3D scans of the FAUST benchmark [4] (red)
PC3
References
(i,j)2E
5. Example application
PC2
Shape training data
x5
J. Black
Systems, 2Bernstein Center for Computational Neuroscience, 3ITC-CNR
1. Motivation
Popular models used in CV are part-based [1][2]
+: efficient inference with message-passing
-: only pose is modeled
Zuffi,
1Michael
Low
stitching
cost
High
stitching
cost