1. No category

Faculty of Science
Anisotropic Distribution on Manifolds:
Template Estimation and MPPs
IPMI 2015, Sabhal Mor Ostaig, Scotland
Stefan Sommer
Department of Computer Science, University of Copenhagen
July 9, 2015
Slide 1/25
Intrinsic Statistics in Geometric Spaces
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 2/25
Most Probable Paths to Samples
• Euclidean:
T
−1
• density pt (x , y ) ∝ e−(x −y ) Σ (x −y )
• transition density of stationary diffusion processes
• x − y most probable path from y to x
• Manifolds:
• construct family of anisotropic Gaussian-like
distributions NM (y , Σ)
• what is the most probable path from y to x?
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 3/25
Some Statistics on Manifolds
2
• Frechét mean: argminx ∈M N1 ∑N
i =1 d (x , yi )
• PGA (Fletcher et al., ’04); GPCA (Huckeman et al.,
’10); HCA (Sommer, ’13); PNS (Jung et al., ’12); BS
(Pennec, ’15)
PGA
GPCA
HCA
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 4/25
PGA, GPCA, HCA, PNS, . . .
• search for explicitly constructed parametric
subspaces: geodesic sprays, geodesics, iterated
development, . . .
• in general manifolds, subspaces are not totally
geodesic (6≈ linear subspaces)
• projections to subspaces are problematic: geodesics
may be dense on tori
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 5/25
Geometry is Infinitesimal
Euclidean
norm kx − y k
vectors
linear subspaces
...
Riemannian
distances d (x , y )
v0 for geodesics
geodesic sprays
...
• difference vs. tangent vector ←→ global vs. local
• geometry suggests “infinitesimal” constructions
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 6/25
Infinitesimally defined Distributions; MLE
• in Rn , Gaussian distributions are transition
distributions of diffusion processes
dXt = dWt
• on (M , g ), Brownian motion is transition distribution of
stochastic process (Eells-Elworthy-Malliavin
construction), or solution to heat diffusion equation
∂
1
p(t , x ) = ∆p(t , x )
∂t
2
2
• infinitesimal dXt vs. global pt (x , y ) ∝ e−kx −y k
• aim: construction of family NM (µ, Σ) of Gaussian-like
distributions allowing MLE of template / covariance
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 7/25
Anisotropic Diffusions and Holonomy
• stationary driftless diffusion SDE in Rn :
dXt = σdWt , σ ∈ M n×d
• diffusion field σ, infinitesimal generator σσT
• curvature: stationary field/generator cannot be
defined due to holonomy
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 8/25
Stochastic Development:
Eells-Elworthy-Malliavin Construction
• Xt : Rn valued Brownian motion (driving process)
• Ut : FM valued (sub-elliptic) diffusion
• Yt : M valued stochastic process (target process)
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 9/25
Stochastic Development:
Eells-Elworthy-Malliavin Construction
• Xt : Rn valued Brownian motion (driving process)
• Ut : FM valued (sub-elliptic) diffusion
• Yt : M valued stochastic process (target process)
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 9/25
Stochastic Development:
Eells-Elworthy-Malliavin Construction
• Xt : Rn valued Brownian motion (driving process)
• Ut : FM valued (sub-elliptic) diffusion
• Yt : M valued stochastic process (target process)
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 9/25
Ut : Frame Bundle Diffusion
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 10/25
MLE of Diffusion Processes
• Eells-Elworthy-Malliavin construction gives map
Z
: FM → Dens(M )
Diff
R
• Diff (FM ) = NM ⊂ Dens(M ): the set of (normalized)
transition densities from FM diffusions
R
• γ = Diff (x , Xα ) = pγ γ0 , the log-likelihood
N
ln L (x , Xα ) = ln L (γ) =
∑ ln pγ(yi )
i =1
• Estimated Template: argmax(x ,Xα )∈FM ln L (x , Xα )
• MLE of data yi under the assumption y ∼ γ ∈ NM
• Diffusion PCA (Sommer ’14): argmax ln L (x , Xα + εI )
generalizing Probabilistic PCA (Tipping, Bishop, ’99;
Zhang, Fletcher ’13)
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 11/25
Estimated Templates
MLE template
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 12/25
Most Probable Paths
• in Rn , straight lines are most probable for stationary
diffusion processes
• Onsager-Machlup functional (σt curve)
1
1
L(σt ) = − kσ0 (t )k2g + R (σ(t ))
2
12
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 13/25
Most Probable Paths
• in Rn , straight lines are most probable for stationary
diffusion processes
• Onsager-Machlup functional (σt curve)
1
1
L(σt ) = − kσ0 (t )k2g + R (σ(t ))
2
12
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 13/25
Most Probable Paths
• in Rn , straight lines are most probable for stationary
diffusion processes
• Onsager-Machlup functional (σt curve)
1
1
L(σt ) = − kσ0 (t )k2g + R (σ(t ))
2
12
• MPP for target process
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 13/25
Most Probable Paths
• in Rn , straight lines are most probable for stationary
diffusion processes
• Onsager-Machlup functional (σt curve)
1
1
L(σt ) = − kσ0 (t )k2g + R (σ(t ))
2
12
• MPP for driving process
R=0
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 13/25
Definition (MPPs for Driving Process)
Let Xt be the driving process for the diffusion Yt and x ∈ M, i.e.
Yt = π(φ(Xt )). Then σ is a most probable path for the driving
process if it satisfies
Z
1
σ = argminc ∈H (Rd ),φ(c )(1)=x
−L(ct )dt
0
Theorem
Let Yα be any frame for To M, and let Yt = π(φ(0,Yα ) (Xt )), i.e. Yt
is the development of Xt starting at (o, Yα ). Then MPPs for the
driving process Xt are geodesics of a lifted sub-Riemannian
metric on FM:
hw , w̃ iFM = Xα−1 π∗ w , Xα−1 π∗ w̃ Rn .
• geodesics equal MPPs for driving process, isotropic case
• if − ln L (x , Xα ) ≈ c + N1 ∑Ni=1 p(MPP(x , yi )). Then Frechét
mean ≈ MLE, isotropic case
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 14/25
MPPs on S2
increasing anisotropy −→
(a) cov. diag(1, 1) (b) cov. diag(2, .5) (c) cov. diag(4, .25)
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 15/25
MPPs on S2
increasing anisotropy −→
(d) cov. diag(1, 1) (e) cov. diag(2, .5) (f) cov. diag(4, .25)
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 15/25
MPP Evolution Equations
• sub-Riemannian Hamilton-Jacobi equations
pq
ẏtk = Gkj (yt )ξt ,j
1 ∂G
ξ˙ t ,k = −
ξt ,p ξt ,q
2 ∂y k
,
• in coordinates (x i ) for M, Xαi for Xα , and W encoding
the inner product W kl = δαβ Xαk Xβl :
jβ
ẋ i = W ij ξj − W ih Γh ξjβ
,
i
i
jβ
Ẋαi = −Γhα W hj ξj + Γkα W kh Γh ξjβ
1 hγ kh kδ
h
k
k
Γk ,i W Γh + Γk γ W kh Γhδ,i ξhγ ξkδ
ξ˙ i = W hl Γl ,δi ξh ξkδ −
2
h
k
k
k
γ
ξ˙ iα = Γk ,iα W kh Γhδ ξhγ ξkδ − W hl,iα Γl δ + W hl Γl ,δiα ξh ξkδ
1 hk
hγ
kδ
kh
W ,iα ξh ξk + Γk W ,iα Γh ξhγ ξkδ
−
2
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 16/25
LDDMM Landmark MPPs
+ horz. var.
isotropic
+ vert. var.
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 17/25
Summary
• infinitesimal definition of anisotropic normal
distributions NM (µ, Σ) on M
R
• diffusion map Diff : FM → Dens(M ) from
Eells-Elworthy-Malliavin construction, stochastic
development
• MLE of template / covariance (in FM)
• MPPs for driving processes generalize geodesics
1
Sommer: Diffusion Processes and PCA on Manifolds, Oberwolfach extended
abstract (Asymptotic Statistics on Stratified Spaces), 2014.
2
Sommer: Anisotropic Distributions on Manifolds: Template Estimation and Most
Probable Paths, Information Processing in Medical Imaging (IPMI) 2015.
3
Sommer: Evolution Equations with Anisotropic Distributions and Diffusion PCA,
Geometric Science of Information (GSI) 2015.
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 18/25
SDEs: Driving, FM and Target Process
• Hi , i = 1 . . . , n horizontal vector fields on F (M ):
1
Hi (u ) = π−
∗ (ui )
• SDE in Rn (driving):
dXt = Idn dBt , X0 = 0
• SDE in FM:
dUt = Hi (Ut ) ◦ dXti ,
U0 = (x0 , Xα ) , Xα ∈ GL(Rn , Tx0 M)
• Process on M (target):
Yt = πFM (Ut )
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 19/25
The Frame Bundle
• the manifold and frames (bases) for the tangent
spaces Tp M
• F (M ) consists of pairs u = (x , Xα ), x ∈ M, Xα frame
for Tx M
• curves in the horizontal part of F (M ) correspond to
curves in M and parallel transport of frames
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 20/25
The Frame Bundle
• the manifold and frames (bases) for the tangent
spaces Tp M
• F (M ) consists of pairs u = (x , Xα ), x ∈ M, Xα frame
for Tx M
• curves in the horizontal part of F (M ) correspond to
curves in M and parallel transport of frames
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 20/25
The Frame Bundle
• the manifold and frames (bases) for the tangent
spaces Tp M
• F (M ) consists of pairs u = (x , Xα ), x ∈ M, Xα frame
for Tx M
• curves in the horizontal part of F (M ) correspond to
curves in M and parallel transport of frames
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 20/25
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 21/25
Anisotropic Diffusions
• development / “rolling without slipping”:
Rt
wt = 0 us−1 ẋs ds , wt ∈ Rη
• development:
• Diffusion map:
paths in Rn ↔ paths on M
R
Diff (x , Xα ) = πF (M ) (U1 )
• in Rn , sample path increments ∆xti = xti +1 − xti are
normally distributed N (0, (∆t )−1 Σ) with
log-probability
ln p̃Σ (xt ) ∝ −
1
∆t
N −1
∑ ∆xtT Σ−1∆xt + c
i
i
i =1
• Formally, we can set
R1
ln p̃Σ (xt ) ∝ − 0 kẋt k2Σ dt + c
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 22/25
Landmark LDDMM
• Christoffel symbols (Michelli et al. ’08)
1
Γk ij = gir g kl g rs,l − g sl g rk,l − g rl g ks,l gsj
2
• mix of transported frame and cometric: F d M bundle
˜ ∈ T ∗ F d M,
of rank d linear maps Rd → Tx M, ξ, ξ
cometric
D E
D E
1
−1
˜
ξ, ξ˜ = δαβ (ξ|π−
X
)(
ξ|π
X
)
+
λ
ξ, ξ˜
α
β
∗
∗
gR
• the whole frame need not be transported
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 23/25
Statistical Manifold: Geometry of Γ
• Densities
n
Dens(M ) = {γ ∈ Ω (M ) :
Z
γ = 1, γ > 0}
M
• Fisher-Rao metric: GγFR (α, β) =
• Γ finite dim. subset of Dens(M )
R αβ
M γ γγ
• properties of
Z
: FM → Dens(M )
Diff
• naturally defined on bundle of symmetric positive T20
tensors
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 24/25
Sub-Riemannian Geometry
• optimal control problem with nonholonomic
constraints
Z 1
xt = arg min
kċt k2Xα,t dt
ct ,c0 =x ,c1 =y
0
• let
−1
−1
hṽ , w̃ iHFM = Xα,
t π∗ (ṽ ), Xα,t π∗ (w̃ ) Rn
on H(xt ,Xα,t ) FM. This defines a sub-Riemannian
metric G on TFM and equivalent problem
Z 1
(xt , Xα,t ) =
arg min
k(ċt , Ċα,t )k2HFM dt
(ct ,Cα,t ),c0 =x ,c1 =y
0
(1)
with constraints (ċt , Ċα,t ) ∈ H(ct ,Cα,t ) FM
Stefan Sommer ([email protected]) (Department of Computer Science, University of Copenhagen) — Anisotropic Distribution on Manifolds: Template Estimation and MPPs
Slide 25/25