Image Blending
Recap
DoF
translation
affine
homography
3D rotation
Constraint
Solver
Recap
DoF
translation
affine
homography
3D rotation
2
Constraint
Solver
Recap
DoF
translation
2
affine
6
homography
8
3D rotation
3
Constraint
Solver
Recap
DoF
Constraint
translation
2
At=b
affine
6
homography
8
3D rotation
3
Solver
Recap
DoF
Constraint
translation
2
At=b
affine
6
At=b
homography
8
At=0
3D rotation
3
Rpi=qi
Solver
Recap
DoF
translation
affine
homography
3D rotation
2
6
8
3
Constraint
At=b
Solver
-1
(A’A) b
-1
At=b
(A’A) b
At=0
Eigen-vector of
(A’A)
Rpi=qi
SVD (singular value
decomposition)
Image Blending
We’ve aligned the images – now what?
Image Blending
Want to seamlessly blend them together
Image Blending
[ Marc Pollefeys ]
An Image From NASA
[ Alyosya Efros ]
Image Feathering
12
Feathering
+
1
0
=
1
0
13
Effect of window size
1
left
1
right
0
0
14
Effect of window size
1
1
0
0
15
Good window size
1
0
“Optimal” window: smooth but not ghosted
• Doesn’t always work...
16
Image feathering
What if you’re blending more than two images?
Image feathering
What if you have more than two images?
• Generate weight map for each image
– typically want large weight at center, small weight at edge
• Each output pixel is a weighted average of inputs
– be sure to divide by sum of weights at the end
Alpha Blending
I3
p
I1
Optional: see Blinn (CGA, 1994) for details:
I2
http://ieeexplore.ieee.org/iel1/38/7531/00310740.pdf?
isNumber=7531&prod=JNL&arnumber=310740&arSt=83&ared
=87&arAuthor=Blinn%2C+J.F.
Encoding blend weights: I(x,y) = (αR, αG, αB, α)
color at p =
Implement this in two steps:
1. accumulate: add up the (α premultiplied) RGBα values at each pixel
2. normalize: divide each pixel’s accumulated RGB by its α value
Q: what if α = 0?
More advanced blending schemes
A quick survey...
Pyramid blending
Create a Laplacian pyramid, blend each level
•
Burt, P. J. and Adelson, E. H., A multiresolution spline with applications to image mosaics, ACM Transactions on
Graphics, 42(4), October 1983, 217-236.
The Laplacian Pyramid
Laplacian Pyramid
Gaussian Pyramid
subs
sub
su
ampl
sa
mp
-am
-
=
le
=
bs
pl
e
e
=
Laplacian
level
4
Laplacian
level
2
Laplacian
level
0
Richard Szeliski
left pyramid
Image Stitching
right pyramid
52
blended pyramid
The Laplacian Pyramid
Blended Laplacian Pyramid
Blended Gaussian
Pyramid
expa
exp
ex
nd
=
an
+
=
nd
+
=
+
pa
d
Laplacian image blend
1. Compute Laplacian pyramid
2. Compute Gaussian pyramid on weight image
3. Blend Laplacians using Gaussian blurred weights
4. Reconstruct the final image
25
Examples
Mix an eye and a hand...
Examples
Mix an eye and a hand...
[ david dmartin, Boston College ]
Examples
Mix an eye and a hand...
[ Chris Cameron, CMU ]
Simplification: Two-band Blending
• Brown & Lowe, 2003
–Only use two bands: high freq. and low freq.
–Blends low freq. smoothly
–Blend high freq. with no smoothing: use binary
mask
2-band Blending
Low frequency (λ > 2 pixels)
High frequency (λ < 2 pixels)
Linear Blending
2-band Blending
Gradient-domain blending
Blend the gradients of the two images, then integrate
For more info: Perez et al, SIGGRAPH 2003
Also called “Poisson” blending
De-Ghosting
Local alignment (deghosting)
Use local optic flow to compensate for small
motions [Shum & Szeliski, ICCV’98]
35
Local alignment (deghosting)
Use local optic flow to compensate for radial
distortion [Shum & Szeliski, ICCV’98]
36
Region-based de-ghosting
Select only one image in regions-of-difference
using weighted vertex cover [Uyttendaele et al., CVPR’01]
37
Cutout-based de-ghosting
•Select only one image per
output pixel, using spatial
continuity
•Blend across seams using
gradient continuity
(“Poisson blending”) [Agarwala et al., SG’2004]
38
Cutout-based compositing
Photomontage [Agarwala et al., SG’2004]
• Interactively blend different images: group portraits
39
Photomontage [Agarwala et al., SG’2004]
Cutout-based compositing
Photomontage [Agarwala et al., SG’2004]
• Interactively blend different images: focus settings
41
Cutout-based compositing
Photomontage [Agarwala et al., SG’2004]
• Interactively blend different images: people’s faces
42
More stitching possibilities
• Video stitching
• High dynamic range image stitching
• see demo…
• Flash + Non-Flash
• Video-based rendering
Computational Photography!
43