Interactive Rendering with
Bidirectional Texture
Functions
Frank Suykens, Karl vom Berge, Ares Lagae, and Philip Dutré
Katholieke Universiteit Leuven, Belgium
Motivation and Goal
Classic 2D Texture
BTF
• Interactive rendering of BTFs
• using graphics hardware
• finding suitable approximation for hardware
rendering of BTFs
• Using a new BRDF factorization method:
• Chained Matrix Factorization
Related work
• BTF measurement
– Dana et al. [1997]
– Sattler et al. [2003]
– Han et al. [2003]
• BTF representations
– Leung et al. [2001]
• BTF synthesis
– Liu et al. [2001]
– Tong et al. [2002]
• BTF rendering
– Meseth et al. [2003]
• Hardware rendering of
complex materials
–
–
–
–
–
–
Kautz et al. [1999, 2000]
Malzbender et al. [2001]
McCool et al. [2001]
McAllister et al. [2002]
Sloan et al. [2002, 2003]
Wang et al. [2003]
Overview
Online HW Rendering
BTF
Acquisition
HW Rendering
Index maps
+
Codebooks
Apparent
BRDFs
BTF
Factorization
Offline Pre-Processing
Factors
Factor
Clustering
Overview
Online HW Rendering
BTF
Acquisition
HW Rendering
Index maps
+
Codebooks
Apparent
BRDFs
BTF
Factorization
Offline Pre-Processing
Factors
Factor
Clustering
Synthetic BTF acquisition
Polygonal Model of BTF Mesostructure
Synthetic BTF acquisition
BTF Texel Plane
Synthetic BTF acquisition
BTF Texel Plane
Synthetic BTF acquisition
BTF Texel Plane
Sampled ABRDFs
Overview
Online HW Rendering
BTF
Acquisition
HW Rendering
Index maps
+
Codebooks
Apparent
BRDFs
BTF
Factorization
Offline Pre-Processing
Factors
Factor
Clustering
BTF factorization
• BTF = number of 4D ABRDFs
• Factorization of each individual ABRDF
⇒ Chained matrix factorization (CMF)
BRDF factorization
4D (A)BRDF
f (ωi , ωo )
Product of 2D factors
≈
P (ω i ) ⋅ Q( h) ⋅ R(ω o )
ωo
ωi
Function of
incident direction
Discretization: e.g.
parabolic map
BRDF factorization
4D (A)BRDF
f (ωi , ωo )
Product of 2D factors
≈
P (ω i ) ⋅ Q( h) ⋅ R(ω o )
h
ωi
Function of
halfway vector
ωo
Discretization: e.g.
parabolic map
BRDF factorization
4D (A)BRDF
f (ωi , ωo )
Product of 2D factors
≈
P (ω i ) ⋅ Q( h) ⋅ R(ω o )
ωo
ωi
Function of
exitant direction
Discretization: e.g.
parabolic map
Chained matrix factorization
• Flexible factorization form
• Factors with well-behaved dynamic range
• Overview
– Matrix representation of a discretized BRDF
– Factorization by matrix decomposition
– Chaining matrix decompositions
Matrix representation of a BRDF
• Discretize 4D BRDF f (ω i , ω o )
– IN:
ω i (θ , ϕ ) : N x M discretization
– OUT: ω o (θ , ϕ ) : N x M discretization
Linearize
into rows
OUT (N x M)
IN
=
(N x M)
(false color image)
Linearize
into
columns
Matrix decomposition
• 1-term singular value decomposition (SVD)
IN
Singular
vectors
OUT
Matrix decomposition
• Reconstruction: multiply singular vectors
IN
×
OUT
Matrix decomposition
• Factor textures
P (ωi )
IN
R (ω o )
×
OUT
Convert to
parabolic maps
Chained matrix factorization
• Matrix decomposition (SVD)
– 2 factors
⇒ good BRDF parameterization
• In-Out
• Halfway – Difference (GSHD), Kautz et al. 1999
• Chained matrix factorization
– More factors!
⇒ sequence of different parameterizations
Chaining matrix decompositions
Original
Original (GSHD)
IN
Resample
Factor Q(h)
Project
(or
SVD)
Half
way
Q (h )
OUT
Difference
Q(h) approx.
Q(h) approx.
IN
Resample
OUT
Half
way
Approximation
Difference
Chaining matrix decompositions
Original
Original / Approximation
Residue
IN
IN
P(ωi )
OUT
OUT
Q(h) approx.
1 term SVD
⇒ 2 more factors
IN
R(ωo )
OUT
Chaining matrix decompositions
• Final approximation
f (ω i , ω o ) ≈ Residue(ω i , ω o ) ⋅ Q( h)
P (ω i ) ⋅ R (ω o)
Chaining matrix decompositions
• Final approximation
f (ω i , ω o ) ≈ P(ω i ) ⋅ Q(h ) ⋅ R(ω o )
• General approximation form of CMF
f (ω i , ω o ) ≈
nrDecomp
∏
k =1
Chained
decompositions




Pk ,t (u k ) ⋅ Qk ,t (vk ) 
∑
t =1

nrTerms
Multi-term SVD
Factors
Factorization results (checker BTF)
– Rendered in software
– Compared to homomorphic factorization
– McCool et al. 2001, same PQR form
Light / view
directions
Reference
images
Homomorphic
factorization
Chained
matrix fact.
Using full floating point factors
Factorization results (checker BTF)
– Rendered in software
– Compared to homomorphic factorization
– McCool et al. 2001, same PQR form
Light / view
directions
Reference
images
Homomorphic
factorization
Chained
matrix fact.
Factors quantized to 8-bit
Overview
Online HW Rendering
BTF
Acquisition
HW Rendering
Index maps
+
Codebooks
Apparent
BRDFs
BTF
Factorization
Offline Pre-Processing
Factors
Factor
Clustering
Factor Clustering
Factor each ABRDF:
f (ωi , ωo ) ≈
P (ωi ) ⋅ Q(h ) ⋅ R (ωo )
P(ωi )`s
Array of all factors P (ωi ) of each ABRDF:
Factor Clustering
Factor each ABRDF:
f (ωi , ωo ) ≈
P (ωi ) ⋅ Q(h ) ⋅ R (ωo )
Q ( h)`s
Array of all factors Q (h) of each ABRDF:
Factor Clustering
Factor each ABRDF:
f (ωi , ωo ) ≈
P (ωi ) ⋅ Q(h ) ⋅ R (ωo )
R(ω o )`s
Array of all factors R (ωo ) of each ABRDF:
Factor Clustering
• K-Means Clustering of three factor arrays P(ωi ) Q(h) R(ωo )
P(ωi )
Q(h)
K-Means
Clustering
R(ωo )
Factor Arrays
Index Maps
Codebooks
256 Factor
Prototypes each
Factor Clustering
• K-Means Clustering of three factor arrays P(ωi ) Q(h) R(ωo )
P(ωi )
Q(h)
K-Means
Clustering
R(ωo )
Factor Arrays
Index Maps
Codebooks
Stored as channels
in a RGB Image
256 Factor
Prototypes each
Overview
Online HW Rendering
BTF
Acquisition
HW Rendering
Index maps
+
Codebooks
Apparent
BRDFs
BTF
Factorization
Offline Pre-Processing
Factors
Factor
Clustering
Hardware rendering
• 8 texture units << 3*256 parabolic maps
• Stack codebooks into 3 3D textures
P(ωi )
Q(h)
Stack in 3D
Textures
R(ωo )
Codebooks
Stacked Codebooks
Hardware rendering
Dependent
texture
access
Index map
x
P codebook
Q codebook
ωi
h
n
x
R codebook
n
n
x
ωo
=
Cosine
P (ωi )
R (ωo )
Q (h)
falloff
This is done in a single rendering pass using a fragment program
BTF
Results
~130 FPS
Results
• ~70 k vertices
• 2 BTFs
• ~ 60 FPS
Data sizes and timings
BTF resolution
ABRDF sampling rate
Fact. Time for all
ABRDFs
Clustering time
BTF data size
Factored data size
Clustered data size
Ratio
Checker
Rings
Unclustered
Clustered
16 x 16
10 x 5
48 min.
64 x 64
10 x 5
21 h.
N/A
7.4 Mb
2.25 Mb
2.25 Mb
3.3:1
35 min.
117.4 Mb
38 Mb
2.25 Mb
86:1
Conclusion
• Chained Matrix Factorization:
– flexible factorization scheme
– ABRDFs as well as BRDFs
– factors with well behaved dynamic range
• Single pass interactive BTF rendering
Future work
• Approximation accuracy
– other ABRDF parametrizations
– other factorization schemes
• Compare PCA compression with Clustering
• Mipmapping
• Try framework on real world BTFs
– CUReT
– Bonn
Acknowledgements
• IWT, FWO, and
K.U.Leuven scholarships
• Marc Van Barel
• K.U.Leuven Computer
Graphics Research
Group
Bilinear texture filtering
• Bilinearly filtering the index map does not
make sense
• Determine the 4 closest samples in the
index map and their weights
• Evaluate the BTF for these samples
• Combine using index map sample weights
• 4 times slower (but still interactive)
Bilinear texture filtering
Mipmapping
• Compute different mip levels with lower
resolution sampling planes
• Trilinear filtering is straightforward to
implement
• But we need explicit access to the mip
levels in the fragment program
• No hardware support right now