NEW SOUTH WALES
Algorithms:
COMP3121/3821/9101/9801
Aleks Ignjatovi´c
School of Computer Science and Engineering
University of New South Wales
LECTURE 4: FAST FOURIER TRANSFORM
COMP3121/3821/9101/9801
1 / 41
Recap: fast multiplication of polynomials
A(x) = An xn + . . . + A0 , B(x) = Bn xn + . . . + B0 two polynomials.
we pad them with zeros to degree 2n, by setting
A2n = . . . = An+1 = B2n = . . . = Bn+1 = 0
so
A(x) = 0 · x2n + . . . + 0 · xn+1 + An xn + . . . + A0
B(x) = 0 · x2n + . . . + 0 · xn+1 + Bn xn + . . . + B0
we saw that in this case we have
A(x) · B(x) =
j
2n
X
X
j=0
!
Ai Bj−i
xj
i=0
COMP3121/3821/9101/9801
2 / 41
Recap: fast multiplication of polynomials
If we let a = hA0 , . . . , An i and b = hB0 , . . . , Bn i, then the sequence
*
a∗b=
j
X
i=0
+2n
Ai Bj−i
j=0
is called the Linear Convolution of sequences a and b. Thus,
a ∗ b = hAn Bn , An−1 Bn + An Bn−1 , An−2 Bn + An−1 Bn−1 + An Bn−2 ,
. . . , A2 B0 + A1 B1 + A0 B2 , A1 B0 + A0 B1 , A0 B0 i
Note that the indices of Ai and Bj−i in the j th term all sum up to j.
COMP3121/3821/9101/9801
3 / 41
Recap: coefficient vs value representation of polynomials
• Every polynomial A(x) of degree n is uniquely determined by its values at any
n + 1 distinct input values for x:
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (xn , A(xn ))}
COMP3121/3821/9101/9801
4 / 41
Recap: coefficient vs value representation of polynomials
• Every polynomial A(x) of degree n is uniquely determined by its values at any
n + 1 distinct input values for x:
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (xn , A(xn ))}
• If A(x) = An xn + An−1 xn−1 + . . . + A0 , we can write in matrix form:

 


1 x0 x20 . . . xn
A(x0 )
A0
0
2
n
 1 x1 x1 . . . x1   A1   A(x1 ) 

 


.
 .
..
..
..
..
..   ..  = 



 ..


.
.
.
.
.
.
A(x
)
A
n
n
1 xn x2n . . . xn
n
COMP3121/3821/9101/9801
(1)
4 / 41
Recap: coefficient vs value representation of polynomials
• Every polynomial A(x) of degree n is uniquely determined by its values at any
n + 1 distinct input values for x:
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (xn , A(xn ))}
• If A(x) = An xn + An−1 xn−1 + . . . + A0 , we can write in matrix form:

 


1 x0 x20 . . . xn
A(x0 )
A0
0
2
n
 1 x1 x1 . . . x1   A1   A(x1 ) 

 


.
 .
..
..
..
..
..   ..  = 



 ..


.
.
.
.
.
.
A(x
)
A
n
n
1 xn x2n . . . xn
n
(1)
• It can be shown that if xi are all distinct, then this matrix is invertible.
COMP3121/3821/9101/9801
4 / 41
Recap: coefficient vs value representation of polynomials
• Thus, if all xi are distinct, given any values A(x0 ), A(x1 ), . . . , A(xn ) the
coefficients A0 , A1 , . . . , An are uniquely determined:

 

−1 
1 x0 x20 . . . xn
A0
A(x0 )
0
 A1   1 x1 x21 . . . xn
  A(x1 ) 
1 

 


 ..  =  .

 
..
.
.
.
.
..
..
..
..  
 .   ..

.
2
n
An
A(xn )
1 xn xn . . . xn
COMP3121/3821/9101/9801
(2)
5 / 41
Recap: coefficient vs value representation of polynomials
• Thus, if all xi are distinct, given any values A(x0 ), A(x1 ), . . . , A(xn ) the
coefficients A0 , A1 , . . . , An are uniquely determined:

 

−1 
1 x0 x20 . . . xn
A0
A(x0 )
0
 A1   1 x1 x21 . . . xn
  A(x1 ) 
1 

 


 ..  =  .

 
..
.
.
.
.
..
..
..
..  
 .   ..

.
2
n
An
A(xn )
1 xn xn . . . xn
(2)
• Equations (1) and (2) show how we can commute between:
COMP3121/3821/9101/9801
5 / 41
Recap: coefficient vs value representation of polynomials
• Thus, if all xi are distinct, given any values A(x0 ), A(x1 ), . . . , A(xn ) the
coefficients A0 , A1 , . . . , An are uniquely determined:

 

−1 
1 x0 x20 . . . xn
A0
A(x0 )
0
 A1   1 x1 x21 . . . xn
  A(x1 ) 
1 

 


 ..  =  .

 
..
.
.
.
.
..
..
..
..  
 .   ..

.
2
n
An
A(xn )
1 xn xn . . . xn
(2)
• Equations (1) and (2) show how we can commute between:
1 a representation of a polynomial A(x) via its coefficients An , An−1 , . . . , A0 , i.e.
A(x) = An xn + . . . + A1 x + A0
COMP3121/3821/9101/9801
5 / 41
Recap: coefficient vs value representation of polynomials
• Thus, if all xi are distinct, given any values A(x0 ), A(x1 ), . . . , A(xn ) the
coefficients A0 , A1 , . . . , An are uniquely determined:

 

−1 
1 x0 x20 . . . xn
A0
A(x0 )
0
 A1   1 x1 x21 . . . xn
  A(x1 ) 
1 

 


 ..  =  .

 
..
.
.
.
.
..
..
..
..  
 .   ..

.
2
n
An
A(xn )
1 xn xn . . . xn
(2)
• Equations (1) and (2) show how we can commute between:
1 a representation of a polynomial A(x) via its coefficients An , An−1 , . . . , A0 , i.e.
A(x) = An xn + . . . + A1 x + A0
2 a representation of a polynomial A(x) via its values
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (xn , A(xn ))}
COMP3121/3821/9101/9801
5 / 41
Recap: our strategy to multiply polynomials fast:
• Given two polynomials of degree at most n,
A(x) = An xn + . . . + A0 ;
B(x) = Bn xn + . . . + B0
COMP3121/3821/9101/9801
6 / 41
Recap: our strategy to multiply polynomials fast:
• Given two polynomials of degree at most n,
A(x) = An xn + . . . + A0 ;
1
B(x) = Bn xn + . . . + B0
convert them into value representation at 2n + 1 distinct points x0 , x1 , . . . , x2n :
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (x2n , A(x2n ))}
B(x) ↔ {(x0 , B(x0 )), (x1 , B(x1 )), . . . , (x2n , B(x2n ))}
COMP3121/3821/9101/9801
6 / 41
Recap: our strategy to multiply polynomials fast:
• Given two polynomials of degree at most n,
A(x) = An xn + . . . + A0 ;
1
B(x) = Bn xn + . . . + B0
convert them into value representation at 2n + 1 distinct points x0 , x1 , . . . , x2n :
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (x2n , A(x2n ))}
B(x) ↔ {(x0 , B(x0 )), (x1 , B(x1 )), . . . , (x2n , B(x2n ))}
2
multiply them point by point using 2n + 1 multiplications:
COMP3121/3821/9101/9801
6 / 41
Recap: our strategy to multiply polynomials fast:
• Given two polynomials of degree at most n,
A(x) = An xn + . . . + A0 ;
1
B(x) = Bn xn + . . . + B0
convert them into value representation at 2n + 1 distinct points x0 , x1 , . . . , x2n :
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (x2n , A(x2n ))}
B(x) ↔ {(x0 , B(x0 )), (x1 , B(x1 )), . . . , (x2n , B(x2n ))}
2
multiply them point by point using 2n + 1 multiplications:
A(x)B(x) ↔ {(x0 , A(x0 )B(x0 )), (x1 , A(x1 )B(x1 )), . . . , (x2n , A(x2n )B(x2n ))}
|
{z
}
{z
}
{z
}
|
|
C(x0 )
C(x1 )
COMP3121/3821/9101/9801
C(x2n )
6 / 41
Recap: our strategy to multiply polynomials fast:
• Given two polynomials of degree at most n,
A(x) = An xn + . . . + A0 ;
1
B(x) = Bn xn + . . . + B0
convert them into value representation at 2n + 1 distinct points x0 , x1 , . . . , x2n :
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (x2n , A(x2n ))}
B(x) ↔ {(x0 , B(x0 )), (x1 , B(x1 )), . . . , (x2n , B(x2n ))}
2
multiply them point by point using 2n + 1 multiplications:
A(x)B(x) ↔ {(x0 , A(x0 )B(x0 )), (x1 , A(x1 )B(x1 )), . . . , (x2n , A(x2n )B(x2n ))}
|
{z
}
{z
}
{z
}
|
|
C(x0 )
3
C(x1 )
C(x2n )
Convert such value representation back to coefficient form
C(x) = C2n x2n + C2n−1 x2n−1 + . . . + C1 x + C0 ;
COMP3121/3821/9101/9801
6 / 41
Recap: our strategy to multiply polynomials fast:
• Given two polynomials of degree at most n,
A(x) = An xn + . . . + A0 ;
1
B(x) = Bn xn + . . . + B0
convert them into value representation at 2n + 1 distinct points x0 , x1 , . . . , x2n :
A(x) ↔ {(x0 , A(x0 )), (x1 , A(x1 )), . . . , (x2n , A(x2n ))}
B(x) ↔ {(x0 , B(x0 )), (x1 , B(x1 )), . . . , (x2n , B(x2n ))}
2
multiply them point by point using 2n + 1 multiplications:
A(x)B(x) ↔ {(x0 , A(x0 )B(x0 )), (x1 , A(x1 )B(x1 )), . . . , (x2n , A(x2n )B(x2n ))}
|
{z
}
{z
}
{z
}
|
|
C(x0 )
3
C(x1 )
C(x2n )
Convert such value representation back to coefficient form
C(x) = C2n x2n + C2n−1 x2n−1 + . . . + C1 x + C0 ;
Key Question: What values should we take for x0 , . . . , x2n to avoid “explosion” of
n
size when we evaluate xn
i while computing A(xi ) = An xi + . . . + A0 ?
COMP3121/3821/9101/9801
6 / 41
Complex numbers revisited
√
Complex numbers z = a + ib can be represented using their modulus |z| = a2 + b2
and their argument, arg(z), which is an angle taking values in (−π, π] and satisfying:
z = |z|ei arg(z) = |z|(cos arg(z) + i sin arg(z)),
see figure below.
!
z!=!a!+!i!b!
|z|!
b!
arg!(z)!
a!
COMP3121/3821/9101/9801
7 / 41
Complex numbers revisited
Recall that
n
z n = |z|ei arg(z)
= |z|n ei n arg(z) = |z|n (cos(n arg(z)) + i sin(n arg(z))),
see the figure.
!
z2 !
|z|2!
z!
2!arg!(z)!
COMP3121/3821/9101/9801
8 / 41
Complex roots of unity
Recall that roots of unity of order n are complex numbers which satisfy z n = 1.
COMP3121/3821/9101/9801
9 / 41
Complex roots of unity
Recall that roots of unity of order n are complex numbers which satisfy z n = 1.
If z n = |z|n (cos(n arg(z)) + i sin(n arg(z))) = 1 then |z| = 1 and arg(z) must be
a multiple of 2π;
COMP3121/3821/9101/9801
9 / 41
Complex roots of unity
Recall that roots of unity of order n are complex numbers which satisfy z n = 1.
If z n = |z|n (cos(n arg(z)) + i sin(n arg(z))) = 1 then |z| = 1 and arg(z) must be
a multiple of 2π;
We denote ωn = ei 2π/n ; such ωn is a primitive root of unity of order n.
COMP3121/3821/9101/9801
9 / 41
Complex roots of unity
Recall that roots of unity of order n are complex numbers which satisfy z n = 1.
If z n = |z|n (cos(n arg(z)) + i sin(n arg(z))) = 1 then |z| = 1 and arg(z) must be
a multiple of 2π;
!
We denote ωn = ei 2π/n ; such ωn is a primitive root of unity of order n.
!! = !! !!!!/! !
|!! | = 1!
2!/!!
1!
COMP3121/3821/9101/9801
9 / 41
Complex roots of unity
A root of unity ω of order n is “primitive” not if it is uncivilised but if all
other roots of unity of the same order can be obtained as its powers ω k .
COMP3121/3821/9101/9801
10 / 41
Complex roots of unity
A root of unity ω of order n is “primitive” not if it is uncivilised but if all
other roots of unity of the same order can be obtained as its powers ω k .
For ωn = ei 2π/n and any integer k we have
((ωn )k )n = (ωn )n k = ((ωn )n )k = 1k = 1
COMP3121/3821/9101/9801
10 / 41
Complex roots of unity
A root of unity ω of order n is “primitive” not if it is uncivilised but if all
other roots of unity of the same order can be obtained as its powers ω k .
For ωn = ei 2π/n and any integer k we have
((ωn )k )n = (ωn )n k = ((ωn )n )k = 1k = 1
Thus, (ωn )k = ωnk is also a root of unity, and it can be shown that it is
primitive just in case k is relatively prime with n.
COMP3121/3821/9101/9801
10 / 41
Complex roots of unity
A root of unity ω of order n is “primitive” not if it is uncivilised but if all
other roots of unity of the same order can be obtained as its powers ω k .
For ωn = ei 2π/n and any integer k we have
((ωn )k )n = (ωn )n k = ((ωn )n )k = 1k = 1
Thus, (ωn )k = ωnk is also a root of unity, and it can be shown that it is
primitive just in case k is relatively prime with n.
Cancelation Lemma:
2π
km
ωkn
= (ωkn )km = (ei kn )km = ei
2πkm
kn
= ei
2πm
n
COMP3121/3821/9101/9801
= (ei
2π
n
)m = (ωn )m = ωnm
10 / 41
Complex roots of unity
A root of unity ω of order n is “primitive” not if it is uncivilised but if all
other roots of unity of the same order can be obtained as its powers ω k .
For ωn = ei 2π/n and any integer k we have
((ωn )k )n = (ωn )n k = ((ωn )n )k = 1k = 1
Thus, (ωn )k = ωnk is also a root of unity, and it can be shown that it is
primitive just in case k is relatively prime with n.
Cancelation Lemma:
2π
km
ωkn
= (ωkn )km = (ei kn )km = ei
2πkm
kn
= ei
2πm
n
= (ei
2π
n
)m = (ωn )m = ωnm
2 k
2k
Thus, in particular, (ω2n
) = ω2n
= ωnk ;
COMP3121/3821/9101/9801
10 / 41
Complex roots of unity
A root of unity ω of order n is “primitive” not if it is uncivilised but if all
other roots of unity of the same order can be obtained as its powers ω k .
For ωn = ei 2π/n and any integer k we have
((ωn )k )n = (ωn )n k = ((ωn )n )k = 1k = 1
Thus, (ωn )k = ωnk is also a root of unity, and it can be shown that it is
primitive just in case k is relatively prime with n.
Cancelation Lemma:
2π
km
ωkn
= (ωkn )km = (ei kn )km = ei
2πkm
kn
= ei
2πm
n
= (ei
2π
n
)m = (ωn )m = ωnm
2 k
2k
Thus, in particular, (ω2n
) = ω2n
= ωnk ;
The squares of roots of unity of order 2n are just the roots of unity of order n.
COMP3121/3821/9101/9801
10 / 41
The Discrete Fourier Transform
Let A = hA0 , A1 , . . . , An i be a sequence of n + 1 real or complex numbers.
COMP3121/3821/9101/9801
11 / 41
The Discrete Fourier Transform
Let A = hA0 , A1 , . . . , An i be a sequence of n + 1 real or complex numbers.
We can form the corresponding polynomial A(x) =
COMP3121/3821/9101/9801
Pn
j=0
Aj xj ,
11 / 41
The Discrete Fourier Transform
Let A = hA0 , A1 , . . . , An i be a sequence of n + 1 real or complex numbers.
We can form the corresponding polynomial A(x) =
Pn
j=0
Aj xj ,
We evaluate it at all complex roots of unity of order n + 1, i.e., we evaluate
k
A(ωn+1
) for all 0 ≤ k ≤ n.
COMP3121/3821/9101/9801
11 / 41
The Discrete Fourier Transform
Let A = hA0 , A1 , . . . , An i be a sequence of n + 1 real or complex numbers.
We can form the corresponding polynomial A(x) =
Pn
j=0
Aj xj ,
We evaluate it at all complex roots of unity of order n + 1, i.e., we evaluate
k
A(ωn+1
) for all 0 ≤ k ≤ n.
2
n
The sequence of values hA(1), A(ωn+1 ), A(ωn+1
), . . . , A(ωn+1
)i, is called the
Discrete Fourier Transform (DFT) of the sequence A = hA0 , A1 , . . . , An i.
COMP3121/3821/9101/9801
11 / 41
The Discrete Fourier Transform
Let A = hA0 , A1 , . . . , An i be a sequence of n + 1 real or complex numbers.
We can form the corresponding polynomial A(x) =
Pn
j=0
Aj xj ,
We evaluate it at all complex roots of unity of order n + 1, i.e., we evaluate
k
A(ωn+1
) for all 0 ≤ k ≤ n.
2
n
The sequence of values hA(1), A(ωn+1 ), A(ωn+1
), . . . , A(ωn+1
)i, is called the
Discrete Fourier Transform (DFT) of the sequence A = hA0 , A1 , . . . , An i.
k
since |ωn+1 | = 1, the powers ωn+1
just rotate around the circle and their sizes
do not “explode”.
COMP3121/3821/9101/9801
11 / 41
Fast multiplication of polynomials
To multiply two polynomials of degree (at most) n we will evaluate them at the roots of
unity of order 2n + 1 (instead of at −n, . . . , −1, 0, 1, . . . , n as in Karatsuba’s method)
COMP3121/3821/9101/9801
12 / 41
Fast multiplication of polynomials
To multiply two polynomials of degree (at most) n we will evaluate them at the roots of
unity of order 2n + 1 (instead of at −n, . . . , −1, 0, 1, . . . , n as in Karatsuba’s method)
this produces the DFT of the (0 padded) sequence of their coefficients
(A0 , A1 , . . . , An , 0, . . . , 0);
| {z }
n
COMP3121/3821/9101/9801
12 / 41
Fast multiplication of polynomials
To multiply two polynomials of degree (at most) n we will evaluate them at the roots of
unity of order 2n + 1 (instead of at −n, . . . , −1, 0, 1, . . . , n as in Karatsuba’s method)
this produces the DFT of the (0 padded) sequence of their coefficients
(A0 , A1 , . . . , An , 0, . . . , 0);
| {z }
n
k
k
we will then multiply the corresponding values A(ω2n+1
) and B(ω2n+1
) for all 0 ≤ k ≤ 2n;
COMP3121/3821/9101/9801
12 / 41
Fast multiplication of polynomials
To multiply two polynomials of degree (at most) n we will evaluate them at the roots of
unity of order 2n + 1 (instead of at −n, . . . , −1, 0, 1, . . . , n as in Karatsuba’s method)
this produces the DFT of the (0 padded) sequence of their coefficients
(A0 , A1 , . . . , An , 0, . . . , 0);
| {z }
n
k
k
we will then multiply the corresponding values A(ω2n+1
) and B(ω2n+1
) for all 0 ≤ k ≤ 2n;
we then use the inverse transformation for DFT, called IDFT, to recover the coefficients of
the product polynomial from its values at these roots of unity:
COMP3121/3821/9101/9801
12 / 41
Fast multiplication of polynomials
To multiply two polynomials of degree (at most) n we will evaluate them at the roots of
unity of order 2n + 1 (instead of at −n, . . . , −1, 0, 1, . . . , n as in Karatsuba’s method)
this produces the DFT of the (0 padded) sequence of their coefficients
(A0 , A1 , . . . , An , 0, . . . , 0);
| {z }
n
k
k
we will then multiply the corresponding values A(ω2n+1
) and B(ω2n+1
) for all 0 ≤ k ≤ 2n;
we then use the inverse transformation for DFT, called IDFT, to recover the coefficients of
the product polynomial from its values at these roots of unity:
n−1
DF T
=⇒
{A(1), A(ω2n+1 ), A(ω2n+1 ), . . . , A(ω2n+1 )}
n−1
DF T
{B(1), B(ω2n+1 ), B(ω2n+1 ), . . . , B(ω2n+1 )}
A(x) = A0 + A1 x + . . . + An−1 x
B(x) = B0 + B1 x + . . . + Bn−1 x
=⇒
2
2n
2
2n
⇓ multiplication
2n
2n
{A(1)B(1), A(ω2n+1 )B(ω2n+1 ), . . . , A(ω2n+1 )B(ω2n+1 )}
C(x) = A(x) · B(x) =
j
2n X
X
j=0
i=0
|
COMP3121/3821/9101/9801
⇓ IDFT
j
Ai Bj−i x .
{z
Cj
}
12 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
so we have to find an efficient way to compute DFT and IDFT;
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
so we have to find an efficient way to compute DFT and IDFT;
For each fixed k such that 0 ≤ k < 2n we have to to evaluate
k
2k
nk
A(x) = A0 + A1 ω2n+1
+ A2 ω2n+1
+ . . . + An ω2n+1
k
2k
nk
B(x) = B0 + B1 ω2n+1
+ B2 ω2n+1
+ . . . + Bn ω2n+1
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
so we have to find an efficient way to compute DFT and IDFT;
For each fixed k such that 0 ≤ k < 2n we have to to evaluate
k
2k
nk
A(x) = A0 + A1 ω2n+1
+ A2 ω2n+1
+ . . . + An ω2n+1
k
2k
nk
B(x) = B0 + B1 ω2n+1
+ B2 ω2n+1
+ . . . + Bn ω2n+1
k
we can precompute all of the values ω2n+1
, but, by brute force, for each k we would
km , for 0 ≤ m ≤ n.
have to do n + 1 multiplications of the form Am · ω2n+1
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
so we have to find an efficient way to compute DFT and IDFT;
For each fixed k such that 0 ≤ k < 2n we have to to evaluate
k
2k
nk
A(x) = A0 + A1 ω2n+1
+ A2 ω2n+1
+ . . . + An ω2n+1
k
2k
nk
B(x) = B0 + B1 ω2n+1
+ B2 ω2n+1
+ . . . + Bn ω2n+1
k
we can precompute all of the values ω2n+1
, but, by brute force, for each k we would
km , for 0 ≤ m ≤ n.
have to do n + 1 multiplications of the form Am · ω2n+1
Thus, since k ranges from 0 to 2n, we would have to do O(n2 ) multiplications.
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
so we have to find an efficient way to compute DFT and IDFT;
For each fixed k such that 0 ≤ k < 2n we have to to evaluate
k
2k
nk
A(x) = A0 + A1 ω2n+1
+ A2 ω2n+1
+ . . . + An ω2n+1
k
2k
nk
B(x) = B0 + B1 ω2n+1
+ B2 ω2n+1
+ . . . + Bn ω2n+1
k
we can precompute all of the values ω2n+1
, but, by brute force, for each k we would
km , for 0 ≤ m ≤ n.
have to do n + 1 multiplications of the form Am · ω2n+1
Thus, since k ranges from 0 to 2n, we would have to do O(n2 ) multiplications.
Can we do it faster??
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
k
k
Multiplying 2n + 1 values of A(ω2n+1
) and B(ω2n+1
) is done in linear time;
so we have to find an efficient way to compute DFT and IDFT;
For each fixed k such that 0 ≤ k < 2n we have to to evaluate
k
2k
nk
A(x) = A0 + A1 ω2n+1
+ A2 ω2n+1
+ . . . + An ω2n+1
k
2k
nk
B(x) = B0 + B1 ω2n+1
+ B2 ω2n+1
+ . . . + Bn ω2n+1
k
we can precompute all of the values ω2n+1
, but, by brute force, for each k we would
km , for 0 ≤ m ≤ n.
have to do n + 1 multiplications of the form Am · ω2n+1
Thus, since k ranges from 0 to 2n, we would have to do O(n2 ) multiplications.
Can we do it faster??
This is precisely what the Fast Fourier Transform (FFT) does; it computes all of
k
the values A(ω2n+1
) in O(n log n) time.
COMP3121/3821/9101/9801
13 / 41
Faster method for multiplication of polynomials
Warning:
a common source of confusion:
COMP3121/3821/9101/9801
14 / 41
Faster method for multiplication of polynomials
Warning:
a common source of confusion:
The Discrete Fourier Transform (DFT) is a transformation of sequences:
given a sequence of real (or complex) numbers
hA0 , A1 , A2 , . . . , An i
its Discrete Fourier Transform is the sequence of the values of the corresponding
polynomial A(x) = A0 + A1 x + A2 x2 + . . . + An xn at all roots of unity of order n + 1:
2
n
hA(1), A(ωn+1 ), A(ωn+1
), . . . , A(ωn+1
)i
COMP3121/3821/9101/9801
14 / 41
Faster method for multiplication of polynomials
Warning:
a common source of confusion:
The Discrete Fourier Transform (DFT) is a transformation of sequences:
given a sequence of real (or complex) numbers
hA0 , A1 , A2 , . . . , An i
its Discrete Fourier Transform is the sequence of the values of the corresponding
polynomial A(x) = A0 + A1 x + A2 x2 + . . . + An xn at all roots of unity of order n + 1:
2
n
hA(1), A(ωn+1 ), A(ωn+1
), . . . , A(ωn+1
)i
The Fast Fourier Transform (FFT) is an algorithm for computing the Discrete
Fourier Transform of a sequence. Thus, the output of the FFT when applied to a
sequence is the DFT of that sequence.
COMP3121/3821/9101/9801
14 / 41
Faster method for multiplication of polynomials
Let A(x) = A0 + A1 x + . . . + An−1 xn−1 ;
COMP3121/3821/9101/9801
15 / 41
Faster method for multiplication of polynomials
Let A(x) = A0 + A1 x + . . . + An−1 xn−1 ;
we can assume that n is a power of 2; otherwise we can pad A(x)
with zero coefficients until its degree becomes equal to the nearest
power of 2.
COMP3121/3821/9101/9801
15 / 41
Faster method for multiplication of polynomials
Let A(x) = A0 + A1 x + . . . + An−1 xn−1 ;
we can assume that n is a power of 2; otherwise we can pad A(x)
with zero coefficients until its degree becomes equal to the nearest
power of 2.
Exercise: show that for every n which is not a power of two the
smallest power of 2 larger than n is smaller than 2n.
COMP3121/3821/9101/9801
15 / 41
Faster method for multiplication of polynomials
Let A(x) = A0 + A1 x + . . . + An−1 xn−1 ;
we can assume that n is a power of 2; otherwise we can pad A(x)
with zero coefficients until its degree becomes equal to the nearest
power of 2.
Exercise: show that for every n which is not a power of two the
smallest power of 2 larger than n is smaller than 2n.
Hint: consider n in binary. How many bits does the nearest power
of two have?
COMP3121/3821/9101/9801
15 / 41
Faster method for multiplication of polynomials
Idea: compute the DFT using divide-and-conquer by splitting the polynomial into
even powers and odd powers:
COMP3121/3821/9101/9801
16 / 41
Faster method for multiplication of polynomials
Idea: compute the DFT using divide-and-conquer by splitting the polynomial into
even powers and odd powers:
A(x) = (A0 + A2 x2 + A4 x4 + . . . + An−2 xn−2 ) + (A1 x + A3 x3 + . . . An−1 xn−1 )
n
= A0 + A2 x2 + A4 (x2 )2 + . . . + An−2 (x2 ) 2 −1
n
+ x(A1 + A3 x2 + A5 (x2 )2 + . . . + An−1 (x2 ) 2 −1 )
COMP3121/3821/9101/9801
16 / 41
Faster method for multiplication of polynomials
Idea: compute the DFT using divide-and-conquer by splitting the polynomial into
even powers and odd powers:
A(x) = (A0 + A2 x2 + A4 x4 + . . . + An−2 xn−2 ) + (A1 x + A3 x3 + . . . An−1 xn−1 )
n
= A0 + A2 x2 + A4 (x2 )2 + . . . + An−2 (x2 ) 2 −1
n
+ x(A1 + A3 x2 + A5 (x2 )2 + . . . + An−1 (x2 ) 2 −1 )
Let us define
n
A0 (y) = A0 + A2 y + A4 y 2 + . . . + An−2 y 2 −1
n
A1 (y) = A1 + A3 y + A5 y 2 + . . . + An−1 y 2 −1
COMP3121/3821/9101/9801
16 / 41
Faster method for multiplication of polynomials
Idea: compute the DFT using divide-and-conquer by splitting the polynomial into
even powers and odd powers:
A(x) = (A0 + A2 x2 + A4 x4 + . . . + An−2 xn−2 ) + (A1 x + A3 x3 + . . . An−1 xn−1 )
n
= A0 + A2 x2 + A4 (x2 )2 + . . . + An−2 (x2 ) 2 −1
n
+ x(A1 + A3 x2 + A5 (x2 )2 + . . . + An−1 (x2 ) 2 −1 )
Let us define
n
A0 (y) = A0 + A2 y + A4 y 2 + . . . + An−2 y 2 −1
n
A1 (y) = A1 + A3 y + A5 y 2 + . . . + An−1 y 2 −1
Then
A(x) = A0 (x2 ) + x A1 (x2 )
COMP3121/3821/9101/9801
16 / 41
Faster method for multiplication of polynomials
Idea: compute the DFT using divide-and-conquer by splitting the polynomial into
even powers and odd powers:
A(x) = (A0 + A2 x2 + A4 x4 + . . . + An−2 xn−2 ) + (A1 x + A3 x3 + . . . An−1 xn−1 )
n
= A0 + A2 x2 + A4 (x2 )2 + . . . + An−2 (x2 ) 2 −1
n
+ x(A1 + A3 x2 + A5 (x2 )2 + . . . + An−1 (x2 ) 2 −1 )
Let us define
n
A0 (y) = A0 + A2 y + A4 y 2 + . . . + An−2 y 2 −1
n
A1 (y) = A1 + A3 y + A5 y 2 + . . . + An−1 y 2 −1
Then
A(x) = A0 (x2 ) + x A1 (x2 )
Note that the degree of the polynomials A0 (y) and A1 (y) is half of the degree of the
polynomial A(x).
COMP3121/3821/9101/9801
16 / 41
Faster method for multiplication of polynomials
Problem of size n:
Evaluate a polynomial of degree n − 1 at n many roots of unity.
COMP3121/3821/9101/9801
17 / 41
Faster method for multiplication of polynomials
Problem of size n:
Evaluate a polynomial of degree n − 1 at n many roots of unity.
Problem of size n/2:
Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity.
COMP3121/3821/9101/9801
17 / 41
Faster method for multiplication of polynomials
Problem of size n:
Evaluate a polynomial of degree n − 1 at n many roots of unity.
Problem of size n/2:
Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity.
We reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1.
COMP3121/3821/9101/9801
17 / 41
Faster method for multiplication of polynomials
Problem of size n:
Evaluate a polynomial of degree n − 1 at n many roots of unity.
Problem of size n/2:
Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity.
We reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1.
0 , ω 1 , ω 2 , . . . , ω n−1 }, the
By the Cancelation Lemma, as x ranges through values {ωn
n
n
n
n−1
2
0
1
2
value of y = x ranges through {ω n , ω n , ω n , . . . , ω n }, and there are only n/2
2
2
2
2
distinct such values.
COMP3121/3821/9101/9801
17 / 41
Faster method for multiplication of polynomials
Problem of size n:
Evaluate a polynomial of degree n − 1 at n many roots of unity.
Problem of size n/2:
Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity.
We reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1.
0 , ω 1 , ω 2 , . . . , ω n−1 }, the
By the Cancelation Lemma, as x ranges through values {ωn
n
n
n
n−1
2
0
1
2
value of y = x ranges through {ω n , ω n , ω n , . . . , ω n }, and there are only n/2
2
2
2
2
distinct such values.
Once we got these n/2 values of A0 (x2 ) and A1 (x2 ) we need n additional
multiplications to obtain the values of
k
k 2
k 1
k 2
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) )
2 k
k 1
2 k
= A0 ((ωn
) ) + ωn
A ((ωn
) )
k 1
A ((ω n )k )
= A0 ((ω n )k ) + ωn
2
COMP3121/3821/9101/9801
2
17 / 41
Faster method for multiplication of polynomials
n
n
Note that by the Cancelation Lemma ωn2 = ω22n = ω2 = −1; thus,
2
COMP3121/3821/9101/9801
18 / 41
Faster method for multiplication of polynomials
n
n
Note that by the Cancelation Lemma ωn2 = ω22n = ω2 = −1; thus,
2
n +k
2
ωn
=
n
2
k
ωn ωn
k
k
= ω2 ωn
= −ωn
COMP3121/3821/9101/9801
18 / 41
Faster method for multiplication of polynomials
n
n
Note that by the Cancelation Lemma ωn2 = ω22n = ω2 = −1; thus,
2
n +k
2
ωn
=
n
2
k
ωn ωn
k
k
= ω2 ωn
= −ωn
We can now simplify evaluation of
k
k 2
k 1
k 2
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) )
for k > n/2 as follows.
COMP3121/3821/9101/9801
18 / 41
Faster method for multiplication of polynomials
n
n
Note that by the Cancelation Lemma ωn2 = ω22n = ω2 = −1; thus,
2
n +k
2
ωn
=
n
2
k
ωn ωn
k
k
= ω2 ωn
= −ωn
We can now simplify evaluation of
k
k 2
k 1
k 2
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) )
for k > n/2 as follows. Let k =
n
2
+ m; then
COMP3121/3821/9101/9801
18 / 41
Faster method for multiplication of polynomials
n
n
Note that by the Cancelation Lemma ωn2 = ω22n = ω2 = −1; thus,
2
n +k
2
ωn
=
n
2
k
ωn ωn
k
k
= ω2 ωn
= −ωn
We can now simplify evaluation of
k
k 2
k 1
k 2
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) )
for k > n/2 as follows. Let k =
n +m
2
A(ωn
n
2
+ m; then
n +m
) = A0 ((ωn2
n +m
)2 ) + ωn2
n +m
A1 ((ωn2
)2 )
n
2
n+2m
m 1
n+2m
= A0 (ωn
) + ωn ωn
A (ωn
)
n 2m
m 1
n 2m
= A0 (ωn
ωn ) + ω2 ωn
A (ωn
ωn )
2m
m 1
2m
= A0 (ωn
) − ωn
A (ωn
)
m 2
m 1
m 2
= A0 ((ωn
) ) − ωn
A ((ωn
) )
m 1
m
= A0 (ω m
n ) − ωn A (ω n )
2
2
COMP3121/3821/9101/9801
18 / 41
Faster method for multiplication of polynomials
n
n
Note that by the Cancelation Lemma ωn2 = ω22n = ω2 = −1; thus,
2
n +k
2
ωn
=
n
2
k
ωn ωn
k
k
= ω2 ωn
= −ωn
We can now simplify evaluation of
k
k 2
k 1
k 2
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) )
for k > n/2 as follows. Let k =
n +m
2
A(ωn
n
2
+ m; then
n +m
) = A0 ((ωn2
n +m
)2 ) + ωn2
n +m
A1 ((ωn2
)2 )
n
2
n+2m
m 1
n+2m
= A0 (ωn
) + ωn ωn
A (ωn
)
n 2m
m 1
n 2m
= A0 (ωn
ωn ) + ω2 ωn
A (ωn
ωn )
2m
m 1
2m
= A0 (ωn
) − ωn
A (ωn
)
m 2
m 1
m 2
= A0 ((ωn
) ) − ωn
A ((ωn
) )
m 1
m
= A0 (ω m
n ) − ωn A (ω n )
2
2
Compare this with the corresponding equation for 0 ≤ m < n/2:
m
m 2
m 1
m 2
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) )
m 1
m
= A0 (ω m
n ) + ωn A (ω n )
2
2
COMP3121/3821/9101/9801
18 / 41
Faster method for multiplication of polynomials
So we can replace evaluations of
k
k 2
k 1
k 2
k 1
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) ) = A0 (ω kn ) + ωn
A (ω kn )
2
2
for k = 0 to k = n − 1 with such evaluations only for k = 0 to k = n/2 − 1 and just let
for m = 0 to m = n/2 − 1
n +m
A(ωn2
m 2
m 1
m 2
k 1
) = A0 ((ωn
) ) − ωn
A ((ωn
) ) = A0 (ω kn ) − ωn
A (ω kn )
2
COMP3121/3821/9101/9801
2
19 / 41
Faster method for multiplication of polynomials
So we can replace evaluations of
k
k 2
k 1
k 2
k 1
A(ωn
) = A0 ((ωn
) ) + ωn
A ((ωn
) ) = A0 (ω kn ) + ωn
A (ω kn )
2
2
for k = 0 to k = n − 1 with such evaluations only for k = 0 to k = n/2 − 1 and just let
for m = 0 to m = n/2 − 1
n +m
A(ωn2
m 2
m 1
m 2
k 1
) = A0 ((ωn
) ) − ωn
A ((ωn
) ) = A0 (ω kn ) − ωn
A (ω kn )
2
2
We can now write a pseudo-code for our FFT algorithm:
COMP3121/3821/9101/9801
19 / 41
FFT algorithm
1: function FFT(A)
2:
n ← length[A]
3:
if n = 1 then return A
4:
else
5:
A[0] ← (A0 , A2 , . . . An−2 );
6:
A[1] ← (A1 , A3 , . . . An−1 );
7:
y [0] ← F F T (A[0] );
8:
y [1] ← F F T (A[1] );
2π
9:
ωn ← ei n ;
10:
ω ← 1;
− 1 do;
11:
for k = 0 to k = n
2
[0]
[1]
12:
yk ← yk + ω · yk ;
[0]
[1]
13:
y n +k ← yk − ω · yk
2
14:
ω ← ω · ωn ;
15:
end for
16:
return y
17:
end if
18: end function
COMP3121/3821/9101/9801
20 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
COMP3121/3821/9101/9801
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
COMP3121/3821/9101/9801
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1
COMP3121/3821/9101/9801
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1 and
0
1
2
n−1
as x ranges through values {ωn
, ωn
, ωn
, . . . , ωn
}, the value of y = x2 ranges through
n−1
0
1
2
{ω n , ω n , ω n , . . . , ω n }, and there are only n/2 distinct such values.
2
2
2
2
COMP3121/3821/9101/9801
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1 and
0
1
2
n−1
as x ranges through values {ωn
, ωn
, ωn
, . . . , ωn
}, the value of y = x2 ranges through
n−1
0
1
2
{ω n , ω n , ω n , . . . , ω n }, and there are only n/2 distinct such values.
2
2
2
2
Once we get these n/2 values of A0 (x2 ) and A1 (x2 ) we need n/2 additional multiplications
to obtain the values of
k
0
k 2
k
1
k 2
0
k
k
1
k
A(ωn ) = A ((ωn ) ) + ωn A ((ωn ) ) = A (ω n ) + ωn A (ω n )
2
COMP3121/3821/9101/9801
2
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1 and
0
1
2
n−1
as x ranges through values {ωn
, ωn
, ωn
, . . . , ωn
}, the value of y = x2 ranges through
n−1
0
1
2
{ω n , ω n , ω n , . . . , ω n }, and there are only n/2 distinct such values.
2
2
2
2
Once we get these n/2 values of A0 (x2 ) and A1 (x2 ) we need n/2 additional multiplications
to obtain the values of
k
0
k 2
k
1
k 2
0
k
k
1
k
A(ωn ) = A ((ωn ) ) + ωn A ((ωn ) ) = A (ω n ) + ωn A (ω n )
2
and
n +k
A(ωn2
0
k 2
k
1
k 2
0
2
k
k
1
k
) = A ((ωn ) ) − ωn A ((ωn ) ) = A (ω n ) − ωn A (ω n )
2
COMP3121/3821/9101/9801
2
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1 and
0
1
2
n−1
as x ranges through values {ωn
, ωn
, ωn
, . . . , ωn
}, the value of y = x2 ranges through
n−1
0
1
2
{ω n , ω n , ω n , . . . , ω n }, and there are only n/2 distinct such values.
2
2
2
2
Once we get these n/2 values of A0 (x2 ) and A1 (x2 ) we need n/2 additional multiplications
to obtain the values of
k
0
k 2
k
1
k 2
0
k
k
1
k
A(ωn ) = A ((ωn ) ) + ωn A ((ωn ) ) = A (ω n ) + ωn A (ω n )
2
and
n +k
A(ωn2
0
k 2
k
1
k 2
0
2
k
k
1
k
) = A ((ωn ) ) − ωn A ((ωn ) ) = A (ω n ) − ωn A (ω n )
2
2
Thus, we reduced a problem of size n to two such problems of size n/2, plus a linear
overhead and so our algorithm’s run time satisfies the recurrence
COMP3121/3821/9101/9801
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1 and
0
1
2
n−1
as x ranges through values {ωn
, ωn
, ωn
, . . . , ωn
}, the value of y = x2 ranges through
n−1
0
1
2
{ω n , ω n , ω n , . . . , ω n }, and there are only n/2 distinct such values.
2
2
2
2
Once we get these n/2 values of A0 (x2 ) and A1 (x2 ) we need n/2 additional multiplications
to obtain the values of
k
0
k 2
k
1
k 2
0
k
k
1
k
A(ωn ) = A ((ωn ) ) + ωn A ((ωn ) ) = A (ω n ) + ωn A (ω n )
2
and
n +k
A(ωn2
0
k 2
k
1
k 2
0
2
k
k
1
k
) = A ((ωn ) ) − ωn A ((ωn ) ) = A (ω n ) − ωn A (ω n )
2
2
Thus, we reduced a problem of size n to two such problems of size n/2, plus a linear
overhead and so our algorithm’s run time satisfies the recurrence
n
T (n) = 2 T
+ cn
2
COMP3121/3821/9101/9801
21 / 41
FFT
To recapitulate:
Problem of size n:
“Evaluate a polynomial of degree n − 1 at n many roots of unity”
has been reduced to two problems of size n/2:
“Evaluate a polynomial of degree n/2 − 1 at n/2 many roots of unity”
because we reduced evaluation of our polynomial A(x) of degree n − 1 to evaluation of two
polynomials A0 (y) and A1 (y) of degree n/2 − 1 and
0
1
2
n−1
as x ranges through values {ωn
, ωn
, ωn
, . . . , ωn
}, the value of y = x2 ranges through
n−1
0
1
2
{ω n , ω n , ω n , . . . , ω n }, and there are only n/2 distinct such values.
2
2
2
2
Once we get these n/2 values of A0 (x2 ) and A1 (x2 ) we need n/2 additional multiplications
to obtain the values of
k
0
k 2
k
1
k 2
0
k
k
1
k
A(ωn ) = A ((ωn ) ) + ωn A ((ωn ) ) = A (ω n ) + ωn A (ω n )
2
and
n +k
A(ωn2
0
k 2
k
1
k 2
0
2
k
k
1
k
) = A ((ωn ) ) − ωn A ((ωn ) ) = A (ω n ) − ωn A (ω n )
2
2
Thus, we reduced a problem of size n to two such problems of size n/2, plus a linear
overhead and so our algorithm’s run time satisfies the recurrence
n
T (n) = 2 T
+ cn
2
The Master Theorem gives T (n) = Θ(n log n).
COMP3121/3821/9101/9801
21 / 41
Recall our strategy for fast multiplication of polynomials
k
Evaluation of a polynomial A(x) = A0 + A1 x + . . . + An−1 xn−1 at roots of unity ωn
of
order n can be represented in the matrix form as follows:









1
1
1
.
.
.
1
1
ωn
2
ωn
.
.
.
n−1
ωn
1
2
ωn
2·2
ωn
.
.
.
2(n−1)
ωn
...
...
...
.
.
.
...

1
n−1
ωn
2·(n−1)
ωn
.
.
.
(n−1)(n−1)
ωn
A0
 A
1


  A2

 .
 .
 .
An










=






A(1)
A(ωn )
2
A(ωn
)
.
.
.
n−1
A(ωn
)





.



2
n−1
0
Thus, from A(ωn
), A(ωn ), A(ωn
), . . . , A(ωn
), we get the coefficients using

A0
 A
1


 A2

 .
 .
 .
An


1

 1





 1
=
 .


 .
 .

1
1
ωn
2
ωn
.
.
.
n−1
ωn
1
2
ωn
2·2
ωn
.
.
.
2(n−1)
ωn
...
...
...
.
.
.
...
1
n−1
ωn
2·(n−1)
ωn
.
.
.
(n−1)(n−1)
ωn
COMP3121/3821/9101/9801
−1 
















A(1)
A(ωn )
2
)
A(ωn
.
.
.
n−1
A(ωn
)





.



22 / 41
Recall our strategy for fast multiplication of polynomials
Another remarkable feature of the roots of unity: to obtain the inverse of the above
matrix, all we have to do is just change the signs of the exponents and divide
everything by n:








1
1
1
..
.
1
1
ωn
2
ωn
..
.
n−1
ωn
1
2
ωn
2·2
ωn
..
.
2(n−1)
ωn



1


n


...
...
...
..
.
...
1
1
1
..
.
1
−1
1
n−1
ωn
2·(n−1)
ωn
..
.
(n−1)(n−1)







=
ωn
1
1
−1
ωn
−2
ωn
−2
ωn
−2·2
ωn
..
.
..
.
−(n−1)
ωn
−2(n−1)
ωn
COMP3121/3821/9101/9801
...
...
...
..
.
...
1
−(n−1)
ωn
−2·(n−1)
ωn
..
.
−(n−1)(n−1)









ωn
23 / 41
Recall our strategy for fast multiplication of polynomials
To see this, note that if we compute the product











1
1
1
...
1

2
ωn
2·2
ωn
...
n−1
ωn
2·(n−1)
ωn
.
.
.
.
.
.
...
(n−1)(n−1)
ωn










1
ωn
1
2
ωn
.
.
.
.
.
.
.
.
.
n−1
ωn
2(n−1)
ωn
1
...
1

−(n−1)
ωn
−2·(n−1)
ωn










1
1
1
1
−1
ωn
−2
ωn
−2
ωn
−2·2
ωn
...
.
.
.
.
.
.
...
−(n−1)(n−1)
ωn
1
.
.
.
.
.
.
.
.
.
1
−(n−1)
ωn
−2(n−1)
ωn
...
...
the (i, j) entry in the product matrix is equal to a product of ith row and j th column:

i
2·i
1 ωn ωn
...





i·(n−1)
ωn





1
−j
ωn
−2j
ωn
.
.
.
−(n−1)j




n−1
n−1

X ik −jk
X (i−j)k

ωn ωn
=
ωn
=


k=0
k=0


ωn
COMP3121/3821/9101/9801
24 / 41
Recall our strategy for fast multiplication of polynomials
We now have two possibilities:
1
i = j: then
n−1
X
(i−j)k
ωn
=
k=0
2
i 6= j: then
Pn−1
k=0
n−1
X
0
ωn =
k=0
n−1
X
1 = n;
k=0
(i−j)k
ωn
represents a geometric series with the ratio ωn :
n−1
X
k=0
(i−j)k
ωn
=
(i−j)n
1 − ωn
i−j
1 − ωn
=
n i−j
1 − (ωn
)
i−j
1 − ωn
=
1−1
i−j
1 − ωn
=0
Thus,

1
i
ωn
2·i
ωn




i·(n−1)
. . . ωn




1
−j
ωn
−2j
ωn
.
.
.
−(n−1)j
ωn




n−1
n if i = j

X (i−j)k

ωn
=
=
0 if i 6= j

k=0


COMP3121/3821/9101/9801
(3)
25 / 41

1
1
1
...
1

1
1
1
...










1
ωn
2
ωn
...
n−1
ωn
1
−1
ωn
−2
ωn
...
1
2
ωn
2·2
ωn
...
2·(n−1)
ωn
1
−2
ωn
−2·2
ωn
...
−2·(n−1)
ωn
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
n−1
ωn










2(n−1)
ωn
...
(n−1)(n−1)
ωn
1
−(n−1)
ωn
−2(n−1)
ωn
...

1
−(n−1)
ωn
−(n−1)(n−1)










ωn

1
0
0
...
0





= n




0
1
0
...
0
0
1
...
.
.
.
.
.
.
.
.
.
.
.
.

0 


0 

. 
. 

. 
0
0
0
...

1
i.e.


















1
1
1
...
1
1
2
ωn
2·2
ωn
...
1
ωn
2
ωn
n−1
ωn
2·(n−1)
ωn
.
.
.
.
.
.
.
.
.
1
n−1
ωn
.
.
.
2(n−1)
ωn
...
...
−1









=

























.
.
.
(n−1)(n−1)
ωn
1
n
1
1
1
...
1
−1
ωn
−2
ωn
−2
ωn
−2·2
ωn
...
.
.
.
−(n−1)
ωn
.
.
.
−2(n−1)
ωn
1
.
.
.
1
COMP3121/3821/9101/9801
...
.
.
.
...
1
−(n−1)
ωn
−2·(n−1)
ωn
.
.
.
−(n−1)(n−1)
ωn


















26 / 41
• We now have

A0
 A
1


 A2

 .
 .
 .
An


1
1
1
...
2

 1
ωn
ωn
...




2
2·2

 1
ωn
ωn
...
=

 .
.
.
.

 .
.
.
.

 .
.
.
.
n−1
2(n−1)
1 ωn
ωn
...

1
1
1
−1
−2
 1
ωn
ωn

−2
−2·2
1 
 1
ωn
ωn
=

n
.
.
.
 .
.
.
 .
.
.
−(n−1)
−2(n−1)
1 ωn
ωn
1
n−1
ωn
2·(n−1)
ωn
.
.
.
−1 








(n−1)(n−1)
ωn
...
...
...
.
.
.
...
1
−(n−1)
ωn
−2·(n−1)
ωn
.
.
.
−(n−1)(n−1)
ωn
COMP3121/3821/9101/9801

A(1)
A(ωn ) 


2
A(ωn
) 
=

.

.

.
n−1
A(ωn )


A(1)
  A(ω ) 
n




2
  A(ωn
) 




.


.


.








n−1
A(ωn
)
27 / 41
• We now have

A0
 A
1


 A2

 .
 .
 .
An


1
1
1
...
2

 1
ωn
ωn
...




2
2·2

 1
ωn
ωn
...
=

 .
.
.
.

 .
.
.
.

 .
.
.
.
n−1
2(n−1)
1 ωn
ωn
...

1
1
1
−1
−2
 1
ωn
ωn

−2
−2·2
1 
 1
ωn
ωn
=

n
.
.
.
 .
.
.
 .
.
.
−(n−1)
−2(n−1)
1 ωn
ωn
1
n−1
ωn
2·(n−1)
ωn
.
.
.
−1 








(n−1)(n−1)
ωn
...
...
...
.
.
.
...
1
−(n−1)
ωn
−2·(n−1)
ωn
.
.
.
−(n−1)(n−1)
ωn

A(1)
A(ωn ) 


2
A(ωn
) 
=

.

.

.
n−1
A(ωn )


A(1)
  A(ω ) 
n




2
  A(ωn
) 




.


.


.








n−1
A(ωn
)
n−1
2
), . . . , A(ωn
)i back to the
• This means that to covert from the valueshA(1), A(ωn ), A(ωn
coefficient form A(x) = A0 + A1 x + A2 x2 + An−1 xn−1 we can use the same FFT algorithm with
−1
the only change that the root of unity ωn is replaced by ωn
= e−i
2π
n
, and that the resulting
values are divided by n.
COMP3121/3821/9101/9801
27 / 41
IFFT algorithm
Inverse Transform:
1: function IFFT(A)
2:
n ← length[A]
3:
if n = 1 then return A
4:
else
5:
A[0] ← (A0 , A2 , . . . An−2 );
6:
A[1] ← (A1 , A3 , . . . An−1 );
7:
y [0] ← F F T (A[0] );
8:
y [1] ← F F T (A[1] );
2π
9:
ωn ← e−i n ;
10:
ω ← 1;
11:
for k = 0 to k = n
− 1 do;
2
[0]
[1]
12:
yk ← yk + ω · yk ;
[0]
[1]
13:
y n +k ← yk − ω · yk
2
14:
ω ← ω · ωn ;
15:
end for
y
16:
return n
;
17:
end if
18: end function
⇐
different from FFT
⇐ different from FFT
COMP3121/3821/9101/9801
28 / 41
Interpretation of DFT
We have followed the textbook (CLRS);
however, what CLRS calls DFT, namely, the sequence
0
1
2
n−1
hA(ωn
), A(ωn
), A(ωn
), . . . , A(ωn
)i
is usually considered the Inverse Discrete Fourier Transform (IDFT) of the sequence
of the coefficients
hA0 , A1 , A2 , . . . , An−1 i
of the polynomial A(x);
−(n−1)
0
−1
−2
hA(ωn
), A(ωn
), A(ωn
), . . . , A(ωn
)i
is considered the “forward operation” i.e., the DFT.
taking this as the “forward operation” has an important conceptual advantage and is
used more often than the textbook’s choice.
COMP3121/3821/9101/9801
29 / 41
Interpretation of DFT
Another “tweak” of DFT: note that

1


 1


 1


 .

 .
 .

1
1
1
ωn
2
ωn
2·2
ωn
2
ωn
.
.
.
n−1
ωn
.
.
.
2(n−1)
ωn
...
1
...
n−1
ωn
2·(n−1)
ωn
...
.
.
.
...
.
.
.
(n−1)(n−1)
ωn

1


 1


 1


 .

 .
 .

1
1
1
...
−1
ωn
−2
ωn
−2
ωn
−2·2
ωn
...
.
.
.
−(n−1)
ωn
.
.
.
−2(n−1)
ωn
...
.
.
.
...


1
−(n−1)
ωn
−2·(n−1)
ωn
.
.
.
−(n−1)(n−1)
ωn
1


 0


 0
= n

 .

 .
 .

0
0
...
0
1
0
...
0
0
1
...
0
.
.
.
.
.
.
.
.
.
.
.
.
0
0
0
...
1



























implies:
COMP3121/3821/9101/9801
30 / 41
Interpretation of DFT

 1
√
 n




 √1
 n




 √1

 n


 .
 .
 .




√1
n
.
.
.
√1
n
√1
n
ωn
√
n
ω2
√n
n
.
.
.
ω2
√n
n
2·2
ωn
√
n
.
.
.
.
.
.
.
.
.
.
.
.
n−1
ωn
√
n
2(n−1)
ωn
√
n
.
.
.

√1
n
 1
 √
 n



n−1

ωn
  √1
√
 n
n



2·(n−1)  
ωn
  √1
√

 n
n


.
 .
.
 .
 .
.



(n−1)(n−1)  
ωn
√
√1
n
n

.
.
.
√1
n
√1
n
−1
ωn
√
n
−2
ωn
√
n
.
.
.
−2
ωn
√
n
−2·2
ωn
√
n
.
.
.
.
.
.
.
.
.
.
.
.
−(n−1)
ωn
√
n
−2(n−1)
ωn
√
n

.
.
.
√1
n
1


 0


 0
= 

 .

 .
 .

0
0
0
1
0
.
.
.






−(n−1)

ωn

√

n



−2·(n−1) 
ωn

√


n


.

.


.



−(n−1)(n−1) 
ωn
√
n

0
...
0


0
...
0 


1
...
0 


.
.
. 

.
.
. 
.
.
. 

0
...
1
Thus, these two matrices are inverses of each other.
COMP3121/3821/9101/9801
31 / 41
Interpretation of DFT
This motivates us to “tweak” the definition of DFT:
COMP3121/3821/9101/9801
32 / 41
Interpretation of DFT
This motivates us to “tweak” the definition of DFT:
Given a sequence of numbers (A0 , A1 , . . . , An−1 ) the Discrete Fourier Transform of
this sequence is the sequence of the values of the polynomial
1
x
xn−1
A∗ (x) = A0 √ + A1 √ + . . . + An−1 √
n
n
n
−k
−k
for x = ωn
for k = 0, . . . , n − 1; i.e., the sequence of values A∗ (ωn
):
−k
A∗ (ωn
) = A0
−k 0
−k 1
−k n−1
(ωn
)
(ωn
)
(ωn
)
+ A1 √
+ . . . + An−1
√
√
n
n
n
COMP3121/3821/9101/9801
32 / 41
Interpretation of DFT
This motivates us to “tweak” the definition of DFT:
Given a sequence of numbers (A0 , A1 , . . . , An−1 ) the Discrete Fourier Transform of
this sequence is the sequence of the values of the polynomial
1
x
xn−1
A∗ (x) = A0 √ + A1 √ + . . . + An−1 √
n
n
n
−k
−k
for x = ωn
for k = 0, . . . , n − 1; i.e., the sequence of values A∗ (ωn
):
−k
A∗ (ωn
) = A0
−k 0
−k 1
−k n−1
(ωn
)
(ωn
)
(ωn
)
+ A1 √
+ . . . + An−1
√
√
n
n
n
Given a sequence of numbers (A0 , A1 , . . . , An−1 ) the Inverse Discrete Fourier
Transform of this sequence is the sequence of the values of the same polynomial
1
x
xn−1
A∗ (x) = A0 √ + A1 √ + . . . + An−1 √
n
n
n
k for k = 0, . . . , n − 1; i.e., the sequence of values A∗ (ω k )
but for x = ωn
n
(ω k )0
(ω k )1
(ω k )n−1
k
A∗ (ωn
) = A0 √n + A1 √n + . . . + An−1 n√
n
n
n
COMP3121/3821/9101/9801
32 / 41
Interpretation of DFT
1: function FFT(A)
2:
n ← length[A]
3:
if n = 1 then return A
4:
else
5:
A[0] ← (A0 , A2 , . . . An−2 );
6:
A[1] ← (A1 , A3 , . . . An−1 );
7:
y [0] ← F F T (A[0] );
8:
y [1] ← F F T (A[1] );
−i 2π
n ;
9:
ωn ← e
10:
ω ← 1;
− 1 do;
11:
for k = 0 to k = n
2
[0]
[1]
12:
yk ← yk + ω · yk ;
[0]
[1]
13:
y n +k ← yk − ω · yk
2
14:
ω ← ω · ωn ;
15:
end for
16:
return √y ;
n
17:
end if
18: end function
1: function IFFT(A)
2:
n ← length[A]
3:
if n = 1 then return A
4:
else
5:
A[0] ← (A0 , A2 , . . . An−2 );
6:
A[1] ← (A1 , A3 , . . . An−1 );
7:
y [0] ← F F T (A[0] );
8:
y [1] ← F F T (A[1] );
i 2π
9:
ωn ← e n ;
10:
ω ← 1;
− 1 do;
11:
for k = 0 to k = n
2
[0]
[1]
12:
yk ← yk + ω · yk ;
[0]
[1]
13:
y n +k ← yk − ω · yk
2
14:
ω ← ω · ωn ;
15:
end for
16:
return √y ;
n
17:
end if
18: end function
COMP3121/3821/9101/9801
33 / 41
Interpretation of DFT
scalar product (also called dot product) of two vectors with real coordinates,
~
x = (x0 , x1 , . . . , xn−1 ) and ~
y = (y0 , y1 , . . . , yn−1 ), denoted by h~
x, ~
y i is defined as
h~
x, ~
yi =
n−1
X
xi yi
i=0
If the coordinates of our vectors are complex numbers, then the scalar product of such
two vectors is defined as
n−1
X
h~
x, ~
yi =
xi yi
i=0
where z denotes the complex conjugate of z, i.e., a + i b = a − i b.
COMP3121/3821/9101/9801
34 / 41
Interpretation of DFT
Note that
2πk
2πk
2πk
2πk
+ i sin
= cos
− i sin
n
n
n
n
−2πk
−2πk
−i 2πk
−k
n
= cos
= ωn
+ i sin
=e
n
n
k = ei
ωn
2πk
n
= cos
Thus, what we had before,

1
k
ωn
2·k
ωn



k·(n−1)

. . . ωn



1
−m
ωn
−2m
ωn
..
.
−(n−1)m
ωn


(
 n−1
X (k−m)j

n if k = m
=
ω
=
n

0 if k 6= m

j=0

(4)
k , ω 2·k , . . . , ω k·(n−1) and
simply means that for k 6= m vectors 1, ωn
n
n
m , ω 2·m , . . . , ω m·(n−1) are orthogonal.
1, ωn
n
n
COMP3121/3821/9101/9801
35 / 41
Interpretation of DFT
If we define
1 k·(n−1)
k
2·k
1, ωn
, ωn
, . . . , ωn
e~k = √
n
COMP3121/3821/9101/9801
36 / 41
Interpretation of DFT
If we define
then
1 k·(n−1)
k
2·k
1, ωn
, ωn
, . . . , ωn
e~k = √
n
p
ke~k k = he~k , e~k i = 1
COMP3121/3821/9101/9801
36 / 41
Interpretation of DFT
If we define
then
and
1 k·(n−1)
k
2·k
1, ωn
, ωn
, . . . , ωn
e~k = √
n
p
ke~k k = he~k , e~k i = 1
he~k , e~m i = 0 for
k 6= m
COMP3121/3821/9101/9801
36 / 41
Interpretation of DFT
If we define
then
and
1 k·(n−1)
k
2·k
1, ωn
, ωn
, . . . , ωn
e~k = √
n
p
ke~k k = he~k , e~k i = 1
he~k , e~m i = 0 for
k 6= m
thus, the set of vectors {e~0 , e~1 , . . . , e~n } is an orthonormal base of the vector space of
all complex valued sequences of length n.
COMP3121/3821/9101/9801
36 / 41
Interpretation of DFT
If we define
then
and
1 k·(n−1)
k
2·k
1, ωn
, ωn
, . . . , ωn
e~k = √
n
p
ke~k k = he~k , e~k i = 1
he~k , e~m i = 0 for
k 6= m
thus, the set of vectors {e~0 , e~1 , . . . , e~n } is an orthonormal base of the vector space of
all complex valued sequences of length n.
~ = (A0 , A1 , A2 , . . . , An−1 ); then for the DFT of this sequence we have
Let A
1
A0 −k 0
A1 −k 1
A2 −k 2
An−1 −k n−1
−k
) = √ (ωn
) + √ (ωn
) + √ (ωn
) + . . . + √ (ωn
)
√ A(ωn
n
n
n
n
n
(ω k )0
(ω k )1
(ω k )2
(ω k )n−1
= A0 √n + A1 √n + A2 √n + . . . + An−1 n√
n
n
n
n
k 0
k )1 (ω k )2
(ωn ) (ωn
(ω k )n−1
n
= (A0 , A1 , A2 , . . . , An−1 ) ,
√ , √ , √ , . . . , n√
n
n
n
n
~ e~k i
= hA,
COMP3121/3821/9101/9801
36 / 41
Interpretation of DFT
If we define
then
and
1 k·(n−1)
k
2·k
1, ωn
, ωn
, . . . , ωn
e~k = √
n
p
ke~k k = he~k , e~k i = 1
he~k , e~m i = 0 for
k 6= m
thus, the set of vectors {e~0 , e~1 , . . . , e~n } is an orthonormal base of the vector space of
all complex valued sequences of length n.
~ = (A0 , A1 , A2 , . . . , An−1 ); then for the DFT of this sequence we have
Let A
1
A0 −k 0
A1 −k 1
A2 −k 2
An−1 −k n−1
−k
) = √ (ωn
) + √ (ωn
) + √ (ωn
) + . . . + √ (ωn
)
√ A(ωn
n
n
n
n
n
(ω k )0
(ω k )1
(ω k )2
(ω k )n−1
= A0 √n + A1 √n + A2 √n + . . . + An−1 n√
n
n
n
n
k 0
k )1 (ω k )2
(ωn ) (ωn
(ω k )n−1
n
= (A0 , A1 , A2 , . . . , An−1 ) ,
√ , √ , √ , . . . , n√
n
n
n
n
~ e~k i
= hA,
~ is simply the sequence of projections of A
~ onto the base
Thus, the DFT of a vector A
vectors e~k , (k = 0, . . . , n − 1).
COMP3121/3821/9101/9801
36 / 41
Interpretation of DFT
~ can
In an n-dimensional vector space V with an orthonormal base B every vector A
be represented as a linear combination of the base vectors with coefficients equal to
~ onto the base vectors, i.e., the scalar product hA,
~ ek i:
the projections of A
c
=
<c
,e
1
>
e
1
+
<c
,e
2
>
e
2
+
<c,e3>
<c
,e
3
>
e
3
~ = hA,
~ e~0 ie~0 + hA,
~ e~1 ie~1 + . . . + hA,
~ ~en−1 i~en−1
A
e2
e3
e1
>
,e 2
<c
< c,e1>
Representing vector c as a linear combination of the
basis vectors e1,e2,e3 with projections as coefficients
COMP3121/3821/9101/9801
37 / 41
Interpretation of DFT
Thus, in our case
~ = hA,
~ e~0 ie~0 + hA,
~ e~1 ie~1 + . . . + hA,
~ ~en−1 i~en−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
COMP3121/3821/9101/9801
)
~en−1
38 / 41
Interpretation of DFT
Thus, in our case
~ = hA,
~ e~0 ie~0 + hA,
~ e~1 ie~1 + . . . + hA,
~ ~en−1 i~en−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
)
~en−1
Looking at the kth cordnate of both the left and the right side we get
Ak =
n−1
0 ) (ω k )0
k )n−1
A(ωn
) (ωn
A(ω 1 ) (ω k )1
A(ωn
√
√n + √ n √n + . . . +
√
√
n
n
n
n
n
n
COMP3121/3821/9101/9801
38 / 41
Interpretation of DFT
Thus, in our case
~ = hA,
~ e~0 ie~0 + hA,
~ e~1 ie~1 + . . . + hA,
~ ~en−1 i~en−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
)
~en−1
Looking at the kth cordnate of both the left and the right side we get
Ak =
n−1
0 ) (ω k )0
k )n−1
A(ωn
) (ωn
A(ω 1 ) (ω k )1
A(ωn
√
√n + √ n √n + . . . +
√
√
n
n
n
n
n
n
Ak is obtained evaluating the polynomial
COMP3121/3821/9101/9801
38 / 41
Interpretation of DFT
Thus, in our case
~ = hA,
~ e~0 ie~0 + hA,
~ e~1 ie~1 + . . . + hA,
~ ~en−1 i~en−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
)
~en−1
Looking at the kth cordnate of both the left and the right side we get
Ak =
n−1
0 ) (ω k )0
k )n−1
A(ωn
) (ωn
A(ω 1 ) (ω k )1
A(ωn
√
√n + √ n √n + . . . +
√
√
n
n
n
n
n
n
Ak is obtained evaluating the polynomial
n−1
0)
A(ωn
A(ω 1 ) x
A(ωn
) xn−1
+ √ n √ + ... +
√
√
√
n
n
n
n
n
COMP3121/3821/9101/9801
38 / 41
Interpretation of DFT
Thus, in our case
~ = hA,
~ e~0 ie~0 + hA,
~ e~1 ie~1 + . . . + hA,
~ ~en−1 i~en−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
)
~en−1
Looking at the kth cordnate of both the left and the right side we get
Ak =
n−1
0 ) (ω k )0
k )n−1
A(ωn
) (ωn
A(ω 1 ) (ω k )1
A(ωn
√
√n + √ n √n + . . . +
√
√
n
n
n
n
n
n
Ak is obtained evaluating the polynomial
n−1
0)
A(ωn
A(ω 1 ) x
A(ωn
) xn−1
+ √ n √ + ... +
√
√
√
n
n
n
n
n
k , which is exactly what the Inverse Discrete Fourier Transform does.
at x = ωn
COMP3121/3821/9101/9801
38 / 41
Interpretation of DFT
Let us denote the usual orthonormal base of Cn by B:
f~0 = (1, 0, 0, 0, . . . , 0), f~1 = (0, 1, 0, 0, . . . , 0), f~2 = (0, 0, 1, 0, . . . , 0), f~n−1 = (0, 0, 0, 0, . . . , 1)
k·(n−1)
ωk
ω 2·k
and by F the base F = {~e0 , ~e1 , . . . , ~en−1 } where e~k = √1n , √nn , √nn , . . . , ωn √n
.
COMP3121/3821/9101/9801
39 / 41
Interpretation of DFT
Let us denote the usual orthonormal base of Cn by B:
f~0 = (1, 0, 0, 0, . . . , 0), f~1 = (0, 1, 0, 0, . . . , 0), f~2 = (0, 0, 1, 0, . . . , 0), f~n−1 = (0, 0, 0, 0, . . . , 1)
k·(n−1)
ωk
ω 2·k
and by F the base F = {~e0 , ~e1 , . . . , ~en−1 } where e~k = √1n , √nn , √nn , . . . , ωn √n
.
then
~ = (A0 , A1 , A2 , . . . , An−1 ) = A0 f~0 + A1 f~1 + A2 f~2 + . . . + An−1 f~n−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
COMP3121/3821/9101/9801
)
~en−1
39 / 41
Interpretation of DFT
Let us denote the usual orthonormal base of Cn by B:
f~0 = (1, 0, 0, 0, . . . , 0), f~1 = (0, 1, 0, 0, . . . , 0), f~2 = (0, 0, 1, 0, . . . , 0), f~n−1 = (0, 0, 0, 0, . . . , 1)
k·(n−1)
ωk
ω 2·k
and by F the base F = {~e0 , ~e1 , . . . , ~en−1 } where e~k = √1n , √nn , √nn , . . . , ωn √n
.
then
~ = (A0 , A1 , A2 , . . . , An−1 ) = A0 f~0 + A1 f~1 + A2 f~2 + . . . + An−1 f~n−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
)
~en−1
DFT is just change of base operation: it transforms the sequence of coordinates
(A0 , A1 , A2 , . . . , An−1 )
in the base B of a vector A into the sequence
−(n−1)
0 ) A(ω −1 )
A(ωn
A(ωn
n
,...,
√ , √
√
n
n
n
)
!
~ in the base F ;
of the coordinates of A
COMP3121/3821/9101/9801
39 / 41
Interpretation of DFT
Let us denote the usual orthonormal base of Cn by B:
f~0 = (1, 0, 0, 0, . . . , 0), f~1 = (0, 1, 0, 0, . . . , 0), f~2 = (0, 0, 1, 0, . . . , 0), f~n−1 = (0, 0, 0, 0, . . . , 1)
k·(n−1)
ωk
ω 2·k
and by F the base F = {~e0 , ~e1 , . . . , ~en−1 } where e~k = √1n , √nn , √nn , . . . , ωn √n
.
then
~ = (A0 , A1 , A2 , . . . , An−1 ) = A0 f~0 + A1 f~1 + A2 f~2 + . . . + An−1 f~n−1
A
−(n−1)
=
−1
0)
A(ωn
)
A(ωn
A(ωn
e~1 + . . . +
√ e~0 + √
√
n
n
n
)
~en−1
DFT is just change of base operation: it transforms the sequence of coordinates
(A0 , A1 , A2 , . . . , An−1 )
in the base B of a vector A into the sequence
−(n−1)
0 ) A(ω −1 )
A(ωn
A(ωn
n
,...,
√ , √
√
n
n
n
)
!
~ in the base F ;
of the coordinates of A
Inverse Discrete Fourier Transform (IDFT) transforms that sequence of the
~ in base B.
coordinates in the base F back to the sequence of coordinates of A
COMP3121/3821/9101/9801
39 / 41
Interpretation of DFT
Note that by replacing n with 2n + 1 we get
0
k
1
k
2n
k
A(ω2n+1
A(ω2n+1
A(ω2n+1
) (ω2n+1
)0
) (ω2n+1
)1
) (ω2n+1
)2n
√
√
√
Ak = √
+ √
+ ... + √
2n + 1
2n + 1
2n + 1
2n + 1
2n + 1
2n + 1
2
2n
A(ω2n+1
A(ω2n+1
) i 2π k·2
) i 2π k·2n
i 2π k·1
e 2n−1 +
e 2n−1 + . . . +
e 2n−1
2n + 1
2n + 1
2n + 1
2n + 1
j
j
n
n X
X
2π k·j
j
A(ω2n+1
) i 2π k·j
A(ω2n+1 ) i arg(A(ω2n+1
)) i 2n−1
=
e 2n−1 =
e
e
2n + 1
2n + 1
j=−n
j=−n
!!
j
n
X A(ω2n+1 ) i 2π k·j +arg(A(ωj
2n+1 ))
=
e 2n−1
2n + 1 =
k
A(ω2n+1
)
e
k·0
i 2π
2n−1
+
1
A(ω2n+1
)
j=−n
1!
COMP3121/3821/9101/9801
40 / 41
Interpretation of DFT
If we let
j
n j
t·j
X
+arg(A(ω2n+1 ))
A(ω2n+1 ) i 2π
e 2n−1
2n + 1 j=−n
j
n
X A(ω2n+1
) 2π · j
2π · j
j
j
=
t + arg(A(ω2n+1
t + arg(A(ω2n+1
)) + i sin
))
cos
2n + 1 2n − 1
2n − 1
j=−n
a(t) =
then Ak = a(k). Thus the sequence hA0 , A1 , . . . , A2n i has been represented as a linear
2πj
for j = −n to j = n.
combination of samples of sinusoids of frequencies 2n+1
COMP3121/3821/9101/9801
41 / 41