Download Report

Journal of Information & Computational Science 11:16 (2014) 5855–5862
Available at http://www.joics.com
November 1, 2014
Image Defogging Approach Based on Dark Channel
Prior with Feedback Mechanism ⋆
Jianlong Xu ∗, Xinfu Chen, Yaming Wang
School of Informatics, Zhejiang Sci-Tech University, Hangzhou 310018, China
Abstract
Aimed at degradation of image gotten in misty weather, an image defogging algorithm based on dark
channel prior with feedback mechanism is proposed. This algorithm can not only retain excellent
treatment eﬀect of the dark channel prior defogging algorithm on the image of non bright areas but also
restore scene perfectly from the image with bright areas. Experimental results show that the algorithm
can eﬃciently alleviate the color distortion caused in process of defogging the image with bright areas
using dark channel prior algorithm, and make the image defogged smoother and more natural.
Keywords: Dark Channel Prior; Defog; Color Coeﬃcient of Variation
1
Introduction
In most cases, the outdoor computer vision system works eﬀectively under normal circumstance
in which system can obtain the image feature clearly and accurately. However, computer vision
system is sensitive to weather, light, and many other factors. An eﬀective image defogging
algorithm plays a very important role in improving system’s robustness and reliability in extreme
weather of fog.
Recently defogging methods for fog-degraded images are of two main types: image enhancement
and image restoration. Image enhancement method is to improve the visual eﬀect of image by
enhancing the contrast and highlighting the details, such as histogram equalization algorithm [1]
and Retinex theory [2] method, but the processing results with these methods were disappointing.
Another kind of method, image restoration is based on the atmospheric physical model which
analyzes light’s scattering and attenuation during the physical process of transmission. This
atmospheric model can more accurately express the inﬂuence from fog, dust, mist and so on.
Fattal [3] proposed a reﬁned image formation model to account for the surface shading and the
scene transmission, he assumed that the transmission and shading are locally uncorrelated and
broke the image into regions of constant albedo, from which the scene can be inferred. However,
this approach may be failed in the cases that the assumption is broken. Tan [4] found that the
⋆
∗
Project supported by the National Nature Science Foundation of China (No. 61272311).
Corresponding author.
Email address: [email protected] (Jianlong Xu).
1548–7741 / Copyright © 2014 Binary Information Press
DOI: 10.12733/jics20104891
5856
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
clear-day images have higher contrast compared with fog-degraded images, and he removed the
haze by maximizing the local contrast of the restored image. Though it produced a remarkable
eﬀect, may be not physically sound. Based on a great deal of statistics data He [5] proposed dark
channel prior, that is in a image most local regions have very low intensity in at least one color
channel. He used the statistical regularity to eﬀectively estimate the transmission, then reﬁned
the transmission with soft matting and ﬁnally got the output image.This algorithm is simple and
eﬃcient. However, when it is used to handle the image with bright areas, dark channel prior is
no longer valid and the restored image often suﬀers from color distortion. To solve this problem,
the causes of distortion in color are analyzed, and then an improved algorithm with feedback
mechanism is proposed.
2
2.1
Defogging Using Dark Channel Prior
Physical Model for Light’s Scattering and Attenuation [6]
In computer vision and image processing, light’s scattering and attenuation during the physical
process of transmission can be expressed as a physical model which is widely used to explain the
formation of a fogged image:
I(x) = J(x)e−rd(x) + A(1 − e−rd(x) )
(1)
where d(x) is the scene depth, r is the medium extinction coeﬃcient. Eq. (1) can be converted
into Eq. (2) by replacing exponential attenuation e−rd(x) with transmission t(x):
I(x) = t(x)J(x) + A(1 − t(x))
(2)
where I(x) is the input image, t(x) is transmission function, vector A is the global atmospheric
light which has nothing to do with image coordinates and usually can be treated as a constant
value, J(x) is the scene need to be restored. So the key to recover the scene J(x) is how to solve
the transmission t(x).
2.2
Defogging with Dark Channel Prior
The dark channel prior is a kind of statistics of the haze-free outdoor images. It is based on a key
observation that most local patches in haze-free outdoor images contain some pixels which have
very low intensities in at least one color channel. For an image J(x), its dark color channel can
be deﬁned as following:
J dark (x) = min ( min J c (y))
(3)
y∈Ω(x) c∈{r,g,b}
where J(c) is a color channel of an image, Ω(x) is the patch with the center x, in He’s paper he
did statistical analysis of quite a large number of pictures and arrived at the conclusion that the
dark color channel of an image tends to be zero: J dark → 0.
Further assuming that the transmission in a local patch Ω(x), is constant, then taking the min
operation among three color channels on Eq. (2), Eq. (4) can be obtained:
(
(
J c (y) )
I c (y) )
=
t(x)
min
min
+ 1 − t(x)
(4)
min min
c
c
y∈Ω(x)
y∈Ω(x)
Ac
Ac
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
5857
As mentioned before, J dark (x) = min ( min J c (y)) = 0. According to Eq. (3) and Eq. (4), a
y∈Ω(x) c∈{r,g,b}
rough estimate of transmission map can be gotten as Eq. (5).
t(x) = 1 − min
y∈Ω(x)
(
min
c
I c (y) )
Ac
(5)
In this paper, guided ﬁlter [7] instead of soft matting is used to reﬁne the transmission map
eﬃciently. Then, the ﬁnal scene J(x) is recovered by:
J(x) =
I(x) − A
+A
t(x)
(6)
where A consists of 3 equivalent components, the component equals the mean value of the top
0.1% maximum gray value of pixels in the dark channel.
2.3
Defect of Defogging with Dark Channel Prior
Fig. 1 shows two groups of defogging results. Fig. 2 shows histograms of intensities of pixels in
dark channels.
As Fig. 2 shows that majority of pixel intensities are of very small value. This statistic gives
very strong support to dark channel prior and He’s method produces impressive visual results
without distortion in color.
(a)
(b)
(c)
(d)
Fig. 1: Defogging results using dark channel prior. (a) Input images; (b) Defogged images; (c) Transmission maps; (d) Dark channels
Dark channel prior is a powerful prior based on the statistics of quite a lot of images, most
of which are non-sky or non bright area images. When the scene objects are inherently similar
to the atmospheric light such as sky regions, the dark channel prior is invalid. Recovered result
by dark channel prior from the image with sky regions suﬀers from color distortion as shown in
Fig. 3.
It can be seen according to Fig. 3 (d) that intensities are distributed randomly and a portion
of the pixel values of intensity are even higher than 200. This is because that intensities of the
bright region pixels in three channels are high. Dark channel prior theory is no longer suitable for
handling bright regions. Jiang et al. [8] did a lot of experiments and his experiments show that
non-bright region is restored quite well with dark channel prior algorithm, but fatal noise and
color distortion are prone to be caused while this algorithm is used to process the bright regions
5858
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
6000
8000
7000
6000
5000
4000
3000
2000
1000
0
5000
4000
3000
2000
1000
0
0
50
100
150
200
250
0
50
(a)
100
150
200
250
(b)
Fig. 2: Statistics of the dark channels. (a) Histogram of the intensity of the pixels in image of woods;
(b) Histogram of the intensity of the pixels in image of ducks
(a)
(b)
12000
10000
8000
6000
4000
2000
0
0
(c)
50
100
150
(d)
200
250
Fig. 3: Color distortion caused by defogging. (a) Input image; (b) The defogged image; (c) Dark channel;
(d) Histogram of the intensity of the pixels in dark channel
such as sky region, large white object and water. This is because that the actual transmission
in bright regions is greater than the transmission estimated by dark channel prior, moreover, the
higher the brightness, the greater is the transmission deviation.
3
Image Defogging Approach Based on Dark Channel
Prior with Feedback Mechanism
On the basis of above discussion, an improved algorithm with feedback mechanism is proposed.
The algorithm process is described as follows:
Step 1. Defog the input image with dark channel prior and get the estimated transmission map.
Step 2. Compare the two images before and after defogging, segment the distorted area by color
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
5859
diﬀerence.
Step 3. Adjust the transmission in color distortion area segmented.
Step 4. Reﬁne the transmission with guided ﬁlter.
Step 5. Recover the scene with the ﬁnal reﬁned transmission.
3.1
Color Diﬀerence between Images before and after Defogged
Color is an important feature of an image, it contains the essential information of an image. The
coeﬃcient of variation is used to measure the variation of a large number of independent data. It
is calculated from the average and standard deviation as follows:
cv =
S tandard Deviation
Average
(7)
Since pixel intensities of three channels in bright regions are close to each other, color coeﬃcient
of variation Ccv is deﬁned as following to describe color diﬀerence of three channels:

N
∑



(I R + I G + I B )



i=1

µ=


3N

√

N

1 ∑


[(I R − µ)2 + (I G − µ)2 + (I B − µ)2 ]


3N

i=1


 ccv =
µ
(8)
where N is the number of pixels within a patch. According to the deﬁnition, a bright white
patch’s Ccv tends to be zero while a colorful patch’s Ccv tends to be much larger. c is deﬁned
as follows to describe the color diﬀerence between images before and after defogged with dark
channel prior. Where Ccv1 is the color coeﬃcient of variation of input image, Ccv2 is the color
coeﬃcient of variation of the image defogged.
c=
|Ccv1 − Ccv2 |
Ccv1 + Ccv2
(9)
It can be seen from Eq. (9) that the larger the diﬀerence between color coeﬃcients of variation,
the larger the value of c is. When the value of c exceeds the threshold, the patch in an image
defogged is identiﬁed as color distortion.
Fig. 4 (a) shows the color diﬀerence map between input image and defogged image, where the
brighter the region, the more severe the distortion is. The color diﬀerence map is converted to
binary one as Fig. 4 (b) shown by suitable threshold. The white regions of Fig. 4 (b) corresponding
to the regions where color distortion are severe, are the areas in which transmissions need to be
adjusted.
5860
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
(a)
(b)
Fig. 4: Color diﬀerence map and color distortion region segmented. (a) Color diﬀerence map; (b) Color
distortion regions
3.2
Adjusting the Transmissions in Color Distortion Regions
The actual transmission can be calculated by Eq. (10):
(
( c ))
1 − min min I A(y)
c
c
y∈Ω(x)
(
tactual =
( c ))
1 − min min J A(y)
c
c
(10)
y∈Ω(x)
1400
1200
1000
800
600
400
200
0
0
he
rs
ot
-0
.1
09
-0
.
09
0.
08
0.
08
-0
.
07
0.
0.
06
-0
.0
7
06
5
-0
.
05
0.
0.
04
-0
.0
04
-0
.
03
0.
-0
.0
02
02
-0
.
0.
01
0.
00.
3
Patch numbers
01
Distribution of Ccv
The dark channel prior is invalid and the denominator cannot be approximately equal to 1 for
the white areas. As a result, actual transmission tactual is larger than the estimated t(x). As
introduction above, the brighter the local region, the more severe the distortion, the less the Ccv
is. The values of Ccv corresponding to all distorted patches in lots of image defogged with dark
channel prior are collected and the statistics show that most values of Ccv have a range of 0 to
0.1. For example, the distribution of Ccv for Fig. 3 (a) is illustrated in Fig. 5.
Fig. 5: Distribution histogram of Ccv
Because the transmission adjusted should share the same range with original transmission which
is less than or equal to 1 (see Eq. (5)), gamma function is used to adjust the transmission in this
paper.
e
t(x) = t(x)min(10Ccv1 ,1)
(11)
According to the above transformation, the brighter the region, the less the Ccv , the larger the
transmission is increased. The adjusted transmission map is illustrated in Fig. 6 (b).
4
Experiment and Analysis
The images with bright region selected from baidu are used to make contrast tests for He’s
algorithm and our algorithm. For the image of grasslands, the patch size is set to 8 × 8 and the
5861
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
(a)
(b)
Fig. 6: Contrast of transmission before and after adjusted. (a) He’s transmission map; (b) Adjusted
transmission map
(a)
(b)
(c)
Fig. 7: Comparison with He’s work. (a) Input images; (b) He’s results; (c) Our results
×104
2.5
×104
2.5
×104
2.5
2.0
2.0
2.0
1.5
1.5
1.5
1.0
1.0
1.0
0.5
0.5
0.5
0
0
50
100
150
200
250
×104
0
0
50
100
150
200
250
×104
0
0
3.0
3.0
3.0
2.5
2.5
2.5
2.0
2.0
2.0
1.5
1.5
1.5
1.0
1.0
1.0
0.5
0.5
0.5
0
0
50
100
150
(a)
200
250
0
0
50
100
50
100
150
200
250
150
200
250
×104
50
100
150
(b)
200
250
0
0
(c)
Fig. 8: Color histograms (a) Histograms of input; (b) He’s histograms; (c) Our histograms
threshold is set to 0.65, 100% of Ccv corresponding to distorted areas have a range of 0 to 0.1.
For the image of mountains, the patch size is set to 10 × 10 and the threshold is set to 0.6, about
91.04% of Ccv corresponding to distorted areas have a range of 0 to 0.1. During guided ﬁltering,
guidance image is the grayscale image of input, local window radius r is set to 30, regularization
parameter ε is set to 10−6 . In Fig. 7, we compare our approach with He’s work.
5862
J. Xu et al. / Journal of Information & Computational Science 11:16 (2014) 5855–5862
The contrast tests show that the bright sky regions restored look smoother and more natural
in our work than He’s. The reason is that this algorithm relieves the color distortion in bright
sky regions by adjusting the transmission with gamma function.
A good defogging approach should restore the image clearly and recover maximal information
of the original image. Image histograms before and after defogged should maintain a high degree
of similarity [9]. Fig. 8 shows color histograms of three channels, the top 3 charts are histograms
of grassland and the bottom 3 charts are histograms of mountains. Fig. 8 indicates that our
histograms can maintain a higher degree of similarity to the histograms of input images than
He’s histograms especially in bright regions such as sky where the three channels intensities are
high.
5
Conclusion
An improved approach with feedback mechanism is proposed to solve the color distortion caused
in process of defogging images with bright areas by dark channel prior. By means of comparing
images before and after defogged with dark channel prior, color distortion regions are segmented
by color diﬀerence, then the transmissions corresponding to distortion regions are adjusted using
gamma function, and ﬁnally the scene is restored with adjusted transmission. Experimental
results show that the algorithm can eﬃciently alleviate the color distortion caused in process of
defogging on the image with bright areas using dark channel prior algorithm, and make the image
defogged smoother and more natural.
References
[1]
Yishu Zhai, Xiaoming Liu, Yayuan Tu, Yaning Chen, An improved fog-degraded image clearness
algorithm [J], Journal of Dalian Maritime University, 33(3), 2007, 55-58
[2]
Daniel J. Jobson, Zia-ur Rahman, Properties and performance of a center/surround retinex [J],
IEEE Transactions on Image Processing, 6(3), 1997, 451-454
[3]
R. Fattal, Single image dehazing [J], SIGGRAPH, ACM Press, 2008, 1-9
[4]
R. Tan, Visibility in bad weather from a single image [C], IEEE Conference on Computer Vision
and Pattern Recognition, 2008, 1-8
[5]
Kaiming He, Jian Sun, Xiaoou Tang, Single image haze removal using dark channel prior [C],
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2009, 1956-1963
[6]
E. J. MeCartney, Opties of the Atmosphere: Seattering by Molecules and Partieles [M], NewYork:
John Wiley and Sons, 1976, 23-32
[7]
K. He, J. Sun, X. Tang, Guied image ﬁltering [C], Proceedings of the 11th European Conference
on Computer Vision: Part I, September 5-11, 2010, Heraklion, Crete, Greece, 1-14
[8]
Jianguo Jiang, Tianfeng Hou, Meibin Qi, Improved image defogging algorithm based on dark color
prior [J], Journal of Circuits and Systems, 16(2), 2011, 7-12
[9]
Dapeng Li, Jing Yu, Chuangbai Xiao, No-reference quality assessment method for defogged images
[J], Journal of Image and Graphics, 16(9), 2011, 1753-1757