2015/05/19
画像認識技術を活用して生物多様性を読み解く
ー生物多様性創出機構としての擬態現象ー!
•  HASHIMOTO, Yoshiaki, M. OHASHI, T. KIMURA, and H.
IKENO
Ant-mimics are particularly abundant and diverse in tropical
forests
Ant mimicking spiders
Ant species number of Lambir
Through the Looking–Glass : reflection of ant-diversity in ant-mimics
Aenictus inflatus
In the tropics, we can frequently encounter ant-mimics
Application of computer vision technology to study ant-mimicry
Batesian mimicry
- Spiders mimic ants to escape predation Invertebrate predators have innate (native) aversion for ants
Ant silhouette
Avoidance
!  We had been continuing ant inventory in
Southeast Asian tropical o for 20 years.
Ant-mimicry: But not limited to defense strategy
! Mimicry as a potential driving force of species diversification
（非学習的にアリ形を避ける）
Avoidance
Ant
!  Ant-mimicry as powerful driving force of evolution
Avoidance
Ant-mimicking spider
Jumping spider
（ハエトリグモ）
Predation
Non-mimicking spider
Huang et al. (2011) "Salticid predation as one potential
driving force of ant mimicry in jumping spiders."
Proceedings of the Royal Society 278: 1356-1364.
!  Ants are the dominant animals in terms of diversity and
biomass in tropics
Wasps and Mantises have also innate (native) aversion for ants, and avoids for ant-mimicking spiders
(Wasps: Edmunds, 1993, Mantises: Nelson et al,2006)
Ant-mimicry as a mechanism to create biodiversity
in tropics ?
“Mimic Study is not easy job!”
擬態の研究は手強いから！
6.1%
45.0%
65.9%
AMS
NMS
Ants
(Total 1,918 samples for 10 years)
Danum Valley
Sakaerate
Lambir Hills
アリ類の多様性が鋳型となって，同所のアリ擬態グモ類の多様性が創出されている
“Yet it is the least known”.
そんな研究は，だれもやってない．どうして？
Frequency of occurrence
1. Danum Valley, Sabah Borneo (2004 ∼2008)
2. Lambir Hills, Sarawak Borneo (2007 ∼)
3. Sakaerate, Thailand (2010 ∼)
1
2015/05/19
Ant-mimicking spiders
Relationships between species number of ants and antmimicking jumping spiders in research sites
Ya m a s a k i , & A h m a d .
"Taxonomic study of the
genus Myrmarachne of
Borneo (Araneae:
Salticidae)." Zootaxa (2013):
501-556.
(365 indiv.)
! 
“Mimic Study is not easy job!”
Method for estimating of mimetic resemblance
Human ranking （人の感覚で判定）
How to estimate mimetic resemblances
Model
Mimic
Rank is 7
and 2
!  Human volunteers (n=21) were shown each mimic photograph in random order on a
projector screen for 20 seconds, alongside model photograph.
!  Human subjects were asked to rank each mimic on a scale of 1 (very poor mimic) to 10
(excellent mimic) for the models
Penney, Heather D., et al. "A comparative analysis of the evolution of imperfect mimicry."
Nature 483.7390 (2012): 461-464.
Method for estimating of mimetic resemblance
To develop new method for mimetic resemblance
estimating
Landmark morphometric （幾何学形態学を使う）
!  We need to handle large samples of 565 ant spices and 62 species of
the mimicking spiders in tropical forests
!  We need objectively and quantitatively estimation to compare
mimetic patterns of ants and the mimicking spiders among different
communities.
!  Landmark placements for caterpillars and snakes in their dorsal and lateral views
We need to develop automatic operation systems for objective analysis
of mimetic resemblance
!  Landmarks were digitized on all photos using tpsDig2 software (Rohlf, 2006)
!  The morphospace occupied by caterpillars with eyespots wewe compared to snakes by PCA
Hossie, T. J., & Sherratt, T. N. (2014). “Does defensive posture increase mimetic fidelity of
caterpillars with eyespots to their putative snake models” Current Zoology, 60(1).
2
2015/05/19
Measurements of mimetic resemblances by using computer
vision technology
1.Body shape
Ant model
Ant model
Measuring the degree of
overlap area on body shape
Ant
Mimic
Original Image
Mimic
XA
Ls
model
Binary imaging
Image resizing
Image resizing
YA
SI =
XA
XA
Measuring body length
Ant
(keeping of aspect ratio)
Ls-|Ls-L|
Ls
Ls: Ant length L: AMS length
Mimic
look like me?
Binary imaging
Size fitting
(Histogram intersection; Swain,1991)
3.Body size
Original Image
Comparing hue-color
histogram of body color
model
Computer
vision
Mimic
Original Image
Binary Imaging
(Original software)
Mimic
2.Body color
Image processing steps for shape similarity analysis
L
Similarity Index : 0<SI<1
•  We developed tools to estimate objectively and quantitatively resemblance between ants and mimic spiders by image recognition
technique
Image processing steps for shape similarity analysis
Ant model
Mimic
Mimic
Image A
image B
image B
Binary images (silhouettes) were set on same canvas size with creating of same margin size
Measurement of body shape similarity for ant-mimicking
spiders to an ant-model
Ant mimicking spiders
Ant model
Rigid Transform
Tetraponera
attenuata
Adjusting location and angle of
image B, for maximization of
overlapping area of two images
(without shape or size altering)
Similarity Index
0<Similarity Index<1
Normalized Cross-Correlation
0.91
0.79
Measuring the degree of overlap area between two images
Programming for the image processing
(Open Source Computer Vision)
Module
Mimetic pattern of Myrmarachne alticephalon
High
! Mimicking spider
“import cv 2.”
OpenCV
Body shape similarity
il
sim
Python
y
arit
sim
il
TSI: 0.90
Free programing language
cv::getRotationMatrix2D
!  Library of modules for
computer vision (more
than 2,500 modules)
High
!  Interpreted, objected
programming
Rotation
cv::WarpAffine
Transformation
!  Easy-to-learn, because of
its clear syntax and
readability
!  Multi-language and crossplatform
!  Glue language
Crmo
Myrmaracne
alticephalon
Poph
similarity
!  OpenCV is written in C+
+ and its primary
interface is in C++ (now
full interfaces in Python,
Java etc.)
!  Presumed ant model
arit
y
Camponotus
saundersi
The most dominant
ant in Danum Valey
The most dominant
mimicking spider in
Danum Valey
Fig. 3D scatter diagram for similarity
indexes of shape, size and color to the
sympatric ant species
!  M. alticephalon have very precise resemblance to just one species of the dominant ants, Camponotus saundersi..
! 
3
2015/05/19
Mimetic pattern of Myrmarachne spider to conspicuous ants
Mimetic pattern of Myrmarachne acromegalis
- not dominant but conspicuous -
- Dominant to dominant -
Mimic spiders
Ant species
Canopy Layer, Lambir NP.
Mimic
M. cornuta
Mimic
Ant model
M. gedongensis
Mimic
Ant model
Ant model
Frequency of occurrence in
canopy layer, Lambir
pilosa
Frequency of occurrence in
canopy layer, Lambir
M. acromegalis
Gnamptogenys
menadensis
Tetraponera
Camponotus
sp.69
(Total 602 samples)
!  Dominant to dominant mimetic association, such as M. alticephalon in Danum, was also found in M. acromegalis in Lambir hill NP.
Conspicuous elongated
body-shape and sting ant
Danum Valey
Conspicuous red
body-color
and bad smelling ants
Danum Valey
! Specific mimetic pattern of Myrmarachne species also found for ant species, which is not dominant but having
conspicuous shape and/or color among sympatric ant species.
! 
! 
Intraspecific variation of body color among Polyrhachis- mimics in Danum Valley
Polyrhachis-mimic species showed
higher intraspecific variation of body
colors than non-polyrhachis mimic
species
M. grossa
M. maxillosa
Interspecific variation of body color among genus Polyrhachis in Danum Valley
Polyrhachis species showed higher
interspecific variation of body colors
than other ant groups.
from a resemblance to the warningly colored species,
Camponotus
P. muelleri
P. armata
M. maxillosa
P. laevigata
Polyrhachis
golden morph
P.javanica
M. biseratensis
M. sabahna
black morph
Crematogaster
P. tibialis
black & red morph
Camponotus!
M. malayana
Non-Polyrhachis
Mimicking species
Principal component
analysis (PCA) plot on body
color for mimicking spiders
Matching of body color variation between Polyrhachis-mimics and their model ants
Crematogaster
Principal component analysis
(PCA) plot on body color for
ant species
Polyrhachis
P. olybria
P. calypso
Intraspecific variation of body color among Polyrhachis- mimics in Danum Valley
Color variation of Polyrhachis-mimics
matched to the variation
of
Polyrhachis ants in same place.
M. grossa
Polyrhachis-mimic species showed
higher intraspecific variation of body
colors than non-polyrhachis mimic
species
M. maxillosa
P. muelleri
P. armata
P. laevigata
Golden morph
P.javanica
M.maxillosa
Polyrhachismimics
M. biseratensis
Black morph
M. sabahna
M. biseratensis
M.malayana
P. tibialis
M. sabahna
Black & red morph
M. grossa
P. olybria
P. calypso
Polyrhachis
M. maxillosa
Principal component analysis
(PCA) plot on body color for
ant and ant-mimic species
M. malayana
Mimic spiders
Non-Polyrhachis
Mimicking species
Principal component
analysis (PCA) plot on body
color for mimicking spiders
4
2015/05/19
“Through the Looking–Glass”
But, more than the technology, what is important is quality
of preparation
- Computer vision -
- Alignment of image posture Image A
image B
Aligning
image
Image A
image B
Incorrect posture to
body axis
By using computer vision technology, we can detect reflection of
ant-diversity in ant-mimics
Determining an angle
degree (θ) for rotation
Creating the rotated
image by the angle
Cropping the rotated
image B to fit image A
!  If necessary, incorrect posture in reference to body axis is aligned by rotated image
But, more than the technology, what is important is quality
of preparation
- Image color management •  Every camera is different. Even new cameras
of the same make and model may have slight
differences in color characteristics.
•  Every monitor has different spectral
properties.
•  Different light sources produces a different
spectral power distribution.
Photography color checker
!  We need to calibrate color components of images to compare them.
Herbarium specimen usually
photographed includes a color
checker
Insect specimen is too small to
include a photography color
checker
できるだけ正確に
But, more than the technology, what is important is quality
of preparation
- Image color management -
Development of automatic analysis software for
behavior mimics of ant-mimicking spiders
Behavioral Mimicry for Ants
photography color checker for
dentist's instruments
White balance and use grayscale to adjust tracking for equal RGB levels
5
2015/05/19
K-track (Automatic tracking and behaviorsegmentation software of multiple honey bees)
Kimura, Toshifumi, et al. "Development of a new method to track multiple honey bees with complex
behaviors on a flat laboratory arena." PloS one 9.1 (2014): e84656.
Image processing steps for behavior analysis
Image processing steps for behavior analysis
Current
frames
Background
image
Foreground
frames
Pixel difference
between successive
frames was used to
erase moving objects.
Using difference
between the current and
background frame,
binary images of moving
objects were created
Background
creation
Background
subtraction
OpenCV Background subtraction model (cv::createBackgroundSubtractorMOG)
Foreground masked frames, which contained only the pixels belonging to moving objects in the scene, were created by using
the background and current frames.
Analysis for walking speeds of AMS, NMS and ANT
t
t+1
t+2
Out puts
t+3
•  Velocity
•  Acceleration
•  Direction
t+4
•  Behaviorsegmentation
Identifying of target by
using spatio-temporal
contextual information
The center position (X,Y) of
moving object was recorded and
tracked
Data log
The location data and the behavioral trajectory of targeted object was logged during the image processing from which the
velocity, acceleration and direction of the object’s movements were calculated.
Analysis for variation of moving directions of AMS,
NMS and ANT
Ant
(Polyrhachis)
Mimicking spider
(M. maxillosa)
Non-mimicking spider
Our goals
• 
Non-mimic
Poor mimic
Good mimic
Ant
!  To develop automatic analysis tools to measure mimetic
resemblance on behaviors between ants and ant-mimicking spiders
Ant
(Polyrhachis)
Non-mimicking spider
!  To detect and categorize characteristic states of behavior for antmimicking and no-mimicking spiders, and ants, respectively
!  To detect differences of behavior states between good and poor
mimics of ant-mimicking spiders.
Mimicking spider
(M. maxillosa)
Ant and ant-mimicking spider walk a zigzag path
6