GIFT: A Geospatial Image and Video Filtering Tool for Computer Vision
Applications
Yinghao Cai, Ying Lu, Seon Ho Kim, Luciano Nocera and Cyrus Shahabi
Integrated Media Systems Center, University of Southern California
Introduction
GIFT Applications and
𝑁 l Motivations:
p: camera location
- Mobile videos are prevalent
 YouTube statistics: ~20% mobile videos,
~3 hours/min upload
𝑑 𝑑 : view orientation
l Panorama generation
- 
1)  Apply circle range query first
α: viewable angle
2)  For each direction group, select the “best” (considering both direction and location) image
R: viewable distance
- Rich sensors are available
- Video frame-level geo-tagged (FOV model)
- 
R
α
Key video frame selection based on geo-metadata
Panorama generation with autoStich
p
Field Of View (FOV) model
l Computer vision applications
Without GIFT, 228 FOVs were selected
- 
Persistent tracking, panorama generation, etc.
- 
Technique needs:
1)  Fine granularity spatial metadata
With GIFT, only 13 FOVs were used
2)  Effective data management on media data
l Persistent tracking
-  GIFT is applied to select video
frames which may cover the target.
Illustration of GIFT for persistent tracking. If the target exits the FOV of one video, GIFT
effectively selects a small number of the most relevant video frames which may cover this target.
Experimental Results of GIFT for Persistent tracking
● Geospatial Image and video Filtering Tool (GIFT)
-  Using the state-of-art geospatial metadata acquisition technologies
● Results with synthetic dataset
-  Efficiently managing videos with the associated geospatial metadata for
indexing and searching
-  Supporting various vision technologies and algorithms to enable scalable
video analysis.
GIFT for Computer Vision
Queries
APP
GIFT
Fast
Fast
Visualization of the synthetic camera trajectories and FOVs
Video
Analytic
s
Processi
ng
SpatioTemporal
Filtering
Images / videos
● Results with real-world dataset
- Use MediaQ to collect mobile videos
with metadata
GIFT Framework
Vision
Applications
Requests
Persistent Tracking
Internet
Filtering Functions
Logical Operations
And
Or
Xor
Spatial
Direction
By orientation
By time
Internet
Results
Images
Directional interval
Inwards
Outwards
Timestamp
Temporal
Query type
FOV Database
FromTime
ToTime
FromToTime
Extraction
Video segments
Rectangle r ange query
kNN
By distance
……
Point query
Circle r ange query
Indexing
Sorting
Panorama generation
Video Database
GIFT
Snapshot query
Continuous query
Video Contents
Conclusion
l  Present the novel GIFT to select key-frames for computer vision applications
l  GIFT can significantly improve the performance for computer vision
applications.
IMSC Retreat 2015