Pose Estimation with RadioControlled Visual Markers
Edwin Rijpkema, Kavitha Muthukrishnan, Stefan Dulman, Koen Langendoen
Delft
University of
Technology
Challenge the future
Introduction and Motivation
•  Pose estimation crucial for many applications
 
Low-cost and high accuracy
•  Many different sensing modalities
•  Vision-based benefits:
 
CCD/CMOS sensor commonly available
 
Low-cost
 
High quality sensor
•  Applications
 
Indoor navigation and tracking
 
Environmental mapping
2 | 23
Vision-based Systems
•  Processes image streams from camera to locate/track
  Unobstrusive
  Problem
tracking
with detection and identity of tracked object
•  Markerless vs. Marker-based systems
•  Marker-based advantages:
  Encoding
identities in markers enforces point correspondences
  Reduced pixel processing
•  Architecture:
 
 
Inside-looking-out (moving camera, static markers)
Outside-looking-in (static cameras)
3 | 23
Contribution
•  Low cost inside-looking-out system combining WSN+vision
  Easy
to scale
•  Using radio-identified LEDs
  Radio
communication aids point correspondence
(matching intensity of LED to the node emitted it)
  Lower
processing cost compared to fiducial marker-based sys.
  Extracting
timing information (when node is blinking)
•  Algorithm for pose estimation
4 | 23
System Overview
•  Outward looking camera unit, pose to be estimated
•  Static LED markers (WSN LEDs)
  LED
flashed sequentially one-at-a-time
5 | 23
Overview & Outline
•  LED detection
•  Camera model and calibration
•  Pose Estimation Algorithm
•  Evaluation
6 | 23
LED detection
•  Find location (u,v) of brightest pixel
•  Define image patch centered by (u,v)
•  Use patch for sub-pixel analysis
7 | 23
Sub-pixel analysis
•  Weighted mean of pixel coordinates
8 | 23
Camera model
•  Pin-hole camera model
•  Describes how a 3D point
is projected into image plane
•  Camera calibration gives:
•  Camera intrinsic parameters M
•  Distortion parameters (radial, tangential)
9 | 23
Coordinate transformation
•  LED markers have known world coordinates, transformed to camera
coordinates by:
  Rotation
matrix
  Translation
vector
•  Camera pose is the inverse transformation:
  Rotation
matrix
  Translation
vector
10 | 23
Overview of algorithms
•  DLT-based algorithm (RefAl)
  Stateless
  Uses
or batch method
vision-only
  Measurements
must be observable
•  Extended Kalman Filtering algorithm
  State-based
  Uses
method
vision and process dynamics
  Measurements
need not be observable
11 | 23
RefAL
•  Given N > 3 LEDs with locations (xi,yi,0)
  Find
α and x, that minimizes the reprojection error
12 | 23
Extended Kalman Filter
  Form of Bayesian Estimation
Time Update
(“Predict”)
Measurement Update
(“Correct”)
  Notion of movement & measurement model
  Recursive state estimator
  State –
  KF deals with linear model, EKF is for non-linear model
  EKF tracks means (state) and covariances (P)
13 | 23
Extended Kalman Filter
•  State
14 | 23
Experimental set up
•  8 sensor nodes (Myrianodes)
•  Firewire camera (640x480, 89 degrees)
•  360 locations X 8 LEDs
15 | 23
Evaluation
•  Using a mix of experimental and synthetic data
  Effect
of number of LEDs
  Effect of camera frame rates
  Effect if measurement noise
16 | 23
Effect of number of LEDs
•  EKF comparable to RefAL (8 LEDs), BUT EKF works under-constrained
•  Performance degrades with less markers
17 | 23
Effect of camera frame rates
•  High frame rate better accuracy
•  EKF prediction improves with higher framerates
•  For 30 fps RefAL = EKF, but EKF performs better for higher fps
18 | 23
Effect of measurement noise
•  More noise results in performance degradation
•  EKF performs better than RefAl
19 | 23
Evaluation: EKF camera trajectory
20 | 23
Computational complexity
•  Execution times per pose estimate in Matlab
•  Extended Kalman Filter (EKF)
 
1.6 ms (1 LED)
 
1.7 ms (4LEDs)
•  Reference Algorithm (RefAl)
 
27.2 ms (4 LEDs)
21 | 23
Conclusions
•  Low cost pose estimation system combining
 
Vision + Wireless sensor network
•  Algorithm based on Extended Kalman filtering
  Feasibility
  Deals
  Low
of using single LED at a time
with low frame rates and high measurement noise level
computational complexity
•  NEED to improve at scale!
22 | 23
Questions
Thank you!
23 | 23