Brain-Computer Interfaces @ AIRLab
Matteo Matteucci – AIRLab, Politecnico di Milano
The Locked-in syndrome
Causes
• Traumatic brain injury
• Brain stem stroke
• Neuro-degenerative
diseases
Effects
• Complete paralysis
• Patient is aware and
awake
• Inability to speak
Matteo Matteucci – Politecnico di Milano
"the closest thing to
being buried alive"
Brain-Computer interfaces
Signal
Acquisition
Processing
Translation
Device
commands
BCI
Perception
World
An alternate communication pathway not requiring muscular activity
Matteo Matteucci – Politecnico di Milano
Inside the Brain
 There are 100 billion (1011) neurons
 Each one is connected to 1000–100,000 others for a total of 1015
connections (synapses)
 An electric potential travels along the axon
4
Matteo Matteucci – Politecnico di Milano
Signals From the Brain
 Invasive approach
• Cortical electrodes
• Electrodes arrays
 Non-invasive approach
• Electroencephalography (EEG)
• Magnetoencephalography (MEG)
• Near-infrared spectroscopy (NIRS)
• Function magnetic resonance (fMRI)
5
Matteo Matteucci – Politecnico di Milano
Electroencephalography
Electrodes displacement
•International 10-20 system
•EOG electrodes
Acquisition device
•Amplification
•Sampling
•Filtering
EEG signal
•Noisy
•Affected by artifacts
But information is there!
Matteo Matteucci – Politecnico di Milano
Event Related Potentials
 P300: Positive wave around 300 ms
after a meaningful/odd stimulus
8
Matteo Matteucci – Politecnico di Milano
Moving ideas …
• Selection of one out of few options by using the P300 wave
• Select robot destination amont a set of pre-defined choiches
[P300 speller by E. Donchin's (UIUC, 1980s)]
A
P300
A
9
Matteo Matteucci – Politecnico di Milano
Machines are not perfect …
A BCI is not 100% accurate … can we do it better?
• Repeate selection
• Ask for confirmation
• Deeper analysis of EEG (ErrP)
ErrP
10
Matteo Matteucci – Politecnico di Milano
2+2=5 No!
Write it Again Sam!
• We can use ErrP to correct wrong choices …
C
A
O
FI
C
O
AI
P300
ErrP
C
A
No!
Ok
O
FI
CI_
C_
_
CIAO_
CIA_
11
Matteo Matteucci – Politecnico di Milano
Event Related Potentials
 P300: Positive wave around 300 ms
after a meaningful/odd stimulus
 ErrP: wave generated
by (machine) errors
12
Matteo Matteucci – Politecnico di Milano
ERP @ Airlab
 Building a complete BCI based on P300 and error potentials (ErrP)
• Recognition of P300 through a genetic algorithm (GA)
• Recognition of ErrP through a statistical method
 Metric to choose design parameters
• Realistic
• Task-based
13
Matteo Matteucci – Politecnico di Milano
How does it look like?
Matteo Matteucci – Politecnico di Milano
Let me introduce LURCH …
BUS
Bedroom
Kitch
en
landmark
Sensors
wheelchair
Navigation
Living Room
Algorithms
User Interface
Feedback
for ErrP EBNeuro Galileo EEG system
Odometry
Movement
command
Motion Control
Matteo Matteucci – Politecnico di Milano
Pattern Recognition
How does it look like?
Matteo Matteucci – Politecnico di Milano
Motor Imagery
The motor cortex
Motor imagery produces contra-lateral
desynchronization of mu and beta rhythms
Left hand
Right hand
Matteo Matteucci – Politecnico di Milano
MI @ AIRLab (and ARCS Lab)
 Development of a motor imagery BCI for a 2 dimensions control:
• Recognition of the intention to
perform 4 +1 movements
(up, down, left, right, and rest)
• Integration with a predictive
language model
 Focus on:
• Automatic (offline) search for user
best frequencies and parameters
• Comparison and improvements of spatial filters
• Improved feature selection process
• Compansation of EOG artifcats
18
Matteo Matteucci – Politecnico di Milano
Mixing the two madalities
Motor Imagery
pipe
Bayesian
classifier
Error
Potentials pipe
Stimulus
presentation
Signal
translation
Signal processing
!
Matteo Matteucci – Politecnico di Milano
How does it look like?
Matteo Matteucci – Politecnico di Milano
How should we evaluate and compare systems?
ITR
Information Transfer Rate (Mutual Information)
ITR = log 2 N + p log 2 p + 1  p  log 2
21
Matteo Matteucci – Politecnico di Milano
1 p
N 1
N: # symbols, p: accuracy
An alternative: The Utility
 Expected average benefit
  bk  E[bk ]
U = E

T

 E [ t k ]
0
bk
b2
b1
t1
t2
bk-1
bK
tk-1 tk tk+1
tK
PL|L
 Computed from models
PX|L
bk+1
PL|L
PBs|Bs
PBs|Bs
PX|Bs
PX|Bs
PBs|L
PX|L
PL|L
PBs|Bs
PBs|Bs
PX|Bs
PBs|Bs
PX|Bs
PBs|L
PX|L
PL|L
PBs|Bs
PBs|L
PX|L
PBs|Bs
PX|Bs
PL|L
PBs|Bs
PBs|Bs
PBs|Bs
Matteo Matteucci – Politecnico di Milano
t
PBs|L
PX|Bs
22
T
PBs|L
A Bit Too Much
 Utility vs. ITR
1
tL =
2p  1
Util = log 2 N 1 max{0,2 p 1}
1 p


ITR = log 2 N + p log 2 p + 1  p log 2
N 1
ITR overestimates
23
Matteo Matteucci – Politecnico di Milano
What’s going on?
 BCI methods and experiments (ongoing)
 Audio based Brain Computer Interface
 Generalization of Utility to generic discrete interfaces
 Test on real users …
 Develop algorithms for autonomous wheelchair driving
 World perception and obstacle avoidance
 User compliant wheelchair trajectory planning
http://www.dei.polimi.it/people/matteucci
http://chrome.ws.dei.polimi.it
Matteo Matteucci – Politecnico di Milano
Brain-Computer Interfaces @ AIRLab
Matteo Matteucci – AIRLab, Politecnico di Milano