Emotion Recognition using the GSR Signal on Android Devices Shuangjiang Li Outline • Emotion Recognition • The GSR Signal • Preliminary Work • Proposed Work • Challenges • Discussion Emotion Recognition • Human-Computer Interaction • • • • • Speech recognition Gesture/Action recognition Facial expression recognition Emotion recognition … • Affective Computing ( Picard @MIT Media Lab around late 90s) Emotion Recognition • Physiological Signals Source: http://biomedikal.in/2011/05/important-physiological-signals-in-the-body/ The GSR Signal • Galvanic Skin Response (GSR) • measuring the electrical conductance of the skin • due to the response of the skin and muscle tissue to external and internal stimuli, the conductance can vary by several microsiemens (unit of ohm). • GSR is highly sensitive to emotions (fear, anger, startle response, etc.) http://en.wikipedia.org/wiki/Skin_conductance The GSR Signal • GSR Sensor • SHIMMER (Sensing Health with Intelligence, Modularity, Mobility and Experimental Reusability) Platform • The goal of SHIMMER is to provide an extremely compact extensible platform for long-term wearable sensing in both connected and disconnected settings using proven system building blocks. • a highly extensible wireless sensor platform • • • • SHIMMER firmware is based on TinyOS Data transmit via Bluetooth Can sense EMG, ECG, GSR, etc. Support Matlab, LabView, Android, C#/.Net etc. http://shimmer.sourceforge.net/ http://www.shimmer-research.com/ Adrian Burns, SHIMMER: An Extensible Platform for Physiological Signal Capture, IEEE EMBS, 2010 Preliminary Work • Emotion recognition based on the GSR signal • Four emotion categories: amusement, fear, relax, sadness • Using GSR + Accelerometer signal • Preprocessing • Using supported accelerometer data • Denoising using median filter • Data rescaling and normalization • Feature Extraction • 6 statistical features + 10 time domain features + 4 frequency domain features + feature selection (SFFS) Preliminary Work • Recognition rate • KNN • 10-fold cross-validation • Subject dependent / single subject Proposed Work • Emotion recognition on Android Devices • Android GUI for reading GSR sensor data • GSR data preprocessing • GSR data classification • Sequential learning Challenge • Emotion signal tend to very noisy. • Emotion signal generally lacks ground truth and emotion is very subjective. • Recognition algorithms on Android devices should be light weight • Dealing with sequential data Discussion Q&A