INRS 2009, Zurich, 12-14 February 2009 The Patient is the Master: How to Control Rehabilitation Robots Robert Riener Sensory-Motor Systems Lab Institute of Robotics and Intelligent Systems, ETH Zurich & Spinal Cord Injury Center, University Hospital Balgrist, Zurich Robot-Aided Rehabilitation Training Lokomat ARMin The Patient is the Master: Why? Improve Adaptation Adapt task and difficulty to the individual patient Increase Participation Let patient physically and mentally participate Increase Motivation Increase short term patient motivation (engagement) and long term patient motivation (compliance) ⇒ Increase Training Efficiency and Outcome How to Make the Patient the Master Cooperative Control Enhance physical participation Virtual Reality Incorporate patient into functional tasks Bio-Cooperative Control Control autonomous functions Enhance mental participation Conventional: Patient is „Slave“ Properties • Position contoller • Given pattern and timing • No interactivity Robot is „Master“, Patient is „Slave“ Patient-Cooperative Control Parameters Position Virtual Assistent Force Patient-Cooperative Control Cooperative Control Path Control Challenge • Support patient but do not restrict him Path Control • Path: virtual tunnel • Patient controls timing of movement • Robot applies assistive and corrective torques A. Duschau-Wicke, H. Vallery, L. Lünenburger Path Control Increases Participation Heart Rate Muscle Activity 1.8 0.24 0.2 0.18 0.16 0.14 0.12 0.1 0.08 0.06 g g g e e in g ing nc nc din win win w w a a s a s t s t S o . t. e d .s ds t. l rm Ini Pr Mi rm e Mi Ini e T T 14 incomplete SCI subjects * 1.6 Hfrz of heart rate Relative increase Normalized muscle activity (BF) 0.22 Position control Path control 1.4 1.2 1 0.8 0.6 0.4 0.2 0 -0.2 Pos Path_a Pos.contr. Path contr. Path Control Increases Variability Path Control 80 80 70 70 60 60 Knee angle [°] Knee angle [°] Position Control 50 40 30 50 40 30 20 20 10 10 0 -20 -10 0 10 20 Hip angle [°] 30 40 iSCI, stroke, CP children 0 -20 -10 0 10 20 Hip angle [°] 30 40 ARMin III Patient-Cooperative Control (xBall , yBall) y (xPatient ,0) x T. Nef K+Bs Patient H. xpatient xball Fsupport Patient I. Pilot Study: Results xpatient xball Fsupport T. Nef How to Make the Patient the Master Cooperative Control Enhance physical participation Virtual Reality Incorporate patient into functional tasks Bio-Cooperative Control Control autonomous functions Enhance mental participation Lokomat and Multimodal VR Lokomat and Multimodal VR 3D Stereo Projection Ventilator Fan 7.1 Sound System Lokomat Pediatric Lokomat Collaboration: Children‘s Hospital USZ, University of Zurich, Hocoma AG Kinder-Lokomat und VR Collaboration: Children‘s Hospital USZ, University of Zurich, Hocoma AG Training Goals Soccer Obstacles Traffic + Force ROM + Speed + Coordination + Cognition + Snow + + + + + ARMin III: 7 Degrees of Freedom ETH Zurich & M. Mihelj, Univ. Ljubljana Hocoma AG & ETH Zurich Selection of ADL Tasks ADL tasks (70) Eating and drinking, dressing, hygiene, household, communication, etc. Suitable for ARMin (48) Important in daily life Performed during conventional therapy Pool of (20) ADL tasks M. Guidali Not dangerous Training of Virtual ADL Tasks Different Generalized Spaces and Subtasks Training of Virtual ADL Tasks Different Generalized Spaces and Subtasks Training of Virtual ADL Tasks Different Generalized Spaces and Subtasks Path Control with ARMin Properties • Patient moves freely within a tunnel around reference trajectory • Adjustable force field inside the tunnel assists the patient 25 Arm Therapy Robot ARMin ARMin III ARMin I 2005 ARMin II 2006 Single case studies I, chronic stroke (n=3) 2007 Single case studies II, chronic stroke (n=4) T. Nef, P. Staubli, V. Klamroth, A. Kollmar et al. 2008 2009 Controlled clinical trial, chronic stroke (n>80) How to Make the Patient the Master Cooperative Control Enhance physical participation Virtual Reality Incorporate patient into functional tasks Bio-Cooperative Control Control autonomous functions Enhance mental participation Bio-Cooperative Control Bio-Cooperative Control Controlling Heart Rate with Gait Speed vTM [km/h] Heart Rate as Function of Gait Speed & Activity 3 2 0 0 4 7 10 Time [min] 13 5 healthy subjects, different activity levels A. König 17 Controlling Heart Rate with Gait Speed Heart Rate as Function of Gait Speed & Activity A. König Gait speed [km/h] Controlling Heart Rate with Gait Speed Fuzzy Controller for Setting Treadmill Speed Heart rate Heart rate error A. König Treadmill speed Controlling Heart Rate with Gait Speed A. König How to Make the Patient the Master Cooperative Control Enhance physical participation Virtual Reality Incorporate patient into functional tasks Bio-Cooperative Control Control autonomous functions Enhance mental participation Yerkes-Dodson‘s Law stress Patient state „good“ arousal M. Bolliger, A. König How to control? exhaustion Bio-Cooperative Control Multimodal Stimulation Biomechanical and Psychophysiological Stimuli Body weight support Sound Graphics Treadmill speed Guidance force Lokomat & Multi-Recordings EMG EEG Blood pressure Spirometry Eye movements Force and position sensors Respiration frequency Heart rate & HRV Skin temperature Skin resistance (GSR) Lokomat & Multi-Recordings Main Physiological Recordings Force and position sensors Respiration frequency Heart rate & HRV Skin temperature Skin resistance (GSR) Preliminary Results: GSR and ECG Galvanic Skin Response SCR (dimless) StDev T1 T2 T3 T4 T5 T6 T7 2 8 10 20 24 25 25 1.3 7.0 14.7 13.4 10.9 7.5 10.8 T1 T2 T3 T4 T5 T6 T7 74 89 85 109 92 100 91 12.1 17.4 9.5 17.6 22.4 24.4 19.4 Heart Rate HR (1/min) StDev M. Bolliger, A. König Bio-Cooperative Control Multimodal Stimulation Physiological Recordings State Interpreter Biomechanical and audiovisual Force and position sensors Physical Effort Arousal Heart rate & HRV Skin resistance etc. Controller Valence Arousal-Valence Space Arousal Valence James Russel, 1979; Gerber et al., 2008 Arousal-Valence Space Arousal Valence James Russel, 1979; Gerber et al., 2008 Exciting and Motivating a Subject Challenged incr guidance force Supporting incr body weight incr speed ? Audiovisual Stimuli Thrilling scary sound action scenes bad weather deep canyon flawy river Challenged Biomechanical Stimuli HR HRV SCR BF Torques Temp Aarousal Valence Physical effort nice sound bright sceneries Calming good weather flat canyon calm river scary sound dark sceneries Thrilling bad weather deep canyon flawy river Challenged low guidance force Demanding low body weight high speed Bored high guidance force Supporting high body weight low speed Overstressed Audiovisual stimuli Biomech. stimuli Controlling Psychophysiology ? HR HRV SCR BF Torques Temp ? HR HRV SCR BF Torques Temp HR HRV SCR BF Torques Temp Arousal Valence Physical effort Arousal Valence Physical effort Arousal Valence Physical effort Take Home Messages The Patient is the Master The robot … • should support the physical effort of the patient • can display interactive functional tasks • can take into account physiological functions • should control mental states and motivate the patient Acknowledgements SMS Lab Team Partners • Balgrist, Hocoma • Kinderklinik USZ • RIC Chicago Support • SNF, IFP • US Dept. Education • Bangerter-Rhyner • ETH Foundation