Analysis of classification performance of fNIRS signals from prefrontal cortex using various temporal windows Noman Naseer1, Keum-Shik Hong2,1, M. Jawad Khan2 and M. Raheel Bhutta1 1 Department of Cogno-Mechatronics Engineering, 2School of Mechanical Engineering, Pusan National University Busan 609-735, Republic of Korea {noman, kshong, jawad, cogno}@pusan.ac.kr Abstract— In this paper we investigate the role of different temporal windows in classification of functional near-infrared spectroscopy (fNIRS) signals corresponding to mental arithmetic and mental counting for development of a brain-computer interface. Signals are acquired from the prefrontal cortex of four healthy subjects during mental arithmetic and mental counting tasks using a continuous-wave fNIRS system, DYNOT: Dynamic Near-Infrared Optical Tomography. Support vector machine is used to classify the mean values of the change in concentration of oxygenated and deoxygenated hemoglobin during different temporal windows. The highest average classification accuracy of 82.4% is achieved during the 2-7 s time window within the total 10 s task period. The averaged classification accuracies achieved using 0-5 s, 1-6 s and 5-10 s temporal windows are 61.6%, 67.4% and 72.5% respectively. These results indicate that using signal mean, calculated during 2-7 s time window, as the features results in higher classification accuracies. Keywords— Brain-computer interface (BCI), functional nearinfrared spectroscopy (fNIRS), Classification, Support vector machines (SVM), Prefrontal cortex, temporal window size. I. INTRODUCTION A brain-computer interface (BCI) technique establishes interface between brain and external devices, by bypassing the peripheral nervous system, using the brain signals [1,2]. BCI technique has made it possible to decode the brain signals and use them for the control of external devices. The current trends in research have shown a great potential in the field of BCI. The recent developments in BCI have reduced the cost of equipment and further research is being carried out for better acquisition and processing of brain signals. BCI modalities are categorized into invasive and non-invasive techniques. The invasive methods acquire brain signals by implanting electrodes into the skull whereas the noninvasive methods do not require any surgical procedure. Although the invasive techniques have recently shown promising results for BCI by showing the possibility of multidimensional control over external devices, the noninvasive techniques are preferred because of fewer complexities involved. Functional near-infrared spectroscopy (fNIRS) is a relatively new brain imaging technique for BCI that uses nearinfrared-range light (650~1000 nm), that can penetrate in the human tissue, to measure the hemodynamic response. It, therefore, provides the information about the cerebral blood flow changes that are caused by the neural activity in the cortical regions of the brain. fNIRS has a good spatial resolution and is relatively inexpensive, portable, safe and overall easy to use. Although fNIRS has been used for the study of cerebral blood flow changes in the past, it’s in the fields of BCI, brain mapping and brain state decoding is relatively new [3-26]. In fNIRS, pairs of near-infrared light emitters and detectors are used to incident light on the scalp and detect the reflected light, respectively. The different wavelength lights are induced on the scalp. These different wavelength lights travel though the head while scattering. These then pass through the cortical areas of brain wherein the wherein the chromophores oxygenated and deoxygenated hemoglobin (HbR) are present. The concentration of these chromophonres changes as a result of neural activation, as the neuronal firing requires consumption of oxygen and glucose from hemoglobin. Some of the light, after being absorbed by the HbO and HbR, is reflected back. These back-reflected lights are detected using a strategically placed detector on the scalp. Since the absorption coefficients of HbO and HbR are different for different wavelengths of lights, the modified Beer-Lambert law can then be used to calculate the changes in concentration of HbX (i.e, HbO and HbR) as explained in Section 2.5. In this paper, classification of mental arithmetic and mental counting was performed using support vector machine. The features used were the mean values of the changes in concentration of HbO and HbR signals. The performance of these features was analyzed in various temporal windows. The 2-7 s time window was found to be the best temporal window for classification of prefrontal activity. Fig. 2. Experimental paradigm: The two brown boxes at the beginning and the end show 20 s rest period while the green box in the middle shows the 10 s task period. Fig. 1. Optode placement on the prefrontal cortex. II. MATERIAL AND METHODS A. Signal Acquisition A continuous-wave fNIRS system DYNOT (Dynamic Near-Infrared Optical Tomography), at a sampling rate of 1.81 Hz was used to acquire brain signals. This system was obtained from NIRx medical technologies, LLC, New York. The system uses near-infrared lights of two wavelengths i.e. 760 nm and 830 nm. B. Subjects Four subjects (mean age: 28.2 ± 2.2) took part in the experiment. None of the subjects had a history of any mental or physical disorders. The verbal consent of each subject was taken after they were informed in detailed about the experiments. The experiments were performed in accordance with the latest Declaration of Helsinki. C. Optode Placement To acquire brain signals corresponding mental arithmetic and mental counting, a total of 3 near-infrared light emitters and 8 detectors were placed on the prefrontal cortex. The optode placement and channel configuration is shown in Fig. 1. The selected channels with the emitter-detector separation of approximately 3 cm, considered for the analysis, are numbered. The source-detector distance of 3 cm was used in accordance with the literature [6-10]. The channels with an emitter-detector distance of more than 3 cm were not considered for analysis as they might not have contained useful information because of their high emitter-detector separation. D. Experimental Paradigm The subjects were seated on a comfortable chair in front of a monitor placed at a distance of approximately 70 cm. The length of experiment for each subject was 750 seconds with a total of fifteen trials in one experimental session. The subjects were asked to relax for first 20 seconds to setup a baseline, after that the subjects were given a visual clue to start the mental arithmetic task. The last 20 seconds was again the rest period to allow the signals to return to the baseline. Fig. 2 shows one complete trial sequence for the experiment. For the mental arithmetic task, the subjects were asked to mentally perform a series of arithmetic calculations that appeared on the screen in a pseudorandom order. These calculations consisted of subtraction of a two-digit number (between 10 and 20) from a three-digit number throughout the task period with successive subtraction of a two-digit number from the result of the previous subtraction (e.g. 625-19, 60612, 594-15, etc). For the mental counting task, the subjects were asked to mentally count the number of times the screen is changed on the monitor. E. Signal Processing and Classification The optical density signals obtained were first converted into the concentration changes of HbO and HbR (ΔcHbO(t), ΔcHbR(t)) using the modified Beer-Lambert Law as follows 1 c (t ) (λ ) HbR (λ1 ) A(t , λ1 ) 1 HbO HbO 1 , c (λ ) HbR (λ 2 ) A(t , λ 2 ) l d HbR (t ) HbO 2 (1) where ΔA(t; λj) (j =1,2) is the unit-less absorbance (optical density) variation of the light emitter of wavelength λj, αHbX(λj) is the extinction coefficient of HbX in µM-1mm-1, d is the unit-less differential path length factor (DPF), and l is the distance (in millimeters) between emitter and detector. The HbO and HbR signals extracted include the physiological noises especially due to respiration, heartbeat and Mayer waves. The signals acquired were therefore lowpass filtered with a cut-off frequency of 0.5 Hz to remove these noises. Normalization was performed after filtering the signals by dividing the signals with the baseline value. The signals after normalization and filtering were then classified using linear support vector machine (SVM). SVM was chosen as a classifier because it has been shown to work well in a number of previous studies. SVM uses hyperplanes to discriminate between the data representing two or more classes [27]. It also introduces a regularization parameter that can allow or penalize classification errors on the training set. The resulting classifier can accommodate outliers and obtain better generalization capabilities. As outliers are common in fNIRS data, this regularized version of SVM may give better results for BCI than the non-regularized version [27]. SVM algorithms attempt to find the optimal solution for the hyperplane that maximizes the distance between the hyperplane and the nearest training samples. These Classifiers are designed to maximize the distance to the nearest training point. The linear decision boundary (separating hyperplane) for both SVM was obtained during a training session prior to the test session. After that the classification into “mental arithmetic” or “mental counting” classes was performed by projecting the test samples, acquired after each experimental trial, on the hyperplane. The classification was performed on the signal mean (SM) of the values of the change in concentration of HbO and HbR. The feature vector hence consisted of thirty 2-dimentional data points for each subject. Four different temporal windows were considered for classification to see the effect selection of temporal windows on classification accuracies. The temporal windows considered were 0-5 s (W0-5), 1-6 s (W1-6), 2-7 s (W2-7) and 5-10 s (W5-10) within the entire 10 s task period. by approximately 2-3 s after the onset of task period and it takes around 5-6 s to reach its peak value. The difference between the classification accuracies acquired using different temporal windows was statistically significant when tested using the t-test. The t-values for classification accuracies obtained using W2-7 versus those obtained using W0-5, W1-6 and W5-10 were found to be 0.00139, 0.0091 and 0.038 respectively, which based on the 5% significance level show that the performance of the features within 2-7 s temporal window was significantly better than the other temporal windows. IV. DISCUSSIONS In the present research, classification of fNIRS signals corresponding to mental arithmetic and mental counting was carried out successfully with an average accuracy of 82.4% using signal mean of HbO and HbR during 2-7 s temporal window within the total 10 s task period. In our previous studies [11,13,28-30], it was shown that using fNIRS, it is possible to classify the different activities versus the rest period with high classification accuracies. The activities considered in those studies were finger tapping, mental arithmetic and mental counting and motor imagery. In this study, however, two different activities were classified, from the prefrontal cortex. The reason for choosing the prefrontal activity as the control paradigm was that it is less likely to be implicated in case of motor disabilities, hence, suitable for providing control commands for patients with motor disabilities. The signal attenuation and hair artifacts are also low when acquiring signals from the prefrontal cortex. , Five-fold cross validation, that mixes data into five segments, one of which is used for testing and the rest four are used for training, was performed to estimate the classification accuracies. The 2-D features spaces for the different temporal windows are shown in Figs. 3-5. III. RESULTS The averaged classification accuracies of subjects using features for different time windows are listed in Table 1. The all-subjects averaged classification accuracy using W2-7 was found to be 82.4%, whereas the same using W0-5, W1-6 and W5-10 were found to be 61.6%, 67.4% and 72.4%, respectively. The higher classification accuracies using W2-7 was in accordance with the literature [8, 9]. This may be attributed to the fact that the hemodynamic response lags the neural activity Fig. 3. The 2-D feature space for 0-5 s temporal window for Sub 3. Fig. 4. The 2-D feature space for 1-6 s temporal window for Sub 3. confirming the result of our previous motor cortex based study. The hemodynamic response varied across the subjects and therefore the classification accuracies also varied, see Table 1. These variations could be attributed to the differences in the shape of the head and different scalp-cortex distances for different subject. Since the hemodynamic response patterns for all four subjects were not similar, more subjects should be recruited for experiments to confirm our findings. The effect of habituation also plays an important role in decreasing the activity strength of the signals acquired from the prefrontal cortex. A continuing exposure to mental arithmetic might result in lower hemodynamic response due to habituation. Further study needs to be devised to investigate the effects of habituation in mental arithmetic-induced hemodynamic response and thereby classification accuracies. Another limitation of our present study is that online classification [7] was not performed. In case of online classification the classification accuracies might decrease. In future, we aim to classify more than 2 activities online for development of a BCI that can have more than two control commands. V. Fig. 5. The 2-D feature space for 2-7 s temporal window for Sub 3. Table 1. SVM CLASSIFICATION ACCURACIES CONCLUSIONS In this paper, we classified two different activities from the prefrontal cortex for development of a BCI and investigated the role of using different temporal windows for classification on classification accuracies. It was shown that it is possible to classify, the fNIRS signals corresponding to mental arithmetic and mental counting and the classification accuracies were improved when the 2-7 s temporal window was use for classification. The average accuracy was found to be of 82.4% using the mean value of the changes in concentration of HbO and HbR signals acquired during the 2-7 s temporal window within the entire 10 s task period. The classified signals can then be used as the two control commands for the control of or communication with the external devices. The results of this research successfully prove the prefrontal activities acquired using fNIRS to be the potential candidates for BCI applications. ACKNOWLEDGMENT Subject No. Classification accuracies (%) W0-5 W1-6 W2-7 W5-10 1 60 73.3 90 80 2 66.6 70 86.6 73.3 3 63.3 66.6 73.3 66.6 4 56.6 60 80 70 Average 61.6 67.4 82.4 72.4 This work was supported by the National Research Foundation of Korea under the Ministry of Science, ICT and Future Planning, Republic of Korea (grant no. NRF-2014R1A2A1A10049727). REFERENCES [1] [2] In another study [8], we performed a detailed temporal analysis of fNIRS signals corresponding to the right- and leftwrist motor imageries. The results showed that the 2-7 s temporal window within the total 10 s task period is the best temporal window for classification. In this study, the same analysis has been carried out on the prefrontal activities, [3] [4] J.R. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller, and T.M. Vaughan, “Brain-computer interfaces for communication and control,” Clin. Neurophysiol. vol. 113, no. 6, pp. 767-791, June 2002. F. Matthews, B.A. Pearlmutter, T.E. Ward, C. Soraghan, and C. Markham, “Hemodynamics for Brain Computer Interfaces,” IEEE Signal Process. Mag. vol. 25, no. 1, pp. 87-94, December 2008. S.M. Coyle, T.E. Ward, and C.M. Markham, “Brain-computer interface using a simplified functional near-infrared spectroscopy system,” J. Neural. Eng. vol. 4, no. 2, pp. 219-226, September 2007. R. Sitaram, H. Zhang, C. Guan, M. Thulasidas, Y. Hoshi, A. Ishikawa, K. Shimizu, and N. Birbaumer, “Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] brain-computer interface,” NeuroImage, vol. 34, pp. 1416-1427, February 2007. M. Naito, Y. Michioka, K. Ozawa, Y. Ito, M. Kiguchi, and T. Kanazaw, “A communication means for totally locked-in ALS patients based on changes in cerebral blood volume measured with near-infrared light,” IEICE Trans. Inf. Syst. vol. 90, pp. 1028-1037, July 2007. S. Luu, and T. Chau, “Decoding subjective preferences from single-trial near-infrared spectroscopy signals,” J. Neural. Eng. vol. 6, no. 1, February 2009. N. Naseer, M.J. Hong, and K.-S. Hong, “Online binary decision decoding using functional near-infrared spectroscopy for development of brain-computer interface,” Exp. Brain Res. vol. 232, no. 2, pp. 555-564, February 2014. N. Naseer and K.-S. Hong, “Classification of functional near-infrared spectroscopy signals corresponding to right- and left-wrist motor imagery for development of a brain-computer interface,” Neurosci. Lett. vol. 553, pp. 84-89, October 2013. N. Naseer, K.-S. Hong, "Determination of temporal window size for classifying the mean value of fNIRS signals from motor imagery," Proceedings of the IEEE International Conference on Robotics, Automation and Mechatronics (IEEE RAM), Manila, Philippines, pp. 237-240, November 12-15, 2013. N. Naseer, K.-S. Hong, “Discrimination of right- and left-wrist motor imagery using fNIRS: towards control of a ball-on-a-beam system,” Proceedings of the 6th International IEEE Engineering in Medicine and Biology Society (IEEE EMBS) conference on Neural Engineering, San Diego, CA. USA, pp. 703-706, November 6-8, 2013. N. Naseer, K.-S. Hong, "Functional near-Infrared spectroscopy based discrimination of mental counting and no-control state for development of a brain-computer interface," Proceedings of the 35th International Conference of IEEE Engineering in Medicine and Biology Society (IEEE EMBS), Osaka, Japan, pp. 1780-1783, July 3-7, 2013. N. Naseer, K.-S. Hong, "Functional near-infrared spectroscopy based classification of prefrontal activity for development of a brain-computer interface," The 10th Annual Congress of Society of Brain Mapping and Therapeutics (SBMT), Baltimore, Maryland, USA, May 12-14, 2013. N. Naseer and K.-S. Hong, “Functional near-infrared spectroscopy based brain activity classification for development of a brain-computer interface,” Proceedings of the IEEE International Conference on Robotics and Artificial Intelligence (ICRAI), Rawalpindi, Pakistan, pp. 174-178, October 22-23, 2012. N. Naseer, K.-S. Hong, M.R. Bhutta, and M.J. Khan, “Improving classification accuracy of covert yes/no response decoding using support vector machines: An fNIRS study,” Proceedings of the IEEE International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 6-9, April 22-24, 2014. M.J. Khan, M.J. Hong, and K.-S. Hong, “Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface,” Front. Hum. Neurosci., vol. 8, 244, pp. 1-10, April 2014. M.J. Khan, K.-S. Hong, M.R. Bhutta, and N. Naseer, “fNIRS based dual movement control command generation using prefrontal activity,” Proceedings of the IEEE International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 244-248, April 22-24, 2014. M.R. Bhutta, K.-S. Hong, N. Naseer, and M.J. Khan, “Water correction algorithm to improve the classification accuracy: A near-infrared spectroscopy study,” Proceedings of the IEEE International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 91-94, April 22-24, 2014. M.J. Khan, K.-S Hong, N. Naseer, M.R. Bhutta, S.H. Yoon, "Hybrid EEG-NIRS BCI for rehabilitation using different brain signals," Proceedings of the Annual Conference of the Society of Instrument and Control Engineers (SICE), Sapporo, Japan, pp. 17681773, September 9-12, 2014 M.R. Bhutta, K.-S. Hong, N. Naseer, M.J. Khan, "Hemodynamic signals based lie detection using a new wireless NIRS system," Proceedings of the Annual Conference of the Society of Instrument and Control Engineers (SICE), Sapporo, Japan, pp. 979-984, September 9-12, 2014. N. Naseer, K.-S. Hong, M.R. Bhutta and M.J. Khan, “ Improving classification accuracy of covert yes/no response decoding using support [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] vector machines: an fNIRS study,” Proceedings of IEEE International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 6-9, April 22-24, 2014. M.R. Bhutta, K.-S Hong, N. Naseer and M.J. Khan, “Water correction algorithm to improve the classification accuracy: a near-Infrared spectroscopy study,” Proceedings of IEEE International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 91-94, April 22-24, 2014. M.J. Khan, K.-S Hong, M.R Bhutta and N. Naseer, “ Dual movement control command generation using prefrontal brain activity,” Proceedings o f IEEE International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 244-248, April 22-24, 2014. N. Naseer, K.-S. Hong, “Decoding answers to four-choice questions using functional near-infrared spectroscopy,” J. Near Infrared Spectrosc. vol. 23, in press, May 2015. N. Naseer, K.-S Hong, “fNIRS-based brain-computer interfaces: a review,” Front. Hum. Neurosci. vol. 9, 3, pp. 1-15, January 2015. K.-S. Hong, N. Naseer, and Y.-H. Kim, “Classification of prefrontal and motor cortex signals for three-class fNIRS-BCI,” Neurosci. Lett. vol. 587, pp. 87-92, February 2015. M.J. Khan, K.-S. Hong, N. Naseer, and M.R. Bhutta, “A hybrid EEGfNIRS BCI: Motor imagery for EEG and mental arithmetic for fNIRS,” Proceedings of the 14th International Conference on Control, Automation and Systems (ICCAS), KINTEX, Korea, pp. 275-278, October 22-25, 2014. F. Lotte, M. Congedo, A. L´ecuyer, F. Lamarche, and B. Arnaldi, “A review of classification algorithms for EEG-based brain–computer interfaces,” J. Neural Eng. vol. 4, no. 2, pp. R1-R13, June 2007. N. Naseer, K.-S. Hong, M.J. Khan and M.R. Bhutta, “Classification of prefrontal and motor cortex activities for development of three-class fNIRS-BCI,” The 20th Annual Meeting of the Organization for Human Brain Mapping (OHMB), Hamburg, Germany, June 8-12, 2014. M.J. Khan, K.-S Hong, N. Naseer and M.R. Bhutta, “Multi-decision detection using EEG-NIRS based hybrid brain-computer interface (BCI),” The 20th Annual Meeting of the Organization for Human Brain Mapping (OHMB), Hamburg, Germany, June 8-12, 2014. M.R Bhutta, K.-S Hong, N. Naseer and M.J. Khan, “Classification of lie and truth in forced choice paradigm: an fNIRS study,” The 20th Annual Meeting of the Organization for Human Brain Mapping (OHMB), June 8-12, 2014.
© Copyright 2024