Analysis of classification performance of fNIRS
signals from prefrontal cortex using various
temporal windows
Noman Naseer1, Keum-Shik Hong2,1, M. Jawad Khan2 and M. Raheel Bhutta1
1
Department of Cogno-Mechatronics Engineering, 2School of Mechanical Engineering,
Pusan National University
Busan 609-735, Republic of Korea
{noman, kshong, jawad, cogno}@pusan.ac.kr
Abstract— In this paper we investigate the role of different
temporal windows in classification of functional near-infrared
spectroscopy (fNIRS) signals corresponding to mental arithmetic
and mental counting for development of a brain-computer
interface. Signals are acquired from the prefrontal cortex of four
healthy subjects during mental arithmetic and mental counting
tasks using a continuous-wave fNIRS system, DYNOT: Dynamic
Near-Infrared Optical Tomography. Support vector machine is
used to classify the mean values of the change in concentration of
oxygenated and deoxygenated hemoglobin during different
temporal windows. The highest average classification accuracy of
82.4% is achieved during the 2-7 s time window within the total
10 s task period. The averaged classification accuracies achieved
using 0-5 s, 1-6 s and 5-10 s temporal windows are 61.6%, 67.4%
and 72.5% respectively. These results indicate that using signal
mean, calculated during 2-7 s time window, as the features results
in higher classification accuracies.
Keywords— Brain-computer interface (BCI), functional nearinfrared spectroscopy (fNIRS), Classification, Support vector
machines (SVM), Prefrontal cortex, temporal window size.
I.
INTRODUCTION
A brain-computer interface (BCI) technique establishes
interface between brain and external devices, by bypassing the
peripheral nervous system, using the brain signals [1,2]. BCI
technique has made it possible to decode the brain signals and
use them for the control of external devices. The current trends
in research have shown a great potential in the field of BCI.
The recent developments in BCI have reduced the cost of
equipment and further research is being carried out for better
acquisition and processing of brain signals. BCI modalities are
categorized into invasive and non-invasive techniques. The
invasive methods acquire brain signals by implanting
electrodes into the skull whereas the noninvasive methods do
not require any surgical procedure. Although the invasive
techniques have recently shown promising results for BCI by
showing the possibility of multidimensional control over
external devices, the noninvasive techniques are preferred
because of fewer complexities involved.
Functional near-infrared spectroscopy (fNIRS) is a
relatively new brain imaging technique for BCI that uses nearinfrared-range light (650~1000 nm), that can penetrate in the
human tissue, to measure the hemodynamic response. It,
therefore, provides the information about the cerebral blood
flow changes that are caused by the neural activity in the
cortical regions of the brain. fNIRS has a good spatial
resolution and is relatively inexpensive, portable, safe and
overall easy to use. Although fNIRS has been used for the
study of cerebral blood flow changes in the past, it’s in the
fields of BCI, brain mapping and brain state decoding is
relatively new [3-26].
In fNIRS, pairs of near-infrared light emitters and
detectors are used to incident light on the scalp and detect the
reflected light, respectively. The different wavelength lights
are induced on the scalp. These different wavelength lights
travel though the head while scattering. These then pass
through the cortical areas of brain wherein the wherein the
chromophores oxygenated and deoxygenated hemoglobin
(HbR) are present. The concentration of these chromophonres
changes as a result of neural activation, as the neuronal firing
requires consumption of oxygen and glucose from
hemoglobin. Some of the light, after being absorbed by the
HbO and HbR, is reflected back. These back-reflected lights
are detected using a strategically placed detector on the scalp.
Since the absorption coefficients of HbO and HbR are
different for different wavelengths of lights, the modified
Beer-Lambert law can then be used to calculate the changes in
concentration of HbX (i.e, HbO and HbR) as explained in
Section 2.5.
In this paper, classification of mental arithmetic and
mental counting was performed using support vector machine.
The features used were the mean values of the changes in
concentration of HbO and HbR signals. The performance of
these features was analyzed in various temporal windows. The
2-7 s time window was found to be the best temporal window
for classification of prefrontal activity.
Fig. 2. Experimental paradigm: The two brown boxes at the beginning and the
end show 20 s rest period while the green box in the middle shows the 10 s
task period.
Fig. 1. Optode placement on the prefrontal cortex.
II.
MATERIAL AND METHODS
A. Signal Acquisition
A continuous-wave fNIRS system DYNOT (Dynamic
Near-Infrared Optical Tomography), at a sampling rate of 1.81
Hz was used to acquire brain signals. This system was
obtained from NIRx medical technologies, LLC, New York.
The system uses near-infrared lights of two wavelengths i.e.
760 nm and 830 nm.
B. Subjects
Four subjects (mean age: 28.2 ± 2.2) took part in the
experiment. None of the subjects had a history of any mental
or physical disorders. The verbal consent of each subject was
taken after they were informed in detailed about the
experiments. The experiments were performed in accordance
with the latest Declaration of Helsinki.
C. Optode Placement
To acquire brain signals corresponding mental arithmetic and
mental counting, a total of 3 near-infrared light emitters and 8
detectors were placed on the prefrontal cortex. The optode
placement and channel configuration is shown in Fig. 1. The
selected channels with the emitter-detector separation of
approximately 3 cm, considered for the analysis, are
numbered. The source-detector distance of 3 cm was used in
accordance with the literature [6-10]. The channels with an
emitter-detector distance of more than 3 cm were not
considered for analysis as they might not have contained
useful information because of their high emitter-detector
separation.
D.
Experimental Paradigm
The subjects were seated on a comfortable chair in front of
a monitor placed at a distance of approximately 70 cm. The
length of experiment for each subject was 750 seconds with a
total of fifteen trials in one experimental session. The subjects
were asked to relax for first 20 seconds to setup a baseline,
after that the subjects were given a visual clue to start the
mental arithmetic task. The last 20 seconds was again the rest
period to allow the signals to return to the baseline. Fig. 2
shows one complete trial sequence for the experiment.
For the mental arithmetic task, the subjects were asked to
mentally perform a series of arithmetic calculations that
appeared on the screen in a pseudorandom order. These
calculations consisted of subtraction of a two-digit number
(between 10 and 20) from a three-digit number throughout the
task period with successive subtraction of a two-digit number
from the result of the previous subtraction (e.g. 625-19, 60612, 594-15, etc). For the mental counting task, the subjects
were asked to mentally count the number of times the screen is
changed on the monitor.
E. Signal Processing and Classification
The optical density signals obtained were first converted
into the concentration changes of HbO and HbR (ΔcHbO(t),
ΔcHbR(t)) using the modified Beer-Lambert Law as follows
1
c
(t ) 
(λ )  HbR (λ1 )   A(t , λ1 )  1
 HbO    HbO 1
,
 

c
 
(λ )  HbR (λ 2 ) A(t , λ 2 ) l  d
 HbR (t )  HbO 2
(1)
where ΔA(t; λj) (j =1,2) is the unit-less absorbance (optical
density) variation of the light emitter of wavelength λj,
αHbX(λj) is the extinction coefficient of HbX in µM-1mm-1, d
is the unit-less differential path length factor (DPF), and l is
the distance (in millimeters) between emitter and detector.
The HbO and HbR signals extracted include the
physiological noises especially due to respiration, heartbeat
and Mayer waves. The signals acquired were therefore lowpass filtered with a cut-off frequency of 0.5 Hz to remove
these noises. Normalization was performed after filtering the
signals by dividing the signals with the baseline value.
The signals after normalization and filtering were then
classified using linear support vector machine (SVM). SVM
was chosen as a classifier because it has been shown to work
well in a number of previous studies. SVM uses hyperplanes
to discriminate between the data representing two or more
classes [27]. It also introduces a regularization parameter that
can allow or penalize classification errors on the training set.
The resulting classifier can accommodate outliers and obtain
better generalization capabilities. As outliers are common in
fNIRS data, this regularized version of SVM may give better
results for BCI than the non-regularized version [27].
SVM algorithms attempt to find the optimal solution for
the hyperplane that maximizes the distance between the
hyperplane and the nearest training samples. These Classifiers
are designed to maximize the distance to the nearest training
point. The linear decision boundary (separating hyperplane)
for both SVM was obtained during a training session prior to
the test session. After that the classification into “mental
arithmetic” or “mental counting” classes was performed by
projecting the test samples, acquired after each experimental
trial, on the hyperplane.
The classification was performed on the signal mean (SM)
of the values of the change in concentration of HbO and HbR.
The feature vector hence consisted of thirty 2-dimentional data
points for each subject. Four different temporal windows were
considered for classification to see the effect selection of
temporal windows on classification accuracies. The temporal
windows considered were 0-5 s (W0-5), 1-6 s (W1-6), 2-7 s
(W2-7) and 5-10 s (W5-10) within the entire 10 s task period.
by approximately 2-3 s after the onset of task period and it
takes around 5-6 s to reach its peak value.
The difference between the classification accuracies
acquired using different temporal windows was statistically
significant when tested using the t-test.
The t-values for classification accuracies obtained using
W2-7 versus those obtained using W0-5, W1-6 and W5-10
were found to be 0.00139, 0.0091 and 0.038 respectively,
which based on the 5% significance level show that the
performance of the features within 2-7 s temporal window was
significantly better than the other temporal windows.
IV.
DISCUSSIONS
In the present research, classification of fNIRS signals
corresponding to mental arithmetic and mental counting was
carried out successfully with an average accuracy of 82.4%
using signal mean of HbO and HbR during 2-7 s temporal
window within the total 10 s task period.
In our previous studies [11,13,28-30], it was shown that
using fNIRS, it is possible to classify the different activities
versus the rest period with high classification accuracies. The
activities considered in those studies were finger tapping,
mental arithmetic and mental counting and motor imagery. In
this study, however, two different activities were classified,
from the prefrontal cortex.
The reason for choosing the prefrontal activity as the
control paradigm was that it is less likely to be implicated in
case of motor disabilities, hence, suitable for providing control
commands for patients with motor disabilities. The signal
attenuation and hair artifacts are also low when acquiring
signals from the prefrontal cortex.
,
Five-fold cross validation, that mixes data into five
segments, one of which is used for testing and the rest four are
used for training, was performed to estimate the classification
accuracies. The 2-D features spaces for the different temporal
windows are shown in Figs. 3-5.
III.
RESULTS
The averaged classification accuracies of subjects using
features for different time windows are listed in Table 1. The
all-subjects averaged classification accuracy using W2-7 was
found to be 82.4%, whereas the same using W0-5, W1-6 and
W5-10 were found to be 61.6%, 67.4% and 72.4%,
respectively.
The higher classification accuracies using W2-7 was in
accordance with the literature [8, 9]. This may be attributed to
the fact that the hemodynamic response lags the neural activity
Fig. 3. The 2-D feature space for 0-5 s temporal window for Sub 3.
Fig. 4. The 2-D feature space for 1-6 s temporal window for Sub 3.
confirming the result of our previous motor cortex based
study. The hemodynamic response varied across the subjects
and therefore the classification accuracies also varied, see
Table 1. These variations could be attributed to the differences
in the shape of the head and different scalp-cortex distances
for different subject. Since the hemodynamic response
patterns for all four subjects were not similar, more subjects
should be recruited for experiments to confirm our findings.
The effect of habituation also plays an important role in
decreasing the activity strength of the signals acquired from
the prefrontal cortex. A continuing exposure to mental
arithmetic might result in lower hemodynamic response due to
habituation. Further study needs to be devised to investigate
the effects of habituation in mental arithmetic-induced
hemodynamic response and thereby classification accuracies.
Another limitation of our present study is that online
classification [7] was not performed. In case of online
classification the classification accuracies might decrease. In
future, we aim to classify more than 2 activities online for
development of a BCI that can have more than two control
commands.
V.
Fig. 5. The 2-D feature space for 2-7 s temporal window for Sub 3.
Table 1.
SVM CLASSIFICATION ACCURACIES
CONCLUSIONS
In this paper, we classified two different activities from the
prefrontal cortex for development of a BCI and investigated
the role of using different temporal windows for classification
on classification accuracies. It was shown that it is possible to
classify, the fNIRS signals corresponding to mental arithmetic
and mental counting and the classification accuracies were
improved when the 2-7 s temporal window was use for
classification. The average accuracy was found to be of 82.4%
using the mean value of the changes in concentration of HbO
and HbR signals acquired during the 2-7 s temporal window
within the entire 10 s task period. The classified signals can
then be used as the two control commands for the control of or
communication with the external devices. The results of this
research successfully prove the prefrontal activities acquired
using fNIRS to be the potential candidates for BCI
applications.
ACKNOWLEDGMENT
Subject
No.
Classification accuracies (%)
W0-5
W1-6
W2-7
W5-10
1
60
73.3
90
80
2
66.6
70
86.6
73.3
3
63.3
66.6
73.3
66.6
4
56.6
60
80
70
Average
61.6
67.4
82.4
72.4
This work was supported by the National Research
Foundation of Korea under the Ministry of Science, ICT and
Future Planning, Republic of Korea (grant no. NRF-2014R1A2A1A10049727).
REFERENCES
[1]
[2]
In another study [8], we performed a detailed temporal
analysis of fNIRS signals corresponding to the right- and leftwrist motor imageries. The results showed that the 2-7 s
temporal window within the total 10 s task period is the best
temporal window for classification. In this study, the same
analysis has been carried out on the prefrontal activities,
[3]
[4]
J.R. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller, and T.M.
Vaughan, “Brain-computer interfaces for communication and control,”
Clin. Neurophysiol. vol. 113, no. 6, pp. 767-791, June 2002.
F. Matthews, B.A. Pearlmutter, T.E. Ward, C. Soraghan, and C.
Markham, “Hemodynamics for Brain Computer Interfaces,” IEEE
Signal Process. Mag. vol. 25, no. 1, pp. 87-94, December 2008.
S.M. Coyle, T.E. Ward, and C.M. Markham, “Brain-computer interface
using a simplified functional near-infrared spectroscopy system,” J.
Neural. Eng. vol. 4, no. 2, pp. 219-226, September 2007.
R. Sitaram, H. Zhang, C. Guan, M. Thulasidas, Y. Hoshi, A. Ishikawa,
K. Shimizu, and N. Birbaumer, “Temporal classification of multichannel
near-infrared spectroscopy signals of motor imagery for developing a
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
brain-computer interface,” NeuroImage, vol. 34, pp. 1416-1427,
February 2007.
M. Naito, Y. Michioka, K. Ozawa, Y. Ito, M. Kiguchi, and T. Kanazaw,
“A communication means for totally locked-in ALS patients based on
changes in cerebral blood volume measured with near-infrared light,”
IEICE Trans. Inf. Syst. vol. 90, pp. 1028-1037, July 2007.
S. Luu, and T. Chau, “Decoding subjective preferences from single-trial
near-infrared spectroscopy signals,” J. Neural. Eng. vol. 6, no. 1,
February 2009.
N. Naseer, M.J. Hong, and K.-S. Hong, “Online binary decision
decoding using functional near-infrared spectroscopy for development of
brain-computer interface,” Exp. Brain Res. vol. 232, no. 2, pp. 555-564,
February 2014.
N. Naseer and K.-S. Hong, “Classiﬁcation of functional near-infrared
spectroscopy signals corresponding to right- and left-wrist motor
imagery for development of a brain-computer interface,” Neurosci. Lett.
vol. 553, pp. 84-89, October 2013.
N. Naseer, K.-S. Hong, "Determination of temporal window size for
classifying the mean value of fNIRS signals from motor
imagery," Proceedings of the IEEE International Conference on
Robotics, Automation and Mechatronics (IEEE RAM), Manila,
Philippines, pp. 237-240, November 12-15, 2013.
N. Naseer, K.-S. Hong, “Discrimination of right- and left-wrist motor
imagery using fNIRS: towards control of a ball-on-a-beam
system,” Proceedings of the 6th International IEEE Engineering in
Medicine and Biology Society (IEEE EMBS) conference on Neural
Engineering, San Diego, CA. USA, pp. 703-706, November 6-8, 2013.
N. Naseer, K.-S. Hong, "Functional near-Infrared spectroscopy based
discrimination of mental counting and no-control state for development
of a brain-computer interface," Proceedings of the 35th International
Conference of IEEE Engineering in Medicine and Biology Society
(IEEE EMBS), Osaka, Japan, pp. 1780-1783, July 3-7, 2013.
N. Naseer, K.-S. Hong, "Functional near-infrared spectroscopy based
classification of prefrontal activity for development of a brain-computer
interface," The 10th Annual Congress of Society of Brain Mapping
and Therapeutics (SBMT), Baltimore, Maryland, USA, May 12-14,
2013.
N. Naseer and K.-S. Hong, “Functional near-infrared spectroscopy based
brain activity classification for development of a brain-computer
interface,” Proceedings of the IEEE International Conference on
Robotics and Artificial Intelligence (ICRAI), Rawalpindi, Pakistan, pp.
174-178, October 22-23, 2012.
N. Naseer, K.-S. Hong, M.R. Bhutta, and M.J. Khan, “Improving
classification accuracy of covert yes/no response decoding using support
vector machines: An fNIRS study,” Proceedings of the IEEE
International Conference on Robotics and Emerging Allied
Technologies in Engineering (iCREATE), Islamabad, Pakistan, pp. 6-9,
April 22-24, 2014.
M.J. Khan, M.J. Hong, and K.-S. Hong, “Decoding of four movement
directions using hybrid NIRS-EEG brain-computer interface,” Front.
Hum. Neurosci., vol. 8, 244, pp. 1-10, April 2014.
M.J. Khan, K.-S. Hong, M.R. Bhutta, and N. Naseer, “fNIRS based dual
movement control command generation using prefrontal activity,”
Proceedings of the IEEE International Conference on Robotics and
Emerging Allied Technologies in Engineering (iCREATE), Islamabad,
Pakistan, pp. 244-248, April 22-24, 2014.
M.R. Bhutta, K.-S. Hong, N. Naseer, and M.J. Khan, “Water correction
algorithm to improve the classification accuracy: A near-infrared
spectroscopy study,” Proceedings of the IEEE International Conference
on Robotics and Emerging Allied Technologies in Engineering
(iCREATE), Islamabad, Pakistan, pp. 91-94, April 22-24, 2014.
M.J. Khan, K.-S Hong, N. Naseer, M.R. Bhutta, S.H. Yoon, "Hybrid
EEG-NIRS BCI for rehabilitation using different brain
signals," Proceedings of the Annual Conference of the Society of
Instrument and Control Engineers (SICE), Sapporo, Japan, pp. 17681773, September 9-12, 2014
M.R. Bhutta, K.-S. Hong, N. Naseer, M.J. Khan, "Hemodynamic signals
based lie detection using a new wireless NIRS system," Proceedings of
the Annual Conference of the Society of Instrument and Control
Engineers (SICE), Sapporo, Japan, pp. 979-984, September 9-12, 2014.
N. Naseer, K.-S. Hong, M.R. Bhutta and M.J. Khan, “ Improving
classification accuracy of covert yes/no response decoding using support
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
vector machines: an fNIRS study,” Proceedings of IEEE International
Conference on Robotics and Emerging Allied Technologies in
Engineering (iCREATE), Islamabad, Pakistan, pp. 6-9, April 22-24,
2014.
M.R. Bhutta, K.-S Hong, N. Naseer and M.J. Khan, “Water correction
algorithm to improve the classification accuracy: a near-Infrared
spectroscopy study,” Proceedings of IEEE International Conference on
Robotics and Emerging Allied Technologies in Engineering
(iCREATE), Islamabad, Pakistan, pp. 91-94, April 22-24, 2014.
M.J. Khan, K.-S Hong, M.R Bhutta and N. Naseer, “ Dual movement
control
command
generation
using
prefrontal
brain
activity,” Proceedings o f IEEE International Conference on Robotics
and Emerging Allied Technologies in Engineering (iCREATE),
Islamabad, Pakistan, pp. 244-248, April 22-24, 2014.
N. Naseer, K.-S. Hong, “Decoding answers to four-choice questions
using functional near-infrared spectroscopy,” J. Near Infrared Spectrosc.
vol. 23, in press, May 2015.
N. Naseer, K.-S Hong, “fNIRS-based brain-computer interfaces: a
review,” Front. Hum. Neurosci. vol. 9, 3, pp. 1-15, January 2015.
K.-S. Hong, N. Naseer, and Y.-H. Kim, “Classification of prefrontal and
motor cortex signals for three-class fNIRS-BCI,” Neurosci. Lett. vol.
587, pp. 87-92, February 2015.
M.J. Khan, K.-S. Hong, N. Naseer, and M.R. Bhutta, “A hybrid EEGfNIRS BCI: Motor imagery for EEG and mental arithmetic for fNIRS,”
Proceedings of the 14th International Conference on Control,
Automation and Systems (ICCAS), KINTEX, Korea, pp. 275-278,
October 22-25, 2014.
F. Lotte, M. Congedo, A. L´ecuyer, F. Lamarche, and B. Arnaldi, “A
review of classification algorithms for EEG-based brain–computer
interfaces,” J. Neural Eng. vol. 4, no. 2, pp. R1-R13, June 2007.
N. Naseer, K.-S. Hong, M.J. Khan and M.R. Bhutta, “Classification of
prefrontal and motor cortex activities for development of three-class
fNIRS-BCI,” The 20th Annual Meeting of the Organization for Human
Brain Mapping (OHMB), Hamburg, Germany, June 8-12, 2014.
M.J. Khan, K.-S Hong, N. Naseer and M.R. Bhutta, “Multi-decision
detection using EEG-NIRS based hybrid brain-computer interface
(BCI),” The 20th Annual Meeting of the Organization for Human Brain
Mapping (OHMB), Hamburg, Germany, June 8-12, 2014.
M.R Bhutta, K.-S Hong, N. Naseer and M.J. Khan, “Classification of lie
and truth in forced choice paradigm: an fNIRS study,” The 20th Annual
Meeting of the Organization for Human Brain Mapping (OHMB), June
8-12, 2014.