1. No category

Howtograsp
aripetomato
Lennart Verhagen
How to grasp a ripe tomato
Lennart Verhagen
Ph.D. thesis, Utrecht University, March 2012
How to grasp a ripe tomato
© 2012 Lennart Verhagen
ISBN 978-94-91027-29-1
A digital version of this thesis can be downloaded from http://www.lennartverhagen.com.
This work is licensed under CreativeCommons BY NC ND . It may be transmitted and reproduced
for non-commercial purposes only, if no alterations are made and the author is attributed.
How to grasp a ripe tomato
Hoe je een rijpe tomaat grijpt
(met een samenvatting in het Nederlands)
Proefschrift
ter verkrijging van de graad van doctor aan de Universiteit Utrecht
op gezag van de rector magnificus, prof. dr. G.J. van der Zwaan,
ingevolge het besluit van het college voor promoties
in het openbaar te verdedigen op
vrijdag 23 maart 2012 des middags te 4.15 uur
door
Lennart Verhagen
geboren op 5 oktober 1982 te Leiden
Promotor:
Prof. dr. A. Postma
Co-promotoren:
Dr. H.C. Dijkerman
Dr. I. Toni
The publication of this thesis was financially supported by MagVenture A/S,
neuroConn GmbH, easycap GmbH, and MedCaT BV.
Contents
1. Introduction
9
2. Biasing perceptuo-motor integration
by object slant and vision
39
3. Methodological considerations in multi-modal
neuroimaging of visually-guided grasping
51
4. A simple and effective method to remove
global nuisance variance in fMRI
67
5. Cortical circuits supporting perceptuo-motor
interactions during grasp planning
87
6. The anterior intraparietal sulcus structures a grasp
plan, integrating metric and perceptual information 111
7. Cortical dynamics of dorsomedial and dorsolateral
parieto-frontal circuits during grasping movements
141
8. Discussion
169
Appendix
Dutch summary
Reference list
Donders series
Acknowledgements
Publication list
Curriculum vitae
189
191
199
223
229
232
233
1
Introduction
10 | chapter 1
i n t r o d u c t i o n | 11
Introduction
Dave: Open the pod bay doors, HAL.
HAL: I’m sorry, Dave. I’m afraid I can’t do that.
In all fairness, HAL really physically couldn’t, even if he would have wanted to. This
iconic fictional super computer, sprung from the brain of science fiction author Arthur
C. Clarke, matches and even surpasses humans in perceptual capabilities, knowledge
and reasoning, but it distinctively lacks the ability to move and act, let alone open
pod bay doors. In real life, a computer has beaten the world chess champion, but the
pieces on the board had to be moved by a human accomplice. Even grandmasters of
Jeopardy, a game-show requiring the processing of natural language and a profound
expanse of encyclopaedic knowledge, have had to acknowledge their better in a
computer called Watson. But while the human contestants could wipe the sweat
of their foreheads and grab a glass of water during the breaks, Watson, the poor
computer, was only equipped with a little mechanical arm merely capable of pressing
the buzzer, nothing more.
When we set out to challenge the capabilities of the human brain we choose
our battlefield in higher cognitive functions, but ignore an ostensibly much
more mundane cerebral function: the control of our actions. The guidance of our
movements, particularly those of our hands and fingers, comes to us so naturally
we easily overlook not only the important role it plays in our everyday life, but also
the distinctiveness of this human characteristic. While modern chess programs can
defeat any human player, any six year old child will exhibit more effortless adaptive
skill in grasping and manipulating the chess pieces than the currently most advanced
electro-mechanical devices. Likewise, while a sick male chimpanzee could still easily
overpower a very fit man, it is the man, not the chimpanzee, who takes up his pencil
and writes down a note of this event in his logbook (if he still can). These differences
are not a matter of mechanical freedom and limitations, but one of action control.
This thesis will investigate the cerebral control of visually-guided grasping
behaviour. Picking up a ripe tomato, for example, or grasping a glass of water. Such
1
12 | chapte r 1
grasping movements are amongst the most ecologically relevant and extensively
studied movements of primates. We can build upon a detailed description of both
the computational and neuroanatomical foundations supporting the transformation
of sensory information into motor parameters guiding the hand and fingers
towards the target object 1-4 . A prominent account of cerebral visuomotor control
distinguishes, both functionally and anatomically, between extrinsic and intrinsic
characteristics of the target object 5 . The arm and hand are guided towards the
object’s extrinsic location in space, while independently the grip of the hand is
formed according to the object’s intrinsic size and shape. However, these metric
parameterizations of visual information alone are not sufficient to understand how
we grasp 6 . For instance, when plucking a tomato we habitually adjust our grip to its
colour and identity. If we would grasp an overripe red tomato in the same way as we
would a green one, things might get messy. Similarly, when we grab a glass of water
we effortlessly incorporate its perceived weight and slipperiness into our motor plan;
we don’t want to drop it or spill the water. These non-metric sources of information
influence our grasping behaviour in a profound way, relying on the transformation of
perceptual and semantic cues into motor code.
In healthy human behaviour perceptual information is incorporated so
seamlessly in action control, that it might not be immediately evident why this
integration would be unexpected in the first place, or what its important implications
could be. However, both points are exemplified by behavioural observations in
neuropsychologically impaired patients.
First, visual agnosia patients, with lesions in occipito-temporal cortex but
spared parietal functioning7, are unable to identify an object or to report on its size
and shape, but they can accurately act upon the same object, adjusting their grip to
the unrecognized metric object features8 . However, when multiple grip solutions are
available they cannot rely on object knowledge to adopt the most appropriate one9 .
For example, if a fork is oriented with the prongs facing towards them, they won’t
orient their grip in order to use the fork, but instead grasp it at the end closest to them.
These findings strongly suggests that object recognition and action guidance rely
on functionally and anatomically distinct processes 10 . Consequentially, perceptuomotor integration is not a straightforward linear computational progression but
instead critically depends on the communication and cooperation between two
otherwise independent and parallel cortical streams.
Second, in contrast to visual agnosia, patients with optic ataxia, associated
with superior parieto-occipital lesions 11 , are often impaired when asked to grasp a
simple cylinder, but they are able to identify objects 12,13 . Importantly, when they
recognize the target object and can rely on prior motor experience, for example
when the cylinder is replaced by a glass of water of the same shape and size, they
are able to act appropriately and accurately14 . Evidently, perceptual and semantic
in t r o d u c t i o n | 13
object knowledge can compensate for the absence of metric grasp guidance, but
more speculatively, these sources of information may have more to offer than
just compensation 15,16 . Hypothetically, they could promote fast and appropriate
adaptive behaviour based on heuristic motor control, forming the basis for complex
object interactions and effective use of tools; functions in which humans excel 17.
This thesis aspires to gain insight into how the human brain incorporates
perceptual information into a motor plan. How does the cortical motor system
anticipate the required grip to make sure we don’t squish an overripe tomato when
picking it up, or spill water when grabbing a glass? As discussed in the following
sections, visuomotor processing is organized in parallel cortical pathways, but the
functional contributions of these segregated circuits are relatively poorly understood.
Even less is known about how these circuits are connected, share information
with each other and with perceptual processing streams, together supporting the
guidance of grasping movements to achieve a common goal. Before embarking
on the main topic of interest, the following sections will describe the outline of the
thesis and sharpen the research question. Subsequently, the neuroanatomy of the
cerebral circuits involved in motor guidance will be examined. These anatomical
foundations are discussed in the context of several influential functional models of
grasp control. Finally, the stage will be set for the novel additions of this thesis to the
understanding of action control by highlighting two recent significant developments
in the understanding of cerebral motor function: corticospinal organization and
predictive coding.
Outlineofthethesis
This thesis aims to understand how the brain controls our grasping movements, and
especially how perceptual information is incorporated into the grasp plan. Chapter
2 proposes to experimentally approach this question by taking advantage of the
distributed processing of binocular and monocular cues of depth when preparing
a grasping movement. The information value of these depth cues differentially
depends on object slant, providing a handle to bias the reliance on either metric or
perceptual cues, irrespective of object characteristics, task difficulty and viewing
conditions.
To characterize the behavioural and cortical correlates of perceptuo-motor
integration, both kinematic measurements and several complementary neuroimaging
and neurointervention modalities are employed in the studies described in this
thesis: functional magnetic resonance imaging (fMRI), electroencephalography
(EEG), and transcranial magnetic stimulation (TMS). The neurophysiological and
methodological underpinnings of these techniques will be discussed in chapter 3.
1
14 | cha p t e r 1
Studying visually-guided movements in an MR environment poses some specific
challenges: the space available inside the scanner bore is restricted, and fMRI data
acquisition is highly susceptible to motion artefacts. Chapter 4 will introduce a novel
protocol to estimate and remove global confounds in the fMRI signal, for example
caused by movements of the hand and head, aiming to improve data quality and
statistical inference. The general applicability of this protocol in fMRI paradigms,
also outside the domain of motor control, is tested in three different datasets.
Having laid the theoretical and methodological foundations, chapter 5
proceeds to describe the cerebral structures supporting the planning of visuallyguided grasping movements. The cortical correlates of increased computational
requirements for visuospatial-motor transformation are isolated from those of
increased reliance on perceptual information for accurate prehensile behaviour.
Chapter 5 further examines how the necessity of incorporating perceptual
information affects the functional coupling of the anterior intraparietal cortex within
the dorsolateral sensorimotor channel, and with the ventral visual stream. In chapter
6 the cortical dynamics of the computational processes underlying perceptuo-motor
integration are further investigated. Using concurrent TMS and EEG, the critical
contribution and temporal dynamics of the anterior intraparietal sulcus in relation
to the visuomotor system is investigated. Chapter 7 describes how the contributions
of the dorsolateral and dorsomedial circuits to visuospatial-motor parameterization
are functionally and temporally integrated, supporting shared guidance of accurate
visuomotor behaviour.
Finally, chapter 8 will provide an overview of the novel findings described in
this thesis and will integrate them with our current understanding of visuomotor
control, informing a revised view on the functional interpretation of the dorsomedial
and dorsolateral parieto-frontal circuits. It will provide an outlook on how this view
influences our understanding of visuomotor control in general and proposes possible
directions of future research.
Restrictingtheresearcharena
Action is central in our behaviour; in fact, it constitutes our behaviour. Provocatively,
it has been suggested to be the reason why animals have a nervous system in the
first place 18 . Accordingly, the study of action control touches upon too many facets
of the brain and behaviour to cover completely, no matter how well intended
the endeavour. This thesis will focus on the control of visually guided grasping
movements, and below, the research arena will be restricted correspondingly, both
functionally and anatomically.
Our hands are the most versatile of our effectors and play a privileged role in
our behaviour19 . Amongst all the movements we can perform with our hands, their
i n t r o d u c t i o n | 15
interactions with objects comprise a central theme, one where humans distinguish
from other animals 20 . As a direct consequence of the focus of this thesis, other
important topics in hand control will only be indirectly addressed, such as reaching
and pointing, eye-hand coordination, hand-body interactions, arbitrary sensorimotor
transformations, or somatosensory and internally guided movements. Similarly, the
current examination is restricted anatomically to the cortical correlates of grasp
guidance3,4 , leaving aside for now the cerebellum and subcortical areas, such as the
basal ganglia and the thalamus. Notwithstanding the critical contributions of these
subcortical areas to motor behaviour, the neocortex of the cerebrum is of particular
interest for the study of perceptuo-motor interactions and perhaps not surprisingly
most strongly developed and enlarged in our phylogenetic history21 .
A further important refinement of the research arena concerns the temporal
evolvement of an action: from vision, via action selection, and motor preparation,
to movement execution, online control and goal monitoring. A subdivision of this
natural evolution will be arguably arbitrary, but this oversimplified cartoon model
with artificially delineated movement phases progressing in a linear fashion will
help to avoid misconceptions. The experiments described in this thesis focus
predominantly on the movement planning phase. Crucially, movement planning is
not equivalent to the selection of an action and its components from competing
options 22,23 , nor to the selection of the goal, target and effector of the action 24,25 .
An everyday example illustrates the sometimes surprisingly subtle differences.
When you’re thirsty you might want to grab a glass of water in order to drink from
it. In that case the goal of the forthcoming glass-grabbing-action is clearly defined,
but selecting the target of the action might be a different matter. The selection is
trivial when only one glass of water present on the countertop, but it can be more
difficult when you’re at a busy party or haven’t done the dishes in a while. You use
perceptual and semantic cues, like shape, colour, texture and object knowledge, to
pick the glass which looks most appropriate 26 . Similarly, the selection of the effector
and action seems quite straightforward: in this example you will probably perform a
reach-to-grasp movement with a power grip 19 . In other circumstances, the effector
selection can strongly depend on the goal and context of the action. For instance,
when you intend to place the glass in a high kitchen cabinet you might hold it only
at the tips of your fingers, and if you want to place it upside down in the dishwasher
you grasp it at the bottom and prepare for an object rotation 24 . For all these cases
perceptual cues might strongly influence the selection of the action, but crucially,
these cues are not strictly necessary for the preparation of the action itself. It might
be argued that once the goal, target, effector and grip type are chosen, the action
itself could be solely defined using metric visuospatial parameters describing the
desired end-positions of the fingers 27,28 . But instead, when you plan and execute
the grasping movement towards the glass of water, you do consider the perceived
slipperiness and weight of the glass and incorporate this information in your
1
16 | cha p t e r 1
movement. This adaptive motor plan evolves during the preparation phase and may
even continue to do so during the early stages of the movement, solely based on
advance information 29 . The scope of the investigations described in this thesis will
be limited to the preparation phase and will ignore, for now, how the brain handles
the feedback we receive, both while moving our hand towards the glass and upon
touch30-32 . This limitation has pragmatic reasons: to keep the experimental inquiries
within practical boundaries, and to prevent neuroimaging measurements from being
infected by potential behavioural changes. But it also has a theoretical rationale:
online movement control, and any other handling of visual and tactile feedback,
is grounded upon a pre-existing movement plan, and their respective underlying
computations may strongly overlap, as will be examined later in this introduction
and in chapters 6 and 7.
Neuroanatomy
Visualprocessingpathways
The neuroanatomical foundations of visually guided voluntary actions, such as
grasping a glass of water, will be discussed following the signal from the eye, through
the brain, to the hand. Light falling on the retina is transformed in neural signals that
travel through the lateral geniculate nuclei (LGN) and terminate in the primary visual
cortex33,34 . From here onwards, visual processing is organized in two functionally and
anatomically segregated circuits: the ventral stream, supporting object recognition
and running inferiorly towards anterior temporal cortex, and the dorsal stream,
controlling goal-directed action and extending into the superior part of the posterior
parietal lobe 10,35,36 . The integration of vision-for-recognition and vision-for-action,
the topic of this thesis, requires the brain to bridge this functional and anatomical
divide.
The two streams redundantly represent several object features, such as
object size and orientation37,38 . Yet, when a perceptual response on these features
is requested, the ventral stream is more active and provides the only critical
representation39 , while the dorsal stream, in contrast, critically contributes to
guidance of a grasping movement towards the same object 40 . Although the exact
functional labels attributed to the ventral and dorsal visual streams might be
debated, one important insight remains undisputed: cortical processing is not only
defined by the input, but also by the task 16,41 .
i n t r o d u c t i o n | 17
Theuntamedjungleoftheparietalcortex
Following the path from eye to hand, the neural processing in the occipital cortex
is passed on to the parietal lobe, and here the cortical organization can no longer
be simply referred to as a visual pathway. On the anterior side of the parietal lobe,
opposite to the border with occipital cortex, somatosensory information first enters
the cerebrum in the postcentral gyrus. Moreover, parietal cortex performs many
computations which could be accurately described as part of a motor, and not a
visual, system 42 . Classically, the parietal lobe has been regarded as multi-modal
association cortex, where visual and tactile information is integrated to build a
spatial representation of the environment35,43,44 . It has received further interest for
its contributions to spatial attention 45-47 and its distinct representations of one’s
own body and peripersonal space43,48-50 , supporting ownership and agency51,52 .
Specifically in humans, it is considered to provide the cortical grounds for a number
of higher-order and potentially culturally characterized functions, such as arithmetic,
and representation of quantity, number and time53-55 .
Following recent developments in human neuroimaging and primate
electrophysiology, more attention has been devoted to the motor processes of the
parietal lobe4,42 . Quite a plethora of proposed motor functions and representations
of the parietal lobe has arisen, many redundant and overlapping 56-58 . This might be
expected from a new field with competing hypotheses, where most data is acquired
with very indirect measurements of neuronal computation. However, a similar view
also arises on the basis of direct electrophysiological measurements of neuronal
activity, for example, when asking a monkey to look at or reach for an object,
presented at several orientations and locations. Neurons with similar preference for
a specific retinal location are often found grouped together, but importantly, they
are interspersed with neurons whose responses are only selective to the objects
orientation, independent of its location 59-61 . Overlapping selectivity for orientation
and location is also present in the occipital cortex62,63 , but in the parietal cortex this
trend of multi-dimensionality is developed to great extent.
The parietal topographical representations according to retinotopy and
somatotopy are prominent64,65 , but many more can be identified: for example
according to visuospatial reference frame (e.g. allocentric, eye-, head-, body-, or
hand-centred coding) 66-68 , movement effector (e.g. eye or hand) 58,69-71 , movement
type (e.g. saccadic, reaching, or grasping) 72,73 , and target space (e.g. peri- vs.
extra-personal space) 74,75 . Evidently, these representations, either topographically
structured or not, are inter-reliant and exhibit a strong cortical overlap. In fact,
this overlap might be a critical factor allowing rapid switching with minimal
computational load between different reference frames and representation
systems (for example, from eye to hand) 76-81 . Potentially, such a high-dimensional
topographical organization could emerge as a consequence of a ‘like attracts like’
1
18 | cha p t e r 1
grouping of multiple organizational gradients on a two dimensional cortical sheet,
as has been proposed for the frontal lateral motor cortex82 . In this framework, the
engagement of specific circuits is dependent on the functional context and requested
output73 , similarly to the task-specific activation of ventral and dorsal visual streams
as encountered above 63 . Such an organization could take advantage of redundant
coding within the parietal lobe, allowing adaptive and compensatory mechanisms83 .
Organizationoftheparietalcortex
Although one should acknowledge that cortical organization is neither strictly
hierarchical nor linear, a view of the parietal cortex as a terra incognita, overgrown
with a wild jungle of organizational gradients and representations, as the paragraph
above might evoke, would ignore a large and relevant body of work on this topic.
In spite of the provocative organizational jungle, studies of cytoarchitectonics
and anatomical connectivity have strongly informed our current understanding of
parietal structure84 . In the interest of the visuomotor focus of this thesis, particular
attention will be paid to the anatomical and functional organization of the parietal
cortex (see figure 1.1 for a schematic overview). To start, the postcentral sulcus
partitions parietal cortex in an anterior and posterior part. The former contains
the primary and secondary somatosensory area, where ascending somatosensory
pathways terminate. The latter part is the largest and most relevant for visuomotor
control. Within the posterior part, the intraparietal sulcus (IPS), a significant
landmark present in all primates, further separates the superior from the inferior
parietal lobule.
For a consistent and detailed labelling of anatomical and functional areas within
the posterior parietal cortex there is currently no other option than to heavily rely on
neuronal properties revealed by electrophysiological and cytoarchitectonic studies
in (macaque) monkeys, despite of recent technological advances. Although inferring
on anatomical homologies between species is notoriously difficult, based on recent
developments in monkey and human functional and anatomical neuroimaging
strong and informative parallels can be drawn between the macaque and human
brain 4,57,87,88 . Most of macaque superior parietal lobule (SPL) is denominated as
area PE 85 , similarly to Brodmann area 7 in humans89-91 . In the anterior bank of the
parietal occipital sulcus (POS) the parieto-occipital complex (PO) 92 encompasses
areas V6 and V6A 59 . The inferior parietal lobule (IPL) of the macaque can be divided
in anterior, intermediate, posterior and occipito-parietal-temporal regions (PF, PFG,
PG, and OPT, respectively) 85,93 . Areas PF and PFG are together putative homologues
of the human supramarginal gyrus (SMG), and areas PG and OPT of the angular
gyrus (AG) 89,94 .
in t r o d u c t i o n | 19
medial view
parietooccipital
sulcus
V6
V6A
F6
PEci
F3
F1
F1
F2
cen
t
sul ral
cus
F7
PE
PE
al
riet
apa
intr ulcus PG
s
PFG
PF
F4
op
PF
F5
ral
late us
c
sul
PGop
c V6A
parieto
occipit
al
sulcus
PGm
PE
PEc
PEc
p
PEi VIP
tal
parie
intra lcus AIP
su
PF
PE
MIP
V6
LIP
PG
PFG
intraparietal sulcus
unfolded
al
r
po
m
te s
r
u
o
ri lc
pe su
su
lateral view
Figure1.1Areas of macaque parietal and frontal motor system
Functional and anatomical labelling of posterior parietal, motor and premotor
cortices according to Pandya and Seltzer85 and the denomination of Rizzolatti and
Matelli 86 . Lateral parietal areas are coloured in yellow and reddish tones. Areas
located inside the intraparietal sulcus are coloured green (see unfolded projection
on the right). Inferior parietal areas are shaded lighter than the superior parietal
areas. Frontal areas are coloured blue, with lighter shades for ventral premotor and
darker shades for dorsal premotor areas. Please see the text for further details.
Figure adapted from Rizzolatti and Matelli 86 .
As mentioned before, two main sources of information, somatosensory
and visual, come together in the parietal cortex, entering from different sides.
Somatosensory information is prevailing in anterior regions of the posterior parietal
cortex (in PF, but especially in PE) but its dominance fades towards posterior areas 95 ;
V6 in the POS is only sensitive to visual information 59,96 . There is a third important
modality, next to somatosensation and vision, which characterizes a parietal
neuron’s response: functional motor context. Some neurons only respond to
1
20 | chapter 1
sensory stimulation and are not modulated at all by the task context, such as motor
planning or execution. Others are responsive to both sensory and motor modalities,
and some only to motor aspects, irrespective of the presence of a stimulus 97. Based
on the anterior-posterior gradient of the sensory modalities, one might expect the
latter two neuronal types to be mainly located in the centre of parietal cortex. This
is partly true, as sensorimotor and motor neurons are indeed abundantly found in
the deep divide of the IPS 57. However, within the IPS, there is not a clear gradient, in
fact a mosaic of areas is observed, many of which, both on the extreme anterior and
posterior ends, contain neurons of all three preference types (sensory, sensorimotor,
and motor) 98-102 .
These features make the IPS of particular interest for our investigations of
visuomotor control. In the superior bank of the IPS, the medial intraparietal area
(MIP) is bordered on its posterior side by V6A in the POS, and on its anterior side by
the intraparietal part of area PE (PEip) near the postcentral sulcus. The fundus of the
IPS situates the ventral intraparietal area (VIP). The inferior bank of the IPS includes,
in a posterior to anterior organization, the caudal, lateral and anterior intraparietal
areas (CIP, LIP and AIP) 36,103 .
Anatomyandfunctionoftheparieto-occipitaland
intraparietalsulci
Macaque area V6A, in the anterior bank of the parieto-occipital sulcus, is responsive
to both visual59,104 and somatosensory stimuli 61,105 . Recently, many visuomotor and
motor properties of V6A have been revealed 60,106 . The computations implemented in
V6A support spatial visuomotor transformations and have been strongly associated
with the guidance of the arm and hand in peripersonal space 107. However, it has
become clear that V6A is not only sensitive to a target’s location, but also to its
orientation and even to the shape of a target object, coding grip type in a prehension
task 108,109 . Although V6A is considered a single functional area, its ventral section
seems most specialized in the visuomotor control, and the dorsal part in the
somatomotor control of reaching and grasping movements 61 .
The putative human homologue of V6A (hV6A) is located in the superior bank of the
parieto-occipital sulcus (sPOS) 110,111 , just anterior to human area V6 112 . In accordance
to its functional attributions in the monkey, this region has been robustly implicated
in the guidance of the arm and hand during reaching and grasping movements,
both on the basis of functional neuroimaging experiments 72,113 and behavioural
observations of optic ataxia patients with lesions in superior parieto-occipital
cortex 11,13 . However, if and how sPOS could contribute to prehensile behaviour and
grip formation is currently under debate, both in health and disease 109,114-116 . This
topic will be discussed further in the introduction in the context of models of grasp
introduc tion | 21
control. In chapter 7 these questions will be experimentally addressed, informing a
revised interpretation of the functional role of sPOS in the visuospatial guidance of
the arm, hand and fingers in reaching and grasping movements.
Just anterior to V6A, in the superior bank of the IPS, area MIP is also often
implicated in the visuomotor control of reaching movements 117,118 . Accordingly, area
MIP is sometimes considered together with V6A as part of the parietal reach region.
An unfortunate misnomer as both these regions are not exclusively involved in reach
guidance, nor are they the only parietal areas to do so 58 . In macaques, MIP is also active
when using a joystick to guide a spot to a visual target 119 , and seems crucial for spatial
movement corrections based on visual and proprioceptive feedback mechanisms 120 ,
suggesting is it more generally involved in the generation and updating of spatial
movement vectors. These spatial visuomotor computations are grounded in a
gaze centred reference frame 121 . The functional properties of the putative human
homologue mIPS shares these characteristics: it’s important in visuospatial motor
transformation and guidance in reference to gaze coordinates 28,67,122 , both during
movements performed with the arm and hand 25,110,111 , but also when controlling a
joystick 123 .
Area VIP, in the fundus of the IPS, contributes to the motor control of the face
and head 124 , but also contributes to arm movements towards the mouth and face,
and defensive movements protecting the head 125,126 . Importantly, these movements
seem to be coded in head centred coordinates 127, revealing a different reference
frame preference than MIP. More generally, it codes a multisensory area of space
around the head, with matching somatosensory and visual receptive fields 128 . This
multimodal characteristic of VIP is extended to vestibular and auditory stimuli
presented in the context of head movements and peri-facial space 129 . This suggests
that, in accordance with our discussion of MIP and of organizational gradients in
general, parietal organisation is not critically dependent on the effector but rather
functionally specific73 . Recently, a human homologue of VIP in the IPS has been
identified that equals its macaque counterpart’s functional characteristics 66,130,131 .
Area LIP contributes to the predictive, preparatory and executive control of
saccades 132 , updating the spatial representation over several eye movements 133,134 .
Accordingly, the functional role of LIP has often been contrasted to that of MIP in the
superior medial bank of the IPS. The last two decades have seen a fierce debate over
the characterization of the function of parietal LIP, arguing if it is best described as
oculomotor control 135 or as spatial attention 46 . The two are bound at the hip, or eye
in this case, and intermediate proposals have arisen; for example suggesting LIP to
code a spatial salience map instead 136,137. Interestingly, a recent study proposes that
a ventro-dorsal gradient within LIP distinguishes between oculomotor contributions
and spatial attention 138 . The human homologue of LIP (hLIP) has been identified in
the IPS on the basis of retinotopy65 , involvement in saccade control 139 , eye-centred
1
22 | chapter 1
oculomotor vector representations updated over gaze shifts 67, and visuospatial
attention orienting 140-142 . Moreover, macaque LIP and its human counterpart have
also been proposed to play a role in a series of non-spatial tasks and complex cognitive
functions, including time perception, decision making, and reward expectancy57. It is
currently unclear if the contribution of hLIP is actually critical for these functions or
if hLIP is collaterally activated.
Area CIP in the most posterior and inferior part of the IPS is selective for threedimensional object features 143 . It is especially sensitive to orientation cues based on
stereoscopic disparity and linear perspective 144 . These qualities are in accordance
with those of the posterior part of the human IPS (cIPS) 37,145 . Interestingly, also
the most anterior part of the IPS codes 3D object characteristics, but, whereas
cIPS seems most responsive to surface orientation in isolated object features, the
anterior IPS is more concerned with the object as a whole 146 especially in the context
of object-hand interactions 147,148 .
In fact, the anterior part of the macaque intraparietal sulcus is selective to
many features of an object, such as its size, shape and orientation 98 , but not to its
location 100 . AIP codes object properties in a fast, coarse and metric fashion, distinct
from object representations in the ventral visual stream (e.g. in area TE in inferior
temporal cortex), and particularly well-suited to guide grip formation and object
manipulation 40 . Indeed, almost all neurons in AIP reveal object feature selectivity
only in the context of a prehension task, either during object presentation, movement
planning and execution 149 , and even shortly afterwards 150 . These visuomotor
neurons can be classified in three types according to their firing rate in different visual
conditions 99 . Neurons that only fire when movements are performed in the light,
but not in the dark are called ‘visual dominant’ neurons. ‘Motor dominant’ neurons
discharge equally during movements performed in either light or dark conditions.
Lastly, the ‘visual-and-motor’ neurons are characterized by their intermediate
response, firing in both conditions, but stronger in the light. Inactivation of AIP leads
to impaired preshaping of the hand in relation to object metrics 151 . Accordingly, AIP
has been considered as a central part of a dedicated ‘grasp-circuit’69 .
Based on both anatomical and functional properties the anterior part of human
intraparietal sulcus (aIPS) has now been firmly identified as the human homologue of
macaque AIP (hAIP) 152,153 . Grasping objects most prominently activates the IPS close
to the postcentral sulcus 154-157. At this site, different sources of depth information
converge to create higher-order representations of object shape, whereas more
posterior regions in the IPS are sensitive to lower-order features, such as depth,
orientation, shape and object location37,38,158,159 . Similarly to macaque AIP, human
aIPS provides physical object properties for grip formation 72,160,161 , but seems less
involved in controlling the transport component of the grasping movement towards
the target 113 . Interestingly, the contributions of aIPS to object interactions seem
introduction | 23
not to be limited to prehension. It is also imperative in tactile shape processing and
transfer of object information between visual and somatosensory systems 162 . In
primates, and especially in humans, aIPS’ contributions to hand-object interactions
are extended to include the use and observation of tools 163-166 . Lastly, the aIPS is also
active when observing object-oriented actions performed by someone else 167-169 ,
implicating it in action observation and intention understanding 170-172 . The diverse
functions of aIPS have been extensively studied over the last decade but are still
an active topic of debate. Further on in the introduction, following the anatomical
description of the motor system, the functional contributions of aIPS to grasp control
will be discussed in more detail. Chapters 5, 6 and 7 will explore these contributions
experimentally, building towards an integrative approach of perceptuo-motor
interactions and grasp control.
Frontalmotorsystem
The motor system in the frontal lobe encompasses the primary motor cortex in the
precentral gyrus and more anterior the lateral and medial premotor cortices (see
figure 1.1 for a schematic overview). The lateral premotor cortex is visible from the
surface, but most of the medial premotor cortex is obscured, folded into the medial
longitudinal fissure. The lateral premotor cortex includes the ventral premotor
area (PMv in humans, areas F4 and F5 in macaque) and the dorsal premotor area
(PMd in humans, areas F2 and F7 in macaque) 173,174 . PMv supports spatially guided
movements, especially those involving the hand and fingers, with a particular focus
on prehension and object interactions 175 . Additionally, PMv codes more abstract
aspects of motor behaviour, such as the goal of an action independent of the
effector or kinematic characteristics 176,177, and importantly, the observed actions of
others 178-180 . It has been suggested that PMv contains an action vocabulary which
can be accessed both to guide one’s own actions 176 and to observe and understand
those of others 181 .
In contrast, the dorsal premotor cortex has been associated with reaching
movements in peripersonal space involving proximal muscles of the arm 95,117,
and more generally, with the selection, preparation, execution and evaluation
of movements towards visuospatial targets 23,182,183 . Moreover, PMd is critical in
arbitrary, in contrast to spatial, sensorimotor behaviour, for example when you
are waiting for a traffic light and press down the gas pedal as soon as it turns to
green 114,184 . This requires the transformation of arbitrary rules to motor code without
a direct spatial congruency between the stimulus and the action 26,185 .
The medial premotor cortex situates the supplementary motor area (SMA,
designated F3 in macaque) and pre-supplementary motor area (pre-SMA, F6
in macaque) 186 . These areas are closely linked to the motor cortex, not only
1
24 | chapter 1
topographically, but also in their neural representations. They preferentially code
motor actions and are less sensitive to visual information 187, but they do receive
strong contributions from somatosensory areas 188 . They assist the motor cortex in
action execution by holding online a pre-specified action or even multiple potential
actions 183,189 . SMA and pre-SMA convert arbitrary stimulus-response mappings to
appropriate movements 190 and support task initiation and switching, movement
sequencing and action monitoring 191 .
Parieto-frontalsensorimotorcircuits
Parietal and frontal cortical areas are heavily interconnected, forming a parietofrontal network69,182,192 . The characterization of connectivity patterns within this
network relies heavily on invasive investigations of the monkey brain, even more so
than the definition of cytoarchitectonically discrete areas, as previously described.
Hence, the following description will mainly focus on macaque tracer injection
studies revealing anterograde and retrograde connections leaving and entering the
injection site, and use the labelling of macaque cortical areas accordingly.
The organization of the parietal cortex is characterized by multiple overlapping
gradients and organizational rules. The same could be said of its connections with
frontal areas, but still a few basic, albeit conditional, principles can be described. First,
a hierarchical organisation of areas, if present, is broadly maintained. The general
anterior to posterior hierarchy of the frontal cortex is by approximation mirrored by
the hierarchical posterior to anterior organisation of the (somatosensory) parietal
cortex, with strong connections between the primary somatosensory area (SI) and
anterior PE to the primary motor cortex (F1 in macaque monkey) 193,194 . Second,
parietal and frontal areas with a preference for coding movements of the same
type and effector are most strongly interconnected. For example, parietal area LIP,
important for planning saccadic eye movements, strongly projects to the frontal
and supplementary eye fields (FEF and SEF) 195,196 . A similar argumentation might
hold for the strong connections between AIP and F5 (regarding object interactions
with the hand and finger), and VIP and F4 (regarding arm movements near the
face) 197. Lastly, and most arguably most prominently, the parieto-frontal network
is organized along the ventro-dorsal axis in two largely segregated circuits86,198,199 .
To avoid confusion with the ventral and dorsal visual streams, these parieto-frontal
circuits are labelled according to their relatively medial and lateral position within,
and as an extension of, the dorsal visual stream.
The dorsomedial circuit connects superior posterior parietal cortex (PE in SPL,
V6A and MIP in superior POS and IPS) with dorsal premotor cortex (mainly F2 and
F7) 106,182,200 . In contrast, the dorsolateral circuit connects AIP, VIP (in inferior and
lateral IPS), and PF (in IPL) with ventral premotor cortex (F5 and F4) 2,86,197. The
introduction | 25
separation between dorsomedial and dorsolateral circuits is phylogenetically old,
and has been suggested to form the basis of two major distinct protogradations
(directions of progressive phylogenetic cortical differentiation), one originating in
the medial longitudinal fissure, the other in the lateral sulcus (Sylvian fissure) 201,202 .
The parieto-frontal connections are bi-directional: AIP does not only project onto
F5, F5 also projects onto AIP197,203,204 . This breaks a strict hierarchy between parietal
and frontal areas and allows reverberated processing of action and perception.
The human cortical organization of the parieto-frontal connections has similarly
been proposed to be divided in two largely segregated dorsomedial and dorsolateral
circuits 2,199,205 . Although the anatomical foundations seem relatively undisputed,
several competing proposals for a functional division of labour have arisen in recent
years. Further on in the introduction several models of grasp control will be examined,
focussing specifically on how they relate to the two parieto-frontal circuits. The
functional interpretation of these distinct networks will form an integral part of this
thesis and will be discussed extensively in chapters 5, 6, and 7.
Primarymotorcortexandthecorticospinaltract
The premotor areas project strongly onto the primary motor cortex, the most
prominent cortical site were movements are structured 193 . In premotor regions, the
control of proximal (e.g. the arm) and distal muscles (e.g. the fingers) is organized
along a topographical gradient, whereby PMd mainly controls the former and PMv
the latter206-208 . This segregation is partly maintained in the primary motor cortex,
where most of the neurons projecting to proximal muscles are found medial and
superior to those innervating distal muscles 209 , although there is considerable
overlap between these neuronal populations 210,211 . Interestingly, while proximal
muscles receive projections from the motor cortex in both hemispheres 212 , distal
muscles are strictly unilaterally innervated213,214 .
The motor cortex, and hence its descending signals to the spinal cord, is
intimately regulated in a cortico-subcortico-cortical loop 215-218 . The motor cortex
stimulates the striatum, one of the main basal ganglia. From here the signal can
travel along two main routes through the different nuclei of the basal ganglia. The
routes, whose activity is influenced by the neurotransmitter dopamine, differ in the
strength by which they inhibit the thalamus, which, in turn, translates to either a
weak or strong stimulation of the motor cortex. Large pyramidal cells in the motor
cortex direct the motor command down towards the muscles via long axons running
along the corticospinal tract, through the brain stem and the spinal cord 219 . These
upper motor neurons do not directly stimulate the muscle, but first synapse in the
anterior horn of the spinal cord either on spinal interneurons or immediately on alpha-
1
26 | chapter 1
motoneurons 220-222 . The latter do, in fact, contract the muscle fibre, completing the
neuroanatomical journey from eye to hand.
Perceptuo-motorintegration
Putativepathwayssupportingperceptuo-motor
integration
The path described from eye to hand, through the visual and motor systems, is
complemented by many alternatives, detours and shortcuts. Some by-pass the
cerebral cortex altogether, others have an extra stop in the brainstem and travel
down via alternative ways (e.g. the reticulospinal tract 221). Below, specifically those
connections with the main visuomotor circuit are highlighted that are important
for the central question in this thesis: how does the brain incorporate perceptual
information into a motor plan? The pathways described thus far are commonly
suggested to support visuomotor transformations based principally on metric
information as processed in the parietal cortex. Indeed, metric parameterization is
a prominent factor structuring cortical organization and computation. However, it
is apparent that also perceptual information, such as object identity, can decisively
shape movement planning. For example, when we grasp a tomato and adjust our
grip to its ripeness as indexed by its colour. Several anatomical connections exist
between the visuomotor network as described above, and the ventral visual stream
in the temporal lobe, that could potentially allow the incorporation of perceptual
information into a motor plan.
First, the premotor areas, especially PMv, receive input from anterior prefrontal
areas 223 . Recently, in macaques, strong connections have been described between
premotor area F5 and areas 46v and 12r224 . These prefrontal areas are vital for topdown control over action, receiving and combining information from many other
brain areas 225-227. They can retrieve object knowledge from memory and organize
actions based on behavioural goals 228,229 , contextual information, and abstract
rules. Additionally, the prefrontal cortex keeps intentions and sensory information
online, temporally integrating both past and future events and actions 230,231 . Area
12r, for example, is not only connected with the ventral premotor area, but also
with the anterior part of the temporal lobe (area TE) where the ventral visual stream
terminates 232,233 , with the anterior part of the intraparietal sulcus (AIP) 204 , and
the secondary somatosensory area (SII) 234 . In humans, the interactions of central
parts of the inferior parietal cortex with lateral prefrontal cortex are even more
prominent, and extend further forward to anterior prefrontal regions 88 . As such,
introduction | 27
(anterior) prefrontal cortex brings together parietal areas important for hand-object
interactions, temporal regions were perceptual information processed, and the
frontal motor system. This may be a key route via which perceptual information may
structure motor planning.
Second, inferior parietal areas PF and PG are directly connected with the
superior bank of the STS 195,235,236 , containing the superior temporal polysensory
area (STP) 237. Moreover, recent studies have described direct reciprocal projections
between the anterior part of the intraparietal sulcus (AIP) and the inferior temporal
lobe, specifically with areas TEO and TE in the inferior bank of the superior temporal
sulcus (STS) and with the middle temporal gyrus 204 . These connections allow AIP
direct access to object identity and object knowledge processed in the ventral visual
stream 6,42 .
Third, AIP is a true network hub within the parietal cortex. Needless to say
it is heavily interconnected with its neighbours, areas PF, LIP and VIP235,238 , but it
additionally shows longer-range connections with the secondary somatosensory
area SII 203,204 , and crucially, with V6A in the parietal occipital sulcus 106 . Potentially,
the pathway from SII to AIP to TE could mediate tactile object recognition without
reliance on visual representations 43,162 . The direct connection with V6A is an
alternative to more indirect pathways (e.g. AIP-LIP-V6A) 106,238 and links together
two critical nodes in the dorsolateral and dorsomedial parietal frontal circuits.
To summarize, there are several anatomical routes via which perceptual
information processed in the ventral visual stream, like object identity, could
influence motor planning56 . First, the anterior temporal cortex projects on
prefrontal areas, which in turn can influence processing in the dorsolateral parietofrontal circuit via strong projections to the ventral premotor cortex and the anterior
intraparietal sulcus. Second, inferior temporal areas can also directly inform
the anterior intraparietal cortex and allow perceptual information to enter the
dorsolateral circuit, and reversely bring somatosensory and motor information to
the temporal lobe. Finally, information can be exchanged between the dorsolateral
and dorsomedial circuits at the level of PMv and PMd in the frontal lobe, but also in
the parietal lobe, between AIP and V6A.
Twoparieto-frontalcircuitsandmodelsofgraspcontrol
The parieto-frontal connections, comprising the two largely segregated dorsomedial
and dorsolateral circuits, occupy a pivotal role in the cortical network mediating
voluntary visuomotor behaviour. Although the relevance of these two circuits for
visuomotor control is undisputed, in recent years there have been several competing
proposals of their respective functional roles. For example, the functional segregation
of the dorsomedial and dorsolateral circuits has been suggested to be grounded in a
1
28 | chapter 1
dissociation between online control vs. action organization, respectively86 , or along
the same line, between online control vs. planning32 . These suggestions are evidently
not mutually exclusive; rather they focus on slightly different aspects of motor
preparation. Critically, they both argue that the contributions of the two circuits are
temporally distinctive, where processing in the dorsolateral stream precedes that of
the dorsomedial in an inherently sequentially dependent fashion.
In light of these accounts, it might be considered surprising that recently the
critical contribution of the dorsolateral stream, and specifically that of the anterior
part of the intraparietal sulcus, in the online control of movement execution has been
highlighted 239,240 . Moreover, in a study investigating parieto-frontal connectivity
using dynamic causal modelling 241 the dorsolateral circuit has been associated
with grasp guidance when a high degree of online control is required, while the
dorsomedial circuit is engaged when action control can rely on a pre-specified
movement plan 205 . At first sight, the former two proposals seem fundamentally
incompatible with the latter two. Chapters 6 and 7 will comprehensively examine the
temporal dynamics of two pivotal nodes in the parieto-frontal circuits during motor
preparation and execution, and suggest a putative clarification how these different
interpretations might be unified.
Several alternative models do not directly address the temporal dynamics of
the movement evolution, but instead consider the type of movement performed.
One significant proposal classifies actions based on their goal and context,
and distinguishes between ‘acting on’ and ‘acting with’ objects, relying on the
dorsomedial and dorsolateral circuits, respectively242 . Only in the latter movement
category conceptual knowledge of objects and object pragmatics are used to guide
the movement.
An influential account argues that grasping movements are organized in a modular
fashion, consisting of independently controlled transport and grip components 5,243 .
The transport component is hypothesised to be supported by the dorsomedial
circuit, sPOS especially, and guide the arm and hand towards the extrinsic location of
the object 28,95,113,122,244 . In contrast, the grip component is proposed to be controlled
by the dorsolateral circuit, with a privileged role for aIPS, shaping the hand and
fingers according to intrinsic object properties such as size and shape40,72,155,160 . This
model is in line with the electrophysiological characterizations of the parietal and
frontal nodes described above. To reiterate, superior parietal areas V6A and MIP
(human sPOS and mIPS), and frontal PMd are most sensitive to movements of the
arm, as required by reaching 107,117, whereas AIP (human aIPS) and PMv respond most
strongly to hand and finger movements, as additionally involved in grasping 151,175 .
Second, whereas many neurons in V6A are selective for target location 60 , those
in AIP instead represent object size, shape and orientation irrespective of its
location 98-100 , supporting a three-dimensional shape representation37 well suited to
introduction | 29
guide grip formation 40 . A transport-grip framework can powerfully explain both the
robustly enhanced activation in aIPS when grasping an object compared to merely
touching it with the knuckles 152,156,245 , and the increased involvement of the superior
parieto-occipital sulcus (sPOS) with increased transport distance113 .
Thetransport-gripdichotomyandindependentfinger
control
The two-channel model of transport-grip control can be regarded as intuitive in
several respects. Indeed, this is how many industrial robotic arms operate: moving
the arm to the target, opening the gripper, moving a bit closer, and closing the
gripper. Importantly, both in robots and humans it inherently requires two largely
independent controllers transforming metric visual information to motor code. This
independency is not uncontroversial and has been debated on several grounds. Some
have highlighted the covariance of many kinematic parameters originally proposed
to characterize either the transport or grip component individually246 . Recent
electrophysiological recordings have shown that neurons in V6A, PMd and PMv are
modulated by both transport and grip aspects of a grasping movement 109,247. From
a theoretical perspective, some authors have adopted a more integrative approach
and argue that both transport and grip characteristics of prehensile behaviour
can be explained by the same or an integrated computational mechanism 27,248-251 .
Simulation studies of grasping movements have revealed that many kinematic
findings originally considered to speak for a transport-grip distinction still emerge if
the digits, and not the transport and grip components, are controlled separately252 .
However, currently these theoretical models lack a neuroanatomical foundation and
it is not immediately clear how they could be grounded in the segregated dorsolateral
and dorsomedial circuits. This topic will be further addressed in chapter 7.
Our ability to use tools provides an interesting theoretical and experimental
challenge to these models. Can our dexterous ability to grasp, manipulate and use
tools be explained by the models of grasp control described above? For example,
our ability to use a small stick or rake to bring something out of reach closer. When
merely grasping the rake, it can be approached just like any other stick or object,
but when using the rake the critical aspect to control is no longer the fingers, but
the tip of the rake itself. The transport-grip and individual-digit-control models do
not address how this shift in control (distalization of the end-effector253) might be
computationally supported. In fact, it might be argued that once the rake is held
firmly in the hand, the grip itself no longer needs to be planned or controlled. Or
at least not more than when holding and moving a stick that cannot be used as
a rake (for example, by attaching the distal end of a sliding stick to a table with a
universal joint). Strictly speaking, if the dorsolateral stream (and aIPS in particular)
1
30 | chapter 1
is considered to form a grasping circuit, it would not be expected to be differentially
involved when using a rake or moving a stick. But instead, when these two tasks are
directly compared, aIPS and PMv are more active when functionally using a rake,
irrespective of the grip formation or arm movements 254 . Interestingly, the increased
activation of the dorsolateral stream is accompanied by an enhanced involvement
of the inferior temporal cortex (IT, or macaque area TEO). This trio of regions (aIPS,
PMv and IT) is consistently reported in neuroimaging studies of tool use 17,164-166 . The
connectivity and activation strength within this network tend to be lateralized with
a bias towards the left hemisphere 255,256 .
Neuropsychologicaleffectsofparietallesions
Damage to parietal cortex often affects a patient’s reaching and grasping behaviour.
Although lesion sites and accompanying behavioural impairments can vary greatly,
two specific visuomotor disorders associated with damage in (left) parietal cortex
are well described. First, optic ataxia, as briefly discussed earlier in the introduction,
is characterized by lesions of superior parieto-occipital cortex in the dorsomedial
circuit 11 . Second, ideomotor apraxia is distinguished by left lateralized inferior
parieto-frontal lesions, often impinging on the anterior intraparietal sulcus 257,258 .
Interestingly, under optimal conditions both optic ataxia and ideomotor apraxia
patients often show normal reaching and grasping behaviour: towards objects
presented visually, in front of their body, and in the centre of their focus. However,
when the circumstances are suboptimal they reveal diverging deficits. Optic
ataxia causes impaired reaching performance, especially in the periphery of the
visual field contralateral to the affected hemisphere 12,13,259 . Ideomotor apraxia, in
contrast, is associated with grasping and hand shaping deficits, in particular when
asked to mimic grasping an object that is not visually present or when to imitate
meaningless gestures 260,261 . These specific impairments might roughly resemble
a distinction between the control of the transport and grip component, but the
specific circumstances under which the deficits are most pronounced suggest a
more intricate picture.
In optic ataxia, the reaching deficit towards targets in the periphery is sometimes
accompanied by an additional grasping impairment 13,262 . It has been suggested that
the affected grasping behaviour is a secondary consequence of perturbed reaching 116 .
However, while patient AT with bilateral medial parieto-occipital lesions is impaired
when grasping a simple cylinder that she has not encountered before, she is
surprisingly capable to accurately reach for and grasp a familiar tube of lipstick of the
same size and shape presented at the same location as the cylinder14 . This suggests
that her impairments with unfamiliar target objects might not be a specific transport
or grip deficit, but more a general impairment of spatial visuomotor transformation
int ro duc tion | 31
(potentially the computational mechanisms proposed by the individual-digit-control
models 251), which can be overcome by using previous experience based on object
knowledge. This finding is substantiated by behavioural observations of optic ataxia
patient IG, who showed impaired hand shaping for novel objects, but her behaviour
improved with repeated presentations 115 . In summary, optic ataxia patients seem
to be able to overcome their perturbed spatial parameterization if they can rely on
object knowledge and prior experience.
Observations of the behaviour of ideomotor apraxia patients reveal a
complementary pattern and suggest a similar dichotomy. As mentioned, when
ideomotor apraxia patients are visually presented the target object, they are able to
appropriately adjust their transport kinematics and shape their hand according to the
object’s metrics 261 . However, similarly to visual agnosia patients, their prehension is
often functionally inappropriate, for example skilfully grasping a spoon, but at the
wrong end9,260,263 . Importantly, ideomotor apraxia patients do not perform worse
than healthy controls when asked to grasp an object which was presented only
shortly ago 264 , suggesting that the functionally inappropriate grasp of tools and
objects is not a problem of short term visuomotor memory, but more specifically of
stored object knowledge which is built up over multiple interactions with the object.
Paradoxically, these patients are less impaired when grasping non-familiar compared
to familiar objects 265 . This might imply that when grasping an unfamiliar object they
can rely on spared transformations of current metric sensory information. However,
when an object is recognized they intend to rely on object knowledge, but fail to do
so owing to the nature of their lesion, hampering accurate grasping behaviour.
Evidently, it must be concluded that metric parameterization of visual
information, as mechanistically described by both the transport-grip and the
individual digit control models, is not sufficient to fully understand human grasping
behaviour. Below, two recent developments in the understanding of the workings
of the motor system are highlighted, setting up the stage to support the functional
interpretation of perceptuo-motor interactions in grasp control as described in this
thesis.
1
32 | chapter 1
Twodevelopmentsintheunderstanding
ofthemotorsystem
Corticospinalorganization
In a series of landmark studies Penfield electrically stimulated the human motor
cortex of epilepsy patients and mapped their responses 266 . The somatotopic
organization of the motor cortex was known before, but Penfield popularized the
notion of the cortical motor homunculus, where all our motile body parts have their
own dedicated field in the precentral gyrus, with disproportionately large sections
devoted to our hands and face. Accordingly, the motor cortex is classically assumed
to have a virtually direct control over our muscles, guiding our effectors in parallel
and independently. Provocatively put, according to popular notion stimulation of one
single site in the motor cortex is assumed lead to a single muscle to contract, causing
for example one single finger to flex. This view is now known to be a misconception.
First of all, direct muscle control and independent finger representations
are fundamentally incompatible with human anatomy. The ring finger, for
instance, cannot be moved independently. When you hold your hand up, with all
fingers extended, try to bend your fingers while keeping only the ring finger fully
extended. The muscles flexing and extending the ring finger are shared by all
others. Interestingly, only the index finger and the little finger do have independent
flexors, allowing us to point with these fingers alone and no others. Moreover, the
extrinsic finger flexors are located in the forearm and hence do not only cause the
fingers to flex, but also the wrist. Several wrist extensors have to be coordinated
to continuously counteract the forces of finger flexion and keep the wrist stable.
Clearly, skeletomuscular control of finger and hand movements is an act of exquisite
balance with simultaneous convergence and divergence from muscles to fingers. This
interlocking pattern in skeletomuscular mechanism is also observed in the neuronal
organization of the corticospinal tract. One higher pyramidal motor neuron in the
precentral gyrus projects on multiple alpha-motoneurons in the spinal cord 267, and
vice versa, one alpha-motoneuron is innervated by multiple higher motor neurons 268 ,
creating a neural network.
Second, neighbouring – supposedly effector specific – primary motor fields
show considerable overlap. Moreover, many effectors are represented multiple
times, both in primary motor and premotor areas. Although a coarse somatotopy
can still be defined, more subtle aspects, for example individual fingers, are most
likely not individually presented, let alone delineated269 . Instead, the control and
representation of muscles and fingers in the primary motor cortex seems to be the
organized in adaptive synergies encompassing multiple fingers 270,271 .
introduction | 33
Third, most of the claims supporting a highly somatotopic organization are
based on very brief electrical stimulation of the cortex, in much shorter trains than
the motor cortex is naturally active during ecologically valid movements. When
the stimulation trains are extended, for example to 500 ms, a different pattern
emerges 210 . Instead of a twitch or single contraction, longer stimulation evokes more
ethologically relevant movements, for example a defensive raising of the arm, or
bringing the hand to the face and opening the mouth simultaneously. Furthermore,
the effector preference of a neuron revealed by short burst stimulation does not
necessarily agree with that neuron’s tuning observed during naturalistic motor
preparation and execution 247. Accordingly, some authors have proposed that instead
of a true somatotopic organization, multiple representations of ethologically
relevant movements and actions are folded onto the two dimensional cortical
sheet 82,272 . This hypothesis has been described earlier in the introduction in relation
to the organization of the posterior parietal cortex.
Fourth, amongst all these considerations, the fact that we show great dexterity
in individuated finger manipulations should not be ignored. This is a remarkable
skill and is, actually, a relatively recent development in our phylogenetic tree.
Many prosimians are only able to grasp an object in a sweeping motion with their
whole hand273 , similarly to how we would clench our fist around a baseball bat. In
contrast, humans can grasp a small object using a precision grip with the index and
thumb 19 . The emergence of this fine motor control in evolution is accompanied
by some striking adaptations in the corticospinal motor system 273,274 . The cortical
higher motor neurons of prosimians and monkeys lacking precision grip skills
terminate only in the dorsal part of the spinal cord on spinal motor interneurons.
Those that do exhibit precision grip additionally have direct cortical projections on
alpha-motoneurons (which directly innervate muscle fibres) in the anterior horn of
the spinal cord. Thus, this pathway crosses just one synapse between cortex and
muscle. In summary, following the phylogenetic tree from prosimians to monkeys to
primates, the number of cortical terminations in the spinal cord increases, especially
in the anterior horn where motoneurons reside 213 .
Fifth, these spinal alterations follow tread with parallel reorganization of the
cortex 21 . The primary motor cortex of monkeys and primates can be divided in two
regions, distinguished by the absence or presence of direct cortico-motoneuronal
projections 211 . An evolutionary old region, shared by most mammals, is located on
the surface of the precentral gyrus and lacks direct projections. More posteriorly, in
the bank of the central sulcus, another region can be differentiated by its abundant
direct connections with spinal motoneurons.
Sixth, although the primary motor cortex is the main cortical site projecting
down the corticospinal tract, it is by far not the only one 220,222 . In fact, about half of
the upper motor neurons axons originate elsewhere in the motor and sensorimotor
1
34 | chapter 1
system 270,275 . All premotor areas, including PMv and PMd, and even some
somatosensory and parietal areas, have direct projections to the spinal cord 276 .
To summarize, evolutionary novel adaptations at the core of the motor system
have allowed the human sensorimotor circuits to occupy a very prominent role in the
neocortex 21 . They highlight the almost direct, but integrated influence that nearly
all parts of the motor system exhibit on the muscles, allowing fast, adaptive and fine
control of movements, supporting human behaviour as investigated in this thesis.
Predictivecoding
The neural circuitry controlling our actions is strongly shaped at all organizational levels
by many feedback and feedforward connections. For instance, a motoneuron in the
spinal cord firing to contract a muscle, also projects on a Renshaw-cell which inhibits
the same motoneuron, stabilizing its activity277. But also at higher levels feedback
mechanisms shape the control of our behaviour. For example, when intending to
pick up a glass brim-full of water, one will probably attentively look at the hand when
closing in on the glass. If an error in the reach is noticed, the movements of the hand
and fingers are adjusted, and so forth, until the fingers touch the glass. At that point
the visual feedback is accompanied by tactile information guiding fine adjustments
until the glass is firmly held and can be lifted. Computationally this behaviour could
be implemented in a recurrent iterative feedback loop. First, the movement of the
hand and fingers to a certain target configuration is prepared and initiated. During
execution, the retinal image of the scene is transformed to a representation of the
current position of the hand and fingers. This representation is compared to the
desired configuration and if found deviating a new adjustment plan is prepared and
executed. Implemented as such, this would result in a slow and stuttered reiterative
loop. It would be more effective, both computationally and in time, if the comparison
between desired state and actual state could already be made at the sensory level,
where the information enters the system 30 . If a discrepancy is found at that early
level, the motor adjustments could be immediate planned instead of having to wait
until the sensory information has reached the representational level of the target
configuration. In essence, this target configuration has then been translated into a
target sensory state. This translation requires the initial motor planning to not only
send a command to the executive motor system but also to send an efference copy
to an inverse model converting it to the sensory level 278-281 . The efference copy of the
motor command, the inverse model, and the following corollary discharge (as it is
know in neurobiology282), are often regarded together as a forward model 283 .
Crucially, the computational advantages of forward modelling are not limited
to the application in online movement control. In fact, recently it has been proposed
introduction | 35
that any neural system with local forward and backward connections could predict
its own input with minimal error according to the ‘free-energy’ principle 284 . This
inherent design could form the foundation of many, if not all, cortical mechanisms 285 .
A stable state of the environment, or of a neural system for that matter, has no
information value and therefore requires a low computational load. Accordingly, to
minimize and optimize the computational load of a neural process, be it perception
or action, it is advantageous to predictively code the upcoming information and value
highly any mismatches of the predicted with the actual state, so called prediction
errors. In that sense, forward modelling (and backward projection of predictions)
does not only play a vital role in online control, but might also be crucial for the initial
planning of an action itself on the basis of current and prior sensory information.
This notion further diminishes the already vague line separating the computational
mechanisms of action preparation from those of online control. In fact, both these
classically delineated movement phases might rely on very similar or even equal
cortical constructs.
This point is illustrated by a familiar example in this introduction: grasping a
tomato. If you recognize the object as a tomato you might have certain expectations
on how the interactions will develop. For example, based on its colour you might
expect the tomato to be soft or hard, and adjust your grip accordingly. Or more
generally, you might have an expectation on the size and shape of the tomato and
where you need to place your fingers. This information could be processed in parallel
to parameterization of the metric visuospatial information present in the current
retinal image. Speculatively, object knowledge might provide a template of a grasp
plan appropriate for the identified object, serving as a prior for subsequent spatial
visuomotor transformations. This is a potential mechanism by which perceptual
information can inform efficient heuristics in adaptive visuomotor control.
Conclusion
At the start of this introduction we met HAL, the fictional supercomputer who could
do everything except move. In other words, he couldn’t do anything at all. HAL is an
acronym for Heuristically programmed ALgorithmic computer, forecasting at a time
when simple electronic calculators were the size of large typewriters, the modern
appreciation of heuristics in any efficient artificial or natural intelligent system.
But only recently, the truly central role that our ability to act plays in the human
brain, both anatomically and functionally, has been appreciated. Furthermore, the
recognition of the potential of heuristics and predictive coding for the computations
implemented in this pivotal motor system is currently growing.
1
36 | chapter 1
As described in this introduction, perception and action are initially segregated
in the human brain, both anatomically and computationally. Ecologically relevant
adaptive behaviour, however, relies on the successful integration of these
modalities. Based on novel insights in neuroanatomy several potential pathways
that could mediate this integration have been highlighted. Evolutionary adaptations
at the level of fundamental corticospinal networks have allowed the development
of fine motor skills, vital for the precise manipulation of tools. Furthermore, the
coupling of perception and action is not a unidirectional stream, but rests on an
intricate visuomotor circuitry. At many levels within this system the fundamental
mechanisms of predictive coding are deeply implemented, certainly in the online
control of movements, but potentially also in visuomotor and perceptuo-motor
transformations supporting action preparation. These principles might provide a
mechanism by which object knowledge could inform the visuomotor system with
previously acquired priors, serving as a heuristic template for ongoing construction
of a motor plan.
The potential power of perceptual information for motor preparation is evident.
But how is this information incorporated? Which neural substrates support the
interaction? How are the computational mechanisms organized, both temporally
during the evolution of the motor plan, and spatially over the distributed cortical
parieto-frontal circuits? And perhaps more importantly, how do these segregated
sensorimotor channels interact within parietal cortex and communicate with
perceptual processing in the temporal lobe, together supporting a common motor
goal?
introduc tion | 37
1
2
Biasing perceptuomotor integration by
object slant and vision
40 | chapter 2
bia sing p erce ptuo - motor inte gration | 41
Biasingperceptuo-motorintegrationby
objectslantandvision
An important feature of our adaptive grasping behaviour is the ability to incorporate
perceptual information into a motor plan. For example, when plucking a ripe red
tomato we accurately anticipate the appropriate grip force; and when grasping
a glass of water we adjust our grip to its perceived weight and slipperiness. In an
experimental setting working with slippery glasses full of water and overripe
tomatoes is not straightforward, even less so in combination with modern
neuroimaging equipment. One way to experimentally manipulate visuomotor
reliance on perceptual information, is to ask subjects to grasp several objects
differing in familiarity, for example as in behavioural studies of optic ataxia patient
AT14 . Although this could be an intuitive approach, changing the identity of the
grasping target inevitably entails that other object properties are altered as well,
obfuscating the cortical computations and experimental inference. Moreover, such a
design only emphasizes how perceptual and metric cues are differentially employed
for motor guidance, but not how they are integrated in grasp planning.
The experiments described in this thesis have aimed to keep all object
characteristics and other experimental factors constant, except the requirement to
integrate perceptual cues in motor preparation. Accordingly, within one experiment
always the same object was presented in the same place: its intrinsic properties,
such as its size and shape, and its extrinsic location remained stable. Crucially, the
object could be rotated to different orientations in depth around its pitch axis (figure
2.1). This manipulation places emphasis on accurate processing of depth information
to guide the hand and fingers. The reliance on perceptual information in the
prehension task was manipulated by taking advantage of the distributed processing
of stereoscopic disparity and monocular pictorial cues of depth such as texture and
shading37,38 . Stereoscopic disparity provides metric information, while texture and
shading are perceptually processed to infer depth. Importantly, these different
sources of depth information both directly concern the same spatial properties of
the object to be grasped, making it possible to directly compare their contributions.
2
42 | chapter 2
Binocularandmonoculardepthcues
In the context of grasp control, the critical processing of binocular disparity
informing object depth and orientation takes place in the parieto-occipital cortex
of the dorsal visual stream, with a particular focus on the posterior and anterior
parts of the intraparietal sulcus (cIPS and aIPS, respectively) 37,158,286,287. In contrast,
monocular pictorial depth cues supporting object representation are processed in
occipito-temporal cortex along the ventral visual stream38,39,288-290 . The construction
of a three-dimensional representation from two dimensional pictorial cues relies
heavily on computations implemented in the lateral occipital complex (LOC) 291,292 .
When subjects can view the object to be grasped with two eyes, i.e. binocularly,
both stereoscopic and pictorial cues of depth are available in the visual input. For
the spatial guidance of movements, stereoscopic cues processed in the dorsal
stream are generally preferred above pictorial cues 293-299 . However, if subjects are
only allowed monocularly vision of the object, i.e. with one eye, they are forced to
rely on pictorial cues of depth to estimate the object’s depth and orientation, after
all stereoscopic disparity cues are not accessible. Under such monocular viewing
conditions, the ventral contributions to depth estimation are critical in guiding
prehension, as revealed by the behaviour of visual agnosia patients with ventral
lesions300-303 . Patient DF, with several lesions amongst which a prominent bilateral
inferior occipito-temporal lesion encompassing the LOtv region 7, shows preserved
visuomotor behaviour when allowed binocular vision, but crucially she is unable
to adjust her grip according to the slant of a target object when only provided
monocular vision301 .
Unfortunately, a direct comparison of binocular and monocular viewing
conditions during grasp would not unequivocally reveal how pictorial information
is incorporated into a motor plan. First, the information value of these viewing
conditions is not matched. Of course, concerning stereoscopic cues this is inherent
and trivial, as stereoscopic cues are absent under monocular viewing condition. But
one might argue that also pictorial information is richer when based on the input of
two eyes instead of one. Second, the main focus of interest here is not on stereoscopic
depth estimation per se, but on how visuomotor transformations based on these
depth estimations guide grasp planning. Third, the reliability of stereoscopic and
pictorial cues of depth strongly and differentially depends on the slant of the target
object. In other words, for different orientations of the object in depth, the relative
reliability and information value of binocular and monocular viewing conditions
is different as well. Instead of regarding this effect of slant a nuisance, it can be
exploited as an opportunity. The effect of object slant on the information value of
depth cues is complex, but vital to the experiments described in chapters 5, 6 and 7,
and will therefore be discussed in detail.
biasing perceptuo-motor integration | 43
Theinteractionofobjectslantand
vision
Informationvalueofdepthcuesasafunctionofobject
slant
In this thesis the configuration of a cuboid object is often referred to as ‘vertically’ or
‘horizontally’ slanted. These terms could be confusing to the naive reader; therefore
a clear definition of these configurations is essential. When the object is vertically
slanted, it is suspended upright in front of the subject with its largest surface facing
the subject directly, in parallel with the subject’s frontal-coronal plane. The subject
grasps the object with the index finger on the top of the object and the thumb on
the bottom. When the slant of the object increases, the object is rotated around its
pitch axis (along the midsagittal plane) with the top of the object moving away from
the subject (and the bottom towards the subject). When the object is horizontally
slanted, it is rotated to a flat configuration with its largest surface in parallel with the
tabletop, along the subject’s transversal plane. The subject grasps the object with
the index finger on the far end of the object and the thumb on the close end (figure
2.1).
With increasing object slant, from vertical to horizontal, pictorial cues such
as texture and shading, become more relevant and reliable. Simultaneously, the
information value of disparity cues actually decreases. To make these two effects of
object slant more intuitive they will be discussed in detail one by one, first texture
cues and later stereoscopic disparity. Imagine a vertically oriented rectangular object
(a cuboid), positioned at eye height, its front patterned with circles. That object is
rotated away from you; the retinal projection of the pattern on the front side of the
object will be compressed vertically. Soon the circles will turn into ellipses and when
the object is nearly horizontal, they will be almost completely flattened. If the object
is rotated just slightly near the vertical orientation, for instance from 0 to 30 degrees
relative to the frontoparallel plane, the pictorial image on the retina will not change
dramatically. A circle on the frontal surface of the object will still strongly resemble
a circle. However, if the same object is rotated by the same angle near its horizontal
orientation, from 60 to 90 degrees, the pictorial image will change drastically. The
deformation of the circles to flattened ellipses and finally to a line is dependent on
increasing slant, in fact the relationship between pictorial deformation and object
slant approximates a sine function.
Stereoscopic disparity behaves differently over the same rotations. In the near
vertical rotation, from 0 to 30, the top and bottom part of the object will have moved
2
44 | chapter 2
considerably in depth (if the object is 6 cm high they will have been displaced by 1.5
cm each). This change in depth is directly related to the change in disparity between
the two retinal images. If the object is rotated from 60 to 90 degrees, the same
endpoints of the object will have moved only slightly in depth (4 mm for the same
6 cm object), and accordingly the retinal disparity will have changed only ever so
subtly. To summarize, if the object is near vertically slanted, stereoscopic disparity
object configurations
interaction of object slant x vision
visual evidence
z - position (cm)
40
35
30
vision
binocular
monocular
source
metric
perceptual
object inference
slant
-25
y - post-20
ion (cm
)
-15
0
x-
5
)
(cm
ion
st
po
precision
stored knowledge
line of sight
action prior
0
15
30
45
60
75
90
degrees away from the frontal plane
experience
less
more
precision
Figure2.1 Experimental manipulations of object slant and vision
The target object could be oriented in seven different slants in depth, from vertical
(blue) to horizontal (orange) relative to the subject’s frontoparallel plane (left
panels). The viewing conditions of the object, binocular (red) or monocular (green),
bias the processing of depth cues to either metric or perceptual sources in the dorsal
and ventral visual streams, respectively (top right panel). The information value
of binocular and monocular depth cues is differentially dependent on the slant of
the presented object (middle right panel). More specifically, when presented with
a (near) vertically oriented object, the precision of the inference of the object’s
configuration is highest when allowed binocular vision. Reversely, for (near)
horizontally slanted objects monocular cues of depth have a higher precision. With
accumulating task experience knowledge of the available object configurations
grows and the precision of the associated stored action priors (bottom left panel).
Importantly, these action priors are selected and retrieved on the basis of the
preceding inference of the object’s configuration (middle right panel).
biasing perceptuo-motor integration | 45
cues are most sensitive, while pictorial cues are less discriminative; for near horizontal
objects the relationship is reversed: disparity is less sensitive, while pictorial cues are
more informative (figure 2.1).
Next to stereoscopic disparity, as described above, informative binocular depth
cues for grasping are also provided by vergence304-306 . However, to be consistent
and avoid confusion, stereoscopic disparity will be used to refer to binocular cues
of depth in general. Importantly, all binocular cues share very similar properties
with respect to object slant. Their information value for orientation estimation
has a direct monotonously increasing relationship with object slant. Moreover,
there is an almost direct mapping of disparity and vergence to extrinsic position
in space. Accordingly, stereo vision could in principle give an absolute measure of
the target positions of the thumb and index finger on the object. For inferences of
object orientation (especially in the case of complex objects) absolute estimates at
several points need to be integrated to construct a veridical surface representation,
and further integration is required to represent an object as a whole. Nonetheless,
the sensitivity of both first- and second-order stereoscopic information to describe
spatial object features still monotonously changes with object slant. Therefore, in the
context of grasp guidance, the processing of object features based on stereoscopic
and vergence cues will be referred to as metric object information.
Within limits, monocular cues, such as texture and linear perspective, might
develop continuously as a function of object slant, as long as the object’s target
surface is tilted within a restricted visible range307. But in contrast to stereo vision,
this is bounded by sharp transitions and not a general property of monocular cues.
This non-continuity of the information value is even stronger for other monocular
depth cues, such as shading, reflexion and occlusion. Because the retina only
provides a two dimensional image the inference of monocular depth is relative
in all cases, in sharp contrast to binocular information. Monocular vision is not
directly informative of absolute extrinsic position; it only provides a pictorial image
that needs to be processed perceptually to inform three-dimensional object and
space representation. Accordingly, pictorial monocular depth cues are considered
representative for perceptual object information.
Although the processing of stereoscopic and pictorial cues relies on
fundamentally different computations and neuroanatomical foundations, they do, in
the end, inform the same spatial aspects of visuomotor control. In other words, their
main difference lies in their processing pathways, not in the domain and modality
of the information that they provide. This makes the interaction of multiple sources
of depth information a feasible and pragmatic platform to study the integration of
metric and perceptual information in general.
It could be argued that as pictorial cues are also available during binocular
viewing conditions, they might contribute to the grasp planning when binocularly
2
46 | chapter 2
viewing near horizontal objects 293 . However, as mentioned, disparity cues are
preferred to inform grasp control, and object slant estimation might suffer from
ambiguity between different cues 294 . Hence, the reliability of slant identification
following binocular vision is higher for vertically compared to horizontally slanted
objects. Accordingly, the interaction of object slant and viewing conditions captures
two effects: 1) the increase in reliance on incorporation of perceptual information,
when grasping near horizontal compared to near vertical objects under monocular
viewing conditions, and 2) the reliance on metric cues, which is maximized with
binocular vision of vertically slanted objects.
In summary, by providing either binocular or monocular vision of a slanted
object the processing of depth information serving grasp control is biased to
either the dorsal or ventral visual stream, respectively. By the nature and cortical
site of the processing of depth cues they can be classified as metric (binocular) or
perceptual (monocular). As the slant of a grasped object increases from vertical to
horizontal, perceptual cues of depth become more relevant to estimate the object’s
slant, whereas the reliability of metric cues decreases. Accordingly, the interaction
of object vision and slant allows us to manipulate the integration of metric and
perceptual information into a motor plan.
Storedobjectconfigurationknowledge
As addressed in the general introduction, investigations of the computational
mechanisms of motor control have classically been focussed on the metric
parameterization of sensory information, and less on the influence of object
knowledge. Accordingly, most of the attention to spatial depth processing has
been devoted to binocular cues, which allow absolute visuomotor transformations.
Recently, the interest in monocular cues of depth and orientation has grown.
Some studies emphasize that the relevance of these cues should not be
underestimated 293,307,308 , while others doubt the preference for binocular cues
in action guidance altogether309 . Behavioural experiments comparing binocular
and monocular viewing conditions have led to conflicting interpretations. This
inconsistency is partly a result of differences in the experimental setup, for instance,
the effects of object slant on cue reliability (as described above) is not always taken
into account. But potentially more important, a relative disadvantage of monocular
vision could be compensated by using object knowledge as acquired during the
experiment. Especially when a relatively small set of objects or object configurations
is used, experience with object interactions can be easily build up over repeated
presentations310,311 . The sample sizes used in chapters 5, 6 and 7 are relatively small
(either 4 or 7 different object configurations) and the object to be grasped was always
a very simple rectangular prism. This will most likely have decreased the relative
advantage of stereo vision to accurately guide the fingers towards the target. But
biasing perceptuo -motor integration | 47
crucially, the critical effect of interest is not a direct comparison of binocular and
monocular viewing conditions, but instead the interaction of object vision and slant,
which is unaffected by these general shifts in cue preference.
In fact, focussing on this within object interaction allows us to investigate the
incorporation of object knowledge independently of general task competence.
With repeated presentation of an object to be grasped, object knowledge is
acquired, but at the same time, subjects will also build up general task experience
and show improved performance accordingly (figure 2.1). To dissociate between
these aspecific effects and object knowledge again the interaction of object vision
and slant can be exploited. The configuration of horizontally slanted objects is
most reliably estimated on the basis of monocular cues, while identifying vertically
slanted objects benefits most from binocular cues: the interaction of object vision
and slant manipulates the ease and reliability by which the object’s configuration
can be identified.
Accordingly, the influence of familiarity with the object’s configurations on
motor preparation can be investigated by tracking the cortical correlates of this
within-object interaction. The advantages of this approach are threefold: 1) the
development of the interaction over time is independent of any general effects
of task learning and experience, 2) there is no need to use different objects to
manipulate familiarity (which would also influence other object features), and 3) the
acquired object knowledge concerns the configuration of the target, and therefore
can directly impact the visuomotor planning of the grasping movement. Chapter 6
will describe a study examining how object configuration knowledge acquired over
trials is incorporated into the motor plan.
Graspingobjectsslantedindepth
The most commonly accepted model of visually guided grasp control emphasises the
modularity of reach-to-grasp movements 5 . The extrinsic location of an object to be
grasped informs the initial arm transport, while a complementary grip component
shapes the hand according to intrinsic object metrics such as its size 243 . These two
components are suggested to map onto two separate parieto-frontal circuits: a
dorsomedial channel guiding the transport component, and a dorsolateral channel
shaping the grip component 2,72,113 . Interestingly, an important object feature, its
orientation, does not fit well into any of these two components. If an object is rotated
around a central axis, neither its extrinsic, nor its intrinsic properties change, as both
the object’s position and its shape are independent of its orientation. Moreover, the
orientation of an object in space creates strong dependencies between the control
of arm and finger movements during prehension 312 . The difficulties of classifying
2
48 | chapter 2
object orientation as either extrinsic or intrinsic have been recognized early in the
development of the hypothesis of modular control of grasping 1,243 . Some have
proposed that guidance of grip orientation relies on the fine coordination and
integration of the transport and grip components, but leave open how this could
be implemented computationally246,312 or neuroanatomically249,313 . Others suggest
that object orientation constitutes a third control component by means of wrist
flexion314 . However, wrist flexion or extension only contributes to grip orientation
in a subset of all possible orientation axes, for example the wrist is flexed when
grasping objects slanted in the midsagittal plane (around its pitch axis), but not for
those oriented in the frontoparallel plane (around its roll axis). Moreover, to date no
account has addressed how object orientation would be cortically controlled by the
two parieto-frontal channels in the context of the transport-grip dichotomy.
In opposition to the notion of a modular structuring of prehension, a more
integrative account of grasp guidance has been put forward 251 , as discussed in the
general introduction. This model assumes that in grasping movements not the
transport and grip components, but the individual digits are controlled. In that
framework grasping can be considered a variation of pointing with two digits,
guiding the digits to their respective target end-points independently. In that light
the three crucial features of an object, its location, size and orientation, all affect the
same factor: the position of the digit end-points in space, and hence do not need to
be controlled individually.
Crucially, in both models, the two-channel transport-grip dichotomy and
individual-digit-control, grasping an object horizontally slanted in depth will be
computationally more demanding than grasping a vertical object in the frontoparallel
plane. The former often requires computationally challenging visuospatial
parameterization to specify the location of visually-inaccessible finger end-points
and potential hindrances from the object itself in the fingers’ trajectories. In other
words, increasing object slant, from vertical to horizontal, will increase the transport
complexity, be it the transport of the preshaped hand, or of the fingers individually.
This notion will form the basis of our investigations of the cortical correlate of
planning a grasping movement towards an object slanted in depth in chapters 5 and
7.
biasing perceptuo-motor integration | 49
2
3
Methodological
considerations in multimodal neuroimaging of
visually-guided grasping
52 | chapter 3
multi-modal neuroimaging of grasping | 53
Methodologicalconsiderationsinmultimodalneuroimagingofvisually-guided
grasping
To understand the mechanisms of the human brain, we cannot go about without
trying to qualify and quantify its neural activity. In the general introduction many
studies have been described which are based on single cell recordings in monkeys,
measuring the spiking activity of individual neurons. Although these spiking action
potentials might be regarded as possibly the most direct index of neuronal activity,
measuring them requires very invasive procedures. Fortunately, the past decades
have seen great progress in the development of imaging techniques, which have
propelled the non-invasive quantification of neural activity, especially in humans.
Also the experiments described in this thesis rely heavily on the application of several
non-invasive neuroimaging techniques. Inferring from their measurements requires
a coarse understanding of both basic neurophysiological mechanisms and how the
applied neuroimaging techniques relate to neural activity.
Electrophysiologicalrecordings
Theneuron
Action potentials fired by a neuron have a fixed amplitude, and accordingly, the
signal arriving at the end of an axon at the presynaptic terminal is virtually binary.
However, the resulting post-synaptic potential (PSP) is not a one-to-one copy; in
fact, it is highly modulated, forming a graded, almost analogue signal. This graded
PSP, in turn, determines if the innervated neuron will fire action potentials of its own.
The sign of the PSP, either depolarizing (excitatory) or hyperpolarizing (inhibitory),
3
54 | chapter 3
is dependent on the type of neurotransmitter and ion channels innervated, and
its amplitude is a result of the complex chemical signalling efficacy. Importantly, a
single neuron receives input from many axons, in the order of thousands. Because
of the spatial spread of a PSP and its sluggish return to the resting potential, PSPs
are spatially and temporally additive. In fact, hardly ever will a single depolarisation
of one synapse on a dendrite evoke an action potential. To build up the PSP towards
the threshold the depolarisations need to follow either in rapid succession or
come from multiple synapses; usually both summation mechanisms are employed
simultaneously. This mechanism allows a neuron to manipulate the strength of its
outgoing signal by the number of action potentials it initiates within a limited time,
even though every individual action potential has a fixed strength. As a result, action
potentials often appear in bursts, so called spike-trains.
Electroencephalography
The activity of a single neuron can be recorded by bringing a very fine electrode
tip close to the cell and its axon. The changes in extracellular voltage recorded by
the electrode are usually a reflection of the neuron’s action potentials. When a
larger microelectrode is placed in the neural tissue, it records a summation of the
membrane potentials of many neurons in the vicinity, called the local field potential.
In contrast to single-unit recordings, the local field potential is not determined
by action potentials, but mainly by PSPs, or their extracellular return currents
to be precise. Accordingly, single-unit and local field potential recordings do not
necessarily measure the same component of neural activity. The former might be
more related to the neuronal output of a region and the latter to its input. If the PSPs
are mainly inhibitory, these measurements could actually be anti-correlated.
If many PSPs arrive on a large enough assembly of neurons with dendrites
sufficiently aligned, as is often the case for large pyramidal cells, the associated
extracellular current might be sufficiently strong to be measureable through the
skull at the scalp 315 . This is the principle of electroencephalography (EEG, figure 3.2),
a technique applied in chapters 6 and 7. Even though the electrodes are placed on
the scalp it can give a relatively direct measure of neural activity. The recordings
generally have a very high temporal resolution, picking up potential changes on
a millisecond scale. The electric field is, unfortunately, spatially distorted and
smoothed (even temporally filtered to a degree) by the conductance of the neural
tissue, skull and skin, resulting in a very low spatial sensitivity of EEG. To a moderate
extent this can be dealt with by high-density recordings and advanced analysis
techniques316 . However, EEG has an inherent drawback: as electric currents get
weaker the further they spread: it is only sensitive to electric currents originating
in the superficial parts of the cerebral cortex. Moreover, areas where the dendrites
multi-modal neuroimaging of grasping | 55
are not aligned, or in an unfavourable orientation with respect to the electrodes on
the scalp, will not contribute to the EEG signal. Some of these issues are partially
addressed by magnetic encephalography (MEG). This advanced technology
measures the tiny summed magnetic fields created by many changing dendrite
potentials in synchrony317,318 . Crucially, these magnetic fields pass through the skull
more easily, resulting in a better spatial inference compared to EEG. However, for
MEG the orientation of the pyramidal dendrites where the measured PSPs originate
is even more critical and the strength of a magnetic field decays quickly with distance,
limiting the spatial extent from where magnetic fields can be detected.
A stimulus, for example the visual presentation of an object, can evoke a
surge of synchronous neural activity. This will be recorded by EEG as an evoked
potential (or field by MEG). Commonly, evoked potentials are so small that they are
unrecognizably hidden within the background signal. They only become apparent
when many trials are averaged with respect to the occurrence of the repeated event.
The background signal is regarded a nuisance in the analysis of evoked potentials.
However, this signal contains a reflection of ongoing neuronal activity. Neurons tend
to operate in synchronous ensembles, firing together in a rhythmic pattern. As a
consequence of the temporal and spatial summation of PSPs, as described above,
this supports efficient information transfer. These rhythmic bursts induce oscillatory
modulations of the PSPs: alternating time windows when it is easier or harder to
elicit a postsynaptic action potential. This oscillatory activity is powerful enough to
dominate the electrical landscape of a neuronal ensemble and it reflects strongly on
the EEG signal.
Oscillatory activity can be characterized on three dimensions: frequency,
amplitude, and phase. Especially the first two are of importance for the current
treatise. The frequency of the observed neural oscillations can range from 1 to 250
Hz or even higher. Importantly, they can be coarsely classified in several constrained
frequency bands, denoted with Greek letters, for instance: alpha (~8-12 Hz), beta
(~16-24 Hz), and gamma (~40-120 Hz). Different bands have divergent underlying
physiological mechanisms and functions. High frequency oscillations (i.e. higher than
40 Hz) are an indication of active computational processing and are tightly linked
to spiking activity319,320 . Low frequency oscillations (below 30 Hz) can entrain faster
ones, for example when only allowing the fast oscillations to occur in the troughs of
the slow cycle321-323 . This creates a hierarchy where slow oscillations can control the
occurrence of faster ones and as a result can inhibit computational processing324-327.
The experiments described in this thesis will focus on low frequency oscillations,
between 8 and 30 Hz. This spectrum covers two important frequency bands, the
alpha band from 8-12 Hz and the beta band, peaking between 18-24 Hz. As a
consequence of the neurophysiological underpinnings, the oscillatory power in these
frequency bands is an index of cortical activity: power enhancement signifies local
3
56 | chapter 3
inhibition, while power suppression is a reflexion of high computational load321,328 .
Alpha power is especially, but not exclusively, prevailing in occipito-parietal cortex;
beta power, on the other hand, is prominently observed over central parieto-frontal
cortex, strongly associated with somatosensory and motor processing318,325,329 .
FunctionalMagneticResonanceImaging
Non-invasive electrophysiological recording techniques, such as EEG, allow a
relatively direct measurement of neural activity with a high temporal but poor spatial
resolution. These properties are complemented, both positively and negatively, by
another tool in the neuroscientist’s shed: functional magnetic resonance imaging
(fMRI, figure 3.1) 330-332 . This technique takes advantage of the magnetic properties
of the oxygen transporting protein haemoglobin, abundant in red blood cells. These
properties are dependent on oxygen binding to the protein’s iron sites. Accordingly,
in fMRI the relative levels of oxygenated and deoxygenated haemoglobin can be
quantified as a blood-oxygenated-level dependent signal (BOLD) 333,334 . Importantly,
this signal can be measured accurately across the whole brain with a very high spatial
resolution, even in volumes as small as 1 mm cubed (a volumetric pixel, or voxel).
The relationship between BOLD and neural activity pivots around an indirect
haemodynamic response, although its exact mechanisms and characteristics are not
completely resolved and are still actively debated. Neurons require energy to function
and take up glucose and oxygen from the blood in surrounding capillaries. If a group
of neurons is active they might need more energy to replenish the energy cost of
repeatedly generating action potentials and returning to equilibrium. Paradoxically,
the strongest haemodynamic response to active neurons is an increase in oxygenated
haemoglobin, and not a decrease as might have been expected335 . This is caused by a
local increase in blood flow and volume, potentially a consequence of the dilation of
smooth capillary muscles, which might be initiated by a signal cascade involving the
release of diffuse nitric oxide by the neurons themselves and other chemical signals
from nearby supporting glial cells336,337. The haemodynamic response, as quantified
by the BOLD signal, is very sluggish, especially with respect to the neural activity
itself. It usually peaks about 6 seconds after its initiation and might take more than
20 seconds to completely return to baseline. Fortunately, multiple BOLD responses
add reasonably linearly and temporally overlapping neural events can be separated
by making use of advances in experimental design and analysis. However, the high
spatial resolution of fMRI comes at the cost of a low temporal resolution. One scan
of the full brain, with thousands of voxels of 3 mm cubed, currently takes about 2
seconds to complete. This is faster than the BOLD response, but much slower than
many effects that can interfere with BOLD measurements. The blood is pumping
multi-modal neuroimaging of grasping | 57
rhythmically with the heartbeat, the chest is going up and down with breathing,
but most detrimental, even the smallest movements of the head have a very strong
deleterious effect on the signal of interest.
To account for head movements consecutive scans are spatially realigned before
the dataset enters statistical analysis 338,339 . But even after realignment, over 90%
of the temporal variance in the data can be attributed to residual motion artefacts.
In fMRI analyses, one scan is assumed to be acquired instantaneously, while in fact
it usually takes 2 seconds or more to complete. Movements, on the other hand,
are continuous and do not concur with the acquisition time. Consequentially, the
head motion within one acquisition period cannot be fully accounted for by spatial
realignment of the time-series. The signal of interest is so sensitive to motion
that even sub millimetre movements can have severe effects. Clearly, motion
artefacts present a challenge for all fMRI studies, let alone for those that are aimed
to investigate the cortical control of hand and arm movements. In the following
paragraphs several problems faced when studying visually-guided hand movements
in the MR environment will be addressed.
fMRIofvisually-guidedgrasping
An MR scanner bore is not the most forgiving place to perform ethologically valid
hand movements, first and foremost because of the very restricted space available.
Additionally, when the hand and arm move through the magnetic field they cause (just
as any other moving conductive body would) changing field inhomogeneities that
affect the signal quality340,341 . Moreover, the movements of the arm are mechanically
transferred via the shoulder and neck to small movements of the head. This transfer
of motion can be reduced by fixating the upper arm and only allowing the lower arm,
wrist and hand to move24 , but unfortunately the motion transfer cannot be excluded
completely. Likewise, co-contractions of neck and shoulder muscles are also hard to
prevent. An additional challenge is posed by the occurrence of the artefact relative
to the task. When studying action control, the motion artefacts caused by the
movements of the hand are inherently temporally linked to the timing of the task. As
such, artefacts might obscure the task-related activation, simply by adding variance
and decreasing the signal-to-noise ratio. Fortunately, here the sluggishness of the
haemodynamic response comes to good use, as the peak of the BOLD response
associated with the cortical correlate of the task is delayed by 6 seconds relative
to the initiation of the movement, while the corresponding artefact is virtually
instantaneous342 . However, if the incidence of the artefact and its temporal lag with
the BOLD-peak are relatively constant over the course of the experiment, general
linear model regressors designed to capture the task-related activity can still share
an unacceptably large part of their variance with nuisance regressors describing the
3
58 | chapter 3
Figure3.1 Studying visually-guided grasping in the MRI scanner
Photographs of an experimental setup aimed to study visually-guided grasping in
an MR environment. The subject lays supine with the head coil tilted (top panel).
In this position the subject can directly see and grasp the target object suspended
above the pelvis. The subject’s vision is controlled by LCD shutter glasses (bottom
left panel; the right glass is opaque allowing only left monocular vision). The whole
setup fits in the scanner bore, but the arch suspending the target object remains
accessible from outside (bottom right panel).
motion artefact. Especially fast event-related experimental designs can be sensitive
to this potential covariance of task and confound. In response to these challenges
some researchers have asked their subjects to grasp an object presented via a
mirror suspended above their head 157. Completing this task requires an additional
and unnatural visuospatial transformation of the spatial object features, potentially
engaging an altered cortical network 245,343 . In the experiment described in chapter
5, a direct and unconstrained viewing of the target object was promoted while
minimizing the detrimental effects of hand, arm and head motion on the taskrelevant fMRI signal. The employed approach rests on a collection of procedures and
principles (figure 3.1).
First, the whole head coil was tilted forward within the scanner bore by 30°. This
adaptation to the conventional fully supine fMRI setup is only possible when using
phased-array receive-only head coils instead of the more conventional circularly
multi-modal neuroimaging of grasping | 59
polarized head coils344 . Combined with a slight tilt of the head within the coil this
allowed direct foveal vision of the target object.
Second, when vision of the object was occluded during rest periods in between
trials, a visual anchor was provided for the eyes by means of two LEDs. This
prevented wandering of the eyes’ fixation during rest, and it excluded the need
for a corrective eye movement at the onset of each trial when the object became
visible. Eye movements can cause co-contraction of neck muscles (e.g. small
nodding movements accompanying a vertical saccade) 345 and engage an additional
cortical network that might obfuscate the correlate of grasp guidance. To limit
these detrimental effects some studies required subjects to fixate during the whole
experiment on an LED positioned above the target object 113,152,245 . Maintaining
fixation requires active control and inhibiting the habitual tendency to focus on the
object to be grasped. Moreover, fixating above the target causes the object to be
presented only in the lower peripheral visual field. It is conceivable that this artificial
restriction might alter the cortical circuit involved in grasp guidance 74 . Our approach
overcomes these issues, minimizing eye movements during rest, while still allowing
free gaze of the object during task performance.
Third, tight mechanical constrains on the subject’s body and head were kept to
a minimum. This might seem counterproductive if one aims to prevent the subject
from moving, but restrained subjects proved to be more likely to tense up over
the course of the experiment and produce jolting and erratic head movements.
Even though these movements are typically of small amplitude, they are fast and
consequentially often more confounding than slow movements of larger amplitude
(as long as the subject stays within the field of view, of course). Subject instruction,
training and comfort are considered more important factors in reducing head
movement confounds than physical restraints.
Fourth, across the whole experimental and analytical pipeline several additional
precautions were taken (see chapter 5 for details): a novel experimental design was
adopted to dissociate the planning of the grasping movement from its execution
and the stereotypical return movement back to the starting position, a dedicated
diagnostic tool was developed qualifying and quantifying potential confounds
before a dataset is entered into statistical analysis346 , and both linear and nonlinear
effects of motion were modelled using a Volterra series expansion of the movement
parameters347,348 .
Fifth, the remaining nuisance variance was indexed by estimating its contribution
to non-grey matter compartment signals. The rationale being that, for example,
white matter signal changes cannot be attributed to neural activity, but they are
similarly infected with the same confounding artefacts as grey matter. The resulting
confound index can be used to model and remove nuisance variance from the
3
60 | chapter 3
signal of interest, leading to lower residual error and higher power of the statistical
inference. This procedure is described in detail in chapter 4.
These protocols and procedures make the study of visually-guided grasping
behaviour in the MR environment feasible, while still allowing subjects to perform
ecologically relevant movements with free gaze.
TranscranialMagneticStimulation
Above, several neuroimaging techniques have been introduced by which neural
activity can be quantified. These observational methods provide mainly correlational
evidence of the relation between brain and behaviour. They can be complemented
by techniques that allow us to intervene with neural activity, supporting a causal
inference of the brain-behaviour relationship. Ever since Galvani first made a frog
leg twitch, electrical stimulation has aided our understanding of the nervous system.
Obviously, direct electrical stimulation is an invasive technique, only employed in
humans when severe medical reasons justify it, for example in the case of deep
brain stimulation of specific Parkinson patients. Using indirect stimulation from the
surface of the head to evoke action potentials in the cortex would require too high
currents in order to overcome the electrical insulation of the skull. However, recent
years have seen a promising reappraisal of low-intensity electrical stimulation at
the scalp (transcranial direct current stimulation, tDCS). These externally applied
weak currents do not directly depolarize neurons, but can inhibit or facilitate
cortical excitability by slightly modulating the intracranial electric properties 349,350 .
Unfortunately, these currents lack the required spatial focus to allow detailed
neuroanatomical inference of stimulation effects.
The disadvantages of electrical stimulation are to a large extent addressed by
the development of transcranial magnetic stimulation (TMS, figure 3.2) 351 . During
TMS, a very rapidly changing electric pulse is passed through a conductive coil held
above the head. Following Faraday’s law of electromagnetic induction, this changing
current generates a pulsed magnetic field that penetrates the scalp and skull reaching
the neural tissue of the cortex underneath. There, the changing magnetic field in turn
induces an electric current that, if strong enough, can depolarize neurons and evoke
action potentials. The strength of the magnetic field decays quickly with distance
allowing only the surface of the cortex to be reached. The electrophysiological
underpinnings of the effect of TMS are relatively poorly understood. In fact, there
is no such thing as ‘the’ effect of TMS. The efficacy of the stimulation and type of
neuronal populations excited depend on many factors: the current direction and
orientation, the stimulus strength, the pulse shape, and importantly, the neural
state at the moment of stimulation 352,353 . The most thoroughly studied and described
m ul t i - m o d al n e u r o im a gin g o f g r a s pin g | 61
application of TMS is over the primary motor cortex. A single pulse can directly
and indirectly innervate pyramidal corticospinal neurons, evoking descending
waves which, if strong enough, elicit a muscle twitch352 . The effect of motor cortex
stimulation can be quantified at the level of the muscle by measuring the motor
evoked potential (MEP) with electromyography (EMG). The amplitude of an MEP
is an index of the motor corticospinal excitability and efficacy of the stimulation 354 .
Paired-pulse protocols, where the test pulse over primary motor cortex is preceded
by a conditioning pulse (at the same or another site), have allowed the functional and
temporal description of cortico-cortical connections within the motor system355,356 .
Crucially, the enforced depolarisations also have a secondary effect: they
interfere with ongoing neural processes, often disrupting local computation357.
This principle is elegantly illustrated by the consequences of delivering TMS over
occipital cortex. Here, the primary effect of stimulation is to elicit the perception
of non-existent flashes of light (phosphenes), similar to evoking muscle twitches by
stimulation over the motor cortex. But additionally, as a secondary consequence,
short visual stimuli may be rendered undetectable for a brief period of time358,359 .
Disruptive effects of single pulse TMS are transient and stimulus detection is only
hampered if the TMS is delivered about 80 ms after stimulus presentation (i.e. the
time it takes a retinal signal to reach and be processed in the visual cortex), and
not 40 or 160 ms. Interestingly, repetitive stimulation can locally influence cortical
excitability and computational processes for extended periods of time, even up
to half an hour after the stimulation train has ended or longer360-362 . The type and
efficacy of these effects are highly dependent on the duration, frequency and shape
of the repetitive TMS protocol363,364 .
To summarize, TMS has provided human neuroscience with a powerful
neurointervention tool, moving beyond correlational analyses, allowing non-invasive
investigation of the causal relationship between brain and behaviour365,366 .
CombinedEEG-TMS
In this thesis TMS is used to study the critical contribution of two important nodes
in the dorsomedial and dorsolateral parieto-frontal circuits to grasp planning:
sPOS and aIPS, respectively. Importantly, both the cerebral and behavioural
consequences of stimulation were quantified by means of concurrent TMS, EEG and
kinematic recordings (figure 3.2). Compared to coarser indices of behaviour, such
as reaction and movement durations, advanced kinematic analyses allow more
informative and sensitive inference of the causal relationship between sPOS, aIPS
and grasping behaviour. With simultaneous recording of EEG, the consequences of
stimulation can be investigated at a level as close to the computational processes
as non-invasive neuroimaging allows, with high sensitivity to the effects of TMS
3
62 | chapter 3
perturbation. Moreover, behavioural measurements (no matter how advanced) will
always be more distant in time and cause from the moment of stimulation 367. Lastly,
the electrophysiological correlates of the task and stimulation can be quantified on
spatial, temporal and spectral dimensions, informing on the nature and dynamics of
the underlying computational processes. Combining these neuroimaging modalities
is unfortunately not without costs.
EEG and TMS are fundamentally incompatible: EEG detects the slightest
microvolt changes at the scalp (~1,000,000 times weaker than a small battery), and
TMS produces fast changing magnetic pulses up to 2 Tesla in strength (~40,000 times
stronger than the earth’s magnetic field). Only recently dedicated EEG systems that
remain operational during TMS have become commercially available. The technical
and practical details of combining TMS and EEG have been discussed in several
articles and reviews368-380 . However, no consensus has been reached on how to best
cope with the numerous EEG artefacts induced by TMS; therefore our approach in
this matter will be explained, especially where it differs from that of some others.
The TMS evoked confounds in the EEG signal can be very diverse of nature.
Indirect muscular (contraction of scalp, face and/or neck muscles), somatosensory
(stimulation of sensory nerves in the scalp), and auditory (click noise of the TMS
coil discharging) effects need to be considered but are often manageable. The
direct effects of the TMS pulse on the EEG signal, on the other hand, are truly
overwhelming. The pulse induces strong eddy currents in the electrodes; if the
amplifier continues to record, they cause the signal to clip out of range, and evoke
step-responses in the operational amplifier (op-amp) and resistor-capacitor (RC)
circuits of the amplifier hardware. How the electronics in the EEG cap and amplifier
react to the pulse is highly dependent on the equipment used, but with a modern
system often about 2-3 ms of data will be clipped and completely lost for analysis.
Moreover, a strong ringing artefact (step-response of the op-amp and low-pass
filter) generally obscures the genuine EEG signal in the first 6-7 ms following the TMS
pulse. Although this additive ringing could potentially be modelled and filtered369 ,
in our experience this does not lead to sufficiently consistent results: the artefact
was either still partially present, or the true EEG signal was affected by the filtering
procedure (too much variance explained by the filter). For the analyses performed
in chapters 6 and 7 data samples acquired in the first 10 ms following the TMS pulse
were excluded.
On top of and following the ringing artefact often one or more exponentially
decaying step-responses are observed, caused by capacitor systems. The most
obvious source of these exponential processes are the several RC circuits in the
amplifier, but also the electrode-electrolyte-scalp circuit might become polarized by
the TMS and start to act as a capacitor370,371 . In the experiments described in chapters
6 and 7, mono-phasic TMS pulses were applied, inducing a mainly unidirectional
multi-modal neuroimaging of grasping | 63
3
Figure3.2 Concurrent TMS and EEG
Photographs of a pilot study investigating the feasibility of transcranial magnetic
stimulation (TMS) during concurrent recording of electroencephalography (EEG).
Left panel: the TMS coil, as seen from the perspective of the experimenter, is held
over the subject’s head, who is wearing an EEG cap. Right panel: a subject sitting in
the experimental setup as used in the experiments described in chapters 6 and 7.
The subject’s head is stabilized by a chin rest and facing mechanical shutter glasses.
Magnetic kinematic trackers are attached to the subjects right hand. The index
finger and thumb are together pressing down the starting button.
current flow in the cortex381 . Such pulses have a relatively robust and well defined
effect on the cortex even at low stimulation intensities, but they also have a higher
probability of polarizing the electrode-electrolyte-scalp circuit compared to biphasic
TMS pulses. As the observed exponential decay artefacts were not consistent
over EEG sensors or repetitions, the data following each TMS pulse was screened
separately in all sensors. To cope with multiple overlapping exponential processes
with varying decay constants, an iterative least-squares fitting algorithm with a
logarithmically adaptive time window was developed.
First, the exponent with the smallest decay constant (describing a step-response
with a slow decay, for example lasting up to ~400 ms after the pulse) was fitted based
on the longest time window furthest from the pulse. Next, informed by the first
approximated function and based on a shorter and earlier time window, a second
exponent could be fitted with a larger decay constant. This iterative algorithm could
64 | chapter 3
maximally identify four individual exponential processes. In the end, all additive
exponential processes were estimated simultaneously (using informed priors based
on the individual functions) to find the best approximate complex step-response and
remove it from the data. Trials where this protocol did not result in satisfactory data
quality were discarded. All surviving trials subsequently entered an independent
component analysis (ICA), describing the data, originally structured as different
signals coming from individual sensors, as a collection of mixed independent
components with maximized entropy382 . In effect, this analysis is able to isolate
independent linear sources, for example cortical sources of the electrical signal, but
also induced artefacts383,384 . ICA was used to select and remove components related
to eye movements, blinks, muscle contraction and remaining TMS artefacts, such as
those caused by the recharging of the TMS coil (a weak current passing through the
TMS coil in preparation for the next pulse).
By capitalizing a combination of dedicated experimental setup and hardware
(see chapters 6 and 7), data clipping, artefact fitting and independent component
selection, EEG could be successfully recorded with minimal confounds of concurrent
TMS.
multi-modal neuroimaging of grasping | 65
3
4
A simple and effective
method to remove
global nuisance
variance in fMRI
68 | chapter 4
Abstract
In fMRI time-series many confounding sources contribute to the signal,
adding to nuisance variance and decreasing the power of statistical
inference of task-related local signal changes. Global variance is a prominent
confounding source, known to induce false deactivations in task-related
fMRI, or false anti-correlations in resting-state fMRI. Yet, ignoring global
signal changes is not a viable solution, as true activations might be missed
and functional connectivity strength might be falsely overestimated. Here,
we propose a novel and simple procedure to account for global confounds,
without introducing false deactivations or anti-correlations.
The procedure relies on using anatomically informed indices of
global T2* signal independent from local BOLD changes. Following brain
segmentation in different compartments (white matter, cerebrospinal fluid,
and a residual compartment outside the head) we consider the mean signal
for each compartment at each image of an fMRI time-series. We show that
modelling these compartment time courses as additive nuisance covariates
in the context of the General Linear Model reduces residual model error and
increases statistical power, both at single subject and group levels, across a
wide range of fMRI protocols. Accordingly, we suggest that this procedure is
generally beneficial for fMRI analyses, both in investigations of task-related
changes and of functional connectivity in resting-state fMRI.
removing global nuisance in fMRI | 69
Introduction
fMRI is a well-established tool for studying brain-behaviour relationships, but those
relationships are concealed within many sources of variance that contributes to the
measured signal. Besides a host of physiological (e.g. breathing, heart pulsation,
swallowing) and physical processes (e.g. spatial and temporal alterations of MR
scanner baseline, thermal noise) movements of the subject provide a major source
of nuisance variance in fMRI studies. Subject’s movements inside the scanner
often affect the signal over the whole volume scanned, resulting in head motion,
as well as spatial and temporal inhomogeneities in the main magnetic field. Head
motion is partially compensated for by spatially realigning subsequent scans under
the assumption of rigid-body transformations 339 . However, head movements
occur continuously and within the period of volume acquisition, violating those
assumptions. Given that spatial realignment of fMRI time-series only corrects
for a portion of the induced motion artefacts, additional head motion effects are
considered in fMRI time-series analyses by including realignment parameters in the
General Linear Model (GLM) as covariates of no interest338 . For instance, non-linear
effects of motion can be partially accounted for by considering a Volterra series
expansion of the realignment parameters 347,348 . Differently from the effects of
head motion, the spatial and temporal inhomogeneities of the main magnetic field
introduced by body motion are usually ignored. This is surprising, since this nuisance
factor can have considerable effects on the MR signal. For instance, even the
rhythmic contractions of the diaphragm during breathing considerably influence the
MR signal340 , let alone the field disturbances caused by head motion or movements
of the arm and hand341 . Such movements might be part of a task 25 (for instance,
see chapter 5 of this thesis), or they could be inadvertent (e.g. tremor-dominant
Parkinson patients or children385,386). As the head and body movements are likely
to be temporally linked to task-related activity, it is crucial to correctly account for
these confounds.
Many sources of nuisance variance, especially those causing inhomogeities
of the magnetic field, affect the signal over the whole volume scanned. Estimates
of the global signal over the image can be straightforwardly considered as either
additive (i.e. representing a covariate of no interest) 387 or multiplicative effects
(i.e. global scaling or global normalization) 388 . Yet, these procedures are seldom
implemented in conventional fMRI analyses, given their potential for introducing
false BOLD deactivations389 . When the global signal carries variation caused by
blood-oxygenation-level dependent (BOLD) responses in the grey matter, then
removing the global signal will also alter BOLD-related effects390,391 . The issue of
global signal confounds is possibly even more acute in resting-state fMRI studies,
where the covariance of two regions over time is interpreted as a sign of functional
4
70 | chapter 4
connectivity between these regions. Ignoring global signal changes might decrease
the sensitivity of the resting-state analyses and subsequent inference392 . For
example, when two regions are found to highly correlate one wants to exclude that
their covariance is caused by a third, potentially global, source of signal change.
On the other hand, when the global signal is informed by strong, but local BOLD
changes, removing them from the signal might introduce false anti-correlations393,394 ,
similarly to the notion described above for task-related fMRI 390 . Some researchers
have suggested that positive correlations are only interpretable when the global
signal has been accounted for, while negative correlations are only valid when
refraining from global signal consideration395 .There have been several proposals
on how to address global nuisance variance, without removing true or introducing
false local changes341,389,391,396-400 . Unfortunately, they have not been widely adopted
by the fMRI community; many of these proposals involve advanced modelling,
complex additional data processing steps and dedicated solutions. Here, we suggest
a simple, robust procedure to account for global signal confounds, applicable to both
task-related fMRI paradigms and resting state connectivity studies. We propose to
index global signal confounds by considering signal changes of white matter (WM),
cerebrospinal fluid (CSF) and a residual compartment outside the head (RC). These
compartment signals are not influenced by changes in BOLD induced by neural
activity in the grey matter (GM), and are as such not informed by the experimental
design or by relevant BOLD changes during resting state studies. The resulting
compartment time-series (and a Volterra expansion to capture non-linear effects)
can be included in standard first level GLMs of fMRI analyses as additive covariates
of no interest.
We hypothesize that when these compartment indices of global disturbance are
considered in the GLM, the statistical inference of fMRI analyses should improve,
as previously unexplained or falsely attributed variance is now correctly taken into
account. This is expected to lead to a better description of the cerebral correlates
of the task within each subject. Crucially, improved within-subject inferences could
lead to reduced between-subjects variability in task-related or resting-state BOLD
signal, a critical factor influencing the power of group level fMRI analyses. We tested
this hypothesis in three different experimental contexts, sampling different levels
of expected noise: 1) a visually-guided grasping task, 2) a study involving Parkinson
patients affected by tremor, and 3) a language perception study involving minimal
movements by the subjects. We report the effects of the proposed procedure
on mean Z-scores and residual sum-of-squares of both single subject and group
analyses.
removing global nuisance in fMRI | 71
Methods
We use the datasets from three published experiments (including chapter 5 of this
thesis) 401,402 to test our proposed procedure to account for global nuisance variance.
We report on the methods of those experiments as far as they are relevant for this
study. In the following sections the study from Verhagen et al. (chapter 5 of this
thesis) is labelled ‘Grasping’, the study from Helmich et al. 401 is labelled ‘Parkinson’,
and the study from Menenti et al. 402 is labelled ‘NoMovement’.
Subjects
Nineteen healthy young men (22 ± 2 years, mean ± standard deviation) participated
in study 1 (Grasping), 19 idiopathic Parkinson’s disease patients (13 men; 53 ± 9
years) participated in study 2 (Parkinson), and 32 healthy volunteers (13 men; 22 ±
3 years) in study 3 (NoMovement), after giving written informed consent according
to institutional guidelines of the local ethics committee (CMO region ArnhemNijmegen, The Netherlands). All participants had normal or corrected-to-normal
vision and were right-handed (>80% on the Edinburgh Handedness Inventory) 403 .
Experimentaldesignandsetup
The experimental designs were constrained by the need to minimize correlations
between regressors describing possible sources of confounds and regressors
describing cerebral correlates of the tasks. The timing of stimulus presentation and
the occurrences of levels were pseudo randomized in all experiments.
Grasping: Subjects were asked to grasp a rectangular object suspended above
their pelvis with their right hand, starting from a button box positioned on their
abdomen. This prehension task involved moving the hand and lower arm, rotating
around the elbow. Care was taken to minimize the carry-over of arm movement to
head motion by restraining the subject with Velcro straps and soft cushions. Liquid
crystal density (LCD) glasses controlled the vision of the object, allowing only vision
during movement planning. Light emitting diodes (LED) provided an anchor for eye
movements during inter-trial intervals when the object was not visible, in order to
further reduce task-related head motion. Subjects performed a total of 240 trials in
~35 minutes of scanning.
Parkinson: Line drawings of left and right hands in various orientations were
presented on a screen visible via a mirror above the patients’ heads. They were
asked to indicate on each trial if the presented hand was either a left or a right
hand. Subjects solve this task by mentally rotating their own hand from its current
4
72 | chapter 4
position into the stimulus orientation for comparison 404 . Using this motor imagery
paradigm, motor planning can be investigated in the absence of execution 405 . The
only instructed instances of motor execution in study 2 were button presses made
with the left or right big toe on each trial, and 30 changes of lower arm posture across
the whole scanning session. At the beginning of each block of 16 trials a cartoon
indicated the subjects to position their arms in one of three possible postures (both
extended, only left arm flexed across the abdomen, only right arm flexed across the
abdomen). In total, subjects performed 480 trials over ~40 minutes.
NoMovement: Subjects were presented on each trial a small text on a screen visible
via a mirror above the head coil. After three context sentences a critical test sentence
was presented word by word, which could be in congruence with common world
knowledge or not, and in congruence with the context sentences or not (constituting
a 2 x 2 factorial design). Subjects were asked to read the context and the critical
sentences passively. To ensure the subjects remained attentive on the stimuli, a
content question about the preceding text and two possible answers were shown
on 10% of the trials. Subjects were asked to select their answer with a button press
using their index finger. In total, subjects saw 132 trials in ~60 minutes divided over
four subsequent runs.
Imageacquisition
Grasping: Images were acquired on a 3 Tesla Trio MRI system (Siemens, Erlangen,
Germany), using the body coil for radio frequency transmission, and an eightchannel phased array surface head coil for signal reception. Blood oxygenation leveldependent (BOLD) sensitive functional images were acquired using a single shot
gradient echo-planar imaging (EPI) sequence (repetition time (TR)/echo time (TE),
2030/30 ms; 32 transversal slices; distance factor, 17%; effective voxel size, 3.5 x 3.5
x 3.5 mm; field-of-view (FOV), 224 mm). After the functional scan, high-resolution
anatomical images were acquired using an MP-RAGE GRAPPA sequence with an
acceleration factor of 2 (TR/TE/inversion time (IT), 2300/2.92/1100 ms; voxel size,
1.0 x 1.0 x 1.0 mm; 192 sagittal slices; field of view (FOV), 256 mm).
Parkinson: Images were acquired on a 1.5 Tesla Sonata MRI system (Siemens,
Erlangen, Germany), using the standard head coil for radio frequency transmission
and signal reception. BOLD sensitive functional images were acquired using a single
shot gradient EPI-sequence (TR/TE, 2560/40 ms; 32 axial slices; voxel size, 3.5 x 3.5
x 3.5 mm; FOV, 224 mm). High resolution anatomical images were acquired using an
MP-RAGE sequence (TR/TE, 3350/3.39 ms; voxel size, 1.0 x 1.0 x 1.0 mm, 176 sagittal
slices; FOV, 256 mm).
removing global nuisance in fMRI | 73
NoMovement: Images were acquired on a 3 Tesla Tim-Trio MRI system (Siemens,
Erlangen, Germany), using the body coil for radio frequency transmission, and a
twelve-channel phased array surface head coil for signal reception. BOLD sensitive
functional images were acquired using a single shot gradient EPI-sequence (TR, 1860
ms; 28 transversal slices covering at least the temporal and frontal lobes; distance
factor, 10%; effective voxel size, 3.0 x 3.0 x 3.3 mm; FOV, 224 mm). High resolution
anatomical images were acquired using an MP-RAGE sequence (TR, 2300 ms; voxel
size, 1.0 x 1.0 x 1.0 mm, 192 sagittal slices; FOV, 256 mm).
Imagepre-processing
We aimed to keep the image pre-processing and analysis as close to the methods
of the original publications. However, to make comparisons across studies possible,
some adjustments had to be made. For example, we used the same software package
to pre-process and analyse all imaging datasets: SPM5 (Statistical Parametric
Mapping, http://www.fil.ion.ucl.ac.uk/spm/; the Parkinson dataset was originally
analysed using SPM2). Before analysis, the images were spatially realigned using a
least-squares approach that estimates six rigid body transformations (translations,
rotations) by minimizing head movements between each image and the reference
image. The time-series for each voxel were temporally adjusted to the first slice in
time to correct for differences in slice time acquisition. Each participant’s structural
image was spatially coregistered to the mean of the functional images and spatially
normalized onto a Montreal Neurological Institute (MNI) aligned T1 template using
both linear and nonlinear transformations in a probabilistic generative model that
combines image registration, tissue classification, and bias correction 406 . This
procedure also provided structural images of segmented grey matter, white matter,
and cerebrospinal fluid, used later in the analysis. Subsequently, functional images
were normalized and resampled at an isotropic voxel size of 2 mm using the same
transformation matrix applied to the structural images. Finally, the normalized
functional images were spatially smoothed using an isotropic 8 mm full-width at
half-maximum (FWHM) Gaussian kernel.
Compartmentsignalchanges
We considered MR signals sampled from white matter, cerebral-spinal fluid, and
outside the skull as indices of global signal that are not informed by the experimental
design. These signals were obtained as follows.
First, the residual compartment (RC) compartment was created by taking the
binary inverse of the sum of the compartment probability maps obtained from the
segmentation of the structural image: grey matter (GM), white matter (WM) and
4
74 | c h a p t e r 4
6 a.u.
whilte matter
10 a.u.
grey matter
4 a.u.
whole image
cerebrospinal fluid (CSF). This provided a binary mask that included only voxels that
not attributed to GM, WM, or CSF. The median intensity of all inclusively masked
voxels from the structural image proved to be an applicable and robust cut-off below
which voxels belonging to neither the skull, nor the ghosting image survived, leaving
only voxels corresponding to empty air outside the brain, skull and ghosting in the
0.25 a.u.
residual
CSF
10 a.u.
2 minutes
Figure4.1 Representative segmented compartments and their time-series
The structural image and four segmented compartments are shown for one
representative subject: grey matter, white matter, cerebrospinal fluid and a residual
compartment. The thresholded (>99% probability) masks are represented as
coloured overlays. The respective mean time-series (right panels) are in consistent
but arbitrary units. The time-series of the white matter, cerebrospinal fluid and the
residual compartment were considered in the context of the General Linear Model
as additive nuisance covariates.
removing global nuisance in fMRI | 75
mask. These three images (WM, CSF, and RC) were resampled to match the voxel
size and bounding box of the normalized mean functional image.
Second, voxels with a compartment probability of less than 99% (as provided by
the SPM5 probability masks) were excluded to prevent accidental misclassifications
or overlap of signal from one compartment to the other (figure 4.1). These
compartment masks could, hypothetically, be further refined by combining them
with an exclusive mask of the grey matter (GM) compartment convolved with a
sphere, for example with a radius of 1 cm. This would ensure that no voxels survive
the mask that are within 1 cm of high probability grey matter voxels and prevent
any leakage of GM signal to the WM compartment by suboptimal segmentation
or image smoothing. Furthermore, any spurious voxel extensions away from the
main body of the compartment can be eroded away by further smoothing and
subsequent thresholding, or by more advanced methods as available in image
processing packages (e.g. Image Processing ToolboxTM of MATLAB, Mathworks,
Natick, Massachusetts, USA). However, in this report we have refrained from
incorporating these hypothetical improvements, as we aimed to test the most easily
implementable protocol based solely on standard fMRI processing techniques.
Third, the WM, CSF and RC images were applied as spatial masks onto the
spatially normalized and realigned fMRI time-series to extract the mean intensity
of the included voxels for each scan in time. In this fashion three separate time
courses were obtained, one for each image compartment, describing the global
changes occurring in each compartment during the scanning session (figure 4.1:
representative example time courses of the global signal, GM, WM, CSF, and RC).
GeneralLinearModels
The pre-processed fMRI time-series were analysed using an event-related approach
in the context of the General Linear Model, using standard multiple regression
procedures. The specific model factors in each experiment are not relevant for the
purposes of this study. It should be noted, however, that in all experiments all trial
phases were explicitly modelled and incorrect trials were modelled in a separate
regressor of no interest. Each of the resulting model regressors was convolved with
a canonical haemodynamic response function. In experiment 1 (Grasping) the model
regressors were also convolved with the temporal and dispersion derivatives of this
canonical haemodynamic response function 407.The potential confounding effects of
residual head movement-related effects were modelled using a Volterra expansion
of the time-series of the estimated head movements during scanning, including
the original, the squared, the first order derivative, and the squared first order
derivative347. We specifically chose to model the linear, quadratic, spin-history and
quadratic spin-history effects of head motion with these 24 additional covariates
4
76 | chapter 4
of no interest to make sure that we accounted for as much of the head motionrelated variance as possible in a way that can still be easily implemented in standard
analyses. Finally, the fMRI time-series were high-pass filtered (cut-off 128 s) to
remove low-frequency confounds, such as scanner drifts. Temporal autocorrelation
was modelled as a first-order autoregressive process.
To test our hypotheses, for each subject we built three additional critical general
linear models based on the basic model described above (labelled ‘Basic’). For the
first test model (labelled ‘GlobScale’), the model regressors were normalized to the
global mean of the functional images using the global scaling option implemented
in SPM5. All other specifications remained as in the Basic model. For the second
model (labelled ‘CompSignal’), the WM, CSF and RC time courses were included in
the multiple regression analysis as additional covariates of no interest. Finally, in the
third model (labelled ‘CompSignalExp’), we considered a Volterra expansion of these
covariates, similar to the expansion of the head-motion parameters: linear, squared,
first-order derivative, and squared first-order derivative, resulting in 12 additional
covariates.
For each subject, linear contrasts pertaining to the regressors of interest (study
1, six; study 2, eight; study 3, four) were calculated. Consistent effects across subjects
were tested using random-effects analyses on these contrasts images by means of
within-subject ANOVAs. In total 12 random-effects analyses were performed, four
for each experiment.
In this study including the compartment signals as additive covariates in multiple
regression analyses was compared to the global scaling method as implemented
in SPM5 and a basic model without any form of global normalization. In effect,
we are directly comparing two different ways to account for global variation: as
additive terms in the model (representing covariates of no interest in the context
of a GLM) 387, or as a multiplicative consideration of global signal (i.e. global scaling,
which also scales the error variance) 388 . We chose to model the global nuisance
variance as an additive term for multiple reasons. First, this allows considering all
three compartments (WM, CSF, and RC) separately. Second, a Volterra expansion
can be applied easily on the covariates to account for non-linear effects of global
disturbances. Third, several reports comment on the relatively small difference
between considering the global signal as a multiplicative or additive term in the
General Linear Model389,408 . We expected the largest difference when comparing
models with or without global signal consideration (regardless of method), and
when comparing global scaling with GM contributions to modelling only WM, CSF
and RC signal variation. Fourth, in most fMRI analysis packages adding an additional
user-specified covariate to the General Linear Model is very straightforward. This
ensures the ease of implementation of this procedure and improves the replicability
of this study.
removing global nuisance in fMRI | 77
Statisticalinference
In this study we aimed to dissociate global signal confounds from local BOLD changes.
Comparing statistical maps (thresholded for multiple comparisons) across competing
models would provide biased results 409 . For example, when analysing a dataset that is
expected to be confounded by movement effects (e.g. when the subjects performed
arm movements while in the scanner bore), more artefactual false activations are
expected to survive a certain threshold when motion parameters are not included in
the model compared to a model that does consider these parameters. Calculating an
average statistical value over all voxels, or counting the number of voxels surviving
a certain threshold, is likely to be strongly influenced by motion artefacts. In that
case, one might falsely conclude that the model that removes the least number of
artefacts would be the best one. Therefore we limited our test volume to a volume
of interest (VOI) appropriate for each experiment, instead of calculating an average
index over the whole brain. Our inference will necessarily be limited to our search
space, but this is an essential concession to minimize the bias.
While task-related activity is expected to be common over subjects, artefacts
are generally not shared. Accordingly, confounds which might lead to high statistical
values in single subject analyses, could introduce inter-subject variance in the group
level analysis, lowering the chances of observing a significant effect over the whole
group. In other words, a model that explains confounding variance could lower the
average statistical value in each subject individually, but will result in more accurate
single subject descriptions, reducing the variance on the group level. Generally, fMRI
studies are designed to make inferences on the group level, not on single subject
basis. Accordingly, the critical test of the competing models is performed on the
group level.
The volumes of interest to which we restricted our tests were chosen for their
known robust BOLD activation in tasks similar to the ones used in the experiments.
Spherical VOIs were drawn with a radius of 8 mm around the following coordinates
(MNI space). In study 1 (Grasping) subjects were asked to grasp an object with their
right hand, so we chose the left primary motor cortex (M1: [-34 -24 62], chapter 5
of this thesis). In study 2 (Parkinson), subjects were viewing line drawings of hands
presented on a screen visible via a mirror, so we chose left primary/secondary visual
cortex (V1/V2: [-24 -96 12]) 401 . In study 3 (NoMovement), subjects were attending
sentences presented on a screen. In each trial the non-critical sentences, providing
context for the critical sentence, were modelled in a separate regressor of no
interest, in effect capturing the low-level BOLD effects of viewing stimuli passively.
Therefore, we chose the left middle temporal gyrus (MTG: [-52 -42 0]) 402 , known to
be involved in language perception.
4
78 | chapter 4
In each model we created an F-contrast testing if the intensity of a voxel deviated
significantly from zero (either positively or negatively) for the regressors of interest.
The F-contrast excluded the regressors of no interest (e.g. misses or baseline) and
spanned the exact same regressors in the first as in the second level analyses. We
considered two indicators of model strength to assess our hypotheses.
The mean Z-score of the voxels included in the volume of interest under this
F-contrast (thresholded at p<0.05 uncorrected at the voxel level) was taken as the
first indicator of model strength. This value increases with increasing model fit. The
better the model considers the variance in the data, the higher the mean Z-score
of the regressors of interest. The residual sum-of-squares (RSS) in the context of
a General Linear Model is provided for each voxel by the SPM5 model estimation
procedure. We considered the mean RSS in the same volume of interest as used
to obtain the mean Z-score as our second indicator. This index decreases with
increasing model fit. The less variance remains unexplained, the lower the mean
residual sum-of-squares.
On the first level, these indices were calculated for each model of each subject
separately ((4 x 19) + (4 x 19) + (4 x 32)). We performed paired t-tests on these first
level indices to directly test which model performed the best (i.e. resulted in the
highest Z-scores and lowest RSS). On the second level, the exact same indicators
were calculated, but now only once for each model in each experiment (4 x 3).
Because each study resulted in only one random-effects second level analysis per
model, differences between the models can only be qualitatively compared and not
be statistically tested. The Z-scores and RSS are described as a relative percentage
of the value of the control model (Basic).
Results
Firstlevelanalyses
In the Basic models no global variance was accounted for, while in the GlobScale
models global scaling as implemented in SPM5 was applied. In the CompSignal and
CompSignalExp models WM, CSF and residual compartment signal regressors were
included in the multivariate analyses as additive covariates. In the CompSignalExp
models an additional Volterra expansion of their linear components was considered.
In experiments 1 and 2 (Grasping and Parkinson) considering global signal
changes in first level multiple regression analyses did not strongly affect the mean
Z-scores (figure 4.2A). In the Parkinson study they tended to decrease when global
variance was accounted for compared to the Basic model (GlobScale: p=0.008;
removing global nuisance in fMRI | 79
200
150
z-score
Basic
GlobScale
CompSignal
CompSignalExp
100
50
C
200
relative z-score (%)
0
150
Grasping
150
100
residual sum-of-squares
50
0
Parkinson NoMovement
Grasping
Parkinson NoMovement
Second level (group) analyses
z-score
residual sum-of-squares
D
Basic
GlobScale
CompSignal
CompSignalExp
100
50
0
B
relative rss (%)
250
Grasping
150
relative rss (%)
A
relative z-score (%)
First level (single subject) analyses
Parkinson NoMovement
4
100
50
0
Grasping
Parkinson NoMovement
Figure4.2 Statistical indices of model fit
Three different ways to account for global confounds were examined: normalising
the fMRI time-series to the mean whole image intensity (including grey matter;
GlobScale), considering compartment time-series (excluding grey matter) as
additive nuisance covariates (CompSignal), and considering a Volterra expansion
of these compartment covariates (CompSignalExp). These approaches were
contrasted to a basic model without any global signal consideration (Basic). The
fitness of all tested models was characterized by average z-scores (left panels) and
residual sum-of-squares (right panels) in regions of interest in three fMRI paradigms
(Grasping, Parkinson, and NoMovement). A-B. Z-scores and residual sum-of
squares for first level (single subject) analyses relative to the Basic model without
global confound consideration. C-D. Relative z-scores and residual sum-of-squares
for second level (group analyses).
80 | chapter 4
CompSignal: p=0.062; CompSignalExp: p=0.046). The first level Z-scores in the
CompSignal and CompSignalExp models were slightly higher than when global
scaling was applied (CompSignal: p=0.044; CompSignalExp: p=0.083). In experiment
3 (NoMovement), strong and significant improvements in Z-scores were found with
global signal inclusion (figure 4.2A; all p<0.001). In this study the compartment
signal covariates performed significantly better compared to the global scaling
(CompSignal: p=0.001; CompSignalExp: p=0.034).
The mean residual sum-of-squares revealed a consistent effect of global signal
inclusion: they decreased drastically and significantly (figure 4.2B; all p<0.001). In
all studies the CompSignalExp method was superior over the CompSignal method
(all p<0.001), which was in turn superior over the GlobScale method in explaining
variance in the general linear models (all p<0.001). This indicates that in a model
without global normalization, especially without considering the compartment
signal changes, a significant amount of variance would remain unexplained, adding
to the error term.
Secondlevelanalyses
For group fMRI studies the most informative test of the behaviour of the competing
models accounting for global variance is performed on the second level analysis.
Here, we observed that in all studies the residual sum-of-squares decreased
sharply (models GlobScale, CompSignal, and CompSignalExp) compared to the
model without global normalization (Basic) (figure 4.2D). Especially when global
variance was modelled as an additive factor (CompSignal, and CompSignalExp) in
the experiments where confounds could be expected a-priori (studies 1 (Grasping)
and 2 (Parkinson)), the RSS was strongly reduced, up to 50%. This effect was further
substantiated by the behaviour of the Z-scores (figure 4.2C). The two additive
models of compartment signals perform clearly better than the Basic model, but also
outperform the global scaling model, especially in the datasets were global signal
disturbances could be expected (Grasping and Parkinson studies). Even though the
models cannot be statistically contrasted to each other, the size and consistency of
the effect adds to the reliability of these findings.
Discussion
fMRI time-series are confounded by global signal disturbances, for example induced
by head or body movements and other physiological and physical processes.
fMRI analyses aim to support inferences of local changes of BOLD. Accordingly,
accounting for global nuisance variance in the image processing or analysis could
removing global nuisance in fMRI | 81
reduce the residual error of the analytical model and dissociate confounding
covariance from BOLD changes of interest. Importantly, the global signal, i.e. the
mean intensity of the whole scanning volume over time, also includes local BOLD
changes390 . Removing it from the fMRI time-series could induce false deactivations
in task-related fMRI analyses 389 , or false anti-correlations in the context of functional
connectivity analyses of resting state fMRI 394 . Global signal normalization has both
potential beneficial and detrimental effects on fMRI inference and its application
is debated. Here, we propose to characterize global nuisance by indexing its
contribution to compartment signals that are not informed by local task-related
BOLD changes in grey matter: white matter, cerebrospinal fluid and a residual
compartment outside the head. Considering these compartment time-series as
nuisance covariates in the context of a GLM explained additional variance at the
subject-level, leading to higher Z-scores in regions showing task-related activity at
the group level analyses. In contrast, modelling the whole volume signal including
grey matter led to inconsistent results at the group level. This indicates that global
signal disturbances can be effectively and robustly accounted for, and that this
simple procedure has a general advantageous effect on statistical inference of fMRI
analyses, without causing false deactivations or anti-correlations.
Implementation
In this study, we addressed the detrimental effects of global signal inclusion in
fMRI analysis, simply by leaving out the grey matter contribution to the global
signal estimate. Using standard brain image segmentation techniques we created
conservative subject specific masks of WM, CSF, and a residual compartment
outside the head, and used those to obtain three time-series of the mean intensities
of the voxels included in each mask. These time-series (and a Volterra expansion of
them) can be included in first level multiple regression analyses as additive nuisance
covariates.
Our proposed procedure can be implemented easily in standard imaging
pre-processing procedures and multiple regression analyses in the context of the
General Linear Model. It is sensitive to subject specific anatomical features and can
be fully automated. Interference of the experimenter and qualitative decisions are
not necessary. This procedure operates on the first level alone and therefore has no
effect on the number of degrees of freedom on the second level. More specifically,
the way in which the second level random effects analysis is performed is not affected
in any way by this procedure to remove global confounds.
4
82 | chapter 4
Application
We have tested the proposed procedure on three different datasets, aiming to
investigate whether beneficial effects of the procedure can be generalized across
different fMRI protocols. One study involved subjects performing grasping
movements, another study involved Parkinson’s disease patients with tremor, and a
third study was representative of fMRI protocols with occasional flexion of the index
finger (a button press). Especially the first two studies have an a-priori high potential
to be confounded by motion artefacts. In these studies, the aim of including
compartment signal covariates in the analysis was twofold: to model previously
unexplained variance and increase the statistical sensitivity, but also to counteract
potential of artefactually induced false positives that obfuscate the true task-related
BOLD response. Accordingly, removing global confounds in the data could both lead
to an increase in significant task-related activity, but also to a decrease of falsely
overestimated artefacts, together improving the balance between model sensitivity
and specificity. On the first level analyses of the Grasping and Parkinson datasets
we observed a decrease of the Z-scores in the VOIs. Speculatively, the compartment
signal covariates might have prevented global signal changes to be falsely attributed
to local BOLD changes of interest. In that case, a lower Z-score would actually be
closer to the truth than an artificially high Z-score. The drastic decrease of the residual
sum-of-squares in the Grasping and Parkinson studies provides indirect support for
this speculation (figure 4.2B). The increase in model specificity could be of relevance
for fMRI studies focusing on single subject analyses, such as patient assessments or
pre-operation functional mapping.
For most studies, the relevant inference occurs at the second level. In this
context, we show that considering compartment signals provides higher sensitivity
than neglecting global normalization or global scaling method (figure 4.2C-D).
Noteworthy, the proposed method did not result in decreased Z-scores in any of the
three experiments, whereas global scaling did (in the Parkinson study; figure 4.2C).
This indicates that the proposed protocol is generally applicable to task-related fMRI
analyses, irrespective of the specific paradigm or a-priori risk of confounds.
Based on the effects of the compartment signal covariates in task-related fMRI
studies, we speculate that beneficial effects will also be prominent when applied to
functional connectivity analyses of resting state fMRI 410,411 . These analyses quantify
the coupling between intrinsic BOLD fluctuations in different regions of the brain 412 .
But because the largest contribution to a region’s signal variance is shared by the rest
of the brain 413 , measures of covariance might be biased. The procedure proposed
in this study might allow the unbiased interpretation of both positive and negative
correlations between two brain regions. This would be a more sensitive approach
than inferring positive correlations only when the global signal is considered, and
negative correlations when the global signal is ignored394,395,398 .
removing global nuisance in fMRI | 83
Interpretationalissues
The time courses of the WM, CSF and RC compartments are expected to strongly
overlap, but their sensitivity to index global confounds might differ. Accordingly,
they are considered separately in the analysis. WM and CSF compartments border
the GM, increasing the chances of unintended inclusion of GM signal as a result of
suboptimal compartment segmentation or image smoothing. For this reason we
used a high posterior probability threshold (99%) to select compartment specific
voxels.
The segmentation of the CSF compartment might include veins draining
blood from the GM, which may show delayed haemodynamic responses and
consequentially inform the CSF time course with (time-shifted) task-related BOLD
effects. Restricting the CSF compartment to include only the ventricles partly
addresses this problem, but the ventricular signal may still show drainage effects
and is susceptible to local motion artefacts, and might therefore not be a good index
of the global signal disturbances.
Theoretically, the RC signal, measured in the air surrounding the head (and
excluding the image ghost), should not result in a measurable BOLD signal. In
practice, it provides a good index of physical scanner noise without being influenced
by physiological processes and noise. Accordingly, RC intensity was shown to
significantly correlate with the posture of the hand and arm over the pelvis within
the static magnetic field (paired t-test, p=0.009) 346 .
Because most of the variance of the compartment regressors is expected to
be shared, every additional compartment signal covariate or its Volterra expansion
will improve the model less dramatically. This is illustrated by the small increase in
Z-scores observed when including the Volterra expansion in addition to only the
linear component (figure 4.2B-C).
We have performed a quantitative analysis of the effectiveness of the proposed
procedure and have refrained from qualitative interpretations. We aimed to make our
analyses reproducible, independent of researcher and lab, and to draw inferences on
the effects of the considering compartment signals beyond the three experiments
described in this report. We feel that the two indicators we chose (mean Z-score and
mean residual sum-of-squares) are both easy to obtain and interpret, while being
powerful and informative enough to test our hypotheses.
The search volumes of our indicators was limited to spherical volumes of
interest with a radius of 8 mm. Importantly, drawing inferences on model quality
is only informative in voxels with variance that is represented by the regressors of
interest. In other words, it is not of interest if the mean Z-scores of voxels with no
behavioural correlate increase or decrease. Furthermore, the chances of finding
false deactivations when erroneously considering the global signal containing GM
4
84 | chapter 4
variance390 are highest in voxels with a strong BOLD response. By testing only those
voxels, we can more specifically claim that the proposed procedure to account for
global disturbances does not lead to false BOLD deactivations.
Although here we report on only a few regions of interest in a few studies, these
areas and studies cover a wide range of different fMRI paradigms. Furthermore, this
procedure has been successfully applied in several studies not included in this report,
covering an experimentally diverse set of experiments 24,25,205,414-416 .
Conclusion
We have proposed a novel and simple method to remove global signal disturbances
in fMRI analyses without introducing false deactivations or anti-correlations. We
have considered global nuisance by segmenting the brain and including non-grey
matter compartment specific time courses as additive covariates in single subject
multiple regression analyses in the context of the General Linear Model. This
procedure is easy to implement in standard processing techniques and results in a
robust improvement of the inference of statistical test at the group level compared
to not including global nuisance indices and to standard global normalization, across
a wide range of experimental paradigms. We propose that this procedure could be
generally beneficial in both task-related fMRI and functional connectivity analysis of
resting state fMRI.
removing global nuisance in fMRI | 85
4
5
Cortical circuits
supporting perceptuomotor interactions
during grasp planning
88 | chapter 5
Abstract
Adaptive behaviour relies on the integration of perceptual and motor
processes. In this study we aimed at characterizing the cerebral processes
underlying perceptuo-motor interactions evoked during grasping
movements in healthy humans, as measured by means of functional
magnetic resonance imaging. We manipulated the viewing conditions
(binocular or monocular) during planning of a grasping movement,
while parametrically varying the slant of the grasped object. This design
manipulates the relative relevance and availability of different depth cues
necessary for accurate planning of the grasping movement, biasing visual
information processing towards either the dorsal visual stream (binocular
vision) or the ventral visual stream (monocular vision).
Two critical nodes of the dorsomedial visuomotor stream (sPOS and
PMd) increased their activity with increasing object slant, irrespective of
viewing conditions. In contrast, areas in both the dorsolateral visuomotor
stream (aIPS and PMv) and in the ventral visual stream (LOtv) showed
differential slant-related responses, with activity increasing when monocular
viewing conditions and increasing slant required the processing of pictorial
depth cues. These conditions also increased the functional coupling of aIPS
with both LOtv and PMv.
These findings support the view that the dorsomedial stream is
automatically involved in processing visuospatial parameters for grasping
on the basis of current information, but irrespective of viewing conditions or
pictorial object characteristics. In contrast, the dorsolateral stream appears
to shape motor behaviour by integrating perceptual information processed
in the ventral stream into the grasp plan.
grasping perceptuo-motor interactions | 89
Introduction
Visual cortical processing is organized in functionally and anatomically separated
circuits 10,417, with object recognition and action guidance relying on processes and
representations distributed along two distinct pathways, the ventral and the dorsal
visual stream, respectively. Both streams originate in the striate cortex, but whereas
the ventral stream terminates in the anterior part of the temporal lobe, the dorsal
stream reaches into the superior part of the posterior parietal lobe36 . Subsequent
anatomical and functional studies have documented a further distinction between
dorsomedial and dorsolateral circuits within the parieto-frontal network86,198,199 .
It has been recently suggested that the relative contribution of these two circuits
is a function of the degree of online control required by a grasping movement 205 .
However, both the planning and control adaptive behaviour ultimately requires
the integration of perceptual and motor abilities, for instance when we adjust our
grasping movement to the ripeness of a tomato as judged by its colour. Here we
extend the scope of previous studies by investigating the cerebral mechanisms
supporting the integration of perceptual and motor abilities during the planning of
goal-oriented behaviour.
Recent findings on the role of binocular vision in visuomotor behaviour provide an
empirical possibility for experimental manipulations of this integration. Processing
depth information along the dorsal stream is particularly dependent on binocular
inputs, like stereoscopic disparity99,143 and vergence305,306 . For instance, in visual
agnosia patients, visuomotor performance is disturbed during monocular viewing of
a target301-303 . This suggests that, when binocular vision is unavailable, the extraction
of depth information (that is crucial for visuomotor control) relies on pictorial cues
like texture, illumination gradients, and perspective. These cues are processed along
the ventral visual stream 289,290,300 .
In the current study we biased visual processing towards either the dorsal or
the ventral stream by manipulating viewing conditions (monocular or binocular
vision) during the planning of a grasping movement. Both viewing conditions are
likely to engage both streams, but the ventral stream should increase its functional
integration with the dorsal stream as the relevance of pictorial cues of depth
information increases. Accordingly, we independently manipulated the relevance of
these cues by varying the slant of the object to be grasped. Increasing the object slant
will increases the importance of depth information provided by pictorial cues like
texture307,418 , while the relevance of binocular cues decreases 293 . We used this design
while measuring cerebral activity with functional magnetic resonance imaging
(fMRI) to test the hypothesis that grasping a horizontal cuboid under monocular
vision increases the functional couplings between the two visual streams as pictorial
cues become more relevant for motor planning.
5
90 | chapter 5
Methods
Participants
Nineteen healthy right-handed (Edinburgh Handedness Inventory403: 97 ± 8%; mean
± SD) adult (22 ± 2 years; mean ± SD) male volunteers were recruited as participants
and received a small financial reward. They all had normal or corrected-to-normal
visual acuity, stereoscopic depth discrimination thresholds at 60 arcsec or less (TNO
stereographs, Laméris Ootech BV, Nieuwegein, the Netherlands, 1972), and they
gave informed consent according to the institutional guidelines of the local ethics
committee (CMO region Arnhem-Nijmegen, the Netherlands).
Experimentalsetup
Subjects lay supine in the MR scanner. The standard mattress of the scanner bed was
removed to position the subject’s body lower within the bore of the scanner and to
aid in comfortably tilting the subject’s head forward by 30 degrees. In this way, the
subject had a direct line of sight of the object to grasp (figure 5.1A). An eight-channel
phased-array receiver head coil was tightly fitted to the subject’s head with foam
wedges. The subject’s right upper arm was immobilized with foam wedges and a
Velcro strap band across the lower chest, such that the forearm could comfortably
lie horizontally on the subject’s abdomen and allow for the grasping movements
performed in this study (see below). An optical response button box (MRI Devices,
Waukesha, WI, USA) was positioned on the left side of the abdomen and served as a
‘home-key’ in between trials. Using this button box, reaction times (RT, i.e. the time
interval from stimulus onset until the release of a home-key) and movement times
(MT, i.e. the time interval from the home-key release until the subsequent home-key
press) were measured on each trial during scanning.
The subject was asked to grasp a black right rectangular prism (6 x 4 x 2 cm,
‘target object’) along its longest dimension with the thumb and index finger. The
target object was positioned at one end of the scanner bore, along the subject’s
midsagittal plane, above the subject’s pelvis, by means of an adjustable polymer
and wooden frame (figure 5.1C). Behind the target object a white cloth served as
a homogeneous background for the object and to block the subject’s view of the
scanner room. The target object could be rotated around its pitch axis along the
subject’s sagittal plane, in steps of 30 degrees between 0 and 90 degrees from the
vertical plane, by means of a pulley system manually operated by an experimenter
standing inside the scanner room (figure 5.1E). Conductive wires on the object
surface (‘touch sensor’) detected changes in capacitance induced by the contact
g r a s p i n g p e r c e p t u o - m o t o r i n t e r a c t i o n s | 91
A
2 3
4
1
5
B
LED
vision
action
time
C
3-8 s
rest
D
~0.6 s ~0.6 s
~1 s
prepare grasp hold & return
E
2
3
4
1
Figure5.1 Experimental setup and trial time course
A. The subject was asked to perform visually-guided grasping movements while
lying supine in the MR scanner. The phased-array receiver head-coil (5) was tilted
forward by about 30 degrees allowing a direct line of sight of the target obejct. The
subject could comfortably move his hand from the home-key placed on the abdomen
(not shown) to the object (3, detail panel C) by rotating his forearm around the
elbow and his hand around the wrist. The experimenter could vary the orientation
of the object (from 0 to 90 degrees deviation from the vertical plane, in 4 steps of 30
degrees) by turning a wheel (1, detail panel E). MR-compatible liquid crystal shutter
glasses controlled the vision (3, panel D shows the glasses in the monocular-right
condition). During scanning the target object was located inside the MR-bore, while
the turning wheel was outside the bore (E). A white curtain (not shown) provided a
stable and homogeneous visual background for the subject and blocked sight of the
experimenter. B. Each trial started with a variable rest period (3-8 sec). During this
period the subject’s hand was resting on the home-key and the experimenter could
rotate the object in the instructed orientation for the upcoming trial. Opaque LCD
glasses blocked the vision, but two LEDs (2, detail panel C) provided an anchor for
fixation. At the end of this rest period, the LEDs turned off and either the left, right
or both shutters (4) became transparent, providing either monocular or binocular
vision of the target object. The subject was instructed to grasp the object using a
precision grip (index finger and thumb) as quickly and accurately as possible after
the target object became visible, holding it briefly before returning to the home-key
on their abdomen. When the subject’s hand released the home-key the shutters
closed. The touch between the subject’s fingers and the object was detected by a
capacitance based sensor. In the fMRI analysis the combined response and grasping
phases (indicated by the first grey block) were used to estimate the main effect of
the grasp phase. The following phases during which the subject held the object and
moved his hand back to the home-key (as indicated by the second grey block) was
used for main effect of the hold-return phase.
5
92 | chapter 5
between the subject’s finger and the target object. This measurement enabled us
to subdivide the whole movement period of each trial into two relevantly distinct
phases: the reach-to-grasp movement phase (MT, i.e. the time interval from release
of the home-key until contact of at least one finger on the object) and the holdreturn phase (HRT, i.e. the time interval from finger-object contact until return on
the home-key).
The subject’s vision was controlled by means of MR-compatible liquid crystal
shutter goggles (‘shutters’, Translucent Technologies, Toronto, Ontario, Canada).
The shutters could assume either a transparent (‘open’) or an opaque (‘closed’)
configuration (transition times (10-90%): open = 23 ms, close<0.5 ms), and they
were positioned in front of the left and the right eye of the subject by means of an
adjustable polymer frame, ensuring a consistent visual field of view across subjects
(figure 5.1D). The shutters covering the left and the right eye were independently
controlled and they allowed us to manipulate the type (i.e. monocular or binocular
vision) and the timing of visual information available to the subject. When the
shutters were closed, a pair of LEDs positioned on the left and on the right of the
object generated two bright spots on the shutter glasses, providing an anchor point
for the subject’s gaze (figure 5.1C). The LEDs were off when the LCD shutter glasses
were open. During this period the subjects planned the grasping movement. This
procedure ensured that the subjects fixated the target object343 . In addition, during
the training session they were instructed to focus on the target object. The LEDs
remained off during the execution of the movement when the shutter glasses were
closed. No explicit instructions were given to the subjects concerning eye movements
during this phase. After the return of the hand to the home-key the shutter glasses
remained closed, and the LEDs were turned on. Subjects were told that the LEDs
served as an anchor for their fixation, and that at the beginning of the next trial
the object would appear at the same location. A previous study revealed that this
procedure was important for minimizing head-motion artefacts 346 (see chapter 3 of
this thesis for details).
The shutters, the LEDs, the touch sensor, and the home-key were controlled by a
computer running Presentation software (version 10.1, Neurobehavioural Systems,
San Francisco, CA, USA) located outside the scanner room. Custom filters and fibre
optic cables were used for the connections.
Each trial started with the shutters closed, the LEDs on, and the subject’s
right hand on the home-key (figure 5.1B). During a pseudo-randomized interval
(3000-8000 ms, uniform distribution, steps of 1 ms) the target object was rotated
by the experimenter according to a pre-specified orientation, communicated to
the experimenter via MR-compatible headphones. At the end of this time interval,
the left, the right, or both shutters opened, thus allowing left monocular, right
monocular, or binocular vision of the target object, respectively. Concurrently, the
g r a s p in g p e r c e p t u o - m o t o r in t e r a c t i o n s | 93
LEDs were turned off. Subjects were instructed to grasp the object as quickly and
accurately as possible as soon as the target object became visible, briefly holding
it between their thumb and index finger before returning to the home-key. When
the subject left the home key, the shutters closed. When the subject returned to the
home-key, the LEDs were turned on and a new trial started. A session had a total
duration of about 30 minutes and consisted of 240 trials organized in a 3 x 4 design
(factor vision, with 3 levels: monocular left, monocular right, binocular; and factor
slant, with 4 levels: 0°, 30°, 60° or 90°). Subjects were instructed and trained inside
the scanner room until stable error-free performance was achieved
Behaviouralanalysis
The fMRI model collapsed RT and MT periods into a single regressor (grasp phase,
see Image analysis below), therefore we performed an analysis of variance on the
same temporal interval (i.e. the trial-by-trial sum of RT and MT) that considered the
factors vision (3 levels: monocular left, monocular right, binocular) and slant (4 levels:
0°, 30°, 60° or 90° deviation from the vertical plane) and their interaction (p<0.05).
To validate the reliability of the behavioural measurements collected in the
MR scanner and to illustrate the effect of object orientation on the variability of
the grasping movement we also performed additional kinematics measurements
on one subject. The experimental settings were identical to those used in the fMRI
experiment, but the measurements were performed outside the scanner in order
to be able to sample the position and orientation of the index finger and thumb
during task performance (miniBIRD tracking system, Ascension Technology Ltd.).
Movements were sampled at 103 Hz, with markers on the wrist, thumb and index
finger. Data was low-passed filtered (fourth order Butterworth filter at 10 Hz), and
the tangential speed of the three markers was calculated. Similarly to the experiment
in the MR scanner, the release of the home-key at the starting position marked the
onset of the movement while the touch sensor on the object marked the offset of
the grasping movement. End-point variance of the marker positions 30 ms before
the movement offset was expressed by the volume of 95% confidence ellipsoids 419 ,
calculated separately for each marker and each object orientation. No statistical
significance was inferred on these kinematics data as only one subject was measured
for validation purposes.
Imageacquisition
Images were acquired on a 3 Tesla Trio MRI system (Siemens, Erlangen, Germany),
using the body coil for radio frequency transmission, and an eight channel phased
array surface head coil for signal reception. BOLD (blood-oxygenation-level
5
94 | chapter 5
dependent) sensitive functional images were acquired using a single shot gradient
EPI sequence (TR/TE 2030/30 ms, 32 transversal slices, distance factor 17%,
effective voxel size 3.5 x 3.5 x 3.5 mm). Following the functional scan, high-resolution
anatomical images were acquired using an MP-RAGE GRAPPA sequence with an
acceleration factor of 2 (TR/TE/TI 2300/2.92/1100 ms, voxel size 1 x 1 x 1 mm, 192
sagittal slices, FoV 256 mm).
Imageanalysis
Imaging data were pre-processed and analysed using SPM5 (Statistical Parametric
Mapping, http://www.fil.ion.ucl.ac.uk/spm/). The first five volumes of each
participant’s dataset were discarded to allow for T1 equilibration. Prior to analysis,
the image time-series were spatially realigned using a least-squares approach that
estimates six rigid body transformations (three translations and three rotations) by
minimizing head-movements between each image and the reference image. The
time-series for each voxel were temporally realigned to the first slice in time to correct
for differences in slice time acquisition. Subsequently, images were normalized onto
a MNI-aligned EPI template using both linear and non-linear transformations and
resampled at an isotropic voxel size of 2 mm in a probabilistic generative model
that combines image registration, tissue classification and bias correction 406 . This
procedure also provided mean images of segmented grey matter, white matter and
cerebrospinal fluid. Finally, the normalized images were spatially smoothed using
an isotropic 10 mm full-width-at-half-maximum Gaussian kernel. Each participant’s
structural image was spatially coregistered to the mean of the functional images
and spatially normalized by using the same transformation matrix applied to the
functional images.
The fMRI time-series were analysed using an event-related approach in the
context of the General Linear Model, using standard multiple regression procedures.
For each trial, we considered two effects, modelled as square-wave functions:
grasp and hold-return phases. The grasp phase was taken as the time interval
from the opening of the shutters until the contact between the target object and
the subject’s hand – this effect relates to the preparation and the execution of the
grasping movement and it corresponds to the grasp interval (first grey block in figure
5.1B). We combined planning and execution phases in a single explanatory variable
because the current experimental setting does not allow us to disambiguate the
relative contributions of these movement phases. In this experiment, movement
planning was quickly followed by movement execution on each and every trial, and
the haemodynamic (BOLD) consequences of these two movement phases would be
indistinguishable (collinear). Previous studies have distinguished these movement
phases by introducing variable delays between instruction cues and movement
gra sping p erce ptuo -motor interac tions | 95
execution 420,421 , or by using a proportion of no-go trials 422,423 . In this experiment, we
opted not to do so in order to avoid transforming the task into a delayed grasping
protocol424 , or introducing motor inhibition effects 425 .
The hold-return phase was taken as the interval from the contact between
the target object and the subject’s hand, until the return of the subject’s hand on
the home-key (second grey block in figure 5.1B). The effect of the grasp phase
was partitioned in three conditions according to the factor vision (i.e. monocular
left, monocular right, and binocular trials). For each of these conditions, we also
considered the linear parametric effect of the factor slant (i.e. the deviation of the
target object from the vertical: 0°, 30°, 60° or 90°). On a subject-by-subject basis,
trials in which the sum of RT and MT was more than 3 times the standard deviation
from the mean (3 ± 2%; mean percentage of misses ± SD over all subjects) were
labelled as misses and included in a separate regressor of no interest. Each of the
resulting eight regressors [monocular left (main & parametric), monocular right
(main & parametric), binocular (main & parametric), hold-return (main), misses
(main)] were convolved with a canonical haemodynamic response function, and its
temporal and dispersion derivatives 407. The potential confounding effects of residual
head-movement related effects were modelled using both the original, the squared,
and the first order derivatives of the time-series of the estimated head movements
during scanning347. The intensity changes attributable to the movements of the arm
and hand through the magnetic field were accounted for by using the time-series
of the mean signal from the white matter and cerebrospinal fluid (chapter 4 of this
thesis). Finally, the fMRI time-series were high-pass filtered (cut-off 128s) to remove
low frequency confounds, such as scanner drifts. Temporal autocorrelation was
modelled as a first-order autoregressive process.
For each subject, linear contrasts pertaining to the six regressors of interest
(monocular left [main & parametric], monocular right [main & parametric], binocular
[main & parametric]) were calculated. Consistent effects across subjects were tested
using a random-effects analysis on these contrast images by means of a within
subject analysis of variance.
Statisticalinference
Statistical inference (p<0.05) was performed at the voxel-level, correcting for
multiple comparisons (false-discovery rate426) over the search volume. When
assessing the effects of movement (i.e. the null hypothesis that there was no effect
evoked by planning and executing the grasping movements), the effects of vision
(i.e. monocular vs. binocular and vice versa), and the effects of slant (i.e. common
effects of increasing slant deviation of the target object from the vertical plane
across both monocular and binocular viewing conditions) the search volume covered
5
96 | chapter 5
the whole brain. When assessing the effects of slant across vision (i.e. the differential
effects of increasing object slant between monocular and binocular conditions), the
search volume was limited to the combined volume of two representative sets of
cortical areas, crucially involved in goal-directed action and object recognition,
respectively. In the dorsal visual stream, the anterior part of the intraparietal sulcus
(aIPS) 72 and the rostro-ventral premotor region (PMv in humans, F5 in macaque) 207
are necessary for preshaping the hand during grasping 151,160,175 . In the ventral visual
stream, the lateral occipital complex (LOC) 291,427,428 plays a crucial role in object
recognition, including the perception of geometrical shape and volumetric features
of objects 429 and the integration of multimodal object features (lateral occipital
tactile-visual region: LOtv) 292,427. Using the WFU PickAtlas SPM5 toolbox 430 , we
drew three spherical volumes of interest (VOIs) with a radius of 8 mm and centred
at the following coordinates (MNI space): LOtv [-48 -64 -16] 292; aIPS [-42 -42 +48] 72;
PMv [-60 +16 +24] 207.
Latencyanddurationestimation
To explore differential orientation-related changes in the latency and duration of
the BOLD responses during monocular and binocular trials, we calculated temporal
shift and dispersion effects following the method of Henson and colleagues 431 . In
this framework, changes in the latency and duration of the BOLD response can be
indexed by relating the sign and magnitude of the partial derivative with respect
to time (temporal derivative, β2) and the partial derivative with respect to duration
(dispersion derivative, β3) of the haemodynamic response model 407 to the sign and
magnitude of the canonical component (β1). Positive values of β2 indicate an earlier
haemodynamic response, negative values of β2 indicate a later haemodynamic
response. Positive values of β3 indicate a shorter haemodynamic response, negative
values of β3 a longer haemodynamic response. However, the relationship between
the derivative/canonical ratios (β2/β1, β3/β1) and the actual latency of the BOLD
response is non-linear because the parameterization ignores high-order terms 431 .
Therefore, we transformed the derivative/canonical ratio using the (approximately)
sigmoidal logistic function 2C/(1+exp(D(βd/β1))-C (where C=1.78, D=3.10, β1 is the
parameter estimate for the canonical component, and βd is the parameter estimate
for either the temporal (β2) or dispersion (β3) partial derivative) 431 . This approach
normalizes condition-specific changes in BOLD latency and duration to the overall
amplitude of the corresponding BOLD response. Latency and dispersion maps, with
respect to the moment of stimulus presentation, were calculated for each subject,
separately for both monocular and binocular trials, as a function of the target
object orientation, generating six SPMs for each subject (analogously to the effects
described for the canonical response, see Image analysis above). After smoothing
these SPMs (10mm FWHM isotropic Gaussian kernel), we entered them into a
g r a s pin g p e r c e p t u o - m o t o r in t e r a c t io n s | 97
random-effects analysis of a within subject analysis of variance. We considered this
analysis as a post-hoc test of BOLD latency and duration on the effects revealed by
the main analysis of BOLD amplitudes. Therefore, we only assessed the significance
(p<0.05) of latency and dispersion effects on the peak voxels revealed in the main
analysis (see tables 5.1, 5.2 and 5.3).
Analysisoffunctionalcoupling(PPI)
We used the psychophysiological interaction (PPI) method to test for changes in
functional connectivity between the three VOIs described above432 . This method
makes inferences about regionally specific responses caused by the interaction
between an experimentally manipulated psychological factor and the physiological
activity measured in a given index area. The analysis was constructed to test for
differences in the regression slope of the activity in a set of target areas on the
activity in the index area, depending on the viewing condition (monocular left,
monocular right or binocular) and on the object slant. For each subject and for each
VOI, the physiological activity of the index area was defined by the first eigenvariate
of the time-series of all voxels within the volumes of interest that showed activation
in response to our task (p<0.05 uncorrected). For each VOI, statistical inference was
performed at the voxel level over a search volume defined by the combined volumes
of the two remaining VOIs (p<0.05, corrected with false-discovery rate426).
Anatomicalinference
Anatomical details of significant signal changes were obtained by superimposing the
relevant SPMs on the structural images of the subjects. The atlas of Duvernoy and
colleagues 433 was used to identify relevant anatomical landmarks. When applicable,
Brodmann Areas (BA) were assigned on the basis of the SPM Anatomy Toolbox 434 .
When the literature used for VOI selection reported the stereotactic coordinates
in Talairach space, these coordinates were converted to coordinates in MNI space
by a non-linear transform of Talairach to MNI (http://imaging.mrc-cbu.cam.ac.uk/
imaging/MniTalairach).
5
98 | chapter 5
Results
Behaviouralresults
Subjects’ grasp phase (sum of RT and MT) was influenced by the viewing conditions
(F(2,36)=10.88, p<0.001), being longer in the monocular trials than in the binocular
trials (post-hoc comparison of means: F(1,18)=10.67, p=0.004, Bonferroni corrected;
figure 5.2A), but there was no effect of object slant nor an interaction between
vision and slant (F(3,54)=0.18, p=0.911; F(6,108)=0.38, p=0.889; figure 5.2A). Figure 5.2B
shows the volume of the 95% confidence ellipsoid describing the end-point positions
of the index finger, thumb and wrist 30 ms before the end of the movement in one
subject. The end-point variance of the index finger increases with increasing object
slant irrespective of viewing condition, but this relationship does not hold for the
wrist and thumb.
Cerebraleffects–BOLDamplitude
1400
monocular
binocular
1300
1200
B
95% CI volume (cm3)
A
grasp phase duration (ms)
Planning and executing visually-guided grasping movements with the right hand
evoked activity in visual cortex (bilaterally) and in the dorsolateral portions of
left parietal and frontal cortex (table 5.1). Figure 5.3A illustrates the anatomical
200
150
index
thumb
wrist
100
50
0
1100
object slant
object slant
Figure5.2 Behavioural effects of object slant and vision
A. Collapsed reaction and movement times (mean ± SE over subjects) were longer
in trials with monocular (green) than in trials with binocular viewing conditions
(red), but did not change as a function of object deviation from the vertical plane
(0°, 30°, 60° or 90°). B. Volumes of the 95% confidence ellipsoids describing the
end-point positions (taken 30 ms before the first touch with the object) of the index
finger, thumb and wrist as a function of object slant. The end-point variance was not
significantly different for monocular and binocular trials.
g r a s p i n g p e r c e p t u o - m o t o r i n t e r a c t i o n s | 99
A
y = -24
16
M1 [-34 -24 62]
12
8
4
monocular
binocular
0
C
contrast estimate (SEM)
contrast estimate (SEM)
B
x = -12
object slant
10
V2 [-12 -96 14]
8
6
4
2
0
object slant
Figure5.3 Cerebral effects – object vision
A. Spatial distribution of cerebral activity related to planning and executing visuallyguided grasping movements with the right hand. Colours from red to yellow show
the common activity across monocular and binocular viewing conditions, whereas
colours from blue to green show the distribution of differential activity of binocular
versus monocular viewing conditions. There were no significant differential effects
when contrasting monocular with binocular conditions. The left side of the image
represents the left side of the brain (neurological convention). The image shows
the relevant SPM{t}s (p<0.05 false-discovery rate corrected over the whole brain,
random effect analysis, only voxels within 16 mm from the cortical surface are
displayed) superimposed on a rendered brain surface, a coronal section and a
sagittal section of a structural T1 image. The prehension task evoked activity in
visual cortex (bilaterally) and in the dorsolateral portions of left parietal and frontal
cortex, as well as in the occipito-temporal region bilaterally, and in the right anterior
parietal cortex, in the inferior parietal lobule (table 5.1). Binocular viewing induced
more activity compared to monocular viewing in striate and peri-striate cortex. B-C.
Cerebral responses over precentral (M1) and peri-striate cortex (V2), respectively
(numbers in square brackets indicate MNI stereotactic coordinates for the local
maxima, see table 5.1). The graphs show parameter estimates (in SEM units) of the
cerebral responses evoked by the prehension task over object orientations for the
monocular (green) and binocular (red) viewing conditions separately.
5
10 0 | chapter 5
A
z = 50
2
sPOS [-18 -72 50]
1
4
0
-1
2
β2
β3
β1
-2
0
-2
C
contrast estimate (SEM)
contrast estimate (SEM)
B
x = -20
object slant
2
PMd [-22 -4 56]
1
6
-1
4
β2
0
β1
β3
-2
2
0
object slant
Figure5.4 Cerebral effects – object slant
A. Cerebral activity related to the prehension task increased as a function of
increasing slant of the target object in occipital cortex (bilaterally), along the
parieto-occipital sulcus and in the anterior part of the dorsal precentral gyrus (table
5.2). The image shows the relevant SPM{t} (p<0.05 false-discovery rate corrected)
superimposed on a rendered brain surface, and an axial and a sagittal section. B-C.
Cerebral responses over the superior parieto-occipital sulcus (sPOS) and the anterior
part of the superior precentral gyrus (PMd). The graphs show parameter estimates
(in SEM units) for the effect evoked by the prehension task over object orientations
for both viewing conditions separately. In these regions, the cerebral effects of the
monocular (green) and binocular (red) viewing condition were comparable, but
a strong effect of increasing object slant on cerebral activity can be seen in sPOS
and PMd, independent of viewing conditions. The histograms (insets) represent the
differential BOLD amplitude (β1), latency (β2), and duration (β3) relative to the slant
x vision interaction. More specifically, they characterize the differential cerebral
activity between monocular and binocular viewing conditions, increasing with
object slant. The histograms show that there is no significant difference in the effect
of increasing object slant between monocular and binocular viewing conditions in
sPOS and PMd, neither in BOLD amplitude (β1), latency (β2), nor duration (β3).
Other conventions as in figure 5.3.
grasping perceptuo -motor interac tions | 101
distribution of cerebral activity evoked during both monocular and binocular viewing
conditions (in red-yellow). There were also significant effects in the cerebellum
(bilaterally), the occipito-temporal region (bilaterally), and in the right superior
parietal lobule (table 5.1).
There were also robust differential effects between binocular and monocular
viewing conditions, namely in striate and peri-striate regions (figure 5.3A in bluegreen, table 5.1). Large parts of this cluster overlap with V1 (30%) and V2 (23%). The
local maxima of these responses fall within the 80% probability range of area V1 and
V2 435 . Given the anatomical location of this effect, it appears more likely to be related
to the difference in viewing conditions than differences in planning and/or execution
of the movements. The effect fits with the predominance of neurons sensitive
to stereoscopic visual input in the striate cortex 436,437. There were no significant
differential effects when contrasting monocular with binocular conditions.
Grasping movements towards a target object which slant deviates progressively
from the vertical plane increased cerebral activity along the parieto-occipital sulcus
(sPOS) and in the anterior part of the dorsal precentral gyrus (PMd) (figure 5.4, table
5.2). It might be argued that these effects could be driven by residual betweensubjects behavioural differences in planning and/or execution of the movements.
To address this issue, we assessed the subject-by-subject relationship between
the magnitude of cerebral and behavioural effects. For each subject, we calculated
the parameter estimates of the BOLD signal evoked in sPOS and PMd (and also in
LOtv, aIPS, PMv), separately for monocular and binocular viewing conditions, and
irrespective of object slant. The behavioural effects we considered were the mean
RT, MT and grasp phase time. Regression analysis showed that the cerebral (BOLD)
variance in those regions could not be explained by the behavioural effects. The
same outcome emerged when considering the relationship between cerebral and
behavioural changes as a function of object slant. These results indicate that neither
the main nor the parametric effect of viewing condition on brain activity could be
explained by timing differences in planning and executing the movement.
This experiment used different viewing conditions (monocular, binocular) to
probe differences in the cerebral responses evoked during the grasping of an object
at different slants (slant x vision interaction). We expected that cortical regions
involved in processing object slant on the basis of pictorial cues in relation to motor
planning would increase their activity more strongly across object orientations in the
monocular than in the binocular condition. The left LOtv, a cortical region known to
be important for object perception, showed this response pattern (figure 5.5, table
5.3). Similar and significant response patterns were also found in two regions of the
dorsolateral visuomotor stream, aIPS and PMv (figure 5.5, table 5.3). The absence of
behavioural differences in the slant x vision interaction makes it unlikely that these
cerebral effects are a consequence of behavioural differences.
5
102 | chapter 5
Cerebraleffects–BOLDlatencyandduration
We performed a post-hoc test on the BOLD latency and duration of the local maxima
revealed in the main analysis. The results are shown in the histograms (insets) of
figures 5.4 and 5.5. These histograms illustrate the differential BOLD amplitude (β1),
latency (β2), and duration (β3) relative to the slant x vision interaction. Accordingly,
the significantly positive β1 values observed in LOtv, aIPS, PMv (figure 5.5B-C) index
the effects detailed in the associated scatter plots, namely that the parametric
effect of object slant is stronger in the monocular than the binocular condition.
Analogously, the β1 values observed in sPOS and PMd are not significantly different
from zero, indicating the absence of a significant slant x vision interaction (figure
5.4B-C). Crucially, the BOLD latency and duration analyses indicate that the slant
x vision interaction evokes earlier BOLD responses in PMv (positive β2), and longer
BOLD responses in aIPS and PMv (negative β3). Temporal modulation of the BOLD
response as a function of viewing conditions was absent in sPOS and PMd (figure
5.4B-C).
Cerebraleffects–functionalcoupling
We assessed the functional relevance of the changes in BOLD signal by testing
whether LOtv, aIPS, and PMv increased their inter-regional couplings as processing
object slant on the basis of pictorial cues became increasingly relevant for accurate
motor planning. We found increased couplings between aIPS and both LOtv
(p=0.017) and PMv (p=0.020) as a function of object slant during the monocular trials
(as compared to the binocular trials).
Discussion
The current study investigated cerebral processes supporting the integration of
perceptual and motor processes by asking subjects to reach and grasp an object
under binocular or monocular viewing conditions. In the latter condition, pictorial
cues of depth information became more relevant for planning an appropriate
grasping movement as the object’s slant increased. Under these circumstances,
the anterior intraparietal region (aIPS) increased its functional coupling with both
frontal premotor regions (PMv) and occipito-temporal perceptual areas (LOtv). In
the following sections we elaborate on the implications of these findings for models
of the neural control of grasping movements.
grasping perceptuo -motor interac tions | 103
B
contrast estimate (SEM)
A
4
6
4
2
0
-2
earlier
β3
β1
β2
longer
2
0
object slant
6
4
monocular
binocular
aIPS [-42 -44 42]
2
0
β2
β3
β1
longer
-2
2
0
D
PMv [-58 10 28]
contrast estimate (SEM)
contrast estimate (SEM)
C
4
4
6
4
LOtv [-54 -62 -14]
2
0
-2
β3
β1
β2
2
0
object slant
Figure 5.5 Cerebral effects – perceptuo-motor interactions
A. Spatial distribution of the differential cerebral activity increasing as a function
of object slant between monocular and binocular viewing conditions. The image
shows the relevant SPM{t} (p<0.05 uncorrected for display purposes) superimposed
on a rendered brain surface (see table 5.3). There were no significant differential
effects when contrasting the parametric effects of increasing object slant under
binocular viewing conditions with the parametric effects under monocular viewing
conditions. B-D. Cerebral responses over the anterior inferior intraparietal sulcus
(aIPS), the inferior precentral gyrus (PMv) and the inferior occipital gyrus (LOtv).
The graphs show parameter estimates of the cerebral responses evoked by the
prehension task over different object orientations for the monocular (green) and
binocular (red) viewing conditions separately. The histograms (insets) illustrate
the differential BOLD amplitude (β1), latency (β2), and duration (β3) relative to the
slant x vision interaction, i.e. the differential cerebral activity between monocular
and binocular viewing conditions, increasing with object slant (as in figure 5.4).
It can be seen that in aIPS, PMv, and LOtv the parametric effect of object slant is
stronger in the monocular than the binocular condition (positive β1). In addition, in
PMv the BOLD response starts earlier in time (positive β2) and that in aIPS and PMv
the BOLD response is extended in time (negative β3) under monocular compared to
binocular viewing conditions with increasing object slant.
5
visual cortex
V5
cuneus (BA 17-18)
middle occipital gyrus
sup. occipital gyrus (BA17-18)
V1-V2
V1-V3
lingual gyrus
sup. parietal lobule (BA 2)
binocular > monocular viewing
motor cortex
somatosensory
lobules IV-VI
functionalregion
sup. precentral gyrus (BA 4)
sup. postcentral gyrus (BA1-3)
cerebellum
monocular and binocular viewing
anatomicalregion
left
right
left
left
right
left
right
left
left, right
left
right
right
4.70
4.42
12.96
12.86
12.16
8.72
11.77
6.43
9.57
7.18
5.04
5.94
t-value
0.015
<0.001
p-value
4111
46668
voxels
-12
12
-34
-38
28
-34
10
-14
6
-48
44
36
-96
-92
-24
-36
-54
-56
-58
-68
-90
-70
-74
-46
14
22
62
60
-20
-24
-8
2
22
4
14
62
localmaximum
maximum is given. Cluster size is given in number of voxels.
Statistical inference (p<0.05) was performed at the voxel level,
correcting for multiple comparisons using the false-discovery
rate approach over the search volume. V1-5: visual areas 1-5.
hemisphere
MNI stereotactic coordinates of the local maxima of regions
showing an effect of reaching and grasping with the right
hand (both the common and differential effects of monocular
and binocular viewing conditions are listed). For large clusters
spanning several anatomical regions more than one local
Table5.1 Common and differential cerebral activity evoked during monocular and binocular trials
10 4 | chapter 5
hV6A
V5
V4
V2
PMd
parieto-occipital sulcus (sPOS)
middle occipital gyrus
posterior inferior occipital gyrus
middle occipital gyrus
ant. sup. precentral gyrus (BA 6)
left
left
left
right
left
hemisphere
6057
1085
639
<0.001
0.007
0.001
6.05
4.62
4.35
4.04
6.01
voxels
p-value
t-value
-18
-44
-50
20
-22
-72
-72
-68
-102
-4
50
8
-16
4
56
localmaximum
slant. hV6A: human homologue of anterior visual area 6; PMd:
dorsal premotor cortex. Other conventions as in table 5.1.
functionalregion
LOtv
PMv
hAIP
anatomicalregion
inferior occipital gyrus
inferior precentral gyrus (BA 44)
anterior intraparietal sulcus (aIPS)
left
left
left
p-value
0.049
0.049
0.049
t-value
3.07
3.02
2.71
105
80
208
voxels
-54
-58
-42
-62
10
-44
-14
28
42
localmaximum
occipital tactile-visual region; PMv: ventral premotor cortex;
hAIP: human homologue of anterior intraparietal area. Other
conventions as in table 5.1.
hemisphere
MNI stereotactic coordinates of the local maxima of regions
showing a positive interaction between viewing conditions
(monocular, binocular) and increasing object slant. LOtv: lateral
Table5.3 Differential cerebral activity between monocular and binocular viewing conditions as a function of object slant
functionalregion
anatomicalregion
MNI stereotactic coordinates of the local maxima of regions
showing cerebral activity that increased as a function of object
Table5.2 Cerebral activity increasing as a function of object slant, independent of viewing conditions
grasping perceptuo -motor interac tions | 105
5
10 6 | chapter 5
Behaviouraleffects
During MR-scanning, subjects performed an ecologically relevant prehension task,
i.e. with a direct line of sight of the target object, without mismatches between
visual target information and proprioceptive input related to the moving arm. The
grasping movement was performed without online visual feedback, i.e. differences
between viewing conditions were confined to the presentation of the target object
prior to the start of the movement. This difference in the initial viewing conditions
influenced subjects’ performance, with longer planning and execution times when
only monocular object cues were available (figure 5.2A). This result confirms previous
observations 296,299,438 , and corroborate the sensitivity of the behavioural measures
acquired during MR-scanning. Crucially, the duration of the grasp phase (RT + MT)
was not influenced by the object slant, excluding the possibility that orientationrelated cerebral effects could be a by-product of behavioural differences. This
result might appear at odds with previous studies reporting increasing movement
times with increasing slant when subjects viewed the object during grasping,
with movement time ending when both fingers stopped moving 314,439 . However,
experimental differences may account for this: the current setting prevented online
visual feedback, and the first contact of either thumb or index finger with the object
defined the end of the movement. Accordingly, the effect of increasing object slant
becomes evident in the end-point variance of the grasping movement (irrespective
of viewing conditions, figure 5.2B), suggesting that computational demands for
specifying the appropriate movement parameters towards slanted objects were
increased.
Cerebraleffects
Reaching and grasping a rectangular prism with the right hand increased metabolic
activity in large portions of occipito-temporal (bilaterally) and parieto-frontal regions
(mainly left hemisphere, figure 5.3). Within these spatially distributed responses,
activity in the medial parieto-occipital sulcus (sPOS, putative human homologue
of area V6A (hV6A)) 110,111,440 and in the superior precentral gyrus (putative area
PMd) 207 increased proportionally to the slant of the object, irrespective of viewing
conditions (figure 5.4). The effect is unlikely to be a by-product of longer periods of
motor output, somatosensory feedback, or visual inspection of the object evoked
by grasping more horizontal objects, since the orientation-related changes in sPOS
and PMd activity were not associated with changes in grasp phase duration or with
lengthening of the BOLD response (figure 5.4B-C). Rather, given that the end-point
variance of the index finger increased with increasing object slant irrespective of
viewing conditions (figure 5.2B), we suggest that the orientation-related increases
in sPOS and PMd activity could result from the increased computational demands
grasping perceptuo -motor interac tions | 107
of specifying appropriate grasping parameters with greater predicted variability of
the movement. This hypothesis fits with the general notion that sPOS and PMd, two
crucial nodes of the dorsomedial visuomotor stream 60,198 , are involved in processing
visuospatial information for visual control of grasping movements 107,205,441 . These
two regions are known to be critically involved in encoding the spatial location of a
movement target31,67 and to integrate target- and effector-related information 25,442 .
The present findings provide further support for the hypothesis that the dorsomedial
visuomotor stream is involved in the specification of spatial parameters for grasping
movements on the basis of visual information acquired before response onset31,205 ,
irrespective of viewing conditions or target characteristics.
The activity pattern of sPOS and PMd can be contrasted with the responses
found in aIPS (putative human homologue of area AIP, hAIP) 4,152 and PMv (both
part of the dorsolateral visuomotor stream 2), and in LOtv (part of the ventral visual
stream; figure 5.5). Four effects were evident. First, these three regions increased
their activity when a combination of monocular viewing conditions and object
slant increased the relevance of pictorial cues to determine the orientation of the
object to be grasped. This slant x vision interaction did not evoke corresponding
behavioural differences (figure 5.2), and the subject-by-subject variability in
planning and execution time did not account for variations in the activity of these
three regions. Second, during binocular vision, while aIPS and PMv decreased their
activity with increasing object slant (p<0.05; figure 5.5B-C), LOtv showed a stable
level of activity across object orientations (figure 5.5D). Third, aIPS increased its
functional coupling with both PMv and LOtv during monocular trials as the slant of
the target object increased. Fourth, both PMv and aIPS BOLD responses extended
over longer portions of monocular trials as the object slant increased (figure 5.5BC). These temporal modulations were regionally specific (being absent in sPOS and
PMd, figure 5.4B-C), differential in nature (being relative to the interaction between
viewing conditions and object slant), and occurred over and above changes in BOLD
magnitude (see Methods). Therefore, they cannot be accounted for by generic
differences in neurovascular coupling between cerebral regions, or by-products of
increased BOLD responses 443 .
These findings suggest a specific scenario of perceptuo-motor interactions
involving the dorsomedial, the dorsolateral, and the ventral visual streams. Namely,
when grasping movements are organized on the basis of depth information
obtained by processing pictorial cues (i.e. monocular trials with slanted objects), the
dorsolateral stream enhances its coupling with the ventral stream, increases and
lengthens its contribution to the visuomotor process, presumably by strengthening
its intrinsic inter-regional connectivity. In other words, when a high degree of
visuomotor precision is required, but online visual feedback is absent and perceptual
information is necessary for planning a correct movement, then the dorsolateral
5
108 | chapter 5
stream appears to support the visuomotor process on the basis of perceptual
information provided by the ventral stream. In contrast, when depth information
is less relevant for organizing grasping movements (i.e. trials with near-vertical
objects), or when it can be obtained by binocular cues (i.e. stereoscopic disparity
or vergence), then the contributions of the dorsolateral stream to the sensorimotor
transformation are reduced, and the perceptual processes in the ventral stream
are not involved in organizing the grasping movement. Under these circumstances
the visuomotor process could be driven by the dorsomedial stream relying on
information accumulated immediately before motor execution.
Interpretationalissues
In this study we have prevented online visual control during action performance, and
separated the effects driven by the grasping movements from those evoked by the
stereotypical return movement to the home-key. The lack of online visual feedback
might have increased LOtv activity, given that this region is particularly responsive
during delayed actions72,444 . However, the lack of online visual feedback cannot
explain the differential responses found in LOtv, given that online visual feedback
was absent during both monocular and binocular trials.
The current study provides specific information for understanding the influence
of perceptual processes on the motor system during the initial stages of sensorimotor
transformations. However, this focus might also limit the relevance of this study
for models of online action control. It remains to be seen whether the reported
interactions between perceptual and visuomotor processes are specifically evoked
by the online control component of visually-guided movements 445 .
In addition to task-related changes in BOLD signals, we have also explored
changes in inter-regional couplings among a-priori defined regions that are important
for prehensile behaviour72,207,292 . Accordingly, the scope of the results is limited by
the exploratory nature of the present connectivity analysis 432 . For instance, it is
conceivable that LOtv could influence the dorsolateral visuomotor stream not only
through aIPS, but also via the ventral prefrontal cortex 199,446-448 . This issue could be
addressed by using more sophisticated and hypothesis-driven models of effective
connectivity241 .
Conclusion
We have assessed the cerebral processes underlying perceptuo-motor interactions
during grasping movements directed towards objects at different slants and
viewed under either monocular or binocular conditions. We found that dorsomedial
parieto-frontal regions (sPOS, PMd) were involved in the grasping movements
grasping perceptuo -motor interac tions | 109
irrespective of viewing conditions. In contrast, perceptual information processed in
the ventral stream (LOtv) influenced the visuomotor process through dorsolateral
parieto-frontal regions (aIPS, PMv). These results point to different functional roles
of dorsomedial and dorsolateral parieto-frontal circuits. The latter circuit might
provide a privileged computational ground for incorporating perceptual information
into a sensorimotor transformation, whereas the dorsomedial circuit might support
motor planning on the basis of current visuospatial information.
5
6
The anterior intraparietal
sulcus structures a
grasp plan, integrating
metric and perceptual
information
112 | c h a p t e r 6
Abstract
The control of prehensile behaviour by visuospatial parameterization is
one of the most well-studied functions of the primate brain. Computations
implemented in the anterior intraparietal sulcus (aIPS) play a privileged role,
contributing to the guidance of finger movements according to intrinsic
object metrics like size and shape. However, metric coding of an object
alone is not sufficient to understand how we pluck a fruit and adjust our
grip to its colour and identity, or when we use familiar objects, including
tools. In addition to the well-known role of aIPS in processing metric object
features during grasping, we hypothesized that this region is also necessary
for incorporating perceptual information – such as pictorial and semantic
cues – into a motor plan. We tested this hypothesis by manipulating the
availability and reliability of stereoscopic and pictorial depth cues during
motor preparation, while we transiently perturbed neuronal processing
in aIPS using transcranial magnetic stimulation. We measured both
the cerebral responses immediately following the stimulation using
electroencephalography and kinematically tracked the movements of the
hand and fingers. We found that the contributions of aIPS are necessary
during the early planning stage, concerned with defining the structure of the
grasping movement on the basis of knowledge of the object configuration,
and associated with both metric and perceptual cues. These findings open
the way for a better understanding of the role of aIPS in apparently disparate
functions like grasping, online control and tool-use.
a I P S s t r u c t u r e s a g r a s p p l a n | 113
Introduction
Visually-guided grasping movements are central to the behaviour of most primates,
and illustrative of how sensory input is transformed into metric parameters for
guiding our hand and fingers towards an object 1,2 . In macaques, the cerebral
circuits supporting those transformations are organized in a dorsomedial pathway
comprising the superior parietal lobe and the dorsal premotor area (PMd), and a
dorsolateral pathway encompassing parts of the intraparietal sulcus (IPS) and the
ventral premotor area (PMv) 86,106,198 . An influential interpretation of these anatomical
circuits has linked the dorsomedial circuit with guidance of the arm according to
the spatial location of a grasped object 4,113 . In contrast, the dorsolateral circuit is
thought to guide finger movements according to metrics of intrinsic object features
like size and shape40,98,149 . Yet, spatial parameterization of a grasping movement is
not sufficient to understand how we grasp. For instance, when adjusting our grip
while plucking a fruit according to its colour and identity, we also need to transform
perceptual information, such as pictorial and semantic cues, into motor codes 26,114 .
Here, we test the hypothesis that the anterior part of the human intraparietal
sulcus (aIPS) plays a causal role in incorporating metric, pictorial and semantic
cues into a motor plan for visually guided grasping. This hypothesis is grounded
on macaque and human functional neuroanatomy. There are mono-synaptic
connections between AIP and occipito-temporal cortex involved in perceptual
processing 204,236 , and stronger inter-regional coupling between aIPS and the
lateral occipital complex when perceptual information is integrated into motor
commands (chapter 5 of this thesis). We tested this hypothesis by quantifying
electrophysiological and kinematic consequences of single-pulse transcranial
magnetic stimulation (TMS) to aIPS while subjects prepared and executed grasping
movements. We focused on movement preparation, a stage when pictorial and
semantic cues could still influence the evolving structure of the movement 29 , and we
manipulated those influences in two ways. First, we exploited the fact that, as the
slant of a grasped object increases from vertical to horizontal, pictorial cues of depth
(e.g. texture, shading) become more relevant to estimate the object’s slant, whereas
the information value of stereoscopic disparity cues decreases 294 . Accordingly, we
asked subjects to grasp horizontally slanted objects during monocular vision, a
condition where depth estimation relies on pictorial cues processed in the temporal
lobe38,300 ; or to grasp vertically slanted objects during binocular vision, a condition
where the most informative depth information is provided by stereoscopic cues,
processed in the parieto-occipital cortex37,286,287. Second, we assessed whether, as
subjects build their knowledge of the grasped object across trials, this factor might
become a dominant prior in organizing the grasping movement and interfere with
the evidence provided on each trial by both stereoscopic and pictorial cues.
6
114 | c h a p t e r 6
Methods
Subjects
Thirty healthy, right handed subjects (age: 21 ± 2 (mean ± SD); 9 males; handedness 403:
89 ± 11 (mean ± SD)) participated in the experiment. All participants conformed to
standard EEG and TMS exclusion criteria (including neither a personal history of
neurological or psychiatric disorders, epilepsy, migraine, or cardiac arrhythmia, nor
in close relatives). All subjects had normal or corrected to normal vision and had a
stereoscopic disparity threshold of at least 120 arcminute (49 ± 29 arcminute (mean
± SD), TNO stereographs; Laméris Ootech BV, Nieuwegein, the Netherlands). The
study was approved by the local ethics committee and a written informed consent
was obtained prior to the start of the experiment according to the declaration of
Helsinki. Two subjects were excluded from analyses because their mean reaction
times were more than 1.5 times the inter-quartile range (IQR) below the first quartile
of the group. Four additional subjects were excluded from the electrophysiological
analyses because of high electrode impedances, muscle artefacts, or eye blink
artefacts.
Experimentalsetup
Subjects were seated at a table, wearing earplugs, with their head stabilized on a chin
rest. Their right hand was resting on a button box, pressing down the button marking
the starting position (home-key). Their eyes were facing two mechanical shutters
that could be opened and closed independently in 3.4 ms (figure 6.1A). A black prism
(6 x 6 x 2 cm) serving as a target object was positioned along the midsagittal plane
in front of the subject. It was displayed against a white background at a comfortable
reaching distance (~25 cm in front and ~30 cm above the starting position of the right
hand) and spanned a visual angle of ~7° when oriented vertically (figure 6.1A, 6.2A).
Experimentaltask
Subjects were asked to grasp the object with their right hand, slide it from its support,
hold it in mid-air, place it back into its support, and return to press the home-key
(figure 6.1B). The subjects were instructed to perform the task accurately. Before
trial onset, subjects held their right hand on the home-key, while closed shutters
occluded their vision. After a random interval (2-4 s), in which the object could be
rotated to one of seven slants (0°, 15°,..., 90° rotating away from the frontal plane),
either one or two shutters opened. This allowed monocular or binocular vision of the
a I P S s t r u c t u r e s a g r a s p p l a n | 115
A experimental setup
B trial timecourse
bino
shutters
vision
mono
object
action
home-key
TMS
time
2-4 s
rest
C experimental factors
vision
aIPS
TMS site
monocular
~1 s
grasp
~2.5 s
hold & return
D TMS sites
vertex
binocular
~0.7 s
planning
vertex
(control)
aIPS
line of sight
vertical
horizontal
slant
Figure6.1 Experimental setup and design
A-B. The subject grasped a target object (6 x 6 x 2 cm). At trial onset the subject was
resting with the right hand on a home-key, the head stabilized by a chin-rest, and
vision occluded by shutters. After an interval (2-4 s) one or both shutters opened,
providing either monocular or binocular vision of the target object, positioned in
front of the subject. The object could be oriented at 7 different slants in depth, from
vertical to horizontal relative to the fronto-parallel plane [0°:15°:90°]. During the
planning phase a single TMS pulse was delivered between either 100-200 ms or 300400 ms after stimulus presentation, constituting an early and late TMS-time window
(indicated on the trial time course with grey blocks). Visual feedback was withheld
during movement execution. C. The experimental design considered three factors:
object vision (2 levels: binocular vision, in red; monocular vision, in green); object
slant (2 levels, vertical slants [0°, 15°, 30°, in blue]; horizontal slants [60°, 75°, 90°,
in orange]); TMS-site (2 levels: aIPS, in magenta; vertex, in grey). D. Projection of
the focus of stimulation of the TMS coil onto the brain surface of one representative
subject, illustrating the two intended TMS targets: the anterior intraparietal cortex
(magenta, aIPS) and the vertex of the head (grey), serving as a control site.
6
116 | c h a p t e r 6
object to be grasped. The shutters closed as soon as the subjects left the home-key
to grasp the object. The absence of visual feedback during movement execution,
and the requirement to hold the object against gravity, ensured that the subjects
prepared an accurate grasping movement.
A trial lasted between 6 and 11 s (average duration: ~8 s). Subjects performed
4 blocks of 84 trials each (~11 min per block), a total of 336 trials for the whole
experiment (~60 min including short breaks between each block). Two additional
blocks (with 2 x 84 = 168 trials) were also collected in the same cohort of subjects;
these data are described in chapter 7.
Experimentalintervention–TMS
On each trial, we delivered a single TMS pulse at a randomized time within two
temporal windows (early: 100-200 ms after trial onset; late: 300-400 ms after trial
onset). Mono-phasic TMS pulses were delivered through a figure-of-eight coil (70
mm diameter), connected to a Magstim MonoPulse machine (Magstim Company,
Whitland, Wales, UK) at 100% of active motor threshold (AMT). The motor
threshold was determined in both a resting (RMT) and active state. During RMT
determination, subjects rested their hand on a pillow, while for AMT subjects were
asked to keep their first dorsal interosseous muscle (FDI) continuously contracted
at 15% of their individual maximum voluntary contraction, as measured with
standard electromyogram (EMG) recordings (10 KHz). RMT and AMT were defined
as the minimum stimulation intensity over contralateral motor cortex that elicited
a motor-evoked potential (MEP) in the FDI muscle higher than 50 μV and 200 μV
peak-to-peak, respectively, in at least 5 out of 10 successive stimulations 449 . Average
RMT and AMT were 43 ± 7% and 33 ± 7% of maximum stimulator output (MSO),
respectively.
During each of the four blocks of trials, TMS was applied over either the site of
interest (aIPS), or a site controlling for non-specific TMS effects (the vertex of the
head). The order of TMS sites was counterbalanced within and between subjects:
in half of the subjects the two TMS sites were ordered over the four blocks in an
A-B-B-A order, in the other half in a B-A-A-B order. Continuous online stereotactic
guidance of the TMS coil was incompatible with the experimental setup, therefore
the following offline procedure was employed to localize the aIPS site. First, we
acquired structural MRI scans of six pilot subjects (different from the main cohort
tested in this study). Second, we transformed the stereotactic coordinates of aIPS
(as reported in chapter 5 of this thesis) into the native space of each subject (using
an iterative unified normalization and segmentation procedure in SPM5, Statistical
Parametric Mapping; http://www.fil.ion.ucl.ac.uk/spm/) 406 . Third, using a frameless
stereotactic neuronavigation system (Brainsight, Rogue Research Inc, Montreal,
a I P S s t r u c t u r e s a g r a s p p l a n | 117
Quebec, Canada), the TMS coil was positioned over the location on the subject’s head
most closely corresponding to the subject-specific aIPS coordinates (figure 6.1D).
This procedure was performed while the subjects wore an electroencephalography
(EEG) cap, and it revealed that the inter-subject variance in coil position for optimal
aIPS stimulation was within 7 mm. Furthermore, the subject-specific optimal
coil position had a consistent spatial relation with the 10-10 system for electrode
positioning 450,451 , falling close to electrode CP3. Therefore, we mapped this site
derived from these 6 subjects on the EEG caps configured according to the 10-10
system and used this mapped location to position and orient the TMS coil for aIPS
stimulation during the experiment. The vertex control site was defined as the point
where both the sagittal midline from nasion to inion and the coronal line from ear to
ear were dissected in the middle (figure 6.1D).
Experimentaldesign
We considered three experimental factors: (1) TMS-site (2 levels: control, aIPS), (2)
object slant (2 levels: vertical [0°, 15°, 30°], horizontal [60°, 75°, 90°]), and (3) object
vision (2 levels: binocular, monocular) (Figure 6.1C). This experimental design tests
for biases in visual information processing through the interaction of object slant and
vision, and the effects that TMS over aIPS evokes over those biases. This approach is
based on the fact that binocular disparity cues are most informative when objects are
vertically slanted (compared to when they are horizontally slanted) 294 . In contrast,
monocular pictorial cues are most informative for discriminating horizontally slanted
objects 293,307. More precisely, the information value of stereoscopic disparity cues
is expected to change as the cosine of object slant from vertical to horizontal (the
clockwise derivative of the horizontal projection of the slant), while the information
value of pictorial cues changes as the sine of slant (the clockwise derivative of
its vertical projection). Given that binocular cues are generally preferred during
prehension 295-297, grasping horizontally slanted objects under binocular vision could
lead to the preferential processing of stereoscopic cues even when those cues are
less informative for the identification of object slant 294 .
The seven object slants were binned in two factorial levels of object slant (vertical
and horizontal) to comply with two analytical and statistical requirements. First, to
maximize the number of trials per analytical cell. Second, as described below, the
non-parametric permutation tests we performed to analyse the electrophysiological
data allows only the direct comparison of two levels per experimental factor.
Accordingly, we hypothesized that object slant identification (and thus the grasp
plan) is most accurate when objects are vertically slanted and binocular vision is
provided (binocular-vertical, relying on stereoscopic cues), or when objects are
6
118 | c h a p t e r 6
horizontally presented under monocular viewing conditions (monocular-horizontal,
relying on pictorial cues).
We arbitrarily chose to provide monocular vision only to the right eye. We did
not mix left and right eye vision to keep the set of monocular trials homogeneous.
To prevent a preference for the right eye, also in binocular trials, we included left eye
monocular trials as catch trials (~8% of the total). In summary, each cell of the 2 x 2 x
2 design contained 33 trials; a total of 264 trials were included in the analysis, and 72
additional trials were considered catch trials (44 trials with the object slanted at 45°,
28 trials allowing only monocular vision of the left eye).
In post-hoc analyses, we considered two additional factors, describing the trial
interval during which the TMS intervention was delivered [factor TMS-time, with
2 levels: early (100-200 ms), late (300-400 ms)], and the experimental run during
which the outcome measures were collected [factor TMS-run, with 2 levels: first half
of the experiment (blocks 1 and 2), second half of the experiment (blocks 3 and 4)].
Experimental conditions were pseudo-randomized and evenly distributed across
trials.
Experimentalmeasurements–handkinematics
We sampled the position and orientation of four sensors at 250 Hz, using an
electromagnetic tracking system (Polhemus LIBERTY, Polhemus, Colchester,
Vermont, USA). Three sensors were positioned on the subject’s hand: on the nail of
the thumb, the nail of the index finger, and on top of the first metacarpophalangeal
joint (MCPJ) (figure 6.2A). The fourth sensor was positioned on the object to be
grasped, along the prism’s axis of rotation.
Experimentalmeasurements–EEG
We sampled the subjects’ electroencephalogram (EEG) using 32 Ag/AgCl electrodes,
organized on a flat-tip cap (BrainProducts GmbH, Gilching, Germany) according to
the 10-10 system 451 . Electrical voltage was sampled at 5000 Hz using amplifiers with
a high dynamic range and capable of direct current recording (MR+ DC BrainAmp,
BrainProducts GmbH). Before digitization, the EEG data was low-pass filtered at
1000 Hz. Electrical artefacts associated with the TMS pulse were minimized by
the use of tip electrodes (rather than typical circular electrodes) and by positioning
the EEG cables perpendicularly to the orientation of the TMS coil when applicable.
Furthermore, to minimize the temporal spread of TMS-induced artefacts, no highpass filter was applied at acquisition.
a I P S s t r u c t u r e s a g r a s p p l a n | 119
Dataanalysis–handkinematics
Kinematic data were analysed offline using a MATLAB toolbox developed in-house
(Mathworks, Natick, Massachusetts, USA), following a multi-step procedure.
First, the TMS-induced artefact on the sensor time-series was removed (20
ms window following the TMS pulse). Since the TMS pulse was delivered when the
subject’s hand was resting on the home-key, the kinematic data included in that
window were interpolated with a piecewise cubic Hermite polynomial.
Second, the kinematic data was low-pass filtered at 15 Hz using a 6th order
Butterworth filter.
Third, we calculated velocity and acceleration (defined as the first and second
order numerical gradient approximate derivatives of the position over time) for
each sensor (figure 6.2C), and additionally for a virtual sensor (referred to as ‘grip
sensor’) positioned halfway on the axis between the thumb and index finger sensors.
Furthermore we calculated grip aperture (defined as the Euclidian distance between
the sensors placed on the thumb and index finger), grip orientation (defined as the
angle along the midsagittal plane between a vertical vector pointing upwards and
the thumb-index finger axis, figure 6.2B), and grip velocity (defined as the first order
numerical gradient approximate derivative of the grip aperture over time, figure
6.2D).
Fourth, we calculated a series of parameters describing the grasping movement.
Given the complex characteristics of the grasping and displacement movements
used in this study (figure 6.2A), we took particular care to use a robust estimate of
the onset and offset of the different phases of the movement: we combined multiple
sources of information formalized in both binary and continuous functions 452 . We
used these multidimensional criteria because, in the present experimental conditions,
the grasping movement cannot be approximated by a simple ballistic movement of
the arm or the hand. In fact, given that the movement was only described a-priori by
the end state of the hand on the target object and that visual feedback was withheld
during movement execution, we predicted that the experimental manipulations
would mainly influence the accuracy of the final phase of the grasping movement.
Accordingly, it becomes important to include in the analysis small corrective
movements based on somatosensory feedback, for example when one finger
touches the object before the other, leading to small changes in grip velocity. We
have formalized this observation by distinguishing between a transport phase of the
movement, when the MCPJ is moving in a ballistic fashion, and an approach phase,
when the MCPJ is not moving, but the thumb and index finger are still approaching
the surface of the prism (figure 6.2, transport: blue, approach: red). We defined the
following events according to these criteria:
6
120 | chapte r 6
Kinematic analyses of a representative trial
A
sensor
position
movement phase
thumb
transport
index
approach
MCPJ
40
∆ approach
grip rotation
35
positive:
grip rotates further
during approach phase
30
-25
y - pos -20
tion (cm
)
0
-15
ion
ost
p
x-
)
(cm
50
transport
approach
90º
grip aperture
grip velocity
15
100
0º
line of sight
D
sagittal velocity
150
negative:
grip rotates
back during
approach
phase
80
40
10
0
5
transport
-40
approach
0
0
0
0.2
0.4
0.6
0.8
time after movement onset (s)
1
grip velocity (cm/s)
sagittal velocity (cm/s)
C
5
grip aperture (cm)
z - position (cm)
B
-80
0
0.2
0.4
0.6
0.8
1
time after movement onset (s)
Figure6.2 Kinematic analyses
A. Representative trajectories of magnetic sensors (light-red circles) positioned on
the index finger (dashed line), thumb (dotted line), and first metacarpophalangeal
joint (MCPJ, dot-dashed line) during a single trial with the object slanted at 45°.
During the transport phase (in blue) the MCPJ is moving in a ballistic fashion. During
the approach phase (in red) the MCPJ is not moving, but the thumb and index finger
are still approaching the surface of the prism. B. Relative approach grip orientation
(∆AGO), defined as the signed difference between grip orientation at the onset of
the approach phase (dashed line) and grip orientation 100 ms after movement offset
(continuous line). C. Representative velocity profiles of MCPJ, thumb, and index
finger along the midsagittal plane, with finger adjustments defining the approach
phase. D. Representative profiles of grip aperture (solid line) and grip velocity
(dashed line), with grip adjustments after the transport phase ended.
aIP S s t r uc t ur e s a gra sp plan | 121
A. Movementonset
1. The fingers must be resting in the starting position: the thumb and index
finger are placed within 7 cm of the home-key, continuously over at least 1
second.
2. The hand must start moving: the velocity of thumb and index finger increases
above 0.02 m/s along the midsagittal plane.
3. When 1. and 2. are satisfied, then movement onset is defined as the point
in time with the lowest thumb and index finger velocity, the smallest grip
aperture, and the largest distance along the midsagittal plane between the
grip sensor and the sensor placed on the object.
B. Movementoffset
1. The fingers must be and stay on the object: the thumb and index finger are
placed within 7 cm of the centre of mass of the prism, continuously over at
least 0.2 seconds.
2. The object must be grasped and held: grip velocity decreases below 0.05
m/s for at least 0.2 seconds.
3. When 1. and 2. are satisfied, then movement offset is defined as the point in
time with the lowest thumb, index finger, and grip velocity, and the smallest
distance between the thumb, index finger, and object.
C. Approachphaseonset
1. The ballistic movement must have ended: the velocity of MCPJ decreases
below 0.3 m/s until movement offset.
2. When 1. is satisfied, then the approach phase onset is defined as the point in
time of the first minimum in MCPJ velocity.
In addition to kinematic parameters previously used to describe grasping
movements 243 (reaction time (RT), movement duration (MT), mean velocity
(MV), peak velocity (PV), relative time to peak velocity as a fraction of MT (rtPV),
maximum grip aperture (MGA), relative time to maximum grip aperture as a fraction
of MT (rtMGA)), we characterized the approach phase of the movement by using the
following three parameters: 1) approach grip aperture (AGA): the grip aperture at the
start of the approach phase, 2) differential approach grip orientation (∆AGO): the grip
orientation at the start of the approach phase relative to the orientation measured
100 ms after movement offset, 3) the integral of the approach grip velocity (∑AGV),
taken over the whole approach phase. When the grip will be closing continuously
during the approach phase the ∑AGV will be negative, and more so when the hand
is closing down further. If the grip needs to be adjusted and re-opened before finally
grasping the object this ∑AGV will become more positive, possibly even above zero.
Hence the ∑AGV parameter is a comprehensive description of the closing phase of
the grip, including any potential adjustments, during the approach phase. A full list
of the kinematic parameters we considered is displayed in table 6.1.
6
122 | chapte r 6
Statisticalinference–handkinematics
We excluded trials where the sensors moved outside the field of view of the Polhemus
system, trials where subjects started moving before or at the time of the TMS pulse,
and trials where the main kinematic parameters deviated from the first or third
quartile by more than three IQRs.
Statistical inference of the kinematic measurements was drawn using the SPSS
software package (SPSS 16.0, SPSS Inc, Chicago, Illinois, USA). All parameters were
checked on skewness and kurtosis, and if found deviating were log transformed
to commit to the assumption of normal distribution (this was applicable for
the trajectory length and mean velocity in the approach phase: ATL and AMV,
respectively; see tables 6.1 and 6.2). Trials were averaged for each experimental
condition and the resulting means were entered in a univariate repeated measures
ANOVA testing for effects between conditions within subjects. The three
parameters describing the approach phase (AGA, ∑AGV, ∆AGO) were also entered in
a within subjects multivariate repeated measures MANOVA to account for potential
interdependencies.
Dataanalysis–EEG
Electrophysiological data were analysed offline using a MATLAB toolbox (FieldTrip,
http://www.ru.nl/neuroimaging/fieldtrip) 453 , following a multi-step procedure.
First, we excluded trials with eye movements, blinks, and muscle artefacts (on
the basis of visual inspection of the data).
Second, each EEG sensor was screened (in a 1 s window following the TMS pulse)
for the following types of TMS-induced artefacts (please see chapter 3 of this thesis
for details):
A. saturation of the EEG signal (~2 ms after the TMS pulse), with consequent
ringing artefacts (~7 ms after the TMS pulse) caused by the 1000 Hz low-pass
filter. We discarded data from the time-window between -0.2 ms to +10 ms from
the TMS pulse.
B. slow step-response (~60-400 ms after the TMS pulse), attributable to stepresponses in the resistor-capacitor circuits and operational amplifier of the
amplifier hardware, and possibly to polarization of the electrode-electrolytescalp circuit370,371 . These step-responses were fitted and removed using an
iterative least-squares fitting algorithm (implemented in FieldTrip) when
possible, otherwise the trial was discarded.
aIP S s t r uc t ur e s a gra sp plan | 123
Third, the data were re-referenced to the average signal of all sensors to remove any
spatial effects on voltage differences with respect to the localization of the reference
electrode.
Fourth, all trials surviving the exclusion criteria were entered into an independent
component analysis, in order to identify and remove residual signals related to eye
movements, blinks, muscle tension and TMS artefacts.
Fifth, power line noise was removed (using a discrete Fourier transform notch
filter at 50, 100 and 150 Hz) and the resulting electrophysiological data were bandpass filtered (0.75-150 Hz, 6th order Butterworth filter) and down-sampled to 500
Hz.
Sixth, for each trial, we calculated time-frequency representations (TFRs) of
oscillatory power, focusing on the alpha (8-12 Hz) and beta (18-24 Hz) frequency
bands. We used a Fourier transform approach (8-35 Hz, in steps of 1 Hz) applied to
sliding time windows (200 ms, sliding in 20 ms steps) multiplied by a Hanning taper
(resulting in a frequency smoothing of 5 Hz). For each experimental condition,
power estimates were averaged over trials, log-transformed, and related to a
baseline period from the same trials (relative change from -700 to -200 ms before
trial onset). It should be emphasized that we were able to generate a continuous
averaged estimate of the power in the alpha and beta bands, un-affected by TMS
intervention. More precisely, given the trial-by-trial temporal variation in TMS
delivery (see Experimental intervention – TMS), the time window from which no
power estimate could be obtained on each individual trial occurred at different time
points across different trials, allowing to create a continuous average over trials.
Within single trials a 220 ms window without oscillatory data arises as a direct result
of not allowing EEG data less than 10 ms after the TMS pulse to be considered in the
200 ms Hanning taper window. The time bins with EEG data nearest to a given TMS
pulse were centred at 100 ms before (window: [-200 0] ms) or 105 ms after (window:
[10 210] ms) the pulse. Combined with the fact that the Hanning taper slid with
20 ms intervals this resulted in a time window of 220 ms lacking oscillatory power
estimation in each trial.
Seventh, the individual effects were grand averaged over subjects to report the
group effects.
Statisticalinference–EEG
Within each subject, the difference between conditions was quantified as the
difference of the relative log transformed mean power changes. Statistical inference
(p<0.05) was performed at the group level (random effects analysis) using a nonparametric randomization test controlling for multiple comparisons over the large
6
124 | chapte r 6
search space given by multiple sensors, frequency bands, and time-intervals (SFT
points) 454 . This procedure involves the following steps.
First, all SFT points are identified for which the t statistics for the difference
between conditions over subjects exceeds a threshold (p>0.05).
Second, contiguous SFT points exceeding the threshold are grouped in clusters,
the t-values from each SFT point of a cluster are added, and this cumulative t-value
is used for inferential statistics at the cluster-level.
Third, a Monte Carlo estimate of the permutation p-value of the cluster is
obtained by comparing the cluster-level test statistic to a randomization null
distribution assuming no difference between conditions. This distribution is obtained
by randomly swapping the conditions in subjects and calculating the cluster-level
test statistic multiple times. Using 10000 random draws the Monte Carlo p-value is
an accurate estimate of the true p-value.
Results
Behaviouralresults–taskperformancefollowingcontrol
TMSoververtex
Effects of object slant: As the slant of the target object increased from vertical to
horizontal, the subject’s hand needed to travel over a longer path. The kinematic
measurements clearly isolated this basic effect (main effect of slant within the vertex
session; see tables 6.1 and 6.2; figure 6.3). Subjects reached a higher peak velocity
relatively earlier in the movement (PV and rtPV, figure 6.3B), while the maximum
grip aperture occurred relatively later for more horizontally slanted objects (rtMGA).
Grip orientation at the onset of the approach phase closely matched the object’s
vertical orientations, whereas it was slanted back during trials when the object was
horizontally oriented (∆AGO, figure 6.4C). In line with previous reports314,439 , the
maximum and approach grip aperture became smaller as object’s slant increased
(MGA and AGA). In contrast to these widespread effects of object slant on grasping
behaviour, this factor did not influence the duration of movement planning (i.e.
the interval between stimulus presentation and movement onset, RT: F(1,22)=0.26,
p=0.615).
Effects of object vision: Planning the grasping movement when only monocular cues
were available led to slightly shorter reaction times, possibly a consequence of the
additional time required to converge the eyes on the object when binocular vision
was available. The shorter reaction time evoked during monocular trials resulted in
aIP S s t r uc t ur e s a gra sp plan | 125
larger planning uncertainties in the transport phase of the movement, as illustrated
by longer movement duration, longer travelled path and reduced mean velocity
(tables 6.1 and 6.2, figure 6.3A).
Effects of slant x vision interaction: The current experimental design focuses on
the interaction of slant and vision factors. This approach is based on the fact that
binocular disparity cues are more informative than monocular pictorial cues when
objects are vertically slanted. In contrast, monocular pictorial cues are more
informative than binocular cues for discriminating horizontally slanted objects 293,307.
Accordingly, we hypothesized that object slant identification (and thus the grasp
plan) is most accurate when objects are vertically slanted and binocular vision is
provided (binocular-vertical trials, relying on stereoscopic cues), or when objects are
horizontally presented under monocular viewing conditions (monocular-horizontal
trials, relying on pictorial cues).
This slant x vision interaction was observed in several kinematic parameters
specifically related to the spatial consistency of the grasping movement during
the approach phase (F(1,22)=4.70, p=0.012 on a multivariate test across approach
parameters on the slant x vision interaction; table 6.1, figure 6.4A-C). There were
neither corresponding effects on the duration of the transport and approach
phases, nor on early markers of planning accuracy, like the maximum grip
aperture. However, during trials with informative depth cues (binocular-vertical
Kinematic effects of task performance - vertex TMS
1.15
movement time
B
106
1.00
0.95
PV (cm/s)
MT (s)
1.10
1.05
108
104
102
100
peak velocity
C
MGA (cm)
A
12.5
max grip aperture
12.0
11.5
11.0
Figure6.3 Behavioural effects following TMS over vertex (control)
Kinematic parameters (mean ± standard error) measured during trials with binocular
vision (in red) or monocular vision (in green), grouped by object slant (in blue: nearvertical slants; in orange: near-horizontal slants). A. Grasping movement durations
(encompassing both the transport and approach phases) increased with increasing
object slant and were longer in trials with monocular compared to binocular vision.
B-C. The peak velocity of the virtual grip sensor (mean of thumb and index finger
sensors) was higher, while the maximal grip aperture was smaller in trials where the
object was slanted more horizontally.
6
RT
MT
TMT
rAMT
TL
TTL
ATL
MV
TMV
AMV
PV
rtPV
MGA
rtMGA
AGA
ΔAGO
ΣAGV
reaction time
movement duration
trasport duration
relative approach duration
trajectory length
transport trajectory
approach trajectory (log)
mean velocity
mean transport velocity
mean approach velocity (log)
peak velocity
rel. time to peak velocity
maximum grip aperture
rel. time to max. grip apt.
approach grip aperture
Δ approach grip orientation
Σ approach grip velocity
multivariate approach phase
parameter
0.000 (34.67)
0.029 (5.44)
0.000 (25.26)
0.000 (75.33)
0.000 (30.41)
0.000 (49.89)
0.000 (132.21)
0.000 (143.23)
0.000 (96.17)
0.000 (30.73)
0.000 (103.89)
0.005 (9.73)
0.000 (337.76)
0.000 (289.89)
0.001 (13.70)
0.029 (5.49)
slant
TMSoververtex
0.047 (4.43)
0.030 (5.38)
0.000 (17.95)
0.046 (4.46)
0.009 (8.12)
0.013 (7.38)
0.069 (3.65; ns)
0.002 (12.87)
0.007 (8.69)
0.005 (9.97)
0.012 (4.70)
slantx
vision
0.035 (5.03)
site
aIPSvs.vertex
0.087 (3.21; ns)
0.010 (7.85)
sitex
vision
0.006 (9.23)
0.043 (4.59)
0.027 (5.65)
0.030 (3.65)
0.069 (3.67; ns)
0.080 (3.37; ns)
0.034 (5.09)
sitex
slantxvision
a significant modulation). Statistical inference was performed
using two univariate repeated measures ANOVA. The parameters
describing the grasping strategy during the approach phase
(AGA, ∆AGO and ∑AGV) also entered a multivariate repeated
measures MANOVA. To maintain clarity only statistics with
associated probabilities below α=0.10 are shown. Trends with
0.05<p<0.1 are printed in italics and indicated with ‘ns’.
0.048 (4.39)
0.000 (23.57)
0.003 (11.01)
vision
Statistical probabilities and F-statistics (in brackets; df: 1,22)
of investigated kinematic parameters for experimental effects
after control stimulation (vertex) and for the differential effect
between stimulation over vertex and aIPS. Presented are the
effects of object slant and vision and their interaction during
vertex runs, and the main and interaction effects of stimulation
site (site x slant is not represented as no parameters revealed
Table6.1 Probabilities and F-statistics of experimental manipulations on behavioural parameters
126 | chapte r 6
RT
MT
TMT
rAMT
TL
TTL
ATL
MV
TMV
AMV
PV
rtPV
MGA
rtMGA
AGA
ΔAGO
ΣAGV
ms
ms
ms
%
cm
cm
cm
cm/s
cm/s
cm/s
cm/s
%
cm
%
cm
deg
cm
vision
slant
site
680 (35)
1020 (48)
797 (29)
19.99 (1.05)
40.72 (0.38)
39.20 (0.38)
1.37 (0.17)
42.27 (1.92)
50.87 (1.96)
6.13 (0.28)
102.4 (4.34)
33.05 (0.67)
12.05 (0.17)
66.64 (0.69)
10.59 (0.13)
0.41 (0.40)
-1.55 (0.11)
binocular
binocular
680 (36)
1089 (50)
874 (31)
17.34 (1.13)
44.62 (0.47)
43.27 (0.47)
1.13 (0.15)
43.06 (2.01)
50.97 (1.92)
5.10 (0.25)
105.1 (4.77)
30.62 (0.58)
11.30 (0.20)
69.66 (0.99)
10.02 (0.11)
5.00 (0.55)
-0.75 (0.11)
665 (33)
1055 (50)
818 (30)
20.15 (1.09)
41.15 (0.41)
39.54 (0.41)
1.43 (0.17)
41.44 (1.92)
50.22 (2.01)
5.90 (0.25)
102.6 (4.35)
32.22 (0.60)
12.09 (0.15)
65.23 (0.73)
10.51 (0.13)
-0.25 (0.49)
-1.46 (0.12)
binocular
647 (33)
1006 (42)
794 (27)
19.37 (0.98)
40.86 (0.43)
39.48 (0.38)
1.27 (0.13)
42.77 (1.83)
51.36 (1.80)
6.05 (0.26)
103.0 (3.78)
32.77 (0.68)
12.01 (0.16)
66.21 (0.77)
10.47 (0.13)
-0.31 (0.51)
-1.44 (0.12)
659 (35)
1112 (49)
890 (35)
18.14 (1.02)
44.79 (0.42)
43.47 (0.40)
1.12 (0.10)
42.48 (1.96)
50.68 (2.03)
5.12 (0.20)
104.5 (4.78)
30.59 (0.62)
11.27 (0.18)
69.28 (1.09)
10.16 (0.11)
5.46 (0.52)
-0.90 (0.11)
638 (35)
1035 (42)
801 (27)
20.80 (0.88)
41.27 (0.48)
39.69 (0.45)
1.37 (0.12)
41.77 (1.73)
50.99 (1.86)
6.24 (0.27)
102.4 (3.80)
32.06 (0.69)
12.10 (0.17)
65.23 (0.87)
10.57 (0.13)
-0.35 (0.56)
-1.52 (0.12)
monocular
vertical
649 (35)
1085 (42)
870 (28)
17.97 (0.92)
44.74 (0.53)
43.45 (0.49)
1.13 (0.11)
42.94 (1.60)
51.43 (1.69)
5.07 (0.22)
105.2 (4.28)
30.56 (0.66)
11.28 (0.20)
69.15 (1.12)
10.03 (0.12)
4.87 (0.51)
-0.79 (0.11)
639 (32)
1088 (40)
872 (28)
17.90 (0.88)
44.92 (0.50)
43.61 (0.47)
1.14 (0.11)
42.92 (1.56)
51.38 (1.74)
5.04 (0.24)
104.8 (4.27)
30.38 (0.70)
11.30 (0.18)
68.91 (1.08)
10.14 (0.11)
5.02 (0.52)
-0.87 (0.10)
monocular
horizontal
binocular
aIPS
used to denote the parameters are spelled out in table
6.1. Parameters ATL and AMV were log-transformed
before entering statistical analysis (see Methods and table
6.1), but are here reported as means and SEMs of the
original parameter values.
monocular
horizontal
monocular
vertical
vertex
Mean and standard error of the mean (SEM, in brackets)
of behavioural parameters, reported separately for the
experimental conditions of the factors TMS-site (vertex
and aIPS), object slant (vertical and horizontal), and
object vision (binocular and monocular). Abbreviations
Table6.2 Behavioural parameter values by experimental condition
aIP S s t r uc t ur e s a gra sp plan | 127
6
128 | chapte r 6
and monocular-horizontal), the approach grip orientation was relatively better
aligned with the object orientation and the fingers closed faster and further when
approaching the object. In contrast, during trials with less informative depth cues
(binocular-horizontal and monocular-vertical) subjects had a smaller grip aperture at
the onset of the approach phase, and their fingers closed more slowly (figure 6.4). We
interpret these observations as a sign that under visually sub-optimal circumstances,
subjects created the conditions for increased reliance on somatosensory feedback,
bringing the fingertips closer to the object and with a lower approach speed. Given
the absence of visual feedback during movement execution, these kinematic
events occurring late in the movement are a particularly sensitive index of planning
inaccuracies, and they suggest that our experimental manipulation was successful
in inducing subtle but reliable changes limited to kinematic aspects of the approach
phase, rather than dramatic effects leading to adjustments affecting the whole
grasping movement. Accordingly, we infer that during trials with informative
stereoscopic and pictorial depth cues, the subjects could better adjust their grasping
movements to the orientation of the graspable surfaces on the basis of depth
information derived from visual cues acquired before movement onset.
Behaviouralresults–taskperformancefollowingTMS
overaIPS
Effects of TMS-site x slant x vision interaction: Having established the presence
of a relative planning advantage of informative depth cues (binocular-vertical
and monocular-horizontal) in the TMS-control session, we proceeded to test the
hypothesis that aIPS stimulation would affect the ability of the brain to incorporate
these stereoscopic and pictorial depth cues into the motor plan. We found that
aIPS stimulation had strong deleterious effects on several kinematic parameters
specifically related to the approach phase of the movement, removing the planning
advantage observed when informative depth cues were present during the
TMS-control session (F(1,22)=3.65, p=0.030 on a multivariate test across approach
parameters on the site x slant x vision interaction, figure 6.4D-F). More precisely, the
relatively better aligned grip orientation and faster and further closing of the initially
larger grip aperture observed during the approach phase in the TMS-control session
were diminished after TMS over aIPS. Taken together, these findings indicate that
aIPS stimulation removed the relative planning advantages seen in those trials
where maximally informative visual cues were available.
aIP S s t r uc t ur e s a gra sp plan | 129
Kinematic effects of the approach phase - vertex TMS
10.5
B
10.0
9.5
-2.0
∑ approach
grip velocity
C
∆AGR (degº)
11.0
approach
grip aperture
∑AGV (cm)
AGA (cm)
A
-1.5
-1.0
8
∆ approach
grip rotation
6
4
2
0
-2
-0.5
Kinematic effects of the approach phase - aIPS vs. vertex TMS
approach
grip aperture
0.2
0
-0.2
E
vertex
aIPS
-0.4
∑ approach
grip velocity
-0.2
0
-0.2
F
∆AGR (degº)
0.4
∑AGV (cm)
AGA (cm)
D
vertex
aIPS
2
∆ approach
grip rotation
1
0
-1
vertex
aIPS
Figure6.4 Differential behavioural effects - TMS over vertex and aIPS
A-C. Behavioural effect following TMS over vertex. A. The grip aperture at the start
of the approach phase decreased with increasing object slant, but also showed
an additional interaction between object slant and vision, being larger when the
available depth cues were optimal for the presented slant. B. These effects were
also present in the integral of the grip velocity over the approach phase. Subjects
closed their fingers faster and further following binocular vision of vertically
oriented objects, and after monocular vision of horizontally oriented objects. A
faster and further grip closing leads to a more negative value, hence the reversal
of the coding on the y-axis. C. The grip orientation (see figure 6.2B) was rotated
further during the approach phase when the object was near horizontal, but
also when the object’s slant could be reliably identified (binocular-vertical and
monocular-horizontal). Difference between the kinematic parameters measured
after TMS over aIPS and vertex. Other conventions as in figure 6.3. D-F. Contrast
estimates of the object slant x vision interaction following either TMS over vertex
(grey) or aIPS (magenta). The contrast estimates of the vertex effects are the same
as those represented for object slant and vision conditions separately in panels A-C.
Movement features related to the grasp approach phase were altered by TMS over
aIPS, more precisely during trials with informative depth cues (binocular-vertical
and monocular-horizontal). During these trials, the approach grip aperture and
the approach grip velocity integral became smaller, while the grip orientation was
rotated less far during the approach phase.
6
130 | c h a p t e r 6
EEG effects of task performance - irrespective of TMS site
time-locked to stimulus onset
vertex TMS
A
beta
18-24 Hz
power change (%log)
60
B
30
0
-30
alpha
8-12 Hz
-60
0 - 0.5 s
vertex TMS
C
vertex and aIPS
30
frequency (Hz)
left motor
C3 ( )
0.5 - 1.0 s
β-power change (%log)
25
0
20
-20
15
-40
10
-60
30
frequency (Hz)
occipital
O1, Oz, O2 ( )
D
TMS site
vertex
aIPS
contrast
α-power change (%log)
25
0
20
-20
15
-40
10
-60
-0.5
0
0.5
time from stimulus (s)
1
-0.5
0
0.5
time from stimulus (s)
1
Figure6.5 Electrophysiological effects following TMS over vertex (control)
Time-locked either to the presentation of the stimulus (lef panel) or to the onset
of the movement (right panel). A-B. Topographies of oscillatory power changes in
the beta band (18-24 Hz; A) and in the alpha band (8-12 Hz; B) relative to baseline
([-700 -200] ms before opening of the shutters) measured during the vertex session.
There were strong power reductions over left motor cortex (beta band; electrode
C3; diamond) and over visual cortex (alpha band; electrodes O1, Oz and O2; stars).
a I P S s t r u c t u r e s a g r a s p p la n | 131
EEG effects of task performance - irrespective of TMS site
time-locked to movement onset
vertex TMS
A
beta
18-24 Hz
power change (%log)
60
30
0
B
-30
alpha
8-12 Hz
-60
0 - 0.5 s
0.5 - 1.0 s
vertex and aIPS
vertex TMS
25
-20
20
15
-40
10
-60
-20
20
15
frequency (Hz)
25
occipital
O1, Oz, O2 ( )
0
-40
10
-60
0
0.5
1
time from movement (s)
D
30
α-power change (%log)
-0.5
left motor
C3 ( )
0
frequency (Hz)
TMS site
vertex
aIPS
contrast
C
30
β-power change (%log)
-0.5
0
0.5
time from movement (s)
1
C-D outer columns. Power changes over time in electrode C3 (C) and O1-Oz-O2
(D) show that the power reduction in the alpha and beta bands covered the whole
duration of the motor planning phase. C-D inner columns. Power changes over time
in the beta and alpha bands measured at C3 and O1-Oz-O2 during control TMS
(grey) or TMS over aIPS (magenta; between-session difference in black) reveal no
main effect of stimulation site. Shaded area surrounding the grand average curves
represents standard error of the mean.
6
132 | c h a p t e r 6
EEGresults–oscillatoryactivityfollowingTMSover
vertex
As the shutters opened and the subjects could see the object and prepare their
movement, ongoing cerebral oscillations were strongly suppressed in the alpha band
(8-12 Hz; p<0.001; strongest effect over parieto-occipital electrodes, figure 6.5B),
and in the beta band (18-24 Hz; p<0.001; strongest effect over left fronto-parietal
electrodes, figure 6.5A). The time-frequency representations (TFRs) of occipital
electrodes O1, Oz and O2 (figure 6.5D, identified with stars in panel B) indicate that
the occipital alpha suppression covered the whole duration of the motor planning
phase, returning slightly above baseline after movement onset and concurrent
shutter closure (figure 6.5B, D). Similarly, the beta power suppression in electrode C3
(figure 6.5C, identified with a diamond in panel A) was sustained during both motor
planning and execution phases. These observations reproduce well-known effects
linking alpha and beta suppression to increased computational load over visual and
motor areas, respectively318,325 . The TFR derived from occipital electrodes timelocked to stimulus presentation (figure 6.5D – left panel) reveals an additional burst
of synchronization in high beta frequencies, caused by visually evoked potentials
phase-locked to the opening of the shutters.
Effects of object slant: There were strong modulations of task-related oscillatory
activity as a function of object slant. Increasing object slant from vertical to
horizontal led to a stronger alpha suppression over medial parietal and frontal
regions. This differential effect started late during movement planning, riding on
top of the overall task-related sustained alpha suppression, and continued during
movement execution (p=0.004; see chapter 7 of this thesis for details).
Effects of object vision: There were no significant differential effects between trials
performed with monocular or binocular vision on oscillatory activity in the alpha and
beta band.
Effects of slant x vision interaction: There was enhanced power suppression in the
beta band over electrode C3 when reliable visual cues were available (stereoscopic
during binocular-vertical and pictorial during monocular-horizontal trials; p=0.002;
figure 6.6A left panel). This enhancement of beta suppression was spatially localized
over the left motor cortex, confined to the beta frequency band, and occurred early
in the movement planning (220-400 ms after stimulus presentation; figure 6.6B
left panel). This finding fits with the kinematic effects described above (figure 6.3),
providing a clear neurophysiological correlate for the notion that when informative
depth cues are available subjects could plan a more accurate grasping movement.
a I P S s t r u c t u r e s a g r a s p p la n | 133
Effects of TMS-run x slant x vision interaction: Analysis of the influence of task
practice on the magnitude of the enhanced beta suppression when informative
visual cues were available revealed that this effect was stronger in the second half of
the experiment (p=0.013, figure 6.6D). This finding suggests that, as subjects build
their knowledge of the configurations of the grasped object across trials, this factor
comes to play a significant role in the ability of the motor cortex to incorporate
informative stereoscopic and pictorial depth cues into the motor plan.
EEGresults–oscillatoryactivityfollowingTMSoveraIPS
In the previous section, we reported task-related effects in the TMS-control session
and considered every trial epoch (solid lines in figure 6.5C-D, central panels). In this
section, we focus on the differences between aIPS and vertex stimulation, and for
this purpose we consider only trial epochs that followed a TMS pulse, in order to
make sure that the differences are about the consequences of TMS alone (dashed
lines in figure 6.5C-D, central panels). There were neither effects of aIPS stimulation
over task-related oscillatory activity (main effect of site: figure 6.5C-D, central panels;
grey: vertex; magenta: aIPS; black: aIPS-vertex contrast), nor differences driven by
the manipulation of viewing conditions (main effect of vision). Furthermore, a posthoc analysis on the consequences of delivering the TMS pulse either early (100-200
ms after stimulus presentation) or late (300-400 ms) did not reveal any differential
effect either. In the following section, we focus on the interaction that constitutes
the principal focus of the current experimental design. To recapitulate, we have
provided evidence for the presence of enhanced beta suppression over electrode C3
under informative stereoscopic and pictorial conditions in the TMS-control session,
paralleling the presence of a relative planning advantage as shown by the kinematic
data. Here, we proceed to test the hypothesis that aIPS stimulation would affect that
enhanced beta suppression, specifically when informative depth cues are available.
Effects of TMS-site x slant x vision interaction: The enhanced suppression in the beta
band over electrode C3 when informative visual cues were available was disrupted
following aIPS perturbation (p=0.009; figure 6.6A right panel). This disruption
occurred at the same location (figure 6.6A), frequency band, and time (260-460
ms after stimulus; figure 6.6B) as the oscillatory effect observed in the slant x
vision interaction following TMS over vertex. The early occurrence of the effect, in
combination with the sliding time-window employed in the time-frequency analysis
(which imposes a time-delay after the TMS pulse of at least 100 ms where no
oscillatory power estimation is possible) implies that the effect can only have arisen
as a consequence of TMS pulses delivered early after stimulus presentation (100-200
ms) and not later (300-400 ms). This interaction is supported by all simple effects
of slant and vision (figure 6.6C, comparable to the behavioural effects presented in
6
13 4 | c h a p t e r 6
EEG-TMS effects of informative spatial and pictorial cues
A
control TMS
more > less informative cues
aIPS vs. control
more > less informative cues
power change (%log)
20
10
0
beta
18-24 Hz
-10
-20
0.22 - 0.40 s
0.26 - 0.46 s
after stimulus onset
after stimulus onset
B
frequency (Hz)
C3 ( )
30
25
20
15
10
0
0.2
0.4
0.6
0.8
1
0
0.2
time after stimulus (s)
aIPS vs. control
informative cues: slant x vision
10
5
0
-5
-10
0.6
0.8
effect of learned
object knowledge
D
20
β-power change (%log)
β-power change (%log)
C
0.4
time after stimulus (s)
10
���
1
st
2nd
0
-10
-20
control
aIPS
1
a I P S s t r u c t u r e s a g r a s p p la n | 135
figure 6.4), with potentially the strongest contribution when the object is vertically
slanted (post-hoc p=0.094).
Effects of TMS-run x TMS-site x slant x vision interaction: Analysis of the influence
of familiarity with the object’s configurations on the magnitude of the disruption
of enhanced beta suppression when informative stereoscopic or pictorial cues were
available revealed that the effect of aIPS perturbation was stronger in the second
half of the experiment (p=0.018; figure 6.6D). It can be seen that in the first half
of the experiment aIPS stimulation merely cancels the potential advantage of
informative depth information. Conversely, in the second half, motor planning
is actually hampered when aIPS is disturbed, even when accurate visual cues are
available.
Figure6.6 Differential EEG effects - TMS over vertex and aIPS
A. Topographies of differential oscillatory power changes in the beta band for the
interaction between object slant and viewing conditions, measured after TMS over
aIPS (as compared to vertex; right panel). For comparison, the same effect measured
during the vertex session is shown in the left panel. B. Time-frequency plots of
power in electrode C3 (identified with a diamond in panels A) for the contrasts
illustrated in panel A. The frequency and time intervals of significant effects (p<0.05
corrected for multiple comparisons) are indicated with grey boxes. C. The site x slant
x vision interaction effects on beta power in electrode C3 are plotted separately
for the object slant (vertical: blue; horizontal: orange) and vision (binocular: red;
monocular: green) conditions (similarly to figure 6.4). These contrast estimates are
obtained by averaging power over the significant time and frequency points (grey
boxes in panel B) and subtracting the vertex effects from those measured during
the aIPS session. They show that the three way interaction is driven by all simple
effects constituting the slant x vision interaction. D. The contrast estimates of the
slant x vision interaction are plotted as a function of TMS-site and run (first run:
yellow; second run: cyan). In control sessions the enhanced motor beta suppression
following reliable depth cues grew stronger with increased experience across trials.
aIPS perturbation in the first half of the experiment cancelled the advantage of
reliable depth cues, but disruption of aIPS in the second half of the experiment led
to an impairment of motor beta suppression when otherwise optimal visual cues
are offered.
6
136 | c h a p t e r 6
Discussion
We have investigated the cortical dynamics supporting the planning of grasping
movements, focusing on the role of the anterior intraparietal sulcus in the integration
of metric and perceptual object features into a motor plan. We exploited the fact that
showing horizontally slanted objects under monocular vision, or vertically slanted
objects under binocular vision, provides maximally informative depth cues for
identifying the orientation of an object to be grasped 294 . When these depth cues were
available, there was enhanced suppression of oscillatory power in the beta band over
motor cortex (MC beta suppression), early during the planning phase. The MC beta
suppression, a well-known neurophysiological hallmark of motor preparation 455 , was
accompanied by behavioural signs of enhanced planning accuracy, namely increased
spatial consistency of the fingers’ movements just before contact with the object
(i.e. during the approach phase). These findings indicate that the grasp plan was
sensitive to the information value of the available depth cues, and that both metric
and perceptual cues were effectively integrated into the motor plan. We used singlepulse TMS to challenge aIPS, the region hypothesized to support this integrative
mechanism. This intervention decreased MC beta suppression, 200-400 ms after
stimulus presentation, and the spatial consistency of the grip during the approach
phase. These findings indicate that stimulation of aIPS reduced the ability of the
prehension system to integrate metric and perceptual depth cues into the motor
plan, and that this integration occurs early during planning. Finally, we considered
whether this integrative mechanism also contributes to incorporate knowledge of
the object configuration developed over the course of the experiment. We observed
that as this knowledge increased, MC beta suppression was enhanced, early during
planning. Critically, aIPS stimulation reversed this effect, even when informative
depth cues were available, an indication that this region favours abstract object
knowledge over current sensory evidence for planning a grasping movement. In the
following sections, we discuss these findings, elaborating on their implications for
understanding the cerebral control of grasping movements.
aIPSincorporatesbothmetricandperceptualdepthcues
intoagraspplan
It is known that macaque AIP and human aIPS are involved in controlling the
fingers’ movements according to intrinsic object metrics during visually guided
grasping 4,149,160,239 . The present kinematic and electrophysiological data confirm
those observations, and provide novel evidence on the characteristics of aIPS
contributions. When either metric or perceptual depth cues were most reliable (i.e.
during grasping of vertical objects viewed binocularly, or horizontal objects viewed
a I P S s t r u c t u r e s a g r a s p p la n | 137
monocularly), we observed an enhanced preparatory beta suppression over the
motor cortex contralateral to the grasping hand. In these conditions, disruption
of neuronal processing in aIPS caused a robust and specific reduction of MC beta
suppression. Given that beta suppression is known to originate from the motor
cortex and to index its computational load318,325,329 , these observations indicate
that disturbing aIPS during the planning phase of a grasping movement reduces
the ability of the motor cortex to adequately organize the movement. Moreover,
because subjects grasped the object without visual feedback, the functional scope
of this planning impairment can be captured by the behavioural consequences of
the TMS intervention. We found that disturbing aIPS selectively impaired subjects’
ability to adjust their grasping movements to the orientation of the graspable
surfaces, leaving early phases of the movement unaffected. These behavioural and
electrophysiological findings extend previous observations (chapter 5 of this thesis),
showing that aIPS is indispensable for integrating both metric and perceptual
information into motor commands during the organization of a grasp plan.
aIPSprovidesafastmotorplanforgrasping
Besides providing evidence for a causal role of aIPS in organizing grasping according
to metric and perceptual cues, this study details the temporal dynamics of aIPS
contribution to grasping. There are two main findings. First, motor planning was
altered (as indexed by reduced MC beta suppression) only when aIPS was stimulated
early (<200 ms after stimulus onset). Second, the electrophysiological consequences
of that perturbation were also early (260-460 ms after stimulus onset). These findings
suggest that the computations implemented in aIPS are necessary to develop the
initial structure of the grasping movement, in line with recent neurophysiological
data from macaque AIP40,149 . These findings also fit with other TMS studies
showing that aIPS is necessary to rapidly adjust prehension when intrinsic object’s
characteristics change 161,239,456 , with the emphasis on fast generation of a new grasp
plan, rather than online control.
aIPSincorporatesobjectconfigurationknowledgeintoa
graspplan
Previous work has clearly detailed how aIPS conveys current object properties to the
motor cortex 160 . Here we have assessed whether aIPS is also involved in conveying
to the motor system more abstract sources of information. We considered whether
subjects made use of the knowledge of the object’s configuration accumulated
across trials when it could be reliably estimated. Although there were no detectable
changes in subjects’ behaviour, and the stimuli remained the same, there was
6
138 | c h a p t e r 6
increased MC beta suppression in the second half of the experiment when
maximally informative cues were available with control stimulation. Crucially, aIPS
stimulation had a stronger effect on visuomotor integration in the second half of
the experiment, presumably preventing the subjects from using informative depth
cues even when available (of either stereoscopic of pictorial nature). This finding
suggests that, as object knowledge becomes accessible, the sensorimotor system
comes to strongly depend on prior experience to structure the movement plan. This
observation fits with impairments observed in patients with ideomotor apraxia,
i.e. patients with left lateralized inferior parieto-frontal lesions, often including the
anterior intraparietal sulcus 257,258 . Although apraxia patients adjust their transport
kinematics and preshape their hand according to the metrics of a target object,
their prehension is often functionally inappropriate 260,263 . For example, they might
skilfully grasp a spoon, but at the wrong end. The link with the present findings is
that, paradoxically, these patients are even more impaired when grasping familiar
than non-familiar objects 265 . Similarly, in this study subjects’ preparatory activity
is even more disturbed after subjects had become familiar with the properties of
the grasped object. In both cases, currently available sensory evidence appears
overridden by previously acquired priors.
Interpretationalissues
This study manipulated the sources and relative reliability of depth cues necessary
to organize an effective grasping movement, introducing combinations of viewing
conditions and object slants that required subjects to estimate object’s depth either
through perceptual cues (when grasping horizontally slanted objects with monocular
vision) or stereoscopic disparity cues (when grasping vertically slanted objects with
binocular vision). It might be argued that, as monocularly-derived perceptual cues
remain available during binocular vision, grasping horizontal objects with binocular
or monocular vision should not differ. Yet, the kinematic data indicate that subjects
were more accurate when grasping horizontally slanted objects under monocular
than binocular vision. This effect fits with the known preferential processing of
binocular cues during grasping 295-297, extending those observations to a situation
when binocular cues are ambiguous for the identification of object slant. More
generally, both kinematic and electrophysiological data converge towards the notion
that, when the object slant and vision combinations were maximally informative for
estimating object depth, subjects prepared a more accurate grasp plan, approaching
the graspable surfaces of the object on the basis of depth information derived from
visual cues acquired before movement onset.
It should be emphasized that, rather than creating ‘virtual lesions’, our aim was
to perturb the sensorimotor system within its physiological range. Reassuringly,
a I P S s t r u c t u r e s a g r a s p p la n | 139
we did not observe any main effects of stimulation site on oscillatory power or
kinematic behaviour. In fact, the TMS-driven oscillatory changes were higher order
interactions between experimental factors, restricted to the beta band over left
motor cortex, and temporally remote from the TMS intervention.
Conclusion
An influential theory of how grasping is computationally and cerebrally controlled
distinguishes between transport and grip components, with aIPS supporting the
parameterization of finger movements according to intrinsic objects metrics 2,72,113 .
Here we qualify and extend the contributions of aIPS during grasping movements,
showing that the computations implemented in aIPS are critical during the early
planning stage, concerned with defining the structure of the grasping movement on
the basis of abstract object configuration knowledge, and deal with both metric and
perceptual cues. These findings open the way for understanding why aIPS lesions
can affect apparently disparate functions like grasping 152 , online control 239 , and tool
use 163,256,258,265 .
6
7
Cortical dynamics
of dorsomedial and
dorsolateral parietofrontal circuits during
grasping movements
142 | cha p t e r 7
Abstract
The cortical control of grasping movements is anatomically and functionally
distributed over dorsomedial and dorsolateral parieto-frontal circuits. An
influential framework has linked these circuits to the control of independent
arm-transport and hand-grip components, based on extrinsic and intrinsic
object properties, respectively. However, recent evidence has shown that
both circuits are responsive to similar object features and body parts, raising
the issue of what differentiates the contributions of these two grasp-related
cortical circuits.
We test the hypothesis that the computations performed in these
circuits differ mainly in the level of sensorimotor abstraction at which they
operate. The suggestion is that the dorsolateral circuit structures grasping
movements according to abstract object knowledge, while the dorsomedial
circuit parameterizes movements based on currently available metric
information. We record the electrophysiological and kinematic responses
to two challenges designed to probe the functional dependencies between
these circuits. First, we manipulated the slant of the object to be grasped,
increasing the demands for accurate sensorimotor transformation. Second,
we applied TMS to perturb computations during movement planning in two
critical nodes of the dorsomedial and dorsolateral channels: the superior
parieto-occipital sulcus (sPOS) and the anterior intraparietal sulcus (aIPS).
When the accuracy demands of the sensorimotor transformation
increased, enhanced alpha power suppression emerged over medial
parieto-frontal regions. TMS over either aIPS or sPOS disturbed this effect,
but aIPS contributions preceded those of sPOS. These results indicate that
the dorsomedial and dorsolateral parieto-frontal circuits are functionally
commutable, but temporally dependent. We suggest that the dorsolateral
circuit provides an initial structure to the motor plan on the basis of abstract
object knowledge, a structure then used by the dorsomedial circuit as a
prior for guiding the parameterization of the movement operating on metric
information accumulated before motor execution.
p arie t o - f r o nt al cir cuit s d uring g r a s p | 143
Introduction
Controlling the hand and fingers to grasp an object appears fairly straightforward, at
least until we consider the wide range of object characteristics that are incorporated.
It has been suggested that the human brain controls grasping movements by using
particular object features to guide specific body parts 1,2 . The hypothesis is that the
arm is moved according to the position in space of the object to be grasped (extrinsic
properties). This transport component is controlled by a dorsomedial parietofrontal circuit that includes the human equivalent of area V6A (in the superior
parieto-occipital sulcus, sPOS) and the dorsal premotor cortex (PMd) 25,110,111,457.
The fingers are moved according to intrinsic object properties, such as object shape
and size. This grip component is controlled by a dorsolateral parieto-frontal circuit
that includes the anterior intraparietal sulcus (aIPS) and the ventral premotor
cortex (PMv) 113,152,184 . However, recent reports have highlighted a shared control of
transport and grip components by the two parieto-frontal circuit, arguing against
this functional dichotomy. For instance, both sPOS and aIPS encode intrinsic object
features during grasping (chapter 5 of this thesis) 98,109 and contribute to the guidance
of reaching movements to extrinsic object properties 110,458 .
Here, we test the hypothesis that the computations performed in the dorsomedial
and dorsolateral parieto-frontal circuits differ mainly in the level of sensorimotor
abstraction at which they operate, rather than by virtue of selectivity to particular
object properties or body parts 72 . The suggestion is that the dorsolateral circuit
structures grasping movements according to abstract object knowledge, while
the dorsomedial circuit parameterizes movements based on currently available
metric information. These computational characteristics give rise to a temporal
dependency between the contributions of these circuits: the dorsolateral circuit
constructs object-based movement templates that can be used by the dorsomedial
circuit as priors during motor planning.
This hypothesis is grounded on observations from macaque and human functional
anatomy. For instance, TMS studies have shown that dorsomedial contributions
follow those of the dorsolateral circuit 207,456 , with aIPS contributions occurring early
during the planning of a grasping movement and structuring the grasp plan on the
basis of abstract object knowledge (chapters 5 and 6 of this thesis) 40 . The presence
of multiple monosynaptic connections between these two parieto-frontal circuits
(sPOS – aIPS; PMd – PMv) points to the relevance of these interactions 106,204,224,459 .
This hypothesis predicts that the contributions of the dorsomedial and
dorsolateral circuits should be temporally distinct, but functionally commutable in
relation to visuospatial object properties or body parts. Accordingly, we recorded the
electrophysiological and kinematic responses to two challenges designed to probe
the functional dependencies between the dorsomedial and dorsolateral circuits.
7
14 4 | cha p t e r 7
First, we manipulated the slant of the object to be grasped, increasing the demands
for accurate sensorimotor transformation. Grasping a slanted object often requires
access to object knowledge, necessary to specify the location of invisible finger
end-points and potential hindrances from the object itself on the fingers’ trajectory.
Grasping a slanted object also creates strong dependencies between the control of
arm and finger movements during prehension 312,439 , without changing the intrinsic
size or shape of the object, nor its extrinsic location in space 1 . Second, we perturbed
neuronal processing in sPOS and aIPS with low-intensity single-pulse transcranial
magnetic stimulation (TMS), creating transient and localized disturbances to
either the dorsomedial or the dorsolateral circuit while subjects planned grasping
movements.
Methods
Subjects
The dataset used in this study was acquired as part of a larger dataset, elements of
which have been described elsewhere (chapter 6 of this thesis). These studies have
used the same cohort of subjects, applying the same inclusion and exclusion criteria
and basic data pre-processing steps. Thirty healthy, right handed subjects (sex: 9
males; age: 21 ± 2 (mean ± SD); handedness 403: 89 ± 11 (mean ± SD)) participated
in the experiment. All subjects conformed to standard EEG and TMS exclusion
criteria (including neither a personal history of neurological or psychiatric disorders,
epilepsy, migraine, or cardiac arrhythmia, nor in close relatives). All participants had
normal or corrected to normal vision and had a stereoscopic disparity threshold of
120 arcminute or higher (49 ± 29 arcminute (mean ± SD), TNO stereographs; Laméris
Ootech BV, Nieuwegein, the Netherlands). The study was approved by the local
ethics committee and a written informed consent was obtained prior to the start
of the experiment in accordance to the declaration of Helsinki. Two subjects were
excluded from further analyses because their mean reaction time was more than 1.5
times the inter-quartile range (IQR) below the first quartile of the group. The data
of four additional subjects were excluded from the electrophysiological analyses
because of high electrode impedances, muscle artefacts, or eye blink artefacts in
the electroencephalogram (EEG).
p arie t o - f r o nt al cir cuit s d uring g r a s p | 145
Experimentalsetupandtask
Subjects were seated at a table, with their head stabilized on a chin rest. Their right
hand was resting on a button box, pressing down the button marking the starting
position (home-key). They were facing two mechanical shutters that could be
opened and closed independently in 3.4 ms (Figure 1A) to control their vision. A black
prism (6 x 6 x 2 cm), serving as a target object, was positioned along the midsagittal
plane in front of the subject, displayed against a white background, at a comfortable
reaching distance (~25 cm in front and ~30 cm above the starting position of the right
hand). The object could be oriented in different slants around its pitch axis, in seven
steps between the frontal and transversal plane relative to the subject (0°, 15°,...,
90°). In the vertical orientation the object was upright, with the object’s largest
surface (6 x 6 cm) parallel to the subject’s frontal plane. With increasing slant, the
top of the object moved away from the subject, towards a horizontal orientation
parallel to table surface. It spanned a visual angle of ~7° when oriented vertically
(Figure 1A, 2A).
Each trial started with subjects holding their right hand on the home-key, their
vision occluded by closed shutters. After a random time interval (2-4 s), in which
the object was rotated to one of seven slants, either one or both shutters opened,
allowing monocular or binocular vision of the object to be grasped. As soon as
subjects started to grasp the object (i.e. they left the home-key), their vision was
occluded. Subjects grasped the object with their right hand, removed it from its
support, held it in mid-air, placed it back into its support, and returned to press
the home-key (Figure 1B). Subjects were instructed to perform the whole task at
natural speed. The absence of visual feedback during movement execution, and the
requirement to hold the object against gravity, further ensured that the subjects
prepared an accurate grasping movement.
A trial lasted between 6 and 11 s (average duration: ~8 s). Subjects performed
6 blocks of 84 trials each (~11 min per block), a total of 504 trials for the whole
experiment (~60 min including short breaks between each block).
Experimentaldesign
We considered three factors: (1) TMS-site (see below; 3 levels: vertex, sPOS and
aIPS), (2) object slant (2 levels: vertical [0°, 15°, 30°], horizontal [60°, 75°, 90°]), and
(3) object vision (2 levels: binocular, monocular right) (figure 7.1C). Additionally, in
post-hoc analyses, we considered the factor TMS-time (2 levels: early 100-200 ms,
late 300-400 ms).
The seven object slants were binned in two factorial levels of object slant (vertical
and horizontal) to comply with two analytical and statistical requirements. First, to
7
14 6 | cha p t e r 7
maximize the number of trials per analytical cell. Second, as described below, the
non-parametric permutation tests we performed to analyse the electrophysiological
data allows only the direct comparison of two levels per experimental factor. We
arbitrarily chose to provide monocular vision only to the right eye. We did not mix
left and right eye vision to keep the set of monocular trials homogeneous. To prevent
a preference for the right eye, also in binocular trials, we included left eye monocular
trials as catch trials (~8% of the total).
In summary, each cell of the 3 x 2 x 2 design contained 33 trials; a total of 396
trials were included in the analysis, and 108 additional trials were considered catch
trials (66 trials with objects slanted at 45°, and 42 trials allowing only monocular
vision of the left eye). Experimental conditions were pseudo-randomized and evenly
distributed across trials.
Experimentalintervention–TMS
TMS pulses were delivered through a figure-of-eight coil (70 mm diameter),
connected to a Magstim MonoPulse machine (Magstim Company, Whitland, Wales,
UK). On each trial, we delivered a single mono-phasic low-intensity TMS pulse at a
randomized time within two temporal windows (early: 100-200 ms after trial onset;
late: 300-400 ms after trial onset). During each of the six blocks of trials, TMS was
applied over one of three different sites, (1) the superior bank of the left parietooccipital sulcus (sPOS), (2) the anterior intraparietal sulcus (aIPS), the intraparietal
sulcus near the postcentral sulcus, and (3) the vertex of the head as a site controlling
for non-specific TMS effects. sPOS is a putative human homologue of macaque
visual area 6 (V6A). aIPS is a putative human homologue of the macaque anterior
intraparietal area (AIP). The order of TMS sites was counterbalanced within and
between subjects. The pulses were delivered at a particularly low intensity: at 100%
of the subject’s individual active motor threshold. In the following paragraphs we
will further elaborate on the anatomical localization and intensity of stimulation,
discussing both the employed protocols and consequences for functional specificity
of the perturbation by TMS.
Continuous online stereotactic guidance of the TMS coil was incompatible with
the experimental setup. Therefore, we mapped the sPOS and aIPS sites off-line with
respect to the 10-10 system for EEG electrode positioning in six additional reference
subjects, before we acquired the dataset used in this study450,451 . We transformed the
stereotactic coordinates of sPOS and aIPS (chapter 5 of this thesis) into the native
space of each reference subject as defined by their structural MRI scan (employing
an iterative unified normalization and segmentation procedure in SPM5, Statistical
Parametric Mapping; http://www.fil.ion.ucl.ac.uk/spm/) 406 . Using a stereotactic
neuronavigation system (Brainsight, Rogue Research Inc, Montreal, Quebec,
p arie t o - f r o nt al cir cuit s d uring g r a s p | 147
A experimental setup
B trial timecourse
shutters
bino
object
vision
mono
action
TMS
time
2-4 s
rest
home-key
D TMS sites
C experimental factors
vision
~0.7 s
planning
aIPS
binocular
x TMS site
monocular
vertex
sPOS
aIPS
~1 s
grasp
~2.5 s
hold & return
vertex
sPOS
vertical horizontal
slant
Figure7.1 Experimental setup and manipulations
A-B. The subject was instructed to prepare and perform a grasping movement,
grasping and displacing the target object before returning to the starting position.
At the start of each trial the subject was resting for a random interval (2-4 s) with
the right hand on a home-key, the head stabilized by a chin-rest, and the vision
occluded by shutter glasses. These glasses could open individually, providing either
monocular or binocular vision of the target object: a black prism measuring 6 x 6 x
2 cm positioned in front of the subject on the sagittal midline. The object could be
oriented at 7 different slants along the midsagittal plane, from vertical to horizontal
relative to the fronto-parallel plane [0°:15°:90°]. As soon as the subject released the
home-key the shutter glasses closed, preventing visual feedback of the movement.
During the planning phase a single TMS pulse was delivered between either 100200 ms or 300-400 ms after stimulus presentation. These early and late TMS time
windows are indicated on the trial time course with grey blocks. C. The experimental
design considered three factors: object vision (2 levels: binocular vision, in red;
monocular vision, in green); object slant (2 levels, vertical object slants [0°, 15°,
30°, in blue]; and horizontal slants [60°, 75°, 90°, in orange]); TMS-site (3 levels:
vertex, in grey; sPOS, in cyan; aIPS, in magenta). See Methods section for further
details.D. TMS target locations for all sites in one exemplary subject: the vertex of
the head (grey) serving as a control site, the superior bank of the parieto-occipital
sulcus (sPOS, cyan), and the anterior intraparietal sulcus (aIPS, magenta). The sites
are projected on a curvilinear reconstruction of the structural MRI of the exemplary
subject, shown here at 6 mm in depth from the cortical surface.
7
14 8 | cha p t e r 7
Canada), the TMS coil was positioned and oriented according to the subject-specific
aIPS coordinates (Figure 7.1D). The average configurations of the coil for optimal
sPOS and aIPS stimulation were marked on the EEG caps that were subsequently
used in the main experiment. The centre of the coil for sPOS stimulation fell just left
of the axis connecting electrodes POz and Pz, while for aIPS stimulation the centre
of the coil was close to electrode CP3. This offline procedure proved to be robust as
the positioning of the coil did not deviate more than 7 mm across the six additional
reference subjects, i.e. within the expected spatial range of the effective magnetic
field changes by the TMS pulse. The vertex site, serving as a control site, was defined
as the point where both the sagittal midline from nasion to inion and the coronal line
from ear to ear were dissected in the middle (Figure 7.1D).
All TMS pulses were delivered at 100% of the subject’s individual active motor
threshold (AMT). AMT was defined as the lowest intensity to elicit motor evoked
potentials, measured using standard electromyogram recordings, in at least 5 out of
10 successive stimulations (please see chapter 6 of this thesis for details) 449 . Average
AMT was reached at 33 ± 7% of maximum stimulator output. Subjects wore earplugs
both during AMT determination and during the experimental sessions to minimize
the acoustic effect of TMS delivery.
We chose to stimulate at this low-intensity for three reasons. First, to minimize
the somatic and auditory stimulation of the subject. Second, to minimize any TMS
artefacts in electroencephalographic and kinematic recordings (please see below
for details). Third, and most importantly, by lowering the intensity we increased
the effective functional specificity of the perturbation by the TMS pulse. When
stimulating at the active motor threshold the effectiveness of the pulse is critically
dependent on the activity of the underlying cortex. The neural processes of an area
that is endogenously activated by the task are more strongly perturbed than those
of an area that is not engaged by the task (when at a similar distance from the coil). In
this study we target the coil at the scalp-projected coordinates of a group activation
in sPOS and aIPS elicited during a similar task (chapter 5 of this thesis). Although the
locations of the intended target functional regions in sPOS and aIPS might differ
from subject to subject, the increased functional specificity of the TMS perturbation
by adopting a low-intensity could be expected to improve the consistency of the
functional interference of the TMS pulse across subjects.
Handkinematics–dataacquisition
We sampled the position and orientation of four sensors at 250 Hz, using an
electromagnetic tracking system (Polhemus LIBERTY, Polhemus, Colchester,
Vermont, USA) (see chapter 6 of this thesis for details). Three sensors captured
the subject’s hand, positioned on the nail of the thumb, the nail of the index finger,
p arie t o - f r o nt al cir cuit s d uring g r a s p | 149
and on top of the first metacarpophalangeal joint (MCPJ). The fourth sensor was
positioned on the object to be grasped, along the prism’s axis of rotation.
Handkinematics–dataanalysis
Kinematic data were analysed offline using a MATLAB toolbox developed in-house
(Mathworks, Natick, Massachusetts, USA), following a multi-step procedure
(chapter 6 of this thesis).
First, the TMS-induced artefact on the sensor time-series was removed (20 ms
window following the TMS pulse). The TMS pulse was delivered when the subject’s
hand was still resting on the home-key, allowing the interpolation of the artefactual
time window using a piecewise cubic Hermite polynomial.
Second, the time-series were low-pass filtered using a 6th order Butterworth
filter and a virtual sensor (referred to as the ‘grip sensor’) was defined as the mean
position of the thumb and index finger sensors.
Third, we calculated velocity and acceleration (defined as the first and second
order numerical gradient approximate derivatives of the position over time) for each
sensor. Furthermore, we calculated grip aperture (defined as the Euclidian distance
between the sensors placed on the thumb and index finger), grip orientation
(defined as the angle along the midsagittal plane between a vertical vector pointing
upwards and the thumb-index finger axis), and grip velocity (defined as the first
order numerical gradient approximate derivative of the grip aperture over time).
Fourth, in each trial we defined the on- and offsets of the phases of all movements.
The kinematic traces of the grasping and subsequent displacement movements did
not allow a simple ballistic description of the transport of the arm, hand and fingers.
The final phase of the grasping movement could include small corrective movements
based on somatosensory feedback, for example when one finger touched the object
before the other, leading to small changes in grip aperture and velocity. To appreciate
the different characteristics of the main displacement of the hand and fingers to the
object and the final phase, including small adjustments, we distinguished between
a transport phase of the movement, when the MCPJ is moving in a ballistic fashion,
and an approach phase, when the MCPJ is not moving but the thumb and index
fingers are still approaching the target surfaces of the object to be grasped. Given
this complex quality, we took particular care to robustly estimate the onset and
offset of the movement and its different phases. We employed a multiple sources of
information approach, formalized in both binary and continuous functions weighted
to find optimal points of transition 452 . Movement onset was defined as the point
in time when the fingers just started moving (>0.02 m/s) while they were near the
home-key, but their velocity and the grip aperture were as small as possible. To
identify the movement offset, the grip sensor must be and stay close to the centre
7
15 0 | c h a p t e r 7
of mass of the object, the grip velocity must be decreasing and stay low (<0.05 m/s),
while the velocities of the sensors and the grip, and the grip aperture must be as
small as possible. The transport phase ends and the approach phase begins when
the velocity of the MCPJ has reached its first minimum after falling below 0.3 m/s
without returning above that threshold. A detailed specification of these criteria can
be found in chapter 6 of this thesis.
Fifth, we calculated a series of parameters characterizing the kinematics of the
grasping movement. The movement planning and execution stages (encompassing
the transport and approach phase) were described using commonly used kinematic
parameters 243: reaction time (RT), movement duration (MT), trajectory length (TL),
mean velocity (MV), peak velocity (PV), relative time to peak velocity as a fraction of
MT (rtPV), maximum grip aperture (MGA), relative time to maximum grip aperture
as a fraction of MT (rtMGA). Additionally, to characterize the transport phase of
the movement we defined its specific duration (TMT), trajectory length (TTL), and
mean velocity (TMV). The approach phase of the movement was further described
by the approach grip aperture (AGA, the grip aperture at the start of the approach
phase), and the differential approach grip orientation (∆AGO, the grip orientation
at the start of the approach phase relative to the orientation measured 100 ms after
movement offset).
Handkinematics–statisticalinference
We excluded trials where the sensors moved outside the field of view of the Polhemus
system, trials where subjects started moving before or at the time of the TMS pulse,
and trials where the main kinematic parameters deviated by more than three IQRs
from the first or third quartile. Statistical inference of the kinematic measurements
was drawn using the SPSS software package (SPSS 16.0, SPSS Inc, Chicago, Illinois,
USA). All parameters were checked on skewness and kurtosis, and found to commit
to the assumption of normal distribution. Trials were averaged for each experimental
condition and subject; the resulting means were entered in a univariate repeated
measures ANOVA testing for effects between conditions within subjects.
EEG–dataacquisition
Encephalograms were acquired from 32 Ag/AgCl electrodes, organized on a flat-tip
cap (BrainProducts GmbH, Gilching, Germany) according to the 10-10 system 451 .
Electrical voltage was sampled at 5000 Hz using an amplifier with a high dynamic
range and capable of direct current recording (MR+ DC BrainAmp, BrainProducts
GmbH). The EEG data was low-pass filtered at 1000 Hz and subsequently digitized.
Electrical artefacts associated with the TMS pulse were minimized during acquisition
p a r i e t o - f r o n t a l c i r c u i t s d u r i n g g r a s p | 151
by (1) the use of tip electrodes (rather than classical circular electrodes), and by (2)
positioning the EEG cables perpendicularly to the orientation of the TMS coil when
applicable. Furthermore, (3) no high-pass filter was applied at acquisition to minimize
the temporal spread of TMS induced artefacts.
EEG–dataanalysis
Electrophysiological data were analysed offline using a MATLAB toolbox (FieldTrip,
http://www.ru.nl/neuroimaging/fieldtrip) 453 , following a multi-step procedure.
First, we excluded trials with eye movements, blinks, and muscle artefacts on
the basis of visual inspection of the data.
Second, each EEG channel was screened in a 1 second window following the TMS
pulse for several known types of TMS induced artefacts. Data in the time-window
where the signal could suffer from saturation or consequent ringing artefacts caused
by amplifier electronics and the 1000 Hz low-pass filter (-0.2 ms to +10 ms relative
to the TMS pulse) were discarded and not included in further analyses. Slow stepresponses (lasting to ~60-400 ms after the TMS pulse) 370,371 were fitted and removed
using an iterative least-squares optimization algorithm (implemented in FieldTrip)
when possible, or otherwise the trial was discarded.
Third, the data were re-referenced to the average signal of all channels to remove
any spatial effects on voltage differences with respect to the fixed localization of the
reference electrode on the EEG-cap.
Fourth, all trials surviving the exclusion criteria were entered into an independent
component analysis383,384 . Residual signals related to eye movements, blinks, muscle
tension and TMS artefacts were identified and removed379 .
Fifth, power line noise was removed using discrete Fourier transform notch
filters at 50, 100 and 150 Hz. The resulting electrophysiological data were band-pass
filtered between 0.75 and 150 Hz using a 6th order Butterworth filter and downsampled to 500 Hz.
Sixth, time-frequency representations (TFRs) of oscillatory power were
calculated for each trial and sensor. We focused on the alpha (9-12 Hz) and beta
(18-24 Hz) frequency bands, using a Fourier transform approach (8-35 Hz, in steps
of 1 Hz) applied to sliding time windows (sliding in 20 ms steps) multiplied by a
Hanning taper (resulting in a frequency smoothing of 5 Hz). We performed the timefrequency analyses with two different time-window durations, one of 200 ms and
one lasting a more conventional 400 ms. The relatively short sliding time window
of 200 ms was chosen to prioritize temporal above spectral specificity, allowing
estimation of oscillatory power shortly before and after the TMS pulse, while still
maintaining enough spectral specificity to clearly separate the frequency bands of
7
152 | c h a p t e r 7
interest. Using these configurations the theoretical time bins with EEG data closest
to a given TMS pulse were centred at 100 ms before (window: [-200 0] ms) or 105 ms
after (window: [10 210] ms) the pulse. Combined with the fact that the Hanning taper
was 200 ms wide and slid with 20 ms intervals, this resulted in a time window of 220
ms lacking oscillatory power estimation in each individual trial. However, it should
be emphasized that by employing this short time-window for the Hanning taper we
were able to generate a continuous average estimate of power in the alpha and beta
bands, un-affected by TMS intervention. Given the trial-by-trial temporal variation
in TMS delivery (see Experimental intervention - transcranial magnetic stimulation
above), the time window from which no power estimate could be obtained on each
individual trial occurred at different time points across different trials. The Hanning
taper time-window of 400 ms was employed for analyses where the temporal
specificity was less crucial, i.e. when the analyses focused on the execution phase
of the movement, with epochs further away in time from the disruptive TMS pulse.
Seventh, for each experimental condition, power estimates were averaged
over trials, log-transformed, and related to a baseline period from the same trials
(relative change from [-700 -200] ms before stimulus presentation). Subsequently,
the individual effects were grand averaged over subjects to report on group effects.
EEG–statisticalinference
Within each subject, the difference between conditions was quantified as the
difference of the relative log transformed mean power changes. Statistical inference
(p<0.05) was performed on the group level (random effects analysis) using a nonparametric randomization test controlling for multiple comparisons over the large
search space constituted by multiple sensors, frequency bands, and time intervals
(SFT points) 454 . This procedure involves the following steps.
First, all SFT points are identified for which the t-statistics for the difference
between conditions over subjects exceeds an arbitrary threshold (p>0.05,
uncorrected).
Second, continuous SFT points exceeding the threshold are grouped in clusters,
the t-statistics from each SFT point of a cluster are added, and this cumulative
t-statistic is used for inferential statistics at the cluster-level.
Third, a Monte Carlo estimate of the permutation p-value of the cluster is
obtained by comparing the cluster-level test statistic to a randomization null
distribution assuming no difference between conditions. This distribution is obtained
by randomly swapping the conditions within subjects and calculating the clusterlevel test statistic 10000 times, resulting in an accurate estimate of the true p-value.
p a r i e t o - f r o n t a l c i r c u i t s d u r i n g g r a s p | 153
Results
Behaviouraleffects:taskperformancefollowingTMS
oververtex
Effect of object slant: Increasing object slant from vertical to horizontal requires many
basic changes in kinematic behaviour. First of all, the orientation of the grip needs to
be adjusted to the orientation of the object, which is mainly addressed by a flexion
of the wrist. Second, the subject’s hand needs to travel over a longer path. Third, the
direction of final approach of the fingers relative to the principal displacement of the
hand changes with object slant, in order to maintain a stable approach angle relative
to the target surfaces. When the object is slanted vertically, the final approach of the
fingers is perpendicular to the principal displacement direction, while it is tangential
for horizontally slanted objects. The kinematic measurement clearly isolated these
basic effects (main effect of slant within the vertex session; see table 7.1; figure 7.2).
Subjects travelled a longer path at the same speed, resulting in longer movement
durations (MT, TL, TMT and TMV), while they reached a higher peak velocity earlier
in the movement (PV and rtPV). The grip orientation at the onset of approach phase
followed the object’s orientation, but while the grip orientation matched the object
closely when the object was slanted vertically, for more horizontal slants the grip
was slanted further back on approach compared to the object (∆AGO). In contrast to
these widespread effects on the grasping kinematics, object slant did not influence
the duration of movement planning (RT: F(1,22)=0.26, p=0.615).
Grip shape and aperture as a function of object slant: We observed that with increasing
slant the maximum and approach grip aperture became smaller, while the maximum
grip aperture occurred relatively later (MGA and rtMGA in table 7.1; figure 7.2E-F;
figure 7.3A). This finding has been consistently reported in the literature314,439,460 ,
although the interpretations of its causes vary. For instance, the reduced maximum
grip aperture evoked by horizontally-oriented objects has been related to a change
in planning uncertainty262 , to a perceived foreshortening of the object314 , and to a
change in approach strategy and confidence439 . In the present experimental setup, it
appears that the reduction in grip aperture with increasing slant is a consequence of
skeletomuscular limitations. As clarified below, this is suggested by the fact that the
difference in grip aperture between different target slants arose surprisingly early, at
around 30% of the total movement time, in accordance with hand shape constraints.
The size of the effect of slant was stable over time, until the maximum grip
aperture was reached, and largely independent of the grip aperture itself. We plotted
the position of the three sensors on the hand at 60% of the movement time, viewed
perpendicularly to the plane defined by these points and relative to the thumb
7
15 4 | c h a p t e r 7
Behavioural effects - TMS over vertex
movement time
PV (cm/s)
MT (ms)
1100
C
1050
1000
950
52
51
50
49
peak velocity
106
104
102
E
100
µ transport velocity
D
rtPV (%MT)
TMV (cm/s)
B
108
MGA (cm)
1150
34
32
30
28
12.5
max grip aperture
12.0
11.5
11.0
rel. time to PV
F
rtMGA (%MT)
A
72
rel. time to MGA
70
68
66
64
Figure 7.2. Behavioural effects after TMS over vertex (control session). A-F. Bar
plots showing kinematic parameter values of the slant x vision interaction (mean ±
standard error). Colour coding is as in figure 7.1C: bars are grouped by object slant
(blue: vertical; orange: horizontal), the bar face colour codes for object vision (red:
binocular; green: monocular). A. Grasping movement durations (encompassing
both the transport and approach phases) increased with increasing object slant and
were longer in trials with monocular compared to binocular vision. B. The mean
transport velocity was only affected by vision, being smaller following monocular
viewing conditions. C-D. In trials where the object was slanted more horizontally
the peak velocity of the virtual grip sensor (mean of thumb and index finger sensors)
was larger and reached later in the movement. E-F. The maximal grip aperture was
smaller and reached later when grasping near horizontal compared to near vertical
objects.
sensor (figure 7.3B). This revealed that the shape of the grip changes asymmetrically
with target slant, more specifically, with increasing slant the position of the MCPJ
relative to the median of the thumb-index axis is shifted further back in the direction
of the thumb. We formalized grip shape asymmetry as the distance between the
intersections of the altitude and the median of the MCPJ with the thumb-index
axis (figure 7.3B-C). This measure of hand shape is independent from grip aperture
under the assumption that both the thumb and index finger are able to move and
contribute equally to the opening of the hand. In a canonical, comfortable grip the
MCPJ amplitude is bisecting the line connecting the thumb and index finger and the
p a r i e t o - f r o n t a l c i r c u i t s d u r i n g g r a s p | 155
asymmetry measure will be close to zero. When the MCPJ is skewed further back
the degree of asymmetry will become more negative, reaching its maximum when
the thumb and index finger are pointing forward along the same line. A negative
asymmetry restricts the degrees of freedom to form a hand shape and reduces the
potential grip aperture.
Multiple factors might contribute to the more negative asymmetry observed
when grasping more horizontally slanted objects. First, when the object is oriented
in a horizontal fashion, the target points for the individual digits lie along roughly
the same line as seen from the starting position. Combined with the preference for
a perpendicular approach of the target surfaces with the independent digits, this
might result in a more asymmetric hand shape, limiting the grip aperture 251 . Second,
when a horizontal object is grasped, the hand and arm need to be lifted higher and
the wrist is flexed further. Optimization of muscle force and joint torque when
grasping an object and avoiding obstacles (which might be the same object) might
also lead to a more asymmetric grip towards more horizontally slanted objects 461 .
The temporal development of the difference in grip aperture as a function of
object slant can inform us about its potential causes. If the grip aperture variation
would be a reflection of changes in planning uncertainty, than the difference
Behavioural effects - TMS over vertex
B
8
4
0
grip shape
12
8
a
median
4
b
0
0 20 40 60 80 100
relative movement time (%)
0
4
8
distance (cm)
C
grip asymmetry (cm)
grip aperture (cm)
12
distnance (cm)
A
grip aperture
1
grip asymmetry
0
-1
-2
-3
asymmetry = a-b
2
-4
0 20 40 60 80 100
relative movement time (%)
Figure 7.3. Effect of object slant on grip after TMS over vertex (control session).
A. Development of grip aperture relative to total movement duration. The effect
of object slant on grip aperture was already pronounced relatively early during the
movement. B. Average relative sensor positions (constituting the grip shape) at 60%
of the movement time. The sensor positions are represented relative to the thumb,
irrespective of object and hand orientation (i.e. projected on the plane encompassing
all three sensors). We described the shape of the grip using an asymmetry index: the
distance in cm between the point where the median and the amplitude of the MCPJ
intersect with the line connecting the thumb and index sensors (a-b/2). A negative
grip asymmetry limits the grip aperture. C. Development of grip asymmetry relative
to total movement duration. The grip asymmetry can explain the deviations in grip
aperture with object slant as depicted in panel A. Other conventions as in figure 7.2.
7
156 | c h a p t e r 7
would be expected to be most pronounced around the moment of maximum grip
aperture (~60% of MT) 243,262 . Alternatively, if the grip aperture would be reduced
towards horizontally slanted objects as a consequence of a foreshortened view of
the object314 , than this difference would be expected to sustain until corrected by
feedback of the grip in relation to the object. In this experiment, visual feedback
was withheld during movement execution, leaving only somatosensory information
acquired at the end of the movement phase when the object is touched to correct an
erroneously anticipated reduction in required grip aperture for horizontally slanted
objects. In fact, the temporal development of the grip aperture we observed is not
in agreement with these two explanations. The change in grip aperture for different
target object slants arose very early in the movement period (30% of MT), sustained
without peaking around the maximum grip aperture, and disappeared well before
somatosensory feedback could be incorporated after object contact (figure 7.3A).
These temporal dynamics are in accordance with the development of the grip
asymmetry, which diverged at the same early point during the movement period
and remained stable, irrespective of the grip aperture itself (figure 7.3C), suggesting
that skeletomuscular constraints cause the observed reduction in grip aperture with
increasing object slant.
Effect of vision: When binocular vision of the object was allowed subjects took slightly
longer to plan the grasping movement compared to trials when only monocular
cues were available. This increase in reaction time might be a consequence of the
additional time required to converge and focus on the object when binocular vision
was available. However, the shorter reaction time evoked during monocular trials
resulted in larger planning uncertainties, as illustrated by longer movement duration,
longer travelled path and reduced mean velocity in the transport phase (table 7.1;
figure 7.2A-B). Differential effects of vision on grasping kinematics as a function of
object slant are described in another report (chapter 6 of this thesis).
Behaviouralresults–taskperformancefollowingTMS
oversPOSandaIPS
Effect of TMS-site: Stimulation of sPOS had no significant main effect on any of
the behavioural parameters tested in this study. After stimulation of aIPS subjects
started moving slightly sooner after stimulus presentation with respect to control
stimulation, irrespectively of object slant or viewing conditions (table 7.1).
Effect of TMS-site x slant and TMS-site x vision interactions: The site x slant
interaction did not elicit any significant behavioural effects, however, the site x vision
interaction modulated a few parameters. After stimulation of either sPOS or aIPS,
compared to vertex stimulation, subjects tended to have a shorter transport phase
reaction time
movement duration
trasport duration
trajectory length
transport trajectory
mean velocity
mean transport velocity
peak velocity
rel. time to peak velocity
maximum grip aperture
rel. time to max. grip apt.
approach grip aperture
Δ approach grip orientation
RT
MT
TMT
TL
TTL
MV
TMV
PV
rtPV
MGA
rtMGA
AGA
ΔAGO
parameter
0.048 (4.39)
0.000 (23.57)
0.003 (11.01)
0.009 (8.12)
0.013 (7.38)
0.000 (17.95)
0.000 (30.73)
0.000 (103.89)
0.000 (337.76)
0.000 (289.89)
0.029 (5.49)
0.000 (132.21)
0.000 (49.89)
0.000 (30.41)
0.047 (4.43)
0.030 (5.38)
0.043 (4.59)
0.046 (4.47)
0.000 (75.33)
0.000 (25.26)
0.042 (4.64)
0.069 (3.66; ns)
0.016 (6.74)
sitexvision
0.012 (7.44)
0.095 (3.04; ns)
site
0.029 (5.44)
0.046 (4.46)
vision
sPOSvs.vertex
0.035 (5.03)
site
0.010 (7.85)
0.087 (3.21; ns)
sitexvision
aIPSvs.vertex
as no parameter showed a significant effect. The interaction of
slant x vision is described in detail in another report (chapter 6
of this thesis) and as such not included here. Statistical inference
was performed using two univariate repeated measures ANOVA.
To maintain clarity only statistics with associated probabilities
below α=0.10 are shown. Trends with 0.05<p<0.1 are printed in
italics and labelled with ‘ns’.
slant
TMSoververtex
Statistical probabilities and F-statistics (in brackets; df: 1,22)
of investigated kinematic parameters for experimental effects
after control stimulation (vertex) and for the differential effect
between stimulation over vertex vs. sPOS and vertex vs. aIPS.
Presented are the main effects of object slant and vision, the
main effect of stimulation site, and the interaction effects of
site x vision. The interactions of site x slant are not represented
Table7.1 Statistical probabilities and F-statistics of behavioural effects
p a r i e t o - f r o n t a l c i r c u i t s d u r i n g g r a s p | 157
7
15 8 | c h a p t e r 7
of the movement following monocular vision (TMT), diminishing the difference
in duration that was exhibited after control stimulation. After sPOS stimulation a
higher mean transport velocity accompanied this change during monocular trials
(TMV). Surprisingly, a higher peak velocity was elicited following binocular vision,
although it was reached slightly sooner during monocular trials (PV and rtPV).
Oscillatory effects of the task - TMS over vertex
time-locked to stimulus onset
time-locked to movement onset
A
beta
18-24 Hz
power change (%log)
60
B
alpha
8-12 Hz
0 - 0.5 s
C
left motor
C3 ( )
frequency (Hz)
0 - 0.5 s
0.5 - 1.0 s
30
TMS site
vertex
sPOS
aIPS
-20
20
15
-40
10
-60
0
25
-20
20
-40
15
-60
10
30
α-power change (%log)
0
0
20
25
-20
-20
20
15
-40
-40
15
10
-60
-60
10
-0.5
0
0.5
time from stimulus (s)
1
-0.5
0
0.5
time from stimulus (s)
1 -0.5
0
0.5
1
time from movement (s)
-0.5
0
0.5
1
time from movement (s)
Figure7.4 Oscillatory effects after TMS over vertex (control session)
Time-locked either to the onset of the stimulus (two leftmost columns) or the
movement (two rightmost columns). A-B. Topographies of oscillatory power
changes in the beta band (18-24 Hz; A) and in the alpha band (8-12 Hz; B) relative to
baseline ([-700 -200] ms before opening of the shutters) measured during the vertex
session. There were strong power reductions over left motor cortex (beta band;
electrode C3; diamond) and over visual cortex (alpha band; electrodes O1, Oz and
O2; stars). C-D outer columns. Power changes over time in electrode C3 (C) and O1Oz-O2 (D), showing that the power reduction in the alpha and beta bands covered
the whole duration of the motor planning phase. C-D central columns. Temporal
dynamics of alpha and beta power changes at C3 and O1-Oz-O2 as a function of time
(vertex: grey, sPOS: cyan, aIPS: magenta) reveal no main effect of TMS site. Shaded
areas around the grand average curves represent standard error of the mean.
frequency (Hz)
25
frequency (Hz)
frequency (Hz)
-60
β-power change (%log)
0
30
occipital
O1, Oz, O2 ( )
0
-30
0.5 - 1.0 s
30
D
30
p a r i e t o - f r o n t a l c i r c u i t s d u r i n g g r a s p | 159
EEGresults–maineffects
Effect of task: Stimulus presentation and subsequent motor preparation led to
strong suppression of both alpha (8-12 Hz) and beta (18-24 Hz) oscillations (figure
7.4), a known marker of increased computational load in the visual and motor
systems 318,321,329 . Over occipital electrodes, alpha power was initially reduced (while
subjects were allowed sight of the object), but it returned to and above baseline
after movement onset (when object vision was prevented; figure 7.4B,D, right
panels). In contrast, over left and right motor/somatosensory cortices, alpha power
remained suppressed during movement execution (figure 7.4B-C, right panels). Beta
Differential oscillatory effects of TMS-site
sPOS vs. vertex
α-power change (%log)
10
5
0
-5
-10
0.24 - 0.92 s
CP5 ( )
-30
-45
-50
0
-20
-20
-40
-40
0
0.2 0.4 0.6 0.8 1
time after stimulus (s)
site x
TMS-time
-40
0
-60
early late
-35
after stimulus onset
α-power chagne (%log)
B
C
α-power change (%log)
A
vertex
sPOS
aIPS
-60
-0.4 -0.2 0 0.2 0.4 0.6
time after movement (s)
Figure 7.5. Differential oscillatory effects of TMS over sPOS. A-C. Main effect of
TMS over sPOS compared to vertex on oscillatory power. Significant differences
(p<0.05 corrected for multiple comparisons) were only found in the alpha frequency
band (8-12 Hz). A. Topography of significant differential main effect between sPOS
and vertex stimulation in the alpha band.B. Development of average power in the
alpha band over time plotted separately for trials in vertex (grey) and sPOS (cyan).
Significant time windows are indicated with grey boxes, relative to stimulus (left
panel, [240 920] ms) and movement onset (right panel, [-360 600] ms) C. Power
averaged over significant time and frequency points (grey boxes in panel B) in
electrode CP5 plotted as a function of TMS-site and TMS-time (early, 100-200 ms:
darker shades; late, 300-400 ms: lighter shades). The enhanced alpha suppression
in CP5 was only present following sPOS stimulation and not affected by the timing
of the TMS pulse delivery.
7
16 0 | cha p t e r 7
Differential oscillatory effects of object slant
A
time-locked to stimulus onset
B
time-locked to movement onset
α-power change (%log)
10
5
0
-0.06 - 0.06 s
-5
0.06 - 0.18 s
0.18 - 0.30 s
-10
0.48 - 1.00 s
after stimulus onset
0.42 - 0.54 s
E
0
Fz ( )
-20
-40
-60
F
0
-20
-40
-60
0
0.2 0.4 0.6 0.8 1
time after stimulus (s)
-0.2 0 0.2 0.4 0.6
time after movement (s)
α-power chagne (%log)
0.30 - 0.42 s
POz, Pz ( )
α-power chagne (%log)
horizontal
α-power chagne (%log)
parietal
POz, Pz ( )
D
α-power change (%log)
frontal
Fz ( )
C
effect of slant
vertical
-20
slant x vision
-25
-30
-35
-40
-30
-35
-40
-45
-50
Figure7.6 EEG effects of object slant - TMS over vertex (control session)
A. Differential topography of power in the alpha band after control stimulation
as a factor of slant in the significant time window (time-locked to stimulus onset;
p<0.05 corrected for multiple comparisons). Alpha is suppressed stronger during
preparation and execution of a grasping movement towards more horizontally
compared to vertically slanted objects.B. Differential topographical representations
of the effect of object slant on alpha power, time locked to movement execution.
The development of the topography over time is shown by 5 sequential timewindows spanning the significant epoch ([-60 440] ms relative to movement onset).
C. Changes in oscillatory power over time in electrode Fz (indicated with a diamond
in panel A) shown separately for trials with vertical (blue) and horizontal objects
(orange) and the differential contrast (black), time-locked to the stimulus (left panel)
and movement onset (right panel). Significant time windows are indicated with
grey boxes ([480 1000] ms after stimulus onset, left panels; [-60 440] ms relative to
movement onset, right panels). D. Changes in average power of electrodes POz and
Pz (indicated with stars in panel A).
p a r i e t o - f r o n t al cir c ui t s d u r in g g r a s p | 161
oscillatory power was most strongly suppressed over the motor and somatosensory
cortices contra-lateral to the moving hand during both preparation and execution
(figure 7.4A,C).
Effect of TMS-site: The main effects of task described above were largely unaffected
by the site of transcranial magnetic stimulation (figure 7.4C-D, middle panels),
with one important exception. TMS over sPOS (compared to a control stimulation
over the vertex of the head) lead to a stronger suppression in the alpha frequency
band over left lateral parietal cortex, localized around electrode CP5, sustained
throughout movement planning and the early stages of execution (p=0.014, figure
7.5A-B). These characteristics indicate that this effect is different from the beta
band suppression evoked by preparatory motor activity and centred on C3 (figure
7.4A). The alpha suppression evoked by TMS over sPOS was neither modulated by
the type of vision allowed (binocular or monocular: post-hoc p=0.96), nor by the
timing of the TMS pulse (between 100-200 or 300-400 ms after stimulus onset: posthoc p=0.83, figure 7.5C). Crucially, the effect was not accompanied by a change in
kinematic behaviour (table 7.1), suggesting that the enhanced alpha suppression in
lateral parietal regions might have a compensatory function following perturbation
of sPOS by TMS.
EEGresults–effectofvision
Effect of vision: No significant differences in oscillatory power were found between
trials with monocular and binocular vision.
Effect of TMS-site x vision interaction: There were also no significant interactions
observed between TMS-site and object vision.
EEGresults–effectofslantfollowingvertexstimulation
Effect of object slant: As object slant increased from a vertical to horizontal
orientation, there was increased alpha suppression over medial parietal and frontal
areas (p=0.004; figure 7.6A-B). Differently from the task-related alpha suppression
Figure7.6 continued
E-F. Changes in power (for electrode Fz in panel E; electrodes POz-Pz in panel F)
averaged over the significant time and frequency points (grey boxes in panel C
and D) are plotted separately for the slant (vertical: blue; horizontal: orange) and
vision (binocular: red; monocular: green) conditions. These contrast estimates show
that the effect of slant is present following both monocular and binocular viewing
conditions.
7
162 | cha p t e r 7
observed over occipital electrodes (figure 7.4B,D), this slant-related alpha
suppression arose just prior to movement onset and persisted during the first half
of the movement, when vision of the hand and object was prevented (figure 7.6C-D).
Effect of slant x vision interaction: The enhanced alpha suppression in trials where
the object is presented in a more horizontal orientation was not modulated by
the type of depth information allowed by vision, either stereoscopic (binocular) or
pictorial (monocular) (post-hoc p=0.65; figure 7.6E-F). Importantly, stereoscopic
depth cues are more informative for estimating the configuration of horizontally
than for vertically slanted objects, while the information value of pictorial cues is
inversely dependent on object slant 294 . Therefore, observing the same relationship
between alpha suppression and object slant irrespective of the visual condition
entails that this effect is not dependent on the information value of the available
depth cues. Effects of the slant x vision interaction on beta suppression are reported
in chapter 6 of this thesis.
EEGresults–effectofslantaftersPOSandaIPS
stimulation
Effect of TMS-site x slant interaction: Stimulation of both sPOS and aIPS diminished
the slant-related alpha suppression observed over medial parieto-frontal cortex
after vertex stimulation (sPOS: p=0.017; aIPS: p=0.006; figure 7.7A). The spatial
distribution of these perturbation effects strongly overlapped with the slant-related
alpha suppression observed after vertex stimulation (figure 7.6A-B), but the timing
of those effects differed. sPOS stimulation induced a late alteration of the slantrelated alpha suppression ([300 580] ms after movement onset; figure 7.7B-C, left
panels), whereas aIPS stimulation altered the slant-related alpha suppression early
during movement performance ([-40 380] ms relative to movement onset; figure
7.7B-C, right panels). In this period, stimulation over the two parietal sites also
differed significantly from each other ([80 260] ms after movement onset; p=0.025).
Effect of TMS-time x TMS-site x slant interaction: As expected, during control
stimulation the size of the alpha enhancement for horizontally slanted objects was
not affected by the timing of the TMS pulse (p=0.66). However, post-hoc analyses
revealed that the perturbation was significantly stronger when the TMS pulse was
delivered over sPOS in the late time-window, between 300-400 ms after stimulus
presentation (p=0.026; figure 7.7D-E). No significant difference effect of the timing
of the TMS pulse was found in aIPS sessions, neither when directly compared to the
effect of TMS-time during vertex sessions (p=0.34), nor when compared to the sPOS
sessions (p=0.29).
p arie t o - f r o nt al cir cuit s d uring g r a s p | 163
Differential oscillatory effects of object TMS-site x object slant
sPOS vs. vertex
aIPS vs. vertex
A
effect of slant
(horizontal > vertical)
after stimulation over:
α-power change (%log)
10
5
vertex
sPOS
aIPS
0
-5
-10
Fz ( )
0
-20
-40
-60
E
0
-20
-40
-60
-0.2 0 0.2 0.4 0.6 0.8
time after movement (s)
0 0.2 0.4 0.6 0.8
time after movement (s)
α-power change (%log)
α-power chagne (%log)
D
POz, Pz ( )
parietal
POz, Pz ( )
C
α-power chagne (%log)
frontal
Fz ( )
B
-0.04 - 0.38 s
after movement onset
α-power change (%log)
0.30 - 0.58 s
after movement onset
8
site x slant x
TMS-time
early late
4
0
-4
-8
vertex
sPOS
aIPS
vertex
sPOS
aIPS
8
4
0
-4
-8
Figure7.7 EEG effects of TMS-site x object slant
A-E. Oscillatory effects of the site x slant interaction contrasting stimulation
over sPOS (left column) and aIPS (middle column) with vertex. A. Differential
topographies of alpha power, representing significant modulation of the effect of
slant (grasping horizontal > vertical objects) by stimulation site. The stronger alpha
suppression after control stimulation for horizontal objects is disrupted both by
sPOS and aIPS stimulation, but these disruptions differ in the temporal domain (aIPS:
[-40 380] ms; sPOS: [300 580] ms relative to movement onset).B-C. The differential
effects of slant are plotted separately following vertex (grey), sPOS (cyan), and aIPS
stimulation (magenta) for electrode Fz (panels B) and the average of electrodes
POz and Pz (panels C). Significant time windows are indicated with grey boxes.
D-E. The contrast estimates of the slant effect are plotted as a function of TMS-site
and TMS-time (early, 100-200 ms: darker shades; late, 300-400 ms: lighter shades).
The contrast estimates are averaged over the significant time and frequency points
(grey boxes in panel C and D) and represented separately for electrode Fz (panel E)
and the average of electrodes POz-Pz (panel F). The perturbation of the slant effect
by sPOS stimulation was significantly stronger following TMS in the late (300-400
ms after stimulus onset) compared to the early time-window (100-200 ms).
7
16 4 | cha p t e r 7
Discussion
We have investigated parieto-frontal circuits supporting visuomotor transformations
during planning and execution of grasping movements, focusing on the temporal
dynamics of aIPS and sPOS contributions. We challenged the visuomotor system
by asking subjects to grasp an object oriented at different slants, while maintaining
constant object position, shape, and size. When subjects grasped horizontally slanted
objects, putting high demands on accurate visuospatial parameterization, there was
enhanced suppression of oscillatory power in the alpha band (8-12 Hz) over medial
parietal and frontal areas, starting shortly before movement onset. This slantrelated alpha suppression effect indicates that object orientation is incorporated in
the grasp plan towards the end of the preparation stage, and confirms that these
computations are implemented in the dorsomedial parieto-frontal circuit (chapter 5
of this thesis). When this circuit was challenged by applying TMS over sPOS during
planning, the slant-related alpha suppression over dorsomedial parieto-frontal
regions was disturbed late during movement execution. A spatially and spectrally
overlapping effect was observed after TMS over aIPS, but starting earlier, shortly
before movement onset. These findings indicate that sPOS contributions to the
control of grasping follow those of aIPS.
Objectorientationisincorporatedinthegraspplanata
latestage,bythedorsomedialcircuit
Planning and executing a grasping movement led to strong suppression of oscillatory
alpha power over occipital, parietal and frontal electrodes (figure 7.4B), an indication
of increased computational load over those regions 321,328,329 . When the slant of the
target object was increased, alpha suppression over dorsomedial parieto-frontal
regions increased further, starting shortly before movement onset (figure 7.6). This
scalp topography matches the spatial distribution of increased BOLD activity evoked
by grasping objects at increasing slants, previously found to be centred on sPOS and
PMd (chapter 5 of this thesis). Here we show that the slant-related alpha suppression
measured over the dorsomedial circuit starts late during movement planning and
continues during the first part of the movement. The timing of this effect closely
matches that of visuomotor neurons recorded from area V6A in macaques 60,108 ,
and it fits with the suggestion that the dorsomedial circuit supports grasping by
fine-tuning the motor plan on the basis of information accumulated before motor
execution (chapters 5 and 6 of this thesis). The enhanced alpha suppression with
increased object slant, i.e. with increased visuospatial kinematic demands, was not
dependent on the information value of the provided depth cues. This highlights that
p arie t o - f r o nt al cir cuit s d uring g r a s p | 165
the processes in this dorsomedial circuit directly concern the required motor output,
and are not fully driven by the visual input.
sPOSandaIPSsupportgraspingatdifferenttimes
The dorsomedial parieto-frontal distribution of the slant-related alpha suppression
provides confirmatory evidence that the dorsomedial circuit is selectively involved in
adjusting the motor plan to the current object orientation (chapter 5 of this thesis).
Accordingly, interfering with either sPOS or aIPS processing during movement
planning led to a reduction of the slant-related alpha suppression over the same
dorsomedial circuit (figure 7.7A). This finding fits with the observation that both
sPOS and aIPS contains neurons coding object orientation 98,108 , and that these
two regions are directly connected in macaques 106,204 . Crucially, there were clear
differences in the timing of the consequences of interference with either sPOS or
aIPS. The early component of the slant-related alpha suppression effect, around
movement onset, was disturbed by aIPS stimulation, whereas sPOS stimulation
affected a later component of that effect. The temporal asymmetry of this effect
suggests that sPOS computations could be influenced by target features processed
in aIPS early during planning. This notion is further substantiated by the observation
that the information value of visual depth cues did not directly influence the slantrelated alpha suppression effect. This confirms that the dorsomedial circuit can
not only specify motor parameters on the basis of raw visual cues of depth directly
accessible from early visual areas 60 , but also on abstract object knowledge, as for
instance processed in the dorsolateral stream (chapters 5 and 6 of this thesis).
ThedorsolateralcircuitcancompensateforsPOS
perturbationduringgraspplanningandexecution
When sPOS activity was disturbed during motor preparation with low-intensity
single-pulse TMS, there was a sustained increase in alpha suppression over the left
aIPS region, starting early in the movement planning and continuing well into the
execution phase (figure 7.5). The timing and topography of this effect was clearly
different from the slant-related alpha suppression described above (figure 7.6CD). The sPOS interference did not give rise to any detectable changes in grasping
kinematics, suggesting that the dorsolateral circuit could compensate for a transient
sPOS alteration, as reflected in the increase in alpha suppression over the left aIPS
region. It remains to be seen how exactly this compensation occurs. One possibility
is that the dorsolateral circuit gathers abstract object knowledge and uses these
priors to constrain the grasp plan, complementing processing of (insufficient)
visuospatial cues (chapter 6 of this thesis). This suggestion fits with prehensile
7
16 6 | cha p t e r 7
behaviour observed in patients with optic ataxia, i.e. patients with bilateral lesions in
superior parieto-occipital cortex 11 . They are impaired when asked to grasp a simple
cylindrical object they have not encountered before. Surprisingly, they show spared
prehensile behaviour towards familiar objects of the same size and shape, a tube of
lipstick for example 14 . Interestingly, if the novel objects are presented repeatedly,
the initially impaired hand shaping improves with increasing familiarity115 . The
behaviour of these patients suggests that object knowledge and prior experience is
sufficient to structure their grasping movements.
Interpretationalissues
There is a temporal gap between TMS delivery (100-200 or 300-400 ms after object
presentation) and the slant-related alpha suppression effect (480-1000 ms after
object presentation). This gap excludes that the interference effect is a consequence
of different cortical excitability at the time of TMS intervention, and it suggests that
disturbing ongoing computations (as indexed by the overall suppression of alpha
power observed over parieto-occipital regions, figure 7.4B,D) can have temporally
remote electrophysiological consequences, at least with the sensitivity afforded by
EEG.
The alpha suppression effects evoked by TMS were neither accompanied by
behavioural alterations, nor by beta suppression effects over motor cortex. Although
these results might be taken as limitations of this study, in fact they indicate that
we were successful in keeping brain activity and behavioural responses within their
physiological range. We kept TMS intensity deliberately low (100% AMT), to ensure
that the TMS effects reflect modulations of a physiological circuit, rather than byproducts of strategic changes in behaviour.
It should be emphasized that the alpha suppression observed over the aIPS region
after sPOS interference has different properties than the slant-related alpha
suppression described above, the latter occurring later and being slant-dependent.
The enhanced alpha suppression over the aIPS region is also different from the
oscillatory correlates of movement preparation observed over the motor cortex, the
latter occurring in the beta band over fronto-central regions.
Conclusion
Influential accounts of the cerebral control of grasping have suggested a functional
dichotomy between transport and grip components driven by particular object
features and implemented in dedicated parieto-frontal circuits 2,113 . Others have
suggested that the two parieto-frontal circuits distinguish between action planning
p arie t o - f r o nt al cir cuit s d uring g r a s p | 167
and online-control32,86 . The findings of this study do not support those suggestions,
but are more easily reconciled with theoretical frameworks that predict integrated
control mechanisms 249,251 . We qualify those mechanisms by showing a temporal
dependency between the contributions of the dorsomedial and dorsolateral
parieto-frontal circuits. Planning a grasping movement relies on early contributions
from aIPS, followed by later contributions from sPOS. We suggest that this temporal
dependency is a consequence of the different levels of sensorimotor abstraction at
which those circuits operate. Namely, aIPS provides an initial structure to the motor
plan on the basis of abstract object knowledge. This movement structure can then be
used by sPOS as a prior for guiding the parameterization of the movement operating
on metric information accumulated before motor execution.
7
8
Discussion
17 0 | c h a p t e r 8
d i s c u s s i o n | 17 1
Introduction
How do we grasp a ripe tomato? It seems too embarrassingly simple to ask. We reach
for it, we open our hand, and once we’re near enough, we close our hand around it.
Right? To do this without error, it is important to know, first of all, where the tomato
is located relative to our body so we have an idea where to reach to, and second, to
estimate the size of the tomato to know how far to open our grip. But how then, do
we so fluently and adaptively anticipate our grip, making sure we do not squish a red
overripe tomato, while grasping a hard green tomato firmly enough to pluck it from
the plant?
Our manual interactions with objects form a central theme in our motor
behaviour. It requires transforming visuospatial information into motor code to
guide the fingers to the desired end-points. But objects, especially the ones we
are familiar with, often have rich motor associations, beyond a metric visuospatial
representation, based on prior experience and semantic knowledge. Even before
seeing a specific tomato we have an idea on how to shape the movements of our
fingers to grasp it. We know that we should not clutch an overripe tomato too firmly.
We anticipate the perceived slipperiness and weight of a glass of water when we intend
to grasp it. When we pick up a knife and fork, we know how to hold them in order
to make use of them. Humans are exceptionally skilled in hand-object interactions
and tool use; understanding how we accomplish these feats will enlighten us about
important aspects of the human brain and our behaviour.
Corticalorganisationofgraspguidance
Perceptual processing of visual information, such as identifying the colour of an
object and recognizing that it is a tomato, is organized along a specific processing
pathway in the brain: the ventral visual stream running along inferior occipitotemporal cortex 10,35 . On the other hand, controlling visually-guided actions towards
objects relies on two largely segregated parieto-frontal channels: a dorsomedial
circuit including the superior part of the parieto-occipital sulcus (sPOS) and dorsal
8
17 2 | c h a p t e r 8
premotor area (PMd) in frontal cortex, and a dorsolateral circuit centred around
the anterior part of the intraparietal sulcus (aIPS) and the ventral premotor area
(PMv) 2,198 .
An influential account of grasp control emphasises the modularity of grasping
movements 1,5 . The extrinsic location of a target object informs the initial arm
transport, while a complementary grip component shapes the hand according to
intrinsic object metrics such as its size 243 . These two components are suggested
to map onto the segregated parieto-frontal channels: the dorsomedial channel
guiding the transport component, and the dorsolateral channel shaping the grip
component 2,72,113 .
In the past three decades studies of grasp control have brought to light how these
circuits are anatomically and functionally dissociated. However, for the successful
performance of a prehension task, even as seemingly simple as grasping a ripe
tomato, we often have to rely on computations implemented in all three channels.
Accordingly, in recent years it has been emphasized that for adaptive and efficient
grasp control the processing in these circuits has to be integrated, but to date this
integration has not been directly investigated. This thesis aimed to help in closing
this gap in our current understanding of grasp control.
Thefocusofthisthesis
The main question of this thesis addresses how perceptual information is integrated
with metric visuospatial cues to guide grasping movements. More specifically, how
do the separate ventral, dorsolateral and dorsomedial circuits share information
to accomplish a common grasping goal, and how is their integration functionally
implemented in the known cortical anatomy? On the basis of available literature
and supplemented by the present studies, an updated account of grasp control is
proposed, describing how perceptual information is incorporated into a motor plan.
This restructured model challenges the framework of separately controlled transport
and grip components and makes new predictions about the functional and temporal
characteristics of the computations performed in the two parieto-frontal channels.
This thesis focuses on two manipulations: 1) biasing the reliance on either
perceptual or metric cues for grasp planning, and 2) altering the demands for
accurate visuomotor transformations. Chapter 2 described how these two
manipulations are experimentally operationalized. First, the manipulation of
perceptuo-motor integration relied on the distributed processing of monocular
(perceptual) and binocular (metric) cues of depth that guide grasping movements
towards an object oriented in depth 287,300,301 . Importantly, the information value
of these cues differentially depends on the slant of the object to be grasped 294,307,
allowing us to track how both perceptual and metric cues contribute to an abstract
d i s c u s s i o n | 17 3
representation of the object’s configuration informing grasp planning. Second,
increasing the slant of an object to be grasped (from vertical to horizontal) increases
the computational demands for accurate visuospatial parameterization guiding the
fingers to the object312,461 , without changing the intrinsic size or shape of the object,
nor its extrinsic location in space 1 .
The experiments described in thesis were designed to isolate and characterize
the behavioural and neural correlates of perceptuo-motor integration in grasp control
by means of a combination of complementary techniques: kinematic tracking,
fMRI, EEG and TMS. It is not a trivial endeavour to combine EEG and TMS, nor to
ensure that a subject is able to perform ecologically valid visually-guided grasping
movements inside an MR bore. In chapter 3 our approach to these challenges is
described. Chapter 4 introduced a novel and generally applicable way to account
for global nuisance variance in the analysis of fMRI time-series. When taking these
experimental considerations into account, fMRI allowed for isolating the cortical
areas involved in grasping and an examination of how their coupling changes as a
function of task demands (chapter 5). The oscillatory cortical activity associated
with the prehension task was measured using EEG, providing further insight in the
underlying computations on the basis of the spectral and temporal characteristics
of these oscillatory components (chapters 6 and 7). The oscillatory and kinematic
consequences of transient TMS interference with two nodes important for grasp
planning were recorded: aIPS and sPOS, as identified in the fMRI study reported in
chapter 5. In that way the causal contributions of these critical nodes were tracked,
allowing a description of the functional and temporal dependencies between the
dorsomedial and dorsolateral circuits.
Summary
aIPSstructuresagraspplanbasedonmetricand
perceptualinformation
Given that the prehension task used in this thesis only instructed the end-state
of the hand in relation to the object, and that visual feedback was withheld
during movement execution, the experimental manipulation of perceptuo-motor
integration was expected to mainly influence the accuracy of the final phase of the
grasping movement. Indeed, when the configuration of the object could be most
reliably estimated, either based on metric or perceptual cues of depth, we observed
increased spatial consistency of the fingers’ movements in relation to the object just
before contact, a reflection of improved planning accuracy (chapter 6).
8
174 | c h a p t e r 8
Using fMRI we found that pivotal nodes in the dorsolateral circuit (aIPS
and PMv) and the ventral visual stream (lateral occipital complex, LOC) showed
enhanced activity when grasp planning relied on pictorial cues of depth (chapter 5).
These conditions also increased the functional coupling of aIPS with both LOC and
PMv. Together, this suggests that aIPS not only transforms metric object features
(obtained by stereoscopic cues), but also incorporates perceptual information (as
provided by monocular cues) to support adaptive prehensile behaviour.
This observation was further specified by investigating the oscillatory effects of
providing maximally informative metric or perceptual depth cues for grasp planning
(chapter 6). Under these viewing conditions we found enhanced suppression of beta
power over the left motor cortex contralateral to the grasping hand, a well-known
hallmark of motor preparation325,329,455 . Critically, this effect occurred very early in
the movement planning phase, and it was especially prominent when experience
had accumulated over the course of the experiment. In other words, when the
object’s configuration is relatively easy to identify, and when you are familiar with
the object’s interactions, then motor preparation is robustly enhanced in the very
early phases of movement planning.
To verify that this specific motor planning advantage critically depends on the
computations performed in aIPS, TMS was applied to interfere with local neural
processing and recorded the behavioural and oscillatory consequences (chapter 6).
When either metric or perceptual cues of object configuration were most informative,
disruption of processing in aIPS led to a specific decrease of the spatial consistency
of the fingers’ movements when approaching the object, in effect removing the
planning advantage of informative depth cues. Moreover, in the same conditions
we observed a robust and specific reduction of motor cortex beta suppression,
early during movement planning. These observations indicate that perturbing aIPS
during the planning phase of a grasping movement reduces the ability of the motor
cortex to adequately organize the movement. Crucially, aIPS interference had a
stronger effect on visuomotor integration in the second half of the experiment,
preventing subjects from using informative depth cues even when available. This
finding suggests that, as object configuration knowledge becomes accessible, the
sensorimotor system comes to critically depend on prior experience as mediated by
aIPS to structure the movement plan.
In summary, aIPS is necessary during the early planning stage, it organizes the
structure of the grasping movement on the basis of both metric and perceptual cues,
making use of abstract object configuration knowledge and prior experience.
d i s c u s s i o n | 17 5
sPOSsubsequentlyparameterizesagraspplan
Grasping a slanted object often necessitates specifying the location of visually
inaccessible finger end-points and potential hindrances from the object itself on the
fingers’ trajectory. This increase in planning demands was captured by many robust
changes in both transport and grip kinematic parameters (chapter 7). The maximum
grip aperture could be expected to increase with increasing prehension difficulty,
as a sign of a more conservative grasping strategy2,243 . But in fact, we observed a
reduction in grip aperture with increasing slant. In chapter 7 this effect is suggested
to be a consequence of skeletomuscular constraints, and not of a changed grasping
strategy or a foreshortened view of the target object (as previously suggested314,439).
When planning a grasping movement towards an object slanted in depth, we
observed, as indexed by fMRI, that areas in the dorsomedial parieto-frontal circuit
(sPOS and PMd) increased their activity with increasing demands on accurate
visuomotor transformations, regardless of the type and information value of the
available visual depth cues (chapter 5).
This finding is confirmed by the oscillatory correlates of increasing visuospatial
demands in grasp planning. We found enhanced alpha power suppression, an index of
increased computational load, emerging over medial parieto-frontal regions (chapter
7). This effect of object configuration started shortly before movement onset, riding
on top of the overall task-related sustained alpha suppression, and continued during
the early stages of movement execution. The timing of this effect closely matches
that of visuomotor neurons recorded from area V6A in macaques 60,108 . It suggests
that the dorsomedial circuit supports grasping by fine-tuning the motor plan on
the basis of information accumulated before motor execution (chapters 5 and 7).
Processing in the dorsomedial circuit, including sPOS, is better characterized by the
demands of the required motor output, than the characteristics of the visual input,
as it is not influenced by the information value of the provided visual depth cues
(chapters 5 and 7). This implies that the dorsomedial circuit can also specify motor
parameters informed by abstract representations of an object’s configuration,
rather than only based on raw visual cues of depth directly accessible from early
visual areas 60 .
TMS was applied during movement planning to perturb computations in aIPS
and sPOS, and examined the functional, causal and temporal dynamics of their
contributions. We found that interfering with either aIPS or sPOS led to a reduction
of the orientation-related alpha suppression over the same dorsomedial regions, but
crucially, there were clear differences in the timing of their contributions (chapter 7).
Stimulation of aIPS disturbed an early component of the alpha suppression effect
around movement onset, whereas sPOS stimulation affected a later component
of that effect. The temporal asymmetry of these contributions suggests that
8
176 | c h a p t e r 8
dorsomedial computations depend on abstract target features processed early
during planning in aIPS (chapter 6). Following TMS over sPOS, irrespective of viewing
conditions and object orientation, we found sustained enhanced alpha power
suppression over the left aIPS region, an indication that increased computational
load in the dorsolateral circuit compensates for transient perturbations in the
dorsomedial circuit (chapter 7).
In summary, these results indicate that the dorsolateral and dorsomedial
parieto-frontal circuits are functionally commutable, but temporally dependent.
Whereas the dorsolateral circuit structures initial planning based on metric and
perceptual cues, the dorsomedial circuits fine-tunes the motor plan by transforming
current visuospatial information to motor code.
Howtograsparipetomato
Taken together, the studies described in this thesis provide a novel account of
the functional interpretation of the dorsomedial and dorsolateral parieto-frontal
circuits in the control of visually-guided grasping movements. They suggest that
the computations performed in these distinct circuits differ mainly in the level of
sensorimotor abstraction at which they operate. The dorsolateral circuit provides
an initial structure to the motor plan on the basis of abstract object knowledge. This
initial structure forms a template that is used by the dorsomedial circuit as a prior for
subsequent parameterization of the movement, operating on current visuospatial
information accumulated before motor execution.
So how do we grasp a ripe tomato? The findings reported in this thesis inform
a speculative and simplified model of how the brain might solve this task. When
we look at a tomato we intend to grasp, several features of the visual image are
processed in parallel: a spatial representation of the scene, the metrics of the object
to be grasped, and its identity and perceptual characteristics. The anterior part
of the intraparietal sulcus (aIPS) in the dorsolateral parieto-frontal circuit rapidly
constructs an object-related motor template of the appropriate action, retrieving and
integrating both semantic and pragmatic knowledge of the tomato. The formation
of his template incorporates both a metric three-dimensional representation of the
object, supported by dorsal occipito-parietal cortex, and the identity of the object,
a red ripe tomato in this case, relying on the ventral visual stream in the inferior
occipito-temporal cortex.
The ripe-tomato-related motor template provided by aIPS is shared with the
ventral premotor area (PMv), transforming it to motor code shaping the movements
of the effectors involved. But the initial template also serves as a prior for subsequent
computations implemented in the medial superior part of the parieto-occipital
sulcus (sPOS) in the dorsomedial circuit, building and refining the grasp plan based
d i s c u s s i o n | 17 7
on current visuospatial information. In fact, these visuomotor transformations
performed in the dorsomedial circuit do not necessarily need to represent the
transport of the arm and the opening of the hand separately. Potentially, the
grasping movement could be controlled in an integrated fashion, where the fingers
are guided towards absolute spatial representations of their respective target endpoints 251 , albeit informed by an object-related initial motor template. Together,
these computations allow accurate planning of the movements of the fingers to the
tomato, based on prior experience and semantic object knowledge, and adjusted to
the specific metrics and configuration of the tomato to be grasped.
In the following sections this novel account of grasp control will be reviewed in
the context of existing models of movement control and functional interpretations
of the distinct parieto-frontal circuits. Some speculative predictions will be provided,
that could be formally tested in the future.
Discussion
Gripforce
When grasping a ripe tomato, multiple facets of the grip are anticipated: not on only
the fingers’ spatial configuration in relation to the tomato, but of course also the
force of the grip. This thesis has primarily focussed on the spatial configuration of
the object to be grasped: both the perceptual and metric cues that were manipulated
specifically considered this domain. However, also the incorporation of grip force
cues seems to rely on a similar dorsolateral parieto-frontal network 462,463 , but the
temporal dynamics of the contributions of aIPS to grip force control differ slightly
from its contributions to grip shape464 . Potentially, whereas aIPS operates in
congruence with the dorsomedial parieto-frontal circuit to define the fingers’ spatial
configuration, its influence on grip force control might be assisted by a circuit pivoting
around the secondary somatosensory area (SII), which is robustly connected in
macaque to AIP204 , ventral premotor area F5 224 , and prefrontal area 12r234 . Critically,
the putative control of grip force by aIPS and SII is likely to be influenced by semantic
knowledge about the relation between colour and ripeness of the tomato as stored
in the temporal cortex, similarly to the control of grip shape and finger movements
as examined in this thesis.
8
17 8 | c h a p t e r 8
GraspplanninginaBayesianframework
The account that the dorsolateral and dorsomedial parieto-frontal circuits differ
mainly in the level of sensorimotor abstraction at which they operate, fits well
with recent proposals of general computational mechanisms supporting predictive
coding 284 . The relation between the computations proposed to be implemented
in these distinct circuits can be viewed in light of a Bayesian framework 465 .
Speculatively, in that context, the object-related motor template supplied by aIPS
is handled by sPOS as an informative prior (i.e. prior probability), which is combined
with the observed visuospatial evidence (i.e. conditional and marginal likelihoods) to
inform the estimation of the required sensorimotor transformations (i.e. posterior
probability). The way aIPS constructs this motor planning prior could also be
considered in the same framework: aIPS infers the current sensory state, in particular
that of the object to be grasped, based on multiple sources of information, and
retrieves an appropriate previously constructed prior from storage. In this Bayesian
process two stages can be defined: 1) a set of priors based on previously acquired
experience and knowledge of different object configurations, and 2) the inference
of the current object configuration to select and retrieve the appropriate prior(s).
Both of these levels are associated with their own precision, which were manipulated
separately in the prehension task investigated in chapter 6.
At the start of the experiment, subjects’ knowledge of the possible configurations
of the object to be grasped could be regarded as a flat (Bayesian) prior with minimal
confidence (i.e. precision). Of course, subjects had most likely encountered simple
rectangular objects before in their lives, even at different slants, but in the context
of the experimental duration, their object configuration knowledge was relatively
uninformed in the first half of the experiment. Across repeated encounters with
the object configurations presented in the experiment, experience accumulates,
building an informed prior with reasonable confidence. We found that more
informed object-interaction priors led to enhanced oscillatory activity in the motor
cortex early during motor preparation, signifying improved motor planning.
From this set of previously acquired priors, the most appropriate needs to be
retrieved based on the current sensory state. Again, this might be implemented
as a Bayesian inferential process, with its own confidence. For example, if the
target object would be presented only very briefly, or in dim lighting conditions,
the estimation of the current configuration of the object will be imprecise, with a
probability distribution spanning several possible configurations (and associated
priors). The prehension task used in this thesis was designed to manipulate the
confidence of this inferential process independently of viewing conditions or
object configuration, by exploiting the differential dependency of the information
value of binocular and monocular depth cues on object slant (see chapter 2 of this
thesis). Accordingly, the interaction of object vision and slant manipulated the
d i s c u s s i o n | 17 9
confidence by which a previously constructed prior could be retrieved. We observed
that with high confidence (i.e. with high reliability of the available depth cues), the
spatial consistency of the fingers’ movements was better adjusted to the object’s
configuration, and oscillatory beta power was enhanced over left motor cortex early
during planning, both reflections of improved motor planning.
Onlinecontrol
This thesis is focused on the preparation phase of a grasping movement In fact,
visual feedback was withheld during the execution phase altogether. Nonetheless,
the account of grasping developed here could possibly be generalized to the online
control of movements. As discussed in the general introduction (chapter 1), online
movement control relies heavily on predictive coding, combining the currently
observed sensory state with the state predicted based on previous sensory input and
motor commands. Accordingly, the Bayesian framework proposed above to underlie
the computational mechanisms of the dorsolateral and dorsomedial circuits, could
be extended in a straightforward way to also support a fluent integration of grasp
planning and execution56,249,466 . This speculative prediction needs to be formally
tested, but it might be supported by recent findings highlighting the importance
of aIPS in online grasp control 239,458,462 . Accordingly, the findings described in this
thesis do not entail that the contributions of aIPS to grasp control are restricted in
time to the early stages of motor planning (as might be suggested by the findings
reported in chapter 6), but instead suggest that the template constructed by aIPS
is kept online (chapter 7) and continuously updated in the dorsolateral circuit 161 .
There are indications that the temporal dependency between the dorsolateral
and dorsomedial circuits proposed in chapter 7 is maintained during movement
execution 207,240,456 , suggesting that the computational principles implemented in
these channels remain similar across all stages of a grasping movement.
This integrated account of movement planning and execution might shed light
on some conflicting functional interpretations of the temporal characteristics of
the two parieto-frontal circuits 32,86,205,239 . It suggests that the primary functional
organizational principle separating the two channels is not, as often suggested,
a distinction between action organization and online control 86,172 , but rather a
difference in the level of abstraction at which these channels are performing
sensorimotor transformations. Accordingly, any observed temporal differentiation
of the dorsolateral and dorsomedial contributions to grasp planning and execution
is likely to be a secondary consequence of the neural dynamics of the computations
performed by the two streams.
8
18 0 | cha p t e r 8
Transportandgripcomponentsofagraspingmovement
As addressed earlier in this general discussion, and especially in chapters 5, 6 and 7,
the findings reported in this thesis are not fully congruent with an influential account
of grasp control: the two-channel transport-grip dichotomy1,2,5 . However, that does
not imply that the current findings contradict two robust observations associated
with this hypothesis.
First, activity in sPOS is enhanced with increasing transport distance, both in
the context of reaching and grasping movements 113 . This observation has been
interpreted as an indication that sPOS controls the transport component, becoming
more active when computational demands on arm and wrist guidance increase. Our
findings advocate for a more general function of sPOS: we also observed stronger
activation in sPOS when the transport complexity of the individual fingers (and
not the wrist or arm) was enhanced with increased object slant (from vertical to
horizontal; see chapters 2 and 5). This implies that sPOS activity is enhanced when
computational demands on visuospatial parameterization of either arm, wrist and
finger movements are high (chapter 5). These similar findings, where in one case
mainly the arm transport is affected 113 , and in the other the finger transport (chapter
5), might together be explained by a less modular and more integrated model of
arm, hand and finger control. For example, Smeets and Brenner251 have proposed
that in grasping, similarly to reaching, the fingers and not separate transport and
grip components are controlled independently. Indeed, during grasping the eyes
are not focused on the hand, but mainly on the index finger and its target endposition 467, with a preference for occluded parts of the target 468 . Increasing object
slant increases the demands on accurate visuospatial transformations in a similar
way as (partial) object occlusion.
Second, aIPS is taxed more when performing grasping movements, with
the fingers enclosing the object, compared to reaching movements with the
knuckles touching the object 155-157,245 . An obvious difference between these two
movements is the requirement to shape the hand and fingers. Accordingly, aIPS
is often considered to control the grip component of a grasping movement 2,72 . In
contrast, the independent finger control hypothesis emphasises the increased
visuospatial and skeletomuscular complexity of reach-to-grasp compared to reachto-touch movements 27,252 , potentially requiring the contribution of an additional
network. However, it seems unlikely that kinematic complexity alone can explain
the dissociation between the engagement of sPOS and aIPS 113 . If the fingers are
assumed to be controlled independently, but irrespective of an transport and grip
component, it does not seem parsimonious that an increase of complexity by
increased transport distance can be handled within the same network (involving
sPOS), while an additional network (involving aIPS) is required to cope with an
increase of complexity by using two fingers instead of one. In light of the model
dis cussio n | 181
proposed in this thesis, the critical difference between enclosing an object with two
fingers, and touching it with the knuckles, is suggested to revolve around the fact
that especially the former can greatly benefit from incorporating an object-related
sensorimotor template.
This framework predicts that aIPS is engaged in addition to sPOS when object
characteristics are important to guide a reach-to-touch movement. This hypothesis
might be operationalized by comparing two reaching movements, one where the
object simply needs to be touched, and one where the object needs to be pushed by
a force that can be anticipated on the basis of visual object features, perhaps its size,
or its material. The accuracy demands (affecting the end-point-error of the digit) of
reach-to-touch movements might be expected to be lower than those of reach-topush movements, but this could be addressed by requiring subject to hit a specific
target on the object, for instance implemented as a visual spot or a small button that
must be pressed by a force independently of the features of the object. Under these
circumstances the kinematics of the hand and finger transport are expected to be
equal between to two conditions 252 , but the relevance of object-interaction priors to
inform the movement are hypothesised to differ.
Grasping movements constitute the majority of our behavioural interactions
with objects, and consequentially, object-related motor priors might also typically
concern finger guidance in grasping. Conversely, the location of a target is often
not standardized, and needs to be incorporated on the basis of current conditions,
often affecting the movements of the arm in space. In other words, a template of
an object-interaction could be expected to mainly inform the shape of the fingers
and hand, while the refinement of the motor plan based on current visuospatial
parameterization, will often require additional control of more proximal muscles. In
that view, a distinction between controlling the grip (fingers) and transport (arm)
component might be a secondary consequence of the computational principles
governing the dorsolateral and dorsomedial parieto-frontal circuits, in that same
way that a contrast in action organization vs. online control is also considered a
secondary consequence of the same computations.
Importantly, even though the processing by aIPS could be expected to mainly
govern finger movements and grip shape, there is no a-priori reason why this
contribution would be exclusive and could not influence the guidance of the arm,
often considered part of a transport component. To test this hypothesis one could
exploit the fact that some familiar objects are also strongly associated with a
particular contextual location. A knife for instance, will most often be encountered
on the right side of a plate. Might this relative location also be incorporated in a
putative knife-grasping-template? When grasping a familiar knife in its familiar
location is compared to grasping the same knife in an unfamiliar location, several
factors might influence the proficiency of the grasping movement. However, these
8
182 | cha p t e r 8
confounds could potentially be addressed by comparing the grasping of the knife
to the grasping of a novel object with the same prehension metrics. The spatial
consistency of the fingers in relation to the target would be expected to be higher
when grasping a familiar compared to a novel object, if both are presented in the
relative location associated with the familiar object. However, if the familiar object
would be presented in a novel location, the prior template might potentially hamper
the planning of the grasping movement, resulting in a less well-adapted arm, hand
and finger coordination compared to grasping of the novel object in the same
location. The latter will mostly rely on visuospatial parameterization on the basis of
current information, not on a template.
Opticataxiaandideomotorapraxia
The functional and anatomical separation of parietal cortex in two largely
segregated circuits might be reflected in the dissociable behaviour observed in
two neuropsychological disorders: optic ataxia (associated with lesions in superior
parieto-occipital cortex) 11,13 and ideomotor apraxia (associated with lesions in the
left dorsolateral parieto-frontal circuit) 257,260,261 . Although the behavioural deficits
of these patients are diverse, and the anatomical extent of the lesions can vary
considerably (even within one group), an integration of our functional interpretation
of the two parieto-frontal circuits with observations of impaired behaviour following
lesions in these circuits, might benefit the specification and understanding of both
our proposal and of these neurological disorders.
Optic ataxia patients are often most affected in reaching (but also grasping)
towards targets in the periphery of the visual field contralateral to the lesion site 12,13 .
When the sPOS is bilaterally damaged, again movements to peripheral targets
are most strongly impaired, while centrally performed actions remain relatively
spared 469,470 . This might be a reflection of distinct networks engaged in the
processing of movements towards the centre or the periphery of vision 74 , or because
the deficit is only apparent when the visuomotor system is fully taxed 12,13 . However,
the behaviour of a well-studied bilaterally affected optic ataxia patient (AT) might
inform a different interpretation. She did not only reveal increased proficiency of
her grasp when acting upon objects in central compared to peripheral vision, but
also when asked to grasp familiar compared to novel objects presented in the same
location 14 . This spared behaviour might be mediated by the dorsolateral circuit
including aIPS (which was unaffected in AT), using abstract object information to
guide the grasping movement. As familiar objects are canonically viewed in central
vision, and given the fact that object-interaction templates of actions cannot
be appropriately tuned a-priori for all possible locations in the reachable space,
abstract object knowledge and prior experience might be most suited to inform
dis cussio n | 183
accurate behaviour in the central visual working space, not the periphery. In that
light, impaired reaching and grasping in the visual periphery of optic ataxia patients
might not be a deficit of acting within the periphery per se, but a general deficit of
visuospatial parameterization compensated only in central vision by the availability
of appropriate visuomotor templates defined in the dorsolateral circuit. Similarly,
the temporal dynamics of the concrete visuospatial parameterization in sPOS might
explain why lesions and TMS perturbations of sPOS and superior parietal cortex
cause deficits in the online adaptation to a change in the visuospatial conditions of
the target31,32,445,469 . Likewise, these dynamics might explain why TMS interference
with aIPS leads to impaired re-planning of grip shape in the context of grasping
movements towards objects 239,471 .
The same mechanisms might underlie the seemingly paradoxical observation
that ideomotor apraxia patients tend to be more impaired when grasping familiar
compared to non familiar objects 265 . Under those circumstances currently available
sensory evidence appears to be overridden by previously acquired priors that fail
to be effectively incorporated. On the other hand, the proposed computations
implemented in sPOS might allow ideomotor apraxia patients to guide their hand
and fingers to an object in a kinematically valid way, although they do not incorporate
semantic knowledge of an object when grasping 260 , leading to functionally
inappropriate behaviour, for instance skilfully grasping a spoon, but at the wrong
end263. Similarly, ideomotor apraxia patients show impairments in hand shaping,
in particular when asked to mimic grasping an object that is not visually present,
or when asked to imitate meaningless gestures 260,261 . Under these conditions,
visuospatial information of the target is not available and the performance of the
task is fully dependent on the availability and quality of prior defined visuomotor
templates associated with the object or instructed gesture. On the basis of the
findings reported in this thesis, it is feasible to assume that the dorsolateral
visuomotor stream supports the processing of these prior acquired templates.
Parallelprocessingintheparieto-frontalcircuits
The findings reported in chapter 7 suggest that the dorsolateral and dorsomedial
parieto-frontal circuits perform commutable functions, but with different temporal
dynamics. Although these dynamics imply a temporal hierarchy (where computations
in sPOS follow those in aIPS), these visuomotor streams do not seem to be strictly
co-dependent. In fact, the observation that both optic ataxia and ideomotor apraxia
patients often perform reasonably well under canonical circumstances (grasping
and reaching in central vision towards objects with congruent spatial and semantic
features), could be interpreted as an indication of the commutable functions
8
18 4 | cha p t e r 8
performed by the two parieto-frontal circuits and might explain the lack of true
dissociations in their behavioural repertoire 15,472 .
The frontal aspects of both channels (PMv and PMd) heavily project onto the
primary motor cortex, but also have direct access to the corticospinal tract 193,275 .
This suggests that the two visuomotor streams form largely parallel pathways
whereby each is potentially able to influence the control of both proximal and
distal muscles, albeit for different functions 210,270 . This integrated organization of
the corticospinal motor system has been proposed to specifically support the fine
control of individuated finger movements 211,269 , relying on phylogenetically recent
adaptations in the corticospinal circuit 21,213 , necessary for dexterous handling of
objects and tools 273,274 .
Dexterous grip guidance requires more than controlling just the aperture of the
opposition space between the thumb and index finger. There are numerous grip
types beyond the precision and power grips discussed in the general introduction 19 .
Many of them do not have a direct spatial relationship to the object to grasp, but are
more closely related to the function that they perform on or with the object 473 . The
studies described in this thesis have only investigated the control of precision grip in
prehension, with a particular focus on the accurate guidance of the fingers and the
opposition space that they create in accordance with the target object. However,
the notion that the dorsolateral stream might use abstract object knowledge to
construct a grasp template might be generalized to include all types of hand-object
interactions. Recent evolutionary adaptations allow PMv to execute a strong
influence over both the primary motor cortex and the corticospinal tract. Such
adaptations, in combination with the simultaneously integrated and individuated
control of both distal and proximal muscles, might allow these templates to directly
inform and shape the behavioural repertoire, particularly that of object-oriented
actions.
Actionrepresentationsinthedorsolateralcircuit
Chapters 5 and 6 suggest that the dorsolateral parieto-frontal circuit structures
a grasp plan based on abstract object knowledge. This finding is in congruence
with recent studies highlighting the abstract nature of action representations in
the dorsolateral circuit. The involvement of aIPS in movement guidance is mainly
shaped by the function and not just by the effector of the movement73 . Accordingly,
the representations of hand-object interactions in aIPS have been suggested to
act as templates for the control of movements executed with other effectors 474 .
Moreover, both aIPS and PMv support the organization of movements based on
the goal of an action, independently of the required kinematics 172,475 . Moreover,
the influence of aIPS on grasp guidance is context and goal dependent 169,239 . This
dis cussio n | 185
abstract and conceptual representation of object-interactions might also support
the contributions of the dorsolateral circuit in the use of tools 163,165 , irrespective of
grip formation, arm movements, or grip-functionality of the tool 254 .
There is a strong overlap in the neural circuitry involved in planning an objectoriented action and observing a similar action being performed by someone else:
two areas important in grasp control, aIPS and PMv are also pivotal in action
observation 167-170 . In macaque monkeys neurons have been identified in these areas
that respond both when the monkey moves himself, and when he observes someone
else performing the same movement, so called mirror neurons 179 . Although these
neurons are characterized by a correspondence between the executed and observed
movement to which they respond, this congruency is not necessarily very strict 180 .
Mirror neurons are capable of representing the goal, even the distant goal, of an
action, both during execution and observation. The dominant interpretation of
these findings is that the motor system, through the mirror mechanism, provides
the cortical foundations for understanding the actions of others 181,476 . Following
that view, action perception and understanding would be shaped by our own motor
system 477.
In accordance with these reports, the dorsolateral stream is often implicated in
a diverse set of conceptual functions, such as object awareness, action on objects,
and action perception 86,172,477. This account of a semantic role of the dorsolateral
parieto-frontal circuit in action guidance resembles our functional interpretation of
grasp control by the dorsolateral circuit on the basis of abstract object knowledge.
However, the integration of perceptual-semantic information processed along the
ventral visual stream into motor planning has often been suggested to operate on
the basis of high-level aspects of prehensile behaviour. In this thesis we show that this
integration can already occur at a relatively low-level of computation, concerning the
orientation of the object to be grasped. Moreover, in contrast to the former account
our proposal specifically addresses the temporal dynamics of the computations
performed, and the mechanisms supporting the information flow between distinct
streams to achieve a common grasping goal. Accordingly, the current findings might
inform investigations of the semantic and perceptual functions of the dorsolateral
parieto-frontal circuit, potentially supporting action observation and tool use.
8
18 6 | cha p t e r 8
Conclusion
Based on the experiments reported in this thesis, I propose a novel functional
interpretation of the contributions of the dorsolateral and dorsomedial parietofrontal circuits to visually-guided grasp control. I suggest that these distinct circuits
differ mainly in the abstraction level of the sensorimotor transformations they
perform. The dorsolateral circuit integrates metric and perceptual object information
to construct an initial template for the motor plan. This structure is subsequently
used by the dorsomedial circuit as a prior for parameterization of the movement, on
the basis of current visuospatial information, accumulated before motor execution.
This account provides a computational basis that can explain both the appearance of
a functional dissociation between the control of a transport and grip component of a
grasping movement, and between action organization and online control.
Conceptual and sensorimotor processes are often regarded and investigated in
separation, or folded one into another. In both views the exceptionally interesting
outcomes of their potential integration are ignored. It appears relevant to test
whether the new account of grasping outlined above could generalize to other
behaviours that rely on the integration of perceptual-semantic information
and visuospatial transformations, such as tool use, and gesture production and
comprehension. Namely, it is conceivable that the mechanisms described in this
thesis might provide a general template for understanding the integration of
conceptual and sensorimotor processes.
dis cussio n | 187
8
A
Appendix
19 0 | a p p e n di x
d u t c h s u m m a r y | 191
Nederlandsesamenvatting
Hoe grijp je een rijpe tomaat? Het lijkt zo’n simpele vraag. We reiken ernaar, we
openen onze hand, en als we er bijna zijn, sluiten we onze hand om de tomaat. Toch?
Als dat alles is, dan hoeven we maar twee dingen te weten. Ten eerste, waar de
tomaat zich bevindt ten opzichte van ons lichaam, zodat we weten waar we naartoe
moeten reiken. Daarnaast, hoe groot de tomaat is, om in te schatten hoe ver we
onze hand moeten openen. Maar als dat inderdaad alles is, hoe passen we dan zo
moeiteloos en flexibel onze grijpbeweging aan zodat we een overrijpe tomaat niet
fijnknijpen, maar een harde groene tomaat juist wel stevig genoeg vastpakken om
hem van de plant te plukken? Hiervoor hebben we perceptuele informatie van de
tomaat nodig. Deze informatie wordt door onze hersenen verwerkt op een heel
andere manier en in een heel ander gebied dan de positie en grootte van de tomaat.
Hoe wordt die informatie dan geïntegreerd in onze bewegingen? Deze eenvoudige
vragen kunnen verregaande gevolgen hebben. De integratie van perceptueel begrip
en bewegingsaansturing liggen namelijk aan de basis van gedrag waar mensen in
uitblinken ten opzichte van vele andere dieren: bijvoorbeeld het manipuleren van
objecten, het gebruiken van gereedschap, en het maken van gebaren.
De interacties tussen onze handen en objecten in onze omgeving is een zeer
belangrijke component van ons gedrag. Het vereist het transformeren van visueelspatiale informatie naar spierinspanningen om onze vingers naar de beoogde posities
te leiden. In eerste opzicht lijkt een kwantitatieve visueel-spatiale representatie van
objecten hiervoor te voldoen. Maar objecten, zeker degene waar we vertrouwd mee
zijn, hebben vaak een veel rijkere associatieve representatie, bijvoorbeeld gebaseerd
op eerdere ervaringen en semantische kennis. Zelfs voordat we een object zien,
kunnen we al een idee vormen over hoe we onze vingers moeten bewegen om het te
grijpen. We weten bijvoorbeeld dat we een overrijpe tomaat niet te stevig moeten
beetpakken. We anticiperen de gladheid en het gewicht van een glas water zodra we
de intentie vormen om hem te grijpen. En wanneer we een mes en vork oppakken
weten we hoe we ze moeten beethouden om ze goed te kunnen gebruiken. Mensen
hebben een bijzondere gave voor het manipuleren van objecten en het gebruiken
A
192 | a p p e n di x
van gereedschap; het bestuderen van deze gave kan ook inzicht geven in andere
belangrijke aspecten van de menselijke hersenen en ons gedrag.
Corticaleaansturingvandegrijpbeweging
Perceptuele verwerking van visuele informatie, zoals het identificeren van de kleur
van een object en het herkennen dat het om een tomaat gaat, is georganiseerd
in een specifiek verwerkingscircuit in de hersenen: de ventrale visuele stroom
in de inferieure occipitale en temporele cortex 10,35 . Daartegenover, motorische
verwerking van visuele informatie, voor het aansturen van bewegingen, berust juist
op twee dorsaal gelegen parallelle verwerkingscircuits. Deze twee circuits verbinden
beide de pariëtale met de frontale cortex. De eerste is een dorsomediaal kanaal,
waartoe onder andere het superieure deel van de pariëto-occipitale sulcus (sPOS) en
het dorsale premotor gebied (PMd) behoren. Het tweede circuit is een dorsolateraal
kanaal en omvat onder andere het anterieure gedeelte van de intrapariëtale sulcus
(aIPS) en het ventrale premotor gebied (PMv) 2,198 .
Een invloedrijk model van grijpaansturing stelt de modulariteit van
grijpbewegingen centraal 1,5 . Het transport van de hand tijdens de initiële
reikbeweging wordt bepaald door de extrinsieke locatie van een doel object. De
opening en configuratie van de hand wordt daarentegen bepaald door intrinsieke
eigenschappen van het object, zoals de vorm en grootte 243 . Deze twee componenten
worden vaak gekoppeld aan de twee parallelle pariëto-frontale kanalen: het
dorsomediale circuit stuurt de transport component aan, terwijl het dorsolaterale
kanaal de grijp component vormt 2,72,113 .
Vele studies hebben nauwkeurig beschreven hoe deze ventrale en dorsale circuits
zowel in anatomie als in functie gescheiden zijn, en dit is nog steeds een bloeiende
tak van onderzoek. Echter, voor een succesvolle grijpbeweging, zelfs voor zoiets
ogenschijnlijk gemakkelijks als het grijpen van een rijpe tomaat, maken we gebruik
van processen die zich afspelen in alle drie de kanalen tegelijkertijd. Recentelijk wordt
er daarom steeds vaker benadrukt dat voor een efficiënte en adaptieve aansturing
van grijpbewegingen de processen in deze gescheiden circuits moeten worden
geïntegreerd. Echter, tot op heden is deze integratie nog niet direct bestudeerd. Met
dit proefschrift hoop ik dit gat in ons begrip van bewegingsaansturing een beetje
kleiner te maken.
Defocusvanditproefschrift
In dit proefschrift wordt onderzocht hoe perceptuele en spatiaal kwantitatieve
informatie wordt geïncorporeerd in de aansturing van grijpbewegingen. Hoe delen
de gescheiden ventrale, dorsolaterale en dorsomediale circuits informatie om
d u t c h s u m m a r y | 193
samen te zorgen dat een gemeenschappelijk grijpdoel wordt behaald? En hoe is
deze integratie functioneel geïmplementeerd in de bekende corticale anatomie? Op
basis van de bestaande literatuur en de in dit proefschrift beschreven experimenten
wordt een bijgewerkt model van de aansturing van grijpbewegingen voorgesteld. Dit
geherstructureerde model beschrijft niet alleen hoe perceptuele informatie wordt
geïntegreerd in een bewegingsplan, maar legt ook een basis voor fundamentele
mechanismen in de aansturing van onze bewegingen. Het biedt een verklaring
voor het feit dat de transport en grijp componenten onafhankelijk lijken te worden
gecontroleerd, en maakt nieuwe voorspellingen over de functionele en temporele
karakteristieken van de processen in de twee pariëto-frontale circuits.
In de studies beschreven in dit proefschrift hebben de proefpersonen een
saaie taak: ze moeten keer op keer een rechthoekig blokje grijpen tussen duim
en wijsvinger. Hierbij staan twee experimentele factoren centraal: 1) de voorkeur
voor perceptuele of spatiaal kwantitatieve informatie voor het plannen van
grijpbewegingen, en 2) de vereiste complexiteit van accurate bewegingsplanning.
Hoofdstuk 2 beschrijft in detail hoe deze twee factoren worden gemanipuleerd
in de experimenten. Ten eerste, de manipulatie van perceptuo-motor integratie
is gebaseerd op de gescheiden verwerking van monoculaire (perceptuele) en
binoculaire (kwantitatieve) diepte indicatoren. Deze twee typen indicatoren zijn
essentieel voor het plannen van een grijpbeweging naar een object dat in de diepte
georiënteerd is (weggedraaid van de persoon af) 287,300,301 . Echter, de informatie
waarde (betrouwbaarheid) van deze indicatoren is verschillend afhankelijk van de
oriëntatie van het object 294,307. Dit maakt het mogelijk om de invloed van perceptuele
en spatiaal kwantitatieve componenten op grijpplanning onafhankelijk van elkaar
te bestuderen. Ten tweede, de oriëntatie van het object beïnvloed ook de visueel
spatiale complexiteit van het vereiste bewegingsplan 312,416 . Als het object verticaal
georiënteerd is, recht voor het lichaam, dan zijn de beoogde eindposities van de
vingers zichtbaar en zonder hindernissen bereikbaar. Echter, hoe verder het object
naar een horizontale configuratie toe gedraaid is, hoe minder goed de doellocatie
van de wijsvinger zichtbaar en bereikbaar is. Het object vormt steeds meer een
hindernis in het beoogde traject. Deze manipulatie van object oriëntatie (van
verticaal naar horizontaal) beïnvloed dus de complexiteit van het bewegingsplan,
zonder de intrinsieke vorm of grootte, noch de extrinsieke locatie van het object als
geheel te veranderen 1 .
De experimenten in dit proefschrift zijn ontworpen om de integratie van
perceptuele en motorische domeinen te bestuderen in zowel de hersenen als
het gedrag. Hiervoor zijn verschillende complementaire technieken gebruikt:
bewegingssensoren
(kinematica),
functionele
magnetische
resonantie
beeldvorming (fMRI), elektro-encefalografie (EEG), en transcraniële magnetische
stimulatie (TMS). Het is geen eenvoudige opgave om EEG met TMS te combineren,
A
19 4 | a p p e n di x
of om een proefpersoon natuurlijke grijpbewegingen te laten maken terwijl zijn
hersenactiviteit wordt gemeten in de MRI scanner. Onze aanpak van deze technische
problemen is beschreven in hoofdstuk 3. In hoofdstuk 4 wordt een nieuwe en breed
toepasbare techniek geïntroduceerd om globale ruis in de analyse van fMRI data te
verminderen. Hiermee is het mogelijk om de corticale gebieden die verantwoordelijk
zijn voor de aansturing van grijpbewegingen te identificeren en hun functionele
koppeling te bestuderen (hoofdstuk 5). De elektrische activiteit van de hersenen
werd tijdens de grijptaak gemeten met EEG. We richtten ons voornamelijk op
corticale oscillaties in de zogenaamde alpha en beta frequentie domeinen (8-12
en 18-24 Hz, respectievelijk). Een onderdrukking van deze alpha en beta oscillaties
duidt op een verhoogde neurale activiteit. Zo geven de spectrale en temporele
karakteristieken van deze oscillaties inzicht in de onderliggende neurale processen
en de interactie tussen de betrokken gebieden (hoofdstukken 6 en 7). Met behulp
van TMS kan de hersenactiviteit lokaal en tijdelijk worden verstoord, waarna de
gevolgen van deze interferentie voor de corticale oscillaties en de kinematica van de
grijpbewegingen kunnen worden bestudeerd. Op deze manier is de causale rol van
twee belangrijke gebieden (aIPS en sPOS, zoals beschreven in hoofdstuk 5), en de
functionele en temporele afhankelijkheid tussen de dorsomediale en dorsolaterale
circuits onderzocht.
aIPSstructureerteengrijpplanopbasisvan
kwantitatieveenperceptueleinformatie
De kwaliteit van de planning van een grijpbeweging komt goed tot uiting tijdens
de laatste fase van de uitvoering van de beweging, vlak voordat het object
wordt aangeraakt. Juist in deze fase verraadde de kinematica van de hand dat
proefpersonen een nauwkeuriger bewegingsplan hadden wanneer de configuratie
van het object goed kon worden afgeleid aan de hand van de aangeboden visuele
informatie, zowel wanneer die informatie kwantitatief of perceptueel van aard was.
De spatiale congruentie tussen de hand en het object was sterker en de vingers
werden met meer vertrouwen naar hun eindposities gebracht (hoofdstuk 6).
Met behulp van fMRI vonden we dat belangrijke gebieden in het dorsolaterale
circuit (aIPS en PMv) en in de ventrale visuele stroom (laterale occipitale complex,
LOC) actiever waren wanneer perceptuele informatie moest worden geïncorporeerd
om een goede planning van de grijpbeweging te ondersteunen(hoofdstuk 5). Ook
werd dan een versterkte functionele koppeling tussen aIPS en zowel LOC als PMv
waargenomen. Samen suggereren deze bevindingen dat aIPS niet alleen maar
spatiaal kwantitatieve object informatie verwerkt, zoals eerder al bekend was, maar
ook perceptuele informatie incorporeert om adaptief grijpgedrag te ondersteunen.
d u t ch s u m m a r y | 195
Deze observatie werd verder ondersteund door het bestuderen van corticale
oscillaties (hoofdstuk 6), met speciale aandacht voor het beta ritme (18-24 Hz).
De onderdrukking van beta oscillaties is een bekende indicator van een goede
bewegingsplanning325,329,445 . Als het bewegingsplan kon worden gebaseerd op
betrouwbare kwantitatieve of perceptuele informatie, dan werd een versterkte
onderdrukking van beta oscillaties over de linker motor cortex (die de rechter hand
aanstuurt) waargenomen. Dit effect had twee belangrijke eigenschappen. Ten eerste,
de versterkte beta onderdrukking trad op aan het begin van de bewegingsplanning.
Ten tweede, deze onderdrukking was extra prominent als de proefpersoon
gedurende het experiment meer ervaring had opgedaan met de mogelijke object
configuraties en de vereiste hand-object interacties. Met andere woorden, wanneer
je de configuratie van een object relatief gemakkelijk kan bepalen, en wanneer je
bekend bent met de interacties met het object, dan is de motorische hersenactiviteit
versterkt aan het begin van de bewegingsplanning.
Om te bevestigen dat juist aIPS causaal verantwoordelijk is voor deze specifieke
verbetering van de bewegingsplanning hebben we met behulp van TMS neurale
processen tijdelijk verstoord en de effecten hiervan op het gedrag en de corticale
oscillaties bestudeerd (hoofdstuk 6). Na TMS van aIPS was het voordeel van
betrouwbare diepte informatie voor de nauwkeurigheid van het einde van de
grijpbeweging verdwenen. Daarnaast was ook de verhoogde beta onderdrukking
aan het begin van de bewegingsplanning sterk verminderd. In de tweede helft van het
experiment was dit effect van TMS zelfs zo sterk dat de bewegingsplanning nu beter
was wanneer diepte informatie juist het minst betrouwbaar was. Als proefpersonen
meer ervaring hadden met de object configuraties zorgde TMS verstoring van aIPS
dat de informatieve diepte indicatoren en opgeslagen ervaringen nu juist niet goed
geïntegreerd konden worden. Met andere woorden, wanneer de object configuratie
wordt herkend verwacht het corticale motor systeem gebruik te kunnen maken
van eerdere ervaringen. Maar als aIPS verstoord is, kan deze kennis en ervaring
niet worden geïncorporeerd en is de bewegingsplanning dus minder goed, ook al is
eigenlijk wel optimale visuele informatie aangeboden.
Samenvattend, aIPS is noodzakelijk tijdens de initiatie van de bewegingsplanning,
en structureert de grijpbeweging op basis van zowel kwantitatieve als perceptuele
informatie, gebruikmakend van abstracte kennis van object configuraties en eerdere
ervaringen.
VervolgensspecificeertsPOSeenspatiaalgrijpplan
Bij het grijpen van objecten zijn beoogde doellocaties van de vingers vaak niet
direct zichtbaar, of vormt het object zelf een mogelijke hindernis op het geplande
bewegingstraject. Dit is niet zozeer van toepassing voor verticale objecten, maar
A
19 6 | a p p e n di x
juist wel voor objecten die horizontaal in de diepte georiënteerd zijn. Deze toename
in de complexiteit van de bewegingsplanning is goed terug te zien in de kinematica
van de hand en vingers, zowel bij transport als bij grijp parameters (hoofdstuk 7).
De maximale opening van de hand wordt vaak beschouwd als een indicator van de
moeilijkheid van de beweging: hoe moeilijker, hoe groter de ‘zekerheidsmarge’2,234 .
Echter, deze handopening is juist kleiner bij het grijpen van meer horizontaal,
dan verticaal georiënteerde objecten. In hoofdstuk 7 is aangetoond dat dit een
consequentie is van beperkingen in de biomechanica van de hand, en niet zoals
eerder gesuggereerd een gevolg van een andere grijpstrategie of omdat het object
kleiner lijkt314,439 .
Gebieden in het dorsomediale circuit, sPOS en PMd, zijn actief betrokken bij het
plannen van grijpbewegingen. De activiteit in deze gebieden, zoals gemeten met
fMRI, nam toe hoe verder het object naar een horizontale configuratie geroteerd
was, dus hoe complexer de vereiste visuo-motor transformaties. Deze toename in
activiteit werd alleen beïnvloed door de vereiste beweging, en niet door het type
diepte informatie dat beschikbaar was (hoofdstuk 5). De processen in deze gebieden
worden dus sterk bepaald door de gewenste uitkomst, en niet door de aangeboden
invoer van informatie. Deze bevindingen werden bevestigd met EEG metingen. Bij
hogere eisen aan de visuospatiale bewegingsplanning waren oscillaties in het alpha
ritme (8-12 Hz) sterker onderdrukt in mediale pariëtale en frontale gebieden, een
indicatie van een verhoogde corticale activiteit (hoofdstuk 7). Dit effect ontstond
aan het eind van de planningsfase en ging door tijdens het begin van de beweging.
Dit suggereert dat het dorsomediale circuit betrokken is bij de precieze specificering
en optimalisatie van het grijpplan aan de hand van actuele visuele informatie
(hoofdstukken 5 en 7).
Door middel van TMS werd de neurale activiteit in aIPS en sPOS verstoord.
Hierdoor kon de functionele, causale en temporele dynamiek van deze gebieden
worden bestudeerd. Verstoring van zowel aIPS als sPOS leidde tot een vermindering
van de hierboven beschreven onderdrukking van de alpha oscillaties in de
dorsomediale pariëto-frontale gebieden. aIPS verstoring verminderde de alpha
onderdrukking rond het begin van de beweging, terwijl TMS over sPOS een latere
component beïnvloedde. Dit belangrijke verschil in tijd suggereert dat dorsomediale
processen afhankelijk zijn van de verwerking van abstracte object eigenschappen in
aIPS (hoofdstuk 6). Na TMS over sPOS vonden we ook een aanhoudend versterkte
onderdrukking van het alpha ritme in de buurt van de linker aIPS, onafhankelijk
van de configuratie van het object of de aangeboden visuele informatie. Dit is
mogelijk een indicatie dat een verhoogde activiteit in het dorsolaterale circuit kan
compenseren voor een tijdelijke verstoring van sPOS (hoofdstuk 7).
Samenvattend, deze resultaten geven aan dat de dorsolaterale en dorsomediale
pariëto-frontale circuits samenwerken, maar beide op een verschillend moment
d u t ch s u m m a r y | 197
bijdragen aan de bewegingsplanning. Het dorsolaterale circuit initialiseert en
structureert de planning, waarna het dorsomediale circuit het plan verfijnt en
actuele visuospatiale informatie transformeert naar bewegingsaansturing.
Hoejeeenrijpetomaatgrijpt
Samengenomen bieden de studies in dit proefschrift een nieuwe interpretatie
van de rol van de dorsomediale en dorsolaterale pariëto-frontale circuits in de
aansturing van grijpbewegingen. De processen in deze twee netwerken zijn
hiërarchisch georganiseerd en verschillen in het abstractieniveau van de sensorimotor transformaties. Het dorsolaterale circuit creëert een initiële structuur voor
het bewegingsplan op basis van abstracte representaties van het object. Dit initiële
plan kan vervolgens dienen als een soort sjabloon voor verdere en nauwkeurige
specificatie van het bewegingsplan door het dorsomediale circuit aan de hand van
actuele visuospatiale informatie.
Dus, hoe grijp je dan een rijpe tomaat? De bevindingen beschreven in dit
proefschrift geven een speculatief en vereenvoudigt begrip van hoe de hersenen
deze taak mogelijk uitvoeren. Wanneer we naar een tomaat kijken die we willen
pakken worden verschillende eigenschappen van het visuele beeld parallel verwerkt:
een spatiale representatie van de omgeving, de afmetingen van het object, en
zijn identiteit en perceptuele karakteristieken. Het anterieure gedeelte van de
intrapariëtale sulcus (aIPS) in het dorsolaterale pariëto-frontale circuit construeert
snel een object-gerelateerd sjabloon voor het bewegingsplan, gebaseerd op
semantische en pragmatische kennis van de tomaat. De constructie van dit sjabloon
incorporeert zowel een kwantitatieve driedimensionale representatie van het
object, zoals aangeleverd door de dorsale occipito-pariëtale cortex, als de identiteit
van het object, een rode rijpe tomaat in dit geval, zoals verwerkt door het ventrale
visuele kanaal in de inferieure occipito-temporale cortex.
Het rijpe-tomaat-sjabloon wordt gedeeld met de ventrale premotor cortex
(PMv), die het transformeert om de bewegingen van de hand en vingers te vormen.
Maar hetzelfde sjabloon kan ook dienen als een zogenaamde ‘prior’ voor latere
processen in het mediale superieure deel van de pariëto-occipitale sulcus (sPOS)
in het dorsomediale circuit. Hierbij wordt een grijpplan gecreëerd en verfijnd op
basis van actuele visuospatiale informatie. Mogelijk behelzen deze visuo-motor
transformaties niet het transport van de arm en de opening van de hand als losse
modules, maar de grijpbeweging als geheel. Daarbij zouden dan de vingers nagenoeg
individueel worden geleid naar hun beoogde doellocaties 251 . Samen verzorgen deze
processen een nauwkeurige planning van de beweging van de vingers naar de tomaat,
gebaseerd op eerdere ervaring en semantische kennis van het object, aangepast
naar de specifieke spatiale karakteristieken en configuratie van de tomaat.
A
198 | a p p e n di x
Conclusie
De experimenten beschreven in dit proefschrift vormen de basis voor een nieuw
model van de corticale aansturing van grijpbewegingen door de dorsolaterale en
dorsomediale pariëto-frontale circuits. Ik stel voor dat de processen in deze circuits
weliswaar dezelfde bewegingscomponenten aansturen, maar functioneel en
representatief hiërarchisch georganiseerd zijn. De sensori-motor transformaties in
het dorsolaterale circuit zijn object gerelateerd en van een hoger abstractieniveau
dan de transformaties in het dorsomediale circuit. Het dorsolaterale circuit
incorporeert zowel spatiaal kwantitatieve als perceptuele object informatie in de
formatie van een initieel sjabloon voor het bewegingsplan. Dit sjabloon kan worden
gebruikt door het dorsomediale circuit als een ‘prior’ voor ruimtelijke specificatie van
de beweging, gebaseerd op actuele visuospatiale informatie. Dit model verschaft
een mechanistische basis dat zowel het ogenschijnlijke onderscheid tussen transport
en een grijp componenten, als het onderscheid tussen bewegingsplanning en online
controle kan verklaren.
Conceptuele, sensorische en motorische processen in de hersenen worden
vaak als strikt gescheiden, of juist als gelijk beschouwd. In beide benaderingen
worden de bijzonder interessante gevolgen van potentiële interacties tussen deze
domeinen genegeerd. Mogelijk kan het nieuwe model zoals hierboven uiteengezet
worden gegeneraliseerd naar andere onderdelen van het menselijk gedrag die
ook afhankelijk zijn van de integratie van perceptueel-semantische informatie en
visuospatiale transformaties, zoals het gebruik van gereedschap, of het maken en
begrijpen van gebaren. Zo zouden de in dit proefschrift beschreven mechanismen
een structuur kunnen vormen voor de integratie van conceptuele, sensorische en
motorische processen.
r e f e r e n c e l i s t | 1 99
Referencelist
1.
Arbib MA (1981) Perceptual structures and
distributed motor control. In: Motor Control
(VB Brooks, ed), pp 1449-1490 Baltimore:
William and Wilkins.
9.
2.
Jeannerod M, Arbib MA, Rizzolatti G &
Sakata H. Grasping objects: the cortical
mechanisms of visuomotor transformation.
TrendsNeurosci18: 314-20 (1995).
10. Goodale MA & Milner AD. Separate visual
pathways for perception and action. Trends
Neurosci15: 20-5 (1992).
3.
Brochier T & Umilta MA. Cortical control of
grasp in non-human primates. Curr Opin
Neurobiol17: 637-43 (2007).
4.
Culham JC & Valyear KF. Human parietal
cortex in action. Curr Opin Neurobiol 16:
205-12 (2006).
5.
Jeannerod M (1981) Intersegmental
coordination during reaching at natural
visual objects. In: Attential and performance
IX (J Long & Baddeley A, eds), pp 153-168
Hillsdale NJ: Erlbaum.
6.
Rossetti Y & Pisella L. Several ‘vision for
action’ systems: a guide to dissociating and
integrating dorsal and ventral functions
(Tutorial). Common Mechanisms in
PerceptionandAction19: 62-119 (2002).
7.
James TW, Culham J, Humphrey GK, Milner
AD & Goodale MA. Ventral occipital lesions
impair object recognition but not objectdirected grasping: an fMRI study. Brain126:
2463-75 (2003).
8.
Goodale MA, Milner AD, Jakobson LS
& Carey DP. A neurological dissociation
between perceiving objects and grasping
them. Nature349: 154-6 (1991).
Carey DP, Harvey M & Milner AD. Visuomotor
sensitivity for shape and orientation
in a patient with visual form agnosia.
Neuropsychologia34: 329-37 (1996).
11. Karnath HO & Perenin MT. Cortical control
of visually guided reaching: evidence from
patients with optic ataxia. CerebCortex15:
1561-9 (2005).
12. Perenin MT & Vighetto A (1983) Optic
ataxia: A specific disorder in visuomotor
coordination. In: Spatially oriented behavior
(A Hein & Jeannerod M, eds), pp 305-326
New York: Springer-Verlag.
13. Perenin MT & Vighetto A. Optic ataxia: a
specific disruption in visuomotor mechanisms.
I. Different aspects of the deficit in reaching
for objects. Brain111 ( Pt 3): 643-74 (1988).
14. Jeannerod M, Decety J & Michel F.
Impairment
of
grasping
movements
following a bilateral posterior parietal lesion.
Neuropsychologia32: 369-80 (1994).
15. Himmelbach M & Karnath HO. Dorsal and
ventral stream interaction: contributions from
optic ataxia. J Cogn Neurosci 17: 632-40
(2005).
16. Jeannerod M & Jacob P. Visual cognition: a
new look at the two-visual systems model.
Neuropsychologia43: 301-12 (2005).
A
200 | appendix
17. Frey SH. What puts the how in where? Tool
use and the divided visual streams hypothesis.
Cortex43: 368-75 (2007).
18. Wolpert DM, Ghahramani Z & Flanagan JR.
Perspectives and problems in motor learning.
TrendsCognSci5: 487-494 (2001).
19. Napier JR. The prehensile movements of the
human hand. J Bone Joint Surg Br 38-B:
902-13 (1956).
20. Parker CE. Behavioral diversity in ten species
of nonhuman primates. J Comp Physiol
Psychol87: 930-7 (1974).
21. Kaas JH. The evolution of the complex sensory
and motor systems of the human brain. Brain
ResBull75: 384-90 (2008).
22. Rushworth MF, Johansen-Berg H, Gobel SM
& Devlin JT. The left parietal and premotor
cortices: motor attention and selection.
Neuroimage20 Suppl 1: S89-100 (2003).
23. Rowe JB, Toni I, Josephs O, Frackowiak RS
& Passingham RE. The prefrontal cortex:
response selection or maintenance within
working memory? Science 288: 1656-60
(2000).
24. Majdandzic J, et al. The role of immediate
and final goals in action planning: an fMRI
study. Neuroimage37: 589-98 (2007).
25. Beurze SM, de Lange FP, Toni I & Medendorp
WP. Integration of target and effector
information in the human brain during reach
planning. JNeurophysiol97: 188-99 (2007).
26. Passingham RE, Toni I, Schluter N &
Rushworth MF. How do visual instructions
influence the motor system? NovartisFound
Symp218: 129-41; discussion 141-6 (1998).
27. Smeets JB, Brenner E & Martin J. Grasping
Occam’s razor. AdvExpMedBiol629: 499522 (2009).
28. Medendorp WP, Beurze SM, Van Pelt S
& Van Der Werf J. Behavioral and cortical
mechanisms for spatial coding and action
planning. Cortex44: 587-97 (2008).
29. Churchland MM, Cunningham JP, Kaufman
MT, Ryu SI & Shenoy KV. Cortical preparatory
activity: representation of movement or first
cog in a dynamical machine? Neuron 68:
387-400 (2010).
30. Wolpert DM & Flanagan JR. Motor prediction.
CurrBiol11: R729-32 (2001).
31. Pisella L, et al. An ‘automatic pilot’ for the
hand in human posterior parietal cortex:
toward reinterpreting optic ataxia. Nat
Neurosci3: 729-36 (2000).
32. Glover S. Separate visual representations in
the planning and control of action. Behav
BrainSci27: 3-24; discussion 24-78 (2004).
33. Leventhal AG, Rodieck RW & Dreher B.
Retinal ganglion cell classes in the Old World
monkey: morphology and central projections.
Science213: 1139-42 (1981).
34. Hendrickson AE, Wilson JR & Ogren MP. The
neuroanatomical organization of pathways
between the dorsal lateral geniculate nucleus
and visual cortex in Old World and New World
primates. JCompNeurol182: 123-36 (1978).
35. Mishkin M & Ungerleider LG. Contribution
of striate inputs to the visuospatial functions
of parieto-preoccipital cortex in monkeys.
BehavBrainRes6: 57-77 (1982).
36. Felleman DJ & Van Essen DC. Distributed
hierarchical processing in the primate cerebral
cortex. CerebCortex1: 1-47 (1991).
37. Georgieva S, Peeters R, Kolster H, Todd
JT & Orban GA. The processing of threedimensional shape from disparity in the
human brain. JNeurosci29: 727-42 (2009).
38. Georgieva SS, Todd JT, Peeters R & Orban
GA. The extraction of 3D shape from texture
and shading in the human brain. Cereb
Cortex18: 2416-38 (2008).
39. Janssen P, Vogels R & Orban GA. Selectivity
for 3D shape that reveals distinct areas within
macaque inferior temporal cortex. Science
288: 2054-6 (2000).
reference list | 201
40. Srivastava S, Orban GA, De Maziere PA
& Janssen P. A distinct representation of
three-dimensional shape in macaque anterior
intraparietal area: fast, metric, and coarse. J
Neurosci29: 10613-26 (2009).
41. Milner AD & Goodale MA (2006) The visual
brain in action (2nd ed.). Oxford; New York:
Oxford University Press.
42. Fogassi L & Luppino G. Motor functions of
the parietal lobe. Curr Opin Neurobiol 15:
626-31 (2005).
43. Dijkerman HC & de Haan EH. Somatosensory
processes subserving perception and action.
Behav Brain Sci 30: 189-201; discussion
201-39 (2007).
44. Kravitz DJ, Saleem KS, Baker CI & Mishkin
M. A new neural framework for visuospatial
processing. Nat Rev Neurosci 12: 217-30
(2011).
45. Rushworth MF, Paus T & Sipila PK. Attention
systems and the organization of the human
parietal cortex. J Neurosci 21: 5262-71
(2001).
46. Colby CL & Goldberg ME. Space and
attention in parietal cortex. Annu Rev
Neurosci22: 319-49 (1999).
47. Corbetta M & Shulman GL. Control of goaldirected and stimulus-driven attention in the
brain. NatRevNeurosci3: 201-15 (2002).
48. Serino A & Haggard P. Touch and the body.
NeurosciBiobehavRev34: 224-36 (2010).
49. de Vignemont F. Body schema and body
image--pros and cons. Neuropsychologia
48: 669-80 (2010).
50. Paillard J (1999) A double dissociation in
deafferented patients. In: Motor control,
today and tomorrow (GN Gantchev, Mori S &
Massion J, eds), pp 197-214 Sofia, Bulgaria:
Academic Publishing House.
51. Tsakiris M, Schutz-Bosbach S & Gallagher
S. On agency and body-ownership:
phenomenological
and
neurocognitive
reflections. Conscious Cogn 16: 645-60
(2007).
52. Makin TR, Holmes NP & Ehrsson HH. On the
other hand: dummy hands and peripersonal
space. BehavBrainRes191: 1-10 (2008).
53. Bueti D & Walsh V. The parietal cortex and
the representation of time, space, number and
other magnitudes. PhilosTransRSocLond
BBiolSci364: 1831-40 (2009).
54. Hubbard EM, Piazza M, Pinel P & Dehaene
S. Interactions between number and space in
parietal cortex. NatRevNeurosci6: 435-48
(2005).
55. Dehaene S, Molko N, Cohen L & Wilson
AJ. Arithmetic and the brain. Curr Opin
Neurobiol14: 218-24 (2004).
56. Grafton ST. The cognitive neuroscience of
prehension: recent developments. ExpBrain
Res204: 475-91 (2010).
57. Grefkes C & Fink GR. The functional
organization of the intraparietal sulcus in
humans and monkeys. J Anat 207: 3-17
(2005).
58. Andersen RA & Cui H. Intention, action
planning, and decision making in parietalfrontal circuits. Neuron63: 568-83 (2009).
59. Galletti C, Fattori P, Battaglini PP, Shipp S
& Zeki S. Functional demarcation of a border
between areas V6 and V6A in the superior
parietal gyrus of the macaque monkey. EurJ
Neurosci8: 30-52 (1996).
60. Galletti C, Kutz DF, Gamberini M, Breveglieri
R & Fattori P. Role of the medial parietooccipital cortex in the control of reaching and
grasping movements. Exp Brain Res 153:
158-70 (2003).
61. Gamberini M, Galletti C, Bosco A, Breveglieri
R & Fattori P. Is the Medial Posterior Parietal
Area V6A a Single Functional Area? J
Neurosci31: 5145-57 (2011).
62. Dow BM. Orientation and color columns
in monkey visual cortex. Cereb Cortex 12:
1005-15 (2002).
A
202 | appendix
63. Aflalo TN & Graziano MS. Organization
of the macaque extrastriate visual cortex
re-examined using the principle of spatial
continuity of function. J Neurophysiol 105:
305-20 (2011).
64. Mountcastle VB. Modality and topographic
properties of single neurons of cat’s somatic
sensory cortex. J Neurophysiol 20: 408-34
(1957).
65. Sereno MI, Pitzalis S & Martinez A.
Mapping of contralateral space in retinotopic
coordinates by a parietal cortical area in
humans. Science294: 1350-4 (2001).
66. Sereno MI & Huang RS. A human parietal
face area contains aligned head-centered
visual and tactile maps. Nat Neurosci 9:
1337-43 (2006).
67. Medendorp WP, Goltz HC, Vilis T & Crawford
JD. Gaze-centered updating of visual space in
human parietal cortex. JNeurosci23: 620914 (2003).
68. Cohen YE & Andersen RA. A common
reference frame for movement plans in the
posterior parietal cortex. Nat Rev Neurosci
3: 553-62 (2002).
69. Rizzolatti G, Luppino G & Matelli M. The
organization of the cortical motor system:
new concepts. Electroencephalogr Clin
Neurophysiol106: 283-96 (1998).
70. Colby CL. Action-oriented spatial reference
frames in cortex. Neuron20: 15-24 (1998).
71. Hinkley LB, Krubitzer LA, Padberg J &
Disbrow EA. Visual-manual exploration
and posterior parietal cortex in humans. J
Neurophysiol102: 3433-46 (2009).
74. Prado J, et al. Two cortical systems for
reaching in central and peripheral vision.
Neuron48: 849-58 (2005).
75. Ladavas E & Serino A. Action-dependent
plasticity
in
peripersonal
space
representations. Cogn Neuropsychol 25:
1099-113 (2008).
76. van Pelt S, Toni I, Diedrichsen J & Medendorp
WP. Repetition suppression dissociates
spatial frames of reference in human saccade
generation. J Neurophysiol 104: 1239-48
(2010).
77. Beurze SM, Toni I, Pisella L & Medendorp
WP. Reference frames for reach planning in
human parietofrontal cortex. JNeurophysiol
104: 1736-45 (2010).
78. Bernier PM & Grafton ST. Human posterior
parietal cortex flexibly determines reference
frames for reaching based on sensory context.
Neuron68: 776-88 (2010).
79. Buneo CA & Andersen RA. The posterior
parietal cortex: sensorimotor interface for the
planning and online control of visually guided
movements. Neuropsychologia 44: 2594606 (2006).
80. Pouget A & Snyder LH. Computational
approaches to sensorimotor transformations.
NatNeurosci3 Suppl: 1192-8 (2000).
81. McGuire LM & Sabes PN. Sensory
transformations and the use of multiple
reference frames for reach planning. Nat
Neurosci12: 1056-61 (2009).
82. Graziano MS & Aflalo TN. Mapping
behavioral repertoire onto the cortex. Neuron
56: 239-51 (2007).
72. Culham JC, Cavina-Pratesi C & Singhal A. The
role of parietal cortex in visuomotor control:
what have we learned from neuroimaging?
Neuropsychologia44: 2668-84 (2006).
83. Graziano MS & Aflalo TN. Rethinking
cortical organization: moving away from
discrete areas arranged in hierarchies.
Neuroscientist13: 138-47 (2007).
73. Heed T, Beurze SM, Toni I, Roder B &
Medendorp WP. Functional rather than
effector-specific organization of human
posterior parietal cortex. J Neurosci 31:
3066-76 (2011).
84. Amunts K, Schleicher A & Zilles K.
Cytoarchitecture of the cerebral cortex--more
than localization. Neuroimage 37: 1061-5;
discussion 1066-8 (2007).
reference list | 203
85. Pandya DN & Seltzer B. Intrinsic connections
and architectonics of posterior parietal cortex
in the rhesus monkey. J Comp Neurol 204:
196-210 (1982).
96. Galletti C, et al. The cortical connections
of area V6: an occipito-parietal network
processing visual information. EurJNeurosci
13: 1572-88 (2001).
86. Rizzolatti G & Matelli M. Two different
streams form the dorsal visual system:
anatomy and functions. ExpBrainRes153:
146-57 (2003).
97. Rizzolatti G, Fogassi L & Gallese V. Parietal
cortex: from sight to action. Curr Opin
Neurobiol7: 562-7 (1997).
87. Sereno MI & Tootell RB. From monkeys to
humans: what do we now know about brain
homologies? CurrOpinNeurobiol15: 135-44
(2005).
88. Mars RB, et al. Diffusion-weighted imaging
tractography-based parcellation of the
human parietal cortex and comparison with
human and macaque resting-state functional
connectivity. JNeurosci31: 4087-100 (2011).
89. von Economo C & Koskinas GN (1925)
Die Cytoarchitektonik der Hirnrinde des
erwachsenen Menschen. Wien: Springer
Verlag.
90. Brodmann K. Beiträge zur histologischen
Lokalisation
der
Grosshirnrinde.
VI.
Mitteilung: Die Cortexgliederung des
Menschen. J Psychol Neurol 10: 231-246
(1908).
91. Scheperjans F, et al. Observer-independent
cytoarchitectonic mapping of the human
superior parietal cortex. Cereb Cortex 18:
846-67 (2008).
92. Colby CL, Gattass R, Olson CR & Gross
CG. Topographical organization of cortical
afferents to extrastriate visual area PO in the
macaque: a dual tracer study. JCompNeurol
269: 392-413 (1988).
93. Gregoriou GG, Borra E, Matelli M & Luppino
G. Architectonic organization of the inferior
parietal convexity of the macaque monkey. J
CompNeurol496: 422-51 (2006).
94. Caspers S, et al. The human inferior parietal
cortex: cytoarchitectonic parcellation and
interindividual variability. Neuroimage 33:
430-48 (2006).
95. Burnod Y, et al. Parieto-frontal coding of
reaching: an integrated framework. Exp
BrainRes129: 325-46 (1999).
98. Murata A, Gallese V, Luppino G, Kaseda M &
Sakata H. Selectivity for the shape, size, and
orientation of objects for grasping in neurons
of monkey parietal area AIP. JNeurophysiol
83: 2580-601 (2000).
99. Sakata H, Taira M, Murata A & Mine S. Neural
mechanisms of visual guidance of hand action
in the parietal cortex of the monkey. Cereb
Cortex5: 429-38 (1995).
100. Taira M, Mine S, Georgopoulos AP, Murata
A & Sakata H. Parietal cortex neurons of the
monkey related to the visual guidance of hand
movement. ExpBrainRes83: 29-36 (1990).
101. Bosco A, Breveglieri R, Chinellato E, Galletti
C & Fattori P. Reaching activity in the medial
posterior parietal cortex of monkeys is
modulated by visual feedback. JNeurosci30:
14773-85 (2010).
102. Zhang M & Barash S. Persistent LIP activity
in memory antisaccades: working memory
for a sensorimotor transformation. J
Neurophysiol91: 1424-41 (2004).
103. Lewis JW & Van Essen DC. Mapping of
architectonic subdivisions in the macaque
monkey, with emphasis on parieto-occipital
cortex. JCompNeurol428: 79-111 (2000).
104. Galletti C, Fattori P, Kutz DF & Gamberini
M. Brain location and visual topography of
cortical area V6A in the macaque monkey. Eur
JNeurosci11: 575-82 (1999).
105. Breveglieri R, Kutz DF, Fattori P, Gamberini
M & Galletti C. Somatosensory cells in the
parieto-occipital area V6A of the macaque.
Neuroreport13: 2113-6 (2002).
106. Gamberini M, et al. Cortical connections of
the visuomotor parietooccipital area V6Ad of
the macaque monkey. J Comp Neurol 513:
622-42 (2009).
A
204 | appendix
107. Fattori P, Gamberini M, Kutz DF & Galletti
C. ‘Arm-reaching’ neurons in the parietal area
V6A of the macaque monkey. EurJNeurosci
13: 2309-13 (2001).
108. Fattori P, et al. Hand orientation during reachto-grasp movements modulates neuronal
activity in the medial posterior parietal area
V6A. JNeurosci29: 1928-36 (2009).
109. Fattori P, et al. The dorsomedial pathway
is not just for reaching: grasping neurons in
the medial parieto-occipital cortex of the
macaque monkey. J Neurosci 30: 342-9
(2010).
110. Filimon F, Nelson JD, Huang RS & Sereno
MI. Multiple parietal reach regions in
humans: cortical representations for visual
and proprioceptive feedback during on-line
reaching. JNeurosci29: 2961-71 (2009).
111. Connolly JD, Andersen RA & Goodale MA.
FMRI evidence for a ‘parietal reach region’ in
the human brain. ExpBrainRes153: 140-5
(2003).
112. Pitzalis S, et al. Wide-field retinotopy defines
human cortical visual area v6. JNeurosci26:
7962-73 (2006).
113. Cavina-Pratesi C, et al. Functional
magnetic resonance imaging reveals the
neural substrates of arm transport and
grip formation in reach-to-grasp actions in
humans. JNeurosci30: 10306-23 (2010).
114. Grol MJ, de Lange FP, Verstraten FA,
Passingham RE & Toni I. Cerebral changes
during performance of overlearned arbitrary
visuomotor associations. JNeurosci26: 11725 (2006).
115. Milner AD, et al. Grasping the past. delay can
improve visuomotor performance. Curr Biol
11: 1896-901 (2001).
116. Cavina-Pratesi C, Ietswaart M, Humphreys
GW, Lestou V & Milner AD. Impaired grasping
in a patient with optic ataxia: primary
visuomotor deficit or secondary consequence
of misreaching? Neuropsychologia48: 22634 (2010).
117. Johnson PB, Ferraina S, Bianchi L & Caminiti
R. Cortical networks for visual reaching:
physiological and anatomical organization of
frontal and parietal lobe arm regions. Cereb
Cortex6: 102-19 (1996).
118. Snyder LH, Batista AP & Andersen RA.
Coding of intention in the posterior parietal
cortex. Nature386: 167-70 (1997).
119. Eskandar EN & Assad JA. Dissociation of
visual, motor and predictive signals in parietal
cortex during visual guidance. NatNeurosci
2: 88-93 (1999).
120. Kalaska JF, Cisek P & Gosselin-Kessiby N.
Mechanisms of selection and guidance of
reaching movements in the parietal lobe. Adv
Neurol93: 97-119 (2003).
121. Batista AP, Buneo CA, Snyder LH &
Andersen RA. Reach plans in eye-centered
coordinates. Science285: 257-60 (1999).
122. van der Werf J, Jensen O, Fries P &
Medendorp WP. Neuronal synchronization in
human posterior parietal cortex during reach
planning. JNeurosci30: 1402-12 (2010).
123. Grefkes C, Ritzl A, Zilles K & Fink GR.
Human medial intraparietal cortex subserves
visuomotor
coordinate
transformation.
Neuroimage23: 1494-506 (2004).
124. Klam F & Graf W. Vestibular signals of
posterior parietal cortex neurons during
active and passive head movements in
macaque monkeys. AnnNYAcadSci1004:
271-82 (2003).
125. Cooke DF & Graziano MS. Super-flinchers
and nerves of steel: defensive movements
altered by chemical manipulation of a cortical
motor area. Neuron43: 585-93 (2004).
126. Cooke DF, Taylor CS, Moore T & Graziano
MS. Complex movements evoked by
microstimulation of the ventral intraparietal
area. ProcNatlAcadSciUSA100: 6163-8
(2003).
127. Colby CL, Duhamel JR & Goldberg ME.
Ventral intraparietal area of the macaque:
anatomic location and visual response
properties. J Neurophysiol 69: 902-14
(1993).
reference list | 205
128. Duhamel JR, Colby CL & Goldberg ME.
Ventral intraparietal area of the macaque:
congruent visual and somatic response
properties. J Neurophysiol 79: 126-36
(1998).
139. Muri RM, Iba-Zizen MT, Derosier C, Cabanis
EA & Pierrot-Deseilligny C. Location of the
human posterior eye field with functional
magnetic resonance imaging. J Neurol
NeurosurgPsychiatry60: 445-8 (1996).
129. Bremmer F, Klam F, Duhamel JR, Ben
Hamed S & Graf W. Visual-vestibular
interactive responses in the macaque ventral
intraparietal area (VIP). Eur J Neurosci 16:
1569-86 (2002).
140. Corbetta M, et al. A common network
of functional areas for attention and eye
movements. Neuron21: 761-73 (1998).
130. Bremmer F, et al. Polymodal motion
processing in posterior parietal and premotor
cortex: a human fMRI study strongly implies
equivalencies between humans and monkeys.
Neuron29: 287-96 (2001).
131. Bartels A, Zeki S & Logothetis NK. Natural
vision reveals regional specialization to local
motion and to contrast-invariant, global flow
in the human brain. CerebCortex18: 705-17
(2008).
132. Gnadt JW & Andersen RA. Memory related
motor planning activity in posterior parietal
cortex of macaque. ExpBrainRes70: 216-20
(1988).
133. Duhamel JR, Colby CL & Goldberg ME. The
updating of the representation of visual space
in parietal cortex by intended eye movements.
Science255: 90-2 (1992).
134. Mazzoni P, Bracewell RM, Barash S &
Andersen RA. Motor intention activity in
the macaque’s lateral intraparietal area.
I. Dissociation of motor plan from sensory
memory. JNeurophysiol76: 1439-56 (1996).
135. Andersen RA & Buneo CA. Intentional
maps in posterior parietal cortex. Annu Rev
Neurosci25: 189-220 (2002).
136. Bisley JW & Goldberg ME. Neuronal activity
in the lateral intraparietal area and spatial
attention. Science299: 81-6 (2003).
137. Bisley JW & Goldberg ME. Attention,
intention, and priority in the parietal lobe.
AnnuRevNeurosci33: 1-21 (2010).
138. Liu Y, Yttri EA & Snyder LH. Intention and
attention: different functional roles for LIPd
and LIPv. NatNeurosci13: 495-500 (2010).
141. Corbetta M, Shulman GL, Miezin FM
& Petersen SE. Superior parietal cortex
activation during spatial attention shifts and
visual feature conjunction. Science270: 8025 (1995).
142. Fink GR, Dolan RJ, Halligan PW, Marshall JC
& Frith CD. Space-based and object-based
visual attention: shared and specific neural
domains. Brain120 ( Pt 11): 2013-28 (1997).
143. Shikata E, Tanaka Y, Nakamura H, Taira M
& Sakata H. Selectivity of the parietal visual
neurones in 3D orientation of surface of
stereoscopic stimuli. Neuroreport 7: 238994 (1996).
144. Tsutsui K, Jiang M, Yara K, Sakata H & Taira
M. Integration of perspective and disparity
cues in surface-orientation-selective neurons
of area CIP. J Neurophysiol 86: 2856-67
(2001).
145. Shikata E, et al. Surface orientation
discrimination activates caudal and anterior
intraparietal sulcus in humans: an eventrelated fMRI study. J Neurophysiol 85:
1309-14 (2001).
146. Faillenot I, Sunaert S, Van Hecke P & Orban
GA. Orientation discrimination of objects
and gratings compared: an fMRI study. EurJ
Neurosci13: 585-96 (2001).
147. Sakata H. The role of the parietal cortex in
grasping. AdvNeurol93: 121-39 (2003).
148. Shikata E, et al. Functional properties and
interaction of the anterior and posterior
intraparietal areas in humans. EurJNeurosci
17: 1105-10 (2003).
A
206 | appendix
149. Sakaguchi Y, Ishida F, Shimizu T & Murata A.
Time course of information representation of
macaque AIP neurons in hand manipulation
task revealed by information analysis. J
Neurophysiol(2010).
150. Murata A, Gallese V, Kaseda M & Sakata H.
Parietal neurons related to memory-guided
hand manipulation. J Neurophysiol 75:
2180-6 (1996).
151. Gallese V, Murata A, Kaseda M, Niki N &
Sakata H. Deficit of hand preshaping after
muscimol injection in monkey parietal cortex.
Neuroreport5: 1525-9 (1994).
152. Binkofski F, et al. Human anterior intraparietal
area subserves prehension: a combined
lesion and functional MRI activation study.
Neurology50: 1253-9 (1998).
153. Choi HJ, et al. Cytoarchitectonic identification
and probabilistic mapping of two distinct
areas within the anterior ventral bank of the
human intraparietal sulcus. J Comp Neurol
495: 53-69 (2006).
154. Kroliczak G, Cavina-Pratesi C, Goodman DA
& Culham JC. What does the brain do when
you fake it? An FMRI study of pantomimed
and real grasping. JNeurophysiol97: 241022 (2007).
155. Cavina-Pratesi C, Goodale MA & Culham JC.
FMRI reveals a dissociation between grasping
and perceiving the size of real 3D objects.
PLoSOne2: e424 (2007).
156. Begliomini C, Wall MB, Smith AT & Castiello
U. Differential cortical activity for precision
and whole-hand visually guided grasping in
humans. EurJNeurosci25: 1245-52 (2007).
157. Frey SH, Vinton D, Norlund R & Grafton
ST. Cortical topography of human anterior
intraparietal cortex active during visually
guided grasping. BrainResCognBrainRes
23: 397-405 (2005).
158. Durand JB, Peeters R, Norman JF, Todd JT
& Orban GA. Parietal regions processing
visual 3D shape extracted from disparity.
Neuroimage46: 1114-26 (2009).
159. Janssen P, Srivastava S, Ombelet S & Orban
GA. Coding of shape and position in macaque
lateral intraparietal area. J Neurosci 28:
6679-90 (2008).
160. Davare M, Rothwell JC & Lemon RN. Causal
connectivity between the human anterior
intraparietal area and premotor cortex during
grasp. CurrBiol20: 176-81 (2010).
161. Rice NJ, Tunik E & Grafton ST. The anterior
intraparietal sulcus mediates grasp execution,
independent of requirement to update:
new insights from transcranial magnetic
stimulation. JNeurosci26: 8176-82 (2006).
162. Grefkes C, Weiss PH, Zilles K & Fink GR.
Crossmodal processing of object features in
human anterior intraparietal cortex: an fMRI
study implies equivalencies between humans
and monkeys. Neuron35: 173-84 (2002).
163. Johnson-Frey SH. The neural bases of
complex tool use in humans. TrendsCognSci
8: 71-8 (2004).
164. Creem-Regehr SH & Lee JN. Neural
representations of graspable objects: are
tools special? BrainResCognBrainRes22:
457-69 (2005).
165. Stark A & Zohary E. Parietal mapping of
visuomotor transformations during human
tool grasping. Cereb Cortex 18: 2358-68
(2008).
166. Vingerhoets G, Vandekerckhove E, Honore
P, Vandemaele P & Achten E. Neural
correlates of pantomiming familiar and
unfamiliar tools: Action semantics versus
mechanical problem solving? Hum Brain
Mapp(2010).
167. Shmuelof L & Zohary E. A mirror
representation of others’ actions in the
human anterior parietal cortex. J Neurosci
26: 9736-42 (2006).
168. Vingerhoets G, et al. Multifocal intraparietal
activation during discrimination of action
intention in observed tool grasping.
Neuroscience169: 1158-67 (2010).
169. Hamilton AF & Grafton ST. Goal
representation in human anterior intraparietal
sulcus. JNeurosci26: 1133-7 (2006).
reference list | 207
170. Nelissen K, et al. Action observation circuits
in the macaque monkey cortex. J Neurosci
31: 3743-56 (2011).
171. Van Overwalle F & Baetens K. Understanding
others’ actions and goals by mirror and
mentalizing systems: a meta-analysis.
Neuroimage48: 564-84 (2009).
172. Fogassi L, et al. Parietal lobe: from action
organization to intention understanding.
Science308: 662-7 (2005).
173. Barbas H & Pandya DN. Architecture and
frontal cortical connections of the premotor
cortex (area 6) in the rhesus monkey. JComp
Neurol256: 211-28 (1987).
174. Matelli M, Luppino G & Rizzolatti G.
Architecture of superior and mesial area 6 and
the adjacent cingulate cortex in the macaque
monkey. JCompNeurol311: 445-62 (1991).
175. Fogassi L, et al. Cortical mechanism for the
visual guidance of hand grasping movements
in the monkey: A reversible inactivation study.
Brain124: 571-86 (2001).
176. Rizzolatti G, et al. Functional organization
of inferior area 6 in the macaque monkey. II.
Area F5 and the control of distal movements.
ExpBrainRes71: 491-507 (1988).
177. Umilta MA, et al. When pliers become fingers
in the monkey motor system. ProcNatlAcad
SciUSA105: 2209-13 (2008).
178. Umilta MA, et al. I know what you are doing.
a neurophysiological study. Neuron 31: 15565 (2001).
179. di Pellegrino G, Fadiga L, Fogassi L, Gallese
V & Rizzolatti G. Understanding motor
events: a neurophysiological study. ExpBrain
Res91: 176-80 (1992).
180. Gallese V, Fadiga L, Fogassi L & Rizzolatti
G. Action recognition in the premotor cortex.
Brain119 ( Pt 2): 593-609 (1996).
181. Rizzolatti G, Fogassi L & Gallese V.
Neurophysiological mechanisms underlying
the understanding and imitation of action.
NatRevNeurosci2: 661-70 (2001).
182. Wise SP, Boussaoud D, Johnson PB &
Caminiti R. Premotor and parietal cortex:
corticocortical connectivity and combinatorial
computations. AnnuRevNeurosci20: 25-42
(1997).
183. Mars RB, Hulstijn W & Toni I. Selection,
preparation, and monitoring: current
approaches to studying the neural control of
action. Cortex44: 479-81 (2008).
184. Toni I, Rushworth MF & Passingham RE.
Neural correlates of visuomotor associations.
Spatial rules compared with arbitrary rules.
ExpBrainRes141: 359-69 (2001).
185. Passingham RE. Premotor cortex: sensory
cues and movement. Behav Brain Res 18:
175-85 (1985).
186. Rushworth MF, Buckley MJ, Behrens TE,
Walton ME & Bannerman DM. Functional
organization of the medial frontal cortex.
CurrOpinNeurobiol17: 220-7 (2007).
187. Hoshi E & Tanji J. Differential roles of
neuronal activity in the supplementary
and presupplementary motor areas: from
information retrieval to motor planning and
execution. J Neurophysiol 92: 3482-99
(2004).
188. Luppino G, Matelli M, Camarda R &
Rizzolatti G. Corticocortical connections of
area F3 (SMA-proper) and area F6 (pre-SMA)
in the macaque monkey. JCompNeurol338:
114-40 (1993).
189. Mars RB, Coles MG, Hulstijn W & Toni I.
Delay-related cerebral activity and motor
preparation. Cortex44: 507-20 (2008).
190. Nachev P, Wydell H, O’Neill K, Husain M &
Kennard C. The role of the pre-supplementary
motor area in the control of action.
Neuroimage36 Suppl 2: T155-63 (2007).
191. Rushworth MF, Walton ME, Kennerley SW &
Bannerman DM. Action sets and decisions in
the medial frontal cortex. TrendsCognSci8:
410-7 (2004).
192. Matelli M, Camarda R, Glickstein M &
Rizzolatti G. Afferent and efferent projections
of the inferior area 6 in the macaque monkey.
JCompNeurol251: 281-98 (1986).
A
208 | appendix
193. Porter R & Lemon R (1993) Corticospinal
function and voluntary movement. Oxford;
New York: Oxford University Press;
Clarendon Press.
202. Sanides F. The cyto-myeloarchitecture of
the human frontal lobe and its relation to
phylogenetic differentiation of the cerebral
cortex. JHirnforsch47: 269-82 (1964).
194. Lacquaniti F, Guigon E, Bianchi L, Ferraina
S & Caminiti R. Representing spatial
information for limb movement: role of area
5 in the monkey. Cereb Cortex 5: 391-409
(1995).
203. Uddin LQ, et al. Dissociable connectivity
within human angular gyrus and intraparietal
sulcus: evidence from functional and
structural connectivity. Cereb Cortex 20:
2636-46 (2010).
195. Andersen RA, Asanuma C, Essick G &
Siegel RM. Corticocortical connections of
anatomically and physiologically defined
subdivisions within the inferior parietal lobule.
JCompNeurol296: 65-113 (1990).
204. Borra E, et al. Cortical connections of the
macaque anterior intraparietal (AIP) area.
CerebCortex18: 1094-111 (2008).
196. Schall JD, Morel A, King DJ & Bullier J.
Topography of visual cortex connections with
frontal eye field in macaque: convergence
and segregation of processing streams. J
Neurosci15: 4464-87 (1995).
197. Luppino G, Murata A, Govoni P & Matelli
M. Largely segregated parietofrontal
connections linking rostral intraparietal
cortex (areas AIP and VIP) and the ventral
premotor cortex (areas F5 and F4). ExpBrain
Res128: 181-7 (1999).
198. Tanne-Gariepy J, Rouiller EM & Boussaoud
D. Parietal inputs to dorsal versus ventral
premotor areas in the macaque monkey:
evidence for largely segregated visuomotor
pathways. ExpBrainRes145: 91-103 (2002).
199. Tomassini V, et al. Diffusion-weighted
imaging tractography-based parcellation
of the human lateral premotor cortex
identifies dorsal and ventral subregions with
anatomical and functional specializations. J
Neurosci27: 10259-69 (2007).
200. Caminiti R, Ferraina S & Johnson PB. The
sources of visual information to the primate
frontal lobe: a novel role for the superior
parietal lobule. Cereb Cortex 6: 319-28
(1996).
201. Hoshi E & Tanji J. Distinctions between dorsal
and ventral premotor areas: anatomical
connectivity and functional properties. Curr
OpinNeurobiol17: 234-42 (2007).
205. Grol MJ, et al. Parieto-frontal connectivity
during visually guided grasping. J Neurosci
27: 11877-87 (2007).
206. Kurata K & Tanji J. Premotor cortex neurons
in macaques: activity before distal and
proximal forelimb movements. JNeurosci6:
403-11 (1986).
207. Davare M, Andres M, Cosnard G, Thonnard
JL & Olivier E. Dissociating the role of ventral
and dorsal premotor cortex in precision
grasping. JNeurosci26: 2260-8 (2006).
208. Tokuno H & Tanji J. Input organization of
distal and proximal forelimb areas in the
monkey primary motor cortex: a retrograde
double labeling study. J Comp Neurol 333:
199-209 (1993).
209. Kwan HC, MacKay WA, Murphy JT & Wong
YC. Spatial organization of precentral cortex
in awake primates. II. Motor outputs. J
Neurophysiol41: 1120-31 (1978).
210. Graziano MS, Taylor CS & Moore T. Complex
movements evoked by microstimulation of
precentral cortex. Neuron34: 841-51 (2002).
211. Rathelot JA & Strick PL. Subdivisions of
primary motor cortex based on corticomotoneuronal cells. ProcNatlAcadSciUS
A106: 918-23 (2009).
212. Colebatch JG, Rothwell JC, Day BL,
Thompson PD & Marsden CD. Cortical
outflow to proximal arm muscles in man.
Brain113 ( Pt 6): 1843-56 (1990).
reference list | 209
213. Kuypers HG (1981) Anatomy of the
descending pathways. In: Handbook of
physiology. The nervous system II (JM
Brookhart & Mountcastle VB, eds), pp 597666 Bethesda, MD: American Physiological
Society.
214. Dijkerman HC, Vargha-Khadem F, Polkey
CE & Weiskrantz L. Ipsilesional and
contralesional sensorimotor function after
hemispherectomy: differences between distal
and proximal function. Neuropsychologia
46: 886-901 (2008).
215. Kunzle H. Bilateral projections from precentral
motor cortex to the putamen and other parts
of the basal ganglia. An autoradiographic
study in Macaca fascicularis. Brain Res 88:
195-209 (1975).
216. Graybiel AM, Aosaki T, Flaherty AW &
Kimura M. The basal ganglia and adaptive
motor control. Science265: 1826-31 (1994).
217. Middleton FA & Strick PL. Basal ganglia and
cerebellar loops: motor and cognitive circuits.
BrainResBrainResRev31: 236-50 (2000).
218. Alexander GE, DeLong MR & Strick
PL. Parallel organization of functionally
segregated circuits linking basal ganglia and
cortex. AnnuRevNeurosci9: 357-81 (1986).
219. Nathan PW, Smith MC & Deacon P. The
corticospinal tracts in man. Course and
location of fibres at different segmental
levels. Brain113 ( Pt 2): 303-24 (1990).
220. Lemon RN, Kirkwood PA, Maier MA,
Nakajima K & Nathan P. Direct and indirect
pathways for corticospinal control of upper
limb motoneurons in the primate. ProgBrain
Res143: 263-79 (2004).
221. Brinkman J & Kuypers HG. Cerebral control
of contralateral and ipsilateral arm, hand and
finger movements in the split-brain rhesus
monkey. Brain96: 653-74 (1973).
222. Maier MA, et al. Differences in the
corticospinal projection from primary motor
cortex and supplementary motor area
to macaque upper limb motoneurons: an
anatomical and electrophysiological study.
CerebCortex12: 281-96 (2002).
223. Petrides M. Lateral prefrontal cortex:
architectonic and functional organization.
Philos Trans R Soc Lond B Biol Sci 360:
781-95 (2005).
224. Gerbella M, Belmalih A, Borra E, Rozzi S
& Luppino G. Cortical connections of the
anterior (F5a) subdivision of the macaque
ventral premotor area F5. BrainStructFunct
216: 43-65 (2011).
225. Fuster JM (2008) The prefrontal cortex.
London: Academic Press.
226. Miller EK & Cohen JD. An integrative theory
of prefrontal cortex function. Annu Rev
Neurosci24: 167-202 (2001).
227. Burgess PW, Dumontheil I & Gilbert SJ.
The gateway hypothesis of rostral prefrontal
cortex (area 10) function. Trends Cogn Sci
11: 290-8 (2007).
228. Badre D & D’Esposito M. Is the rostro-caudal
axis of the frontal lobe hierarchical? NatRev
Neurosci10: 659-69 (2009).
229. Badre D, Kayser AS & D’Esposito M. Frontal
cortex and the discovery of abstract action
rules. Neuron66: 315-26 (2010).
230. Koechlin E, Basso G, Pietrini P, Panzer S &
Grafman J. The role of the anterior prefrontal
cortex in human cognition. Nature399: 14851 (1999).
231. Burgess PW, Quayle A & Frith CD. Brain
regions involved in prospective memory as
determined by positron emission tomography.
Neuropsychologia39: 545-55 (2001).
232. Borra E, Ichinohe N, Sato T, Tanifuji M &
Rockland KS. Cortical connections to area
TE in monkey: hybrid modular and distributed
organization. Cereb Cortex 20: 257-70
(2010).
233. Gerbella M, Belmalih A, Borra E, Rozzi S
& Luppino G. Cortical connections of the
macaque caudal ventrolateral prefrontal
areas 45A and 45B. CerebCortex20: 141-68
(2010).
A
210 | appendix
234. Borra E, Gerbella M, Rozzi S & Luppino G.
Anatomical evidence for a possible role of
the macaque prefrontal area 12r in action
organization and cognitive motor functions.
in 7th Forum of European Neuroscience. 2010.
Amsterdam, the Netherlands.
235. Lewis JW & Van Essen DC. Corticocortical
connections of visual, sensorimotor, and
multimodal processing areas in the parietal
lobe of the macaque monkey. JCompNeurol
428: 112-37 (2000).
236. Rozzi S, et al. Cortical connections of the
inferior parietal cortical convexity of the
macaque monkey. Cereb Cortex 16: 1389417 (2006).
237. Bruce C, Desimone R & Gross CG. Visual
properties of neurons in a polysensory area
in superior temporal sulcus of the macaque. J
Neurophysiol46: 369-84 (1981).
238. Nakamura H, et al. From three-dimensional
space vision to prehensile hand movements:
the lateral intraparietal area links the area
V3A and the anterior intraparietal area in
macaques. JNeurosci21: 8174-87 (2001).
239. Tunik E, Frey SH & Grafton ST. Virtual lesions
of the anterior intraparietal area disrupt goaldependent on-line adjustments of grasp. Nat
Neurosci8: 505-11 (2005).
240. Tunik E, Ortigue S, Adamovich SV &
Grafton ST. Differential recruitment of
anterior intraparietal sulcus and superior
parietal lobule during visually guided
grasping revealed by electrical neuroimaging.
JNeurosci28: 13615-20 (2008).
241. Friston KJ, Harrison L & Penny W. Dynamic
causal modelling. Neuroimage19: 1273-302
(2003).
244. Buneo CA, Jarvis MR, Batista AP & Andersen
RA. Direct visuomotor transformations for
reaching. Nature416: 632-6 (2002).
245. Culham JC, et al. Visually guided grasping
produces fMRI activation in dorsal but not
ventral stream brain areas. Exp Brain Res
153: 180-9 (2003).
246. Jeannerod M, Paulignan Y & Weiss P.
Grasping an object: one movement, several
components. Novartis Found Symp 218:
5-16; discussion 16-20 (1998).
247. Stark E, Asher I & Abeles M. Encoding of
reach and grasp by single neurons in premotor
cortex is independent of recording site. J
Neurophysiol97: 3351-64 (2007).
248. Zaal FT, Bootsma RJ & van Wieringen PC.
Coordination in prehension. Informationbased coupling of reaching and grasping. Exp
BrainRes119: 427-35 (1998).
249. Ulloa A & Bullock D. A neural network
simulating human reach-grasp coordination
by continuous updating of vector positioning
commands. Neural Netw 16: 1141-60
(2003).
250. Haggard P & Wing A. Coordinated responses
following mechanical perturbation of the arm
during prehension. ExpBrainRes102: 48394 (1995).
251. Smeets JB & Brenner E. A new view on
grasping. MotorControl3: 237-71 (1999).
252. Smeets JB, Martin J & Brenner E. Similarities
between digits’ movements in grasping,
touching and pushing. Exp Brain Res 203:
339-46 (2010).
253. Arbib MA, Bonaiuto JB, Jacobs S & Frey
SH. Tool use and the distalization of the endeffector. PsycholRes73: 441-62 (2009).
242. Johnson SH & Grafton ST. From ‘acting on’
to ‘acting with’: the functional anatomy of
object-oriented action schemata. ProgBrain
Res142: 127-39 (2003).
254. Obayashi S, et al. Functional brain mapping
of monkey tool use. Neuroimage14: 853-61
(2001).
243. Jeannerod M. The timing of natural
prehension movements. J Mot Behav 16:
235-54 (1984).
255. Ramayya AG, Glasser MF & Rilling JK. A DTI
investigation of neural substrates supporting
tool use. CerebCortex20: 507-16 (2010).
r e f e r e n c e l i s t | 211
256. Frey SH. Tool use, communicative gesture
and cerebral asymmetries in the modern
human brain. Philos Trans R Soc Lond B
BiolSci363: 1951-7 (2008).
267. Shinoda Y, Yokota J & Futami T. Divergent
projection of individual corticospinal axons
to motoneurons of multiple muscles in the
monkey. NeurosciLett23: 7-12 (1981).
257. Haaland KY, Harrington DL & Knight RT.
Neural representations of skilled movement.
Brain123 ( Pt 11): 2306-13 (2000).
268. Andersen P, Hagan PJ, Phillips CG &
Powell TP. Mapping by microstimulation of
overlapping projections from area 4 to motor
units of the baboon’s hand. ProcRSocLond
BBiolSci188: 31-6 (1975).
258. Buxbaum LJ, Johnson-Frey SH & BartlettWilliams M. Deficient internal models for
planning hand-object interactions in apraxia.
Neuropsychologia43: 917-29 (2005).
259. Dijkerman HC, et al. Reaching errors in optic
ataxia are linked to eye position rather than
head or body position. Neuropsychologia
44: 2766-73 (2006).
260. Buxbaum LJ, Sirigu A, Schwartz MF
& Klatzky R. Cognitive representations
of hand posture in ideomotor apraxia.
Neuropsychologia41: 1091-113 (2003).
269. Schieber MH & Hibbard LS. How somatotopic
is the motor cortex hand area? Science261:
489-92 (1993).
270. Rathelot JA & Strick PL. Muscle
representation in the macaque motor cortex:
an anatomical perspective. Proc Natl Acad
SciUSA103: 8257-62 (2006).
271. Pesyna C, Pundi K & Flanders M.
Coordination of hand shape. J Neurosci 31:
3757-65 (2011).
261. De Renzi E, Motti F & Nichelli P. Imitating
gestures. A quantitative approach to
ideomotor apraxia. Arch Neurol 37: 6-10
(1980).
272. Meier JD, Aflalo TN, Kastner S & Graziano
MS. Complex organization of human primary
motor cortex: a high-resolution fMRI study. J
Neurophysiol100: 1800-12 (2008).
262. Jeannerod M. Mechanisms of visuomotor
coordination: a study in normal and braindamaged subjects. Neuropsychologia 24:
41-78 (1986).
273. Bortoff GA & Strick PL. Corticospinal
terminations in two new-world primates:
further evidence that corticomotoneuronal
connections provide part of the neural
substrate for manual dexterity. J Neurosci
13: 5105-18 (1993).
263. Sirigu A, et al. A selective impairment of
hand posture for object utilization in apraxia.
Cortex31: 41-55 (1995).
264. Ietswaart M, Carey DP, Della Sala S &
Dijkhuizen RS. Memory-driven movements
in limb apraxia: is there evidence for impaired
communication between the dorsal and the
ventral streams? Neuropsychologia 39:
950-61 (2001).
265. Dawson AM, Buxbaum LJ & Duff SV. The
impact of left hemisphere stroke on force
control with familiar and novel objects:
neuroanatomic substrates and relationship to
apraxia. BrainRes1317: 124-36 (2010).
266. Penfield W & Jasper HH (1954) Epilepsy and
the functional anatomy of the human brain.
Boston,: Little.
274. Lemon RN & Griffiths J. Comparing the
function of the corticospinal system in
different species: organizational differences
for motor specialization? Muscle Nerve 32:
261-79 (2005).
275. Dum RP & Strick PL. Motor areas in the
frontal lobe of the primate. Physiol Behav
77: 677-82 (2002).
276. Murata Y, et al. Effects of motor training
on the recovery of manual dexterity after
primary motor cortex lesion in macaque
monkeys. JNeurophysiol99: 773-86 (2008).
277. Eccles JC, Fatt P & Koketsu K. Cholinergic
and inhibitory synapses in a pathway from
motor-axon collaterals to motoneurones. J
Physiol126: 524-62 (1954).
A
212 | ap p e ndix
278. Angel RW. Efference copy in the control of
movement. Neurology26: 1164-8 (1976).
279. Bell CC. An efference copy which is modified
by reafferent input. Science 214: 450-53
(1981).
280. Blakemore SJ, Frith CD & Wolpert DM.
Spatio-temporal prediction modulates the
perception of self-produced stimuli. J Cogn
Neurosci11: 551-9 (1999).
281. Helmholtz Hv (1867) Handbuch
physiologischen Optik. Leipzig,: Voss.
der
282. Poulet JF & Hedwig B. New insights into
corollary discharges mediated by identified
neural pathways. TrendsNeurosci30: 14-21
(2007).
283. Wolpert DM & Miall RC. Forward models for
physiological motor control. NeuralNetw9:
1265-1279 (1996).
284. Friston KJ, Kilner J & Harrison L. A free
energy principle for the brain. JPhysiolParis
100: 70-87 (2006).
285. Friston KJ. The free-energy principle: a
unified brain theory? Nat Rev Neurosci 11:
127-38 (2010).
286. Parker AJ. Binocular depth perception and
the cerebral cortex. Nat Rev Neurosci 8:
379-91 (2007).
287. Minini L, Parker AJ & Bridge H. Neural
modulation by binocular disparity greatest in
human dorsal visual stream. JNeurophysiol
104: 169-78 (2010).
288. Nelissen K, et al. The extraction of depth
structure from shading and texture in the
macaque brain. PLoSOne4: e8306 (2009).
289. Huxlin KR, Saunders RC, Marchionini D,
Pham HA & Merigan WH. Perceptual deficits
after lesions of inferotemporal cortex in
macaques. CerebCortex10: 671-83 (2000).
290. Kovacs G, et al. Effects of surface cues on
macaque inferior temporal cortical responses.
CerebCortex13: 178-88 (2003).
291. Kourtzi Z & Kanwisher N. Representation of
perceived object shape by the human lateral
occipital complex. Science 293: 1506-9
(2001).
292. Amedi A, Jacobson G, Hendler T, Malach
R & Zohary E. Convergence of visual and
tactile shape processing in the human lateral
occipital complex. CerebCortex12: 1202-12
(2002).
293. Knill DC & Saunders JA. Do humans optimally
integrate stereo and texture information for
judgments of surface slant? Vision Res 43:
2539-58 (2003).
294. Knill DC. Reaching for visual cues to depth:
the brain combines depth cues differently for
motor control and perception. JVis5: 103-15
(2005).
295. Servos P, Goodale MA & Jakobson LS. The
role of binocular vision in prehension - a
kinematic analysis. Vision Research 32:
1513-1521 (1992).
296. Jackson SR, Jones CA, Newport R &
Pritchard C. A kinematic analysis of goaldirected prehension movements executed
under binocular, monocular, and memoryguided viewing conditions. VisualCognition
4: 113-142 (1997).
297. Bradshaw MF, et al. Binocular cues and the
control of prehension. Spat Vis 17: 95-110
(2004).
298. Jackson SR, Newport R & Shaw A. Monocular
vision leads to a dissociation between grip
force and grip aperture scaling during reachto-grasp movements. Curr Biol 12: 237-40
(2002).
299. Loftus A, Servos P, Goodale MA,
Mendarozqueta N & Mon-Williams M. When
two eyes are better than one in prehension:
monocular viewing and end-point variance.
ExpBrainRes158: 317-27 (2004).
300. Mon-Williams M, Tresilian JR, McIntosh
RD & Milner AD. Monocular and binocular
distance cues: insights from visual form
agnosia I (of III). ExpBrainRes139: 127-36
(2001).
r e f e r e n c e li s t | 213
301. Dijkerman HC, Milner AD & Carey DP. The
perception and prehension of objects oriented
in the depth plane. I. Effects of visual form
agnosia. ExpBrainRes112: 442-51 (1996).
312. Gentilucci M, Daprati E, Gangitano M,
Saetti MC & Toni I. On orienting the hand to
reach and grasp an object. Neuroreport 7:
589-92 (1996).
302. Dijkerman HC, Le S, Demonet JF & Milner
AD. Visuomotor performance in a patient
with visual agnosia due to an early lesion.
BrainResCognBrainRes20: 12-25 (2004).
313. Haggard P & Wing A. Coordination of hand
aperture with the spatial path of hand
transport. ExpBrainRes118: 286-92 (1998).
303. Marotta JJ, Behrmann M & Goodale MA.
The removal of binocular cues disrupts the
calibration of grasping in patients with visual
form agnosia. Exp Brain Res 116: 113-21
(1997).
304. Mon-Williams M & Dijkerman HC. The use
of vergence information in the programming
of prehension. Exp Brain Res 128: 578-82
(1999).
305. Mon-Williams M, Tresilian JR & Roberts A.
Vergence provides veridical depth perception
from horizontal retinal image disparities. Exp
BrainRes133: 407-13 (2000).
306. Quinlan DJ & Culham JC. fMRI reveals a
preference for near viewing in the human
parieto-occipital cortex. Neuroimage 36:
167-87 (2007).
307. Knill DC. Surface orientation from texture:
ideal observers, generic observers and the
information content of texture cues. Vision
Res38: 1655-82 (1998).
308. Young MJ, Landy MS & Maloney LT. A
perturbation analysis of depth perception
from combinations of texture and motion
cues. VisionRes33: 2685-96 (1993).
309. Keefe BD, Hibbard PB & Watt SJ. Depthcue integration in grasp programming:
No evidence for a binocular specialism.
Neuropsychologia(2011).
310. Melmoth DR, Finlay AL, Morgan MJ & Grant
S. Grasping deficits and adaptations in adults
with stereo vision losses. InvestOphthalmol
VisSci50: 3711-20 (2009).
311. Keefe BD & Watt SJ. The role of binocular
vision in grasping: a small stimulus-set
distorts results. Exp Brain Res 194: 435-44
(2009).
314. Mamassian P. Prehension of objects oriented
in three-dimensional space. Exp Brain Res
114: 235-45 (1997).
315. Berger H. Über das elektroenkephalogramm
des menschen. ArchPsychiatrNervenkr87:
527-570 (1929).
316. Liljestrom M, Kujala J, Jensen O & Salmelin
R. Neuromagnetic localization of rhythmic
activity in the human brain: a comparison
of three methods. Neuroimage 25: 734-45
(2005).
317. Hamalainen M, Hari R, Ilmoniemi
RJ, Knuutila J & Lounasmaa OV.
Magnetoencephalography
Theory,
Instrumentation, and Applications to
Noninvasive Studies of the Working Human
Brain. ReviewsofModernPhysics65: 413497 (1993).
318. Hari R & Salmelin R. Human cortical
oscillations: a neuromagnetic view through
the skull. TrendsNeurosci20: 44-9 (1997).
319. Jensen O, Kaiser J & Lachaux JP. Human
gamma-frequency oscillations associated
with attention and memory. TrendsNeurosci
30: 317-24 (2007).
320. Fries
P.
Neuronal
gamma-band
synchronization as a fundamental process in
cortical computation. Annu Rev Neurosci
32: 209-24 (2009).
321. Jensen O & Mazaheri A. Shaping functional
architecture by oscillatory alpha activity:
gating by inhibition. FrontHumNeurosci4:
186 (2010).
322. Jensen O & Colgin LL. Cross-frequency
coupling between neuronal oscillations.
TrendsCognSci11: 267-9 (2007).
A
214 | a p p e n dix
323. Osipova D, Hermes D & Jensen O. Gamma
power is phase-locked to posterior alpha
activity. PLoSOne3: e3990 (2008).
324. Mazaheri A, Nieuwenhuis IL, van Dijk H &
Jensen O. Prestimulus alpha and mu activity
predicts failure to inhibit motor responses.
HumBrainMapp30: 1791-800 (2009).
325. Jensen O, et al. On the human sensorimotorcortex beta rhythm: sources and modeling.
Neuroimage26: 347-55 (2005).
326. Lakatos P, Karmos G, Mehta AD, Ulbert I
& Schroeder CE. Entrainment of neuronal
oscillations as a mechanism of attentional
selection. Science320: 110-3 (2008).
327. Schroeder CE & Lakatos P. Low-frequency
neuronal oscillations as instruments of
sensory selection. TrendsNeurosci32: 9-18
(2009).
334. Ogawa S, et al. Functional brain mapping by
blood oxygenation level-dependent contrast
magnetic resonance imaging. A comparison
of signal characteristics with a biophysical
model. BiophysJ64: 803-12 (1993).
335. Buxton RB, Wong EC & Frank LR. Dynamics
of blood flow and oxygenation changes
during brain activation: the balloon model.
MagnResonMed39: 855-64 (1998).
336. Stefanovic B, Schwindt W, Hoehn M & Silva
AC. Functional uncoupling of hemodynamic
from neuronal response by inhibition of
neuronal nitric oxide synthase. JCerebBlood
FlowMetab27: 741-54 (2007).
337. Rossi DJ. Another BOLD role for astrocytes:
coupling blood flow to neural activity. Nat
Neurosci9: 159-61 (2006).
338. Friston KJ, et al. Analysis of fMRI time-series
revisited. Neuroimage2: 45-53 (1995).
328. Klimesch W, Sauseng P & Hanslmayr S.
EEG alpha oscillations: The inhibition-timing
hypothesis. Brain Research Reviews 53:
63-88 (2007).
339. Friston KJ, et al. Spatial registration and
normalization of images. Human Brain
Mapping3: 165-189 (1995).
329. Neuper C, Wortz M & Pfurtscheller G.
ERD/ERS patterns reflecting sensorimotor
activation and deactivation. ProgBrainRes
159: 211-22 (2006).
340. Raj D, Anderson AW & Gore JC. Respiratory
effects in human functional magnetic
resonance imaging due to bulk susceptibility
changes. PhysMedBiol46: 3331-40 (2001).
330. Norris DG. Principles of magnetic resonance
assessment of brain function. JMagnReson
Imaging23: 794-807 (2006).
341. Diedrichsen J & Shadmehr R. Detecting and
adjusting for artifacts in fMRI time series
data. Neuroimage27: 624-34 (2005).
331. Ogawa S, et al. Intrinsic signal changes
accompanying
sensory
stimulation:
functional brain mapping with magnetic
resonance imaging. ProcNatlAcadSciUS
A89: 5951-5 (1992).
342. Birn RM, Bandettini PA, Cox RW & Shaker
R. Event-related fMRI of tasks involving brief
motion. HumBrainMapp7: 106-14 (1999).
332. Bandettini PA, Jesmanowicz A, Wong EC
& Hyde JS. Processing strategies for timecourse data sets in functional MRI of the
human brain. Magn Reson Med 30: 161-73
(1993).
333. Ogawa S, Lee TM, Kay AR & Tank DW. Brain
magnetic resonance imaging with contrast
dependent on blood oxygenation. Proc Natl
AcadSciUSA87: 9868-72 (1990).
343. Johansson RS, Westling G, Backstrom A &
Flanagan JR. Eye-hand coordination in object
manipulation. JNeurosci21: 6917-32 (2001).
344. Porter JR, Wright SM & Reykowski A. A
16-element phased-array head coil. Magn
ResonMed40: 272-9 (1998).
345. Corneil BD, Olivier E & Munoz DP. Visual
responses on neck muscles reveal selective
gating that prevents express saccades.
Neuron42: 831-41 (2004).
r e f e r e n c e li s t | 215
346. Verhagen L, Grol MJ, Dijkerman HC & Toni
I. Studying visually-guided reach-to-grasp
movements in an MR-environment. in
12th Annual meeting of the Organization
for Human Brain Mapping (HBM). 2006.
Florence, Italy.
356. Davare M, Lemon R & Olivier E. Selective
modulation of interactions between ventral
premotor cortex and primary motor cortex
during precision grasping in humans. J
Physiol586: 2735-42 (2008).
347. Lund TE, Norgaard MD, Rostrup E, Rowe JB
& Paulson OB. Motion or activity: their role
in intra- and inter-subject variation in fMRI.
Neuroimage26: 960-4 (2005).
357. Harris JA, Clifford CW & Miniussi C. The
functional effect of transcranial magnetic
stimulation: signal suppression or neural
noise generation? JCognNeurosci20: 73440 (2008).
348. Friston KJ, Williams S, Howard R, Frackowiak
RS & Turner R. Movement-related effects in
fMRI time-series. MagnResonMed35: 34655 (1996).
358. Amassian VE, et al. Suppression of visual
perception by magnetic coil stimulation of
human occipital cortex. Electroencephalogr
ClinNeurophysiol74: 458-62 (1989).
349. Nitsche MA & Paulus W. Excitability changes
induced in the human motor cortex by weak
transcranial direct current stimulation. J
Physiol527 Pt 3: 633-9 (2000).
359. Maccabee PJ, et al. Magnetic coil stimulation
of human visual cortex: studies of perception.
Electroencephalogr Clin Neurophysiol
Suppl43: 111-20 (1991).
350. Siebner HR, et al. Preconditioning of lowfrequency repetitive transcranial magnetic
stimulation with transcranial direct current
stimulation: evidence for homeostatic
plasticity in the human motor cortex. J
Neurosci24: 3379-85 (2004).
360. Pascual-Leone A, Gates JR & Dhuna A.
Induction of speech arrest and counting
errors with rapid-rate transcranial magnetic
stimulation. Neurology41: 697-702 (1991).
351. Barker AT, Jalinous R & Freeston IL. Noninvasive magnetic stimulation of human
motor cortex. Lancet1: 1106-7 (1985).
361. Pascual-Leone A, Valls-Sole J, Wassermann
EM & Hallett M. Responses to rapid-rate
transcranial magnetic stimulation of the
human motor cortex. Brain117 ( Pt 4): 847-58
(1994).
352. Di Lazzaro V, Ziemann U & Lemon RN. State
of the art: Physiology of transcranial motor
cortex stimulation. Brain Stimul 1: 345-62
(2008).
362. Chen R, et al. Depression of motor cortex
excitability by low-frequency transcranial
magnetic stimulation. Neurology 48: 1398403 (1997).
353. Silvanto J, Muggleton NG, Cowey A & Walsh
V. Neural adaptation reveals state-dependent
effects of transcranial magnetic stimulation.
EurJNeurosci25: 1874-81 (2007).
363. Huang YZ, Edwards MJ, Rounis E, Bhatia
KP & Rothwell JC. Theta burst stimulation of
the human motor cortex. Neuron 45: 201-6
(2005).
354. Pascual-Leone A, et al. Study and
modulation of human cortical excitability
with transcranial magnetic stimulation. JClin
Neurophysiol15: 333-43 (1998).
364. Di Lazzaro V, et al. The effects of motor cortex
rTMS on corticospinal descending activity.
ClinNeurophysiol121: 464-73 (2010).
355. Kujirai T, et al. Corticocortical inhibition in
human motor cortex. J Physiol 471: 501-19
(1993).
365. Walsh V & Cowey A. Transcranial magnetic
stimulation and cognitive neuroscience. Nat
RevNeurosci1: 73-9 (2000).
366. Pascual-Leone A, Walsh V & Rothwell
J. Transcranial magnetic stimulation in
cognitive
neuroscience--virtual
lesion,
chronometry, and functional connectivity.
CurrOpinNeurobiol10: 232-7 (2000).
A
216 | a p p e n dix
367. Wilkinson D & Halligan P. The relevance of
behavioural measures for functional-imaging
studies of cognition. Nat Rev Neurosci 5:
67-73 (2004).
368. Komssi S & Kahkonen S. The novelty value of
the combined use of electroencephalography
and transcranial magnetic stimulation for
neuroscience research. Brain Research
Reviews52: 183-92 (2006).
369. Morbidi F, et al. Off-line removal of
TMS-induced
artifacts
on
human
electroencephalography by Kalman filter. J
NeurosciMethods162: 293-302 (2007).
370. Litvak V, et al. Artifact correction and source
analysis of early electroencephalographic
responses evoked by transcranial magnetic
stimulation over primary motor cortex.
Neuroimage37: 56-70 (2007).
377. Thut G & Pascual-Leone A. A review of
combined TMS-EEG studies to characterize
lasting effects of repetitive TMS and assess
their usefulness in cognitive and clinical
neuroscience. Brain Topogr 22: 219-32
(2010).
378. Veniero D, Bortoletto M & Miniussi C. TMSEEG co-registration: on TMS-induced artifact.
ClinNeurophysiol120: 1392-9 (2009).
379. Korhonen RJ, et al. Removal of large
muscle artifacts from transcranial magnetic
stimulation-evoked EEG by independent
component analysis. MedBiolEngComput
(2011).
380. Maki H & Ilmoniemi RJ. Projecting out
muscle artifacts from TMS-evoked EEG.
Neuroimage54: 2706-10 (2011).
371. Ilmoniemi RJ & Kičić D. Methodology for
combined TMS and EEG. Brain Topogr 22:
233-48 (2010).
381. Sommer M, et al. Half sine, monophasic and
biphasic transcranial magnetic stimulation of
the human motor cortex. ClinNeurophysiol
117: 838-44 (2006).
372. Miniussi C & Thut G. Combining TMS
and EEG offers new prospects in cognitive
neuroscience. Brain Topogr 22: 249-56
(2010).
382. Bell AJ & Sejnowski TJ. An informationmaximization approach to blind separation
and blind deconvolution. Neural Comput 7:
1129-59 (1995).
373. Taylor PC, Walsh V & Eimer M. Combining
TMS and EEG to study cognitive function and
cortico-cortico interactions. BehavBrainRes
191: 141-7 (2008).
383. Makeig S, Debener S, Onton J & Delorme A.
Mining event-related brain dynamics. Trends
CognSci8: 204-10 (2004).
374. Virtanen J, Ruohonen J, Naatanen R &
Ilmoniemi RJ. Instrumentation for the
measurement of electric brain responses to
transcranial magnetic stimulation. MedBiol
EngComput37: 322-6 (1999).
375. Ilmoniemi RJ, et al. Neuronal responses
to magnetic stimulation reveal cortical
reactivity and connectivity. Neuroreport 8:
3537-40 (1997).
376. Thut G & Miniussi C. New insights into
rhythmic brain activity from TMS-EEG
studies. TrendsCognSci13: 182-9 (2009).
384. Lee TW, Girolami M & Sejnowski TJ.
Independent component analysis using
an extended infomax algorithm for mixed
subgaussian and supergaussian sources.
NeuralComput11: 417-41 (1999).
385. Helmich RC, Janssen MJ, Oyen WJ, Bloem
BR & Toni I. Pallidal dysfunction drives a
cerebellothalamic circuit into Parkinson
tremor. AnnNeurol69: 269-81 (2011).
386. Bunge SA, Dudukovic NM, Thomason ME,
Vaidya CJ & Gabrieli JD. Immature frontal
lobe contributions to cognitive control in
children: evidence from fMRI. Neuron 33:
301-11 (2002).
387. Friston KJ, et al. The relationship between
global and local changes in PET scans. J
CerebBloodFlowMetab10: 458-66 (1990).
r e f e r e n c e l i s t | 217
388. Fox PT & Raichle ME. Stimulus rate
dependence of regional cerebral blood flow
in human striate cortex, demonstrated
by positron emission tomography. J
Neurophysiol51: 1109-20 (1984).
399. Peeters RR & Van der Linden A. A data
post-processing protocol for dynamic MRI
data to discriminate brain activity from global
physiological effects. Magn Reson Imaging
20: 503-10 (2002).
389. Gavrilescu M, et al. Simulation of the
effects of global normalization procedures
in functional MRI. Neuroimage 17: 532-42
(2002).
400. Andersson JL, Ashburner J & Friston K. A
global estimator unbiased by local changes.
Neuroimage13: 1193-206 (2001).
390. Aguirre GK, Zarahn E & D’Esposito M. The
inferential impact of global signal covariates
in functional neuroimaging analyses.
Neuroimage8: 302-6 (1998).
391. Andersson JL. How to estimate global
activity independent of changes in local
activity. Neuroimage6: 237-44 (1997).
392. Fox MD, Zhang D, Snyder AZ & Raichle ME.
The global signal and observed anticorrelated
resting state brain networks. JNeurophysiol
101: 3270-83 (2009).
401. Helmich RC, de Lange FP, Bloem BR & Toni I.
Cerebral compensation during motor imagery
in Parkinson’s disease. Neuropsychologia
45: 2201-15 (2007).
402. Menenti L, Petersson KM, Scheeringa R &
Hagoort P. When elephants fly: differential
sensitivity of right and left inferior frontal gyri
to discourse and world knowledge. J Cogn
Neurosci21: 2358-68 (2009).
403. Oldfield RC. The assessment and analysis
of handedness: the Edinburgh inventory.
Neuropsychologia9: 97-113 (1971).
393. Laurienti PJ. Deactivations, global signal,
and the default mode of brain function. J
CognNeurosci16: 1481-3 (2004).
404. Parsons
LM.
Imagined
spatial
transformations of one’s hands and feet.
CognPsychol19: 178-241 (1987).
394. Murphy K, Birn RM, Handwerker DA, Jones
TB & Bandettini PA. The impact of global
signal regression on resting state correlations:
are anti-correlated networks introduced?
Neuroimage44: 893-905 (2009).
405. de Lange FP, Hagoort P & Toni I. Neural
topography and content of movement
representations. JCognNeurosci17: 97-112
(2005).
395. Weissenbacher A, et al. Correlations and
anticorrelations in resting-state functional
connectivity MRI: a quantitative comparison
of preprocessing strategies. Neuroimage47:
1408-16 (2009).
396. Desjardins AE, Kiehl KA & Liddle PF. Removal
of confounding effects of global signal in
functional MRI analyses. Neuroimage 13:
751-8 (2001).
397. Macey PM, Macey KE, Kumar R & Harper
RM. A method for removal of global effects
from fMRI time series. Neuroimage22: 3606 (2004).
398. Chang C & Glover GH. Effects of model-based
physiological noise correction on default mode
network anti-correlations and correlations.
Neuroimage47: 1448-59 (2009).
406. Ashburner J & Friston KJ. Unified
segmentation. Neuroimage 26: 839-51
(2005).
407. Friston KJ, Josephs O, Rees G & Turner R.
Nonlinear event-related responses in fMRI.
MagnResonMed39: 41-52 (1998).
408. Petersson KM, Nichols TE, Poline JB
& Holmes AP. Statistical limitations in
functional neuroimaging. I. Non-inferential
methods and statistical models. PhilosTrans
RSocLondBBiolSci354: 1239-60 (1999).
409. Smith SM, et al. Variability in fMRI: a reexamination of inter-session differences.
HumBrainMapp24: 248-57 (2005).
A
218 | a p p e n dix
410. Biswal B, Yetkin FZ, Haughton VM & Hyde
JS. Functional connectivity in the motor
cortex of resting human brain using echoplanar MRI. Magn Reson Med 34: 537-41
(1995).
411. Damoiseaux JS, et al. Consistent restingstate networks across healthy subjects. Proc
NatlAcadSciUSA103: 13848-53 (2006).
412. Fox MD & Raichle ME. Spontaneous
fluctuations in brain activity observed with
functional magnetic resonance imaging. Nat
RevNeurosci8: 700-11 (2007).
413. Salvador R, et al. Neurophysiological
architecture of functional magnetic resonance
images of human brain. Cereb Cortex 15:
1332-42 (2005).
414. Willems RM, Hagoort P & Casasanto D.
Body-specific representations of action verbs:
neural evidence from right- and left-handers.
PsycholSci21: 67-74 (2010).
415. Volman I, Toni I, Verhagen L & Roelofs
K. Endogenous Testosterone Modulates
Prefrontal-Amygdala Connectivity during
Social Emotional Behavior. Cereb Cortex
(2011).
416. Takashima A, et al. Shift from hippocampal
to neocortical centered retrieval network
with consolidation. JNeurosci29: 10087-93
(2009).
417. Jeannerod M & Rossetti Y. Visuomotor
coordination as a dissociable visual function:
experimental and clinical evidence. Baillieres
ClinNeurol2: 439-60 (1993).
418. Knill DC. Discrimination of planar surface
slant from texture: human and ideal observers
compared. VisionRes38: 1683-711 (1998).
419. McIntyre J, Stratta F & Lacquaniti F. Shortterm memory for reaching to visual targets:
psychophysical evidence for body-centered
reference frames. J Neurosci 18: 8423-35
(1998).
420. Toni I, Schluter ND, Josephs O, Friston K &
Passingham RE. Signal-, set- and movementrelated activity in the human brain: an eventrelated fMRI study. Cereb Cortex 9: 35-49
(1999).
421. Toni I, Thoenissen D & Zilles K. Movement
preparation
and
motor
intention.
Neuroimage14: S110-7 (2001).
422. Shulman GL, et al. Reactivation of networks
involved in preparatory states. CerebCortex
12: 590-600 (2002).
423. Thoenissen D, Zilles K & Toni I. Differential
involvement of parietal and precentral
regions in movement preparation and motor
intention. JNeurosci22: 9024-34 (2002).
424. Hu Y, Eagleson R & Goodale MA. The effects
of delay on the kinematics of grasping. Exp
BrainRes126: 109-16 (1999).
425. Mars RB, Piekema C, Coles MG, Hulstijn
W & Toni I. On the programming and
reprogramming of actions. CerebCortex17:
2972-9 (2007).
426. Benjamini Y & Hochberg Y. Controlling the
false discovery rate - a practical and powerful
approach to multiple testing. Journal of
the Royal Statistical Society Series
B-Methodological57: 289-300 (1995).
427. Amedi A, Malach R, Hendler T, Peled S
& Zohary E. Visuo-haptic object-related
activation in the ventral visual pathway. Nat
Neurosci4: 324-30 (2001).
428. Grill-Spector K, et al. Differential processing
of objects under various viewing conditions in
the human lateral occipital complex. Neuron
24: 187-203 (1999).
429. Moore C & Engel SA. Neural response to
perception of volume in the lateral occipital
complex. Neuron29: 277-86 (2001).
430. Maldjian JA, Laurienti PJ, Kraft RA &
Burdette JH. An automated method for
neuroanatomic
and
cytoarchitectonic
atlas-based interrogation of fMRI data sets.
Neuroimage19: 1233-9 (2003).
431. Henson RN, Price CJ, Rugg MD, Turner R &
Friston KJ. Detecting latency differences in
event-related BOLD responses: application
to words versus nonwords and initial versus
repeated face presentations. Neuroimage
15: 83-97 (2002).
r e f e r e n c e lis t | 219
432. Friston KJ, et al. Psychophysiological and
modulatory interactions in neuroimaging.
Neuroimage6: 218-29 (1997).
443. Friston KJ, et al. Event-related fMRI:
characterizing
differential
responses.
Neuroimage7: 30-40 (1998).
433. Duvernoy HM, Cabanis EA & Vannson JL
(1991) The human brain: surface, threedimensional sectional anatomy and MRI.
Vienna: Springer.
444. Cohen NR, Cross ES, Tunik E, Grafton ST
& Culham JC. Ventral and dorsal stream
contributions to the online control of
immediate and delayed grasping: a TMS
approach. Neuropsychologia 47: 1553-62
(2009).
434. Eickhoff SB, et al. A new SPM toolbox for
combining probabilistic cytoarchitectonic
maps and functional imaging data.
Neuroimage25: 1325-35 (2005).
435. Amunts K, Malikovic A, Mohlberg H,
Schormann T & Zilles K. Brodmann’s areas 17
and 18 brought into stereotaxic space-where
and how variable? Neuroimage 11: 66-84
(2000).
436. Anzai A, Ohzawa I & Freeman RD. Neural
mechanisms for processing binocular
information II. Complex cells. JNeurophysiol
82: 909-24 (1999).
437. Anzai A, Ohzawa I & Freeman RD. Neural
mechanisms for processing binocular
information I. Simple cells. J Neurophysiol
82: 891-908 (1999).
438. Bradshaw MF & Elliott KM. The role of
binocular information in the ‘on-line’ control
of prehension. SpatVis16: 295-309 (2003).
439. van Bergen E, van Swieten LM, Williams JH
& Mon-Williams M. The effect of orientation
on prehension movement time. Exp Brain
Res178: 180-93 (2007).
440. Fattori P, Pitzalis S & Galletti C. The cortical
visual area V6 in macaque and human brains.
JPhysiolParis103: 88-97 (2009).
441. Fattori P, Kutz DF, Breveglieri R, Marzocchi
N & Galletti C. Spatial tuning of reaching
activity in the medial parieto-occipital
cortex (area V6A) of macaque monkey. EurJ
Neurosci22: 956-72 (2005).
442. Pesaran B, Nelson MJ & Andersen RA.
Dorsal premotor neurons encode the relative
position of the hand, eye, and goal during
reach planning. Neuron51: 125-34 (2006).
445. Glover S. Optic ataxia as a deficit specific
to the on-line control of actions. Neurosci
BiobehavRev27: 447-56 (2003).
446. Croxson PL, et al. Quantitative investigation
of connections of the prefrontal cortex in
the human and macaque using probabilistic
diffusion tractography. JNeurosci25: 885466 (2005).
447. Petrides M & Pandya DN. Comparative
cytoarchitectonic analysis of the human and
the macaque ventrolateral prefrontal cortex
and corticocortical connection patterns in the
monkey. EurJNeurosci16: 291-310 (2002).
448. Ungerleider LG, Gaffan D & Pelak VS.
Projections from inferior temporal cortex to
prefrontal cortex via the uncinate fascicle in
rhesus monkeys. Exp Brain Res 76: 473-84
(1989).
449. Rossini PM, et al. Non-invasive electrical and
magnetic stimulation of the brain, spinal cord
and roots: basic principles and procedures
for routine clinical application. Report of an
IFCN committee. Electroencephalogr Clin
Neurophysiol91: 79-92 (1994).
450. Jasper HH. The ten-twenty electrode
system of the International Federation.
ElectroencephalogrClinNeurophysiol10:
371-375 (1958).
451. Chatrian GE, Lettich E & Nelson PL. Ten
percent electrode system for topographic
studies of spontaneous and evoked EEG
activity. AmJEEGTechnol25: 83-92 (1985).
452. Schot WD, Brenner E & Smeets JB. Robust
movement segmentation by combining
multiple sources of information. J Neurosci
Methods187: 147-55 (2010).
A
220 | appendix
453. Oostenveld R, Fries P, Maris E & Schoffelen
JM. FieldTrip: Open source software for
advanced analysis of MEG, EEG, and invasive
electrophysiological data. Comput Intell
Neurosci2011: 156869 (2011).
463. Ehrsson HH, Fagergren E & Forssberg
H. Differential fronto-parietal activation
depending on force used in a precision grip
task: an fMRI study. J Neurophysiol 85:
2613-23 (2001).
454. Maris E & Oostenveld R. Nonparametric
statistical testing of EEG- and MEG-data. J
NeurosciMethods164: 177-90 (2007).
464. Davare M, Andres M, Clerget E, Thonnard
JL & Olivier E. Temporal dissociation between
hand shaping and grip force scaling in the
anterior intraparietal area. J Neurosci 27:
3974-80 (2007).
455. Brown
P.
Abnormal
oscillatory
synchronisation in the motor system leads to
impaired movement. Curr Opin Neurobiol
17: 656-64 (2007).
456. Taubert M, et al. Inhibition of the anterior
intraparietal area and the dorsal premotor
cortex interfere with arbitrary visuo-motor
mapping. Clin Neurophysiol 121: 408-13
(2010).
457. Hoshi E & Tanji J. Differential involvement of
neurons in the dorsal and ventral premotor
cortex during processing of visual signals for
action planning. J Neurophysiol 95: 3596616 (2006).
458. Reichenbach A, Bresciani JP, Peer A,
Bulthoff HH & Thielscher A. Contributions
of the PPC to online control of visually guided
reaching movements assessed with fMRIguided TMS. CerebCortex(2010).
459. Gharbawie OA, Stepniewska I & Kaas JH.
Cortical Connections of Functional Zones in
Posterior Parietal Cortex and Frontal Cortex
Motor Regions in New World Monkeys. Cereb
Cortex(2011).
460. Desmurget M, Grea H & Prablanc C. Final
posture of the upper limb depends on the
initial position of the hand during prehension
movements. Exp Brain Res 119: 511-6
(1998).
461. Vaughan J, Rosenbaum DA & Meulenbroek
RG. Planning reaching and grasping
movements: the problem of obstacle
avoidance. MotorControl5: 116-35 (2001).
462. Dafotakis M, Sparing R, Eickhoff SB, Fink
GR & Nowak DA. On the role of the ventral
premotor cortex and anterior intraparietal
area for predictive and reactive scaling of grip
force. BrainRes1228: 73-80 (2008).
465. Knill DC & Pouget A. The Bayesian brain:
the role of uncertainty in neural coding and
computation. Trends Neurosci 27: 712-9
(2004).
466. Wolpert DM & Ghahramani Z. Computational
principles of movement neuroscience. Nat
Neurosci3 Suppl: 1212-7 (2000).
467. Brouwer AM, Franz VH & Gegenfurtner KR.
Differences in fixations between grasping and
viewing objects. JVis9: 18 1-24 (2009).
468. de Grave DD, Hesse C, Brouwer AM & Franz
VH. Fixation locations when grasping partly
occluded objects. JVis8: 5 1-11 (2008).
469. Grea H, et al. A lesion of the posterior parietal
cortex disrupts on-line adjustments during
aiming movements. Neuropsychologia 40:
2471-80 (2002).
470. Milner AD, Paulignan Y, Dijkerman HC,
Michel F & Jeannerod M. A paradoxical
improvement of misreaching in optic ataxia:
new evidence for two separate neural systems
for visual localization. Proc Biol Sci 266:
2225-9 (1999).
471. Glover S, Miall RC & Rushworth MF.
Parietal rTMS disrupts the initiation but not
the execution of on-line adjustments to a
perturbation of object size. JCognNeurosci
17: 124-36 (2005).
472. Pisella L, Binkofski F, Lasek K, Toni I &
Rossetti Y. No double-dissociation between
optic ataxia and visual agnosia: multiple
sub-streams for multiple visuo-manual
integrations. Neuropsychologia 44: 273448 (2006).
reference list | 221
473. Macfarlane NB & Graziano MS. Diversity of
grip in Macaca mulatta. ExpBrainRes197:
255-68 (2009).
476. Iacoboni M, et al. Grasping the intentions of
others with one’s own mirror neuron system.
PLoSBiol3: e79 (2005).
474. Jastorff J, Begliomini C, Fabbri-Destro M,
Rizzolatti G & Orban GA. Coding observed
motor acts: different organizational principles
in the parietal and premotor cortex of
humans. JNeurophysiol104: 128-40 (2010).
477. Rizzolatti G & Craighero L. The mirrorneuron system. Annu Rev Neurosci 27:
169-92 (2004).
475. Raos V, Umilta MA, Murata A, Fogassi L &
Gallese V. Functional properties of graspingrelated neurons in the ventral premotor area
F5 of the macaque monkey. J Neurophysiol
95: 709-29 (2006).
A
222 | appendix
donders series | 223
DondersGraduateSchool
forCognitiveNeuroscienceSeries
1.
van Aalderen-Smeets, S.I. (2007). Neural
dynamics of visual selection. Maastricht
University, Maastricht, the Netherlands.
2.
Schoffelen,
J.M.
(2007).
Neuronal
communication through coherence in the
human motor system. Radboud University
Nijmegen, Nijmegen, the Netherlands.
3.
4.
5.
6.
7.
8.
de Lange, F.P. (2008). Neural mechanisms
of motor imagery. Radboud University
Nijmegen, Nijmegen, the Netherlands.
Grol, M.J. (2008). Parieto-frontal circuitry
in visuomotor control. Utrecht University,
Utrecht, the Netherlands.
Bauer, M. (2008). Functional roles of rhythmic
neuronal activity in the human visual and
somatosensory system. Radboud University
Nijmegen, Nijmegen, the Netherlands.
Mazaheri, A. (2008). The Influence of
Ongoing Oscillatory Brain Activity on
Evoked Responses and Behaviour. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
Hooijmans, C.R. (2008). Impact of nutritional
lipids and vascular factors in Alzheimer’s
Disease. Radboud University Nijmegen,
Nijmegen, the Netherlands.
Gaszner, B. (2008). Plastic responses to
stress by the rodent urocortinergic EdingerWestphal nucleus. Radboud University
Nijmegen, Nijmegen, the Netherlands.
9.
Willems, R.M. (2009). Neural reflections of
meaning in gesture, language and action.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
10. van Pelt, S. (2009). Dynamic neural
representations of human visuomotor space.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
11. Lommertzen, J. (2009). Visuomotor coupling
at different levels of complexity. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
12. Poljac, E. (2009). Dynamics of cognitive
control in task switching: Looking beyond the
switch cost. Radboud University Nijmegen,
Nijmegen, the Netherlands.
13. Poser, B.A. (2009). Techniques for BOLD
and blood volume weighted fMRI. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
14. Baggio, G. (2009). Semantics and the
electrophysiology of meaning. Tense,
aspect, event structure. Radboud University
Nijmegen, Nijmegen, the Netherlands.
15. van Wingen, G.A. (2009). Biological
determinants of amygdala functioning.
Radboud University Nijmegen Medical
Centre, Nijmegen, the Netherlands.
A
224 | appendix
16. Bakker, M. (2009). Supraspinal control
of walking: lessons from motor imagery.
Radboud University Nijmegen Medical
Centre, Nijmegen, the Netherlands.
17. Aarts, E. (2009). Resisting temptation:
the role of the anterior cingulate cortex
in adjusting cognitive control. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
26. Horlings,
C.G.C.
(2010).
A
Weak
balance; balance and falls in patients
with neuromuscular disorders. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
27. Snaphaan, L.J.A.E. (2010). Epidemiology
of post-stroke behavioural consequences.
Radboud University Nijmegen Medical
Centre, Nijmegen, the Netherlands.
18. Prinz, S. (2009). Waterbath stunning of
chickens – Effects of electrical parameters
on the electroencephalogram and physical
reflexes of broilers. Radboud University
Nijmegen, Nijmegen, the Netherlands.
28. Dado – van Beek, H.E.A. (2010). The
regulation of cerebral perfusion in patients
with
Alzheimer’s
disease.
Radboud
University Nijmegen Medical Centre,
Nijmegen, the Netherlands.
19. Knippenberg, J.M.J. (2009). The N150 of
the Auditory Evoked Potential from the
rat amygdala: In search for its functional
significance. Radboud University Nijmegen,
Nijmegen, the Netherlands.
29. Derks, N.M. (2010). The role of the nonpreganglionic Edinger-Westphal nucleus in
sex-dependent stress adaptation in rodents.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
20. Dumont,
G.J.H.
(2009).
Cognitive
and
physiological
effects
of
3 , 4-methylenediox ymethamphetamine
(MDMA or ’ecstasy’) in combination with
alcohol or cannabis in humans. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
30. Wyczesany, M. (2010). Covariation of mood
and brain activity. Integration of subjective
self-report data with quantitative EEG
measures. Radboud University Nijmegen,
Nijmegen, the Netherlands.
21. Pijnacker, J. (2010). Defeasible inference in
autism: a behavioral and electrophysiogical
approach. Radboud University Nijmegen,
Nijmegen, the Netherlands.
22. de Vrijer, M. (2010). Multisensory integration
in spatial orientation. Radboud University
Nijmegen, Nijmegen, the Netherlands.
23. Vergeer, M. (2010). Perceptual visibility
and appearance: Effects of color and form.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
24. Levy, J. (2010). In Cerebro Unveiling
Unconscious Mechanisms during Reading.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
25. Treder, M. S. (2010). Symmetry in (inter)
action. Radboud University Nijmegen,
Nijmegen, the Netherlands.
31. Beurze, S.M. (2010). Cortical mechanisms
for reach planning. Radboud University
Nijmegen, Nijmegen, the Netherlands.
32. van Dijk, J.P. (2010). On the Number of
Motor Units. Radboud University Nijmegen,
Nijmegen, the Netherlands.
33. Lapatki, B.G. (2010). The Facial Musculature
– Characterization at a Motor Unit Level.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
34. Kok, P. (2010). Word Order and Verb Inflection
in Agrammatic Sentence Production.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
35. van Elk, M. (2010). Action semantics:
Functional and neural dynamics. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
36. Majdandzic, J. (2010). Cerebral mechanisms
of processing action goals in self and others.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
donders series | 225
37. Snijders, T.M. (2010). More than words –
neural and genetic dynamics of syntactic
unification. Radboud University Nijmegen,
Nijmegen, the Netherlands.
48. Lamers, M.J.M. (2011). Levels of selective
attention in action planning. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
38. Grootens, K.P. (2010). Cognitive dysfunction
and effects of antipsychotics in schizophrenia
and borderline personality disorder. Radboud
University Nijmegen Medical Centre,
Nijmegen, the Netherlands.
49. van der Werf, J. (2011). Cortical oscillatory
activity in human visuomotor integration.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
39. Nieuwenhuis, I.L.C. (2010). Memory
consolidation: A process of integration –
Converging evidence from MEG, fMRI and
behavior. Radboud University Nijmegen
Medical Centre, Nijmegen, the Netherlands.
40. Menenti, L.M.E. (2010). The right language:
differential hemispheric contributions to
language production and comprehension
in context. Radboud University Nijmegen,
Nijmegen, the Netherlands.
41. van Dijk, H.P. (2010). The state of the brain,
how alpha oscillations shape behaviour and
event related responses.Radboud University
Nijmegen, Nijmegen, the Netherlands.
42. Meulenbroek, O.V. (2010). Neural correlates
of episodic memory in healthy aging and
Alzheimer’s disease. Radboud University
Nijmegen, Nijmegen, the Netherlands.
43. Oude Nijhuis, L.B. (2010). Modulation
of human balance reactions. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
44. Qin, S. (2010). Adaptive memory: imaging
medial temporal and prefrontal memory
systems. Radboud University Nijmegen,
Nijmegen, the Netherlands.
45. Timmer, N.M. (2011). The interaction of
heparan sulfate proteoglycans with the
amyloid β protein. Radboud University
Nijmegen, Nijmegen, the Netherlands.
50. Scheeringa, R. (2011). On the relation
between oscillatory EEG activity and the
BOLD signal. Radboud University Nijmegen,
Nijmegen, the Netherlands.
51. Bögels, S. (2011). The role of prosody in
language comprehension: when prosodic
breaks and pitch accents come into play.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
52. Ossewaarde, L. (2011). The mood cycle:
hormonal influences on the female brain.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
53. Kuribara, M. (2011). Environment-induced
activation and growth of pituitary
melanotrope cells of Xenopus laevis.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
54. Helmich,
R.C.G.
(2011).
Cerebral
reorganization in Parkinson’s disease.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
55. Boelen, D. (2011). Order out of chaos?
Assessment and treatment of executive
disorders in brain-injured patients. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
56. Koopmans, P.J. (2011). fMRI of cortical
layers. Radboud University Nijmegen,
Nijmegen, the Netherlands.
46. Crajé, C. (2011). (A)typical motor planning
and motor imagery. Radboud University
Nijmegen, Nijmegen, the Netherlands.
57. van der Linden, M.H. (2011). Experiencebased cortical plasticity in object category
representation.
Radboud
University
Nijmegen, Nijmegen, the Netherlands.
47. van Grootel, T.J. (2011). On the role of eye
and head position in spatial localisation
behaviour. Radboud University Nijmegen,
Nijmegen, the Netherlands.
58. Kleine, B.U. (2011). Motor unit discharges Physiological and diagnostic studies in ALS.
Radboud University Nijmegen Medical
Centre, Nijmegen, the Netherlands.
A
226 | appendix
59. Paulus, M. (2011). Development of action
perception: Neurocognitive mechanisms
underlying children’s processing of others’
actions. Radboud University Nijmegen,
Nijmegen, the Netherlands.
68. Xu, L. (2011). The non-preganglionic EdingerWestphal nucleus: an integration center
for energy balance and stress adaptation.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
60. Tieleman, A.A. (2011). Myotonic dystrophy
type 2. A newly diagnosed disease in the
Netherlands. Radboud University Nijmegen
Medical Centre, Nijmegen, the Netherlands.
69. Schellekens,
A.F.A.
(2011).
Geneenvironment interaction and intermediate
phenotypes in alcohol dependence. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
61. van Leeuwen, T.M. (2011). ‘How one can
see what is not there’: Neural mechanisms
of grapheme-colour synaesthesia. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
62. van Tilborg, I.A.D.A. (2011). Procedural
learning in cognitively impaired patients and
its application in clinical practice. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
63. Bruinsma, I.B. (2011). Amyloidogenic proteins
in Alzheimer’s disease and Parkinson’s
disease: interaction with chaperones
and inflammation. Radboud University
Nijmegen, Nijmegen, the Netherlands.
64. Voermans, N. (2011). Neuromuscular
features of Ehlers-Danlos syndrome and
Marfan syndrome; expanding the phenotype
of inherited connective tissue disorders and
investigating the role of the extracellular
matrix in muscle. Radboud University
Nijmegen Medical Centre, Nijmegen, the
Netherlands.
65. Reelick, M. (2011). One step at a time.
Disentangling the complexity of preventing
falls in frail older persons. Radboud
University Nijmegen Medical Centre,
Nijmegen, the Netherlands.
66. Buur, P.F. (2011). Imaging in motion.
Applications of multi-echo fMRI. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
67. Schaefer, R.S. (2011). Measuring the
mind’s ear: EEG of music imagery. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
70. van Marle, H.J.F. (2011). The amygdala on
alert: A neuroimaging investigation into
amygdala function during acute stress and its
aftermath. Radboud University Nijmegen,
Nijmegen, the Netherlands.
71. de Laat, K.F. (2011). Motor performance in
individuals with cerebral small vessel disease:
an MRI study. Radboud University Nijmegen
Medical Centre, Nijmegen, the Netherlands.
72. Mädebach, A. (2011). Lexical access in
speaking: Studies on lexical selection and
cascading activation. Radboud University
Nijmegen, Nijmegen, the Netherlands.
73. Poelmans, G.J.V. (2011). Genes and protein
networks for neurodevelopmental disorders.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
74. van Norden, A.G.W. (2011). Cognitive
function in elderly individuals with cerebral
small vessel disease. An MRI study. Radboud
University Nijmegen Medical Centre,
Nijmegen, the Netherlands.
75. Jansen, E.J.R. (2011). New insights into
V-ATPase functioning: the role of its accessory
subunit Ac45 and a novel brain-specific Ac45
paralog. Radboud University Nijmegen,
Nijmegen, the Netherlands.
76. Haaxma, C.A. (2011). New perspectives
on preclinical and early stage Parkinson’s
disease. Radboud University Nijmegen
Medical Centre, Nijmegen, the Netherlands.
77. Haegens, S. (2012). On the functional
role of oscillatory neuronal activity in the
somatosensory system. Radboud University
Nijmegen, Nijmegen, the Netherlands.
donders series | 227
78. van
Barneveld,
D.C.P.B.M.
(2012).
Integration of exteroceptive and interoceptive
cues in spatial localization. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
83. Vermeer, S. (2012). Clinical and genetic
characterisation of Autosomal Recessive
Cerebellar Ataxias. Radboud University
Nijmegen Medical Centre, Nijmegen, the
Netherlands.
79. Spies, P.E. (2012). The reflection of Alzheimer
disease in CSF. Radboud University
Nijmegen Medical Centre, Nijmegen, the
Netherlands.
84. Vrins, S. (2012). Shaping object boundaries:
contextual effects in infants and adults.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
80. Helle, M. (2012). Artery-specific perfusion
measurements in the cerebral vasculature
by magnetic resonance imaging. Radboud
University Nijmegen, Nijmegen, the
Netherlands.
85. Weber, K.M. (2012). The language learning
brain: Evidence from second language and
bilingual studies of syntactic processing.
Radboud University Nijmegen, Nijmegen,
the Netherlands.
81. Egetemeir, J. (2012). Neural correlates of
real-life joint action. Radboud University
Nijmegen, Nijmegen, the Netherlands.
86. Verhagen, L. (2012). How to grasp a ripe
tomato. Utrecht University, Utrecht, the
Netherlands.
82. Janssen, L. (2012). Planning and execution
of (bi)manual grasping. Radboud University
Nijmegen, Nijmegen, the Netherlands.
A
228 | appendix
thank you | 229
Acknowledgements
When I was younger I thought that scientists were supposed to have grey hear,
wear white lab coats, and scribble with chalk on black boards. I thought that genius
insight came spontaneous to individual thinkers who shouted “Eureka” when they
understood.
I am glad that this is not the case, although I do like chalk and black boards. I
am especially happy about the fact that scientists are not alone as Archimedes was
in his bath. Science is hard work, but luckily it is fun work. Not only because of the
challenges, the discoveries, or the insight, but perhaps mainly because of the people
with whom you share those with.
Ivan, you are my scientific father, my mentor. And as a good student I will adopt
your ways to express feelings and thoughts: in classic tongue. I have spent a lot of
time on the plains of Asphodel, wondering how many angles can dance on the head
of a pin, but you have always inspired me to aim for mount Olympus. I have stood
besides Tantalus grasping for tomatoes and have visited Kafka’s Castle many times.
But you taught me to continue my quest like Ahab did, and to never make a Faustian
deal. You are like Daedalus, providing both the challenge of the labyrinth and the
freedom of flight. I am not as heroic as Theseus, but I have taken your wings. I have
flown high in the sky, and close to the see, but unlike Icarus I am still flying. Now it’s
time to land (and continue by bike, as Dutchmen do).
Chris, you are always extremely kind and knowledgeable. You have shared
with me your tremendous theoretical comprehension of the subject. Sometimes it
almost felt like I was continuing where your PhD project left off; I hope I have not let
you down. It cannot have been easy to supervise a student in another city who dives
from one methodology into the other, but you have never complained. Instead, you
have always been supportive and I am grateful for your guidance and supervision.
Albert, I have arrived in your flock too late to benefit from you scientific
supervision, but I am thankful that you have helped me through the Kafkian Castle
that constitutes the final phase of a PhD project.
Peter, I cannot exactly remember how we first became friends, all I know is
that suddenly we were standing on stage organizing an in promptu dance contest.
We are great friends and there is no way it could be otherwise, because what other
roommate would also like to install an Orval tap in the office?
Arjen, although in hairdo and physical fitness you are my opposite, we have
shared a connection from the day you joined the Intention and Action team. I am
impressed by your boldness and talent, both in science and on the racing track. We
are the proof that there is no such thing as too much coffee, as it brings me a very
inspiring and fruitful collaboration (is that figure ready yet?).
A
230 | appendix
My time at the Donders has been decisively shaped by the early members of
the Intention and Action group. Meike, I have learned more from you than you will
ever accept to believe. Thank you. Jasminka, Floris, and Rogier you have created
an inspirational environment that further ignited my interest in the brain. Rick, it
was a privilege and a great pleasure to have spent both our internships and our PhDs
together at the Donders.
The current I&A group continues to provide a stimulating and challenging
environment. The range of topics and methods we discuss in depth amazes me. I
would especially like to thank Marius, Miriam, Sasha, Frank, Florian, Idil, Mark and
Iris for your help and exciting discussions.
I have spent less time in the Utrecht lab, but I have always felt very welcome.
Thank you, especially Helen and Alyanne, for the fruitful and fun conversations we
have had. I am most grateful to Marjolein. You have been my roommate during my
Utrecht days, but also an exceptional friend and a talented colleague. I will not forget
our many passionate discussions over so many ristretto’s and cafè latte’s.
Also in Nijmegen I have been fortunate to collaborate intensively with a talented
colleague. Anny, our cooperation started partly as a miraculous twist of fate, but
developed very fruitfully. You are a brilliant researcher and I hope to continue
working with you in your undoubtedly bright future. Ruud, Ellen and recently Nina,
during your internship you have (had) to endure me as a co-supervisor; I’m glad you
survived. Pieter, Karin, Ole and David, thank you for our collaborations, and Harold,
Marco, Jody, Jacinta, Nici, Jeroen, Willemijn, Patrizia, and Karl-Magnus for often
entertaining and always interesting discussions at conferences and in Nijmegen.
Dear subjects, thank you for lying or sitting still for hours, grasping the same
object hundreds of times, for being scanned, scrubbed, measured, covered in tape,
and zapped. IngeL, Marius and many others, thank you for all your help with data
acquisition (and for rotating the object thousands of times).
The Donders is blessed with the best technical assistance any researcher could
ever whish for. Erik, Marek, Paul, Sander, Bram, and recently Uriël, and from
the other side of the campus, Pascal, Jos and Norbert, you are the indispensible.
Subjects can be replaced by pumpkins, supervisors should keep a healthy distance
from the experimental setup, but without you this thesis would not have been. I
really enjoyed working together with you guys, I cannot thank you enough.
Moniek, Dick, Gijs, and recently Til and Tom, you share with me a strange
passion to hit subjects on the head with a magnetic pulse and see what happens.
I am proud of the Donders Laboratory for Non-Invasive Brain Stimulation we have
build together, and grateful for everything I have learned from you.
The extremely supportive climate at the Donders is completed by the amazingly
flexible work of Tildie, Sandra, Nicole, and Arthur.
t hank you | 231
I have worked in an excellent scientific environment, but luckily there is more to
life at the Donders, making it a terrific place to waste a PhD. I remember discussions,
dialogues, arguments, drinks, diners, jokes, and parties (although sometimes I
wished to forget the day after). Pieter and Jenny, you have been very fun and
gezellige officemates in the many years we have occupied room 1.24 together. Thank
you, Hanneke for being a great friend and insightful researcher, Laura for your big
heart and many talents, René for never believing what I say, Erno, Atsuko, Ingrid,
Benedikt, Markus, Jan-Mathijs and Robert for your knowledge and opinions.
Lindsey, Katrien, Ali, Eelke, Esther, Olga, Guido, Roel, Marieke, Martine, Marlieke,
Tessa, Joost, Sean, Stephen, Marcel, the many people who I have already named
above, and everybody at the FAD, for all the great times and beers we have shared.
Heren van Ad Absurdum, Joost, Jeffrey, Ivar, Philip, Maurits en Peter, bedankt
dat jullie mij zo nu en dan goed laten genieten van het leven buiten de wetenschap.
Laten we dat vaker doen, ik ben bang dat ik anders het contact met de normale
wereld verlies. Marjolein, Judith, bedankt dat jullie zeker het begin van mijn AiO
periode zo’n mooie tijd hebben gemaakt en dat we nog steeds goede vrienden zijn.
Michel, Nienke, Hilco, Daniël, Rolf en alle andere leden van de Golden Snikkel, ik
geniet graag met jullie van het Bourgondische leven.
Herco, ik ken je al lang en we hebben veel gedeeld, tijdens de studie, in het
verenigingsleven, persoonlijk, en op vakantie. Dos cervezas, of gewoon een dubbele
bakken borrel? Cyrille en Wieteke, bedankt voor jullie geweldige vriendschap die
landsgrenzen en carrières overstijgt.
Lieve Inez, wat ongelofelijk fijn dat jij mijn familie bent komen versterken! Kom
je mee naar buiten vogels kijken en bioloogje spelen? Lieve Ewold en Kiki, jullie zijn
altijd warm en gastvrij, en ik hoop nog veel tijd door te brengen met jullie gezinnetje.
Oma, Opa, en Ome Arie, bedankt dat jullie mijn jeugd zo zorgeloos en liefdevol
hebben gemaakt. Anneke en Arie, jullie hebben mij altijd overal in gesteund, zonder
mij te sturen. Alhoewel, soms verdenk ik jou ervan, Arie, dat je van vroegs af aan een
geheim masterplan hebt gehad om je eigen natuurkundige en bioloog te kweken.
Anneke, het is niet makkelijk om drie wetenschappers (in spe) in huis te hebben,
maar jouw liefde verwarmt ons allen en brengt ons bij elkaar.
Lieve Inge, dankzij jou heb ik er een boel nieuwe vrienden bij, en met AnneMarie, Rob, Birgit, Huub, opa, en oma, ook een hele lieve familie. Sta me nog
een laatste klassieke vergelijking toe: net zoals Dante in la Divina Commedia heb
ik de afgelopen jaren zowel inferno als purgatorio gezien, maar jij hebt me altijd
opgewacht, om mij rond te leiden in paradiso. Ik kan niet wachten om samen onze
toekomst te ontdekken. Ik heb je lief en zal dat altijd blijven doen.
A
232 | appendix
Publicationlist
Verhagen L, Dijkerman HC, Medendorp WP, Toni I (2012). Cortical dynamics of
sensorimotor integration during grasp planning. J Neurosci, in press
Volman I, Roelofs K, Koch S, VerhagenL, Toni I (2011). Anterior prefrontal cortex
inhibition impairs control over social emotional actions. Curr Biol 21(20):1766-70
Volman I, Toni I, VerhagenL, Roelofs K (2011). Endogenous testosterone modulates
prefrontal-amygdala connectivity during social emotional behavior. Cereb
Cortex 21(10):2282-90
Lachlan RF, Verhagen L, Peters S, ten Cate C (2010). Are there species-universal
categories in bird song phonology and syntax? A comparative study of
chaffinches (Fringilla coelebs), zebra finches (Taenopygia guttata), and swamp
sparrows (Melospiza Georgiana). J Comp Psychol 124(1):92-108
Kammers MP, Verhagen L, Dijkerman HC, Hogendoorn H, De Vignemont F,
Schutter DJ (2009). Is this hand for real? Attenuation of the rubber hand illusion
by transcranial magnetic stimulation over the inferior parietal lobule. J Cogn
Neurosci 21(7):1311-20
Kammers MP, de Vignemont F, VerhagenL, Dijkerman HC (2009). The rubber hand
illusion in action. Neuropsychologia 47(1):204-11
Verhagen L, Dijkerman HC, Grol MJ, Toni I (2008). Perceptuo-motor interactions
during prehension movements. J Neurosci 28(18):4726-35
Grol MJ, Majdandzić J, Stephan KE, Verhagen L, Dijkerman HC, Bekkering H,
Verstraten FA, Toni I (2007). Parieto-frontal connectivity during visually guided
grasping. J Neurosci 27(44):11877-87
Majdandzic J, Grol MJ, van Schie HT, VerhagenL, Toni I, Bekkering H (2007). The
role of immediate and final goals in action planning: an fMRI study. Neuroimage
37(2):589-98
Publicationsinpreparation
VerhagenL, Dijkerman HC, Toni I. Grasping the functional integration of dorsomedial
and dorsolateral parieto-frontal circuits. In preparation for submission
VerhagenL, Dijkerman HC, Helmich RC, Menenti L, Norris DG, Toni I. A simple and
effective method to remove global nuisance variance in fMRI. In preparation for
submission
Verhagen L, Toni I, Dijkerman HC. A paradoxical decrease in grip aperture with
increasing movement difficulty when grasping objects slanted in depth. In
preparation
about the author | 233
Curriculumvitae
Lennart Verhagen was born on the 5 th of October, 1982 in Leiden, the Netherlands.
After he obtained his high school diploma at the Stedelijk Gymnasium Leiden in 2000,
he studied biology at Utrecht University with a focus on cell biology, animal behaviour,
and neurobiology. In the context of the latter he wrote his bachelor thesis entitled
“Top-down influences on bottom-up processing of biological motion”. In 2003 he
enrolled in the master program Neuroscience and Cognition. He did his first research
internschip at the Donders Centre for Cognitive Neuroimaging, Radboud University
Nijmegen, where he studied grasping movements using fMRI working together
with dr. HC Dijkerman and dr. I Toni. His second internship addressed the structural
organization and cultural evolution of zebra finch song, in collaboration with dr. RF
Lachlan and prof. dr. C ten Cate at the Institute of Biology, Leiden University. After
he obtained his master’s degree in 2006, he returned to Nijmegen to start a PhD
project under the supervision of dr. I Toni and dr. HC Dijkerman; a joint collaboration
between the Donders Centre for Cognitive Neuroimaging and the department of
Experimental Psychology, Utrecht University. He investigated how perceptual and
motoric processes are integrated in the brain during grasping movements, using
functional magnetic resonance imaging, electroencephalography, and transcranial
magnetic stimulation. The results of that work are described in this thesis. He is
currently living in Nijmegen together with Inge Volman, and works as a post-doctoral
researcher studying how humans plan and comprehend communicative actions.
A
The research described in this thesis was conducted at the Donders Institute for
Brain, Cognition and Behaviour, Centre for Cognitive Neuroimaging of the Radboud
University Nijmegen, and the Helmholtz Institute, Department of Experimental
Psychology of the Utrecht University. This work was funded by the Netherlands
Organization for Scientific Research (NWO), grant 400-04-379 to prof. dr. H Chris
Dijkerman and prof. dr. Ivan Toni.
ISBN 978-94-91027-29-1