1. No category

Networked Interaction in Foreign Language
Acquisition and Research
Grant Agreement Number - 2008 – 4467 / 001 - 001
NIFLAR
Guideline on how to conduct research on
networked interaction
Deliverable number
3.1
Dissemination level
Public
Delivery date
10 December 2010
Status
Final
Author(s)
Rick de Graaff, Kristi Jauregi
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Table of Contents
1. Aims ............................................................................................................................... 3
a. Aims and background of NIFLAR
b. Aims of networked tasks for intercultural communicative competence
c. Aims and target group of the guide
2. Research guideline for VWC / VW (3.1)
a. developmental design-based research:............................................................. 8
i. description and evaluation of (blended) task design procedure
b. quasi-experimental effect research: ..................................................................10
i. effects of VWC/VW tasks on language proficiency
ii. effects of VWC/VW tasks on ICC
iii. effects of VWC/VW tasks on motivation
iv. differential effects of VWC and VW
c. examples of NIFLAR-related research proposals ............................................. 15
d. appendix: ........................................................................................................... 19
i. research checklists
NB: This research guideline may be combined with the Teacher guide (1.2/2.3/1.3/2.4).
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-2-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
1. Aims
a. Aims and background of NIFLAR
The NIFLAR project aimed at studying the opportunities and effects of innovative e-learning
environments for creating authentic and interactive contexts for foreign language learners
and for (future) teachers. In this project, language learners, (future) teachers and researchers
participated from five European countries. Target languages were Spanish, Portuguese,
Dutch and Russian. Tasks were developed, implemented and evaluated for secondary and
higher education at A2-B2 level, for video-web communication using Adobe Connect, and for
virtual worlds using Second Life and Open Sim.
Within the NIFLAR project, partners collaborate in the elaboration of teacher manuals for
secondary and tertiary education on networked communication, interaction formats, task
development, implementation and evaluation, intercultural awareness, and action research.
b. Aims of networked tasks for intercultural communicative competence
In a FL context there are far fewer opportunities to construct rich learning environments as
compared to a natural L2 context: students have limited possibilities to get significant rich
input (both linguistic and intercultural) and engage in meaningful interaction with native
speakers. Communication processes and settings enhanced in FL classrooms are artificial.
However, goal-oriented and realistic communication in authentic and challenging contexts
between nonnative and native speakers is a necessary context for effective language use
and development.
In order to overcome the limitations of foreign language learning in classroom settings,
networked tasks may provide opportunities for meaningful intercultural interaction between
language learners and native speakers. The application of the tasks that are addressed in
this teacher manual may support language teachers:
−
−
−
To cope with the limitations imposed by the FL context by expanding and enriching it,
favouring authentic, contextualized interaction with peer native speakers. Authentic,
contextualized interaction may be facilitated and enhanced twofold: 1) by video-web
communication, for promoting authentic distant interaction; 2) by communication
settings in a Virtual World, for promoting contextualized networked interaction.
To improve FL teaching approaches by creating opportunities for students to engage
in on-line meaningful interactions with peer native speakers according to the
objectives set by relevant tasks which favour information exchange and intercultural
awareness.
To offer actual practice and hands-on experiences in online collaborative language
learning activities and immersive environments and thus support competence
development to address intercultural concerns and successful integration of so-called
second-generation web technologies into language education.
Furthermore, networked tasks may make FL teacher training more authentic, effective and
innovative by: 1) encouraging pre-service teachers to engage in on-line peer-interactions on
intercultural issues with their future target group; 2) analyze interactions within virtual
environments; 3) reflect upon and be aware about the complexity of learning processes in
order to learn to develop tools directed towards giving answers to potential difficulties.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-3-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
This teacher manual facilitates the development, implementation and evaluation of blended
learning tasks directed to enhance peer interaction in the TL. In order to reach this goal,
formal face-to-face instruction with teacher and classmates is combined with two innovative
e-environments: Video-web communication and Virtual Worlds. With this combination
teachers may aim at making learning and teaching of FLs a more relevant, rewarding and
attractive experience, which contributes to enhance motivation and to achieve higher level of
intercultural communicative competence.
c. Aims and target group of the research manual
One of the opportunities to develop and enhance future language teachers’ competencies in
the use of Computer Mediated Communication (CMC) for interactive and collaborative
language learning is by having them actively participate in action research on the effects of
CMC on intercultural communicative competence (ICC). In the NIFLAR project student
teachers acquired competencies for action research on CMC for ICC by participating in task
development, task completion and task evaluation, for video-web communication and virtual
worlds. Action research took take place with respect to task effects on interaction,
intercultural competence, language proficiency, motivation and added value of the CMC
tools.
Regarding research on networked interaction (WP 3.1) this manual contains guidelines for
(student) teachers, focusing on designing and performing research related to task
development and implementation for VWC and VW, and on the effects of task performance
on language proficiency and intercultural communicative competence (ICC).
This manual aims at supporting (future) language teachers who carry out action research on
the task effects on communication, intercultural competence, language proficiency and
added value of the ICT tools (3.1). This manual therefore particularly focuses on the action
research agenda (future) language learners are engaged in, focusing on the rationale and
effects of such action research for their own professional development process.
2. Research guideline for videowebcommunication (VWC) and virtual worlds (VW)
a. developmental design-based research:
i. description and evaluation of (blended) task design procedure
b. quasi-experimental effect research:
i. effects of VWC/VW tasks on language proficiency
ii. effects of VWC/VW tasks on ICC
iii. effects of VWC/VW tasks on motivation
iv. differential effects of VWC and VW
c. examples of NIFLAR related research proposals
d. appendix
NIFLAR research projects aim at contributing to the body of research in networked
interaction and intercultural communicative competence and in so doing enhance reflexive,
analytical and research skills of pre- and in-service teachers of FLs, and teacher trainers.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-4-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
This deliverable contains a guideline on research methodology on how to plan, pilot and
undertake research projects. According to research objectives quantitative/qualitative
methodologies are described; data collection and analysis systems.
Research is needed in order to know whether (and how) oral synchronous communication
(with Video-web communication and Virtual Worlds) may assist SLA and how it may be
implemented in foreign language curricula. Within SLA research, several studies have been
conducted on asynchronous computer mediated communication aiming to show its relevance
for furthering SLA (Chapelle, 2003; Warschauer 2000; Warschauer & Kern 2000). Recently
synchronous communication through textual chat has been the focus of several studies (Lai
& Zhao, 2006; Jepson, 2005; Pellettieri, 2000; Salaberry, 2000; Shekary & Tahririan, 2006;
Smith, 2004; Tudini, 2003, 2007). However, studies analysing cross-cultural interactions
making use of video web-communication are scarce (Wang, 2006; Jauregi & Bañados,
2008). As for Virtual Worlds, although there is some research available on text-based MultiUser Virtual Environments (MUVEs) (Koenraad & Goedemé, 2002; Shield, 2003; Kötter,
2006), the number of studies on language learner interactions in 3D Virtual Worlds is
extremely limited (Svensson, 2003; Molka-Danielsen et all., 2007).
Related to the NIFLAR aims research can be conducted within both communities of
practices: video-web communication and virtual worlds. Research topics:
1. Role of networked synchronous interactive tools in furthering collaborative learning
and intercultural awareness. Studies analysing cross-cultural interactions making use
of video web-communication are scarce (Wang, 2006; Jauregi & Bañados,.2008)
while studies on language learner interactions in 3D Virtual Worlds at secondary level
are non-existent, to our knowledge. The first research line set within this project
seeks to fill in this gap by studying whether and how online interaction through videoweb communication or Virtual Worlds can further communicative and intercultural
competence in the TL
−
−
−
−
−
−
Does virtual Interaction with native peers making use of video-web
communication tools and Virtual Worlds enhance intercultural awareness?
Are there differences according to the tool being used: video-web
communication or virtual worlds?
Which kinds of tasks / issues are best to enhance ICC for what specific
purposes and under what circumstances?
Can 3D Virtual Worlds be used for valid and reliable language assessments?
For what specific purposes and under what circumstances?
How do learners experience interaction through video-web communication or
in virtual worlds?
Do individual differences as measured by standardised questionnaires have
an influence on performance in and appreciation of video-web communication
or virtual worlds?
In order to investigate these issues, interactive sessions in the TL should bee recorded,
transcribed and analysed focussing on learning moments: language related episodes,
negotiation of intercultural, pragmatic and formal meaning and standardised
questionnaires have been developed in order to assess learners’ experience.
2. Another research line focuses on gaining a deeper understanding of processes
developed in Virtual World settings as related to the video-web communication:
Role of tasks in furthering FL acquisition and intercultural awareness in networked
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-5-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
communication, according to differences in typology and interactive setting: what are
the specific and complementary advantages of video communication and virtual world
settings for effective FL acquisition and intercultural/pragmatic awareness?
1. Research foci NIFLAR:
1.1. Main research focus: To investigate the added value of interaction with NSs
through VWC &/or VW in the development of Intercultural Communicative
Competence of participants (as compared to traditional settings).
1.2. To describe which aspects of ICC are developed in interaction with NSs through
VWC &/or VW.
1.3. To study the role different conditions play in the development of ICC:
•
•
•
•
•
Role of tasks
Role of technique: affordances of the environments, technique working properly or
badly, VWC versus VW
Role of individual issues (language learning proficiency level, ICC level, personal
factors, willingness to communicate…)
Role of pre-service / in-service teacher (+/- involvement)
Degree of subjectivity being reached in interaction among speech partners
Need to focus on task development and ICC development research in language
teacher training
In the departments of Modern Languages and the teacher training institutes involved in the
NIFLAR project, very little attention is paid to research methodology on design-based or
(quasi) experimental research. Many language teachers, as a consequence, have very little
background in setting up, carrying out an evaluating research on task effectiveness and
intercultural or linguistic proficiency of language learners. However, (future) language
teachers need to possess basic research skills on task development and task effects, for the
following reasons:
1. They should be able to systematically analyze and evaluate the potential
effectiveness of tasks as offered for example in course books and online resources.
2. They should be able to adapt tasks to the needs and level of their language students.
3. They should be able to develop effective tasks themselves, based on task design
principles and/or good practice examples.
4. They should be able to evaluate the effect of tasks on the language learning process
of their students, related to relevant aspects of language proficiency and/or
intercultural communicative competence.
5. They should be able to use the evaluation results in order to adapt, refine and
redesign tasks, aiming at meeting best the needs of their students as a group and/or
of individual students.
Teacher competences 1, 2, 3 an 5 relate to a design-based approach to language teaching
and language teaching research; teacher competence 4 relates to an experimental effect
study approach to language teaching and language teaching research. As the aims of many
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-6-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
teacher education programmes point out, good teachers are reflective teachers, that is, are
able to reflect on the effect of their own teaching role on their students’ learning activities and
results. Building future teachers’ competence in reflecting on and evaluating the effects of
their own teaching (such as the tasks they apply), is beneficial for their own teaching
performance, and, as a consequence, for their students’ learning process.
In NIFLAR, therefore, (pre-service) language teachers have been involved in task design,
task evaluation and evaluation of ICC competence development of language learners. The
present research guideline, which was developed and applied for pre-service teachers
involved in the NIFLAR project, may serve as a basis for course materials in future courses
on task-based ICC research for pre-service language teachers. It may also serve as a
starting point for a reflection on the position of research methodology in bachelor’s and
master’s programmes for future language teachers.
Research on learning and educational settings is based on research methodology from
social sciences (Educational science, Cognitive psychology). Therefore, a course syllabus for
pre-service language teachers may focus on the following topics that are common in most
methodology course syllabi in the social sciences:
•
Main research types (empirical-analytical; interpretative; design-based etc.)
•
The empirical cycle (observation-induction-deduction-assessment-evaluation)
•
Preparing a study: problem statement and hypothesis testing
•
Preparing a study: specifying method; addressing reliability and validity
•
Carrying out a study: data collecting (including task development and implementation)
and data analysis
•
Reporting research.
We refer our Dutch pre-service student teachers to:
‘t Hart, H., Boeije, H., & Hox, J. (Eds.) (2005). Onderzoeksmethoden [Research methods].
The Hague: Boom onderwijs.
Oost, H. et al. (2002). Een onderzoek voorbereiden. [Preparing research]. Amersfoort:
Thieme Meulenhoff.
Oost, H. et al. (2002). Een onderzoek uitvoeren. [Carrying out research]. Amersfoort: Thieme
Meulenhoff.
Oost, H. et al. (2002). Een onderzoek rapporteren. [Reporting research]. Amersfoort: Thieme
Meulenhoff.
Oost, H. et al. (2002). Een onderzoek presenteren. [Presenting research]. Amersfoort:
Thieme Meulenhoff.
a. developmental design-based research:
In order to design, implement and evaluate VWC/VW tasks for ICC, we recommend (student)
teachers to take a developmental design-based approach. That is, to carry out an
intervention study design cyclus, in which evaluation results are used for redesign and
reimplementation.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-7-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Educational design research is defined as “the systematic study of designing, developing and
evaluating educational interventions (such as programs, teaching-learning strategies and
materials, products and systems) as solutions for complex problems in educational practice,
which also aims at advancing our knowledge about the characteristics of these interventions
and the processes of designing and developing them.” (T. Plomp (2009), Educational design
research, an introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational
design research (pp. 9-36). Enschede, the Netherlands: SLO.)
The research process in design research encompasses educational design processes. It is –
like all systematic educational and instructional design processes - therefore cyclical in
character: analysis, design, evaluation and revision activities are iterated until a satisfying
balance between ideals (‘the intended’) and realization has been achieved.
Reeves (2006) depicts the stages in design research approach as follows:
1. Identify and analyse problems by researchers & practitioners in collaboration;
2. Development of prototype solutions: informed by state-of-art theory, existing design
principals & technology innovations;
3. Iterative cycles of testing & refinement of solutions in practice;
4. Reflection to produce ‘design principles’ & enhance solution implementation in practice.
Reeves, T.C. (2006). Design research from a technology perspective. In: Van den Akker, J.,
Gravemeijer, K, McKenney, S. & Nieveen, N. (Eds). (2006). Educational design research.
London: Routledge, 52-66.
Although within NIFLAR it was usually not feasible for (student) teachers to carry out iterative
cycles of testing and refinement themselves, they could evaluate, refine and reapply tasks
and task principles developed by other (student) teachers and NIFLAR researchers. As such,
they could contribute to the growing body of knowledge on task development for ICC by
VWC/VW, and on a growing set of tasks that were applied and evaluated in language
teaching practice. The awareness raising tasks (see deliverable 1.2/2.3) were an essential
component in both the design cycle for task development and the learning cycle for student
teachers.
Student teachers involved in NIFLAR have participated through online videoconferencing in
guest lectures by Colpaert (June 2010) and by Hauck (September 2010), focusing on a
design-oriented approach to VWC/VW task research. These lectures are available at
www.niflar.eu:
http://cms.hum.uu.nl/niflar/index.php/home-2/presentations.html
Colpaert takes an educational engineering perspective, related to principles of educational
design research discussed above. According to Colpaert it is not feasible to find clear proofs
or hard evidence for generalizable effects of NIFLAR VWC/VW tasks on ICC or language
proficiency, simple because there are too many uncontrolled variables. On the other hand,
any well-documented task design, implementation and evaluation description is relevant
evidence for success (or failure) is as well, in that specific context. Therefore, a focus on
design-based and case-based descriptions is proposed. An educational engineering
perspective allows for a well-structured construction and description of learning environment,
learning process and related and learning outcome.
Following Colpaert’s approach, we analyze, design, develop, implement and evaluate in an
iterative process aiming at improving teaching and learning.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-8-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Source: Colpaert, videoconference lecture, 29 June 2010.
As such, the design process and its consequences for teaching and learning becomes the
focus of research. Data analysis can focus on a wide range of factors, such as learning
results, resistance and acceptance, motivation and attitudes, content development, software
development, hardware, project management, ... In other words, it provides insight into the
design process and the entire learning environment, allowing for methodologically justified
changes in an iterative procedure. Effects are the outcome of a wide range of related factors,
and should be approached and reported as such.
Further reading:
J. Colpaert. “Elicitation of language learners’ personal goals as design concepts.” Innovation
in Language Learning and Teaching. Taylor and Francis. In print.
L. Cohen, L. Manion & K. Morrisson. Research Methods in Education. 2000.
P. Reason & H. Bradbury. Handbook of Action Research. Participative Inquiry and Practice.
2001.
Hauck (2010) points out that the role of the task designer and the actual process of task
design remains unexploited in published research on (tele-collaborative) task-based
language teaching (TBLT). She advocates that data collection and analysis calls for a
broader sociocultural research paradigm: a more detailed description of the learners, settings
and events in [CALL] contexts, and a better understanding of how exactly all of these factors
interact and operate in real pedagogical contexts. We therefore propose our pre-service
language teachers to take data triangulation perspective on design-oriented TBLT research,
combining:
•
•
•
•
•
A detailed description of the educational design cycle
Pre- and post-questionnaires
Transcripts from VWC or VW recordings
Learner and teacher portfolios
(Semi)-structured interviews
In order to do so, several qualitative research paradigms apply, such as
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
-9-
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
•
•
•
Action research: “a systematic, iterative process of (1) identifying an issue, problem,
or puzzle we wish to investigate in our own context; (2) thinking and planning an
appropriate action to address that concern; (3) carrying out the action; (4) observing
the apparent outcomes of the action; (5) reflecting on the outcomes and on other
possibilities; and (6) repeating these steps again.” (Nunan & Bailey, 2009).
Case study: “a ‘bounded instance’ […] whether those boundaries are physical (a
certain school site, a child), or temporal, which is studied in context, focussing on
observation, description, inference and interpretation, all important facts of
ethnographic and practitioner research.” (Nunan & Bailey, 2009).
Ethnography: investigating a small number of cases (or even just the one case) by
working with unstructured data which have not been coded according to a predetermined set of analytic categories, taking the socio-cultural nature of teaching and
learning languages into consideration and also taking account of the participants’
reflections and intentions.
Hauck, Mirjam (2010). Telecollaboration: at the interface between multimodal and
intercultural communicative competence. In: Guth, Sarah and Helm, Francesca (Eds.)
Telecollaboration 2.0: Language, Literacies and Intercultural Learning in the 21st
Century. Telecollaboration in Education (1). Bern: Peter Lang, pp. 219–248.
Nunan, D. & Bailey, K.M. (2009). Exploring second language classroom research. A
comprehensive guide. Boston: Heinle.
b. quasi-experimental effect research:
NB: This part of the Research guideline is based on the outcome reported in deliverable 3.3
(Global research results).
It is quite challenging to show effects of VWC/VW tasks within NIFLAR. We have aimed at
developing and implementing tasks that promote:
•
•
•
language proficiency
intercultural communicative competence
motivation
distinguishing VWC, VW and classroom contexts. IN the NIFLAR context, most treatments,
that is, series of tasks, are rather limited (1 – 5 times 0.5 – 2 hours), and most tasks allow for
many different types of interaction. Therefore, an (educational engineering) design based
case-study approach, as explained in the previous section, can be a valid and informative
way to address designing, teaching, leaning and interaction issues.
On the other hand, the data collected in many parallel and consecutive task-based
experiments in the NIFLAR project also allow a more quantitative approach, aiming proving
positive effects of VWC/VW tasks on language proficiency, intercultural communicative
competence and willingness to communicate.
A first step for in-service language teachers to set up and carry out (quasi-) experimental and
case-study research themselves, is critically reading and evaluating related research
publications. We recommend, among others:
Brown, J.D. (1988). Understanding research in second language learning. Cambridge
University Press.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 10 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Porte, G.K. (2002). Appraising research in second language learning; a practical approach to
critical analysis of quantitative research. Amsterdam: John Benjamins.
Related to the research questions addressed, different types of semi-experimental and case
studies can be conducted. The most relevant ones for NIFLAR are addressed below.
Studying ICC development in general terms
a) Comparing interaction sessions (session 1 & last session): focus on IC/linguistic
growth:
X____________X_____________X_____________X____________X
Session 1
Session 5
For research validity interactions to be compared should be similar in terms of task
format, content and quantity of interlocutors engaged in conversation.
In order to be able to detect growth sessions should be planned carefully: students
should have one session per week and should participate in as much sessions as
possible (5 better than 3). We shouldn’t expect to be able to detect relevant growth in
three sessions. Questionnaires, interviews / reflections could be used as additional
information sources.
b) Comparing values of pre-test (before interaction begins) with post-tests (after
conclusion of last interaction session)
Pre-test
X___________X____________X_____________X____________X Post-test
Pre- & post-tests should be similar in terms of format, content, speech acts being enacted
and at the right competence level (use CEFR).
We could be using oral tasks (to measure communicative competence) +
questionnaires (to elicit specific intercultural / pragmatic elements).
How to know whether this growth is due to NIFLAR sessions or to the general language
course students follow?
By comparing groups in different conditions:
•
•
•
Control group: does not participate in NIFLAR and follows regular teaching
Experimental group 1: participates in VWC sessions
Experimental group 2: participates in VW sessions (only for Spanish)
The question is whether we can compare small groups of participants as it is the case by
Russian, and Portuguese. And the answer is “yes, we can”. It is indeed possible to
compare conditions with a small quantity of subjects in the form of a case study. When
discussing the results we should make clear that these are not generalizable to other
cohorts and that more research is needed. However, since within NIFLAR different
students at different levels and target languages are participating in the interaction
sessions, we could use all the results in an attempt to give a substantial answer to the
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 11 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
question about the added value of VWC/VW as facilitating tools of native non-native
speaker interaction.
If growth is detected in experimental groups how to know what this growth is due to?
By resorting to the analysis of different data sources:
Interaction data:
a)
b)
c)
d)
How do interlocutors negotiate IC content according to the task being
carried out?
What roles do speech partners have in interaction?
• How involved/passive are speech partners?
• How balanced / unbalanced is individual contribution to the interaction?
• Is any of the speaker controlling the interaction in terms of quantity of
production, topic initiation and change or questions being asked?
What language and culture related episodes are generated during the
sessions? (quantity and quality)
Analysis of the native speaker input FLLs get (pragmatic aspects present in
NS output)
Subjective data:
Questionnaires, interviews, reflections (in blogs, diaries): related to questions: what
have you learned? Will be used for additional information.
Data sources:
o Recoded interactions
o Reflection participants in blogs or diaries
o Analysis reports (pre-service teachers)
o Presentations participants
o Pre-tests & post-tests (to measure growth)
o Pre-questionnaires & post-questionnaires
o Teachers’ reports
o Interviews
Data analysis
In order to measure communicative growth analyse:
•
•
(recorded) pre- & post-tests
(recorded) interactions first & last session
following the description levels of the ECFR & according to the following categories
a)
b)
c)
d)
Range
Fluency
Accuracy
Topic development
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 12 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
e) Coherence
by applying in each case a 10 point scale system.
This will allow us to rate the overall performance in each case and measure possible growth
by applying statistical analysis.
In order for the pre-service teachers to perform rating similarly, in other words, to establish
inter-rater reliability, a training was organised, where different researchers and pre-service
teachers listened to short samples of interaction, valued them per category applying a 10
point scale system and discussed results.
•
On the other hand, subjective information sources on learning (questionnaires /
interviews / reflection documents) have been used to revalidate findings
If growth is detected in experimental groups how to know what this growth is due to?
Analyse Interaction data:
a) How do interlocutors negotiate IC content according to the task being
carried out?
b) What roles do speech partners have in interaction?
• How involved/passive are speech partners?
• How balanced / unbalanced is individual contribution to the interaction?
• Is any of the speaker controlling the interaction in terms of quantity of
production, topic initiation and change or questions being asked?
c) Which language and culture related episodes are generated during the
sessions? (quantity and quality)
Several scholars have advocated the use of a critical events approach, focusing on
key situations to arouse cognitive conflicts in internet-mediated foreign language
exchanges. This can be done at the level of negotiation both for linguistic meaning
and for intercultural understanding. Such situations or instances can be described as
Language-related episodes (LRE) and Culture-related episodes (CRE).
In their presentation at EuroCALL 2009 in Gandia (Spain), Ciekanski and Chanier
promote cognitive conflict and misunderstanding as an opportunity for learning, and,
therefore, as an focus for research. They claim that, probably more so than in any
other kind of communication, language learners interacting with native speakers are
exposed to rules of interaction they do not master. Therefore, they claim, learners
may be able to improve their ICC according to the way they cope with critical events.
Hopkins, too, at EuroCALL 2009, addresses the importance of analyzing multimodal
discourse in VWC/VW, focusing on critical events, in order to identify instances of
interaction viewed as beneficial to second language acquisition. This might relate not
only to language-related episodes, such as negotiation of meaning, negative
feedback and scaffolding, but also to culture-related episodes.
Language related episodes (LRE) (negotiations where linguistic learning is
taking place). LREs are mini-dialogues, in which learners ask or talk about
language, or explicitly or implicitly question their own language use or that of
others (Swain & Lapkin, 1998). In general, an LRE starts with the identification of
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 13 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
a language problem (I) and ends with a resolution (R; RR) (see Shekary &
Tahririan, 2006).
Culture related episodes (negotiations where cultural learning is taking place).
Select and analyse those fragments where speech partners show Byram’s (1997)
five savoirs:
Attitudes: curiosity and openness, readiness to suspend disbelief about other
cultures and belief about one’s own.
Knowledge of social groups and their products and practices in one’s own
and one’s interlocutor’s country, and of the general processes of societal and
individual interaction.
Skills of interpreting and relating: ability to interpret a document or event
from another culture, to explain it and relate it to documents from one’s own.
Skills of discovery and interaction: ability to acquire new knowledge of a
culture and cultural practices and the ability to operate knowledge, attitudes
and skills under the constraints of real-time communication and interaction.
Critical cultural awareness/ political education: an ability to evaluate
critically and on the basis of explicit criteria perspectives, practices and
products in one’s own and other cultures and countries.
Related to Byram’s interaction skills we may include an analysis of a selection of
Müller-Jacquier’s categories of intercultural pragmatics (2003):
o Social meaning
o Speech acts
o Organization of conversation: (turn taking
o Choice & development of topics
o Directness / indirectness
o Politeness strategies
o Register
o Para-verbal factors
o Non-verbal means of expression
o Culture specific values / attitudes
o Culture specific behavior
Conclusion
In NIFLAR, (pre-service) language teachers have been involved in task design, task
evaluation and evaluation of ICC competence development of language learners. The
present research guideline, which was developed and applied for pre-service teachers
involved in the NIFLAR project, may serve as a basis for course materials in future courses
on task-based ICC research for pre-service language teachers. It may also serve as a
starting point for a reflection on the position of research methodology in bachelor’s and
master’s programmes for future language teachers.
c. examples of NIFLAR-related research proposals
researcher / research
team
Revenga García, Gómez Molina
De Graaff, Koenraad
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 14 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
master students and/or
student teachers involved
in the research team?
If not, why not?
other NIFLAR partners
involved in the study
research question
Revenga García is a pre-service teacher at Valencia University.
This study is part of her Master’s thesis
Comenius College, Hilversum
Institut Clot de Moro, Sagunt, Valencia
How do secondary students value their participation in
OpenSim virtual world sessions for ICC?
research methodology
Analysis and comparison of pre and post questionnaires
How does this research
question relate to the main
NIFLAR research aims?
How does this proposal fit
within the NIFLAR
research methodology?
If not, why?
What data needed?
- participants
- level
- task
- …
It seeks to explore the added value of 3D virtual worlds to enhance
ICC in secondary education.
Attempts to address the appreciation of the added value.
Questionnaires
All Dutch and Spanish participants in 3 Spanish and 3 English
tandem sessions in OpenSim, at level A2
dataset available?
Completed pre- and post-Questionnaires
analysis instruments
needed
Descriptive statistics
Analysis instruments
available?
Planning of the study
November-December 2010
Type of publication:
- article (journal?)
- conference paper
- book chapter
- …
Will be included in Master’s thesis, 2011.
researcher / research
team
José Ramón Gómez Molina
research question
Do NIFLAR sessions contribute to enhance technical skills,
teacher training and motivation of pre-service teachers?
research methodology
Analysis and comparison of pre- and post-questionnaires
How doe this research
question relate to the main
It seeks to explore the added value in both virtual environments in
teacher training
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 15 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
NIFLAR research aims?
How does this proposal fit
within the NIFLAR
research methodology?
If not, why?
What data needed?
- participants
- level
- task
- …
Attemps to address the added value
Pre & Postquestionnaires of pre-service teachers
dataset available?
Pre & Postquestionnaires of pre-service teachers
Planning of the study
October-november
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 16 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
researcher / research
team
Jauregi
master students and/or
student teachers involved
in the research team?
If not, why not?
other NIFLAR partners
involved in the study
??
research question
research methodology
How doe this research
question relate to the main
NIFLAR research aims?
How does this proposal fit
within the NIFLAR
research methodology?
If not, why?
What data needed?
- participants
- level
- task
- …
dataset available?
What learning opportunities do emerge in interaction
sessions with native speakers through video-web
communication or Second Life?
Case study. In depth analysis of different issues contributing
to enhance learning: negotiation of production, negotiation of
comprehension, negotiation of culture and compare it with the
interaction on the same task FLls carry out in the traditional
classroom .
It seeks to explore the added value of video-web communication
and or second Life to enhance communicative competence.
Attempts to address the added value.
Recordings of tasks of experimental and control group
Recordings
analysis instruments
needed
Analysis instruments
available?
Planning of the study
December should hopefully be ready
Type of publication:
- article (journal?)
- conference paper
- book chapter
- …
Hopefully publication.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 17 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
researcher / research
team
Silvia, Kristi
master students and/or
student teachers involved
in the research team?
If not, why not?
other NIFLAR partners
involved in the study
To be confirmed
research question
SL experimental group vs. control group: have the Niflar
sessions contributed to students’ learning experiences?
research methodology
Analysis and comparison of pre- and post- tests and pre-, midand post- questionnaires
In depth analysis and comparison of tasks 1 and 5
How doe this research
question relate to the main
NIFLAR research aims?
How does this proposal fit
within the NIFLAR
research methodology?
If not, why?
What data needed?
- participants
- level
- task
- …
It seeks to explore the added value of virtual worlds in the
participants’ learning experiences as opposed to traditional settings
dataset available?
José Ramón
Attempts to address the added value.
Comparing pre-and post-tests and first and last interaction sessions
Pre- and post-tests
Pre- and post-questionnaires
Session recordings
yes
analysis instruments
needed
Analysis instruments
available?
Planning of the study
December 2010
Type of publication:
- article (journal?)
- conference paper
- book chapter
- …
Aimed at potential publications
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 18 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
d. appendix: student teacher Research checklist
Example from the UU Master course Language Education
Manual written assignment Language Education
As part of the Language Education course you prepare a final Group assignment, consisting
of a concise presentation and a written report. This manual addresses the assessment
criteria for the presentation and the report. It also provides some guidelines for your group
process and the final product. It also contains a checklist for feedback and assessment for
presentations and reports.
Written assignment
Aim of the report
By means of the written report students show:
•
they can design language tasks for intercultural communicative competence based
on TBLT and ICC theory
•
they can specifcy these tasks for the CEFR level of the target group language
learners
•
They can evaluate the tasks with respect to:
•
o
feasibility
o
effect on interaction
o
effect on language proficiency
o
role of the teacher and native speaker interlocutor
o
preconditions and added value of the VWC or VW environment being used.
They can relate their evaluation results to theory on TBLT and ICC.
Report outline:
The report contains the following parts. For each of the chapter 1 – 5 one of the students is
responsible; the chapters are written in the target language of the respective students.
0. Introduction:
a. What research questions will be addressed in the report?
b. Which sources from TBLT and ICC theory have been used for your problem
statement?
c. How will your research questions be answered? What methodology will be
used?
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 19 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
1. Tasks
a. Which tasks have been developed?
Describe the tasks by means of the task design grid.
b. Do the tasks meet the requirements for effective tasks?
Evaluate the tasks by means of the task evaluation grid. Provide evidence for
the issues you address in your evaluation.
2. Teacher roles (specific focus depends on your research questions)
a. Creating authentic communicative input
b. Facilitating processing for meaning
c. Facilitating interaction
d. Facilitating intercultural understanding
e. Facilitating correct / adequate language use
f.
Facilitating the use of communication strategies
g. Providing feedback on a – f.
3. Language leaner roles (specific focus depends on your research questions)
Assess one of the language learners with respect to:
a. range, accuracy, fluency, interaction, coherence, and/or appropriateness (see
assessment grid interaction sessions);
b. intercultural understanding.
c. Relate your assessment to the task specifications
4. Role and effects of video-web communication environment (specific focus depends
on your research questions)
a. What has been the affordance of the environment for the task effects?
b. What was the added value of the environment for promoting effective
interaction?
c. What were the constraints of the environment that affected interaction?
d. What has been done to best promote the added value and best avoid the
constraints?
5. Discussion
Provide suggestions for task improvement and implementation, related to your own
research questions: “If you want to apply task X in environment Y in order to meet
goal Z, then XYZ can best be organized as follows, taking into account variables
ABC.”
6. Conclusions
Provide a summary of the answers to your research questions, and suggestions for
further research.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 20 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
7. Sources
Systematically list your bibliographical references. For an APA style sheer, see:
http://www.apastyle.org/apa-style-help.aspx
http://flash1r.apa.org/apastyle/basics/index.htm.
8. Appendices
Provide the completed task design and task evaluation grids, and any other
information relevtn for the assessment of your report.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 21 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Assessment form for written assignments on NIFLAR tasks
Language Education
Educational Master course
– fail; 0 pass; + ecellent
1.
Content
0. introduction:
•
Specification of research question
•
relevance of sources used
•
justification of method proposed
1. tasks:
•
adequate task description
•
justification of task evaluation
analysis and evaluation:
•
2. teacher roles
•
3. learner roles
•
4. role and effects of video-web communication
environment
5. discussion:
•
relevant suggestions for improvement
6. conclusion:
•
relationship with research questions
•
justification related to theory discussed
7, 8. sources and appendices:
•
relevance of references
•
consistence of references (format)
•
originality of references
•
all task descriptions in appendix
2.
0
+
–
0
+
–
0
+
–
0
+
–
0
+
–
0
+
–
–
0
0
+
+
–
–
–
–
–
0
0
0
0
0
+
+
+
+
+
Structure and justification
Effective focus and report structure (chapters)
Jstification of task division (for group assignment)
3.
–
Language use and presentation
Appropriate style for target audience
Spelling and grammar
Lay-out
Lenght
Adequate target language terminology
Total result
Strongest point:
To be improved:
Final result:
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 22 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Manual written assignment Language Education
(original text, in Dutch)
Voor de cursus Language Education bereid je in groepjes een eindopdracht voor, die bestaat
uit een beknopte presentatie en een schriftelijk verslag. In deze handleiding staat uitgewerkt
waaraan de presentatie en het verslag dienen te voldoen, inclusief een aantal adviezen voor
werkwijze en resultaat. Daarnaast bevat de handleiding een feedback- en
beoordelingschecklist voor de presentaties en de verslagen.
Schriftelijk verslag
Doel van het verslag
Met het verslag tonen de studenten aan dat ze:
•
op een wetenschappelijk inzichtelijke en onderbouwde manier taaltaken kunnen
ontwikkelen voor interculturele communicatieve competentie;
•
deze taken kunnen specificeren voor het beoogde ERK-taalvaardigheidsniveau van
de taallleerders;
•
de taken kunnen evalueren op:
•
o
haalbaarheid van de uitvoering,
o
effect op de interactie,
o
effect op het taalverwervingsproces,
o
rol van de docent,
o
de randvoorwaarden en meerwaarde van de gebruikte video-web
communicatieomgeving;
deze evaluatie kunnen onderbouwen met inzichten uit relevante wetenschappelijke
bronnen
Opzet van het verslag:
Het verslag bestaat uit onderstaande onderdelen. Voor ieder van de hoofdstukken 1 t/m 5 is
een student verantwoordelijk; het hoofdstuk is geschreven in de betreffende taal van studie.
0. Inleiding:
a. welke onderzoeksvragen ga je in het verslag beantwoorden?
b. welke wetenschappelijke bronnen heb je gebruikt om deze vragen te
formuleren?
c. op welke manier ga je deze onderzoeksvragen beantwoorden?
1. Taken
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 23 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
a. welke taken heb je ontwikkeld?
Beschrijf de taken met behulp van het task design grid.
b. waarom zijn dit goede taken?
Evalueer een of meer taken met behulp van het task evaluation grid. Motiveer
en onderbouw de gegeven evaluatie.
2. Docentrollen (focus is afhankelijk van de eigen onderzoeksvragen)
welke rollen hebben jullie als docenten gespeeld m.b.t.:
a. het creëren van authentiek communicatief taalaanbod
b. het stimuleren van taalbegrip
c. het stimuleren van interactie
d. het stimuleren van intercultureel bewustzijn
e. het stimuleren van correct en/of adequaat taalgebruik
f.
het stimuleren van taalleerstrategieën
g. het geven van feedback bij bovengenoemde punten
3. Studentrollen (focus is afhankelijk van de eigen onderzoeksvragen)
hoe evalueer je een of meer studenten m.b.t.:
a. range, accuracy, fluency, interaction, coherence, en/of appropriateness (zie
assessment grid interaction sessions);
b. interculturele bijdrage en/of intercultureel bewustzijn.
c. hoe verklaar je deze beoordeling op grond van de uitgevoerde taken?
4. Rol en effectiviteit van de video-webcommunicatieomgeving (focus is afhankelijk van de
eigen onderzoeksvragen)
a. hoe heeft de omgeving bijgedragen aan het effect van de taaltaken?
b. wat was de meerwaarde van de omgeving voor het realiseren van effectieve
interactie?
c. wat waren de beperkingen van de omgeving voor het realiseren van effectieve
interactie?
d. hoe is de meerwaarde zo veel mogelijk benut en zijn de beperkingen zo veel
mogelijk voorkomen?
5. Discussie
adviezen voor verbetering van taken en interactie, geformuleerd aan collega-docenten
(“Als je taaltaak X via omgeving Y wilt inzetten voor het bereiken van leerdoel Z, dan kun
je XYZ het beste als volgt organiseren, en dien je rekening te houden met ABC.”). Focus
hierbij vooral op de eigen onderzoeksvragen.
6. Conclusies
samenvattende beantwoording van de onderzoeksvragen uit hoofdstuk 0, en suggesties
voor vervolgonderzoek.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 24 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
7. Bronnen
Geef een systematisch overzicht van geraadpleegde bronnen.
Gebruikelijk is het om in ons vakgebied de APA Style te hanteren. Zie
http://www.apastyle.org/apa-style-help.aspx
http://flash1r.apa.org/apastyle/basics/index.htm.
8. Bijlagen
Neem de ingevulde task design en task evaluation grids, en andere stukken die relevant
zijn voor de beoordeling, op in de bijlage, plaats ze in de SURFgroepen met een
verwijzing in het verslag.
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 25 -
NIFLAR – 3.1: Guideline on how to conduct research
on networked interaction
Assessment form written assignments on NIFLAR tasks
cursus Language Education
Educatieve Master, i.s.m. IVLOS
invulinstructies: z.o.z.
4.
– onvoldoende; 0 voldoende; + goed
Inhoud
0. inleiding:
•
•
•
helderheid en afbakening onderzoeksvraag
relevantie van gebruikte bronnen
verantwoording van gebruikte methode
–
0
+
–
0
+
–
0
+
–
–
0
0
+
+
–
–
–
–
–
0
0
0
0
0
+
+
+
+
+
1. taken:
•
adequate beschrijving van de taken
•
onderbouwde evaluatie van de taken
analyse en evaluatie:
•
2. docentrollen
•
3. studentrollen
•
4. rol en effectiviteit van video-web
communicatieomgeving
5. discussie:
•
zinvolle en overtuigende verbetersuggesties
6. conclusie:
•
relatie met de onderzoeksvragen
•
onderbouwing en argumentatie mbv besproken
theorie
7, 8. bronnen en bijlagen:
•
relevantie van bronvermelding en
(internet)referenties
•
consistentie van bronvermelding en
(internet)referenties (format)
•
originaliteit van bronvermelding en
(internet)referenties
•
beschikbaarheid van relevante bijlagen
5.
Structuur en verantwoording
doelgerichte opbouw en hoofdstukindeling
taakverdeling en verantwoording (bij groepsopdracht)
6.
Vorm en presentatie
doelgroepgerichte stijl
verzorging: spelling en grammatica
verzorging: lay-out
omvang
juiste taalspecifieke terminologie/woordgebruik
Totaal
sterkste punt:
verbeterpunten:
globaal
eindoordeel:
Project number – 143472-LLP-1-2008-1-NL-KA2-KA2MP
- 26 -