Developing Cooperative Environment Web Services
based on Action Research
Renate Motschnig-Pitrik, Michael Derntl, Juergen Mangler
University of Vienna
Department of Computer Science and Business Informatics
Rathausstrasse 19/9, 1010 Vienna, Austria
{renate.motschnig, michael.derntl, juergen.mangler}@univie.ac.at
Abstract. Current technology has disappointed many users of cooperative
learning environments by its complexity and only slow and tough adaptability
to specific users’ requirements. In order to compensate for these deficiencies,
we use Action Research to guide us in a process in which we co-develop and
improve open source, Web service based modules that directly and intuitively
support learning, cooperation, and facilitation processes based on users’ experiences. In this paper we characterize selected services, sketch our environment
and propose an accompanying research procedure that allows us to assess and
cyclically improve and extend our tools to closely match both their underlying
didactics and users’ needs.
1
Introduction
Several commercial tools supporting cooperative learning do not fully perform as
expected. They tend to offer extensive functionality and hence are sophisticated in
both their use and installation in a given environment. Often, they fail to support their
users in their specific didactic designs or cooperative endeavors. Based on this observation, our goal is to develop a pool or toolbox of open source modules that support
cooperative, blended learning in academic as well as industrial environments. Appropriate web support modules (such as discussion forums, shared workspaces, or collaborative text production) have the potential to improve cooperation and communication in these knowledge-based environments and the potential to include some of the
tacit and informal knowledge. In general the proposed models support organizational
knowledge gathering and management processes. By intention, we resist the temptation to implement an extensive set of services that are universally applicable. Instead,
we focus on a well-defined set of interpersonal and educational values that we strive
to support in our facilitative way of conducting courses. In brief, these values stem
from a Person-Centered [15, 16], humanistic perspective and can be acquired through
self-initiated learning, experiential learning, whole-person learning, solving authentic
problems, learning from peers, active involvement of participants in all aspects of
teaching/learning [12].
In this paper we focus on the strategy and accompanying research procedure while
developing Cooperative Environment Web Services (CEWebS) [9] incrementally.
2
Renate Motschnig-Pitrik, Michael Derntl, Juergen Mangler
Each service module is based on the practice gained from conducting academic
courses, e.g. on Web Engineering, Project Management, etc., modeling them as scenarios and reusable patterns [5], and deriving the required support to be shared between Web service and facilitator [4]. Since typically courses are repeated each year
or semester, there is potential for improvement, redesign, and extension, if instructors
view themselves both as practitioners and researchers. This process results in a methodology that has become known as Participatory Action Research (e.g., [14]).
The paper is organized as follows. The next Section introduces our extended Action Research framework that we employ and appreciate in the context of developing,
extending, and customizing our Web service based environment. Section three briefly
sketches the characteristics of our approach and tracks a selected Web service module
through the individual phases in the Action Research process. Section four presents
the resulting CEWebS architecture and describes its current status and open source
availability. We conclude by sharing our plans for future practice and research in the
realm of creative and effective cooperation and learning.
2
A Framework for Participatory Action Research
Besides the development of a clean technical solution, our goal is to propose a
thoughtful confluence of methods and tools from the social and technical sciences to
allow for systematic research and assessment of blended cooperation and learning
across organizations. Since, typically, educators or project coordinators themselves
research their innovative blended learning practices and apply their learning and insight iteratively in successive course cycles, participatory Action Research (AR), if
carefully adopted, qualifies in providing the overall research framework. We propose
to base this framework on visual, conceptual models of cooperation and learning
processes and to complement it with a mix of qualitative and quantitative methods for
evaluation, specifically designed to overcome the shortcomings of sole AR.
According to Baskerville [2], the ideal domain of AR is characterized by a social
setting where:
· The researcher is actively involved, with expected benefit for research and organization,
· The knowledge obtained can be immediately applied, based on a clear conceptual
framework,
· The research is a (typically cyclical) process linking theory and practice.
In general, AR is appreciated for integrating and concurrently advancing both practical and theoretical aspects. It accompanies real change effected by real actions in
real organizations and thus has immediate validity for the hosting organization. All
three characteristics fully apply in our setting. AR, however, is also criticized [8] for:
· lacking methodological precision,
· lacking controllability, due to the complexity and fuzziness of real environments,
· being subjective or biased, due to the researchers’ deep personal involvement,
· delivering results that are difficult to generalize.
Developing Cooperative Environment Web Services based on Action Research
3
Although this criticism seems to be inherent in the nature of AR, we have begun to
compensate this by complementing AR with a mix of empirical methods. This combined procedure, as will be illustrated below, re-establishes controllability and objectivity to a significant degree and, through the use of explicit conceptual models, provides paths for generalization of results across organizational boundaries.
According to the prevalent AR description by [18], five phases are iterated:
1. Diagnosing,
2. Action planning,
3. Action taking,
4. Evaluating, and
5. Specifying learning.
In order to overcome most of the difficulties of AR mentioned above, we propose
to complement these basic phases by situated investigation methods that contribute
empirical studies regarding those aspects that are amenable to traditional investigation. The resulting framework is sketched in Fig. 1 and elaborated below.
Specifying
Learning
Diagnosing
+ Assessment regarding selected aspects
+ Adaptation/complementation of theory
+ Extension of concept framework
Action Planning
Evaluation
+ Evaluation of didactic issues of interest
+ Platform evaluation
+ Evaluation of evaluation procedure and tools
+ Comparison of results from multiple researchers
+ Comparison of results from multiple research methods
Diagnosing
+ Formulation of hypotheses
+ Formulation of questions of interest
...
+ Conceptual framework
+ Selection of didactical and technological baseline
+ Selection of parameters to be observed
+ Evaluation planning
Action Taking
+ Evaluation of pre-action state
+ Evaluation of post-action state
+ Reflection
Fig. 1. Proposed extensions to the basic Action Research cycle [10].
In each phase we – as practitioners and researchers – consider three interrelated aspects:
· Didactic baseline,
· Learning platform (technological support), and
· Evaluation process.
Diagnosing. Initially, the problems that shall be overcome by the desired change
are investigated. In the context of blended learning we ask questions such as: What is
the organization’s/department’s current situation regarding the use of New Media?
4
Renate Motschnig-Pitrik, Michael Derntl, Juergen Mangler
Who are the persons we deal with? Where do we want to get? What didactic baseline
underlies teaching/learning? Which tools could we use? What are we interested in?
Action Planning. Based on a conceptual framework, the actions that lead to the
desired future state are planned. Since, in our case the desired future state is quite
complex, namely “improved teaching/learning”, it requires controlled experimentation and can only be approached in each cycle. First, the didactic baseline or, in other
words, the learning paradigm (humanistic education in our case) needs to be selected.
Second, we suggest using visual scenarios that model the activity flows in educational
processes as the conceptual framework. This is because such scenarios are formal
enough to allow for comparisons and specifications of learning platform elements,
and are sufficiently intuitive to be understood by educators. Regarding the platform,
the elements/features needed to support the selected didactical approach need to be
selected and adapted and/or implemented (or at least prototyped). Regarding the
evaluation, we need to formulate research questions, select the parameters to be observed along with appropriate methods and tools, and plan the evaluation process. For
example, the questions to be included in online questionnaires need to be formulated
as well as the time frames, in which the questionnaires shall be completed. Further, it
needs to be decided whether and when reaction sheets shall be collected and/or interviews be made.
Action Taking. In this phase the blended course is conducted, the learning platform is used, and the planned data collection processes are performed in order to
allow for a multi-perspective evaluation in the subsequent phase.
Evaluation. Regarding evaluation, we investigate the students’ motivation, expectations, goals and intentions right after the beginning of the course and in end of the
course by having students fill out online questionnaires. The final questionnaire includes additional questions that depend on the specific course situation, such as: the
use of individual platform elements, the usability of platform features, the quality and
quantity of learning, the role of team work, the instructors’ attitudes, etc. Furthermore,
we include questions and comments on the questionnaire. Histograms and appropriate
quantitative methods are then applied to show the effects of the blended course as
objectively as possible. Although we do not have control groups, we do have courses
that are partitioned into groups that different instructors conduct in the same environment. Also, in the case that the same course is taught by the same instructor in consecutive years, loose comparisons regarding the variation of distinct parameters can
be obtained. Thus, iterations, a core feature of AR, allow us to approach comparability between groups, although not as rigidly as in a classical experimental design employing control groups. It is our goal that open source learning platform modules that
support reusable learning scenarios will enable comparability of parameters and results across organizations.
Specifying Learning. Learning ideally should be amenable to generalization in
order to have more effect. In our case, most learning, clearly, comes from our own
experience, both successes and failures, and from discussing one’s practices with
colleagues. Successful course designs are included in a repository of scenarios and
reusable components thereof are specified as patterns [3]. The evaluation results further indicate which of the goals have been achieved, how the learning platform has
Developing Cooperative Environment Web Services based on Action Research
5
been used and in which ways it was/was not found helpful. For example, the histogram in Fig. 2 shows that Web Engineering students preferred working with the CEWebS platform than with the commercial learning platform that was used to host the
document workspaces. All of these results iteratively flow into the next cycle. As a
side effect, the histograms indicate the influence of individual instructors on platform
usage in general (see also [11]).
Fig. 2. General tool support in Web Engineering (1 = low … 5 = high).
3
CEWebS Sample: From User Requirements to Web Services
Diagnosing. In the blended learning courses we conduct students collaboratively
solve problems or elaborate contributions in small teams of about 2 – 5 members.
These teams are usually formed at an early stage during each course. In previous
years, the instructors asked the students in the initial “course kickoff” meeting to
build their desired team constellations. Each team wrote the resulting constellation on
a piece of paper and handed it over to the instructor or to one of the tutors either directly after the meeting or during a subsequent meeting. On this basis the instructor or
a tutor had to manually “import” the hand-written team sheets (including student
identifiers, names, and e-mail addresses) into the learning platform. In a course with
15 or 25 students this may not be a problem, but:
- What about courses with several hundred students?
- What about team sheets with unreadable or erroneous handwriting?
- What about students who want to switch their team during the initial phases?
- What about team sheets getting lost?
- What about typing errors while entering team data?
In one specific case, the Web Engineering lab course conducted last year, we had
over 350 students organized in 12 lab groups, each with about 8 teams. The lesson we
6
Renate Motschnig-Pitrik, Michael Derntl, Juergen Mangler
learned was that we never wanted to have to manually enter any team constellation
into a learning platform again. Therefore we unanimously decided to conceive and to
implement a CEWebS module for Online Team Building for the next iteration.
Action Planning. From the problems observed the requirements were clear: we
had to include an online module where students could build and manage their team
constellations on their own. During the following semester students of another course
were involved in elaborating a list of usability requirements they expected from the
team building module.
The resulting specification is sketched in Fig. 3. The
figure shows a generic visual template that is capable of
supporting the complete team building process on one
single, interactive Web page. The page consists of three
major sections:
· General information on the team building process as
supplied by the instructor or administrator.
· The participant pool, a list of students who have not
yet joined a team. Each participant in the pool is
shown as a hyperlink that executes one of the following actions, depending on the status of the currently
logged-in participant (i.e., the current user):
- If the current user is not yet member of any team,
clicking on a participant in the pool will create a
new team including the selected participant from
the pool as well as the current user.
- If the current user is already member of a team, Fig. 3. Team building Web
template.
the pool participant is added to the current user’s
team. Basically, this resembles an invitation to
join the team.
· A list of currently existing teams (the current team constellation): Each team carries a hyperlink that, when clicked, makes the current user join that team.
Action Taking. The learning platform module was then developed according to
the above specification. This year, students in all teamwork-based courses that employed our learning platform were instructed to use the online team building module.
Due to its simplicity, this took no more than about five minutes during the course.
They immediately adopted the new module, producing smooth and quick team building processes in all courses and groups (for an example see the screenshot in Fig. 4).
Evaluation and Specifying Learning. As all instructors agreed to ask students to
form teams online and we received neither negative feedback on the online module
nor suggestions for improvement, we intend to keep its basic working principle for
the next application cycle. However, there was one problem that caused confusion:
Each time a team was formed for a specific learning activity, it was assigned a new
team number. If the same team constellation is built for different learning activities
within the same course, two equal team constellations may get different numbers
assigned. This issue naturally produced some confusion among all concerned. It will
Developing Cooperative Environment Web Services based on Action Research
7
be resolved for the upcoming semester by patching the team number generator to
produce equal team numbers for equal team constellations.
These results motivate us to further develop CEWebS since it precisely confirms
our thinking and experiences with learning platform usability. In the next questionnaires we plan to include questions regarding individual platform Web services to
guide and prioritize future implementation and maintenance activities.
Fig. 4. Screenshot taken from the online team building module in Web Engineering.
Currently, the following CEWebS modules are available and in use:
· Contributions: allows students to upload contributions and assignments online.
· Diary: enables students to keep track of their work, and facilitators to monitor
teamwork progress online.
· Discussion forum: enables threaded discussions in online forums that may be dedicated to specific activities, instructors, or issues of interest.
· Evaluation: a generic service that allows collecting feedback in the form of questionnaires, reaction sheets, or written evaluations.
· Learning contract: enables management of online learning contracts.
· Participants & teams: service for displaying course participants and collaborative
building of student teams (compare the example above).
· Wiki [23]: allows administrative staff to collaboratively edit and structure the information pages on the learning platform.
8
Renate Motschnig-Pitrik, Michael Derntl, Juergen Mangler
· Workspaces: virtual document storage that enables students and teams to manage
documents for particular learning activities (e.g., learning contracts) online.
· XML Tools: provides simple online tools for experimenting with different XML
technologies (e.g., Schema, DTD, XPath, XSL, WSDL, etc.).
4
The CEWebS Architecture
Recently, Web services have drawn the attention of learning technology researchers
and practitioners. Current approaches include for example:
- Decentralized, integrated support of Web-based learning processes [19],
- Personalization of such processes in intelligent tutoring systems through Webservice-based agents [7],
- Contract-based provision and discovery of distributed Reusable Learning Object
(RLO) repositories [17], or
- Enhancing the functionality and interoperability of existing learning technology
applications [1].
These approaches employ Web services to increase extensibility and flexibility of
existing solutions and to further standards-based development, dissemination, and
exploitation of desired functionality. Thereby, using Web services for blended learning purposes is all about sharing and open development. The particular approach
presented in the following aims at supporting common blended learning scenarios
through Web services that interact within an open, extensible architecture.
Fig. 5. The Cooperative Environment Web Services (CEWebS) architecture.
Developing Cooperative Environment Web Services based on Action Research
9
The architecture of our framework (see Fig. 5) is composed of three main parts:
1. Transformation Engine (hereafter referred to as TE)
2. Web Services (WS)
3. Administration Website
The TE can be imagined as a container, which is dedicated to hold various components according to an initial course structure (e.g., consisting of homepage, workspaces, and discussion forum). The administrator initializes the TE container with the
users (student data) and the WS’s it should contain. Then he or she initializes and
configures every WS with the user information and additional configuration data that
may be required. That data is collected from the administrator via auto-generated,
Web-based initialization wizards.
As the TE just consists of a simple, standardized SOAP (Simple Object Access
Protocol) [21] interface, any existing learning platform can exploit the CEWebS architecture by implementing that interface. Thereby, the TE acts as the “link” component between the host platform, the users, and the WS’s, by providing two main functionalities:
· Translating users’ HTTP requests into Web service invocations
· Transforming the WS response via stylesheets (XSL, CSS) to meet the user interface guidelines given by the “surrounding” environment (e.g., some commercial
learning platform such as WebCT [22]). The response from a WS is required to be
a “raw” XML document that is defined as a subset of XHTML 1.0 [20].
The WS’s are the central parts of the CEWebS architecture. Every TE can talk to
an arbitrary number of WS’s through a well-defined SOAP interface that has to be
implemented by each WS. Each WS can hold multiple instances of user / configuration data, allowing it to be used by more than one TE. Additionally, each WS provides a formal description (XML Schema) of its configuration capabilities, enabling
the Administration Manager to automatically generate user-friendly configuration
wizards.
Finally, the Administration Website is a centralized service and user repository that
additionally holds information about all TE’s available. The Report Manager, which
is part of the Administration Website, allows administrators to collect reports (e.g., a
list of users that missed the submission deadline for some contribution) from each WS
instance that implements the Report interface. Through the report interface the administrator or instructor always receives binary data, e.g. simple HTML pages, spreadsheets, PDF’s, or pictures containing the solicited information.
The core of the architecture as described above is already working stably. Driven
by additional requirements and conclusions drawn from each research/application
cycle, several components – especially the Web services and the Administration
Website – are still under development and perfective maintenance.
5
Discussion and Future Work
We have sketched a framework for cooperative blended learning and an accompanying research methodology that aims at developing and maintaining open-source, us-
10
Renate Motschnig-Pitrik, Michael Derntl, Juergen Mangler
able Web services that stem from and keep particularly close to the users’ needs.
These services can either be used in the form of a configurable toolbox, or equally be
interlinked with commercial platforms. Initial experiences, users’ reactions and empirical studies substantiate the effectiveness of our approach to facilitating cooperative blended learning [13]. From the social or psychological perspective we base our
approach on the Person-Centered Approach stemming from humanistic psychology.
This approach has proven to be most effective in furthering interpersonal relationships
along with cooperation and significant learning.
Further research will take several directions. First, we are conceptually modeling
generic learning processes that support Person-Centered educational values. We call
the resulting conceptual models PCeL patterns [5] in order to support them with appropriate Web design elements in CEWebS [4]. These are intended to provide the
computerized framework for deep and persistent learning on the one hand and to
support and simplify the organization, administration, and evaluation of PCeL courses
on the other hand. Second, we continue with case studies and Action Research on
PCeL and, concurrently, improve the test instruments in order to be able to observe
the effects of changes. Third, we are in the process of populating a virtual community
of persons interested in the Person-Centered Approach in higher education
(http://elearn.pri.univie.ac.at/pca) in order to have a medium to share experiences and
coordinate research aiming to promote a science that is authentic [6] in so far as it
inherently addresses the genuine questions of the researchers as persons involved in
knowledge management and learning processes. Everybody interested in joining is
welcome. Last but most challenging and influential is the field of staff development in
the spirit of co-developing media-, personal-, and interpersonal competences in order
to facilitate cooperative, blended learning most effectively.
References
[1] S. Apelt, "Using web services to extend an application to meet new requirements," IEEE
Learning Technology Newsletter, vol. 6, 2004, pp. 19-20.
[2] R. L. Baskerville, "Investigating Information Systems with Action Research," Communications of the Association for Information Systems, vol. 2, 1999, pp.
http://cais.isworld.org/articles/2-19/.
[3] M. Derntl, "The Person-Centered e-Learning Pattern Repository: Design for Reuse and
Extensibility," Proc. ED-MEDIA'04 - World Conference on Educational Multimedia, Hypermedia, & Telecommunications, Lugano, Switzerland, 2004, pp. 3856-3861.
[4] M. Derntl and J. Mangler, "Web Services for Blended Learning Patterns," Proc. IEEE
International Conference on Advanced Learning Technologies, Joensuu, Finland, 2004
[5] M. Derntl and R. Motschnig-Pitrik, "Patterns for Blended, Person-Centered Learning:
Strategy, Concepts, Experiences, and Evaluation," Proc. 2004 ACM Symposium on Applied Computing (SAC), Nicosia, Cyprus, 2004, pp. 916-923.
[6] R. Hutterer, "Authentic Science - Some Implications of Carl Rogers's Reflections on Science," Person-Centered Review, vol. 5, 1990, pp. 57-76.
[7] K. Kabassi and M. Virvou, "Using Web Services for Personalised Web-based Learning,"
Educational Technology & Society, vol. 6, 2003, pp. 61-71.
Developing Cooperative Environment Web Services based on Action Research
11
[8] N. Kock, "The three threats of action research: a discussion of methodological antidotes in
the context of an information systems study," Decision Support Systems, vol. 1062, 2003,
pp. 1-22.
[9] J. Mangler and M. Derntl, "CEWebS - Cooperative Environment Web Services," Proc. 4th
International Conference on Knowledge Management (I-KNOW '04), Graz, Austria, 2004
[10] R. Motschnig-Pitrik, "An Action Research-Based Framework for Assessing Blended
Learning Scenarios," Proc. ED-MEDIA'04 - World Conference on Educational Multimedia, Hypermedia, & Telecommunications, Lugano, Switzerland, 2004
[11] R. Motschnig-Pitrik, M. Derntl, and J. Mangler, "Web-Support for Learning Contracts:
Concept and Experiences," Proc. Second International Conference on Multimedia and Information & Communication Technologies in Education (m-ICTE'03), Badajoz, Spain,
2003
[12] R. Motschnig-Pitrik and A. Holzinger, "Student-Centered Teaching Meets New Media:
Concept and Case Study," IEEE Educational Technology & Society, vol. 5, 2002, pp. 160172.
[13] R. Motschnig-Pitrik and K. Mallich, "Effects of Person-Centered Attitudes on Professional
and Social Competence in a Blended Learning Paradigm," IEEE Educational Technology
& Society, to appear.
[14] S. Ottosson, "Participation action research - A key to improved knowledge of management," Technovation, vol. 23, 2003, pp. 87-94.
[15] C. R. Rogers, On Becoming a Person - A Psychotherapists View of Psychotherapy. Constable, London, 1961.
[16] C. R. Rogers, Freedom to Learn for the 80's. Charles E. Merrill Publishing Company,
Columbus, OH, 1983.
[17] S. Sanchez, J. Parra, O. Sanjuàn, and M. A. Sicilia, "Learning object repositories as contract-based Web services," IEEE Learning Technology Newsletter, vol. 6, 2004, pp. 16-18.
[18] G. I. Susman and R. D. Evered, "An assessment of the scientific merits of action research,"
Administrative Science Quarterly, vol. 23, 1978, pp. 582-603.
[19] J. Torres, J. M. Dodero, and C. L. Padrón, "A Framework Based on Web Services Composition for the Adaptability of Complex and Dynamic Learning Processes," IEEE Learning
Technology Newsletter, vol. 6, 2004, pp. 7-11.
[20] W3C, "XHTML 1.0 The Extensible HyperText Markup Language (Second Edition),"
available at http://www.w3.org/TR/xhtml1/ [accessed Jul 20, 2004], 2002.
[21] W3C, "SOAP Specifications," available at http://www.w3.org/TR/soap [accessed Oct 8,
2003], 2004.
[22] WebCT Inc., "WebCT - Learning without Limits," available at http://www.webct.com/
[accessed Apr 2, 2004], 2004.
[23] Wikipedia, "Wiki," available at http://en.wikipedia.org/wiki/Wiki [accessed May 10,
2004], 2004.