a) Bid cover sheet JISC Grant Funding 5/11
Cover Sheet for Bids
(All sections must be completed)
Name of JISC Initiative (identify strand
ie A, B or C):
Assessment and Feedback Programme Strand C
Name of Lead Institution:
Name of Proposed Project:
Name(s) of Project Partners(s)
(except commercial sector – see
below)
Kingston University
Uniqurate
University of Glasgow, Harper Adams University College and
University of Strathclyde
This project involves one or more
commercial sector partners
YES / NO (delete as appropriate)
Full Contact Details for Primary Contact:
Name:
Graham Alsop/David Livingstone
Position:
Principal Lecturer/Senior Lecturer
Email:
[email protected]/[email protected]
Tel:
02084172889
Address:
Faculty of Science, Engineering and Computing
School of Computing and Information Systems, Kingston University,
Road, Kingston upon Thames, SURREY KT1 2EE
Length of Project:
Project Start Date:
18 months
1st Sept 2011
Project End Date:
Penrhyn
28 Feb 2013
Total Funding Requested from JISC:
£39,994
Funding requested from JISC broken down across Academic Years (Aug-July)
Aug11 – July12
Aug12 – July13
£24,566
£15,427
Total Institutional Contributions:
£57,438
Outline Project Description
The main objective of this project is to increase the usage of standards-based electronic assessment tools
in HE. This will be achieved by refactoring the available tools that produce content in the Question and Test
Interoperability Specification 2.1 (QTI2.1) format. These tools will extended via an Agile development
approach involving fresh “client” organisations to add new UI elements and features designed to facilitate
their use by novice users.
I have looked at the example FOI form at Appendix A and
included an FOI form in this bid
I have read the Funding Call and associated Terms and
Conditions of Grant at Appendix B
For FE institutions only: Please tick this box if you are an FE
institution in England, please tick this box to confirm that you
meet the eligibility requirement of teaching HE to more than 400
FTE
YES / NO (delete as appropriate)
YES / NO (delete as appropriate)
b) Appropriateness and fit to programme objectives and overall value to the JISC community Introduction 1. The main objective of this project is to increase the usage of standards-based electronic assessment
tools in HE. This will be achieved by refactoring the available tools that produce content in the Question
and Test Interoperability Specification 2.1 (QTI2.1) format. These tools will extended via an Agile
development approach involving fresh “client” organisations to add new UI elements and features
designed to facilitate their use by novice users.
Background 2. Several projects have developed tools for the creation of electronic assessment content that uses the
Question and Test Interoperability Specification version 2.1 (QTI2.1) format. Examples of these projects
are QTITools1, MathAssess2 and FETLAR3,4, which resulted in the question authoring tools Aqurate5
and Mathqurate6, and the test authoring tool Spectatus7.
3. QTI2.1 is important for a number of reasons. It allows for the exchange of question items and tests
among different systems. Thus questions authored and stored in this format will be transferable among
systems (this is being further ensured by the ongoing QTI Implementation, Profiling and Support (QTIIPS) project). Given the current climate in FE and HE there is likely to be a further need to utilize
intelligent ways of undertaking both formative and summative assessment. Maxmising this ability
through an open source approach based upon open standards will support this need.
4. In terms of QTI’s assessment ability the specification provides the means for authors to include
individualised feedback depending on the student’s input, in addition to hints and solutions. Such
feedback can range from “OK, that’s the right answer” to “you have correctly labelled all of the
components apart from the ‘fanbelt’, please look at the solution to see the correct location” or “You
seem to have made an error in calculating the coefficient of x2” and may contain multimedia objects as
appropriate. These comments may be provided in conjunction with the opportunity to submit another
attempt. This capability is valuable for formative and self assessment and enriches the learning
environment for students while giving the tutor a means of addressing problems directly and promptly.
Transferring QTI to a wider community 5. The existing applications have now reached a degree of maturity. However, they were developed either
as proof of concepts or specifically targeted at expert users – expert in this sense meaning either within
a subject area, or with respect to the QTI specification. For example, Aqurate and its complementary
projects represent a showcase of what could be done generally with QTI 2.0, and the subsequent
application Mathqurate was developed for a particular subject group and to test and develop
mathematical extensions to the QTI 2.1 specification. As a result, the software that emerged was
designed for Mathematicians and QTI experts. This was an appropriate outcome for these projects and
their specified target audiences. However, as they stand these tools are only readily usable by those
who are au fait with the specific subject areas and with QTI itself.
6. Clearly, this is not the ideal scenario for potential authors of electronic assessment content with no prior
knowledge or experience with QTI. New users find the existing tools daunting and they are not easily
transferrable to those outside the original project circle. This has had an impact on their uptake and,
arguably, on the uptake of QTI 2.1 as a platform for electronic assessment in HE. Even within
institutions such as ours where expert knowledge of QTI and the current tools is available, the authoring
of QTI e-assessment content remains almost entirely with those who were directly involved in previous
projects. The skills they use come from their immersion in QTI for many years and are not transferable
short of subjecting a new user to similar immersion. Notwithstanding the portability of QTI as a
standard, this raises questions over sustainability of QTI content (institutionally speaking) if these
experts become unavailable in the future.
7. This project will take the existing QTI authoring tools Spectatus and Mathqurate, and transfer them to
two named client institutions. During an iterative Agile development process, the applications will be
1
th
http://www.qtitools.org, accessed 12 July 2011
th
http://www.jisc.ac.uk/whatwedo/projects/mathssasses.aspx, accessed 12 July 2011
3
th
http://www.fetlar.bham.ac.uk, accessed 12 July 2011
4
th
http://aqurate.kingston.ac.uk/fetlar, accessed 12 July 2011
5
th
http://aqurate.kingston.ac.uk/indexold.htm, accessed 12 July 2011
6
th
http://aqurate.kingston.ac.uk/mathqurate, accessed 12 July 2011
7
th
http://aqurate.kingston.ac.uk/spectatus, accessed 12 July 2011
2
modified to detach them from any specific subject area, and to allow QTI naïfs to create questions and
tests without the need to be familiar with the specification. The current expectation is that wizard-style
dialogues will be added to the applications that allow users to build questions from a range of
templates, designed to provide an accessible user experience that allows the creation of content
ranging from simple multiple choice up to more complex questions that evaluate user input. However,
depending on user feedback during the Agile process, this may change (see paragraphs 11-17). In
parallel with application development, support and documentation will be provided to the client
institution on how to best utilise the tools within their institutional context.
8. The ultimate result of the project will be to extend the use of these tools beyond their current “inner
circle”. This will encourage the creation of new content and break the cycle where currently the use of
and authoring of content in QTIv2.1 in HE is confined to a knowledgeable few. Each of the partner
institutions on this project has a need for integrated feedback-rich assessment in STEM disciplines. By
providing tools that extend the number of QTI content authors we address not only this need in the
partner institutions, but also encourage the adoption of open standards elsewhere. This will provide the
means for widespread sharing and repurposing of content and encourages software providers to
incorporate import and export facilities for QTI objects into their systems. This additional flexibility will
increase the efficiency of content production and add value to students, staff and institutions alike.
9. This project is allied to the QTIDI project, which aims to transfer a generalised, standards-based
connector for integrating the MathAssessEngine renderer for QTIv2.1 into VLEs to the same group of
partner institutions. While both projects stand alone, if both are successful it is anticipated that the two
projects will support one another – this has already occurred during the composition of the two bids,
which will be evident in their content. The knowledge base with respect to QTI has been built up by
working on several inter-institutional JISC and HEA funded projects over a number of years. Several
members of the project team are currently involved in another JISC project focusing on QTI (QTI-IPS).
The two new projects will continue to share expertise and personnel, and in combination will provide a
complete QTI-based end-to-end solution, providing the means for academics to author innovative
assessment items and the mechanism for delivering them to students in a form which integrates fully
with their course.
10. The name of this project is taken from the existing question authoring tools, Aquarate and Mathqurate.
The primary objective is that universally accessible and transferable QTI2.1 authoring tools will result –
hence Uniqurate.
c) Quality of proposal and robustness of workplan Project planning, management and development methodology 11. An Agile, iterative approach will be taken towards software development that will be loosely based on
Scrum. It is expected that there will be in the region of six “sprints”. Each sprint will open with an online
planning meeting, which will consider the software outputs of the previous sprint and feed back on the
user experience and recommendations for improvements. This will inform the software development
that will take place during the sprint.
12. By its very nature, an Agile approach mandates an adaptive approach to planning. It should therefore
be noted that the following is indicative, and by design will evolve depending on the needs of the client
institutions through the lifecycle of the project. Similarly, Table 1 shows a list of staff, their roles and the
time they will spend on the project; the times give are averaged out over the lifetime of the project and
are not intended to be rigidly fixed and/or scheduled – for example, it may be that a staff member
spends several days working on the project, and is then inactive for several subsequent weeks. This
flexibility will be important for the success of the Agile approach.
13. It is anticipated that, while the project will run from September 2011 to Feb 2013, the main thrust of
development will take place over approximately 12 months between December 2011/November 2012.
This will allow for the initial planning, for the current versions of the tools to be installed at the client
institutions and to allow sufficient time for the client institutions to make use of the existing tools and
begin to formulate the required feedback that will drive the development process.
"scrum"
1 week
2 months
"sprint"
Product
backlog
document
Software
Increment
sprint planning
meeting
8
Figure 1 - Adapted Scrum approach
14. The first sprint will use the current state of the tools as its basis for discussion, and its planning meeting
will define a “product backlog” document, outlining required features and change controls to the tools.
Subsequent sprints will “tick off” and, potentially, add items to this document depending on the results
of the development work during the sprint and the subsequent feedback from the user community.
15. During each sprint, regular online “scrums” will occur. Ideally these will take the form of a brief online
session at which users and developer will specify what they have done since the last scrum, what their
plans are before the next scrum, and any needs or potential obstacles that may stand in their way. If an
online session is not possible, a concise email will suffice. The intent is to ensure that, despite physical
distance, a continuous line of communication occurs between team members, with a feedbackdevelopment-feedback cycle that enables the user community to shape the tools to their requirements.
16. The same Agile, Scrum-based approach will also be shared by sister project QTIDI.
17. The project will be managed in accordance with the JISC Project Management Guidelines. David
Livingstone will project manage, assisted by Graham Alsop.
Staff List, roles and availability Name (Initials)
Institution
Role
Time allocated
Paul Neve (PN)
Kingston
University
Senior E-learning software developer
– main software developer and
development lead
3 days per week, between
December 2011 and
November 2012 inclusive
Graham Alsop
(GA)
Kingston
University
PI and academic/project champion
and manager
½ day per week
David Livingstone
(DL)
Kingston
University
PI and academic/project champion
and manager
½ day per week
Niall Barr (NB)
University of
Glasgow
QTI expert and software developer –
integration assistance and facilitation
between authoring tools and
renderers (i.e. QTIDI project)
½ day per week
Sue Milne (SM)
Independent
consultant
Documentation and QTI schema
support
9 days over the duration of
the project
Michael Aherne
(MA)
University of
Strathclyde
E-learning developer - on-site support ½ day per week
and roll-out for “client” institution
Sue Barnes (SB)
University of
Strathclyde
Project/Academic champion
½ day per week
Project champion
½ day per week
Roger Greenhalgh Harper Adams
(RG)
8
Adapted from original material from http://openclipart.org.
Name (Initials)
Stephen Spencer
(SS)
Institution
Harper Adams
David White (DW) Harper Adams
Role
Time allocated
E-learning developer - on-site support ½ day per week
and roll-out for “client” institution
Academic champion
½ day per week
Table 1 – staff details
Project scheduling Task
Timing
Deliverable
Documentation/project management tasks
Throughout
Project blog, wiki and
discussion forums
Confirm project plan
Sep 2011
Project Plan
Dissemination/liaison with community
Dec 2011, July,
Conference paper/participation,
Sep, Dec 2012+
online presence, plus
ongoing
involvement at any QTI profiling
meetings
Write final project blog entry
Feb 2013
Final report
Project management
Evaluate software
Evaluate project
Evaulation tasks
Dec 2012 – Jan
Software evaluation report
2013
(also ongoing – see
paragraphs 20-24)
Dec 2012 – Jan
Project evaluation report
2013
Technology transfer/development tasks
Initial deployment of existing tools at
Sep 2011
Existing tools installed and
client institutions
operational at receiving
institutions
First use of tools at client institutions
October 2011
Feedback for Sprint 1
Sprint Planning meeting 1
November 2011
Product backlog document
(PBD)
Sprint 1/ongoing client use of existing
Dec 2011/Jan 2012 Iteration 1 of software deployed
tools
Planning meeting for Sprint 2
Jan 2012
Modified PBD
Sprint 2/client use of iteration 1
Feb/Mar 2012
Iteration 2 of software
Planning meeting for Sprint 3
Mar 2012
Modified PBD
Sprint 3/client use of iteration 2
Apr/May 2012
Iteration 3 of software
Planning meeting for Sprint 4
May 2012
Modified PBD
Sprint 4/client use of iteration 3
June/July 2012
Iteration 4 of software
Planning meeting for Sprint 5
July 2012
Modified PBD
Sprint 5/client use of iteration 5
Aug/Sep 2012
Iteration 5 of software
Planning meeting for Sprint 6
Sep 2012
Modified PBD
Sprint 6/first client use of iteration 6
Oct/Nov 2012
Final version of software
Ongoing use of (final) iteration 6
December 2012Feeds into software/project
ongoing
evaluation report later
Post development support
November 2012Contributes to support site
ongoing
Duration
(days)
4.5
1
6
5
4
4
2
5
0.5
20
2
23
2
21
2
21
2
21
2
16
20+
ongoing
10
Table 2 - project tasks and timings
18. Note that for many tasks, the durations given in table 2 involve several project members and should not
be mistaken for person days. For example, “Sprint 2/client use of iteration 1” involves the lead
!"#$%%
&'($%%
)*+$%%
,"'$%%
-./$%0
1"2$%0
3.4$%0
5#4$%0
3.6$%0
-7/$%0
-78$%0
579$%0
!"#$%0
&'($%0
)*+$%0
,"'$%0
-./$%:
1"2$%:
developer working on new code to accommodate the outputs, while the end-users make use of the
software in their day-to-day work, with a view to informing the next planning meeting.
,*'7;"/.(<*/=>4*?"'(@3./.9";"/(
!"#$%&'()*+*,%-%+'
.#+/0"-(!"#$%&'(!1*+
2033%-0+*'0#+
4+'%"0-(5%6#"'
70+*1(5%6#"'
A+.87.(<*/
8#/'9*"%(:;*1<*'0#+
!"#$%&'(:;*1<*'0#+
B"'C/*8*96@B4./DE"4=F"+"8*#;"/(
4+0'0*1(=%61#>-%+'(#/('##13('#(&10%+'3
70"3'(<3%(#/('##13(?>(&10%+'3
86"0+'(@(61*++0+,
86"0+'(@(&10%+'(<3%(#/(%A03'0+,('##13
86"0+'(B(61*++0+,
86"0+'(B(&10%+'(<3%(#/(0'%"*'0#+(B
86"0+'(C(61*++0+,
86"0+'(C(&10%+'(<3%(#/(0'%"*'0#+(C
86"0+'(D(61*++0+,
86"0+'(D(&10%+'(<3%(#/(0'%"*'0#+(C
86"0+'(E(61*++0+,
86"0+'(E(&10%+'(<3%(#/(0'%"*'0#+(D
86"0+'(F(61*++0+,
86"0+'(F(&10%+'(<3%(#/(0'%"*'0#+(E
Figure 2 - High level overview of proposed project schedule
Risk Analysis Risk
P
(1-5)
2
S
(1-5)
5
Score
(PxS)
10
Delays occur in
development/ feedback
cycle
Withdrawal of client
institution
3
4
12
2
3
6
Changes to existing
software prove to be nonfeasible, either in the
timeframe or with available
expertise
2
3
6
Loss of key staff or
consultants.
Mitigation
Documentation of plans and technical
artefacts will reduce impact and allow for
others to assume responsibility.
Relationships from past projects exist that
allow for experts outside the present
consortium to be drafted in if needed.
Project team are all in post and/or
availability confirmed at this point in time.
The number of development “sprints” or
their duration can be tweaked, depending
on needs, to compensate for such delays.
If required, lead institution from sister
project could assume role of client on this
project. Additionally, new client
institution(s) can be sourced from already
interested parties in the community.
Development personnel involved wrote
the existing software, so are intimately
familiar with it. Agile development
approach will ensure requirements and
specified functionality are realistic.
Table 3 – Risks and Mitigations (P=Probability S=Severity)
Deliverables 19. This project will deliver the following:
•
•
•
•
•
•
•
•
A universally accessible QTI question authoring tool, based on Mathqurate and Aqurate, that can be
used by someone new to both QTI and to the authoring tool itself with a minimum of effort and
tuition.
A universally accessible QTI test authoring tool, based on Spectatus that can be used by someone
new to both QTI and to the authoring tool itself with a minimum of effort and tuition.
• NB: Based on user feedback it may be considered desirable to merge question and test
authoring into a single application.
• Software artefacts will be provided as an installable binary for the major operating systems
and will be available from the project website/blog, as well as from other appropriate online
locations (e.g. JISC Design Studio, KU Learning Technology Research Group web site, etc)
Source code available via SourceForge (or similar open repository), offered under an appropriate
open source licence.
Developer documentation for the software, targeted at those wishing to make changes to the source
code. This to include class diagrams and Javadoc documentation.
Technical documentation for the software, targeted at IT administrators and expert users who will
support end users in the use of the application(s).
User documentation for the software, targeted at end users.
A community web presence, to include a blog, wiki and forum/discussion area so as to promote
inter-institutional communication and collaboration across three areas:
• Software development
• Day to day use. Suggested subforums might include areas such as “tips and tricks”, tutorials
and guides, and/or an area for peer-to-peer support where members of the end-user
community can source input from other end-users
• Content sharing and collaboration
Journal/conference papers over the course of the lifetime of the project
Evaluation Mechanisms 20. The Agile approach that will be taken incorporates a continuous cycle of evaluation throughout the
software development cycle, indicated by the lighter shaded areas in Figure 2. As noted previously, this
will be a crucial part of the evolution of the software, and will take place after each iteration to inform
subsequent development work.
21. Towards the end of the project a period of two months (December 2012-January 2013) will formally be
devoted to evaluation across two discrete strands: software and project, as evident in figure 2.
22. In conjunction with the client institutions and the user community, the software strand will evaluate the
final iteration of the software against the project objective, specifically: can a novice user with no prior
knowledge of the software or of QTI successfully create electronic assessment content? A series of
“goals” will be created where a “guinea pig” user will be asked to author a piece of content ranging from
a simple multiple choice question (MCQ) to a more complex question that evaluates user input – the
precise form of the latter will obviously depend on the functionality that arises from the Agile process.
These “guinea pig” users will be selected either from within the partner institutions or institutions who
have become involved via the online community, but it is key that these users be genuinely new to the
tools, the project and QTI itself. The aim is to determine the feasibility of a truly novice user authoring
content – if such a user succeeds in the specified “goals”, we will have created a truly transferable
piece of software.
23. The second strand of evaluation will focus on the project as a whole, specifically with regard to the
project management and Agile development approaches. This will seek to ascertain the efficiency of
these processes, particularly with respect to the distances involved and the need to adapt the Scrum
methodology to an online, non-physical paradigm, where availability times of team members do not
necessarily coincide. While there is research to indicate that “distributed” Scrum9 can be successful, we
9
Essentially, Scrum spread out over large distances and, potentially, timezones. See
http://www.goodagile.com/distributedscrumprimer/. Also see Fully Distributed Scrum: Linear Scalability of Production between San
Francisco and India – available from http://jeffsutherland.com/SutherlandFullyDistributedScrumSFIndiaAgile2009.pdf. Both
accessed 14th July 2010.
are also proposing to stretch the timescales of the methodology beyond those usually suggested. Our
second evaluatory strand will examine whether this leisurely flavour of Scrum worked, whether it added
value to the project and whether it would be advisable to adopt it for future similar projects.
24. It is expected that evaluation will also continue past the end of the project within the wider community,
via the online arena, and will continue to shape development of the tools beyond the end of the project.
The lighter grey areas in Figure 3 for February 2013 denote this: while the chart ends at this point, this
ongoing evaluation will continue beyond the boundaries of the chart.
IPR position 25. All software to be used or produced within the project already bears an open source licence. All
documentation created by the project will be shared under a Creative Commons Attribution Share-Alike
licence.
d) Engagement with the community 26. Past projects have made their software artefacts available under open source licences and this will
continue to be the case here. Source code will be made available via SourceForge. This is crucial if the
tools are to be truly sustainable, allowing for future development of the tools beyond the life of this
project. Learning technologists within the client institutions will be provided with support in the
deployment of the tools and their modification. In combination with the documentation that will be
available on the wiki, this will enable on-site technology professionals at the client institutions to
continue to support and develop the tools beyond the life of the current project, arming the future
community and facilitating post-project development work. The two client institutions will be able to
support each other and learn from the others’ experience, sharing any new knowledge or modifications,
which should provide the foundation for a development community beyond the life of the project itself.
Over the course of the project, opportunities for dissemination will enable us to demonstrate the tools to
other colleagues. There have been several who have expressed an interest via conferences and other
discussion forums.
27. The QTI-IPS project is currently creating a support site for both technical and academic users of
QTIv2.1, including documentation and annotated example questions and code samples. These facilities
will be available for the client institutions within this project and its sister project, and indeed for others
who adopt the tools, and it is expected that such use will contribute to the sustainability of the support
site itself, as well as that of the tools. Several proposed team members of this project are currently
engaged on the QTI-IPS project, which will provide useful continuity of knowledge and expertise, and
ensure that its resources can quickly be applied here.
28. As noted previously, the relationship with the sister QTIDI project is integral. The lead developer of the
Uniqurate project is embedded on QTIDI, and the PI of QTIDI embedded here. This will not only ensure
constant engagement with the other project, but also vital cross-pollination of knowledge between the
two.
29. To enable the embedding of the agile approach and assure sustainability several complementary forms
of media will be utilised:
• For project documentation a Wiki (and blogs were appropriate) with an RSS feed will be used.
• Output in terms of documentation and code from the Sprints will be published through the Wiki and
on SourceForge.
• For project ‘narrowcast’ communication a Twitter hashtag of #qtitools will be exploited.
• For project ‘broadcast’ communication a Twitter hashtag of #qti will be deployed and encouraged to
be taken up by other related projects.
• The project team and surrounding community are already embedded into the existing CETISAssessment-SIG through recent project work – this will continue.
• The outputs will be exhibited nationally at conferences including ALT-C and CAA, and
internationally via the IMS.
30. In the past OSS Watch have been an invaluable resource, and this is expected to continue in this
project. We expect to liaise closely with them to ensure that both the letter and the spirit of open source
licencing is met, both with respect to the software artefacts we produce, but also the libraries and other
software we may leverage within our tools. The lead developer will liaise with OSS Watch when new
open source libraries and code not directly authored by the project team are introduced into the project,
this contact to take place during and as an integral part of development sprints. This will ensure that the
wider implications of such inclusion are adequately understood, and help avoid potential
incompatibilities between different open source licences.
31. There are already lines of communication in place between individuals on the project team and JISCCETIS, with several already attending JISC programme meetings in their existing capacities. These
pre-existing relationships will be leveraged to ensure appropriate engagement with the programme and
will be supplemented by the links with sister project QTIDI.
32. The team has been contacted by the HE-STEM project10 and we expect to engage with them during the
project.
Stakeholders Stakeholder
Interest / stake
Importance
Senior management at partner
institutions
• HEFCE, JISC and HEA
• IMS
• UK Examination Authorities
• Worldwide Education/Training
community
Software engineers and QTI
specialists
Producers of assessment
resources
Successful outcome to project
Medium
Sustainable, user-friendly tools
conforming to QTIv2.1 standard
made available for creating
assessment resources
High
Will see their investment in QTI
enhanced
Rich cross-platform assessment
items can be authored and
combined into assessments
Future-proofing, sharing and
repurposing of resources is
facilitated
Continued UK involvement in
standards-conformant eassessment systems development
High
End-users
JISC CETIS Assessment SIG
Medium
High
Medium
Table 4 - Stakeholder analysis
33. The adoption of sophisticated authoring tools and the creation of more advanced question types
requires support. Although this project is of relatively short duration, the groundwork for this support
provision can be put in place and its efficacy investigated.
34. The community of e-assessment users needs to be made aware of the benefits of QTI and engaged in
its implementation within their institutions. This is particularly important for those who own large
collections of questions and tests, including the examination authorities in the UK. The project will
enable users to engage with QTI with confidence that the standards-based resources and tools they are
using are sustainable.
Dissemination 35. Along with the dissemination that will happen organically via the online channels within the growing
community, It is expected that the client institutions will disseminate the tools to their colleagues within
the institutions.
36. The software outputs of the project will be disseminated at a JISC-CETIS meeting during 2012 and to at
least two UK national conferences. These will be selected from CAA2012, ALT-C 2012 and MSOR
2012. A presentation at the Scottish E-Assessment Conference in August 2012 is also likely.
10
http://www.hestem.ac.uk/activity/integrating-formative-and-summative-e-assessment-mathematics-teaching-and-learningfirst-an (accessed 14th July 2010)
e) Budget Directly Incurred
August 11– July 12
August 12– July 13
August 13– July 14
TOTAL £
Staff
(Strand A only)
Neve sp33 0.6fte(3dpw) Kingston
£17,252
£8,963
£0
£26,215
Stephen Spencer 0.05FTE(82.5hpa) sp26 Harper Adams e-Learning
£957Developer
£690
£0
£1,647
Michael Aherne 0.05FTE(82.5hpa) sp38 Strathclyde e-Learning
£1,501
Developer
£1,000
£0
£2,501
Niall Barr 0.05FTE(82.5hpa) sp36 Glasgow
£1,294
£933
£0
£2,227
Sue Milne 9 days @ £422pd
£2,110
£1,688
£0
£3,798
Total Directly Incurred Staff (A)
£23,114
£13,274
£0
£36,388
Non-Staff
Travel and expenses
Hardware/software
Dissemination
Evaluation
Other
Total Directly Incurred Non-Staff (B)
August 11– July 12
£600
£0
£400
£0
August 13– July 14 (Strand
TOTAL
A only)
£
£1,000
£1,600
£0
£0
£0
£0
£0
£0
£24,114
£14,874
£0
£
Directly Incurred Total (C)
(A+B=C)
Directly Allocated
August 12– July 13
£600
£0
£400
£600
£
August 11– July 12
August 12– July 13
£1,200
£0
£800
£600
£0
£2,600
£38,988
August 13– July 14 (Strand
TOTAL
A only)
£
Staff
Estates
Other
Directly Allocated Total (D)
£9,075
£3,507
£0
£12,582
£5,560
£2,500
£0
£8,060
£0
£0
£0
£0
£14,634
£6,008
£0
£20,642
Indirect Costs (E)
£21,934
£15,868
£0
£37,802
Total Project Cost (C+D+E)
Amount Requested from JISC
Institutional Contributions
£58,630
£24,566
£34,063
£38,802
£15,427
£23,375
£0
£0
£0
£97,432
£39,994
£57,438
Percentage Contributions over the life of JISC
the project
Partners
41%
No. FTEs used to calculate indirect and
estates charges, and staff included
No FTEs
Total
59%
Which Staff
0.7 Paul Neve Kingston
Stephen Spencer Harper Adams
Michael Aherne Strathclyde
Niall Barr Glasgow
David Livingstone Kingston
Graham Alsop Kingston
Roger GreenHalgh Harper Adams
David White Harper Adams
Sue Barnes Strathclyde
100%
b. Project team experience 37. Paul Neve MSc (Kingston) has a wealth of experience developing software solutions for educational
needs. He was Senior E-Learning Software Developer on the FETLAR project, involved with the
development of the Mathqurate QTI item authoring tool, author of the Spectatus QTI test authoring tool,
and creator of the FETLAR virtual appliance which provided an instantly deployable package of all
FETLAR project outputs, both software and content. He is also the author of the WLab online learning
environment for delivering IT-related practical workshops, which presented at ALT-C 2010. In his
current position as a Learning Technologist at Kingston University he is developing the NoobLab
electronic learning suite for teaching introductory computer programming. He will act as development
and technical lead on this project.
38. David Livingstone BSc MSc (Kingston) will share PI and PM responsibilities with Graham Alsop. He is
a Senior Lecturer in the Faculty of Computing, Information Systems and Mathematics. David’s research
interests include educational technologies, GIS and location-based services. He was the joint recipient
in 2001 of the Journal of Geography in Higher Education Biennial Award for Promoting Excellence in
Teaching and Learning. He has successfully managed, delivered and contributed to previous JISC
development and demonstration projects, including Aqurate, MathAssess and FETLAR.
39. Graham Alsop BSc MSc (Kingston) will share PI and PM responsibilities with David Livingstone. He is
a Principal Lecturer in Computing and Information Systems, Field Leader and has been researching the
use of learning technologies for some years. He leads the Learning technology Research Group in the
School. He was Faculty Educational Technology Leader 2000-2006 (aiding the implementation and use
of Blackboard and other learning technologies into teaching) and Associate Director (Learning and
Teaching) for the New Technology Institute 2002-2004/ He managed the JISC funded Aqurate’s project
2007-08, Aqurate’s contribution to MathAssess 2008, and the follow on involvement in FETLAR.
40. Michael Aherne (Strathclyde) has worked in the education sector as a software developer for over
twelve years. He has been involved in running and supporting Virtual Learning Environments, including
WebCT and Moodle, for over 8 years, and was a lead developer on the COLEG Online Assessment
Project (COLA), which produced QTI items from Microsoft Word templates.
41. Sue Barnes (Strathclyde) has a PhD in Maths from Glasgow University. She taught Maths in
secondary schools for many years and since completing her PhD has managed the Maths Skills
Support Centre and worked as a Learning Technology Advisor at the University of Strathclyde.
42. Niall Barr (Glasgow) has been deeply involved with interoperability standards including QTI 2.0 and
2.1, Tools Interoperability 1.0 and the Common Cartridge. He currently works at the Learning &
Teaching Centre, supporting and developing learning technology. As the University's representative on
the IMS Technical Advisory Board and a member of the QTI 2.1 working group, he will oversee the
development and packaging aspects of the project.
43. Roger Greenhaigh BSc, MSc, DIC, FRES, CertEd (Harper Adams) is Head of Information Services at
Harper Adams University College. He is an IT solutions architect and software development team
leader with further experience of both teaching and eLearning delivery. He has institutional project
leadership experience on a number of HEFCE-, HEIF- and JISC-funded projects including the
NationalRural Knowledge Exchange, OpenFields open-access repository and Ripple Cascade OER
projects. He has particular interests in syndication of digital resources.
44. David White BSc, MSc, CEng, CEnv, FIAgrE, MIMechE (Harper Adams) is a Senior Lecturer in
Engineering at Harper Adams University College. He is a mechanical/agricultural engineer with 20
years experience teaching mathematics, statics and dynamics, the design of agricultural machinery and
off road vehicles at degree and masters level. He has carried out applied research for external
organisations and is currently supervising several postgraduate students. David is Vice-President of the
Institution of Agricultural Engineers.
45. Stephen Spencer (Harper Adams) is an eLearning Developer at Harper Adams University College. He
has responsibility for development and delivery of the Harper Adams Moodle VLE, and has technical
strengths in handling the underpinning PHP, MySQL and XML framework, skills which have been
invaluable in the development of custom VLE plug-ins, VLE/eAssessment platform interoperability, and
user interface development. Stephen also provides a range of end-user support services to the various
types of Harper Adams VLE users.
46. Sue Milne was director of the CALMAT group at Glasgow Caledonian University when they
implemented QTI v2.0 for the MELA e-learning system, and was a consultant to the MathAssess and
FETLAR projects. She has extensive experience of authoring QTI questions over many years, and is
familiar with the QTI2.1 specification and the profiles being developed. She will author the
documentation and act as a QTI expert on this project.