Download Report

D
e
p
a
r
t
me
nto
fC
o
mp
ut
e
rS
c
i
e
nc
ea
ndE
ng
i
ne
e
r
i
ng
Vil
l
eT
.H
e
ikkil
ä
Casest
udie
so
n re
l
e
asepl
anning in agil
ede
ve
l
o
pme
nto
rganiz
at
io
ns
Casest
udie
so
n re
l
e
ase
pl
anning in agil
e
de
ve
l
o
pme
nto
rganiz
at
io
ns
V
i
l
l
eT
.H
e
i
k
k
i
l
ä
A
a
l
t
oU
ni
v
e
r
s
i
t
y
D
O
C
T
O
R
A
L
D
I
S
S
E
R
T
A
T
I
O
N
S
Aalto University publication series
DOCTORAL DISSERTATIONS 6/2015
Case studies on release planning in
agile development organizations
Ville T. Heikkilä
A doctoral dissertation completed for the degree of Doctor of
Science (Technology) to be defended, with the permission of the
Aalto University School of Science, at a public examination held in
Auditorium T2 of the school on 23 January 2015 at noon.
Aalto University
School of Science
Department of Computer Science and Engineering
Software Process Research Group
Supervising professor
Professor Casper Lassenius
Thesis advisor
Doctor Maria Paasivaara
Preliminary examiners
Professor Richard Torkar
Chalmers University of Technology, Sweden
Professor Kari Smolander
Lappeenranta University of Technology, Finland
Opponents
Professor Tony Gorschek
Blekinge Institute of Technology, Sweden
Aalto University publication series
DOCTORAL DISSERTATIONS 6/2015
© Ville T. Heikkilä
ISBN 978-952-60-6045-3 (printed)
ISBN 978-952-60-6046-0 (pdf)
ISSN-L 1799-4934
ISSN 1799-4934 (printed)
ISSN 1799-4942 (pdf)
http://urn.fi/URN:ISBN:978-952-60-6046-0
Images: The cover image is a deritative of "Gears" by William Warby
under Creative Commons Attribution 2.0 Generic, "Waves 2" by Tony
Hisgett under Creative Commons Attribution 2.0 Generic and
"Snowball" by redjar under Creative Commons Attribution-ShareAlike
2.0 Generic. The cover image is published under the Creative
Commons Attribution-ShareAlike 2.0 Generic licence.
Unigrafia Oy
Helsinki 2015
Finland
Abstract
Aalto University, P.O. Box 11000, FI-00076 Aalto www.aalto.fi
Author
Ville T. Heikkilä
Name of the doctoral dissertation
Case studies on release planning in agile development organizations
Publisher School of Science
Unit Department of Computer Science and Engineering
Series Aalto University publication series DOCTORAL DISSERTATIONS 6/2015
Field of research Software Engineering
Manuscript submitted 13 May 2014
Date of the defence 23 January 2015
Permission to publish granted (date) 5 November 2014
Language English
Monograph
Article dissertation (summary + original articles)
Abstract
The goal of software release planning is to plan the appropriate scope for the next release
while taking into account the availability of resources, the priorities of the candidate requirements and any other factors and constraints that affect the requirements selection. Previous
research on release planning has focussed on algorithmic, model-based solutions that expect
the requirements selection factors and release constraints to be known. In reality, the factors
and constraints are rarely known beforehand. Furthermore, modern, agile software development methods are based on rapid development cycles and constant communication between
stakeholders instead of planning and plan execution.
This dissertation presents three case studies on release planning methods that were applied
by two large and one small agile software development organizations. The goals of this research
were to study how the organizations performed release planning, how the release planning
methods affected the team-level planning practices, and what beneﬁts the organizations gained
and what challenges they faced in applying the methods. The cases were studied with
observations, interviews and surveys and with quantitative requirements analysis.
The case organizations employed different release planning methods. In Case Agilefant, a
model-based method was applied by asking stakeholders to vote on the different priorities of
the candidate requirements. In Case F-Secure, the release planning method regularly gathered
all stakeholders of the project into the same space to plan the next release. In Case Ericsson,
the method was continuous and based on the incremental elaboration of requirements. The
team-level planning practices mostly followed the Scrum guidance. The ﬁrst identiﬁed challenge was that the candidate requirements were not identiﬁed, selected and prepared in a way
that supported the release planning methods. The second challenge was the immature adoption
of agile. The beneﬁts identiﬁed in the methods were the mediation of multiple opinions, high
efﬁciency, the creation of high-quality plans and the increased developer motivation.
The results implicate that release planning should not be considered a task that managers
perform in isolation from the developers. The tacit knowledge of the whole organization should
be harnessed in release planning. In addition, this dissertation depicts several challenges that
an organization may face when planning a release and puts forward ways these challenges may
be overcome. The results of this dissertation also implicate that model-based release planning
research should focus on how release planning is performed in practice before proposing new
models and algorithms.
Keywords Scrum, release planning, agile software development, inter-team coordination,
large-scale agile, case study
ISBN (printed) 978-952-60-6045-3
ISBN (pdf) 978-952-60-6046-0
ISSN-L 1799-4934
ISSN (printed) 1799-4934
ISSN (pdf) 1799-4942
Location of publisher Helsinki
Pages 163
Location of printing Helsinki Year 2015
urn http://urn.ﬁ/URN:ISBN:978-952-60-6046-0
Tiivistelmä
Aalto-yliopisto, PL 11000, 00076 Aalto www.aalto.fi
Tekijä
Ville T. Heikkilä
Väitöskirjan nimi
Tapaustutkimuksia julkaisusuunnittelusta ketterissä ohjelmistokehitysorganisaatioissa
Julkaisija Perustieteiden korkeakoulu
Yksikkö Tietotekniikan laitos
Sarja Aalto University publication series DOCTORAL DISSERTATIONS 6/2015
Tutkimusala Ohjelmistotuotanto
Käsikirjoituksen pvm 13.05.2014
Julkaisuluvan myöntämispäivä 05.11.2014
Monografia
Väitöspäivä 23.01.2015
Kieli Englanti
Yhdistelmäväitöskirja (yhteenveto-osa + erillisartikkelit)
Tiivistelmä
Ohjelmistojen julkaisusuunnittelun tavoitteena on tarkoituksenmukaisen sisällön suunnittelu. Julkaisusuunnittelussa on otettava huomioon tarvittavat resurssit, ohjelmistovaatimusten prioriteetit ja erilaiset rajoitteet. Aiheen aikaisempi tutkimus on keskittynyt
algoritmisiin ja mallipohjaisiin ratkaisuihin, joissa vaatimusten valintaperusteiden ja
rajoitteiden pitää olla hyvin määritellyt. Todellisuudessa valintaperusteita ja rajoitteita
pystytään harvoin määrittelemään ennalta riittävän tarkasti. Nykyaikaiset ketterät
ohjelmistokehitysmenetelmät perustuvat nopeaan kehityssykliin ja jatkuvaan kommunikaatioon eri osapuolten välillä, eivät suunnitteluun ja suunnitelman seuraamiseen, kuten
mallipohjaiset menetelmät.
Tämä väitöskirja esittelee kolme tapaustutkimusta julkaisusuunnittelumenetelmistä
kahdessa isossa ja yhdessä pienessä ohjelmistokehitysorganisaatiossa. Tutkimuksen
tavoitteena oli tutkia, miten menetelmiä oli käytetty, miten ne vaikuttivat ohjelmistokehitysryhmätason suunnittelukäytäntöihin ja mitä haasteita ja hyötyjä menetelmiin liittyi.
Tutkimusmenetelminä käytettiin havainnointeja, haastatteluja, kyselyjä ja ohjelmistovaatimusten tilastollista analyysiä.
Tutkitut organisaatiot käyttivät erilaisia julkaisusuunnittelumenetelmiä. Agilefant-organisaatiossa käytettiin mallipohjaista menetelmää, jossa ohjelmiston sidosryhmiä pyydettiin
äänestämään vaatimusten tärkeysjärjestyksestä. F-Secure käytti menetelmää, jossa kaikki
sidosryhmiin kuuluvat kokoontuivat säännöllisesti suunnittelemaan seuraavaa julkaisua
yhdessä. Ericssonin menetelmä oli luonteeltaan jatkuva ja perustui vaatimusten vähittäiseen
tarkentamiseen. Ryhmätasolla suunnittelukäytännöt noudattivat pääosin Scrum-menetelmää.
Ensimmäinen haaste oli, ettei vaatimuksia kyetty tunnistamaan, valitsemaan ja tarkentamaan siten, että ne olisivat tukeneet julkaisusuunnittelua. Toinen haaste oli ketterien
ohjelmistokehitysmenetelmien käyttöönoton keskeneräisyys. Menetelmien hyötyjä olivat
useiden mielipiteiden sovittelu, tehokkuus, korkeatasoisten suunnitelmien luonti ja kehittäjien
parantunut motivaatio.
Tulokset osoittavat, että johtajien ei pitäisi tehdä julkaisusuunnittelua eristyksissä
ohjelmistokehittäjistä. Sen sijaan koko organisaation hiljainen tieto tulisi valjastaa
julkaisusuunnitteluun. Ohjelmistokehitysorganisaatiot voivat hyödyntää väitöskirjassa
tunnistettuihin haasteisiin esitettyjä ratkaisuja julkaisusuunnittelumenetelmiensä
parantamiseksi. Mallipohjaisen julkaisusuunnittelun tutkimuksen tulisi jatkossa keskittyä
siihen, miten julkaisusuunnittelua käytännössä tehdään.
Avainsanat Scrum, julkaisusuunnittelu, ketterä ohjelmistokehitys, pienryhmien koordinointi,
ketteryys isoissa organisaatioissa, tapaustutkimus
ISBN (painettu) 978-952-60-6045-3
ISBN (pdf) 978-952-60-6046-0
ISSN-L 1799-4934
ISSN (painettu) 1799-4934
ISSN (pdf) 1799-4942
Julkaisupaikka Helsinki
Sivumäärä 163
Painopaikka Helsinki
Vuosi 2015
urn http://urn.ﬁ/URN:ISBN:978-952-60-6046-0
Preface
The ﬁrst steps towards this dissertation were taken in 2008 although I did
not realize it at the time. When I was recruited for a summer job in the
ATMAN research project, I could never have predicted that six and a half
years later I would be publicly defending my doctoral dissertation. Between
now and then, I have had the opportunity to work with many intelligent
and interesting people who, in one way or another, have contributed to this
dissertation.
I would like to thank my supervisor, Professor Casper Lassenius, who
has guided my research towards a coherent topic and encouraged me to
ﬁnally put together this dissertation. I would like to thank my dissertation
instructor, Maria Paasivaara, for the help she has given me during the
writing of this dissertation. Anas Jadallah, Kristian Rautiainen, Günther Ruhe, Slinger Jansen, Christian Engblom, Towo Toivola and Janne
Järvinen contributed to the research and writing of the publications that
are included in this dissertation. It would not exist without them. I am
grateful to all the people from F-Secure and Ericsson who were interviewed
or provided data otherwise and to all the people who participated in the
Agilefant release planning case study. The preliminary examination of
this dissertation was done by Professor Richard Torkar and Professor Kari
Smolander. Their comments led to many improvements to the dissertation
and I wish to thank them for that.
The members of the Aalto University Software Process Research Group
have been the best co-workers a young researcher might hope for. Jarno
Vähäniitty, Juha Itkonen, Mika Mäntylä, Jari Vanhanen and Timo Lehtinen have all supported my work by countless discussions on the theory
and practice of conducting and publishing research.
This research was ﬁnancially supported by TEKES (the Finnish Funding
Agency for Innovation) and the participating companies in the ATMAN
1
Preface
research project, and by TEKES as a part of the Cloud Software Finland
and the Need for Speed research programs of DIGILE (Finnish Strategic
Centre for Science, Technology and Innovation in the ﬁeld of ICT and
digital business).
I ﬁnalized this dissertation while I was visiting the SEGAL research
group at the University of Victoria, BC, Canada. I would like to thank
Professor Daniela Damian for accepting me into her research group and
inspiring me to take on a new research challenge.
Finally, I wish to thank my spouse Maria for her support. She has been
very understanding and adaptable to the needs of my work, whether it has
been my odd working hours, travelling or long visits to foreign countries.
Espoo, December 18, 2014,
Ville T. Heikkilä
2
Contents
Preface
1
Contents
3
List of Publications
7
Author’s Contribution
9
1. Introduction
11
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
1.2 Research goals and questions . . . . . . . . . . . . . . . . . .
13
1.3 Scope limitations . . . . . . . . . . . . . . . . . . . . . . . . .
14
1.4 Structure of the dissertation . . . . . . . . . . . . . . . . . . .
15
2. Background and related work
17
2.1 Agile software development and Scrum . . . . . . . . . . . .
17
2.2 Scaling up agile development . . . . . . . . . . . . . . . . . .
19
2.3 Releases and release planning . . . . . . . . . . . . . . . . . .
22
2.4 Software product and project management . . . . . . . . . .
24
3. Research methods
27
3.1 Software engineering research methods . . . . . . . . . . . .
27
3.2 Case study approach . . . . . . . . . . . . . . . . . . . . . . .
29
3.3 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
3.3.1 Case Agilefant . . . . . . . . . . . . . . . . . . . . . . .
30
3.3.2 Case F-Secure . . . . . . . . . . . . . . . . . . . . . . .
31
3.3.3 Case Ericsson . . . . . . . . . . . . . . . . . . . . . . .
32
3.4 Qualitative data analysis . . . . . . . . . . . . . . . . . . . . .
33
3.5 Quantitative data analysis . . . . . . . . . . . . . . . . . . . .
36
4. Summary of the results
39
3
Contents
4.1 Overview and background of the cases . . . . . . . . . . . . .
39
4.1.1 Case Agilefant . . . . . . . . . . . . . . . . . . . . . . .
39
4.1.2 Case F-Secure . . . . . . . . . . . . . . . . . . . . . . .
40
4.1.3 Case Ericsson . . . . . . . . . . . . . . . . . . . . . . .
41
4.2 RQ1: How were the releases planned in the cases studied? .
42
4.2.1 SCERP . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
4.2.2 Release Iteration Planning . . . . . . . . . . . . . . . .
43
4.2.3 Continuous Release Planning . . . . . . . . . . . . . .
46
4.3 RQ2: What were the team-level planning processes, and how
were they affected by the release planning method? . . . . .
47
4.3.1 Case F-Secure . . . . . . . . . . . . . . . . . . . . . . .
48
4.3.2 Case Ericsson . . . . . . . . . . . . . . . . . . . . . . .
48
4.4 RQ3: What kinds of challenges were associated with the
release planning methods? . . . . . . . . . . . . . . . . . . . .
50
4.4.1 Effort spent on release planning . . . . . . . . . . . .
50
4.4.2 Selection of the candidate features . . . . . . . . . . .
50
4.4.3 Identifying feature prioritization criteria . . . . . . .
51
4.4.4 Preparation and deﬁnition of features . . . . . . . . .
51
4.4.5 Balancing between specialized and generalist teams .
52
4.4.6 Immature adoption of the agile mind-set . . . . . . .
52
4.4.7 Planning the product architecture . . . . . . . . . . .
53
4.5 RQ4: What kinds of beneﬁts have the release planning methods realized? . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
4.5.1 Mediation of multiple opinions and perspectives . . .
54
4.5.2 Efﬁciency of the planning method . . . . . . . . . . . .
55
4.5.3 Creation of high-quality plans . . . . . . . . . . . . . .
55
4.5.4 Increased motivation . . . . . . . . . . . . . . . . . . .
56
5. Discussion
5.1 Answers to research questions . . . . . . . . . . . . . . . . . .
5.1.1 How were the releases planned in the cases studied?
57
57
57
5.1.2 What were the team-level planning processes, and
how were they affected by the release planning method? 59
5.1.3 What kinds of challenges were associated with the
release planning methods? . . . . . . . . . . . . . . . .
60
5.1.4 What kinds of beneﬁts have the release planning
4
methods realized? . . . . . . . . . . . . . . . . . . . . .
60
5.2 Threats to validity and limitations . . . . . . . . . . . . . . .
62
Contents
5.2.1 Construct validity . . . . . . . . . . . . . . . . . . . . .
63
5.2.2 Internal validity . . . . . . . . . . . . . . . . . . . . . .
64
5.2.3 External validity . . . . . . . . . . . . . . . . . . . . .
65
5.2.4 Statistical-conclusion validity . . . . . . . . . . . . . .
65
5.2.5 Reliability . . . . . . . . . . . . . . . . . . . . . . . . .
66
5.2.6 Limitations . . . . . . . . . . . . . . . . . . . . . . . . .
66
6. Conclusions
69
6.1 Implications to practice . . . . . . . . . . . . . . . . . . . . . .
69
6.2 Implications to theory
. . . . . . . . . . . . . . . . . . . . . .
70
6.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . .
71
Bibliography
73
Publications
79
5
Contents
6
List of Publications
This thesis consists of an overview and of the following publications which
are referred to in the text by their Roman numerals.
I Ville Heikkilä, Anas Jadallah, Kristian Rautiainen, Günther Ruhe. Rigorous Support for Flexible Planning of Product Releases — A StakeholderCentric Approach and its Initial Evaluation. In Proceedings of the 43th
Hawaii International Conference on System Sciences, Kuai, Hawaii, 10
pages (digital proceedings), January 2010.
II Ville Heikkilä, Kristian Rautiainen, Slinger Jansen. A Revelatory Case
Study on Scaling Agile Release Planning. In Proceedings of the 36th
EUROMICRO Conference on Software Engineering and Advanced Applications, Lille, France, pp. 289–296, September 2010.
III Ville T. Heikkilä, Maria Paasivaara, Casper Lassenius, and Christian
Engblom. Continuous Release Planning in a Large-Scale Scrum Development Organization at Ericsson. In Proceedings of the Agile Processes in
Software Engineering and Extreme Programming, Vienna, Austria, Vol.
149 of Lecture Notes in Business Information Processing, Springer Berlin
Heidelberg, pp. 195–209, June 2013.
IV Ville T. Heikkilä, Maria Paasivaara, and Casper Lassenius. ScrumBut,
but Does It Matter? A Mixed-Method Study of the Planning Process of
a Multi-Team Scrum Organization. In Proceedings of the ACM / IEEE
International Symposium on Empirical Software Engineering and Measurement, Baltimore, Maryland, USA, pp. 85–94, October 2013.
V Ville T. Heikkilä, Maria Paasivaara, Kristian Rautiainen, Casper Lassenius, Towo Toivola, and Janne Järvinen. Operational Release Planning
in Large-Scale Scrum with Multiple Stakeholders — A Longitudinal Case
Study at F-Secure Corporation. Information and Software Technology, in
press, 25 pages, 2014.
7
List of Publications
8
Author’s Contribution
Publication I: “Rigorous Support for Flexible Planning of Product
Releases — A Stakeholder-Centric Approach and its Initial
Evaluation”
The authors of the publication planned the research collaboratively. The
author of this dissertation conducted the case study and analysed the
resulting data. The abstract, related work, introduction and discussion
were written by the authors of the article in collaboration. Case study
design and results were written by the author of this dissertation.
Publication II: “A Revelatory Case Study on Scaling Agile Release
Planning”
The ﬁrst two authors of the publication planned the research and collected
the data. The author of this dissertation wrote the ﬁrst draft of the publication. The text was ﬁnalized in collaboration with Kristian Rautiainen.
Publication III: “Continuous Release Planning in a Large-Scale
Scrum Development Organization at Ericsson”
The authors of the publication planned the research in collaboration. Data
was collaboratively collected by the ﬁrst three authors. The author of this
dissertation analysed the data and wrote the publication.
9
Author’s Contribution
Publication IV: “ScrumBut, but Does It Matter? A Mixed-Method
Study of the Planning Process of a Multi-Team Scrum Organization”
The research was planned by the authors of the publication in collaboration.
The data collection, analysis and writing of the publication were done by
the author of this dissertation.
Publication V: “Operational Release Planning in Large-Scale Scrum
with Multiple Stakeholders — A Longitudinal Case Study at
F-Secure Corporation”
The research was planned by the authors of the publication in collaboration.
The observation data was collaboratively collected by the author of this
dissertation and Kristian Rautiainen. The interviews were conducted
collaboratively by the author and Maria Paasivaara. Data analysis was
conducted by the author. The ﬁrst draft of Section 8 was written by Maria
Paasivaara. Section 8 was later partially re-written by the author. The
rest of the publication was written by the author.
10
1. Introduction
Our suggestion . . . is to give the role of (internal) contract giver for all development activities to Product Management as part of its product responsibility.
. . . The establishment of a such strict separation of contract giver and taker for
development is usually somewhat difﬁcult to start since the developers tend to see
this more as annoying control than as help whereas people on the business side
often feel forced into sharing responsibility for the results of the development.
—Kittlaus and Clough (2009, pages 94–95)
In general the software product manager is dependent on being able to predict
reliably if Development can meet the plan in terms of time and quality. Once
Development has written the project plan, the product manager is advised to
follow the progress of the project in comparison to this project plan closely . . .
—Kittlaus and Clough (2009, page 96)
As the snippets above illustrate, software product management literature
has traditionally seen the development as a separate organization that is
given a set of requirements and is expected to complete the requirements
according to a plan. Modern, agile software development methods take
a completely opposite approach to communication and planning. They
emphasize collaboration between all stakeholders and adapting to changes
instead of following a plan. Combining the business-driven need for predictability and plans with agile software development methods, especially
in large-scale development organizations, is challenging. This dissertation
presents the ﬁrst steps towards solutions to this challenge by describing
three cases of release planning in agile software development organizations.
11
Introduction
1.1
Motivation
Planning the next product release is recognized as a challenging part
of market-driven product development (Fogelström et al., 2010) and an
important success factor in agile software development projects (Chow and
Cao, 2008). The main goal of software release planning is to select an
appropriate scope for a release while taking into account constraints such
as the budget, resources, dependencies and technical aspects, and factors
such as the importance or urgency of the candidate requirements and the
availability of development resources (Carlshamre, 2002; Fogelström et al.,
2010; Ngo-The and Ruhe, 2008). Depending on the deﬁnition, release
planning often includes the most efﬁcient implementation order of the
candidate requirements (Svahnberg et al., 2010). Release planning is
especially vital in market-driven product development, as it puts the
strategy of the company in practice (Svahnberg et al., 2010).
Research on software release planning has focused on model-based release planning designed to create the most valuable, satisfying or risk-free
release plans using a mathematical optimization algorithm (Svahnberg
et al., 2010). The model or simulation is then used to generate one or
a set of optimal or near-optimal release plan(s). However, the empirical
validation of the proposed models is largely weak and the adoption rate of
model-based release planning in industry is unknown (Svahnberg et al.,
2010).
Release planning activities are sometimes called pre-project activities
(Fogelström et al., 2010), indicating that requirements gathering, prioritization and planning are performed before the development begins. In
this traditional, plan-driven approach, a product manager or a product
management ofﬁce is responsible for these activities. Traditional product
and release planning literature largely has not acknowledged the rise of
the modern, agile software development methods that are dependent on
collaboration between different stakeholders (Vähäniitty, 2011). Instead,
it suggests that strict division between responsibilities and detailed communication protocols should be established. In agile software development,
planning is an iterative and incremental activity that is conducted during
the project (Fogelström et al., 2010). Several release planning activities,
such as the market, customer and competitor analysis, are typically performed by product managers or product owners also in agile development
projects. However, most of the release planning activities are performed
12
Introduction
collaboratively with the development organization (Benestad and Hannay,
2011; Schwaber and Beedle, 2002). Disregarding model-based release planning research and practitioner literature, little empirical evidence exists
on how release planning can be conducted in large-scale agile development
organizations.
Considering the importance of release planning for the success of a
development project (Chow and Cao, 2008; Fogelström et al., 2010), the
dominance of the model-based release planning in research (Jantunen
et al., 2011; Lehtola et al., 2004; Svahnberg et al., 2010), and the lack of
solid empirical evidence of successful practices for scaling up agile release
planning, there is an obvious gap in the research of release planning in
modern, large-scale agile software development organizations. The overall
goal of the research presented in this dissertation is to take the ﬁrst
empirical steps to ﬁll that gap.
1.2
Research goals and questions
The overall research question that this dissertation aims to answer is
the following: How can releases be planned in modern, agile software
development organizations?
In order to answer this main research question, the following research
questions are answered in this dissertation:
RQ1: How were the releases planned in the cases studied?
RQ2: What were the team-level planning processes, and how were they
affected by the release planning method?
RQ3: What kinds of challenges were associated with the release planning
methods?
RQ4: What kinds of beneﬁts have the release planning methods realized?
The research presented in this dissertation is based on the pragmatist
epistemological tradition of empirical software engineering research (Petersen and Gencel, 2013; Runeson and Höst, 2009; Sjøberg et al., 2007;
Wohlin et al., 2003). Although both qualitative and quantitative methods
are applied, the goal is to disseminate the studied release planning methods from the realist standpoint instead of focusing on, for example, the
interpretation of the internal realities of the people affected by the method.
This dissertation consists of ﬁve publications and a summary. Table
1.1 summarises how the publications contribute to the overall research
13
Introduction
questions of this dissertation. Complimentary subjects are not necessarily
introduced in the research questions of their respective publications, but
information on the subject is provided in the results of the publication.
Table 1.1. Contribution of each publication to the research questions
Research question
Publication
RQ1
I
M
II
M
III
M
IV
V
RQ2
RQ3
RQ 4
c
c
c
c
c
c
M
M
M
M
M
M
c
Note. M = main subject, c = complimentary subject
1.3
Scope limitations
The term release planning does not have an universally accepted deﬁnition. For example, in a model proposed by van de Weerd et al. (2006)
release planning includes the following processes: requirements prioritization, requirements selection, release deﬁnition, release validation, scope
change management and launch preparation. However, according to Bays
(1999), launch preparations belong to the knowledge area of release management. Release management encompasses the practicalities of getting a
new release available to the market (Bays, 1999). These include the manufacturing of the physical distribution media, preparing the distribution
chain, preparing the customer support, preparing internationalization and
so on. Such practicalities are out of the scope of this dissertation.
Al-Emran and Pfahl (2007) and Fogelström et al. (2009) divide release
planning into strategic release planning and operational release planning.
In strategic release planning, the focus is on selecting the appropriate
requirements for the next public release of the product. In operational
release planning, the focus is on planning how the requirements for the
next release can be best implemented. Such strict separation of strategic and operational release planning does not exist in agile development
organizations (Fogelström et al., 2010).
This dissertation adopts a working deﬁnition of release planning that
encompasses both strategic and operational release planning. Thus, release
14
Introduction
planning is considered an activity where the tentative scope of the next
release is planned and the initial prioritization of requirements and the
tentative assignment of requirements to developers or development teams
is considered. The market-facing knowledge areas of product and project
management are out of the scope of this dissertation. These knowledge
areas include, but are not limited to, market analysis, competitor analysis,
product strategy, roadmapping, product pricing, marketing, recruitment
and project budgeting.
1.4
Structure of the dissertation
This dissertation consists of a summary and ﬁve research articles. These
articles describe three different release planning methods. The ﬁrst is
SCERP, which is a method based on an algorithmic optimization model
and on the votes of the stakeholders. The second method, Release Iteration Planning, is iterative and incremental and focusses on collaborative
planning events in which all stakeholders of the development project participate. The third method is Continuous Release Planning, which is
a continuous decision-making process based on stepwise elaboration of
requirements.
The rest of this summary is organized as follows. Chapter 2 provides
background and related work on agile software development on small scale
and large scale, on software product management and on release planning.
Chapter 3 describes the research methods employed in this dissertation.
Chapter 4 provides a summary of the results of this dissertation. Chapter
5 discusses the results of this dissertation and the threats to the validity of
the results. Chapter 6 concludes and provides directions for future work.
15
Introduction
16
2. Background and related work
The research disseminated in this dissertation covers multiple subject
areas, from software engineering methods to software product management. This section provides background information on these subjects. All
studied cases employed a development process inspired by the Scrum agile
software development method; general information on the history and
principles of agile software development methods and Scrum is described
below in Section 2.1. Although Scrum was originally aimed for small
teams, large organizations have begun to adopt Scrum practices. Two of
the case organizations studied in this research were large. Section 2.2
gives background information on how agile methods and Scrum have been
proposed to be scalable to large organizations. In the software engineering
discipline, release planning has been mostly studied as a mathematical
optimization problem. The deﬁnitions of release and release planning, the
mathematical approach and what little is written on release planning in
Scrum, is discussed in Section 2.3. Traditionally, a product management
organization has been considered responsible for release planning. This
subject and challenges related to it are discussed in Section 2.4.
2.1
Agile software development and Scrum
The agile software development movement began at the beginning of the
21st century when a group of researchers, consultants and practitioners
declared that the software development methods and processes at the time
focussed too much on formal contracts, documents, plans, processes and
tools (Beck et al., 2001; Dingsøyr et al., 2012; Larman, 2004). Although
there is no universal deﬁnition of agile software development methods
(Dingsøyr et al., 2012), in this dissertation the following working deﬁnition
inspired by Larman (2004) is adopted: Agile methods aim to quickly deliver
17
Background and related work
something of value to the customer, cultivate collaboration between the
stakeholders, value competence and technical excellence, and empower the
developers to make decisions on the development team level.
The agile movement produced several agile software development methods, which are sometimes alternatively known as agile software development life-cycle models. At the time of writing of this dissertation, the most
widely used agile software development method was Scrum (VersionOne,
Inc., 2013). Scrum was also used in all three cases described in this dissertation; other agile software development methods are omitted from this
section. Scrum is an iterative and incremental agile method originally
intended for small, independent software development teams (Schwaber
and Beedle, 2002; Schwaber and Sutherland, 2013). Software is developed
during two- to four-week long software development iterations, which are
called sprints.
In addition to software developers who are cross-functional and capable of
implementing and delivering any requirements imagined for the software,
Scrum deﬁnes two Scrum-speciﬁc roles: A Scrum master guides the team in
the correct implementation of the Scrum method, shields the development
team from external disruptions and in every way tries to facilitate and
improve the work of the Scrum team. A product owner is responsible for
creating and prioritizing the requirements for the software and for guiding
the team in the implementation of the requirements.
The requirements that are to be implemented in the software under
development are stored in a product backlog. Originally in Scrum, the
requirements stored in the product backlog were called backlog items
(Schwaber and Beedle, 2002). Backlog items describe end-to-end functionality that is visible and valuable to the end user. Later on, many Scrum
practitioners began to format requirements as user stores (Cohn, 2004), a
practice that was adopted from the eXtreme Programming agile software
development method (Beck et al., 2001). A requirement that follows the
user story formatting gives a short description of whom the requirement is
for, what it is expected to do and why it is needed. In Scrum, the product
owner has the sole authority to add new user stories to the backlog and to
prioritize the backlog.
Every Scrum sprint begins with a sprint planning meeting that is done
in two parts. In part one of the meeting the developers with help from the
product owner ﬁrst estimate the topmost (the high priority) user stories in
the backlog. The developers then select as many user stories from the top of
18
Background and related work
the backlog as they estimate they can develop during the following sprint.
These user stories form the basis of the sprint backlog of the following
sprint. In part two of the sprint planning meeting the developers split
the user stories in the sprint backlog into concrete development tasks and
estimate the tasks. The tasks are linked to the user stories to which they
are related and are added to the sprint backlog.
At the end of the sprint, the developers demonstrate the user stories that
they have completed during the sprint to the product owner in a sprint
review. All the user stories included in a sprint must be completed during
the sprint; otherwise, the sprint is considered a failure. The completed user
stories must be integrated, of publishable quality and work end-to-end.
The goal is that the software could be published for the end users at the
end of any sprint. A sprint retrospective is also conducted during the sprint
review. The goal of the retrospective is to identify problems, good practices
and improvement suggestions based on the preceding sprint.
2.2
Scaling up agile development
At the time of writing, large development organizations have begun to
adopt Scrum practices (VersionOne, Inc., 2013). The early Scrum literature
provided little guidance for strategic release planning, as the focus was on
planning and developing software one sprint at a time in a single team, single project context (Schwaber and Beedle, 2002). The proposed way to scale
up the size of a Scrum development organization is to employ multiple
small Scrum teams that simultaneously develop the same software system
(Schwaber, 2007). There are usually architectural complexities that result
in a network of dependencies between requirements (Benestad and Hannay, 2011; Carlshamre, 2002), which increases the difﬁculty of inter-team
coordination. Several problems related to agile development in the large
enterprise context have been identiﬁed by van Waardenburg and van Vliet
(2013). These included a multi-layered organization structure with strict
communication channels, diffused information, dependencies on external
parties, difﬁculties in developing end-to-end functionality, lack of business
involvement and requirements prioritization when many stakeholders are
involved and they have conﬂicting needs.
Many consultants and practitioners have proposed different kinds of
models for scaling agile development based on their experiences. Three
notably different models are described below to provide an overview of the
19
Background and related work
subject and to allow comparisons to the cases described in this dissertation.
Figure 2.1 illustrates the organization of the development in the three
models.
(a) Schwaber (2007)
(b) Larman and Vodde (2010)
(c) Lefﬁngwell (2011)
Figure 2.1. Organization of large-scale agile development by (a) Schwaber, (b) Larman
and Vodde, and (c) Lefﬁngwell
20
Background and related work
Schwaber (2007) suggests organizing the development using a tree structure. The structure has multiple levels of integration Scrum teams in
the branch nodes and (development) Scrum teams in the leaf nodes. The
integration Scrum teams do not develop functional software but instead
integrate, build and test the software implemented by the (development)
Scrum teams. Both kinds of Scrum teams have a dedicated product owner.
All requirements are listed in a product backlog as user stories. The branch
node product owners are responsible for assigning the sections of the product backlog for the lower level teams. Release planning is performed by
the root node product owner by selecting a subset of the product backlog
as the release product backlog.
Larman and Vodde (2010) propose a two-layer model for a large-scale
development organization. Development teams are arranged as feature
teams that work on a single feature at a time, and the team composition
persists over time. Feature teams are grouped into technical product areas.
Each product area is managed by an area product owner, who in turn is
managed by a product owner. The product owner manages the product
backlog and assigns backlog items to the product areas. Features are large
backlog items that describe functionality that is valuable for the customer.
Features are split into smaller backlog items that can be implemented
during a single sprint. Only the dates of the external releases are planned,
and the content of the release is deﬁned by what is ready at the time of
the release.
Lefﬁngwell (2011) proposes a model for the agile enterprise that consists
of three layers, which are the portfolio, the program and the team layer.
Each level has a corresponding backlog. The portfolio level is planned with
epics, which are "large-scale development initiatives" that span multiple
releases. Epics are split into features, which are planned on the program
level. Features are descriptions of the system behaviour that can be
developed in a single release project. Product management is responsible
for managing the program backlog, which contains the features. Features
are split into stories that can be implemented in a single development
iteration. The development teams are organized as independent teams
that each have a dedicated product owner. Release planning is performed
in release planning events where all stakeholders of the product assemble
to plan the next release together.
Although these models are purportedly based on experiences in real
software development organizations, the empirical validation of the models
21
Background and related work
is weak. As Figure 2.1 illustrates, the organization of the development
differs between the models considerably. Clearly more empirical research
on the organizations of the development and planning is required to reveal
what kinds of beneﬁts and challenges different kinds of agile scaling models
create.
2.3
Releases and release planning
Several different approaches to releasing software are employed in development organizations. The traditional way to approach software releases is
to divide the release into several incremental stages. The most commonly
used stages are the alpha test releases, the beta test releases and the
general availability release (Bays, 1999). Alpha test releases are created
to test the software internally or with a small set of external alpha testers.
Alpha test releases are not expected to be feature complete or free of faults,
but have enough functionality for the test users to test most of the features (Bays, 1999; Mäntylä and Itkonen, In Press). Beta test releases are
done for external users for implicit defect detection. Beta versions of the
software are expected to be feature complete and of high quality, and the
purpose of a beta test release is to test the software in larger variety of different environments than what is internally feasible (Bays, 1999; Mäntylä
and Itkonen, 2013, In Press). Patch and update releases, which correct
problems, and upgrade releases, which add or improve functionality, may
follow the general availability release (Bays, 1999).
The approach to software releases in Scrum does not follow the traditional
staged release model. Each sprint in Scrum is expected to end in a set
of new functionality that is of publishable quality and the product owner
may at any point decide to publish the software (Schwaber and Beedle,
2002). The ﬁrst release of the software in Scrum cannot be considered
a traditional alpha or beta, since it is expected to be of high quality but
it contains only a part of the functionality expected of the software. The
ﬁrst release is followed by further releases that incrementally add new
functionality. The purported beneﬁt from the Scrum release model is that
the development organization gets feedback from the users after each
incremental release and that the releases can be sold to customers earlier
than in the traditional model (Pries and Quigley, 2011).
The continuous delivery model has been proposed as the next evolutionary step from the iterative and incremental Scrum release model (Olsson
22
Background and related work
et al., 2012). In the continuous delivery model, a new version of the software is released and delivered to the users or customers immediately after
it has passed thorough an automated integration and testing process which
ensures that the released software is always of high quality. The purported
beneﬁts from this model are the continuous customer feedback and the
ability to eliminate work that does not produce value for the customer
(Olsson et al., 2012). However, this release model requires that there is a
baseline set of functionality which makes the software usable or valuable
to the customer before the continuous delivery of improvements can begin.
In a software development project, the release plans provide information
on how the goals of the project are reached on a time horizon of several
months (Cohn, 2005). The release plan uniﬁes the expectations about the
likely outcome and timeframe of the next release (Cohn, 2005). According to empirical release planning research, there are several technical
and human factors that make release planning a difﬁcult task. A shared
understanding about the requirements arises during development and
may be weak in the beginning of the release development project (Benestad and Hannay, 2011). The requirement selection criteria are time
dependent and may change both qualitatively and quantitatively during
the release development project (Benestad and Hannay, 2011). The great
majority of requirements have dependencies between them that constrain
the implementation order. These dependencies may be complex and difﬁcult to identify (Benestad and Hannay, 2011; Carlshamre et al., 2001).
Decision makers have difﬁculties expressing how value is created by selecting and prioritizing requirements (Barney et al., 2008) and "gut-feeling",
lobbying, politics, sell-in and strong individuals affect the requirements
prioritization in practice (Lindgren et al., 2008). The size of the client base
affects how the release planning of a product can be performed, and some
customers are strategically more important than others (Barney et al.,
2008). The business perspective is often considered the most important
requirement selection factor, which results in the implicit prioritization
of feature development over system improvement and innovation (Barney
et al., 2008; Fogelström et al., 2009; Lindgren et al., 2008).
The majority of published research on software release planning focusses
on different kinds of mathematical models and simulations that are designed to create the most valuable, satisfying or risk-free release plans
when the candidate requirements can be described in sufﬁcient detail and
key constraints and factors are known and can be estimated with sufﬁcient
23
Background and related work
accuracy (Svahnberg et al., 2010). The produced release plan is then expected to be precisely followed during the project execution. However, the
mathematical approach to release planning has proven to be problematic
in practice, since the above conditions are usually difﬁcult to fulﬁl in practice (Benestad and Hannay, 2011; Boehm, 2000; Cao and Ramesh, 2008;
Carlshamre, 2002; Jantunen et al., 2011).
The early Scrum literature (Schwaber and Beedle, 2002) described a
simple method for release planning in Scrum. The product owner simply
decides, based on either the release date or the intended release content,
which backlog items from the top of the backlog are included in the next
release. In the later Scrum literature, the discussion about release planning varies. Sometimes the subject is completely omitted (Schwaber and
Sutherland, 2013), and sometimes it is discussed as a Scrum extension
(Schwaber, 2007). Release planning has been recognized as an important
success factor in software projects (Chow and Cao, 2008; Fogelström et al.,
2010) and as a "wicked problem" that is difﬁcult to solve in a deterministic
way (Barney et al., 2008; Benestad and Hannay, 2011; Carlshamre et al.,
2001; Fogelström et al., 2009; Lindgren et al., 2008). Nevertheless, empirical research on release planning methods, especially in large-scale agile
development organizations, is scarce.
2.4
Software product and project management
In large software development organizations, there are multiple levels of
planning and management that are performed on different time horizons
and by different actors (Cohn, 2005; Lefﬁngwell, 2011; Rautiainen et al.,
2002). A product manager or a product management organization is an important actor in the management of a software product. A product manager
is a "mini CEO of a product" (Maglyas et al., 2012), and thus principally
responsible for the releases of a product. The role of product management
in a software development organization is not radically different from the
role in any other ﬁeld of business (Ebert, 2009), although Maglyas et al.
(2012) identiﬁed an ongoing confusion about what product management
actually encompasses. According to the traditional product management
literature (Condon (2002); Ferrari (2008); Kittlaus and Clough (2009)), a
product manager has the role of an internal contract giver to the development organization, and the success of the development organization is
measured by a separate quality assurance and veriﬁcation organization.
24
Background and related work
The literature views the communication link between the product manager
and the development as a mostly one-directional pipe where product speciﬁcations are put in and (hopefully) a product ﬁtting those speciﬁcations
comes out in a timely fashion.
Agile software development methods emphasize frequent, direct and
informal communication between the different stakeholders, including
the product management, project management and developers (Dingsøyr
et al., 2012). Thus, the traditional model, where the product management is strictly separated from the development organization, does not ﬁt
the agile development methods where product planning activities are not
strictly separated from the development organization (Fogelström et al.,
2010; Vähäniitty, 2011). Several product planning activities, such as the
market, customer and competitor analysis, are typically performed by the
product managers or product owners. However, many traditional product
and release planning activities, such as the prioritization of requirements,
implementation scheduling and cost/beneﬁt analysis, are performed collaboratively with the development organization (Benestad and Hannay,
2011; Schwaber and Beedle, 2002). Researchers, for example Heikkilä et al.
(2011), Vlaanderen et al. (2011) and Vähäniitty and Rautiainen (2008),
have proposed frameworks for product management in a rapid iterative
and incremental, or agile, software development organization. However,
the empirical evaluation of these frameworks, at the time of writing, was
somewhat lacking.
The traditional view is that software projects can be managed using the
same methods and tools as any other complex project. These have included
project schedules, requirements speciﬁcation documents, work breakdown
structure networks, activity (also known as Gantt) charts, resource allocation charts and myriad of other documents and charts (Pressman,
2010; Sommerville, 2001). The approach taken in agile software development methods, and particularly in Scrum, is to replace the elaborate
documents and detailed plans with the just-in-time elaboration of requirements, rolling wave planning and face-to-face communication (Pries and
Quigley, 2011; Schwaber, 2004). In the traditional, plan-driven project
model the release plan is embedded in the different documents that form
the project plan. Since such a detailed project plan is not created in agile projects, the information needs to be created and disseminated using
other methods. In a small Scrum project, the product owner can communicate the release plan verbally and by splitting a release backlog from
25
Background and related work
the product backlog (Schwaber and Beedle, 2002). Due to the complexities
related to agile development in large organizations, in that context the
simple verbal face-to-face communication of release plans is not sufﬁcient
(Benestad and Hannay, 2011; Carlshamre, 2002; van Waardenburg and
van Vliet, 2013). In large agile organizations, release planning and release
plans replace the formal project plan and detailed documents in providing
the development organization and the stakeholders the information about
the tentative schedule, resourcing and contents of the next release.
26
3. Research methods
In this chapter, the methodology employed in this dissertation is discussed.
First, the research methods employed in software engineering research in
general are described. Second, the case study approach employed in this
thesis is disseminated. Following that, the data collection performed in
each of the cases is described and the qualitative and quantitative data
analysis methods are explicated.
3.1
Software engineering research methods
Software engineering research can be divided into two research streams:
The creation or improvement of technology (processes, tools, models and
so on) for software engineering, and the evaluation and comparison of
the aforementioned technology (Sjøberg et al., 2007). The research in the
former stream is in general more theory-oriented and interested in the
theoretical power or validity of the proposed technologies, while the latter
stream is interested in the empirical, real-world efﬁciency, effectiveness
and applicability of technology. The research presented in this thesis is
based on the study of real-world organizations and is, subsequently, profusely empirical. This section presents an overview of empirical software
research methods and motivates the selection of the research method of
this thesis.
There are four main methods for empirical software engineering research
(Sjøberg et al., 2007): experimentation, surveys, case studies and action
research. Experimentation is performed in a controlled setting. Experimental laboratory research is sometimes referred to as research-in-the-small
due to the inherent limitations of the setting (Wohlin et al., 2003). The
goal of experimentation is to study cause-effect relationships by controlling
independent variables and studying the subsequent changes in dependent
27
Research methods
variables. Careful elimination or control of any variables that may affect
the dependent variables is required to establish cause-effect relationships.
Control of the variables is accomplished by controlling the setting while
the elimination of the effects of any uncontrolled (or unidentiﬁed) variables
is accomplished by random sampling. Quasi-experiments are otherwise
similar to experiments, but random sampling is not performed and more
attention must be given to the identiﬁcation of any and all variables that
signiﬁcantly affect the dependent variables. Experimentation is quantitative research by deﬁnition (Wohlin et al., 2003).
Surveys can be used to study a large number of variables using large
sample sizes (Sjøberg et al., 2007). Surveys can collect both qualitative
and quantitative data and are used to respond "how much" or "what kind
of" questions. Controlling the survey setting is not possible and the sample
of survey respondents is rarely truly random, as surveys are subject to
biases such as self-selection and coverage (Lavrakas, 2008). In addition, it
is difﬁcult to ensure that all survey respondents have sufﬁciently similar
understanding of the survey questions and divergent understanding can
cause measurement bias (Lavrakas, 2008).
Action research differs from the other research methods in that it has
the explicit goal of improving some real-life situation (Sjøberg et al., 2007).
Many different forms of action research exist, but the most common form
used in empirical software engineering research is known as the canonical
action research process (Davison et al., 2012). Canonical action research is
based on the cycle of diagnosis, action planning, intervention, evaluation
and reﬂection. All activities in the cycle should be based on a theory (or
a hypothesis) that informs and guides the activities. Action research has
been criticized based on the lack of objectivity (Sjøberg et al., 2007) due to
its goal of improvement. Although action research is concerned with causes
(via interventions) and effects (via evaluations), its real-life setting makes
it usually very difﬁcult to control or eliminate confounding variables and
assert cause-effect relationships.
The research method employed in the research disseminated in this
dissertation is the case study method (Sjøberg et al., 2007; Yin, 2009).
The method is used to study phenomena in their real-life context. Case
study aims to answer a "how" or "why" question. The case study method is
the most appropriate choice when a contemporary event over which the
researcher has no control is studied. Both qualitative and quantitative
data can be collected and analysed in case study research.
28
Research methods
The case study method was selected due to the difﬁculty of studying largescale agile software development with any of the other methods, and due to
its power to disseminate new and interesting phenomena. Experimental or
quasi-experimental design was not possible due to the nature of the studied
release planning methods. Designs with randomized sampling and control
groups were not feasible because the unit of analysis was the development
organization as a whole. Small-scale experiments with, for example, groups
of students would have been feasible, but the generalizability of such
experiments to real-world large development organizations is questionable.
Due to the lack of a solid theory or even a generally accepted deﬁnition for
large-scale agile release planning, surveys regarding it were considered
premature. The difference between action research and research presented
in Publication V was that in Publication V the goal was not the theorybased improvement of the current situation by intervention. Instead, the
aim was to study the interesting phenomena in the cases as they unwound
(Yin, 2009). Although feedback reports were provided to the company, it
would have been impossible to separate the effects of the feedback reports
from the effects caused by changes independently instigated by the case
organization. Since the theory-based analysis of cause-effect relationships
is one of the main goals of canonical action research, the research presented
in Publication V cannot be considered canonical action research. Other
forms of action research mostly focus on sociology and have explicit social
goals such as self-emancipation, equality and empowerment that have
little to do with the research goals of this dissertation (Davison et al., 2004;
Isador et al., 1948).
The case study method is well suited for studying new and atypical
phenomena, and such studies are some times called revelatory case studies.
The case study method does not require an existing theory or hypothesis
to be applicable, and case studies can be theoretically generalized, which
means that large samples required by statistical generalization are not
necessary (Yin, 2009).
3.2
Case study approach
The publications that belong to this thesis present three case studies of
three different release planning methods in three organizations. Table 3.1
shows the cases, release planning methods and the publications related to
each of the three.
29
Research methods
Table 3.1. Cases, release planning methods and contributing publications
Case
Release planning method
Publication(s)
Agilefant
SCERP
I
F-Secure
Release Iteration Planning
II, V
Ericsson
Continuous Release Planning
III, IV
In Case Agilefant, the researchers initiated the case study by suggesting
that the SCERP method could be applied to the release planning of Agilefant. However, the software, the stakeholders and the candidate features
were real and contemporary, and the researchers did not affect the SCERP
process during the case study. Following the deﬁnition presented by Yin
(2009), the studies of Case Ericsson and Case F-Secure were revelatory
case studies. They presented the ﬁrst empirical studies of new phenomena.
In Case F-Secure and Case Ericsson, the researchers identiﬁed interesting contemporary phenomena and decided to study them. However, the
researchers in both of these cases were not merely neutral observers, as
they provided reports of their observations for the case organizations.
3.3
Data collection
This section gives an overview of the data collection in the cases presented
in this dissertation. Table 3.2 summarizes the collected data. Below, the
data collection is described case by case. For more details on the data
collection, see the respective publications.
Table 3.2. Data collected in the studied cases
Case
Collected data
Agilefant
survey responses, SCERP votes
F-Secure
voice recordings, interviews, survey responses, ﬁeld notes
Ericsson
interviews, user stories∗
Note. ∗ Refers to requirements in the user story format
3.3.1
Case Agilefant
In Case Agilefant, the goal of the case study was the preliminary evaluation of the SCERP method introduced in Publication I. The evaluation
was done by applying the method for the release planning of the Agilefant
30
Research methods
(Vähäniitty and Rautiainen, 2008) software development management
tool. Agilefant users were invited to participate in the release planning
as stakeholders by an open invitation on the Web page of Agilefant. A
Web-based release planning tool was employed to gather the stakeholders’
votes and for the release plan generation. In addition to contributing to the
release planning method, the stakeholders were asked to ﬁll out a survey
concerning the SCERP method. The survey asked the stakeholders to evaluate their understanding of the features and the release planning method,
and to evaluate the release planning method and the list of proposed features. The survey also asked the respondents to estimate the time spent
on the release planning and to provide general feedback on the method.
Nineteen stakeholders voted on the priorities of the candidate features
and responded to the survey. Ten stakeholders voted on the acceptability
of the release plan alternatives generated in SCERP.
3.3.2
Case F-Secure
Case F-Secure was a longitudinal case study (Yin, 2009) of the Release
Iteration Planning method. The case study was conducted by the author of
this dissertation with the other authors of Publication II and Publication
V. For Publication II, we observed two Release Iteration Planning events
(which are called joint release planning events in the publication). During
the events, we made observation notes and recorded presentations and
discussions using voice recorders. After both events, we collaboratively
constructed a short description of the event and the most important observations in a case diary. We also conducted a survey after the ﬁrst two
Release Iteration Planning events. All participants of both events were
invited to answer the survey. The ﬁrst survey was sent to 140 participants
and was answered by 33 respondents, and the second survey was sent
to 136 participants and was answered by 26 respondents. The survey
contained statements that the respondents were asked to evaluate using
a six-point Likert-like scale. An open text ﬁeld was also provided next to
each statement for free text feedback. In addition, the survey contained
questions about the demographic of the respondent and a request to grade
the event on a six-point scale.
For Publication V, we collected data additional to Publication II and
analysed the data more rigorously. We observed three additional Release
Iteration Planning events where we wrote ﬁeld notes and recorded presentations and discussions, and after the events we updated the case diary. In
31
Research methods
Table 3.3. Overview of the data collection in Case F-Secure
Data source
Data collected
Release planning events
Event 1
voice recordings (13h 19min), notes
Event 2
voice recordings (7h 34min), notes
Event 3
voice recordings (5h 28min), notes
Event 4
voice recordings (5h 5min), notes
Event 5
voice recordings (3h 41min), notes
Retrospectives
Retrospective 1
voice recording (1h)
Retrospective 2
voice recordings (56min)
Retrospective 3
voice recordings (4h 45min)
Retrospective 4
ﬁeld notes
Surveys
Event 1 survey
33 responses
Event 2 survey
26 responses
Post-project interviews
Product owner
voice recording (1h 22min)
Scrum master/Facilitator
voice recording (1h)
SPI manager
voice recording (1h)
Product manager A
voice recording (41min)
Product manager B
voice recording (58min)
R&D Line manager
voice recording (48 min)
addition, we observed four retrospectives conducted by the case organization. In the ﬁrst three retrospectives, we recorded discussions and wrote
ﬁeld notes. In the fourth retrospective we were only allowed to write ﬁeld
notes due to conﬁdentiality reasons. After the case project, we conducted
ﬁve post-project interviews. Three of these interviews also covered a second project that had adopted the Release Iteration Planning method. The
research on the second project was based on those three interviews. Table
3.3 presents the data collected in Case F-Secure in detail.
3.3.3
Case Ericsson
Case Ericsson was a single case study of the Continuous Release Planning
method employed by the case organization and of the team-level planning
practices employed by the case organization. The case study was conducted
by the author of this dissertation with the other authors of Publication III
and Publication IV. We collected the data for Publication III by conducting
three rounds of interviews. We conducted the ﬁrst and second rounds
32
Research methods
in the spring of 2011. We selected the ﬁrst-round interviewees based on
suggestions from case organization representatives to get a good overview
of the organization. We ﬁrst interviewed eight people in Finland who were
our key informants (Yin, 2009). They provided us with an overview of the
organization’s history, goals, growth and structure as well as the planning
process used in the organization.
To build deeper understanding about the case organization and to enable
the triangulation of data sources (Patton, 2002), we performed a second
round of interviews. During the second round, we interviewed twenty
people in Finland and eleven people in Hungary. Our goal was to interview
multiple people with the same role in the organization, if available, in
order to reduce personal bias. After a careful analysis of the results from
the ﬁrst two rounds of interviews, we found that we needed to deepen
our understanding of the team-level practices in the case organization
and we performed one additional pair-interview in the spring of 2013. We
interviewed a product owner and a process improvement manager. The
interview focussed on the development teams’ planning processes.
In the Case Ericsson interviews, we employed the general interview guide
(also known as semi-structured interview) approach (Patton, 2002) in order
to maintain adaptability to the roles and individual experiences of the
interviewees while making sure that the relevant topics were explored. We
updated the interview guide constantly based on new insights from the
previous interviews (Patton, 2002). The interview questions were based on
the role of the interviewee and on the subjects we wanted to know more
about. All interviews were voice-recorded.
We collected the quantitative data employed in Publication IV from the
backlog management tool of the case organization. We collected 435 user
stories that had been modiﬁed or created between August 2011 and November 2012. In addition, we received information on the type of work and the
site of the teams that had worked on the user stories. The qualitative data
collected in Case Ericsson were also employed in Publication IV. Table 3.4
presents the data collected in Case Ericsson in detail.
3.4
Qualitative data analysis
In Case Agilefant, the qualitative survey answers were simply read and
any ﬁndings were written out to the case description. In Case F-Secure
and Case Ericsson, we had multiple qualitative data sources which were
33
Research methods
Table 3.4. Overview of the data collection in Case Ericsson
Data source
Data collected
Interviewees
Voice recordings (per interview)
Middle and upper managers (7)
2-3h
Agile coach (1)
2h
Scrum Masters (6)
1-2h
Developers (13)
1-2h
Line managers (3)
2-3h
Product owners (8)
1-2h
Technical specialists / architects (5)
1-2h
Backlog data
Information
User stories (435)
effort estimate, date opened,
date closed, assigned team(s)
employed in the analyses in different ways. The voice recordings were
transcribed and imported into the qualitative analysis program Atlas.ti.
We analysed the transcribed recordings using a qualitative data analysis
method inspired by the classical Grounded Theory (GT) method (Adolph
et al., 2011; Glaser, 2007).
The GT approach was adopted in these cases due to the difﬁculties of
identifying an existing theory or analytical framework that could be used
in the analysis of the data. Initially, we tried to analyse the data using
existing theories on what needs to be considered when release planning is
conducted (Svahnberg et al., 2010). The existing theories were focussed
on mathematical modelling and optimization algorithms. They proposed
different prioritization criteria, constraints, models and algorithms for
release planning. However, when we started to analyse the data using
these theories, we quickly discovered that the interviewees did not see the
release planning process as an algorithmic optimization problem and we
realized that the analysis of the data using the existing theories would not
be fruitful. Since the existing theories were not useful in our analysis, we
turned to the Grounded Theory method.
Our research process in Case F-Secure and Case Ericsson deviated from
the classical GT method in two crucial ways. Subsequently, we characterise
the analysis method as GT-inspired. First, we had quite a lot of pre-existing
knowledge regarding the subjects we wanted to cover in the interviews.
This was because we had previously conducted research on a similar
subject, which had inevitably included reviewing the related literature.
Thus, we did not conduct the interviews with our minds blank of knowledge
34
Research methods
of the speciﬁc subject area of interest, as is required by the classical
GT (Adolph et al., 2011). The second deviation was the use of a voice
recorder to record the interviews in their entirety. We did not want to rely
purely on post-interview memos and on notes made during the interviews
(Adolph et al., 2011). The research tradition in software engineering
research expects that a chain of evidence is established when research
is disseminated (Runeson and Höst, 2009). The chain of evidence should
show concretely how the presented conclusions and results are derived
from the data. In theory, the chain of evidence enables other researchers to
assess the validity of the research by repeating the study. Studies can be
repeated on many levels. For example, other researchers can conduct new
interviews with the same interviewees as in the original study, conduct new
interviews with different interviewees in the same organizational context,
or re-analyse the data from the original interviews. Thus, the recording
of interviews is crucial for the validity and repeatability of research, and
research based purely on informal memos would be considered of low
quality or even unacceptable due to the lack of a rigorous chain of evidence.
Due to the reasons described above, we did not follow the classical GT
method to the letter. Instead, we adopted an abductive (Patton, 2002)
coding approach: We employed our pre-existing knowledge in the analysis,
but we also adapted to new interesting topics we identiﬁed in the data.
In practice, the transcribed data were read, and codes were assigned to a
section of text when that section contained something we were interested
in.
In Case Ericsson, our approach was close to the classical GT approach.
We began the coding of the interview data without a pre-deﬁned list of codes.
Based on our pre-existing knowledge of processes and release planning,
we identiﬁed passages of text that were of interest and coded the passages
with one or more concept(s) that described the topic of the passage. At the
beginning of the coding process, new concepts were frequently added to the
code list. For example, the concepts "technical specialist" and "user story"
were identiﬁed in the process. As the coding progressed, we employed
the constant comparison technique (Adolph et al., 2011) to validate our
concepts: Interesting passages of text were compared with the concepts we
had previously identiﬁed to see if the new passages supported the concepts.
The concepts were then kept, changed, combined, split or abandoned based
on the new passages. We also identiﬁed common themes in the concepts
and arranged the concepts in thematic groups, which were validated using
35
Research methods
the constant comparison technique. For example, the themes "stakeholder"
and "work item" were identiﬁed during the analysis.
In Case F-Secure, our approach was more straightforward. We prepopulated the list of codes with concepts that were related to the events
and the topics we were interested in. For example, these included the
concepts "good in the release planning method" and "development teams".
The list of codes was augmented with additional concepts when interesting
topics we identiﬁed in the data did not ﬁt the existing set of codes. These
additional codes included the concepts "why the method was adopted" and
"dependencies".
Finally, the qualitative descriptions of Case F-Secure and Case Ericsson
were written based on all the available data. In Case F-Secure, this included the ﬁeld notes, survey results, feedback reports and the case diary
in addition to the transcribed and analysed recordings. In Case Ericsson,
the analysis was based on the transcribed and analysed interviews. In
both cases, the resulting publications were reviewed by several authoritative and experienced case organization representatives to ensure that
the results did not include factual errors. Although some minor factual
errors were identiﬁed and corrected, the reviewers considered the case
descriptions to give a fair account of the studied cases.
3.5
Quantitative data analysis
The purpose of the quantitative survey data collected in the case studies
was to provide additional insights into the qualitative data and to increase
the validity of the results with triangulation of data sources (Patton, 2002;
Yin, 2009). In Case Agilefant, we applied simple descriptive data analysis
to the survey results. We calculated the median and standard deviation of
the results of each survey question. The same approach was applied to the
analysis of the quantitative output of the SCERP release planning method.
The survey results in Case F-Secure were analysed descriptively by calculating the median and the ﬁrst and third quartiles and by identifying the
lowest and highest answers that were not outliers.
The focus of Publication IV was the quantitative backlog data collected
from Case Ericsson. We ﬁrst eliminated outliers from the quantitative
data based on the qualitative results from the case. These included backlog
items with properties outside the range deﬁned by the case organization
and special cases. The analysis of the remaining valid backlog items
36
Research methods
was conducted using descriptive analysis, Wilcoxon Signed-Rank Test,
Kandall’s tau-b correlation coefﬁcient and linear regression analysis.
37
Research methods
38
4. Summary of the results
This section summarizes the results of the research. The details of the results of each individual article can be found in the respective articles. This
section is structured according to the overall research questions presented
in Section 1.2. This section begins with an overview and background of the
cases studied. Finally, the results from each case are detailed in relation
to each overall research question of this dissertation.
4.1
Overview and background of the cases
Background information on the cases studied is presented in this section.
In Case Agilefant, the organization and background were mostly out of the
scope of the original case study (Publication I), as the focus of the research
was in the description and evaluation of the SCERP method. Regarding
Case F-Secure and Case Ericsson, this section describes the histories of
the case organizations, the organization of the stakeholders in the cases,
and the requirements management processes employed in the cases.
4.1.1
Case Agilefant
Case Agilefant was a small-scale, illustrative case study of the release
planning of Agilefant (Vähäniitty and Rautiainen, 2008), a Scrum backlog
management tool. Agilefant was a non-commercial open-source software
program which was developed using a process partially based on Scrum.
At the time of the case study, two part-time developers were developing
the software and the development was directed by one product owner.
39
Summary of the results
Figure 4.1. Project organization during the ﬁrst four Release Iteration Planning events in
Case F-Secure1
4.1.2
Case F-Secure
In 2009, the F-Secure corporation employed more than 800 employees and
had subsidiaries around the world. In addition to traditional PC computer
security software, its offerings included mobile security and data security
software. F-Secure began the adoption of the Scrum development life-cycle
model in 2006. The goal was to improve the efﬁciency and decrease the lead
time of software development. When the study of Case F-Secure began in
2009, the different parts of the organization were still in different stages of
adopting Scrum. Some parts had employed Scrum for several years, while
some parts were still working using traditional, plan-driven processes.
We studied two projects in Case F-Secure. The ﬁrst project was the main
focus of the study. The goal of the project was to develop a new version of
an existing product that belonged to an existing product family. Preceding
the project, the back-end and front-end of the product had been developed
in separate organizations. In the project, the development organizations
were combined, and back-end and front-end Scrum teams were created
from the members.
The project organization of the ﬁrst project had deﬁned a four-level hierarchical model of requirements management in the project. Epics formed
the high-level goals of the product for the multi-release time horizon. The
epics were split into more concrete features that described the requirements for a whole release project. Features were expected to encompass
functionality that would create concrete value for the customer or user.
1 Figure originally published in Publication V
40
Summary of the results
Features, in turn, were split into user stories.
Figure 4.1 illustrates the organization of the project at the beginning of
the ﬁrst studied project. The product management team was responsible
for creating the features and epics and prioritizing them. It was supported
by the product architecture team, the user experience team and the engine
development team representatives. At the beginning of the project, there
were nine Finnish development teams and one development team from the
Malaysian development site of the company.
4.1.3
Case Ericsson
Case Ericsson was based on a study of an Ericsson node development unit.
The unit developed a large systems product consisting of both software
and hardware. The development of this product had started over ten
years before the case study, and at the time of the study it was used by
operators all over the world, while the further development of the product
still continued. The case organization had begun the transformation from a
waterfall-like plan-driven process to an agile process in 2009. By 2011, the
whole development organization had adopted a Scrum-based development
process. Before the transformation, the development had been arranged as
a traditional plan-driven project organization where release planning had
begun two years before the release date when the scope of the next release
was decided by the product management.
The case organization in Case Ericsson used a four-level hierarchy of
backlog items to manage requirements. The highest abstraction level work
items were features. Features described on a high level what needed to
be developed. Epics were created based on the features. Epics were subfeatures that could be developed mostly independently of each other. Epics
were split into more concrete user stories. Finally, development tasks were
created based on the user stories.
The product owners formed their own team, which was led by a chief
product owner (chief PO). The product owner team was jointly responsible
for managing the development of the software. The long-term development
was planned by a product manager. The development teams in the case organization consisted of ﬁve to seven developers on each team. The product
owners were not tied to the teams. Instead, one or more product owners
and teams formed a temporary sub-organization around each epic that
was taken under development. The number of teams and product owners
in the sub-organization depended on the size of the epic and the state of
41
Summary of the results
Figure 4.2. Project organization in Case Ericsson2
its development. Figure 4.2 illustrates the development organization in
Case Ericsson.
There were several stakeholders who did not belong to the development
organization. A single product manager (PM) was responsible for the
software part of the product. The PM mostly communicated with the chief
PO. In addition, there was an early phases program manager who was
responsible for managing the early phases of the release planning process.
The product management and the development organization were assisted
by technical specialists, who were people with extensive knowledge of
telecommunications technology.
4.2
RQ1: How were the releases planned in the cases studied?
This section presents the three release planning methods that form the
basis of this dissertation. Case Agilefant presents an algorithmic release
planning method called SCERP and the preliminary evaluation of the
method in the release planning of Agilefant. Case F-Secure presents an
iterative and incremental release planning method, called the Release
Iteration Planning method, employed by F-Secure. Case Ericsson presents
the Continuous Release Planning method employed by Ericsson in Finland
and in Hungary.
2 Figure originally published in Publication III
42
Summary of the results
4.2.1
SCERP
The SCERP release planning method described in Case Agilefant was
based on algorithmic optimization. The priority or importance of the candidate features was deﬁned by a set of stakeholders who voted on how
important the features were based on one criterion or several. Each stakeholder could be assigned a relative weight which reﬂected the stakeholder’s
importance. In SCERP, the development of each feature required deﬁned
amount of one or more resource types. A set amount of each resource was
available during each of the release iterations that were planned. In the
preliminary evaluation of SCERP, the only resource considered was the
time of software developers. In SCERP, each release was given a relative
weight that reﬂected the cost of delay regarding the features that would
come out in the later releases. Based on the prioritization of the features,
stakeholders and releases, as well as on resource availability and consumption, the SCERP method employed an optimization algorithm that
produced several alternative optimal or near-optimal release plans. In the
ﬁnal step of the SCERP method, the stakeholders voted on which plan
they preferred among the several alternatives created by the optimization
algorithm, and the product manager selected the best plan.
In Case Agilefant, the product owner of Agilefant selected 37 candidate
features. The selected prioritization criteria were the value ("What is the
perceived (relative) value of the feature for you?"), the urgency ("How urgently do you need the feature?") and the dissatisfaction ("How dissatisﬁed
you would be if the feature becomes postponed?") of the features. Nineteen
stakeholders voted on the priority of the features. All stakeholders and
criteria were considered equally important. The implementation effort
(in work days) of each feature was estimated by two Agilefant developers.
Two releases were planned using the method. Each had a resource constraint of 22 development days. The release planning tool was employed
to produce ﬁve alternative release plans. In addition, the product owner
independently created a manual release plan. Finally, ten stakeholders
voted on how well each of the release plans matched their needs using a
Web-based tool.
4.2.2
Release Iteration Planning
The Release Iteration Planning method described in Case F-Secure was an
iterative and incremental release planning method based on collaboration
43
Summary of the results
Figure 4.3. An example schedule of a Release Iteration Planning event3
between all stakeholders of the development project. The development
project was divided into release iterations which each lasted for two to four
months. Each intermediate release iteration was planned to end with the
release of a test version of the software. The test versions were expected
to be of releasable quality and contain complete features. The goal was to
gather feedback from the real users and customers of the software. The
ﬁnal release iteration was planned to end in the release of the complete
version of the software. Several improvements were made to the Release
Iteration Planning method in Case F-Secure, and what is described here
was the latest version of the Release Iteration Planning method. For
details on how the method was changed over time, see Publication II and
Publication V.
The focal points of the method were the Release Iteration Planning
events. Each release iteration began with a Release Iteration Planning
event where the whole project organization and relevant stakeholders
gathered in the same space for two to three days. Before each event, the
participants prepared for the event. The product owners and product
managers prepared the features that were expected to be implemented in
the following release iteration. They decided which features were included,
prioritized the features and elaborated the features in such a way the
developers could understand them. They also decided on the tentative
assignment of features to the development teams. The development teams
made sure that they had updated their development progress to the product
backlog. Other stakeholders, such as software architects and usability
experts, made sure their materials and guidelines were up to date. To
illustrate how the events proceeded, Figure 4.3 shows the schedule of the
fourth Release Iteration Planning event we observed in Case F-Secure.
The events began with several presentations. The presentations provided
information on what had been accomplished during the previous release
iteration, described the overall goals of the following release iteration
3 Figure adopted from Publication V
44
Summary of the results
and gave practical guidance on the schedule of the event and on the
planning practices that should be used. If required, technical information
could be presented. This included, among others, software architecture
guidelines, usability guidance and practical development instructions. The
presentations were given at the beginning of the ﬁrst planning day or the
day preceding the ﬁrst planning day.
After the presentations, the developers of each team got together with
their product owner to decide what features the team(s) would start to plan.
The product managers were also present and available in case the teams
needed more information about the features. After the features had been
assigned, the teams broke out to their assigned planning areas; hence this
part of the events was called the planning breakout. With guidance from
their product owner and, if required, from the product managers, other
stakeholders and other teams, the teams started to split their features into
user stories. The user stories were then estimated and scheduled into the
development sprints of the next release iteration.
During the planning, two kinds of status check meetings were conducted,
alternating hourly. The ﬁrst kind was a planning progress status check
meeting. A representative from each development team, the product owners and the product managers participated in these meetings. Each team
representative reported what the team had planned since the previous
status check, what they were going to plan next, and if they had any problems and issues that needed to be solved. After the meeting, a suitable
set of people stayed together to solve the problems and issues that were
raised. The second kind of status check meeting was an architectural
status check. The status check was led by the lead architect, and a representative from each development team participated. The focus was on
handling software-architecture-related risks, dependencies and problems.
At the end of the last planning day, a plan review was conducted. Each
team presented to the other participants what it had planned and what
issues and risks it had identiﬁed. After all teams had presented their
plans, the participants together solved the issues and risks bought up by
the teams. Finally, the product managers and product owners recorded
the created release plans, and the teams transported their release plans to
their regular team spaces at their ofﬁce.
45
Summary of the results
Figure 4.4. The Continuous Release Planning method4
4.2.3
Continuous Release Planning
The Continuous Release Planning method employed in Case Ericsson was
quite different from SCERP and Release Iteration Planning. Instead of
aiming to identify a set of features that were to be developed during the
next release, the Continuous Release Planning method was a process that
continuously supplied the development organization with features. The
decisions were based on the availability of development resources and on
gradually elaborated feature information. Although the dates of upcoming
releases were planned approximately two years ahead of the release date,
the contents of each release were based on the features that were ready in
time for the release.
The Continuous Release Planning method consisted of ﬁve feature decisions (called F0-F4 decisions hereafter) and of two artefacts that were
created during the release planning process. The ﬁrst three feature decisions were made during the early phases of feature development when
no concrete implementation was done yet. The decisions during the early
phases were made by the portfolio steering group, which consisted of all
stakeholders of the development organization. The last two decisions were
made by the development steering group, which consisted of the chief PO
and the product manager. Other stakeholders could also participate in
the development steering group meetings when deemed necessary. Both
steering groups met once a week, and any number of feature decisions
could be made in a single meeting. Figure 4.4 illustrates the Continuous
4 Figure originally published in Publication III
46
Summary of the results
Release Planning method.
The release planning process began with the F0 decision. The product
manager and the early phases program manager had an idea of a feature
that could be implemented. The early phases program manager presented
the idea to the portfolio steering group, which decided to abandon it,
postpone it or advance it. In the latter case, the writing of a one pager
was begun. The one pager was created by a technical specialist and gave a
description of what the feature would be and why it was needed on a high
abstraction level. After a maximum writing time of two weeks, the chief
PO presented the one pager to the portfolio steering group. The steering
group could again decide to abandon, postpone or advance the feature idea.
In the latter case, a virtual team was created out of developers from several
development teams and a product owner. They had two weeks to write
a feature concept study, which provided the information needed to decide
whether the feature should be implemented or abandoned. The chief PO
presented the completed feature concept study in a portfolio steering group
meeting. The ﬁnal feature decision made by the steering group was either
to abandon the feature, to postpone it or to put it into implementation. In
the latter case, the development teams that had participated in the feature
concept study began implementing the feature.
When the chief PO considered the implementation of the feature to be
close to completion, an F3 decision was proposed in a development steering
group meeting. An F3 decision required the chief PO to commit the feature
to a release of the software. If the decision was positive, the marketing of
the feature could begin. After the feature was implemented and tested, the
chief PO proposed an F4 decision for that feature. A positive F4 decision
meant that the feature was ready to be included in a public release of the
software.
4.3
RQ2: What were the team-level planning processes, and how
were they affected by the release planning method?
This section summarises the results regarding the team-level planning
practices in Case F-Secure and Case Ericsson and how the release planning
methods employed in the cases affected the team-level planning practices.
This topic was not studied in Case Agilefant, and subsequently that case
will not be included in this section.
47
Summary of the results
4.3.1
Case F-Secure
In Case F-Secure, the development teams used Scrum as their teamlevel development life-cycle model. During the Release Iteration Planning
events, they planned the sprints of the following release iteration on the
user story level. During the release iteration, the plans of the ﬁrst sprint
made in the Release Iteration Planning event were directly used as the
Scrum sprint plan. During the later sprints, the plans were used as
guidelines for the Scrum sprint planning. The release plans also showed
what dependencies the teams had with each other and when the teams
were planning to initiate or ﬁnish developing the dependent features,
which helped in the dependency management.
4.3.2
Case Ericsson
In Case Ericsson, the development teams followed a planning process
inspired by Scrum. The teams had biweekly sprint planning meetings.
Before each sprint planning meeting, in a sprint review, the teams demonstrated to their product owner what they had accomplished during the
previous sprint. In the sprint planning meetings, the teams planned which
user stories they were going to implement during the following sprint. The
ofﬁcial goal was to get everything included in a sprint completed during
the sprint. However, Case Ericsson revealed that there were discrepancies
between the Scrum process and the teams’ actual conduct.
According to Scrum, new user stories can only be set in progress in a
sprint planning meeting. Similarly, the user stories are ofﬁcially closed
or completed only during a sprint review meeting, when they have been
successfully demonstrated to the product owner. In Case Ericsson, fewer
than a third of the analysed user stories were taken into implementation
during the sprint review and planning day. Approximately a third of the
user stories were closed during the sprint review and planning day. The
main reason for setting new user stories in progress during a development
sprint was that there was not always enough work in the existing user
stories for all team members, especially when the user stories required
speciﬁc skills that not all the team members had. User stories were
reviewed and closed during the sprints to allow the development of any
dependent user stories. In addition, the developers preferred to close user
stories as soon as they were ready, since the responsibility for ﬁxing any
issues related to a user story was lifted from the team when the user story
48
Summary of the results
was closed.
The goal in Scrum is to get all user stories taken into a sprint completed
during that sprint. Since user stories are taken in progress and closed only
once a sprint, the lead time, or the time in the development process, should
then be approximately equal to the sprint length. In Case Ericsson, the
user story development lead time was approximately two times the sprint
length of two weeks. This was explained by two causes. First, many user
stories could not be closed when the team was done with them due to the
external dependencies of the user stories. Second, splitting user stories
into small enough parts to be developed in a single two-week sprint was
considered difﬁcult due to the goal to create end-to-end functionality visible
to the user or customer in every user story and due to the complexity of
the system.
In Case Ericsson, the developers did not participate in the portfolio or
development steering group meetings where feature decisions were made.
However, one or several representative(s) from each development team
assigned to a feature had participated in the writing of the feature concept
study. This gave them insight into the big picture and the reasoning behind
the feature. The continuous release planning process also affected their
team-level planning process. Although the planning process employed
by the teams was inspired by Scrum, we found that several planning
practices employed by the development teams had characteristics of a
more continuous development process.
The Continuous Release Planning method supplied the development
organization new features on a weekly basis. Since several teams usually
developed each feature, the feature that a development team was implementing could be switched if the feature was near to completion or the
team had speciﬁc skills required for the implementation of another feature.
Since several teams were usually assigned to develop the same feature,
the dependencies between the teams often prevented the completion of
user stories. However, these discrepancies between the teams’ planning
practices and Scrum were considered nearly harmless or improvements on
Scrum. The changes were seen to improve the utilization of development
resources and to increase the ﬂexibility of the development organization.
49
Summary of the results
Table 4.1. The challenges and the cases in which they were identiﬁed
Case
Challenge
Agilefant
F-Secure
Effort spent on release planning
X
X
Selection of the candidate features
X
X
Identifying feature prioritization criteria
X
X
Ericsson
Preparation and deﬁnition of features
X
Balancing between specialized and generalist teams
X
Immature adoption of the agile mindset
X
X
Planning the product architecture
X
X
4.4
X
RQ3: What kinds of challenges were associated with the
release planning methods?
The challenges identiﬁed in the cases disseminated in this dissertation
are summarized in Table 4.1. The challenges are described in the sections
below.
4.4.1
Effort spent on release planning
The SCERP release planning method and the Release Iteration Planning
method were considered quite time-consuming. In Case Agilefant, the
product owner spent two days completing the SCERP method. This included the identiﬁcation and invitation of the stakeholders, pre-selection
of the candidate features, elaboration of the candidate features, selection
of the prioritization criteria and other related tasks. However, this did not
include the initial conception and prioritization of the features. In Case FSecure, the whole project organization and several additional stakeholders
spent two to three days in the Release Iteration Planning events, and the
preparatory effort required from the product owners and product mangers
was considerable. However, much of the discussion and planning would
have been required regardless of the release planning method used, and
the Release Iteration Planning method simply made the effort explicit.
4.4.2
Selection of the candidate features
The selection of the candidate features included in the release planning
was a challenge in Case Agilefant and Case F-Secure. In the Case Agilefant
survey, the stakeholders indicated that the list of candidate features did
not contain the most important features for them, and they would have
liked to add features to the list. In Case F-Secure, the product managers
50
Summary of the results
initially brought all features envisioned for the software to the ﬁrst Release Iteration Planning event. The number of features was many times
more than what could be implemented in the following release iteration.
Furthermore, the features were not pre-assigned for the ﬁrst event, and
the assignment of features to teams took considerable time. Since there
was no centralized information on which team was implementing which
feature, the participants could not easily see which features were actually
taken into planning. This challenge was resolved in the later events by the
tentative pre-assignment of the features to the development teams.
4.4.3
Identifying feature prioritization criteria
The feature prioritization criteria were identiﬁed as a challenge in Case
Agilefant and Case F-Secure. In Case Agilefant, the stakeholders tended
to give very similar priorities to the candidate features based on all three
prioritization criteria (the value, urgency and dissatisfaction). The stakeholders also commented in the survey results that they had difﬁculties
differentiating between the prioritization criteria. In Case F-Secure, the
product managers initially considered all features must-haves and were
unwilling to prioritize them. Instead, they considered the most efﬁcient
technical implementation order to be the right criteria for feature prioritization. In the later events, the product managers had produced a
prioritized list of features but still considered most of the features musthaves. This issue was solved before the last release planning event, when
the responsibility for prioritizing and elaborating the features was given
to the product owners.
4.4.4
Preparation and deﬁnition of features
In Case F-Secure, the product managers initially produced feature descriptions that varied wildly in detail level and in form. The features were not
synchronized and the descriptions were too abstract to be useful for the
developers and product owners. These issues slowed down planning in the
ﬁrst release planning event considerably: The teams needed the product
managers to explain the features, and subsequently the product managers became information bottlenecks. This challenge alleviated somewhat
over time as the product managers begun to understand the development
teams’ needs better. Furthermore, the preparation of features performed
by the product managers and the product owners in collaboration seemed
51
Summary of the results
to further improve the situation. In Case Agilefant, the survey responses
indicated that the stakeholders thought they understood the features quite
well based on the given descriptions, and that the only challenge related
to preparation and deﬁnition of features was the effort required of the
product owner.
4.4.5
Balancing between specialized and generalist teams
In Case F-Secure and Case Ericsson, building generalist end-to-end teams
was a challenge. Although the goal in both cases was to build at least
some generalist teams that could implement features end-to-end, both
organizations struggled to ﬁnd the right balance between specialization
and generalist teams. In the ﬁrst case project of Case F-Secure, the teams
were initially specialized either in the back-end or front-end of the software,
because the parts had been previously developed in separate organizations.
Although the goal was to create end-to-end features, no single team was
competent enough to create them alone. The separation between the
front-end and back-end, however, created much communications overhead
between the teams and made it difﬁcult to ensure that all parts of a feature
were implemented. Two end-to-end teams were created before the last
studied Release Iteration Planning event, but these teams did not perform
well. They split internally into front-end and back-end subteams and
struggled to plan the features in end-to-end fashion. The end-to-end teams
were later on disbanded.
In Case Ericsson, the initial goal was to create cross-functional generalist teams that would be able to implement features end-to-end. The
development organization, however, quickly realized that many parts of
the system were complex and difﬁcult and required experience and speciﬁc
skills to develop. Assigning inexperienced developers to those parts of the
system had created large learning overhead and long development lead
times. Subsequently, they began to assign teams to work on features on
which they had at least some experience.
4.4.6
Immature adoption of the agile mind-set
In Case F-Secure and Case Ericsson, the studied development organizations were in the middle of development process change. In Case F-Secure,
the development organization of the ﬁrst studied project had little experience of development in a large-scale Scrum organization. At ﬁrst, the
52
Summary of the results
product managers and the developers had understood little about the principles of agile software development. Consequently, the product managers
and developers expected that the plans created in the Release Iteration
Planning events should be followed rigorously; they did not adapt the plans
to new information they discovered during the release iteration. When
they could not hold on to the plans, rather than trying to adapt to the
new information, they considered the failing plans to be a problem with
the plans and the release planning method. Another issue caused by the
lack of agile mind-set was the lack of communication between the teams
in the ﬁrst few Release Iteration Planning events. By the virtue of the
previous, plan-driven development model, many developers expected that
someone else would manage the coordination between the teams. The
Release Iteration Planning method, however, expects the developers to
coordinate between teams and manage dependencies themselves. These
issues alleviated over time as the product managers and the developers
gained more experience with the agile development model.
In Case Ericsson, the product management was still working with a
plan-driven mind-set. They requested rigorous long-term plans that the
adoption of the Continuous Release Planning method had rendered unavailable. In addition, they pressured the product owner team to make
premature promises on feature completion, which overburdened the development organization. The case organization tried to mitigate this issue by
adding more slack to their estimates and by splitting features into smaller
parts that could be delivered earlier.
4.4.7
Planning the product architecture
In Case F-Secure and Case Ericsson, the practices and responsibilities for
planning the software architecture were a challenge. In Case F-Secure,
the software architects provided the teams with high-level architectural
plans, but the teams found these plans too abstract to be of much use. The
teams were expected to plan the lower-level architecture themselves, but
they lacked the skills and experience to do so. This was an issue especially
at the beginning of the ﬁrst case project.
In Case Ericsson, the responsibility for planning and improving of the
software architecture became unclear when the development organization adopted the Continuous Release Planning method. In the previous,
plan-driven development model, technical specialists had been responsible
for planning the architecture. After the transformation, the role of the
53
Summary of the results
Table 4.2. The beneﬁts and the cases where they were identiﬁed in
Case
Beneﬁt
Agilefant
F-Secure
Ericsson
Mediation of multiple opinions and perspectives
X
X
X
Efﬁciency of the planning method
X
X
X
Creation of high-quality plans
X
Increased motivation
X
X
X
technical specialists was de-emphasized, and the development teams were
expected to take over the role. However, the teams concentrated on developing the features they had been assigned and had little time for system
planning and improvement.
4.5
RQ4: What kinds of beneﬁts have the release planning
methods realized?
The beneﬁts identiﬁed in the cases described in the thesis are summarized
in Table 4.2.
4.5.1
Mediation of multiple opinions and perspectives
The arbitration and mediation of the different opinions and perspectives of
the stakeholders regarding the priorities of features was a beneﬁt created
by the release planning methods presented in this thesis. In Case Agilefant,
we found that the priorities the stakeholders assigned to the different
features varied considerably. SCERP condensed the different opinions into
a single number that could be employed to generate a set of (near-)optimal
release plans. Furthermore, the stakeholders were allowed to express their
priorities the second time when they voted between the alternative release
plans. This revealed that two of the plans generated by SCERP were
regarded as considerably more acceptable than the three other generated
plans or the manual plan.
In Case F-Secure, all stakeholders took part in the Release Iteration
Planning events. This allowed immediate and collaborative decision making regarding the scheduling and priorities of the features. Whenever an
issue regarding the features was identiﬁed, a group of stakeholders with
the knowledge and power to solve the issue could quickly be gathered to
solve it.
In Case Ericsson, the feature decisions were made by the portfolio steer-
54
Summary of the results
ing group, which consisted of stakeholders with many different perspectives. The development teams could express their perspectives in the
planning process when they helped to create the feature concept studies.
The method combined the business knowledge of the portfolio steering
group members with the technical knowledge of the technical specialists
and developers.
4.5.2
Efﬁciency of the planning method
The results from Case Agilefant indicate that each stakeholder spent
only 14 minutes on average to prioritize the features and vote on the
release plan alternatives. This suggests that the method is an efﬁcient
way to plan for the stakeholders. Although the results in Case F-Secure
regarding the overall efﬁciency of the Release Iteration Planning method
were contradictory, the face-to-face communication in the Release Iteration
Planning events enabled the fast decision making and the fast recognition
and mitigation of impediments and risks. This was identiﬁed as clear
improvement over the previous, plan-driven development model where
making similar decisions could take up to several weeks of calendar time.
In Case Ericsson, the multiple feature planning iterations during the
early phases allowed the identiﬁcation and removal of infeasible or tooexpensive features. Since the feature plans were incrementally elaborated,
the cost sunk during the early phases was relatively small. This was a
major beneﬁt of the Continuous Release Planning method compared with
the previous, plan-driven model, in which intricate feature plans were
made before a feature even began to be developed.
4.5.3
Creation of high-quality plans
Case Agilefant illustrates that the plans generated by SCERP were considerably more optimal than the manual plan. The formal optimality of the
generated plans was between 100% and 96.8% while the manual plan was
only 93.4% optimal.
In Case F-Secure, the plan resulting from the Release Iteration Planning
method was not as important as the planning process that led to it. The
plans were actually expected to change during the following development
iteration, although the product managers and developers at ﬁrst had
difﬁculties understanding that this was the intention. Thus, in the Release
Iteration Planning method, the formal quality of the created plan was not
55
Summary of the results
considered relevant.
In Case Ericsson, the feature plans were incrementally elaborated during
the early phases of the Continuous Release Planning method. In the
previous, plan-driven model, the feature plans were very detailed and
intricate and considered being of high quality. Although the Continuous
Release Planning method required less feature planning effort, we did not
ﬁnd that the quality of the plans had decreased from that of the previous
model.
4.5.4
Increased motivation
In Case F-Secure and Case Ericsson, the collaborative nature of the release
planning methods increased the developers’ motivation. In Case F-Secure,
the introductory presentations gave the developers information on the
high-level goals of the project and showed the importance of the project
to the company. All participants could meet face to face, which helped to
create a project spirit. In Case Ericsson, the developers took part in the
early phases of the feature development, which allowed them to see the
motivation behind the features and the big picture of the system. In Case
Agilefant, the effect of the method on the motivation of the developers or
the product owner was not studied.
56
5. Discussion
In this chapter, answers to the research questions are discussed and the
ﬁndings are discussed in comparison with the previous research. Then,
threats to the validity and reliability of the results are discussed.
5.1
Answers to research questions
In this section, answers to the four research questions presented in the
introduction are discussed. The discussion is organized following the order
in which research questions were given.
5.1.1
How were the releases planned in the cases studied?
Even though all studied cases employed a software development method
at least inspired by Scrum, the way releases were planned differed widely
in the three studied cases. Table 5.1 illustrates the main characteristics of
each release planning method.
SCERP requires that the release planning problem is modelled by identiTable 5.1. The characteristics of the release planning methods
Method
Main
Decision maker(s)
characteristic(s)
SCERP
model-based,
Information
providers
product manager
algorithmic
selected
stakeholders
Release Iteration
iterative &
developers, product
relevant
Planning
incremental
owners, product
stakeholders
managers
Continuous
Release Planning
continuous
portfolio &
technical
development
specialists, product
steering groups
owners, developers
57
Discussion
fying the prioritization criteria, resources, and resource constraints before
the release planning is conducted. A selected set of stakeholders vote on
the priorities of the candidate features and the optimization algorithm at
the core of the method creates several alternative plans based on the votes
and the release model. The stakeholders vote on their preferred release
plan, but the ﬁnal decision is made by the product manager or the product
owner.
The Release Iteration Planning method is iterative and incremental
and based on Release Iteration Planning events. Product owners and
product managers make the initial feature prioritization decisions, but
the prioritization can be changed according to the information uncovered
during the events. The development teams make the user story-level
prioritization and scheduling decisions. All relevant stakeholders take
part in the release planning events and provide information and guidance
for the developers, product owners, and product managers.
The portfolio and development steering groups, who were responsible for
release planning decision making in Case Ericsson, met once a week. In
contrast to the two to three months between the Release Iteration Planning events in Case F-Secure, the Continuous Release Planning method
was indeed a continuous process. In addition, the technical specialists,
product owners, and developers who created the one pagers and feature
concept studies worked continuously to create the information needed in
the portfolio steering group meetings to make release planning decisions
during the early phases of the feature development.
Although the Release Iteration Planning and Continuous Release Planning methods are quite different in practice, both employ the knowledge
of the whole development organization and relevant stakeholders in the
creation of the release plans. In SCERP, the way in which the stakeholders
affect the release plans is the prioritization of the candidate features and
voting on the plans. Although the development organization and stakeholders contribute to the prioritization process in all three release planning
methods, the ﬁnal say regarding the prioritization is in the hands of the
product owners and product managers.
A multitude of methods similar to SCERP have been proposed in the
model-based release planning research (Svahnberg et al., 2010). The main
difference to these methods, and the main contribution of SCERP, was the
aim of combining model-based release planning with Scrum. Li et al. (2006)
have proposed a similar, risk-driven method for extreme programming
58
Discussion
release planning. They have also conducted a small-scale experiment,
and found that their method allows the decision makers to create better,
risk-driven release plans. Case Agilefant supports their conclusion that
a model-based release planning method can be successfully applied to
small-scale agile development.
Research literature describing release planning methods similar to the
Release Iteration Planning or the Continuous Release Planning method
could not be identiﬁed. Thus, comparisons can be made only with practitioner literature. The joint release planning method proposed by Lefﬁngwell (2011) is partially based on Case F-Secure and, consequently, is very
similar to the Release Iteration Planning method.
The organization of development proposed by Larman and Vodde (2010) is
similar to the organization in Case Ericsson, and they suggest that release
contents should be based on what is ready in time for the release, which
was also true in Case Ericsson. Nevertheless, the methods for identifying,
elaborating, prioritizing, resourcing, and scheduling features suggested by
Larman and Vodde (2010) are notably different to Case Ericsson.
5.1.2
What were the team-level planning processes, and how
were they affected by the release planning method?
In all three studied cases, the team-level planning process was inspired
by Scrum. The main Scrum planning meetings were conducted in the
cases. In Case Ericsson, the practices associated with Scrum were not
followed to the letter. The most prominent difference to the Scrum-by-thebook (Schwaber and Beedle, 2002; Schwaber and Sutherland, 2013) was
the addition and closing of user stories during the development sprints.
According to a survey by VersionOne, Inc. (2013), although Scrum is the
most-used agile method, practitioners rarely use all the Scrum practices
together. This suggests that what practitioners understand as Scrum is
quite different from what Scrum textbooks describe.
Although only two cases were studied in depth1 , an interesting hypothesis can be created about the effects the release planning methods had on
the team-level planning practices. In Case F-Secure, the Release Iteration Planning method was iterative and incremental. The development
teams in Case F-Secure followed Scrum planning practices in team-level
planning. In Case Ericsson, the Continuous Release Planning method
was continuous in nature, and the planning practices of the development
1 Case Agilefant was an illustrative, small-scale case study.
59
Discussion
teams had characteristics of a continuous process instead of the strict
sprint cycle of Scrum. Consequently, it seems that the nature of the release
planning method inﬂuences the team-level planning practices even when
the methods and practices are not ofﬁcially synchronized between the two
levels of planning.
5.1.3
What kinds of challenges were associated with the release
planning methods?
Two kinds of challenges associated with the methods can be identiﬁed. The
ﬁrst kind is the challenges related to the release planning methods. The
effort spent on release planning, selection of candidate features, identifying
feature prioritization criteria, and preparation and deﬁnition of features
are challenges of the ﬁrst kind. In Case F-Secure, these challenges alleviated over time as the case organization gained more experience in applying
the Release Iteration Planning method and prepared for the events better
than earlier. In Case Agilefant, these challenges were dependent on the exact prioritization criteria and backlog management practices employed in
the case project. With a different set of criteria and with more systematic
backlog management practices, these challenges might have been avoided.
Challenges of the second kind are related to adopting and applying agile
methods, especially on the large scale. The balancing between specialized
and generalist teams, the immature adoption of the agile mind-set and the
planning of the product architecture are all challenges of this kind. The
results from Case Agilefant support this ﬁnding; The case organization
consisted of a product owner and two developers, and it did not face these
challenges. Freudenberg and Sharp (2010) identiﬁed agile on the large
scale as one of the most important research questions in the software
engineering domain. Changes made to the release planning methods
themselves are unlikely to completely remove these challenges, and ﬁnding
solutions requires further research on agile methods on the large scale.
5.1.4
What kinds of beneﬁts have the release planning methods
realized?
Especially in complex, large-scale software development projects, a single
decision maker cannot be expected to have all the information required to
make optimal or even good release planning decisions (Lehtola et al., 2004;
Ruhe and Saliu, 2005). When decision makers have a business-oriented
perspective, they may underrate system improvement and innovation
60
Discussion
related work in comparison to features that are visible to the customers
(Barney et al., 2008; Fogelström et al., 2009; Lindgren et al., 2008). On
the other hand, developers may have a limited understanding about the
needs of the customers and about the long-term business goals of their
organization (Lehtola et al., 2004). All three release planning methods
presented in this thesis allowed the mediation of multiple opinions and
perspectives.
Although some have claimed that the planning done by the business
organization should be strictly separated from the planning done by the
development organization (Ferrari, 2008; Kittlaus and Clough, 2009), the
results of this dissertation suggest that collaborative release planning is
both possible and beneﬁcial in agile software development organizations
regardless of the size of the organization. This may be the most important
ﬁnding in this dissertation.
The release planning methods described in this thesis were all considered
highly effective in different ways. The SCERP release planning method
demanded little time from the participating stakeholders except from the
product owner. The Release Iteration Planning method made the effort
spent on planning visible and enabled efﬁcient face-to-face communication among the stakeholders. The Continuous Release Planning method
allowed the identiﬁcation and elimination of infeasible or too-expensive
features with minimal costs sunk to the feature elaboration. Our results
suggest that a release planning method must be efﬁcient to gain acceptance
in the organization.
Agile software development methods emphasize that the action of collaborative planning and the ability to react to changes are more important
than the creation of precise and detailed plans (Beck et al., 2001; Leffingwell, 2011). This philosophy was embraced in the Release Iteration
Planning method, although at ﬁrst the case organization had difﬁculties
understanding it. The approach taken in the Continuous Release Planning
method was somewhat different. Rather than investing effort on detailed
plans, the method was based on the stepwise elaboration of individual feature information. Eventually, the feature information was expected to be
sufﬁcient to produce releases that satisﬁed the needs of the customers. In
SCERP, the success of the planning was measured by the formal optimality
of the plans created. The central ﬁnding regarding the level of detail and
precision of the plans was that the pursued level should be considered
explicitly in order to avoid excess spending in creating the plans.
61
Discussion
All three release planning methods provided a way to take into account
the needs of the customers and users. In SCERP, the stakeholders of
the software project directly affected the prioritization by voting on the
priorities of the candidate features. The test version releases created in
the Release Iteration Planning method were explicitly created in order
to gather feedback from test users. This feedback could be used both
by the software developers to improve the existing features and by the
management to reconsider the priorities of the upcoming requirements. In
the Continuous Release Planning method, all releases were considered full
releases of the software. The rapid release cycle allowed the management
to quickly react to the needs of the customers and to the changes in the
market.
Existing research on motivating software developers (Beecham et al.,
2008) suggests that their main motivational factors are (1) autonomy, (2)
mastery, and (3) purpose. The developers should be able to make decisions
concerning their own work, they should be given opportunities to improve
their skills and learn new things, and they should have knowledge of the
purpose of their work. Both the Release Iteration Planning method and
the Continuous Release Planning method had the aspects of the three
motivational factors.
In the Release Iteration Planning method, (1) the developers had the
central role in the planning of their team’s work during the release planning events and during the sprints; (2) the developers were responsible
for implementing the end-to-end features, which gave them opportunities
to study and learn new things; (3) the Release Iteration Planning events
gave the developers visibility to the planned future of the product. In
the Continuous Release Planning method, the developers took part in the
early phases of the feature development, which (1) gave them authority
to perform team-level planning as they saw ﬁt; (2) encouraged them to develop features beyond their immediate area of expertise; (3) allowed them
to see the upcoming features. Based on these cases, taking motivation
factors into account when a release planning method is adopted is clearly
an important success factor.
5.2
Threats to validity and limitations
This section describes the threats to the validity and reliability of this
research and the limitations of this research. In the discussion on the
62
Discussion
validity and reliability of our results and analysis, this section relies on
the deﬁnitions proposed by Yin (2009) and by Shadish et al. (2001). The
discussion on validity and reliability is based on the tradition of pragmatist
epistemology in software engineering research (Petersen and Gencel, 2013;
Runeson and Höst, 2009; Sjøberg et al., 2007; Wohlin et al., 2003). Other
frameworks have been proposed to better suit the evaluation of the quality
or trustworthiness of purely qualitative research (Petersen and Gencel,
2013; Seale, 1999). The pragmatist approach to quality is appropriate
for this dissertation for two reasons. First, the research presented in
this dissertation employs mixed methods instead of purely qualitative or
quantitative methods. Second, the goal is to disseminate release planning
methods, not to describe the feelings and experiences of individuals.
The validity of the research is discussed on the basis of ﬁve different
aspects: the internal validity, the construct validity, the external validity,
the statistical-conclusion validity and the reliability (Shadish et al., 2001;
Yin, 2009). Statistical conclusion validity is relevant only in relation to
Case Ericsson, as the two other cases did not include statistical analysis beyond simple descriptive statistics. Finally, the limitations of the case study
research method in comparison to other research methods are discussed.
5.2.1
Construct validity
Construct validity is concerned with how well the constructs (concepts,
themes, categories, hypotheses and so on) created during the analysis
process reﬂect the reality (Shadish et al., 2001; Yin, 2009). The main
way to improve the construct validity of a case study is to triangulate
data sources, investigators, theories, and methods (Jick, 1979; Patton,
2002; Yin, 2009). Of these, data sources, investigators and methods were
triangulated in Case F-Secure and Case Ericsson. Triangulation of theories
was not used, because the research was exploratory and descriptive, not
theory-based and deductive.
The ﬁrst threat to construct validity is the inadequate identiﬁcation
of constructs. The constructs identiﬁed in this research may not fairly
represent the reality due to errors in the coding process or because of
researchers’ bias in the analysis. The ﬁrst threat was somewhat mitigated
by the constant comparison (Adolph et al., 2011) technique that was employed in the data analysis, as the inaccurate constructs should have been
eventually corrected or eliminated in the analysis process. Another way
the ﬁrst threat can be mitigated is investigator triangulation. In Case
63
Discussion
F-Secure and Case Ericsson, at least two researchers analysed the same
data at least partially; in addition, in both cases, at least one experienced
and authoritative member of the case organization reviewed the results.
The second threat is that subjects under study may have reacted to the
presence of the researcher and provided data that gave an overly positive,
or otherwise biased, image of the case organization or themselves. Thus,
the results may not fairly represent their normal conduct and the reality in
the case organization. The second threat was mitigated in Case F-Secure
by the longitudinal nature of the study. Over time, the members of the
case organization got used to the presence of the researchers. In addition,
data source and method triangulation were employed in Case F-Secure by
collecting data from many different sources and by using multiple data
collection methods: observations, interviews, and surveys. The results of
the observations and interviews were triangulated with the results from
the surveys. The quantitative surveys allowed us to compare the opinions
of a larger sample of participants to the results from the qualitative data.
The free text answers to the surveys were analysed alongside the other
qualitative data. Overall, the results from the surveys supported the
ﬁndings from the observations and interviews. In Case Ericsson, the
triangulation of data sources was employed by interviewing wide variety
of subjects from different parts of the organization and with different roles.
Triangulation of methods was employed in the analysis of the team-level
practices in Case Ericsson by analysing both quantitative requirement
data and qualitative interview data.
In Case Agilefant, the main threat to construct validity is the participating stakeholders’ understanding of the features. Although the stakeholders
may have misunderstood the features, the conclusions of the research on
the efﬁciency of the release planning method hold. Furthermore, the
results of the survey indicate that the stakeholders did understand the
candidate features they were presented.
5.2.2
Internal validity
The Internal validity of a descriptive case study is concerned with the
validity of the causal inferences identiﬁed in the analysis (Shadish et al.,
2001; Yin, 2009). The identiﬁed causal relation may be temporally ambiguous, that is, it may be uncertain what is the cause and what is the effect.
Events that happen concurrently or changes occurring naturally over time
may create an illusion of a cause-effect relation.
64
Discussion
In Case F-Secure, several cause-effect relations were identiﬁed between
the changes made to the release planning method and the effects of those
changes. In Case Ericsson, the analysis of cause-effect relations was
limited to the cause-effect relations identiﬁed by the interviewees. In both
cases, it is possible that the results were caused by the aforementioned
threats to internal validity instead of the cause-effect relations identiﬁed
in the research. Only the study of additional, similar cases can reveal how
internally valid the causal inferences presented in this dissertation really
are.
5.2.3
External validity
The external validity of a case study is concerned with the domain the
results can be generalized to (Shadish et al., 2001; Yin, 2009). Threats to
external validity arise from the aspects of the cases that were not identiﬁed or disseminated, but that affected the results. In single-case-study
research, the generalizability of the result to other domains can be considered only on a theoretical level (Yin, 2009). The main way to increase
external validity in single-case-study research is to study and disseminate
the case in enough detail to allow comparisons with other organisations.
This will allow other studies on similar subjects to make comparisons with
the presented case and, eventually, also enables the retrospective analysis
of the generalizability of the results presented in this dissertation. The
results of the statistical analyses that are described in Publication IV are
not generalizable beyond the case presented in the publication. Although
the hypotheses that are put forward in the publication are tested statistically, what is discussed above regarding the generalisability of single
case studies also holds regarding the results of the statistical analyses in
Publication IV.
5.2.4
Statistical-conclusion validity
Statistical-conclusion validity is concerned with the correct use of statistical analysis (Shadish et al., 2001). For example, are the variables truly
measured on the appropriate level? Is the distribution of the data appropriate considering the statistical analysis employed? The quantitative
survey data employed in this research were analysed using descriptive
statistics that required data measured on the ordinal level, that is, using
quartiles, distributions and frequencies. These statistics are widely ac-
65
Discussion
cepted as proper for ordinal data (Coolican, 1999). Consequently, there are
no major threats to the statistical-conclusion validity regarding the survey
results. In Case Ericsson, several statistical analyses were used to analyse
the quantitative backlog data beyond simple descriptive statistics. It is
possible that these statistical analyses were used inappropriately. However, these statistical analyses were employed to afﬁrm results from the
descriptive analysis, and the results were also afﬁrmed by the qualitative
data. It is unlikely that the major conclusions drawn from the case were
invalid because of issues in the statistical-conclusion validity.
5.2.5
Reliability
Reliability of research is concerned with the repeatability of research; if
other researcher or researchers had conducted the same study, would they
have reached the same results (Yin, 2009)? Case study research is very
much based on the cognitive skills of the researcher(s). We may expect
that multiple researchers with equal cognitive skills and, perhaps more
importantly, with equal pre-existing knowledge of the topic would produce
the same results if they followed the same research protocol.
The main threat to the reliability of Case F-Secure and Case Ericsson
is the variability in the data collection. Different researchers might have
observed different things and, because of the semistructured nature of
the interviews, asked different interview questions. In both cases, many
different data sources, a wide variety of interview questions, and several
interviews were used, making the results more reliable. In Case Agilefant,
the researchers did not have an active role in the data collection, which
makes the results robust against threats to reliability. The algorithms the
SCERP method uses to produce release plans are deterministic; the same
inputs to the algorithms always produce the same results, making this
part of the method highly reliable.
5.2.6
Limitations
The case study research method (Yin, 2009) is the most appropriate when
a "how" or "why" question is being asked. There are several important aspects to release planning in agile software development organizations that
could have been answered by employing other research methods, but were
left unanswered by this thesis. The ﬁrst limitation is put forward in Publication V: "Only true experiments could reveal if there is any signiﬁcant
66
Discussion
difference between the effort spent on those [release planning] activities in
plan-driven and agile projects". Based on the research presented in this
thesis, it is not possible to say how quantitatively efﬁcient the presented
methods are in comparison to traditional, plan-driven approaches, or in
comparison to each other. Inferring such information would require quantitative experimentation (Sjøberg et al., 2007). There are many similar
quantitative aspects to agile release planning. For example, what is the
most efﬁcient level of detail for release plans or requirements brought to a
release planning event? Such questions, although important, can not be
answered statistically by the case study method and were not studied in
this dissertation.
The second limitation is that this dissertation does not present information regarding the use of different release planning methods or about
release planning in general in the software industry. Such data can be
collected by conducting industry surveys (Sjøberg et al., 2007). However,
at the time the research was conducted, the terminology related to release
planning was not established in the industry or in the SE research community. The validity of such surveys would have been questionable due
to the measurement bias created by the diverse understanding of the key
terminology (Lavrakas, 2008).
The main goal of this research was the description of the release planning methods, not their improvement. However, this dissertation presents
observations on improvements made to the methods. The third limitation
is the limited validity of the observed cause-effect relationships between
the improvements and their effects. The action research method is aimed
for studying improvements (Sjøberg et al., 2007). The canonical action research method is based on substantive theory or hypothesis that permeates
all steps of the action research cycle (Davison et al., 2012). When the research was conducted, no such theory or even a hypothesis existed for agile
release planning, and consequently the canonical action research method
was not an option. Based on the results presented in this dissertation,
further hypotheses regarding improvements to agile release planning can
be formed and the action research method is one valid option for testing
the hypotheses.
67
Discussion
68
6. Conclusions
In this ﬁnal section of the dissertation summary, the implications of the
results of this research to practice and to theory are provided. Finally,
directions for future work are suggested.
6.1
Implications to practice
This dissertation presents three release planning methods which were
employed in different organizational contexts. Although the goals of this
dissertation did not include the presentation of normative release planning
guidance, practitioners in similar organizations can use the results if they
are looking for a method for release planning. Especially the challenges
faced by the case organizations and the solutions identiﬁed can be useful
for practitioners facing similar challenges.
The results of this dissertation support the previous ﬁndings that release
planning should not be considered a process that is completed before the
development project begins. Mangers should not plan releases in isolation
from the developers. Release planning should not be considered only an
optimization problem that needs to be solved before the development begins.
Instead, whatever is the speciﬁc method chosen, release planning should
be considered a collaborative decision making process that is informed by
all stakeholders relevant to the planned software.
Another ﬁnding that may have important implications to practice is the
connection between the nature of the release planning process and the
team-level planning practices. The results of this dissertation suggest that
the team-level planning practices take on the characteristics of the release
planning method: A continuous release planning method induces continuous team-level practices, and an iterative and incremental method induces
iterative and incremental team-level practices. Consequently, the team-
69
Conclusions
level practices adopted in a software development organization may beneﬁt
from having a similar nature to the employed release planning method,
and vice versa. Since the number of cases was small, this implication must
be considered very hypothetical.
6.2
Implications to theory
The case studies presented in this dissertation were revelatory case studies. Besides the literature written by consultants and practitioners, this
dissertation is the ﬁrst detailed descriptive study of release planning in
large-scale agile software development organizations. The two methods
employed in the large-scale Scrum organizations were both successful, and
the results show that collaborative release planning is possible in such a
context.
The great majority of software release planning research has concentrated on model-driven, algorithmic optimization methods (Svahnberg
et al., 2010). The results presented in this dissertation support the previous ﬁndings that the assumptions made in the algorithmic methods often
do not match reality in software engineering organizations (Benestad and
Hannay, 2011; Boehm, 2000; Cao and Ramesh, 2008; Carlshamre, 2002;
Jantunen et al., 2011).
Further release planning research, whether model-based or not, should
take into account the following ﬁndings of this research when proposing
new methods or processes: Release planning should be seen as a collaborative effort that utilizes the knowledge of the whole organization. Although
decision makers, such as product managers or product owners, have the
last word on the release planning decisions, they cannot be expected to comprehensively explicate the prioritization criteria they employ in making
the decisions. Comprehensive and correct information about a requirement
is rarely available at the beginning of the development of the requirement.
Release planning methods should be capable of reacting to newly uncovered information about the priorities or content of the requirements or the
needs of the customers or users. The release planning methods presented
in this dissertation are based on relatively frequent releases, which is
one way to react to new information and to changing market needs. Dependencies between features are often complex and difﬁcult to identify
before implementation. Consequently, release planning methods should
not expect that all dependencies are known beforehand.
70
Conclusions
6.3
Future work
The most obvious direction for future work is the continued study of release
planning methods employed in large-scale agile development organizations
to ﬁnd out how generalizable the ﬁndings presented in this thesis are.
The two large development organizations described in this thesis both
were somewhat immature in their adoption of Scrum. Studies of release
planning in mature Scrum organizations may reveal more about which
of the challenges identiﬁed in this thesis stem from immature Scrum
adoption and which are related to the release planning methods. Many
development organizations have begun to adopt aspects of lean software
development methods (Poppendieck and Poppendieck, 2003), such as the
Kanban workﬂow management process (Rautiainen, 2011). It would be
interesting to see what kinds of release planning methods are employed
in development organizations that have embraced the lean thinking and
ideology.
Release planning decision support research has concentrated on normative, model-based release planning. It has proposed models on how release
planning should be performed and what kinds of criteria and constraints
should be considered. In the real world, however, the criteria and constraints are difﬁcult to identify, explicate, and employ deterministically.
Based on the results of this thesis, research on tools for release planning
should concentrate on supporting dependency management, architectural
planning, and communication instead of feature prioritization and plan
optimization.
Finally, this dissertation has put forward the following preliminary hypothesis about the relation between the nature of a release planning
process and the nature of a team-level planning processes: The team-level
planning practices are inﬂuenced by the nature of the release planning
process, be it continuous, iterative and incremental, or something else,
and eventually they begin to take on the aspects of the release planning
process. This relationship may also hold in the opposite direction. Further
studies on this subject could produce interesting and high-impact results
on how the development and release planning should be organized.
71
Conclusions
72
Bibliography
Steve Adolph, Wendy Hall, and Philippe Kruchten. Using grounded theory to
study the experience of software development. Empirical Software Engineering,
16(4):487–513, 2011.
Ahmed Al-Emran and Dietmar Pfahl. Operational planning, re-planning and
risk analysis for software releases. In Jürgen Münch and Pekka Abrahamsson,
editors, Proceedings of the 8th International Conference on Product-Focused
Software Process Improvement (PROFES 2007), volume 4589 of Lecture Notes
in Computer Science, pages 315–329. Springer Berlin Heidelberg, July 2007.
Sebastian Barney, Aybüke Aurum, and Claes Wohlin. A product management
challenge: Creating software product value through requirements selection.
Journal of Systems Architecture, 54(6):576–593, 2008.
Michael E. Bays. Software Release Methodology. Prentice-Hall, Upper Saddle
River, NJ, USA, 1999.
Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cocburn, Ward Cunningham, Martin Fowler, James Grenning, Andrew Hunt, Ron Jeffries, Jon Kern,
Brian Maric, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland,
and Dave Thomas. Manifesto for agile software development, 2001. URL
http://agilemanifesto.org/. Retrieved Apr. 4th, 2014.
Sarah Beecham, Nathan Baddoo, Tracy Hall, Hugh Robinson, and Helen Sharp.
Motivation in software engineering: A systematic literature review. Information
and Software Technology, 50(9–10), 2008.
Hans Christian Benestad and Jo E. Hannay. A comparison of model-based and
judgment-based release planning in incremental software projects. In Proceeding of the 33rd International Conference on Software Engineering (ICSE 2011),
pages 766–775, New York, NY, USA, 2011. ACM.
Barry W. Boehm. Requirements that handle IKIWISI, COTS, and rapid change.
Computer, 33(7):99–102, 2000.
Lan Cao and Balasubramaniam Ramesh. Agile requirements engineering practices: An empirical study. Software, 25(1):60–67, 2008.
Pär Carlshamre. Release planning in market-driven software product development: Provoking an understanding. Requirements Engineering, 7(3):139–151,
2002.
73
Bibliography
Pär Carlshamre, Kristian Sandahl, Mikael Lindvall, Bjön Regnell, and Johan
Natt och Dag. An industrial survey of requirements interdependencies in
software product release planning. In Frances M. Titsworth, editor, Proceedings
of the IEEE International Conference on Requirements Engineering (RE ’01),
pages 84–91, Piscataway, NJ, USA, 2001. IEEE Computer Society.
Tsun Chow and Dac-Buu Cao. A survey study of critical success factors in agile
software projects. Journal of Systems and Software,, 81(6):961–971, 2008.
Mike Cohn. User stories applied : for agile software development. Addison-Wesley,
Boston, MA, USA, 2004.
Mike Cohn. Agile estimating and planning. Prentice Hall Professional Technical
Reference, Upper Saddle River, NJ, USA, 2005.
Dan Condon. Software product management : managing software development
from idea to product to marketing to sales. Aspatore, Boston, MA, USA, 2002.
Hugh Coolican. Research methods and statistics in psychology. Hodder &
Stoughton, London, UK, 3rd edition, 1999.
Robert M. Davison, Maris G. Martinsons, and Ned Kock. Principles of canonical
action research. Information Systems Journal, 14(1):65–86, 2004.
Robert M. Davison, Maris G. Martinsons, and Carol X. J. Ou. The roles of theory
in canonical action research. MIS Quarterly, 36(3):763–796, 2012.
Torgeir Dingsøyr, Sridhar Nerur, VenuGobal Balijepally, and Nils Brede Moe. A
decade of agile methodologies: Towards explaining agile software development.
Journal of Systems and Software, 85(6):1213–1221, 2012.
Christof Ebert. Software product management. Crosstalk, The Journal of Defence
Software Development, pages 15–19, January 2009.
Ernani Ferrari. Product Management for Software. Mondo Strategies Press, 2008.
Nina Dzamashvili Fogelström, Mikael Svahnberg, and Tony Gorschek. Investigating impact of business risk on requirements selection decisions. In Proceedings
of the 35th Euromicro Conference on Software Engineering and Advanced Applications (SEAA ’09), pages 217–223, Piscataway, NJ, USA, 2009. IEEE Computer
Society.
Nina Dzamashvili Fogelström, Tony Gorschek, Mikael Svahnberg, and Peo Olsson.
The impact of agile principles on market-driven software product development.
Journal of Software Maintenance and Evolution: Research and Practice, 22(1):
53–80, 2010.
Sallyann Freudenberg and Helen Sharp. The top 10 burning research questions
from practitioners. IEEE Software, 27(5):8–9, 2010.
Barney G. Glaser. Remodeling grounded theory. Historical Social Research,
Supplement, 19:47–68, 2007.
Ville T. Heikkilä, Kristian Rautiainen, and Jarno Vähäniitty, editors. Towards Agile Product and Portfolio Management. Aalto University, Helsinki,
Finland, 2011.
URL http://www.soberit.hut.fi/sprg/projects/atman/
TowardsAgileProductandPortfolioManagement.pdf. Retrieved Apr. 4th, 2014.
74
Bibliography
Chein Isador, Stuart W. Cook, and John Harding. The ﬁeld of action research.
American Psychologist, 3:45–50, 1948.
Sami Jantunen, Laura Lehtola, Donald C. Gause, U. Rex Dumdum, and Raymond J. Barnes. The challenge of release planning. In Proceedings of the
Fifth International Workshop on Software Product Management (IWSPM 2011),
pages 36–45, Piscataway, NJ, USA, 2011. IEEE.
Todd D. Jick. Mixing qualitative and quantitative methods: Triangulation in
action. Administrative Science Quarterly, 24(4):602–611, 1979.
Hans-Bernd Kittlaus and Peter N. Clough. Software product management and
pricing: key success factors for software organizations. Springer, Berlin, Germany, 2009.
Craig Larman. Agile and Iterative Development : A Manager’s Guide. AddisonWesley, Boston, MA, USA, 2004.
Craig Larman and Bas Vodde. Practices for scaling lean & agile development: large,
multisite, and offshore product development with large-scale scrum. AddisonWesley, Upper Saddle River, NJ, USA, 2010.
Paul J. Lavrakas, editor. Encyclopedia of Survey Research Methods. Sage Publications, Inc., 2008.
Dean Lefﬁngwell. Agile software requirements : lean requirements practices for
teams, programs, and the enterprise. Addison-Wesley, Upper Saddle River, NJ,
USA, 2011.
Laura Lehtola, Marjo Kauppinen, and Sari Kujala. Requirements prioritization
challenges in practice. In Frank Bomarius and Hajimu Iida, editors, Proceedings
of the Product Focused Software Process Improvement conference (PROFES
2004), volume 3009 of Lecture Notes in Computer Science, pages 497–508.
Springer Berlin Heidelberg, 2004.
Mingshu Li, Meng Huang, Fengdi Shu, and Juan Li. A risk-driven method for eXtreme programming release planning. In Proceedings of the 28th international
conference on Software engineering (ICSE 2006), pages 423–430, New York, NY,
USA, 2006. ACM.
Markus Lindgren, Christer Norsträm, Anders Wall, and Rikard Land. Importance
of software architecture during release planning. In Proceedings of the 7th
IEEE/IFIP Working Conference on Software Architecture (WICSA 2008), pages
253–256, 2008.
Andrey Maglyas, Uolevi Nikula, and Kari Smolander. What do practitioners mean
when they talk about product management? In Proceedings of the 20th IEEE
International Requirements Engineering Conference (RE 2012), pages 261–266,
2012.
Mika V. Mäntylä and Juha Itkonen. More testers — the effect of crowd size and
time restriction in software testing. Information and Software Technology, 55
(6):986–1003, 2013.
Mika V. Mäntylä and Juha Itkonen. How are software defects found? the role of
implicit defect detection, individual responsibility, documents, and knowledge.
Information and Software Technology, In Press.
75
Bibliography
An Ngo-The and Guenther Ruhe. A systematic approach for solving the wicked
problem of software release planning. Soft Computing — A Fusion of Foundations, Methodologies and Applications, 12(1):95–108, 2008.
Helena Holmström Olsson, Hiva Alahyari, and Jan Bosch. Climbing the "stairway
to heaven" — a mulitiple-case study exploring barriers in the transition from
agile development towards continuous deployment of software. In Proceedings
of the 38th EUROMICRO Conference on Software Engineering and Advanced
Applications (SEAA), pages 392–399, 2012.
Michael Q. Patton. Qualitative research and evaluation methods. Sage Publication,
Thousand Oaks, CA, USA, 3rd edition, 2002.
Kai Petersen and Cigdem Gencel. Worldviews, research methods, and their relationship to validity in empirical software engineering research. In Proceedings
of the Joint Conference of the 23rd International Workshop on Software Measurement and the Eighth International Conference on Software Process and Product
Measurement (IWSM-MENSURA), pages 81–89. IEEE Computer Society, 2013.
Mary Poppendieck and Tom Poppendieck. Lean software development : an agile
toolkit. Addison Wesley, Boston, MA, USA, 2003.
Roger S. Pressman. Software engineering: a practitioner’s approach. McGraw-Hill,
Boston, MA, USA, 7th edition, 2010.
Kim H. Pries and Jon M. Quigley. Scrum Project Management. CRC Press, Boca
Raton, FL, USA, 2011.
Kristian Rautiainen. Kanban for software development. In Ville T. Heikkilä,
Kristian Rautiainen, and Jarno Vähäniitty, editors, Towards Agile Product
and Portfolio Management, chapter 12, pages 184–192. Aalto University,
Helsinki, Finland, 2011. URL http://www.soberit.hut.fi/sprg/projects/
atman/TowardsAgileProductandPortfolioManagement.pdf. Retrieved Apr. 4th,
2014.
Kristian Rautiainen, Casper Lassenius, and Reijo Sulonen. 4cc: A framework
for managing software product development. EMJ - Engineering Management
Journal, 14(2):27–32, 2002.
Guenther Ruhe and Moshood Omolade Saliu. The art and science of software
release planning. Software, IEEE, 22(6):47–53, 2005.
Per Runeson and Martin Höst. Guidelines for conducting and reporting case
study research in software engineering. Empirical Software Engineering, 14(2):
131–164, 2009.
Ken Schwaber. Agile project management with Scrum. Microsoft Press, Redmond,
WA, USA, 2004.
Ken Schwaber. The enterprise and Scrum. Microsoft Press, Redmond, WA, USA,
2007.
Ken Schwaber and Mike Beedle. Agile software development with Scrum. PrenticeHall, Upper Saddle River, NJ, USA, 2002.
Ken Schwaber and Jeff Sutherland. The Scrum guide - the deﬁnitive guide to
Scrum: The rules of the game. Technical report, Scrum.org, July 2013. URL
http://www.scrum.org/Scrum-Guides. Retrieved Apr. 4th, 2014.
76
Bibliography
Clive Seale. Quality in qualitative research. Qualitative Inquiry, 5(4):465–478,
1999.
William R. Shadish, Thomas D. Cook, and Donald T. Campbell. Experimental and
quasi-experimental designs for generalized causal inference. Houghton Mifﬂin,
Boston, MA, USA, 2001.
Dag I. K. Sjøberg, Tore Dybå, and Magne Jørgensen. The future of empirical
methods in software engineering research. In Future of Software Engineering (FOSE’07), pages 358–378, Washington, DC, USA, 2007. IEEE Computer
Society.
Ian Sommerville. Software engineering. Addison-Wesley, 6th edition, 2001.
Mikael Svahnberg, Tony Gorschek, Robert Feldt, Richard Torkar, Saad Bin
Saleem, and Muhammad Usman Shaﬁque. A systematic review on strategic release planning models. Information and Software Technology, 52(3):
237–248, 2010.
Inge van de Weerd, Sjaak Brinkkemper, Richard Nieuwenhuis, Johan Versendaal,
and Lex Bijlsma. Towards a reference framework for software product management. In Proceedings of the IEEE International Conference on Requirements
Engineering (RE ’06), pages 312–315, 2006.
Guus van Waardenburg and Hans van Vliet. When agile meets the enterprise.
Information and Software Technology, 55(12):2154–2171, 2013.
VersionOne, Inc. 7th annual "state of agile development" survey. Technical
report, 2013. URL http://www.versionone.com/pdf/7th-Annual-State-ofAgile-Development-Survey.pdf. Retrieved Apr. 4th, 2014.
Jarno Vähäniitty. The gap in the literature. In Ville T. Heikkilä, Kristian Rautiainen, and Jarno Vähäniitty, editors, Towards Agile Product and
Portfolio Management, chapter 3, pages 38–51. Aalto University, Helsinki,
Finland, 2011.
URL http://www.soberit.hut.fi/sprg/projects/atman/
TowardsAgileProductandPortfolioManagement.pdf. Retrieved Apr. 4th, 2014.
Jarno Vähäniitty and Kristian Rautiainen. Towards a conceptual framework and
tool support for linking long-term product and business planning with agile
software development. In Proceedings of the 1st international workshop on
software development governance, pages 25–28. ACM, 2008.
Kevin Vlaanderen, Slinger Jansen, Sjaak Brinkkemper, and Erik Jaspers. The
agile requirements reﬁnery: Applying SCRUM principles to software product
management. Information and Software Technology, 53(1):58–70, 2011.
Claes Wohlin, Martin Höst, and Kennet Henningsson. Empirical Research Methods in Software Engineering, pages 7–23. Empirical Methods and Studies in
Software Engineering — Experiences from ESERNET. Springer, 2003.
Robert K. Yin. Case study research : design and methods, volume 5 of Applied
social research methods series. Sage Publications, Thousand Oaks, CA, USA,
4th edition, 2009.
77
Bibliography
78
9HSTFMG*agaefd+
I
S
BN9
7
89
5
2
6
0
6
0
4
5
3(
p
ri
nt
e
d
)
I
S
BN9
7
89
5
2
6
0
6
0
4
6
0(
p
d
f
)
I
S
S
N
L1
7
9
9
4
9
34
I
S
S
N1
7
9
9
4
9
34(
p
ri
nt
e
d
)
I
S
S
N1
7
9
9
4
9
4
2(
p
d
f
)
A
a
l
t
oU
ni
v
e
r
s
i
t
y
S
c
h
o
o
lo
fS
c
i
e
nc
e
D
e
p
a
r
t
me
nto
fC
o
mp
ut
e
rS
c
i
e
nc
ea
ndE
ng
i
ne
e
r
i
ng
w
w
w
.
a
a
l
t
o
.
f
i
A
al
t
o
D
D6
/
2
0
1
5
T
h
ego
alo
f so
ft
w
arere
l
e
asepl
anning is t
o
pl
an t
h
eappro
priat
esc
o
pefo
rt
h
ene
xt
re
l
e
asew
h
il
et
aking int
oac
c
o
untt
h
e
avail
abil
it
yo
f re
so
urc
e
s, t
h
eprio
rit
ie
so
ft
h
e
c
andidat
ere
quire
me
nt
s and any o
t
h
e
r
fac
t
o
rs and c
o
nst
raint
st
h
ataffe
c
tt
h
e
re
quire
me
nt
s se
l
e
c
t
io
n.P
re
vio
us re
se
arc
h
h
as fo
c
usse
do
n al
go
rit
h
micso
l
ut
io
ns t
h
at
e
xpe
c
tt
h
ere
quire
me
nt
s se
l
e
c
t
io
nf ac
t
o
rs
and re
l
e
asec
o
nst
raint
st
obekno
w
n.I
n
re
al
it
y, t
h
efac
t
o
rs and c
o
nst
raint
s arerare
l
y
kno
w
n be
fo
re
h
and.Mo
de
rn, agil
eso
ft
w
are
de
ve
l
o
pme
ntme
t
h
o
ds arebase
do
n rapid
de
ve
l
o
pme
ntc
yc
l
e
s and c
o
nst
ant
c
o
mmunic
at
io
n be
t
w
e
e
n st
ake
h
o
l
de
rs
inst
e
ad o
f pl
anning and pl
an e
xe
c
ut
io
n.T
h
is
disse
rt
at
io
n pre
se
nt
st
h
eﬁrstst
e
ps t
o
w
ards
c
l
o
sing t
h
egap be
t
w
e
e
n re
l
e
asepl
anning
re
se
arc
hand prac
t
ic
eby de
sc
ribing t
h
re
e
c
ase
so
f re
l
e
asepl
anning in agil
eso
ft
w
are
de
ve
l
o
pme
nto
rganiz
at
io
ns.
BU
S
I
N
E
S
S+
E
C
O
N
O
M
Y
A
R
T+
D
E
S
I
G
N+
A
R
C
H
I
T
E
C
T
U
R
E
S
C
I
E
N
C
E+
T
E
C
H
N
O
L
O
G
Y
C
R
O
S
S
O
V
E
R
D
O
C
T
O
R
A
L
D
I
S
S
E
R
T
A
T
I
O
N
S