2012 Joint Conference of the 22nd International Workshop on Software Measurement and the 2012 Seventh International
Conference on Software Process and Product Measurement
Quality of Estimations
How to Assess Reliability of Cost Predictions
Dr. Thomas Fehlmann
Eberhard Kranich
Euro Project Office AG
Zurich, Switzerland
e-Mail: [email protected]
Processes, Quality & IT (PQIT)
T-Systems International GmbH
Bonn, Germany
e-Mail: [email protected]
suggest that our inability of predicting software project cost is
the main reason for the inability of today’s industry to create
enough value to pay for the debt bills of industrialized nations.
Instead of great news about high returns from investments in
integrated ICT systems, daily news talk about the next round of
debt crisis meetings.
Abstract— Software Project Cost Prediction is one of the unresolved problems of mankind. While today’s civil engineering
work is more or less under control, software projects are not.
Cost overruns are so frequent that it is wise never trusting any
initial cost estimate but take precaution for higher cost.
Nevertheless, finance managers need reliable estimates in order
to be able to fund software and ICT projects without running
risks. Estimates are usually readily available – for instance based
on functional size and benchmarking. However, the question how
reliable these estimations are is often left out, or answered in a
purely statistical manner that gives no clue to practitioners what
these overall statistical variations means for them.
A. Why is Software Cost Estimation so difficult?
Software engineering is not civil engineering, where you
first create a plan then execute the plan, and all you need to do
is making sure the plan takes all eventualities into due consideration! Software has to explain complex tasks in a language
simple enough such that ICT systems are able to understand
and execute it correctly. It’s a translation process starting with
some actual processes, some explicit and many more implicit
requirements, involves social behavior, organizational capability maturity, ability to communicate, to formulate in different
industry-specific languages, of keeping trust and continual engagement that eventually ends in an integrated men-machine
system creating value.
This paper explains how to make use of Six Sigma's transfer
functions that map cost defined by a committee of GUFPI-ISMA
onto project cost. Transfer functions reverse the process of estimation: they show how much a project costs under suitable assumptions for the cost drivers. If cost drivers can be measured,
and transfer functions can be determined with known accuracy,
not only project cost can be predicted but also the range and
probability for such cost to occur.
Railways are less difficult to operate, and traffic jams are
easier to avoid than make software run as expected.
Keywords— Project Cost Estimation, Cost Drivers, Transfer
Function, Soft Skills, Lean Six Sigma
I.
B. Types of Software Project Cost Estimation
Since the early days, when manual coding was still the
main task of developers, software project managers had attempted to predict software project costs by detail analyzing
tasks and duration. It was the time of sophisticated methodologies based on detailed Work Breakdown Structures (WBS).
While a WBS helped management understanding complexity
of software development, it turned out to be unreliable in predicting the actual work needed. Nevertheless, effort predictions
based on work breakdown structures weren’t all bad – even if
they tended to predict other work than that actually needed to
complete the project. The major problem with these approaches
is that they do not reflect the nature of software development –
namely uncover what needs to be done to complete the project.
INTRODUCTION
Today’s economies heavily suffer from the problems to
conduct software projects within reasonable time and budget
constraints. Due to this, many administrative and legally relevant processes still rely on complicated and error-prone manual
and paper-based procedures, for instance for voting, health
insurance, consumer billing, intercompany transactions; and
eGovernment is still a kind of dream, sixty years after it became technically possible to transport information electronically. Not many inventions in mankind’s history took that long to
take effect! For instance, the steam engine took less than twenty years until railways crossed all Europe; between the construction of the 1885 Daimler/Maybach Petroleum Reitwagen
(Riding Car) and the first mention of traffic jams in The San
Francisco Call of October 20, 1904, there were less than twenty
years.
A recent reaction to WBS-based project estimations is agile
development – not planning for the work details but for allocating the time slots needed to complete the project, and do the
work in fixed-time increments, e.g., sprints.
Obviously, it was less risky to fund railways and car factories than today’s software industry. Consequently, today’s ICT
world is a world of gaming, chatting, entertainment and consumption rather than one of value creation. It is allowed to
978-0-7695-4840-1/12 $26.00 © 2012 IEEE
DOI 10.1109/IWSM-MENSURA.2012.11
However, the most popular approach to cost estimation is
benchmarking – based on own experiences or comparisons
with industry. Because benchmarking always suffered from the
8
The ISBSG database comes with a large list of project attribute parameters that are used to compare different projects:
industry, choice of modeling and coding language, team size,
usage characteristics, methodology approach, architecture, target platform. The database is filtered for large, medium or
small functional size, development platform, and development
type (new, enhancement, re-development, or customization).
Nevertheless, variations between functional size count and
effective effort needed are significant, as shown in [7]. Based
on a sample of 16 MIS Projects in R10 Database of 2009, with
similar project attributes, there is almost no correlation between
functional size and actual effort reported, see Figure 1. On the
contrary, if actual efforts of the same projects are analyzed
using parametric approach based on cost drivers, the actual cost
can be explained perfectly, see Figure 2.
difficulties of collecting reliable and comparable data, we also
consider the so-called Expert Estimation approach as kind of
benchmarking – instead of numerical database using the memories of experienced developers. Expert estimation is particularly successful in agile – collecting Story Points for sizing
software projects and allocating enough sprints [5].
C. The ISBSG Benchmarking Database
Among the numerical database collections, the ISBSG database is certainly the most popular one [11]. It’s an open collection of software development and maintenance projects collected all over the world and across all kind of industries.
While other such collections exist, e.g., the proprietary QSM
collection, none has the advantage of open, standardized and
controlled collection practices. It is relatively easy to estimate a
project – once its functional size is known, that is, when it’s
clear what needs to be build.
II.
PARAMETRIC APPROACHES TO COST PREDICTION
A. Cost Drivers
Parametric approaches are the most promising for predicting software project cost. The idea was made known by Barry
Boehm in 1981 when he started publishing the range of
COCOMO prediction models [4]. He based project estimation
on a number of cost drivers.
900 FP
800 FP
700 FP
600 FP
500 FP
Person Days (PD)
400 FP
300 FP
200 FP
100 FP
0 FP
0
100
200
300
400
500
600
700
800
900
days
Figure 1: ISBSG MIS Projects: Function Points vs. Actual Effort
predicted days
900
800
Cost Driver
700
Low
600
High
Figure 3: Cost Drivers with Different Slopes
500
400
Each of such cost driver functions has a different slope that
models how the cost driver influences overall effort. The slope
is referred as a-Parameter; the selected cost driver impact is
denoted by x. Boehm used some kind of general system characteristics such as: requirements volatility, functional sizing,
technical complexity, impact on current application, communication needs, and so one. These cost drivers are intuitively easy
to understand but behave differently in the requirements elicitation, in the design & development phases, or for application
testing or documentation. Moreover, the cost drivers may behave differently among different products. Total effort prediction is a function of these cost drivers.
300
200
100
0
Medium
0
200
400
600
800
1000
actual days
Figure 2: Cost Driven Estimations vs. Actual Effort
Unfortunately, this happens relatively late in the project,
namely when requirements are known up to a certain degree
and at a defined granularity level. At this point of time, a large
part of the project budget has possibly already been spent to
find out what these functional user requirements eventually
actually are. Nevertheless, for solution design, model-driven
development, scope management of projects, and defect prediction and planning of software operations: functional sizing
is the method of choice.
These cost drivers are good candidates for predicting the effort needed to implement the project tasks. Boehm characterizes project cost by a Cost Driver Profile. Users of the model use
a discrete scale marked with “Low”–“Medium”–“High” characterizing the cost driving force. What the medium is must be
9
defined: it should be fixed such that profiles remain comparable. Thus there is a need to state measurable standard value
ranges for medium profile values, e.g., for team size or for
people’s skills against which comparison is possible. Ideally,
cost drivers should be measurable; however, only quite rough
assessments are usually available for soft factors such as “skills
level”, or “need for communication”.
effort response vector is five times the number of projects considered for process measurement; possibly a few dimensions
less if not all projects cover all five effort types.
The impact of cost drivers varies among the development
stages. Thus, as Figure 3 shows, we may have different impact
even of the same cost driver, depending on the product.
( ) = C. The Estimation Formula
Barry Boehm uses exponential functions for the impact of
cost factors1:
where represents the slope of the cost driver and defines the impact of the cost driver , see Figure 3. The
products refer to the cross-point values for the impact
function ( ). The practical reason for taking an exponential
function is that you don’t have to care for dimensions nor for
any static minimum cost; experience shows, on the other hand,
that cost factors have a tendency to soaring when they start
growing. The a-parameter takes care of all that. The difference
between low impact and medium impact is much less than between medium to high impact. High impact has almost no upper limit, whereas low impact always has a limit: a minimal
cost associated to it.
B. Measuring the Response of the Software Project Process
We need to analyze the response of our process in a way
that allows distinguishing the various contributions from the
cost drivers. An obvious choice is looking at cost per phase: for
instance, distinguishing cost of requirements elicitation, analysis & design, technical implementation, solution integration,
and start of operation phases already allow for analyzing impact of various cost drivers that relate to people and requirements volatility. Another approach is based on the five CMMI
process areas Requirements Development (RD), Technical Solution (TS), Quality Assurance (QA), Product Integration (PI),
and Project Management (PM); however, as the experience of
ISBSG shows, it is very difficult to get reliable results for
phases across organizations, since different, internally developed and applied methodologies are common.
Barry Boehm combines those individual cost driver effects
by multiplication with an exponential factor:
D. Combining with Functional Size
If the model contains more than just one cost driver that
depends from functional size, we cannot use the above formula
(ii). However, since modules should not interfere with each
other, an additive model is more appropriate than the multiplication of influential factors. Let ( ) denote the impact of
functional size where the index1 ≤ ≤ . The functional contributions ( ) sum up for the Functional Cost Driver FD:
TABLE I: MEASURABLE EFFORT TYPES
Tester
Admin
Project Manager
DoIt
Test
Adm
(ii)
where n is the number of cost drivers, and () =
(〈 , , … , 〉) is the total cost influence profile per estimation item for the cost driver profile = 〈 , , … , 〉 .
Note that the represent low–medium–high and can be set
without loss of generality to some equally distanced values
around 1.0 : = 0.5, 1.0, and 1.5 respectively. No impact
means = 0, thus = 1. The Impact Function () can
be calculated based on the cost profile vector 〈 , , … , 〉 for
each cost driver vector that represents an estimation item.
The project process response should be measured by efforts
spent for a few relevant effort components shown in TABLE I.
Talk
Meetings
Meetings,
Chat
Meetings,
Chat
Meetings,
Chat
Meetings,
Chat
() = ( ) = In view of practicality, effort data should be collected as
closely to roles and physical evidence as possible, in order to
allow for comparisons among different organizations and
methodologies. We propose to distinguish effort spent for requirements elicitation in team and stakeholder communications
(Talk), work and rework needed (DoIt), reviews and tests conducted (Test), time needed for technical and financial project
administration (Adm), e.g., documentation, configuration management, time and records keeping, and project management
(PM). Since this kind of effort data is effort spent by the roles
sponsor, developer, tester, administrators, and project managers, it is easier to collect and allows getting more reliable effort
data. Note that cost drivers impact all effort types in the same
way, e.g., high, however a-parameters have different slope.
Roles\Effort Types
Sponsor
Developer
(i)
PM
() = ( )
Design,
Code
(iii)
Integrate,
QA
Exponentiation of the functional contributions ( ) also
yields excellent results, see [7], but makes the model unnecessarily complex. With only one functional cost driver ,
Enable,
Track
Manage
() = (iv)
fixes the logarithmic base for .
The profile vector for the project effort response thus runs
over two levels: over the number of projects estimated or effort-measured, and for each project over the five effort types:
Talk, DoIt, Test, Adm, and PM. These points of measurement
are called Estimation Items. Total dimension of the project
For instance, can be selected such that, say, 512
COSMIC Function Points correspond to = 1; the exponen1
10
Note that Barry Boehm uses and the other way round, see [7].
Six Sigma
Estimation
One Sigma
Estimation
Tolerance
Range
6V
Estimation
Items
Tolerance
Range
Standard
Deviation 1V
Standard
Deviation 1V
6V 5V 4V 3V 2V 1V 1V 2V 3V 4V 5V 6V
6V
FIGURE 4: ONE SIGMA AND SIX SIGMA ESTIMATIONS DEPEND ON THE VARIANCE IN THE ESTIMATION STACK
tial factor indicates the cost driving impact of functional
size.
jects, it is not sufficient if some mature organization keeps collecting effort data and profiling their projects, its customer
must have the possibility to compare and validate those calibration data.
The Calculated Effort in Person Days (PD) for an estimation item with cost driver profile is therefore
While the first problem can be solved in high maturity organizations, see e.g., [7], the new GUFPI-ISMA cost driver
catalogue is a big step towards addressing the third issue, see
section V. The remaining part of this paper focuses on the second problem: how to assess quality of estimations.
() = () ∗ () = (v)
This is a simplification compared to (ii).
E. Calibration
The cost driver vector = 〈 , , … , 〉 represents by the impact of functional size and by , … , the impact of
non-functional cost drivers. If there are enough estimation
items with cost driver profile for which () is known, it is
possible to calculate the a-parameters by multi-linear regression. The a-parameters hold for a series of similar estimation
items. Since the low/medium/high cost driver profiles need to
be taken into account, and since we also allow for no impact in
the profile, at least 4 estimation items – with each cost driver
once in no, low, medium and high profile state – are necessary
for calibration. Such as set of estimation items with known
() is called an Estimation Stack. However, the more estimation items are available, the better for reducing measurement errors by redundancy.
III.
TRANSFER FUNCTIONS
A. Estimation Stacks as Transfer Functions
An estimation stack represents a transfer function that maps
the cost driver vector profile onto an "-ary actual estimation
item efforts vector #$ = 〈 (), (), … % ()〉 using
(ii) for & = 1, . . ". This vector constitutes the response of the
process of creating an estimation stack for the " estimation
items, each estimation item row depending from their cost
driver profile ' = 〈,' , ,' , … , ,' 〉:
#$ = *() = 〈 ,+ , ,- , … , ,/ 〉
(vi)
This transfer function *() can be used for predicting project cost, based on the settings for the cost driver profiles. Note
that if the cost driver profiles remain restricted to discrete values, such as = 0.0, 0.5, 1.0, and 1.5, the number of estimations for a stack is limited to the permutation of all possible
cost driver profiles, thus to 4 possible response predictions –
zero, low, medium, or high. Thus, every response of this cost
prediction model comes with a known variation, with known
accuracy. However, intermediate values for the are also feasible.
So, if cost prediction for software projects is that easy, why
isn’t it current successful standard practice?
F. Quality of Predictions
There are a few problems. The first is certainly the data collection used for calibration. Very few organizations are mature
enough to collect their project data, know their cost drivers,
and keep them under control for a long enough time to successfully predict project cost. And even if data can be collected,
how do we know how accurate the calibration data is? This is
the second problem. Collecting actual data is significantly
more difficult than collecting expert estimations. That’s the
reason why many organizations rely on expert estimations rather than on actual data when calibrating their estimation
stacks. However, the third problem is probably the most intrinsic: since cost prediction is necessary for contracting ICT pro-
B. Selecting Cost Drivers
Transfer functions map process controls into process responses – not the other way round. Since responses are typically known first, before the relevant critical controls, predicting
the critical controls is a relevant issue for understanding transfer functions for processes.
11
estimation items in the stack. It is unclear what happens when
using the stack for predicting new projects. To understand how
to assess quality on an estimation stack for prediction, we need
to turn somewhat more into theory of transfer functions.
Both process controls and process responses are vectors in
a multidimensional event space, namely the space of suspected
cost drivers for the project delivery process. Cost drivers have a
value; they are more or less important. Cost drivers should be
orthogonal to each other, that is, one cost driver value must not
depend from other cost driver values. The condition is that the
value of one cost driver never depends from any combination
of other cost drivers.
D. Analyzing Transfer Functions for Software Projects
The cost driver profiles ' = 〈,' , ,' , … , ,' 〉 that define
the cost for the &
estimation item using the estimation function
(v) yield the matrix 9 = :,' ; of dimensions ( + 1) × " .
denotes the number of cost drivers as before; " is the size of
the estimation stack. Let #$> denote the vector obtained by
actual measurement of all " estimation items. Obviously #$> ≠ #$ but, if the model is capable, #$> ≅ #$ holds.
The aim is to predict how capable the model based on the chosen cost driver profile actually is.
However, cost drivers can compensate each other: if one
cost driver has no impact, other cost drivers might provide the
necessary impact to yield the observed process response.
C. Quality of Estimation Stacks
After calibration, i.e., calculation of the a-parameter by
multi-linear regression, the quality of an estimation stack can
be measured by its variation 3, as seen in Figure 4. This is assuming that project effort follows normal distribution. The
American Association of Cost Engineers has recently put this
into question, see [2], suggesting a double triangular distribution which is skewed to allow for larger cost overruns than
undercuts. In this case, a left-side 36 and a right-side 37 should
be used.
The transfer function can be linearized by looking at the
cost driver. The matrix A = :,' ; defines a linear mapping
B = A = 〈C , … C% 〉 where C' = ,' The vector B = 〈C , … , C% 〉 is called Effort Profile Vector.
With a sufficiently large number of cost drivers, it is always
possible to find suitable a-parameter. Note that positive
a-parameter increase cost; negative parameters would decrease
cost. Sometimes this is not straightforward. For instance, if you
add as a cost driver “Need for extensive documentation”, it is
not clear whether this increases or decreases cost. It might decrease cost of quality assurance and thus of effort type “Test”
but increase effort spent for “DoIt”.
The cost driver profiles ,' must not contradict each other;
this means any pair of the cost profiles must drive cost either
up or down. If some projects react on some cost driver with
cost increase, and others with otherwise same settings behave
differently, it won’t be possible to calculate the a-parameter by
regression analysis. In other words, regression analysis will
yield weird results without giving any hint.
Note that the cost ' of the &
project is not a function of
the organization’s effort profile component C' but of the cost
driver profile components ,' – fixed for the &
project – and
the cost drivers according formula (v). Thus the vector B is not
directly measurable; it only can be determined indirectly by
measuring cost of estimation item ' (), then calculating The sigma value can also be expressed in terms of confidence intervals; a suitable metric for getting the right kind of
management attention. For instance, a variation of 3 = 3.5
corresponds to 99.8% confidence based on the 95th percentile.
However, even if the estimation stack has high confidence,
it only demonstrates that the selected cost drivers model the
Prediction Accuracy #$D − * E #$
#$D : Observed Response of the Process
• Cost Measured
#$ : Predicted Response by Cost Drivers
• Predicted Cost
The Response
#$ = F(x)
Talk Effort
DoIt Effort
Test Effort
Adm Effort
PM Effort
“Analysis”
G
“Transfer”
Cost Driver x5
Cost Driver x4
Cost Driver x3
Cost Driver x2
The Controls x
Cost Driver x1
(vii)
FIGURE 5: OBSERVED RESPONSE AND EXPLAINED RESPONSE
12
F
for the full estimation stack #$, and finally using (vii) to derive B = A. The effort profile vector B is characteristic for the
vector #$ by using (vii).
F. The Quality Criteria for Cost Driver
Let A = :,' ; be the cost profiles matrix as before. The
difference between the effect profile vector AA⊺ BK obtained
from the eigenvector BK and the effect profile B = A obtained
from the cost driver profile is called Convergence Gap:
Characteristic effort profile vectors also exist for the measured (or expert-estimated) estimation items vector #$> , denoted by B> . Clearly, if B> ≅ B, then also #$> ≅ #$. However, given #$> , the effort profile vector B> cannot be easily
measured or calculated.
‖B − BK ‖ = ‖B − AA⊺ BK ‖
(viii)
⊺
Assume the eigenvalue I = 1 in AA BK = IBK . This vector difference (viii) is an indicator for the Prediction Accuracy,
the minimum difference between model estimations and actual
cost measurements:
E. Eigenvectors as Quality Criteria
An Eigenvector H of a square matrix > has the characteristic property that >H = IH; I is called its Eigenvalue. By normalization of H, the eigenvalue can be assumed to I = 1. The
existence of an eigenvector means that repeated application of
the square matrix > keeps the result stable. This is an indication that the measurements are not at random but stable. Eigenvectors therefore are used to level out measurement errors; this
is common practice in physics but also in decision theories like
the Analytic Hierarchy Process (AHP) of Saaty [14] and in
Google’s search algorithms [10].
M#$D − *:E(#$);M
(xi)
Consult [6], [8] and [12] for how to use eigenvectors of
transfer functions for validating cause and effect relations; for
application of eigenvectors with large-dimensional vector
spaces see [10], and [13] introduces the general theory. Compare Figure 6 for a visualization of the convergence gap in the
case of only 3 cost drivers, according an idea presented by
Schurr in [16] when explaining how AHP works.
Assume some expert has characterized an estimation stack
by its cost driver profiles and denote this analysis function
by E; thus = E(#$> ) and #$ = *(). Let B = A be as
defined in (vii). Let A⊺ denote the transpose of the matrix A. A⊺
is called dual as it reverses the cause-effect direction of cost
drivers, thus eliminating errors in cost driver assessments. For
an eigenvector BK , AA⊺ is the inverse, thus = A⊺ BK or
A = AA⊺ BK . The matrix AA⊺ is positive definite and diagonalsymmetric by construction and thus has real eigenvectors.
The calculation of eigenvectors is easily possible with any
industry standard linear algebra package; this is not a topic for
this paper. Thus eigenvector theory validates the choice of cost
drivers but cannot ascertain their correct label and meaning.
Note that the eigenvector calculation does not replace the regression analysis needed to get the a-parameters. It enhanced
their calculation by removing inconsistencies in the cost profiles, thus improving quality of cost driver profiling.
Convergence
Gap small
Convergence
Gap large
Eigenvector
Eigenvector
Cost Driver
Profile Vectors
Cost Driver
Profile Vectors
See Schurr, 2011
an
FIGURE 6: SMALL AND LARGE CONVERGENCE GAP FOR THREE COST DRIVERS (SCHURR [16])
Therefore, if B is near to an eigenvector of AA⊺ , i.e.,
‖BK − B‖ ≅ 0, then the cost driver profile matrix A = :,' ;
defines a stable estimation system in the sense that there are no
contradicting cost profiles, and thus calculating a-parameters
by regression analysis is safe for estimation items. The transpose A⊺ B predicts the solution for the equation A = B, eliminating contradictions injected by estimators. Such an estimation stack meets the quality criteria for model estimations
shown in Figure 5 and can be used to predict other projects that
rely on cost drivers used for the estimation stack.
G. Research Topic: Benefits for Estimation Stacks
The eigenvector property (viii) allows categorizing estimations stacks according the eigenvector criteria. Since many
eigenvectors for AA⊺ exist, it is possible to select one with the
minimum number of cost drivers. For this, it suffices to look at
the vector A⊺ B and identify those cost drivers components
(A⊺ B) whose impacts are close to zero. Thus it seems possible
to create estimation stacks with a limited selection out of the
possible cost driver factors, taking only those that eventually
impact the total cost estimate. This reduces the effort needed
13
TABLE II: GUFPI-ISMA PRODUCTIVITY IMPACT FACTORS
Personal
H1
H2
H3
H4
H5
H6
Process
Domain Knowhow
Personnel Capability
Technology Knowledge
Team Turnover
Management Capability
Team Size
P1
P2
P3
P4
P5
P6
P7
P8
P9
Product
Organization Maturity
Schedule Constraints
Requirement Completeness
Reuse
Project Type
Methodology
Stakeholder Cohesion
Project/Program Integration
Project Logistics
S1
S2
S3
S4
S5
S6
S7
However, this conjecture is yet under investigation and no
practical experience with this method is known so far.
LIMITATIONS
[1]
The consistency check for the matrix A = :,' ; cannot validate the semantics of the cost drivers ; it only limits the difference (xi). The labels must be determined by identifying the
meaning of the cost driver in the real world. However, the
choice of cost drivers can be ascertained in the following sense:
if it is possible to find estimation stacks that allow for consistent selection of cost drivers.
V.
[2]
[3]
[4]
[5]
THE GUFPI-ISMA PRODUCTIVITY IMPACT FACTORS
Since 2009, a working group of GUFPI-ISMA has collected from various sources, including COCOMO and ISBSG, the
cost drivers suspected to account for soft project cost. The advantage of such a collection – if accepted internationally and
used for profiling software projects – is that cost estimations
become comparable between organizations. Such an achievement is probably among the most important in information
technology since van Neumann stored code and data on the
same device.
[6]
[7]
[8]
[9]
However, up to now it is not known whether these cost
drivers are able to explain the observed cost in ICT projects.
Some preliminary research has just started. Also, it is not clear
how such cost drivers can be measured in a repeatable, unambiguous way across different organizations and possibly even
across different types of software projects. Establishing an estimation stack for the Productivity Impact Factors (PIF) is difficult because of the large number of cost drivers. To calculate
the a-parameter for the PIFs, it would be helpful to have sample but representative project data with cost drivers varying for
a few drivers only, not indiscriminately for all.
VI.
Technology
T1
T2
T3
T4
T5
Programming Language
Development Tools
Technical Environment
Technology Change
Technical Constraints
Although the method cannot verify that the selected cost
drivers are correct, it can ascertain that their measurements are
consistent and that the estimation model can be duly used for
predicting project cost. The convergence gap between model
prediction and actual project cost indicates how good the estimation stack used for cost prediction actually is.
for measuring or agreeing on cost driver profiles, and allows
creating estimation stacks for different categories of projects.
IV.
Product Size
Product Architecture
Product Complexity
Other Product Properties
Required Documentation
System Integration
Required Reusability
[10]
[11]
[12]
[13]
CONCLUSION
[14]
The concept of transfer functions is very powerful, and easily adaptable to software development cost estimations. We
have laid the theoretical background for quality of estimations;
the practical implementation is yet another challenge. The reward is huge: reliable project cost estimations, or at least estimations with a known accuracy, will help to develop the information and communication technology to bring the economic benefits that it promised long ago.
[15]
[16]
14
Abran, A. et al., “The COSMIC functional size measurement method Version 3.0.1 - Measurement manual,” COSMIC Corp., Montréal,
Canada (2009)
American Association of Cost Estimators, “Recommended Practice No.
41R-08,” AACE, Inc. (2008)
Buglione, L., Trudel, S.: “Guideline for sizing agile projects with
COSMIC,” Proceedings of the IWSM / MetriKon / Mensura 2010,
Stuttgart, Germany (2010)
Boehm, B.: “COCOMO II,” Addison-Wesley, New York, NY (2002)
Cohn, M.: “Agile estimating and planning,” Prentice Hall, New Jersey
(2005)
Fehlmann, Th., “The impact of linear algebra on QFD,” International
Journal of Quality & Reliability Management, Vol. 21 No. 9, pp. 83-96,
Emerald Group Publishing Ltd., Bradford, UK (2005)
Fehlmann, Th., “Using Six Sigma for project estimations – an
application of statistical methods for software metrics,” MetriKon 2009,
Kaiserslautern, Germany (2009)
Fehlmann, Th., Kranich, E., “Transfer functions, eigenvectors and QFD
in concert,” 17th International QFD Symposium, ISQFD 2011, Stuttgart,
Germany (2011)
Fenton, N.E., Neil, M., Marquez, D., “Using Bayesian networks to
predict software defects and reliability,” Proceedings of the Institution of
Mechanical Engineers, Part O, Journal of Risk and Reliability: p. 701712 (2008)
Gallardo, P. F., “Google's secret and linear algebra,” EMS Newsletter
63, March 2007, 10-15, Universidad Autónoma de Madrid, Spain (2007)
Hill, P. ed., “Practical software project estimation 3rd edition,” McGrawHill, New York, NY (2010)
Hu, M. and Antony, J., “Enhancing design decision-making through
development of proper transfer function in design for Six Sigma ,”
International Journal of Six Sigma and Competitive Advantage 3, 2007,
pp. 33-55 (2007)
Kressner, D., “Numerical methods for general and structured eigenvalue
problems,” Lecture Notes in Computational Science and Engineering,
vol. 46. Springer-Verlag, Heidelberg. Germany (2005)
Saaty, Th., “Decision-making with the AHP: Why is the principal eigenvector necessary,” European Journal of Operational Research vol. 145,
pp. 85--91, Elsevier Science B.V. (2003)
Santillo, L., Moretto, G., on behalf of SBC (GUFPI-ISMA), “A general
taxonomy of productivity impact factors,” Software Benchmarking
Committee, Gruppo Utenti Function Point Italia – Italian Software
Metrics Association, IWSM-MENSURA, Stuttgart (2010)
Schurr, S., “Evaluating AHP Questionnaire Feedback with Statistical
Methods,” 17th International QFD Symposium, ISQFD 2011, Stuttgart,
Germany (2011)