Mar 21, 2010  Often, the staffing level is not changed continuously in a project and approximations of the Rayleigh curve are used: assigning a few people at the start, having the peak team during the coding phase, and then leaving a few people for integration and system testing.

• Rodney Allen
• 3 years ago
• Views:

Transcription

1 Resource Estimation in Software Engineering Lionel C. Briand Carleton University Systems and Computer Engineering Dept. Ottawa, ON K1S 5B6 Canada Isabella Wieczorek Fraunhofer Institute for Experimental Software Engineering (IESE) Sauerwiesen Kaiserslautern International Software Engineering Research Network, Technical Report ISERN

2 Resource Estimation in Software Engineering Lionel C. Briand and Isabella Wieczorek 1 Introduction This paper presents a comprehensive overview of the state of the art in software resource estimation. We describe common estimation methods and also provide an evaluation framework to systematically compare and assess alternative estimation methods. Though we have tried to be as precise and objective as possible, it is inevitable that such a comparison exercise be somewhat subjective. We however, provide as much information as possible, so that the reader can form his or her own opinion on the methods to employ. We also discuss the applications of such estimation methods and provide practical guidelines. Understanding this article does not require any specific expertise in resource estimation or quantitative modeling. However, certain method descriptions are brief and the level of understanding that can be expected from such a text depends, to a certain extent, upon the reader s knowledge. Our objective is to provide the reader with a comprehension of existing software resource estimation methods as well as with the tools to reason about estimation methods and how they relate to the reader s problems. Section 2 briefly describes the problems at hand, the history and the current status of resource estimation in software engineering research and practice. Section 3 provides a comprehensive, though certainly not complete, overview of resource estimation methods. Project sizing, an important issue related to resource estimation, is then discussed in Section 4. Section 5 defines an evaluation framework that allows us to make systematic and justified comparisons in Section 6. Section 7 provides guidelines regarding the selection of appropriate estimation methods, in a given context. Section 8 describes typical scenarios for using resource estimation methods, thereby relating them to software management practice. Section 9 attempts to define important research and practice directions, requiring the collaboration of academia and industry. Section 10 then concludes this article by summarizing the main points made throughout the article. 2 Resource Estimation in Software Engineering This section briefly summarizes current practices regarding resource estimation in software engineering. This will shed light on the importance of the research being conducted and presented in subsequent sections. 2.1 Current Practices and Motivations The estimation of resource expenditures (e.g., effort, schedule) is an essential software project management activity. Inaccuracies in such estimations have repeatedly shown to lead to disasters (Heemstra, 1992). Delivering software on-time and within budget is a critical concern for organizations as underestimating software projects can have detrimental effects on business reputation, competitiveness, and performance (Putnam and Myers 1992; Lederer and Prasad, 1993). On the other hand, overestimated projects can result in poor resource allocation and ISERN

3 missed opportunities to take on other projects. Industry has recognized that by improving project estimation methods, problems with resource allocation and schedule forecasting can be overcome (Heemstra, 1992). Project managers commonly stress the importance of improving estimation accuracy and methods to support better estimates (Lederer et al., 1990, Lederer and Prasad, 1992). Accurate estimates are crucial for better project planning, tracking, and control and pave the way for successful project delivery. Software resource estimation is considered to be more difficult than resource estimation in other industries. Software organizations typically develop new products as opposed to fabricate the same product over and over again. This leads to many difficulties in estimating the resources of a project, especially in early project phases. Many studies have confirmed that a majority of projects overrun their budget (e.g., Jenkins et al., 1984; Heemstra, 1992; Lederer and Prasad, 1992; Lederer and Prasad, 1995). Practitioners are often faced with the lack of explicit cost and resource data collected from past projects (Heemstra, 1992). Moreover, estimators rely more on their personal memory and informal procedures than on documented facts, standards, or arithmetic rules (Hihn, 1991; Lederer and Prasad, 1992). Hence, estimates based on expert judgement and based on the available capacity prove to be quite popular (Heemstra, 1992). If actual data was recorded than only little time is spent to determine reasons for any difference between actual and plan data (van Genuchten, 1991). Many commercial packages are available on the market and one might be tempted to use some of them. However, experience has shown that one should not rely on software estimation tools for an accurate estimate, i.e., no difference was found between organizations using the tools and those not using the tools in terms of accuracy (Lederer and Prasad, 1992). The difficulties in software estimation are related to a variety of practical, measurement, and modeling issues. To address these issues, it is necessary to follow a systematic estimation process that increases the estimation quality and repeatability. A systematic estimation process may be supported by techniques, models, or/and tools. A number of resource estimation methods (i.e., a combination of techniques, models, tools) are currently available and they show contrasting strengths and weaknesses as well as different underlying assumptions. This makes it difficult to decide which one is best suited in a given context. To appropriately select one or more estimation methods, it is necessary to assess and compare their characteristics and their impact in practical estimation contexts. 2.2 History and Brief Overview The study of software resource estimation started as early as the late 1950 s and 1960 s (Norden, 1958; Nelson, 1966). (Brooks, 1975) raised concerns over gutless estimating whereby managers tended to produce wish-derived estimates and emphasized a need for an approach capable of justifying the estimate. The 1970 s proved to be a significant period of estimation methods development. Most of the early methods provided ready-to-use models, including predefined cost drivers that were then applied to obtain direct point estimates (Wolverton, 1974; Walston-Felix, 1977; Putnam, 1978; Albrecht, 1979). At that point, it became clear that there were difficulties in selecting cost drivers from an ever-increasing list of variables that were believed to influence software development effort. Practical experience has shown that early methods either emphasized project sizing, cost drivers, or expert judgement, but never a combination of the three (Conte et al., 1986). Hence, Conte stressed the need for methods that incorporate a combination of analytic equations, statistical data fitting, and expert judgement. ISERN

4 These methods take into account adjustments to nominal estimates made by experts. The bestknown method of this type, proposed by (Boehm, 1981), is the Constructive Cost Model (COCOMO). It provides equations that incorporate system size as the principal effort driver. Predicted development effort is then adjusted to accommodate the influence of 15 additional cost drivers. Other examples of this type of methods are SLIM (Putnam, 1978) and COPMO (Conte et al., 1986). In the 1980 s, widely used parametric methods (Putnam, 1978; Albrecht, 1979; Boehm, 1981; Bailey and Basili, 1981) were compared using data sets of various sizes and environments. Some of the main conclusions were that these models perform poorly when applied uncalibrated to other environments (Kitchenham and Taylor, 1985; Conte et al., 1986; Kemerer, 1987). Moreover, many estimation methods were automated and packaged into commercial tools (Stutzke, 1996). Software development is a complex dynamic process (Abdel-Hamid and Madnick, 1991). We know little about the complex, ever-changing relationships that explain variations in productivity. Therefore, the 1990 s saw the introduction and evaluation of non-parametric modeling techniques based on machine learning algorithms, such as Optimized Set Reduction (Briand et al., 1992), Artificial Neural Networks (Jørgensen, 1995; Finnie at al., 1997), CART regression trees (Srinivasan and Fisher, 1995; Kitchenham, 1998), and Analogy-based estimation (Mukhopadyay et al., 1992; Shepperd and Schofield, 1997; Walkerden and Jeffery, 1999). These methods typically make weak assumptions about the data and some of them produce models that can easily be interpreted and can accommodate complex relationships and interactions between cost drivers. They are therefore well suited to early exploratory investigations and theory construction. In addition, expert judgment and methods combining expert opinion with historical data have been investigated and compared. Examples are, subjective effort estimation (Höst and Wohlin, 1998; Stensrud and Myrtveit, 1998), modeling based on expert knowledge elicitation (Briand et al. 1998a), and techniques combining expert opinion and project data (Chulani et al., 1999). As discussed further below, such approaches are likely to be key in the future developments of the software resource estimation field. 2.3 Status and Main Obstacles Cost estimation techniques have drawn upon a variety of fields, statistics, machine learning, and knowledge acquisition. Given the diversity of estimation techniques one is faced with the difficult exercise of determining which technique would be the best in given circumstances. In order to assess a technique s appropriateness, the underlying assumptions, strengths, and weaknesses have to be known and its performances must be assessed. Homogeneous, company-specific data are believed to form a good basis for accurate resource estimates. Data collection, however, is an expensive, time-consuming process for individual organizations. There have been recent developments in standard data collection. The collaboration of organizations to form multi-organizational data sets provides the possibility for reduced data collection costs, faster data accumulation and shared information benefits. Their administrators offer a standard channel of data collection. Therefore, the pertinent question remains whether multi-organizational data are valuable to estimation methods. ISERN

5 3 Overview of Estimation Methods This section gives a general overview of the resource estimation literature classifies existing cost estimation methods into categories according to their underlying assumptions and modeling characteristics, and describes a selection of methods in more detail with a focus on effort estimation. There is a wealth of research addressing the software resource estimation problem. Research activities can be classed as: 1. Evaluation of effort estimation methods in different contexts. Investigations are aimed at (a) determining which method has the greatest effort prediction accuracy (e.g., Jørgensen 1995; Finnie et al., 1997; Walkerden and Jeffery, 1999; Briand et al., 1999b; Briand et al., 2000) (b) proposing new or combined methods that could provide better estimates (e.g., Conte et al. 1986; Briand et al., 1992; Schepperd and Schofield, 1997; Stensrud and Myrtveit, 1998). 2. Identification of significant cost drivers and productivity factors across different contexts (e.g., Subramanian and Breslawski, 1994; Maxwell et al., 1996; Briand et al., 1999a). 3. Assessment of current industry software development practices (e.g., Heemstra, 1992; Lederer and Prasad, 1992; Lederer and Prasad, 1998) 4. Calibration of effort estimation methods to tailor them to individual organizations (e.g., Miyazaki and Mori, 1985; Cuelenaere, 1987). General overviews and surveys of software cost estimation can be found in several papers. Stutzke (Stutzke, 1996) gives a chronological overview of generic cost estimation methods and tools, such as COCOMO (Constructive Cost Model), SLIM (Software Life Cycle Management), PRICE-S (Parametric Review of Information for Cost Evaluation), or Function Points (FP) 1. Kitchenham and Boehm give overviews and subjective evaluations of well-known cost models (Kitchenham, 1990; Boehm, 1981). Wakerden and Jeffery (Walkerden and Jeffery, 1997) give a comprehensive overview of the cost estimation process and its relation to the Quality Improvement Paradigm (Basili and Rombach, 1988), cost estimation models and practices. Lederer and Prasad (Lederer and Prasad, 1992) derive practical management guidelines based on a survey of 114 IT managers in the US. Heemstra (Heemstra, 1992) reports on findings from studying cost estimation practices of 400 Dutch companies. His work mainly comprises the usage of different methods by different organizations. Wrigley and Dexter (Wrigley and Dexter, 1987) provide a review of cost estimation methods and stress several issues in cost estimation, like the software sizing problem, or the independence of factors impacting development effort. 3.1 Classification Schema Researchers have made a number of attempts to classify software cost models. This is useful, because it permits the evaluation and comparison of model types. Boehm (Boehm, 1981; Boehm, 1984) introduced seven classes: Algorithmic Models, Expert Judgment, Analogy, Parkinson, Price to Win, Top-Down, Bottom-Up. Some of these classes, like Price-to-Win, cannot really be considered to be an estimation technique. Moreover, some classes are not orthogonal, e.g., expert 1 The latter one is not really comparable as this is mostly a sizing technique as discussed in Section 4. ISERN

6 judgment can be used following a bottom-up estimation process. Similarly, it is difficult to distinguish, for example, the Algorithmic and the Top-Down method. Walkerden and Jeffery (Walkerden and Jeffery, 1997) defined a framework consisting of four classes of prediction methods: Empirical, Analogical, Theoretical, and Heuristic. Unfortunately, they state that expert judgment cannot be included in their framework. In addition, the classes Analogy and Heuristic can overlap, as heuristics can be included in the analogy estimation process (see adaptation rules Section 3.5.2). Moreover, it is not evident why methods using Analogy are not empirical as well since certain components of the analogy method can be derived empirically (see similarity functions 3.5.2). Kitchenham (Fenton and Pfleeger, 1996) classifies current approaches to cost estimation into four classes: expert opinion, analogy, decomposition, and models. Here, decomposition can be seen as estimating the effort in a bottom-up manner. Thus, this category overlaps with the other three classes as it not orthogonal to them. Unfortunately, classification schemes are subjective and there is no agreement about the best one (Kitchenham and de Neumann, 1990). Our classification is not fully satisfactory but is designed to follow our argumentation and evaluation regarding the various types of estimation methods. The classification schema is hierarchical, starting from two main categories (Model Based Methods, Non-Model Based Method) that are further refined into sub-categories. The hierarchy should cover all possible types of resource estimation methods, without being overly complicated for our purpose. Such a classification will help us talking in general terms of a certain type of method. Figure 1 summarizes the classification schema we propose for cost estimation methods. The letters in brackets are explained and used in Section 5. Each class is described in the following sub-sections. Estimation Methods Model Based Methods (A) Non-Model Based Methods (B) Generic Model Based (A1) Specific Model Based(A2) Proprietary Not Proprietary Data Driven Composite Methods Figure 1: Classification of Resource Estimation Methods Model Based Methods Model-based estimation methods, in general, involve at least one modeling method, one model, and one model application method (see Section 5.2). An effort estimation model usually takes a ISERN

7 number of inputs (productivity factors and an estimate of system size) and produces an effort point estimate or distribution. Generic Model Based These estimation methods generate models that are assumed to be generally applicable across different contexts. Proprietary: Modeling methods and models are not fully documented or public domain. Not Proprietary: Modeling methods and models are documented and public domain. Specific Model Based: Specific Model Based estimation methods include local models whose validity is only ensured in the context where they have been developed. Data Driven: Modeling methods are based on data analysis, i.e., the models are derived from data. We may distinguish here further between parametric and non-parametric modeling methods. Parametric methods require the a priori specification of a functional relationship between project attributes and cost. The modeling method then tries to fit the underlying data in the best way possible using the assumed functional form. Non-Parametric methods derive models that do not make specific assumptions about the functional relationship between project attributes and cost (although, to a certain extent, there are always assumptions being made). Composite Methods: Models are built based on combining expert opinion and if cost-driver was measured on an ordinal scale, similarly to the COCOMO model), that would mean that the cost of the project would be 120% that of the nominal project. The multipliers for the cost-drivers are modeled as distributions to capture the uncertainty inherent to expert opinion. Triangular distributions can be used (minimum, most likely, and maximum value) and reflect the experts' opinion about each cost-driver's impact on cost. Triangle(0,10,20) Triangle(0,20,50) Figure 6: Triangular Distributions Figure 6 provides examples of triangular distributions. The variance in distribution reflects the uncertainty in expert s opinion. Specific techniques have been designed to quantify interactions but this is out of the scope of this short summary. The second component of COBRA uses data from past similar projects identifying a relationship between cost overhead and productivity. Note that this is a simple bivariate relationship that does not require a large dataset. This is important as it explains why COBRA does not have demanding data requirements, as opposed to data-driven estimation techniques. ISERN

18 Cost Overhead Productivity Figure 7: Example of a Cost Overhead - Productivity Relationship Figure 7 illustrates a linear relationship between productivity and cost overhead that can be obtained from past project data. It indicates that the higher the cost overhead the lower the productivity. Using COBRA the uncertainty of a prediction can be evaluated (for the purpose of risk assessment), a productivity or effort point estimate may be obtained for an actual project, and benchmarking can be performed of the new project against the underlying past project data. We will describe an example scenario regarding the use of COBRA for risk assessment. To produce a cost overhead estimate for an actual project, the project is characterized in terms of the cost factors in the causal model. The actual project values are transformed into parameters of triangular distributions. Running a Monte Carlo simulation, sampling is performed from each of the distributions and each sampled value is summed obtaining a cost overhead estimate. This is repeated many times (e.g., 1000) and a distribution of cost overhead values is generated. In order to interpret the cost overhead values from the distribution, the second component of COBRA is needed. Using the cost overhead - productivity relationship, the amount of money, or effort corresponding to a cost overhead value can be determined. In order to evaluate the probability that a project will be over budget, one can generate the descending cumulative cost overhead distribution for the actual project (Figure 8) P R O 0.80 B A 0.60 B I 0.40 L I 0.20 T Y 0.00 Cost Overhead 1095 Figure 8: Cumulative Probability of Cost Overhead Let us assume that the available budget for a project is 1100 Units and that this project s cost overhead is characterized by the distribution in Figure 8. There is roughly a 80% probability that the project will stay within budget. If this probability represents an acceptable risk in a particular context, the project budget may be approved. ISERN

19 COBRA assumes orthogonal variables, which are defined on approximately interval scales. COBRA does not require a specific measure for system size. But to use the model, system size needs to be estimated in some way. Some advantages of COBRA are (1) its applicability in environment with only little project data is available, (2) the explicit modeling of reusable expert knowledge regarding project estimation, (3) it is more likely to be accepted by an organization s practitioners as the models are rooted in their own experience. The main disadvantages are that COBRA (1) requires experts to be available for interviews and (2) that knowledge elicitation is still a difficult task requiring extensive training and experience Analogy Based Methods Analogy is a problem solving technique (Delany et al., 1998). It solves a new problem by adapting solutions that were used to solve an old problem. One or more cases (i.e., projects) similar to current problem are retrieved and attempted to modify these to fit the current problem parameters. In software effort estimation, every case is a previous project while the current problem is to extract a suitable estimate for the current project. The basic idea is to identify the completed projects that are the most similar (analogues) to a new project. Major decisions to be made by the user are (1) the selection of relevant project attributes (i.e., cost-drivers) (2) the selection of appropriate similarity/distance functions, and (3) the number of analogues to consider for prediction. Similarity functions may be defined with the help of experts. As software datasets commonly contain variables of different scale types (nominal, ordinal, interval, ratio), measures were suggested to combine mixed types of variables. Kaufman and Rousseeuw proposed a dissimilarity coefficient (Kaufman and Rousseeuw, 1990). An example of a simple distance measure is based on the unweighted Euclidean distance using variables normalized between 0 and 1 (Shepperd and Schofield, 1997). The overall distance(p i, P j ) between two projects P i and P j is defined as: n δ (Pik,Pjk ) distance(p,p ) k = 1 i j = n where n is the number of variables. A variable could be, for example, the team size (continuous) or project environment (discrete). The distance regarding a given variable k between two projects P i and P j is δ(p ik, P jk ): δ(p ik,p jk 2 P ik Pjk, if k is continuous maxk mink ) = 0, if k is categorical AND Pik = P 1, if k is categorical AND P P where value max k /min k is the maximum/minimum possible value of variable k. If two projects have the same project environment value, the distance is 0, if their project environment is different, the corresponding distance for this variable is 1. A variation is to compute weighted Euclidean distances where each variable receives a weight according to its perceived importance. ik jk jk ISERN

20 The selection of relevant project attributes may be determined by looking at the optimal combination of variables by implementing a comprehensive search as implemented in the ANGEL tool (AnaloGy SoftwarE tool, Bournemouth University). This is, however, inadequate for a high number of variables and projects, as reported in (Shepperd and Schofield, 1997). Another strategy is proposed by Finnie et al. (Finnie et al., 1997). They applied to all categorical variables a two-tailed t-test to determine variables that show significant influence on productivity. Attributes New Project Retrieved Project 1 Retrieved Project 2 Project Category Real Time Real Time Simulators Programming Language C C C Team Size System Size Effort? Similarity 80% 77% Table 3: Example of Analogy based retrieval For prediction, one may use the effort value of the most similar analogue(s). For example, in Table 3, 1000 would be the predicted value for a new project if the best analogue was used (retrieved project 1). When considering more than one analogue, simply the mean value (mean effort from retrieved project 1 and 2), or a weighted mean may be used for prediction. Adaptation rules may be defined with the help of experts to devise the final estimation for the new project. A simple example of an adaptation rule is an adjustment of the predicted effort using the size of the actual project (e.g., Walkerden and Jeffery, 1999). For example, assume the system size of the new project is 150 units, the system size of the most similar project is 200 units and its effort is 1000 Units. With the application of an adaptation rule the predicted effort is 150/200*1000=750 Units, rather than 1000 Units without applying the adaptation rule. 3.6 Evaluation of Non-Model Based Methods Expert judgment involves the consulting of one or more experts to derive directly one resource expenditure estimate (Hughes, 1996). Experts usually use their experience and understanding of a new project and available information about the new and past projects to derive an estimate. Examples of the information used are design requirements, source code, software tools available, rules of thumb, resources available, size/complexity of the new functions, data from past projects, or feedback from past estimates. In contrast to model-based methods, the procedure to derive the final estimate (i.e., the estimation technique applied, see Section 5, Figure 9) may be more intuitive to some practitioners. One approach is based on estimating as a group. This appears to have advantages such as the enthusiasm of some people may be balanced by the caution of others. However, this approach involves the risk that people with stronger personalities may dominate (Baron and Greenberg, 1990). To reduce these risks and to avoid the causes of bias of individual experts, the adoption of the Delphi technique is suggested (Boehm, 1981; Boehm, 1984; Chulani et al., 1999). First, a group coordinator presents a specification and a form to be filled out. Then, experts anonymously and independently record their estimates for a new project on the form. The group coordinator prepares a summary of the experts responses on a new form and requests another iteration and the rational behind the estimate. The experts again anonymously fill out the forms. The process is iterated for as many rounds as appropriate. No discussion among the experts is taking place during the whole process. One variation of the standard Delphi-Method includes group meetings, ISERN

Resource Estimation in Software Engineering 1

Resource Estimation in Software Engineering 1 Lionel C. Briand and Isabella Wieczorek 1 Introduction This paper presents a comprehensive overview of the state of the art in software resource estimation.

More information

A replicated Assessment and Comparison of Common Software Cost Modeling Techniques

A replicated Assessment and Comparison of Common Software Cost Modeling Techniques Lionel C. Briand Tristen Langley Isabella Wieczorek Carleton University CAESAR, University of Fraunhofer Institute for

More information

A replicated Assessment and Comparison of Common Software Cost Modeling Techniques

A replicated Assessment and Comparison of Common Software Cost Modeling Techniques Lionel Briand Tristen Langley Isabella Wieczorek Carleton University CAESAR, University of Fraunhofer Institute for Systems

More information

Cost Estimation for Web Applications

Melanie Ruhe 1 Siemens AG, Corporate Technology, Software Engineering 3 80730 Munich, Germany melanie.ruhe@siemens.com Cost Estimation for Web Applications Ross Jeffery University of New South Wales School

More information

Multinomial Logistic Regression Applied on Software Productivity Prediction

Multinomial Logistic Regression Applied on Software Productivity Prediction Panagiotis Sentas, Lefteris Angelis, Ioannis Stamelos Department of Informatics, Aristotle University 54124 Thessaloniki, Greece

More information

Project Planning Objectives. Project Estimation. Resources. Software Project Estimation

Project Planning Objectives Project Estimation Providing a framework that allows managers to make responsible estimates of the resources and time required to build a software product. Determining the scope

More information

Software Development Cost and Time Forecasting Using a High Performance Artificial Neural Network Model

Software Development Cost and Time Forecasting Using a High Performance Artificial Neural Network Model Iman Attarzadeh and Siew Hock Ow Department of Software Engineering Faculty of Computer Science &

More information

An Assessment and Comparison of Common Software Cost Estimation Modeling Techniques

An Assessment and Comparison of Common Software Cost Estimation Modeling Techniques Lionel C. Briand, Khaled El Emam Dagmar Surmann, Isabella Wieczorek Fraunhofer Institute for Experimental Solhare Engineering

More information

Simple Predictive Analytics Curtis Seare

Using Excel to Solve Business Problems: Simple Predictive Analytics Curtis Seare Copyright: Vault Analytics July 2010 Contents Section I: Background Information Why use Predictive Analytics? How to use

More information

A Comparison of Calibrated Equations for Software Development Effort Estimation

A Comparison of Calibrated Equations for Software Development Effort Estimation Cuauhtemoc Lopez Martin Edgardo Felipe Riveron Agustin Gutierrez Tornes 3,, 3 Center for Computing Research, National Polytechnic

More information

Measurement Information Model

mcgarry02.qxd 9/7/01 1:27 PM Page 13 2 Information Model This chapter describes one of the fundamental measurement concepts of Practical Software, the Information Model. The Information Model provides

More information

METHODS OF EFFORT ESTIMATION IN SOFTWARE ENGINEERING

I International Symposium Engineering Management And Competitiveness 2011 (EMC2011) June 24-25, 2011, Zrenjanin, Serbia METHODS OF EFFORT ESTIMATION IN SOFTWARE ENGINEERING Jovan Živadinović, Ph.D * High

More information

Current Standard: Mathematical Concepts and Applications Shape, Space, and Measurement- Primary

Shape, Space, and Measurement- Primary A student shall apply concepts of shape, space, and measurement to solve problems involving two- and three-dimensional shapes by demonstrating an understanding of:

More information

MTAT.03.244 Software Economics. Lecture 5: Software Cost Estimation

MTAT.03.244 Software Economics Lecture 5: Software Cost Estimation Marlon Dumas marlon.dumas ät ut. ee Outline Estimating Software Size Estimating Effort Estimating Duration 2 For Discussion It is hopeless

More information

Domain Analysis for the Reuse of Software Development Experiences 1

Domain Analysis for the Reuse of Software Development Experiences 1 V. R. Basili*, L. C. Briand**, W. M. Thomas* * Department of Computer Science University of Maryland College Park, MD, 20742 USA ** CRIM

More information

Software cost estimation

Software cost estimation Sommerville Chapter 26 Objectives To introduce the fundamentals of software costing and pricing To describe three metrics for software productivity assessment To explain why different

More information

Software Engineering. Dilbert on Project Planning. Overview CS / COE 1530. Reading: chapter 3 in textbook Requirements documents due 9/20

Software Engineering CS / COE 1530 Lecture 4 Project Management Dilbert on Project Planning Overview Reading: chapter 3 in textbook Requirements documents due 9/20 1 Tracking project progress Do you understand

More information

Module 11. Software Project Planning. Version 2 CSE IIT, Kharagpur

Module 11 Software Project Planning Lesson 29 Staffing Level Estimation and Scheduling Specific Instructional Objectives At the end of this lesson the student would be able to: Identify why careful planning

More information

STATISTICA Formula Guide: Logistic Regression. Table of Contents

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

More information

Software cost estimation

Software cost estimation Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 26 Slide 1 Objectives To introduce the fundamentals of software costing and pricing To describe three metrics for

More information

Contents. Today Project Management. Project Management. Last Time - Software Development Processes. What is Project Management?

Contents Introduction Software Development Processes Project Management Requirements Engineering Software Construction Group processes Quality Assurance Software Management and Evolution Last Time - Software

More information

Factors Influencing Software Development Productivity - State of the Art and Industrial Experiences

Factors Influencing Software Development Productivity - State of the Art and Industrial Experiences Adam Trendowicz, Jürgen Münch Fraunhofer Institute for Experimental Software Engineering Fraunhofer-Platz

More information

Software Metrics & Software Metrology. Alain Abran. Chapter 4 Quantification and Measurement are Not the Same!

Software Metrics & Software Metrology Alain Abran Chapter 4 Quantification and Measurement are Not the Same! 1 Agenda This chapter covers: The difference between a number & an analysis model. The Measurement

More information

Pragmatic Peer Review Project Contextual Software Cost Estimation A Novel Approach

www.ijcsi.org 692 Pragmatic Peer Review Project Contextual Software Cost Estimation A Novel Approach Manoj Kumar Panda HEAD OF THE DEPT,CE,IT & MCA NUVA COLLEGE OF ENGINEERING & TECH NAGPUR, MAHARASHTRA,INDIA

More information

Cost Estimation Strategies COST ESTIMATION GUIDELINES

Cost Estimation Strategies Algorithmic models (Rayleigh curve Cost in week t = K a t exp(-a t 2 ) Expert judgment (9 step model presented later) Analogy (Use similar systems) Parkinson (Work expands to

More information

Software project cost estimation using AI techniques

Software project cost estimation using AI techniques Rodríguez Montequín, V.; Villanueva Balsera, J.; Alba González, C.; Martínez Huerta, G. Project Management Area University of Oviedo C/Independencia

More information

Models for Product Demand Forecasting with the Use of Judgmental Adjustments to Statistical Forecasts

Page 1 of 20 ISF 2008 Models for Product Demand Forecasting with the Use of Judgmental Adjustments to Statistical Forecasts Andrey Davydenko, Professor Robert Fildes a.davydenko@lancaster.ac.uk Lancaster

More information

Data Analysis. Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) SS Analysis of Experiments - Introduction

Data Analysis Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) Prof. Dr. Dr. h.c. Dieter Rombach Dr. Andreas Jedlitschka SS 2014 Analysis of Experiments - Introduction

More information

Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

Module 11. Software Project Planning. Version 2 CSE IIT, Kharagpur

Module 11 Software Project Planning Lesson 28 COCOMO Model Specific Instructional Objectives At the end of this lesson the student would be able to: Differentiate among organic, semidetached and embedded

More information

Applying Statistics Recommended by Regulatory Documents

Applying Statistics Recommended by Regulatory Documents Steven Walfish President, Statistical Outsourcing Services steven@statisticaloutsourcingservices.com 301-325 325-31293129 About the Speaker Mr. Steven

More information

Handling attrition and non-response in longitudinal data

Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein

More information

Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

More information

Knowledge-Based Systems Engineering Risk Assessment

Knowledge-Based Systems Engineering Risk Assessment Raymond Madachy, Ricardo Valerdi University of Southern California - Center for Systems and Software Engineering Massachusetts Institute of Technology

More information

Software Cost Estimation

Software Cost Estimation 1 Hareton Leung Zhang Fan Department of Computing The Hong Kong Polytechnic University {cshleung, csfzhang}@comp.polyu.edu.hk Abstract Software cost estimation is the process of

More information

Chapter 23 Software Cost Estimation

Chapter 23 Software Cost Estimation Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 23 Slide 1 Software cost estimation Predicting the resources required for a software development process

More information

South Carolina College- and Career-Ready (SCCCR) Probability and Statistics

South Carolina College- and Career-Ready (SCCCR) Probability and Statistics South Carolina College- and Career-Ready Mathematical Process Standards The South Carolina College- and Career-Ready (SCCCR)

More information

11/20/2014. Correlational research is used to describe the relationship between two or more naturally occurring variables.

Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are highly extraverted people less afraid of rejection

More information

Efficiency in Software Development Projects

Efficiency in Software Development Projects Aneesh Chinubhai Dharmsinh Desai University aneeshchinubhai@gmail.com Abstract A number of different factors are thought to influence the efficiency of the software

More information

Multiple Linear Regression in Data Mining

Multiple Linear Regression in Data Mining Contents 2.1. A Review of Multiple Linear Regression 2.2. Illustration of the Regression Process 2.3. Subset Selection in Linear Regression 1 2 Chap. 2 Multiple

More information

The Limits of CBR in Software Project Estimation

Presented at 1998 German Workshop on Case-Based Reasoning The Limits of CBR in Software Project Estimation Sarah Jane Delany 1, Pádraig Cunningham 2 and Wolfgang Wilke 3 1 Dublin Institute of Technology,

More information

IMPROVEMENT AND IMPLEMENTATION OF ANALOGY BASED METHOD FOR SOFTWARE PROJECT COST ESTIMATION

IMPROVEMENT AND IMPLEMENTATION OF ANALOGY BASED METHOD FOR SOFTWARE PROJECT COST ESTIMATION LI YAN-FU (B. Eng), WUHAN UNIVERSITY A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF

More information

ASSIGNMENT 4 PREDICTIVE MODELING AND GAINS CHARTS

DATABASE MARKETING Fall 2015, max 24 credits Dead line 15.10. ASSIGNMENT 4 PREDICTIVE MODELING AND GAINS CHARTS PART A Gains chart with excel Prepare a gains chart from the data in workcoursese27e20100ass4b.xls.

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Software Cost Estimation with Incomplete Data

890 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 27, NO. 10, OCTOBER 2001 Software Cost Estimation with Incomplete Data Kevin Strike, Khaled El Emam, and Nazim Madhavji AbstractÐThe construction of

More information

Calibrating Function Point Backfiring Conversion Ratios Using Neuro-Fuzzy Technique

Western University Scholarship@Western Electrical and Computer Engineering Publications Electrical and Computer Engineering 12-2008 Calibrating Function Point Backfiring Conversion Ratios Using Neuro-Fuzzy

More information

Data Mining - Evaluation of Classifiers

Data Mining - Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010

More information

Introduction to Regression and Data Analysis

Statlab Workshop Introduction to Regression and Data Analysis with Dan Campbell and Sherlock Campbell October 28, 2008 I. The basics A. Types of variables Your variables may take several forms, and it

More information

Project Planning and Project Estimation Techniques. Naveen Aggarwal

Project Planning and Project Estimation Techniques Naveen Aggarwal Responsibilities of a software project manager The job responsibility of a project manager ranges from invisible activities like building

More information

Deducing software process improvement areas from a COCOMO II-based productivity measurement

Deducing software process improvement areas from a COCOMO II-based productivity measurement Lotte De Rore, Monique Snoeck, Geert Poels, Guido Dedene Abstract At the SMEF2006 conference, we presented our

More information

Lecture 14: Cost Estimation

Overview Project management activities Project costing Project scheduling and staffing Project monitoring and review General cost estimation rules Algorithmic Cost Modeling Function point model COCOMO

More information

(Refer Slide Time: 01:52)

Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This

More information

LEARNING OBJECTIVES SCALES OF MEASUREMENT: A REVIEW SCALES OF MEASUREMENT: A REVIEW DESCRIBING RESULTS DESCRIBING RESULTS 8/14/2016

UNDERSTANDING RESEARCH RESULTS: DESCRIPTION AND CORRELATION LEARNING OBJECTIVES Contrast three ways of describing results: Comparing group percentages Correlating scores Comparing group means Describe

More information

STATISTICA. Clustering Techniques. Case Study: Defining Clusters of Shopping Center Patrons. and

Clustering Techniques and STATISTICA Case Study: Defining Clusters of Shopping Center Patrons STATISTICA Solutions for Business Intelligence, Data Mining, Quality Control, and Web-based Analytics Table

More information

The Use of Simulation Techniques for Hybrid Software Cost Estimation and Risk Analysis

The Use of Simulation Techniques for Hybrid Software Cost Estimation and Risk Analysis Michael Kläs, Adam Trendowicz, Axel Wickenkamp, Jürgen Münch Fraunhofer Institute for Experimental Software Engineering

More information

Finally, Article 4, Creating the Project Plan describes how to use your insight into project cost and schedule to create a complete project plan.

Project Cost Adjustments This article describes how to make adjustments to a cost estimate for environmental factors, schedule strategies and software reuse. Author: William Roetzheim Co-Founder, Cost

More information

Prentice Hall Algebra 2 2011 Correlated to: Colorado P-12 Academic Standards for High School Mathematics, Adopted 12/2009

Content Area: Mathematics Grade Level Expectations: High School Standard: Number Sense, Properties, and Operations Understand the structure and properties of our number system. At their most basic level

More information

International Journal of Software and Web Sciences (IJSWS)

International Association of Scientific Innovation and Research (IASIR) (An Association Unifying the Sciences, Engineering, and Applied Research) International Journal of Software and Web Sciences (IJSWS)

More information

Numerical Summarization of Data OPRE 6301

Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting

More information

Insurance Analytics - analýza dat a prediktivní modelování v pojišťovnictví. Pavel Kříž. Seminář z aktuárských věd MFF 4.

Insurance Analytics - analýza dat a prediktivní modelování v pojišťovnictví Pavel Kříž Seminář z aktuárských věd MFF 4. dubna 2014 Summary 1. Application areas of Insurance Analytics 2. Insurance Analytics

More information

An Approach to Find Maintenance Costs Using Cost Drivers of Cocomo Intermediate Model

An Approach to Find Maintenance Costs Using Cost Drivers of Cocomo Intermediate Model C.V.S.R SYAVASYA 1, M.Tech, GITAM UNIVERSITY Abstract: Maintenance of software under several cost drivers is as sort

More information

The Application of Case-Based Reasoning to Early Software Project Cost Estimation and Risk Assessment

The Application of Case-Based Reasoning to Early Software Project Cost Estimation and Risk Assessment Sarah Jane Delany Department of Computer Science DIT Kevin Street, Dublin Pádraig Cunningham Department

More information

Classification and Regression Trees

Classification and Regression Trees Bob Stine Dept of Statistics, School University of Pennsylvania Trees Familiar metaphor Biology Decision tree Medical diagnosis Org chart Properties Recursive, partitioning

More information

DATA INTERPRETATION AND STATISTICS

PholC60 September 001 DATA INTERPRETATION AND STATISTICS Books A easy and systematic introductory text is Essentials of Medical Statistics by Betty Kirkwood, published by Blackwell at about 14. DESCRIPTIVE

More information

On Correlating Performance Metrics

On Correlating Performance Metrics Yiping Ding and Chris Thornley BMC Software, Inc. Kenneth Newman BMC Software, Inc. University of Massachusetts, Boston Performance metrics and their measurements are

More information

Scott Knott Test Based Effective Software Effort Estimation through a Multiple Comparison Algorithms

Scott Knott Test Based Effective Software Effort Estimation through a Multiple Comparison Algorithms N.Padma priya 1, D.Vidyabharathi 2 PG scholar, Department of CSE, SONA College of Technology, Salem,

More information

COMBINING THE METHODS OF FORECASTING AND DECISION-MAKING TO OPTIMISE THE FINANCIAL PERFORMANCE OF SMALL ENTERPRISES

COMBINING THE METHODS OF FORECASTING AND DECISION-MAKING TO OPTIMISE THE FINANCIAL PERFORMANCE OF SMALL ENTERPRISES JULIA IGOREVNA LARIONOVA 1 ANNA NIKOLAEVNA TIKHOMIROVA 2 1, 2 The National Nuclear Research

More information

Demand forecasting & Aggregate planning in a Supply chain. Session Speaker Prof.P.S.Satish

Demand forecasting & Aggregate planning in a Supply chain Session Speaker Prof.P.S.Satish 1 Introduction PEMP-EMM2506 Forecasting provides an estimate of future demand Factors that influence demand and

More information

Computer Science and Software Engineering University of Wisconsin - Platteville 3.Time Management

Computer Science and Software Engineering University of Wisconsin - Platteville 3.Time Management SE 2730 Lecture Notes Yan Shi Based on Introduction to the Personal Software Process by Watts Humphrey

More information

Software cost estimation. Predicting the resources required for a software development process

Software cost estimation Predicting the resources required for a software development process Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 23 Slide 1 Objectives To introduce the fundamentals

More information

Marketing Mix Modelling and Big Data P. M Cain

1) Introduction Marketing Mix Modelling and Big Data P. M Cain Big data is generally defined in terms of the volume and variety of structured and unstructured information. Whereas structured data is stored

More information

Size-Based Software Cost Modelling with Artificial Neural Networks and Genetic Algorithms

9 Size-Based Software Cost Modelling with Artificial Neural Networks and Genetic Algorithms Efi Papatheocharous 1 and Andreas S. Andreou 2 1 Department of Computer Science, University of Cyprus 2 Department

More information

Software Architecture and Engineering Project Cost Management Peter Müller

Software Architecture and Engineering Project Cost Management Peter Müller Chair of Programming Methodology Spring Semester 2012 12. Project Cost Management Overview 2 12. Project Cost Management 12.1

More information

Fairfield Public Schools

Mathematics Fairfield Public Schools AP Statistics AP Statistics BOE Approved 04/08/2014 1 AP STATISTICS Critical Areas of Focus AP Statistics is a rigorous course that offers advanced students an opportunity

More information

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternatives

More information

Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

More information

4. Introduction to Statistics

Statistics for Engineers 4-1 4. Introduction to Statistics Descriptive Statistics Types of data A variate or random variable is a quantity or attribute whose value may vary from one unit of investigation

More information

This unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.

Algebra I Overview View unit yearlong overview here Many of the concepts presented in Algebra I are progressions of concepts that were introduced in grades 6 through 8. The content presented in this course

More information

12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand when to use multiple Understand the multiple equation and what the coefficients represent Understand different methods

More information

Prediction of Stock Performance Using Analytical Techniques

136 JOURNAL OF EMERGING TECHNOLOGIES IN WEB INTELLIGENCE, VOL. 5, NO. 2, MAY 2013 Prediction of Stock Performance Using Analytical Techniques Carol Hargreaves Institute of Systems Science National University

More information

Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics.

Business Course Text Bowerman, Bruce L., Richard T. O'Connell, J. B. Orris, and Dawn C. Porter. Essentials of Business, 2nd edition, McGraw-Hill/Irwin, 2008, ISBN: 978-0-07-331988-9. Required Computing

More information

Literature Survey on Algorithmic Methods for Software Development Cost Estimation

Literature Survey on Algorithmic Methods for Software Development Cost Estimation Mrs. Shubhangi Mahesh Potdar 1 Assistant professor, IBMRD, Ahmednagar, India Email:shubhangipotdar@rediffmail.com Dr. Manimala

More information

PROJECT COST MANAGEMENT

7 PROJECT COST MANAGEMENT Project Cost Management includes the processes required to ensure that the project is completed within the approved budget. Figure 7 1 provides an overview of the following major

More information

DESCRIPTION OF COURSES

DESCRIPTION OF COURSES MGT600 Management, Organizational Policy and Practices The purpose of the course is to enable the students to understand and analyze the management and organizational processes and

More information

Silvermine House Steenberg Office Park, Tokai 7945 Cape Town, South Africa Telephone: +27 21 702 4666 www.spss-sa.com

SPSS-SA Silvermine House Steenberg Office Park, Tokai 7945 Cape Town, South Africa Telephone: +27 21 702 4666 www.spss-sa.com SPSS-SA Training Brochure 2009 TABLE OF CONTENTS 1 SPSS TRAINING COURSES FOCUSING

More information

Curriculum Map Statistics and Probability Honors (348) Saugus High School Saugus Public Schools 2009-2010

Curriculum Map Statistics and Probability Honors (348) Saugus High School Saugus Public Schools 2009-2010 Week 1 Week 2 14.0 Students organize and describe distributions of data by using a number of different

More information

Efficient Indicators to Evaluate the Status of Software Development Effort Estimation inside the Organizations

Efficient Indicators to Evaluate the Status of Software Development Effort Estimation inside the Organizations Elham Khatibi Department of Information System Universiti Teknologi Malaysia (UTM) Skudai

More information

THE HYBRID CART-LOGIT MODEL IN CLASSIFICATION AND DATA MINING. Dan Steinberg and N. Scott Cardell

THE HYBID CAT-LOGIT MODEL IN CLASSIFICATION AND DATA MINING Introduction Dan Steinberg and N. Scott Cardell Most data-mining projects involve classification problems assigning objects to classes whether

More information

Gerry Hobbs, Department of Statistics, West Virginia University

Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit

More information

An introduction to Value-at-Risk Learning Curve September 2003

An introduction to Value-at-Risk Learning Curve September 2003 Value-at-Risk The introduction of Value-at-Risk (VaR) as an accepted methodology for quantifying market risk is part of the evolution of risk

More information

Learning Example. Machine learning and our focus. Another Example. An example: data (loan application) The data and the goal

Learning Example Chapter 18: Learning from Examples 22c:145 An emergency room in a hospital measures 17 variables (e.g., blood pressure, age, etc) of newly admitted patients. A decision is needed: whether

More information

Tail-Dependence an Essential Factor for Correctly Measuring the Benefits of Diversification

Tail-Dependence an Essential Factor for Correctly Measuring the Benefits of Diversification Presented by Work done with Roland Bürgi and Roger Iles New Views on Extreme Events: Coupled Networks, Dragon

More information

12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand linear regression with a single predictor Understand how we assess the fit of a regression model Total Sum of Squares

More information

A HYBRID INTELLIGENT MODEL FOR SOFTWARE COST ESTIMATION

Journal of Computer Science, 9(11):1506-1513, 2013, doi:10.3844/ajbb.2013.1506-1513 A HYBRID INTELLIGENT MODEL FOR SOFTWARE COST ESTIMATION Wei Lin Du 1, Luiz Fernando Capretz 2, Ali Bou Nassif 2, Danny

More information

Comparison of SDLC-2013 Model with Other SDLC Models by Using COCOMO

International Journal of Emerging Science and Engineering (IJESE) Comparison of SDLC-2013 Model with Other SDLC Models by Using COCOMO Naresh Kumar, Pinky Chandwal Abstract There exist a large number of

More information

Lecture 10: Regression Trees

Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,

More information

Statistics. Measurement. Scales of Measurement 7/18/2012

Statistics Measurement Measurement is defined as a set of rules for assigning numbers to represent objects, traits, attributes, or behaviors A variableis something that varies (eye color), a constant does

More information

1. Why the hell do we need statistics?

1. Why the hell do we need statistics? There are three kind of lies: lies, damned lies, and statistics, British Prime Minister Benjamin Disraeli (as credited by Mark Twain): It is easy to lie with statistics,

More information

D-optimal plans in observational studies

D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observational

More information Jump to navigationJump to search

In software development, effort estimation is the process of predicting the most realistic amount of effort (expressed in terms of person-hours or money) required to develop or maintain software based on incomplete, uncertain and noisy input. Effort estimates may be used as input to project plans, iteration plans, budgets, investment analyses, pricing processes and bidding rounds.^[1]

State-of-practice[edit]

Published surveys on estimation practice suggest that expert estimation is the dominant strategy when estimating software development effort.^[2]

Typically, effort estimates are over-optimistic and there is a strong over-confidence in their accuracy. The mean effort overrun seems to be about 30% and not decreasing over time. For a review of effort estimation error surveys, see.^[3] However, the measurement of estimation error is problematic, see Assessing the accuracy of estimates.The strong overconfidence in the accuracy of the effort estimates is illustrated by the finding that, on average, if a software professional is 90% confident or “almost sure” to include the actual effort in a minimum-maximum interval, the observed frequency of including the actual effort is only 60-70%.^[4]

Currently the term “effort estimate” is used to denote as different concepts such as most likely use of effort (modal value), the effort that corresponds to a probability of 50% of not exceeding (median), the planned effort, the budgeted effort or the effort used to propose a bid or price to the client. This is believed to be unfortunate, because communication problems may occur and because the concepts serve different goals.^[5][6]

History[edit]

Software researchers and practitioners have been addressing the problems of effort estimation for software development projects since at least the 1960s; see, e.g., work by Farr ^[7][8] and Nelson.^[9]

Most of the research has focused on the construction of formal software effort estimation models. The early models were typically based on regression analysis or mathematically derived from theories from other domains. Since then a high number of model building approaches have been evaluated, such as approaches founded on case-based reasoning, classification and regression trees, simulation, neural networks, Bayesian statistics, lexical analysis of requirement specifications, genetic programming, linear programming, economic production models, soft computing, fuzzy logic modeling, statistical bootstrapping, and combinations of two or more of these models. The perhaps most common estimation methods today are the parametric estimation models COCOMO, SEER-SEM and SLIM. They have their basis in estimation research conducted in the 1970s and 1980s and are since then updated with new calibration data, with the last major release being COCOMO II in the year 2000. The estimation approaches based on functionality-based size measures, e.g., function points, is also based on research conducted in the 1970s and 1980s, but are re-calibrated with modified size measures and different counting approaches, such as the use case points^[10] or object points in the 1990s.

Estimation approaches[edit]

There are many ways of categorizing estimation approaches, see for example.^[11][12] The top level categories are the following:

• Expert estimation: The quantification step, i.e., the step where the estimate is produced based on judgmental processes.^[13]
• Formal estimation model: The quantification step is based on mechanical processes, e.g., the use of a formula derived from historical data.
• Combination-based estimation: The quantification step is based on a judgmental and mechanical combination of estimates from different sources.

Below are examples of estimation approaches within each category.

Estimation approach	Category	Examples of support of implementation of estimation approach
Analogy-based estimation	Formal estimation model	ANGEL, Weighted Micro Function Points
WBS-based (bottom up) estimation	Expert estimation	Project management software, company specific activity templates
Parametric models	Formal estimation model	COCOMO, SLIM, SEER-SEM, TruePlanning for Software
Size-based estimation models[14]	Formal estimation model	Function Point Analysis,[15]Use Case Analysis, Use Case Points, SSU (Software Size Unit), Story points-based estimation in Agile software development, Object Points
Group estimation	Expert estimation	Planning poker, Wideband delphi
Mechanical combination	Combination-based estimation	Average of an analogy-based and a Work breakdown structure-based effort estimate[16]
Judgmental combination	Combination-based estimation	Expert judgment based on estimates from a parametric model and group estimation

Selection of estimation approaches[edit]

The evidence on differences in estimation accuracy of different estimation approaches and models suggest that there is no “best approach” and that the relative accuracy of one approach or model in comparison to another depends strongly on the context.^[17] This implies that different organizations benefit from different estimation approaches. Findings^[18] that may support the selection of estimation approach based on the expected accuracy of an approach include:

• Expert estimation is on average at least as accurate as model-based effort estimation. In particular, situations with unstable relationships and information of high importance not included in the model may suggest use of expert estimation. This assumes, of course, that experts with relevant experience are available.
• Formal estimation models not tailored to a particular organization’s own context, may be very inaccurate. Use of own historical data is consequently crucial if one cannot be sure that the estimation model’s core relationships (e.g., formula parameters) are based on similar project contexts.
• Formal estimation models may be particularly useful in situations where the model is tailored to the organization’s context (either through use of own historical data or that the model is derived from similar projects and contexts), and it is likely that the experts’ estimates will be subject to a strong degree of wishful thinking.

The most robust finding, in many forecasting domains, is that combination of estimates from independent sources, preferable applying different approaches, will on average improve the estimation accuracy.^[18][19][20]

It is important to be aware of the limitations of each traditional approach to measuring software development productivity.^[21]

In addition, other factors such as ease of understanding and communicating the results of an approach, ease of use of an approach, and cost of introduction of an approach should be considered in a selection process.

Assessing the accuracy of estimates[edit]

The most common measure of the average estimation accuracy is the MMRE (Mean Magnitude of Relative Error), where the MRE of each estimate is defined as:

MRE = ${\displaystyle \frac{\text{actual effort}-\text{estimated effort}}{\text{actual effort}}}$

This measure has been criticized ^[22][23][24] and there are several alternative measures, such as more symmetric measures ^[25], Weighted Mean of Quartiles of relative errors (WMQ)^[26] and Mean Variation from Estimate (MVFE).^[27]

MRE is not reliable if the individual items are skewed. PRED(25) is preferred as a measure of estimation accuracy. PRED(25) measures the percentage of predicted values that are within 25 percent of the actual value.

A high estimation error cannot automatically be interpreted as an indicator of low estimation ability. Alternative, competing or complementing, reasons include low cost control of project, high complexity of development work, and more delivered functionality than originally estimated. A framework for improved use and interpretation of estimation error measurement is included in.^[28]

Psychological issues[edit]

There are many psychological factors potentially explaining the strong tendency towards over-optimistic effort estimates that need to be dealt with to increase accuracy of effort estimates. These factors are essential even when using formal estimation models, because much of the input to these models is judgment-based. Factors that have been demonstrated to be important are: Wishful thinking, anchoring, planning fallacy and cognitive dissonance. A discussion on these and other factors can be found in work by Jørgensen and Grimstad.^[29]

• It's easy to estimate what you know.
• It's hard to estimate what you know you don't know. (known unknowns)
• It's very hard to estimate things that you don't know you don't know. (unknown unknowns)

Humor[edit]

The chronic underestimation of development effort has led to the coinage and popularity of numerous humorous adages, such as ironically referring to a task as a 'small matter of programming' (when much effort is likely required), and citing laws about underestimation:

• Ninety-ninety rule:

The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.^[30]

• Hofstadter's law:

Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.

• Fred Brooks' law:

What one programmer can do in one month, two programmers can do in two months.

Adding to the fact that estimating development efforts is hard, it's worth stating that assigning more resources doesn't always help.

Comparison of development estimation software[edit]

Software	Schedule estimate	Cost estimate	Cost Models	Input	Report Output Format	Supported Programming Languages	Platforms	Cost	License
AFCAA REVIC[32]	Yes	Yes	REVIC	KLOC, Scale Factors, Cost Drivers	proprietary, Text	any	DOS	Free	Proprietary Free for public distribution
Seer for Software[33]	Yes	Yes	SEER-SEM	SLOC, Function points, use cases, bottoms-up, object, features	proprietary, Excel, Microsoft Project, IBM Rational, Oracle Crystal Ball	any	Windows, Any (Web-based)[34]	Commercial	Proprietary
SLIM[35]	Yes	Yes	SLIM	Size (SLOC, Function points, Use Cases, etc.), constraints (size, duration, effort, staff), scale factors, historical projects, historical trends	proprietary, Excel, Microsoft Project, Microsoft PowerPoint, IBM Rational, text, HTML	any	Windows, Any (Web-based)[36]	Commercial	Proprietary
TruePlanning[37]	Yes	Yes	PRICE	Components, Structures, Activities, Cost drivers, Processes, Functional Software Size (Source Lines of Code (SLOC), Function Points, Use Case Conversion Points (UCCP), Predictive Object Points (POPs) etc.)	Excel, CAD	any	Windows	Commercial	Proprietary

See also[edit]

References[edit]

1. ^'What We do and Don't Know about Software Development Effort Estimation'.
2. ^Jørgensen, M. (2004). 'A Review of Studies on Expert Estimation of Software Development Effort'. Journal of Systems and Software. 70 (1–2): 37–60. doi:10.1016/S0164-1212(02)00156-5.
3. ^Molokken, K. Jorgensen, M. (2003). 'A review of software surveys on software effort estimation'. 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings. pp. 223–230. doi:10.1109/ISESE.2003.1237981. ISBN978-0-7695-2002-5.CS1 maint: Multiple names: authors list (link)
4. ^Jørgensen, M. Teigen, K.H. Ribu, K. (2004). 'Better sure than safe? Over-confidence in judgement based software development effort prediction intervals'. Journal of Systems and Software. 70 (1–2): 79–93. doi:10.1016/S0164-1212(02)00160-7. Archived from the original on 2009-08-19.CS1 maint: Multiple names: authors list (link)
5. ^Edwards, J.S. Moores (1994). 'A conflict between the use of estimating and planning tools in the management of information systems'. European Journal of Information Systems. 3 (2): 139–147. doi:10.1057/ejis.1994.14.
6. ^Goodwin, P. (1998). Enhancing judgmental sales forecasting: The role of laboratory research. Forecasting with judgment. G. Wright and P. Goodwin. New York, John Wiley & Sons: 91-112. Hi
7. ^Farr, L. Nanus, B. 'Factors that affect the cost of computer programming, volume I'(PDF).CS1 maint: Multiple names: authors list (link)
8. ^Farr, L. Nanus, B. 'Factors that affect the cost of computer programming, volume II'(PDF).CS1 maint: Multiple names: authors list (link)
9. ^Nelson, E. A. (1966). Management Handbook for the Estimation of Computer Programming Costs. AD-A648750, Systems Development Corp.
10. ^Anda, B. Angelvik, E. Ribu, K. (2002). Improving Estimation Practices by Applying Use Case Models. Lecture Notes in Computer Science. 2559. pp. 383–397. CiteSeerX10.1.1.546.112. doi:10.1007/3-540-36209-6_32. ISBN9783540002345 9783540362098 Check isbn= value: length (help).CS1 maint: Multiple names: authors list (link)
11. ^Briand, L. C. and Wieczorek, I. (2002). Resource estimation in software engineering. Encyclopedia of software engineering. J. J. Marcinak. New York, John Wiley & Sons: 1160-1196.
12. ^Jørgensen, M. Shepperd, M. 'A Systematic Review of Software Development Cost Estimation Studies'.CS1 maint: Multiple names: authors list (link)
13. ^'Custom Software Development Services - Custom App Development - Oxagile'.
14. ^Hill Peter (ISBSG) - Estimation Workbook 2 - published by International Software Benchmarking Standards Group ISBSG - Estimation and Benchmarking Resource Centre
15. ^Morris Pam — Overview of Function Point Analysis Total Metrics - Function Point Resource Centre
16. ^Srinivasa Gopal and Meenakshi D'Souza. 2012. Improving estimation accuracy by using case based reasoning and a combined estimation approach. In Proceedings of the 5th India Software Engineering Conference (ISEC '12). ACM, New York, NY, USA, 75-78. DOI=https://dx.doi.org/10.1145/2134254.2134267
17. ^Shepperd, M. Kadoda, G. (2001). 'Comparing software prediction techniques using simulation'. IEEE Transactions on Software Engineering. 27 (11): 1014–1022. doi:10.1109/32.965341.CS1 maint: Multiple names: authors list (link)
18. ^ ^abJørgensen, M. 'Estimation of Software Development Work Effort:Evidence on Expert Judgment and Formal Models'.
19. ^Winkler, R.L. (1989). 'Combining forecasts: A philosophical basis and some current issues Manager'. International Journal of Forecasting. 5 (4): 605–609. doi:10.1016/0169-2070(89)90018-6.
20. ^Blattberg, R.C. Hoch, S.J. (1990). 'Database Models and Managerial Intuition: 50% Model + 50% Manager'. Management Science. 36 (8): 887–899. doi:10.1287/mnsc.36.8.887. JSTOR2632364.CS1 maint: Multiple names: authors list (link)
21. ^BlueOptima. 'Identifying Reliable, Objective Software Development Metrics'.
22. ^Shepperd, M. Cartwright, M. Kadoda, G. (2000). 'On Building Prediction Systems for Software Engineers'. Empirical Software Engineering. 5 (3): 175–182. doi:10.1023/A:1026582314146.CS1 maint: Multiple names: authors list (link)
23. ^Kitchenham, B. Pickard, L.M. MacDonell, S.G. Shepperd. 'What accuracy statistics really measure'.CS1 maint: Multiple names: authors list (link)
24. ^Foss, T. Stensrud, E. Kitchenham, B. Myrtveit, I. (2003). 'A Simulation Study of the Model Evaluation Criterion MMRE'. IEEE Transactions on Software Engineering. 29 (11): 985–995. CiteSeerX10.1.1.101.5792. doi:10.1109/TSE.2003.1245300.CS1 maint: Multiple names: authors list (link)
25. ^Miyazaki, Y. Terakado, M. Ozaki, K. Nozaki, H. (1994). 'Robust regression for developing software estimation models'. Journal of Systems and Software. 27: 3–16. doi:10.1016/0164-1212(94)90110-4.CS1 maint: Multiple names: authors list (link)
26. ^Lo, B. Gao, X. 'Assessing Software Cost Estimation Models: criteria for accuracy, consistency and regression'.CS1 maint: Multiple names: authors list (link)
27. ^Hughes, R.T. Cunliffe, A. Young-Martos, F. (1998). 'Evaluating software development effort model-building techniquesfor application in a real-time telecommunications environment'. IEE Proceedings - Software. 145: 29. doi:10.1049/ip-sen:19983370.CS1 maint: Multiple names: authors list (link)
28. ^Grimstad, S. Jørgensen, M. (2006). 'A Framework for the Analysis of Software Cost Estimation Accuracy'.CS1 maint: Multiple names: authors list (link)
29. ^Jørgensen, M. Grimstad, S. 'How to Avoid Impact from Irrelevant and Misleading Information When Estimating Software Development Effort'.CS1 maint: Multiple names: authors list (link)
30. ^Bentley, Jon (1985). 'Programming pearls'. Communications of the ACM(fee required) format= requires url= (help). 28 (9): 896–901. doi:10.1145/4284.315122. ISSN0001-0782.
31. ^Gödel, Escher, Bach: An Eternal Golden Braid. 20th anniversary ed., 1999, p. 152. ISBN0-465-02656-7.
32. ^AFCAA Revic 9.2 manual Revic memorial site
33. ^Seer for Software Datasheet Galorath site
34. ^'SEER for Software Estimation - Technical Features'.
35. ^SLIM Suite Overview
36. ^SLIM-WebServices
37. ^TruePlanning Integrated Cost Models PRICE Systems site

External links[edit]

• Mike Cohn's Estimating With Use Case Points from article from Methods & Tools: http://www.methodsandtools.com/archive/archive.php?id=25
• Resources on Software Estimation from Steve McConnell: https://web.archive.org/web/20080501183154/http://www.construx.com/Page.aspx?nid=297