Unit I - Module 2 Cost Estimating Techniques Flashcards

Question

The idea of precision and accuracy manifests in cost estimating as uncertainty and risk.

Answer 1

Uncertainty captures the range of possible outcomes of the estimate which characterizes its precision. This is best done as a probability distribution, usually an empirical one resulting from a Monte Carlo simulation which can then be summarized with a confidence interval (e.g., one million dollars, plus or minus 15 percent). Risk refers to the upward shift applied to the cost estimate range to account for the fact that unadjusted estimates tend to be systematically low. This adjustment is intended to eliminate what appears to be an inherent bias in estimates and improves the estimate's accuracy. For instance, if costs come in on average 20% higher than the estimates, cost estimators may want to add 20% to the estimate. At first, this seems like a high percentage, but it is not atypical of historical cost growth on some programs. The proper treatment of uncertainty and risk is difficult because each involves some probability and statistics, but developing a best attempt is essential to produce meaningful and useful estimates.

Answer 2

Same systems or units are not built repeatedly so that the range of costs can be observed. The system or unit is built once and one possible value is determined, which includes random or unknown elements, causing it to be above or below average. Even when building multiple units, as in a production run, each individual unit has its own random or unknown elements that produce variation around what should be a smooth learning curve.

Answer 3

Cost estimating is sometimes referred to as both a science and an art. In exploring costing techniques, cost estimators strive for the curiosity, creativity, and quest for truth found in both the arts and the sciences while maintaining a firmly analytical mindset. In this context, "creativity" means being versatile in applying various established techniques. The Analogy technique uses an estimate for an existing item to develop an estimate for a similar item. The parametric technique uses an established pattern based on known variables to provide an estimate for an item. By developing a statistical relationship or a cost estimating relationship (CER) between known variables, the analyst can infer the cost value for an item using these known variables. The Build-up technique is based on the concept that an item is the sum of its constituent parts. In this technique, the sum of each constituent element adds up to provide the cost for an item.

Answer 4

A cost estimating analogy is an attempt to estimate costs by drawing a comparison between the item in question and a similar (or analogous) item. Cost estimators develop analogies at the system, subsystem, or component level, and may use multiple analogies at the lower WBS levels to create to a higher level estimate. Generally, some adjustments must be made to the costs of the older item to estimate the new item. These adjustments include those based on programmatic information (e.g., quantity or schedule), physical characteristics (e.g., weight or materials), performance characteristics (e.g., power or pointing accuracy), government or commercial practices, or contract type (e.g., fixed price or cost plus). Economic adjustments for inflation (e.g., converting from constant dollars to then year dollars) are normally considered part of data normalization. For additional information, refer to Module 5 Index Numbers/Inflation. Objectivity is important when making an adjustment. Identify key cost drivers and then determine how the old item relates to the new and how that cost driver affects the costs. Remember that all estimates should be reasonable. The source of the analogy and any adjustments must be logical, credible, and acceptable to an informed

Answer 5

there are many similarities between the old and new systems, adjustments are quantitative, not qualitative, and subjective adjustments are minimized or avoided altogether. An analogy is useful as a cross-check for other methods even when using other more detailed costing techniques. In this case, the estimates should have the same order of magnitude

Answer 6

One advantage is that it can be used before detailed program requirements are known. The more similar the systems, the stronger the analogy and the easier it is to stand up to review. The analogy is also an easy technique to use if a sufficient database exists on an analogous system.

Answer 7

One disadvantage is that there is a tendency to be too subjective in making an analogy. For analogies that require too many subjective adjustments, this technique is not appropriate. An assessment that a new component is 20% more complex without specifying a rationale is not acceptable. Associate the complexity to something less subjective. An appropriate adjustment would be that the new component has 20% more integrated circuits or weighs 20% more than the old component. However, it is often difficult to find sufficient cost, technical, and programmatic data for drawing these types of analogies.

Answer 8

but instead of basing the slope on a number of data points, it is essentially a guess. One point does not determine a line, so the assumption is that the line goes through the origin. Since the analogy is a single data point, it represents a point of departure and any estimate using adjusted analogy constitutes estimating outside the range of the data.

Answer 9

For estimates based on an analogy, there will be uncertainty in both the point of departure and the slope of the adjustment. The point of departure is the analogous system. There is uncertainty in whether or not the chosen point of departure is truly analogous to the new system. An estimate based on an analogy assumes a linear relationship between the old and the new system. The unknown nature of the underlying relationship between the two systems creates uncertainty. If data exists to understand the underlying relationship between the two systems, use the parametric estimating technique.

Answer 10

This means that the new system may have risks that would not have been captured in the costs associated with the old system. Examples include risks associated with new technologies, risks associated with economic conditions (e.g., inflation), and risks associated with the labor environment. There may also be risk characterized by historical growth in the scaling quantity. As a system or project progresses in the program life cycle and the design matures, the scaling quantity used in the analogy may change. Insight into the historical growth of the scaling quantity allows cost estimators to capture the potential for growth as risk. Examples may include size, power, weight, or lines of code. For additional information about quantifying these risks and incorporating them into your cost estimate, refer to Module 9 Cost and Schedule Risk Analysis.

Answer 11

The independent variables are known as cost drivers and could be physical characteristics, performance or operational parameters, programmatic variables, or even other costs. Developing a parametric relationship uses multiple systems to cover a broader range than an analogy. A parametric relationship also allows statistical inferences to be made. These statistical relationships will be able to tell you how well your parametric equation works. When developing a parametric equation, the underlying assumption is that the historical framework on which the parametric relationship is based will remain the same for the new system (e.g., the technology, manufacturing processes, etc., are not drastically changing). A parametric relationship could range in complexity from a simple rule of thumb (e.g., dollars per pound ($/lb)) to a complex regression equation (e.g., Effort = 0.0114 * New SLOC + [0.04 * Reused SLOC]^0.9766). Parametric relationships are commonly referred to as CERs which include rates, factors, and ratios. For additional details on parametric relationships, refer to Module 3 Parametric Estimating. For details on statistically based techniques for developing parametric equations, refer to Module 8 Regression Analysis. Develop CERs using regression whenever possible to enable the statistical inferences previously discussed. Note, however, that many rates, factors, and ratios in use may not be statistically based.

Answer 12

early planning estimates to detailed contract negotiations. Cost estimators generally need an adequate number of relevant data points to develop a parametric estimate. Normalize data so that it can be used to develop a parametric equation that is consistent and complete. Parametric relationships are used early in a program when the design is not well defined. In many programs, changes occur frequently and can easily be reflected in the estimate by adjusting the values of the input parameters as the program becomes better defined.

Answer 13

For example, cost estimators can develop a very detailed estimate of a spacecraft program (e.g., structure, power, or propulsion) and then use a top-level total spacecraft CER to validate the detailed estimate. Ensure that the system being estimated does not fall too far outside the range of the data or parameters used to develop the parametric equation (and preferably not at all). For example, if the new spacecraft is expected to weigh 250 kg and the CER was based on a spacecraft that weighed 2000 kg and higher, then use another CER.

Answer 14

One advantage of the parametric technique is versatility. Parametric estimates can be developed at any level when there is enough data (e.g., system, subsystem, component). As the design changes, cost estimators can quickly and easily capture the effects on the costs by modifying the input parameters. For example, if during the early stages of a program the weight estimates for a system increase by 25%, the cost estimator can plug the new weights into a weight-based CER and reflect the increased weight in the cost estimate. Similarly, analysts can perform sensitivity analysis by varying the input parameters and recording how cost changes with respect to that parameter. A parametric relationship derived through statistical analysis generally has both objective measures of validity, the statistical significance of each estimated coefficient and the model as a whole, and a calculated standard error (a measure used in cost risk analysis). These statistical properties are discussed in Module 8 Regression Analysis. Risk is discussed in more detail in Module 9

Answer 15

Verify that data used in the CER was thoroughly normalized. Data normalization is a crucial step that can be overlooked when using canned CERs. A canned CER is one that someone else developed and the raw data is not accessible. Without understanding how the data is normalized, the cost estimator is trusting that the data was normalized correctly and sufficiently. Data-related issues are discussed in detail in Module 4 Data Collection and Normalization. This concept is referred to as the black box syndrome and occurs when cost estimators plug in the inputs and blindly accept the output without understanding how it is generated. Another disadvantage is that parametric relationships must be updated to capture the most current cost, technical, and programmatic data. If a CER is too dated, it will not provide an adequate cost estimate for the current state. In fact, as technologies change, a dated CER can be exactly wrong. For example, weight often moves from a direct (more weight = more material = more cost) to an indirect (less weight = more advanced material = more cost) relationship.

Answer 16

is a CER for the site activation cost element for a Major Automated Information System (MAIS) developed from 11 data points corresponding to similar systems where installations ranged from seven to 47 workstations, inclusive. In this case, the number of workstations is the cost driver and the equation (Site Act ($K) = 82.8 + 26.5 * Num Wkstn) shows that the estimated cost for site activation is 82.8 thousand dollars plus 26.5 thousand dollars times the number of workstations in the new installation. This CER might provide good estimates provided that the number of workstations in the new installation is between about five and 50. The CER should not be used for independent variable values (number of workstations) much outside the range of the data on which it was originally based. In an extreme case, the cost of site activation for installation with no workstations is about $82.8K, as this is what the equation gives mathematically.

Answer 17

Parametric estimates, however, have the advantage of using statistics to capture the uncertainty in estimating beyond the range of data. The uncertainty in parametric estimates is in the intercept and slope of the regression line and can be measured by the standard error. The standard error is the estimate of the standard deviation of the error in the parametric estimating method. It estimates the standard deviation of the difference between the estimated values and the true values. There is also uncertainty in the distribution around the regression line, which can be estimated by a Prediction Interval (PI). A PI is an interval estimate of a variable. When making an estimate using a regression line where the true value of the regression line is unknown, the PI predicts the distribution around the estimate. The risks associated with estimates based on a parametric estimate can also be characterized as those not included in the historical data set. This means that the new system being estimated may have risks that are also new and did not occur in the historical data. Examples may include risks associated with new technologies, risks associated with economic conditions (e.g., inflation), and risks associated with the labor environment.

Answer 18

As a system or project progresses in the program life cycle and the design matures, the value of the cost driver may grow or change. If cost estimators have insight into the historical growth of the cost driver, they can capture the potential for growth as risk. Examples may include size, power, weight, lines of code, and others. Module 9 Risk Analysis discusses quantifying these risks and incorporating them into cost estimates in more deta

Answer 19

Standards development is the foundation for this technique. Standards can be developed by sources internal or external to a company and are typically available in industry publications. The standards generally reflect an optimal production environment. They capture how long it takes to perform a particular task, based on time and motion studies done in controlled environments. Because no one operates at an optimum level, variance factors (also known as realization factors) are calculated based on measures of a company’s actual experience compared to the standard. The variance factors capture the company’s historical performance against the standard. The labor hours are multiplied by labor rates to determine costs. A detailed parts list or material costs is also required. The parts required for a particular task are identified at the lowest level possible, often down to the nut and bolt level. Consideration is made for quantity and schedule to capture the effects of learning curves and production rate. The stage of the program is also important because the processes and standards used in the manufacture of prototypes are generally different than those for full rate production. The non-touch support (sustaining engineering, quality) labor is often estimated as a factor of touch labor. The build-up technique is discussed in detail in Module 11 Manufacturing Cost Estimating. The lower-level estimating associated with the build-up technique uses IE principles and is sometimes referred to as engineering build-up. This technique may use catalogs to estimate the cost (price) of purchased materials or components and handbooks which contain standards or other information.

Answer 20

The method is also applicable in a touch labor environment where a company is manufacturing a product. Touch labor means that workers (human or robotic) are actually handling the product and performing some sort of work on it as opposed to a support function such as quality assurance (QA) or sustaining engineering. Touch labor may include building a circuit card or assembling an automobile. In these operations, the process is well known and each step of the workflow can be identified, measured, and tracked for performance. The standards and performance factors developed can be used not only to build a database for estimating costs but for managing the work. These metrics are used to identify areas that are performing better or worse than projected.

Answer 21

if anything was overlooked. If time and motion studies are involved, there will be a fairly accurate depiction of the actual process of producing the part or system because the process is mapped down to a very low level of detail. The variance factors applied to standards are based on verifiable actual cost data. The application of the build-up technique is unique to a specific program and manufacturer (contractor). A disadvantage of this technique is that it can be expensive to implement if companies develop their own standards. This method requires extensive data collection and monitoring for variance factors. The product specification must be well known and stable and all product/process changes must be reflected in the estimate. Because many costs (e.g., rework, tooling, quality) are calculated as a percentage of touch labor, small errors at the touch labor level can be magnified into much larger errors. Finally, though it is easy to see what the estimate encompasses at any point, omissions are likely. It is very difficult to anticipate all the actual costs.

Answer 22

The F/A-18 E/F program is currently experiencing a 20% variance (actuals to standards) and an indirect or support labor cost factor of 48% of the touch labor hours. The standards developed to produce the sheet metal element of the new inlet nacelle are 2000 touch labor hours. The estimate is developed using the standard hours and applying the F/A-18 E/F variance factor and support factor. Touch labor hours are estimated to be the standard 2,000 hours times the 1.2 variance factor (100%+20%), or 2,400 hours. Support hours are estimated by applying the 48% factor to the estimated touch labor hours, for 1,152 hours (2,400 times 48%) or a total of 3,552 hours (2,400 + 1.152). Labor rates can then be used to convert these labor hours into costs. Note that build-up may include elements of the other two estimating techniques. Inputs, such as the standard and variance factor in this example, may be developed in whole or in part by analogy to a similar system and the factors used are essentially parametric relationships.

Answer 23

There may be uncertainty in the design specifications, in the performance to standards (usually associated with the labor estimate), and in the unit costs and scrap rates of materials. Risk for estimates developed using a build-up technique may include an adjustment for potential omissions, the historical growth of design specifications, and the difficulty associated with the integration of lower level estimates and/or systems or sub-systems.

Answer 24

Extrapolation from actuals is not one of the three core costing techniques but rather an umbrella term covering the application of other principles. Extrapolation from actuals uses actual costs from past or current items to predict future costs for the same item. There are several variants of extrapolation from actuals, including:

Answer 25

The most basic variant is using averages (e.g. simple or moving) to determine the average actual cost of the units produced to date and using the average cost as the prediction of cost for future units.

Answer 26

This is the most common method used for extrapolation of actuals. Items to consider for this method include the theory used (cumulative average or unit), Learning Curve Slope (LCS), and the theoretical first unit cost (T1). Learning curves, also referred to as cost improvement curves or cost/quantity curves, are discussed in greater detail in Module 7

Answer 27

EACs are a special case of extrapolation from actuals which uses actual cost and schedule data to develop estimates of costs at completion using Earned Value Management (EVM) techniques. Refer to Module 15 Earned Value Management for detailed information on EACs. Note that extrapolation from actuals can be used in conjunction with one of the three basic costing techniques. For example, actual data may be used to construct a parametric estimate or as the basis for an analogy.

Answer 28

One-on-one interviews with experts: Request any documentation available on the subject. Iterate if possible. Round-table discussion: Multiple experts present all sides of an issue, and all the experts stay in a room until a consensus is reached. They document areas of risk in the estimate. Delphi technique: A group of experts provides their answers anonymously to avoid a single person influencing the results in a group environment. The results are summarized and sent back out for coordination/comments. This approach generates a range of opinions and generally results in convergence to a single number or at least tighter range of possible outcomes. This is a specific technique from Operations Research (OR). Using expert opinion or engineering judgment without an accompanying rationale is neither a valid technique nor sound estimating practice. Use insights of functional experts in conjunction with one of the three established costing techniques.

Answer 29

It is hard to justify as a primary estimating method and it is difficult to run risk around an expert opinion. Research further because there is generally additional data that the opinion was based upon. Experts may have their own data, collected over many years and many projects, that were used explicitly or implicitly to develop the estimate. The key is to extract this information from the expert and document it. Expert opinion may be used to corroborate or adjust (as in the analogy technique) objective data. For example, an expert with many years of experience may be able to explain a data point which appears anomalous and help guide the analyst to appropriate treatment of an apparent outlier. It could also be used early in a program’s life cycle for a quick, high-level, Rough Order-of-Magnitude (ROM) estimate. Expert opinion is often used as a secondary method to offer a cross check for an estimate developed using established techniques. A sanity check with experts helps solidify a cost estimate. In some organizations, executives may be comforted knowing that the estimate was validated by a long-standing company expert

Answer 30

Subject Matter Experts (SMEs) usually provide a different perspective and may point out things not previously considered. Interaction with experts allows for a better understanding of the product or process whose cost is being estimated. Remember that expert opinion is completely subjective and can be refuted using objective data. Expert opinion holds little, if any credibility, when not used in conjunction with one of the discussed costing techniques.

Answer 31

Humans are not great random number generators and they tend to understate both variance (spread) and randomness (lack of uniformity). Expert opinion is grounded in anecdotal experience. Unfortunately, faulty recollection of this experience tends to understate effort. The expert may recall getting a report done in a week and thus put down 40 hours, but then forget that they had to work overtime or that the report got returned with review comments that took another couple of days to resolve. In extrapolating from past experience, misremembered or not, experts may be excessively optimistic or conservative.

Answer 32

milestone decisions, budget development, ROM estimates, alternatives comparisons, or proposal development or evaluation? Several different types of estimates are listed above. An estimate for a milestone decision, for example, requires more detail than a ROM.

Answer 33

There are two main ways to structure a cost estimate, top-down an- bottom up. Although any of the costing techniques may be used with either estimating approach, top-down is generally associated with parametric or analogy. It involves known top-level requirements (e.g., weight or power) or parameters to develop an estimate for an entire system. Bottom-up involves working from information at the lowest level to develop an estimate for an entire system. Develop discrete estimates for each element by estimating required labor hours, materials, and other costs and applying direct and indirect rates, and then roll up these lower-level elements to arrive at an estimate for the entire system. Bottom up is most closely associated with the build-up technique, but again, multiple techniques may be used in an estimate.

Answer 34

program phase (e.g., development, production, or O&S), appropriation (e.g., Research, Development, Test, and Evaluation (RDT&E), procurement, or Operations and Maintenance (O&M)), funding source, non-recurring or recurring costs, and direct or indirect costs.

Answer 35

Research which techniques are available from all potential sources. There may be several alternative techniques for any given element. Some customers may have specific models that they want you to use. In addition, many cost estimators maintain a library of costing techniques. Along with existing techniques, consider developing new ones. After researching potential techniques, compare the techniques against the requirements and against each other. To compare techniques look at the underlying data, do statistical comparisons, and make sure that nothing is overlooked or left out. In addition to finding the right techniques, be careful not to use outdated techniques. If a technique was developed based on old data, it may not accurately estimate new technology. Make sure that the technique is for the appropriate program phase. Select the technique most appropriate for the estimate or develop a technique that fills the needs. Remember that multiple techniques can be used in the same estimate. Each technique has strengths and weaknesses and can be applied at different times in the life cycle of a cost estimate. In addition to selecting a primary technique, select a secondary technique as a cross-check.

Answer 36

To increase the credibility of an estimate, check the results using another method. A cross-check is a test of reasonableness. The cross-check may involve using an analogy to compare with the results of a parametric estimate. The cross-check does not have to result in the exact same number, but should be a similar number. At a minimum, it should be the same order of magnitude. Validation of results is more detailed than a cross-check. The intent is to get nearly the same result using different techniques. An example is using a commercially available software estimating package to check the results obtained using another software package. The sense of validation is an Independent Cost Estimate (ICE). Generally, cross-check, validation, and ICE reflect increasing levels of detail, fidelity, and persuasiveness in corroborating a cost estimate.

Answer 37

Documentation is necessary to show the ground rules, assumptions, data, and calculations used to develop an estimate. If there is any doubt about whether to include something, then include it. More information is better than less (within reason). All data used in the analysis should be included in the documentation, with sufficient detail that another analyst would be able to recreate the work based solely on the documentation. This replicability is the acid test for documentation. Documentation is another way to lend credibility to an estimate. A well-documented estimate convinces others that a thorough estimate was developed.

Answer 38

It shows the approximate proportional usage of costing techniques by phase. This figure helps provide an idea of the appropriate time to apply particular techniques. At the beginning of a program, there is more emphasis on using analogies and parametric relationships. In these early phases, gross estimates are expected as detailed estimates are not usually possible due to poor program definition, changing requirements, and scarce cost data. As the program matures, it becomes more defined, additional data is collected, and the estimates get more detailed. Engineering build-up and extrapolation from actuals are used more frequently in the production and deployment phase and O&S phase

Answer 39

including a definition of each technique, when to apply them, how to apply them, and the strengths and weaknesses of each technique. Remember that there are many different techniques, which can be applied in different situations. Carefully consider each technique analytically and select the best technique or techniques for the program.

Answer 40

Definition, Examples, Advantages, Disadvantages, and Risk Implications of each technique Mapping of techniques to life-cycle phases Operating and Support (O&S) Estimating

Answer 41

Schedule Estimating Other advanced topic

Answer 42

The Analogy technique refers to comparing the cost of an item to be estimated to that of a similar item. The Parametric technique uses a mathematical relationship based on historical data to relate cost to one or more technical, performance, cost, or programmatic parameters. The Build-up technique involves estimating costs at the lowest definable level and typically applies to Industrial Engineering (IE). These techniques have different uses and degrees of applicability during a program's life cycle. Cost estimators use one or more of these techniques depending upon the level of fidelity required. Other techniques include expert opinion and extrapolation from actuals. The Expert Opinion technique uses subjective information from Subject Matter Experts (SMEs) to corroborate or adjust cost estimates.

Answer 43

Extrapolation from actuals is an umbrella term referring to the use of actuals from past or current items to predict future costs for the same item. It includes techniques such as learning curve, averages, and earned value management.  For example: contract actuals from a contract that has been in place for an extended period can be used to estimate the follow-on costs for the contract.  For another example: extrapolation from actuals should be used to predict follow-on unit or lot costs when estimating production costs

Answer 44

One-on-one interviews with experts: Request any documentation available on the subject. Iterate if possible. Round-table discussion: Multiple experts present all sides of an issue, and all the experts stay in a room until a consensus is reached. They document areas of risk in the estimate. Delphi technique: A group of experts provides their answers anonymously to avoid a single person influencing the results in a group environment. The results are summarized and sent back out for coordination/comments. This approach generates a range of opinions and generally results in convergence to a single number or at least tighter range of possible outcomes. This is a specific technique from Operations Research (OR). Using expert opinion or engineering judgment without an accompanying rationale is neither a valid technique nor sound estimating practice. Use insights of functional experts in conjunction with one of the three established costing techniques.

Answer 45

For estimates based on an analogy, there are risks not included in the analogous system. This means that the new system may have risks that would not have been captured in the costs associated with the old system. Examples include risks associated with new technologies, risks associated with economic conditions (e.g., inflation), and risks associated with the labor environment. In this image, the solid gray line shows the true underlying cost driver relationship and the two dotted gray lines show plus or minus one standard deviation (sigma) of the associated error term. Note the fundamental assumptions for OLS regression for making appropriate comparisons using the parametric technique later. The green square represents an estimate that is based on an analogy. In this example, the analogous point is circled in red. The line between the estimate and the analogous point is linear and passes through the origin. The error associated with the estimate based on an average was included to determine the uncertainty surrounding the estimate based on an analogy.

Unit I - Module 2 Cost Estimating Techniques Flashcards

(70 cards)