{ "@context": "https://schema.org", "@type": "Organization", "name": "Brainscape", "url": "https://www.brainscape.com/", "logo": "https://www.brainscape.com/pks/images/cms/public-views/shared/Brainscape-logo-c4e172b280b4616f7fda.svg", "sameAs": [ "https://www.facebook.com/Brainscape", "https://x.com/brainscape", "https://www.linkedin.com/company/brainscape", "https://www.instagram.com/brainscape/", "https://www.tiktok.com/@brainscapeu", "https://www.pinterest.com/brainscape/", "https://www.youtube.com/@BrainscapeNY" ], "contactPoint": { "@type": "ContactPoint", "telephone": "(929) 334-4005", "contactType": "customer service", "availableLanguage": ["English"] }, "founder": { "@type": "Person", "name": "Andrew Cohen" }, "description": "Brainscape’s spaced repetition system is proven to DOUBLE learning results! Find, make, and study flashcards online or in our mobile app. Serious learners only.", "address": { "@type": "PostalAddress", "streetAddress": "159 W 25th St, Ste 517", "addressLocality": "New York", "addressRegion": "NY", "postalCode": "10001", "addressCountry": "USA" } }

Unit I - Module 2 Cost Estimating Techniques Flashcards

(70 cards)

1
Q

Which of the following estimating techniques relies heavily on the subjective opinion of an individual?

A. Parametric
B. Extrapolation from Actuals
C. Expert Opinion
D. Analogy
E. Build-Up
F. None of the Above
G. All of the Above

A

C. Expert Opinion

Parametric estimates and estimates based on Analogy use data from completed historical programs to
project future costs.

Extrapolation from Actuals is also data-based, though the data used may be from
either complete or incomplete units.

A BuildiUp estimates at the lowest possible level, and uses industrial engineering techniques, such as time standards, to develop an estimate at a low level.

Expert Opinion, on the other hand, relies on the subject opinion of the individual expert.
In fact, Expert Opinion is not generally accepted as a valid technique in and or itself, but rather the estimator’s judgment and expertise is used in applying one of the other methods.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

In estimating the cost of a “follow-on” contract for a program that has been in existence for an extended period of time for which the actual costs incurred to date on the specific program are readily available, what would be the preferred estimating methodology?

A. Expert Opinion
B. Extrapolation from Actuals
C. Analogy
D. Parametric
E. Build-Up
F. All of the Above
G. None of the Above

A

B. Extrapolation from Actuals

Extrapolation from Actuals uses actuals from past or current items to predict future costs for the same item.

Extrapolation from Actuals is an umbrella term covering several estimating techniques, including learning curve and earned value management (EVM) estimates at complete (EACs), will be explored in subsequent modules.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Which of the following estimating techniques is based upon knowledge of the cost of a similar item and uses adjustment factors for the complexity, technical, or physical differences between the items?

A. Analogy
B. Expert Opinion
C. Extrapolation from Actuals
D. Parametric
E. Build-Up
F. All of the Above
G. None of the Above

A

A. Analogy

A cost estimating analogy is an attempt to estimate costs by drawing a comparison between the item in question and a similar (or analogous) item.

An analogy can be done at the system, subsystem, or component level. Multiple analogies can be used at the lower WBS levels to build up to a higher level
estimate.

Two systems are rarely identical, and adjustments must be made to account for the differences between the old item and the new item. These adjustments are based on complexity, technical, or physical differences between the two items.

One difference between estimating parametrically and estimating with an analogy is that in parametric estimating, the analyst has multiple similar systems used to create a Cost Estimating Relationship, where in an analogy only one historical data point is used.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Top-level technical data, such as the mass properties of an item, would most likely be used to support which estimating technique?

A. Build-Up
B. Parametric
C. Extrapolation From Actuals
D. Analogy
E. Expert Opinion
F. All of the Above
G. None of the Above

A

B. Parametric

The parametric costing technique is a mathematical relationship between certain characteristics (such as weight, thrust, or power) as one or more independent variables of a system and the
system’s cost as a dependent variable.

If the data were at a lower level, such as a detailed mass properties report, that would more likely support the Build-Up technique.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

A quantitative methodology for estimating the touch labor of manufacturing a product using labor standards is most likely to be part of the application of which of the following costing techniques?

A. Expert Opinion
B. Extrapolation From Actuals
C. Build-Up
D. Analogy
E. Parametric
F. All of the Above
G. None of the Above

A

C. Build-Up

The Build-Up method builds estimates for higher-level cost elements by summing or “rolling up” detailed estimates for lower-level cost elements.

A build-up is characterized by estimating at the lowest definable level at which data exist. Standards Development is the cornerstone of this technique.

The standards generally reflect an optimal production environment. They capture how long it takes to perform particular task, based on time and motion studies done in controlled environments.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

True or False. A cost analyst that has the labor standards for manufacturing a landing gear bracket for the B-2 Bomber has enough information to determine the total cost of the landing gear bracket.

A

False

False. Manufacturing a landing gear bracket requires inputs beyond touch labor.
Material costs, for example, must also be included. On the labor side, support labor, ancillary labor, and indirects (overhead) must be considered (see Module 11 Manufacturing Cost Estimating for more
detail on these). This demonstrates one of the pitfalls of build-up estimating: omissions are likely.

It is very difficult to anticipate all costs beforehand. It is useful to cross-check a build up estimate with another estimating technique, like an analogy.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

When using a weight-based parametric estimating technique, it is necessary to know the material type being used (e.g., steel or composites).

True or False.

A

True

The variables used in the Cost Estimating Relationship (CER) developed using a parametric technique must either be constant or the data must be normalized such that differences do no effect the estimate.

The cost-per-pound of Steel vs. Composite Material is not equal. A weight-based parametric estimating technique is influenced by the type of material, and this variable must be known.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Which of the following is a major shortcoming of using expert opinion as an estimating methodology?

A. It relies heavily on performance data
B. It takes a purely quantitative approach to estimating
C. Actual cost data is readily available
D. The opinion of an individual is not refutable
E. The estimate is always understated
F. It is likely that other experts exist with differing opinions
G. None of the Above

A

F. It is likely that other experts exist with differing opinions

Expert opinion is subjective. A different expert will likely have a different, subjective, opinion.

Because of this, without support from another, objective, estimating methodology, expert opinion
has little credibility.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

A relationship between power (in kilowatts) and cost (in thousands of dollars) is an example of which methodology?

A. Extrapolation from Actuals
B. Expert Opinion
C. Analogy
D. Build-Up
E. SWAG Estimating
F. Atmospheric Extraction
G. Parametric

A

G. Parametric

The parametric costing technique is a mathematical relationship between certain characteristics (such as weight, thrust, or power) as one or more independent variables of a system and the
system’s cost as a dependent variable.

The stated relationship between kilowatts and cost is a parametric relationship.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Which methodology would be best to use when estimating the cost of the 20th lot of F/A - 18 C/D containing units 950 through 980 when complete historical costs are available for units 1 through 949?

A. Extrapolation From Actuals
B. Parametric
C. SWAG Estimating
D. Atmospheric Extraction
E. Analogy
F. Expert Opinion
G. Industrial Engineering

A

A. Extrapolation From Actuals

Extrapolation from actuals is best suited for follow–on units/lots when you have existing datafrom current and past production lots.

N.b., this assumes little change in the product design or manufacturing process from the previous units. If large changes exist, careful adjustments may have to be made or some other method chosen.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

When developing a cost estimate, you should always use the same costing technique for every element.

True / False

A

False

False. Some techniques may be better suited than others for certain elements in the estimate.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

You must develop a cost estimate for the structure subsystem of a new communications satellite, COM2. The new satellite is similar to COM1, a communications satellite your company recently built. Through discussion with the structural engineers, you collected the data to the right. What should the estimated cost of the structure subsytem be?

A. $4 M
B. $5 M
C. $6.25 M
D. $10 M
E. None of the Above

A

C. $6.25 M

The data is recent, and the subsystem similar, so the analogous data does not need to be normalized
further. N.b., this is a cartoon example.

Defense of this cost estimate would require more in-depthinformation regarding, amongst other things, what constitutes “similar” and “recent.”

$5M * (250 kg/200 kg)= $6.25 M
or

$5M / 200 kg = 0.025 $M/kg
0.025 $M/kg * 250 kg = $6.25 M

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

If a parametric Cost Estimating Relationship (CER) is given by the equation y = a + bx, which of the following statements is true?

I. x is the independent variable.
II. y usually represents cost.
III. When x increases by 1, y increases by a.

A. I only
B. II only
C. III only
D. I and II only
E. I and III only
F. II and III only
G. I, II, and III

A

D. I and II only

Statements I and II are true: x is the independent variable in the equation and, in a CER, y usually represents costs. Statement III is incorrect. The correct statement is: when x increases by 1, y
increases by b, since b is the slope.

If you remember “rise over run,” you’ll see that the line goes up b units as it goes over 1 unit.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Which of the following pairs are the correct terms for the two main ways to structure a cost estimate?

A. Tops Down / Bottoms Up
B. Top Down/ Bottoms Up
C. Tops Down / Bottom Up
D. Top Down / Bottom Up

A

D. Top Down / Bottom Up

Top Down is generally associated with the use of Parametrics or Analogy; Bottom Up involves working from information at the lowest level to develop an estimate for an entire system.

When referencing either, remember they are both correctly referenced in the singular.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

The standard for an item is 800 touch labor hours, and the documented variance factor is 18%. The corresponding indirect labor factor is 60%. What are the estimated touch labor hours for this item?

A. 800
B. 944
C. 1280
D. 1510.4

A

B. 944

Apply variance to touch labor hours:

800*1.18= 944 hours

Do not add in the additional 60% because that is indirect and the questions asks for touch labor (direct)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

True or False. A Cross-Check should result in exactly the same estimate as the original estimate.

A

False

False. A Cross-Check should be a similar number, of at least the same order of magnitude, but it will likely not be exactly the same as the original estimate. In fact, a cross check that results in exactly the same estimate as the original is suspicious.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Which of the following estimating techniques is an objective method of estimating?

A. Engineering Judgment
B. Delphi Technique
C. Round Table
D. Expert Opinion
E. None of the Above
F. All of the Above

A

E. None of the Above

All estimating techniques mentioned here are different guises for Expert Opinion, and hence inherently subjective.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Which of the following are risks associated with Parametric Estimating?

A. Imprecision of the intercept and slope of the regression line.
B. New technologies are not included in the historical data set.
C. Cost drivers change over time.
D. A and B only
E. B and C only
F. All of the above
G. None of the above

A

E. B and C only

Choice A is a concern, but relates to uncertainty. Choices B and C are risks.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Which of these general statements is true?
I. Analogies, Engineering Build-Ups and Parametric Estimates are all used equally throughout a program life cycle
II. Analogies and Parametric Estimates are more prevalent towards the end of a program life cycle, and Engineering Build-Ups are more prevalent earlier in the life cycle.
III. Analogies and Parametric Estimates are more prevalent towards the beginning of a program life cycle, and Engineering Build-Ups are more prevalent later in the life cycle

A

Statement III is generally true.
When the system is in an early phase (like the design phase), engineering estimates are not possible.
As the program matures, the estimates are more detailed.
Analogies and parametrics can still be useful later in a program’s life cycle, but they tend to be
phased out in favor of Build-Ups and Extrapolation From Actuals for most estimates.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

True or False. All cost estimating methodologies have some weakness

A

True

True, all cost estimating methodologies have some weakness. It is important to understand what those weaknesses are and when to use which methodology.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Analogy, parametric, and build-up are the three primary costing techniques used in developing an estimate.

A

The Analogy technique refers to comparing the cost of an item to be estimated to that of a similar item.

The Parametric technique uses a mathematical relationship based on historical data to relate cost to one or more technical, performance, cost, or programmatic parameters.

The Build-up technique involves estimating costs at the lowest definable level and typically applies to Industrial Engineering (IE).

These costing techniques are discussed within the context to which they apply, specifically a Cost Element Structure (CES), and how they relate to the Work Breakdown Structure (WBS).

Developing a cost and schedule estimate is the practical application of these techniques. Cost estimators typically use multiple techniques: one to derive the primary estimate and one or more to provide a cross-check to give confidence to the estimate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Other techniques include expert opinion and extrapolation from actuals.

A

The Expert Opinion technique uses subjective information from Subject Matter Experts (SMEs) to corroborate or adjust cost estimates.

The Extrapolation from actuals technique uses data from prototypes or complete or partially complete units to project the cost of future units; it may also use earned value data to develop an Estimate At Completion (EAC) for any contract, phase, or program

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

the basic application of cost estimating principles

Past
Present
Future

A

Past
Historical cost
Historical productivity
Past systems data

Present
Current pricing data
Production data
Productivity data
Labor rates
Inflation rates
Normalization

Future
Learning curve analysis
Economic analysis
Parametric estimating
Adjusted analogies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Cost estimates always have some degree of uncertainty and risk, but cost estimators can quantify that uncertainty to accurately support decision making

A

Precision is the spread of the range of outcomes that the estimate produces. A narrow range requires greater precision while a wider range requires less.

By contrast, accuracy conveys whether the range is centered on the true value. If the center of the range is close to the true value, it is an accurate estimate.

Conversely, if it is nowhere near the center then the estimate is inaccurate. The standard illustration provided in many introductory science textbooks is a dartboard. If the darts cluster tightly about a single point, then the throws are precise. If the darts cluster around the bull’s eye, then the throws are accurate even if they are not tightly clustered.

Precision and accuracy in cost estimates are ideal, but accuracy is more important. Precision can give a false sense of security.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
The idea of precision and accuracy manifests in cost estimating as uncertainty and risk.
Uncertainty captures the range of possible outcomes of the estimate which characterizes its precision. This is best done as a probability distribution, usually an empirical one resulting from a Monte Carlo simulation which can then be summarized with a confidence interval (e.g., one million dollars, plus or minus 15 percent). Risk refers to the upward shift applied to the cost estimate range to account for the fact that unadjusted estimates tend to be systematically low. This adjustment is intended to eliminate what appears to be an inherent bias in estimates and improves the estimate's accuracy. For instance, if costs come in on average 20% higher than the estimates, cost estimators may want to add 20% to the estimate. At first, this seems like a high percentage, but it is not atypical of historical cost growth on some programs. The proper treatment of uncertainty and risk is difficult because each involves some probability and statistics, but developing a best attempt is essential to produce meaningful and useful estimates.
26
The practice of cost estimating is not a repeatable experiment.
Same systems or units are not built repeatedly so that the range of costs can be observed. The system or unit is built once and one possible value is determined, which includes random or unknown elements, causing it to be above or below average. Even when building multiple units, as in a production run, each individual unit has its own random or unknown elements that produce variation around what should be a smooth learning curve.
27
Costing techniques are the building blocks of a cost estimate. They provide structure and predict future costs based on historical data. Costing techniques rely on statistical properties, logical relationships, and preference
Cost estimating is sometimes referred to as both a science and an art. In exploring costing techniques, cost estimators strive for the curiosity, creativity, and quest for truth found in both the arts and the sciences while maintaining a firmly analytical mindset. In this context, "creativity" means being versatile in applying various established techniques. The Analogy technique uses an estimate for an existing item to develop an estimate for a similar item. The parametric technique uses an established pattern based on known variables to provide an estimate for an item. By developing a statistical relationship or a cost estimating relationship (CER) between known variables, the analyst can infer the cost value for an item using these known variables. The Build-up technique is based on the concept that an item is the sum of its constituent parts. In this technique, the sum of each constituent element adds up to provide the cost for an item.
28
Costing Techniques Analogy Cost estimators think in analogies by comparing one item to another. In layman’s terms, an analogy is a comparison drawn for illustrative purposes, and it is generally a little more precise than a metaphor or a simile in that it seeks to exploit parallel logical (and sometimes quantitative) relationships
A cost estimating analogy is an attempt to estimate costs by drawing a comparison between the item in question and a similar (or analogous) item. Cost estimators develop analogies at the system, subsystem, or component level, and may use multiple analogies at the lower WBS levels to create to a higher level estimate. Generally, some adjustments must be made to the costs of the older item to estimate the new item. These adjustments include those based on programmatic information (e.g., quantity or schedule), physical characteristics (e.g., weight or materials), performance characteristics (e.g., power or pointing accuracy), government or commercial practices, or contract type (e.g., fixed price or cost plus). Economic adjustments for inflation (e.g., converting from constant dollars to then year dollars) are normally considered part of data normalization. For additional information, refer to Module 5 Index Numbers/Inflation. Objectivity is important when making an adjustment. Identify key cost drivers and then determine how the old item relates to the new and how that cost driver affects the costs. Remember that all estimates should be reasonable. The source of the analogy and any adjustments must be logical, credible, and acceptable to an informed
29
Costing Techniques Analogy Application Cost estimators typically use analogies early in the program life cycle when there is little definition in the new program or a lack of a pre-existing cost model. Most development programs have some heritage in design. The heritage or legacy system is used for comparison to the new system to be estimated. One of the first considerations when assessing the cost of a new development program is the percent of new design versus heritage or reuse. This assessment can be performed at system, subsystem, and component levels. An analogy can also be used when there is not enough data or program definition to develop a cost estimate using a more detailed technique. There should be a strong parallel between the historical system and the item to be estimated. Analogy is a one-for-one comparison. An analogy works best when:
there are many similarities between the old and new systems, adjustments are quantitative, not qualitative, and subjective adjustments are minimized or avoided altogether. An analogy is useful as a cross-check for other methods even when using other more detailed costing techniques. In this case, the estimates should have the same order of magnitude
30
Costing Techniques Analogy Considerations There are several advantages of using an analogy.
One advantage is that it can be used before detailed program requirements are known. The more similar the systems, the stronger the analogy and the easier it is to stand up to review. The analogy is also an easy technique to use if a sufficient database exists on an analogous system.
31
Costing Techniques Analogy Considerations There are also disadvantages to using an analogy
One disadvantage is that there is a tendency to be too subjective in making an analogy. For analogies that require too many subjective adjustments, this technique is not appropriate. An assessment that a new component is 20% more complex without specifying a rationale is not acceptable. Associate the complexity to something less subjective. An appropriate adjustment would be that the new component has 20% more integrated circuits or weighs 20% more than the old component. However, it is often difficult to find sufficient cost, technical, and programmatic data for drawing these types of analogies.
32
Costing Techniques Analogy Comparison In comparing the analogy technique with the parametric technique, an adjusted analogy is like a linear regression
but instead of basing the slope on a number of data points, it is essentially a guess. One point does not determine a line, so the assumption is that the line goes through the origin. Since the analogy is a single data point, it represents a point of departure and any estimate using adjusted analogy constitutes estimating outside the range of the data.
33
Costing Techniques Analogy Uncertainty and Risk Uncertainty is the range of possible outcomes of the estimate
For estimates based on an analogy, there will be uncertainty in both the point of departure and the slope of the adjustment. The point of departure is the analogous system. There is uncertainty in whether or not the chosen point of departure is truly analogous to the new system. An estimate based on an analogy assumes a linear relationship between the old and the new system. The unknown nature of the underlying relationship between the two systems creates uncertainty. If data exists to understand the underlying relationship between the two systems, use the parametric estimating technique.
34
Costing Techniques Analogy Uncertainty and Risk For estimates based on an analogy, there are risks not included in the analogous system
This means that the new system may have risks that would not have been captured in the costs associated with the old system. Examples include risks associated with new technologies, risks associated with economic conditions (e.g., inflation), and risks associated with the labor environment. There may also be risk characterized by historical growth in the scaling quantity. As a system or project progresses in the program life cycle and the design matures, the scaling quantity used in the analogy may change. Insight into the historical growth of the scaling quantity allows cost estimators to capture the potential for growth as risk. Examples may include size, power, weight, or lines of code. For additional information about quantifying these risks and incorporating them into your cost estimate, refer to Module 9 Cost and Schedule Risk Analysis.
35
Costing Techniques Parametric The parametric costing technique is a mathematical relationship between certain characteristics (e.g., weight, thrust, or power) as one or more independent variables of a system and the system’s cost as a dependent variable. These relationships are developed using data collected on similar programs
The independent variables are known as cost drivers and could be physical characteristics, performance or operational parameters, programmatic variables, or even other costs. Developing a parametric relationship uses multiple systems to cover a broader range than an analogy. A parametric relationship also allows statistical inferences to be made. These statistical relationships will be able to tell you how well your parametric equation works. When developing a parametric equation, the underlying assumption is that the historical framework on which the parametric relationship is based will remain the same for the new system (e.g., the technology, manufacturing processes, etc., are not drastically changing). A parametric relationship could range in complexity from a simple rule of thumb (e.g., dollars per pound ($/lb)) to a complex regression equation (e.g., Effort = 0.0114 * New SLOC + [0.04 * Reused SLOC]^0.9766). Parametric relationships are commonly referred to as CERs which include rates, factors, and ratios. For additional details on parametric relationships, refer to Module 3 Parametric Estimating. For details on statistically based techniques for developing parametric equations, refer to Module 8 Regression Analysis. Develop CERs using regression whenever possible to enable the statistical inferences previously discussed. Note, however, that many rates, factors, and ratios in use may not be statistically based.
36
Costing Techniques Parametric The parametric technique can be used in a wide variety of situations. These situations range from
early planning estimates to detailed contract negotiations. Cost estimators generally need an adequate number of relevant data points to develop a parametric estimate. Normalize data so that it can be used to develop a parametric equation that is consistent and complete. Parametric relationships are used early in a program when the design is not well defined. In many programs, changes occur frequently and can easily be reflected in the estimate by adjusting the values of the input parameters as the program becomes better defined.
37
Costing Techniques Parametric The parametric technique is also good as a secondary or cross-check technique
For example, cost estimators can develop a very detailed estimate of a spacecraft program (e.g., structure, power, or propulsion) and then use a top-level total spacecraft CER to validate the detailed estimate. Ensure that the system being estimated does not fall too far outside the range of the data or parameters used to develop the parametric equation (and preferably not at all). For example, if the new spacecraft is expected to weigh 250 kg and the CER was based on a spacecraft that weighed 2000 kg and higher, then use another CER.
38
Costing Techniques Parametric Considerations There are several advantages to the parametric technique
One advantage of the parametric technique is versatility. Parametric estimates can be developed at any level when there is enough data (e.g., system, subsystem, component). As the design changes, cost estimators can quickly and easily capture the effects on the costs by modifying the input parameters. For example, if during the early stages of a program the weight estimates for a system increase by 25%, the cost estimator can plug the new weights into a weight-based CER and reflect the increased weight in the cost estimate. Similarly, analysts can perform sensitivity analysis by varying the input parameters and recording how cost changes with respect to that parameter. A parametric relationship derived through statistical analysis generally has both objective measures of validity, the statistical significance of each estimated coefficient and the model as a whole, and a calculated standard error (a measure used in cost risk analysis). These statistical properties are discussed in Module 8 Regression Analysis. Risk is discussed in more detail in Module 9
39
Costing Techniques Parametric A disadvantage of the parametric technique is that the underlying database upon which the CER is based must be consistent and sufficiently robust.
Verify that data used in the CER was thoroughly normalized. Data normalization is a crucial step that can be overlooked when using canned CERs. A canned CER is one that someone else developed and the raw data is not accessible. Without understanding how the data is normalized, the cost estimator is trusting that the data was normalized correctly and sufficiently. Data-related issues are discussed in detail in Module 4 Data Collection and Normalization. This concept is referred to as the black box syndrome and occurs when cost estimators plug in the inputs and blindly accept the output without understanding how it is generated. Another disadvantage is that parametric relationships must be updated to capture the most current cost, technical, and programmatic data. If a CER is too dated, it will not provide an adequate cost estimate for the current state. In fact, as technologies change, a dated CER can be exactly wrong. For example, weight often moves from a direct (more weight = more material = more cost) to an indirect (less weight = more advanced material = more cost) relationship.
40
Costing Techniques Parametric Automated Information Systems (AIS) are defined in DoD Instruction (DoDI) 5000.02 (Table 1) as “a system of computer hardware, computer software, data or telecommunications that performs functions such as collecting, processing, storing, transmitting, and displaying information. Excluded are computer resources, both hardware and software, that are an integral part of a weapon or weapon system; used for highly sensitive classified programs (as determined by the Secretary of Defense); used for other highly sensitive information technology (IT) programs (as determined by the DoD CIO; or determined by the DAE or designee to be better overseen as a non- AIS program (e.g., a program with a low ratio of RDT&E funding to total program acquisition costs or that requires significant hardware development)”. Additionally, ACAT IA programs are deemed to be MAIS. A MAIS is a DoD acquisition program for an automated information system (AIS) that is either designated by the MDA as a MAIS, or estimated to exceed certain dollar thresholds (See 10 U.S.C. 2445a)[4]. An example of parametric estimating
is a CER for the site activation cost element for a Major Automated Information System (MAIS) developed from 11 data points corresponding to similar systems where installations ranged from seven to 47 workstations, inclusive. In this case, the number of workstations is the cost driver and the equation (Site Act ($K) = 82.8 + 26.5 * Num Wkstn) shows that the estimated cost for site activation is 82.8 thousand dollars plus 26.5 thousand dollars times the number of workstations in the new installation. This CER might provide good estimates provided that the number of workstations in the new installation is between about five and 50. The CER should not be used for independent variable values (number of workstations) much outside the range of the data on which it was originally based. In an extreme case, the cost of site activation for installation with no workstations is about $82.8K, as this is what the equation gives mathematically.
41
Costing Techniques Parametric Uncertainty and Risk The uncertainty and risk inherent in parametric estimates can be characterized similarly to the uncertainty and risk in estimates based on an analogy.
Parametric estimates, however, have the advantage of using statistics to capture the uncertainty in estimating beyond the range of data. The uncertainty in parametric estimates is in the intercept and slope of the regression line and can be measured by the standard error. The standard error is the estimate of the standard deviation of the error in the parametric estimating method. It estimates the standard deviation of the difference between the estimated values and the true values. There is also uncertainty in the distribution around the regression line, which can be estimated by a Prediction Interval (PI). A PI is an interval estimate of a variable. When making an estimate using a regression line where the true value of the regression line is unknown, the PI predicts the distribution around the estimate. The risks associated with estimates based on a parametric estimate can also be characterized as those not included in the historical data set. This means that the new system being estimated may have risks that are also new and did not occur in the historical data. Examples may include risks associated with new technologies, risks associated with economic conditions (e.g., inflation), and risks associated with the labor environment.
42
Costing Techniques Parametric Uncertainty and Risk There may also be risk characterized by historical growth of the cost driver.
As a system or project progresses in the program life cycle and the design matures, the value of the cost driver may grow or change. If cost estimators have insight into the historical growth of the cost driver, they can capture the potential for growth as risk. Examples may include size, power, weight, lines of code, and others. Module 9 Risk Analysis discusses quantifying these risks and incorporating them into cost estimates in more deta
43
Costing Techniques Build Up The build-up method builds estimates for higher-level cost elements by summing or rolling up detailed estimates for lower-level cost elements. While roll-up is common to estimates produced by any combination of the three techniques, the build-up method is characterized by estimating at the lowest definable level at which data exists. There are several elements which can be used to develop estimates at these lower levels using this method.
Standards development is the foundation for this technique. Standards can be developed by sources internal or external to a company and are typically available in industry publications. The standards generally reflect an optimal production environment. They capture how long it takes to perform a particular task, based on time and motion studies done in controlled environments. Because no one operates at an optimum level, variance factors (also known as realization factors) are calculated based on measures of a company’s actual experience compared to the standard. The variance factors capture the company’s historical performance against the standard. The labor hours are multiplied by labor rates to determine costs. A detailed parts list or material costs is also required. The parts required for a particular task are identified at the lowest level possible, often down to the nut and bolt level. Consideration is made for quantity and schedule to capture the effects of learning curves and production rate. The stage of the program is also important because the processes and standards used in the manufacture of prototypes are generally different than those for full rate production. The non-touch support (sustaining engineering, quality) labor is often estimated as a factor of touch labor. The build-up technique is discussed in detail in Module 11 Manufacturing Cost Estimating. The lower-level estimating associated with the build-up technique uses IE principles and is sometimes referred to as engineering build-up. This technique may use catalogs to estimate the cost (price) of purchased materials or components and handbooks which contain standards or other information.
44
Costing Techniques Build Up Application The build-up method is used when detailed information exists at a low level about an item (e.g., how many hours or how many parts).
The method is also applicable in a touch labor environment where a company is manufacturing a product. Touch labor means that workers (human or robotic) are actually handling the product and performing some sort of work on it as opposed to a support function such as quality assurance (QA) or sustaining engineering. Touch labor may include building a circuit card or assembling an automobile. In these operations, the process is well known and each step of the workflow can be identified, measured, and tracked for performance. The standards and performance factors developed can be used not only to build a database for estimating costs but for managing the work. These metrics are used to identify areas that are performing better or worse than projected.
45
Costing Techniques Build Up Considerations There are several advantages to the build-up technique. By developing an estimate at a low level, cost estimators can show exactly what the estimate covers and helps determine
if anything was overlooked. If time and motion studies are involved, there will be a fairly accurate depiction of the actual process of producing the part or system because the process is mapped down to a very low level of detail. The variance factors applied to standards are based on verifiable actual cost data. The application of the build-up technique is unique to a specific program and manufacturer (contractor). A disadvantage of this technique is that it can be expensive to implement if companies develop their own standards. This method requires extensive data collection and monitoring for variance factors. The product specification must be well known and stable and all product/process changes must be reflected in the estimate. Because many costs (e.g., rework, tooling, quality) are calculated as a percentage of touch labor, small errors at the touch labor level can be magnified into much larger errors. Finally, though it is easy to see what the estimate encompasses at any point, omissions are likely. It is very difficult to anticipate all the actual costs.
46
Costing Techniques Build Up Example In this build-up technique example, cost estimators must develop an estimate for the sheet metal element of the Joint Strike Fighter's (JSF) inlet nacelle. The new inlet nacelle is similar to what is currently being produced on the F/A–18 E/F program. The contractor has a stringent work measurement program in place to track performance based on industrial engineering standards.
The F/A-18 E/F program is currently experiencing a 20% variance (actuals to standards) and an indirect or support labor cost factor of 48% of the touch labor hours. The standards developed to produce the sheet metal element of the new inlet nacelle are 2000 touch labor hours. The estimate is developed using the standard hours and applying the F/A-18 E/F variance factor and support factor. Touch labor hours are estimated to be the standard 2,000 hours times the 1.2 variance factor (100%+20%), or 2,400 hours. Support hours are estimated by applying the 48% factor to the estimated touch labor hours, for 1,152 hours (2,400 times 48%) or a total of 3,552 hours (2,400 + 1.152). Labor rates can then be used to convert these labor hours into costs. Note that build-up may include elements of the other two estimating techniques. Inputs, such as the standard and variance factor in this example, may be developed in whole or in part by analogy to a similar system and the factors used are essentially parametric relationships.
47
Costing Techniques Build Up Uncertainty and Risk There are several areas in which uncertainty may affect an estimate based on a build-u
There may be uncertainty in the design specifications, in the performance to standards (usually associated with the labor estimate), and in the unit costs and scrap rates of materials. Risk for estimates developed using a build-up technique may include an adjustment for potential omissions, the historical growth of design specifications, and the difficulty associated with the integration of lower level estimates and/or systems or sub-systems.
48
Costing Techniques Other Techniques Extrapolation from Actuals
Extrapolation from actuals is not one of the three core costing techniques but rather an umbrella term covering the application of other principles. Extrapolation from actuals uses actual costs from past or current items to predict future costs for the same item. There are several variants of extrapolation from actuals, including:
49
Costing Techniques Other Techniques Averages:
The most basic variant is using averages (e.g. simple or moving) to determine the average actual cost of the units produced to date and using the average cost as the prediction of cost for future units.
50
Costing Techniques Other Techniques Learning Curves:
This is the most common method used for extrapolation of actuals. Items to consider for this method include the theory used (cumulative average or unit), Learning Curve Slope (LCS), and the theoretical first unit cost (T1). Learning curves, also referred to as cost improvement curves or cost/quantity curves, are discussed in greater detail in Module 7
51
Costing Techniques Other Techniques Estimate at Completion (EAC)
EACs are a special case of extrapolation from actuals which uses actual cost and schedule data to develop estimates of costs at completion using Earned Value Management (EVM) techniques. Refer to Module 15 Earned Value Management for detailed information on EACs. Note that extrapolation from actuals can be used in conjunction with one of the three basic costing techniques. For example, actual data may be used to construct a parametric estimate or as the basis for an analogy.
52
Costing Techniques Other Techniques Expert Opinion Expert opinion involves using an expert or group of experts to estimate the cost of a system. This is often known as engineering judgment. Expert opinion is generally viewed as too subjective. To alleviate this concern, continue delving into the opinion to determine if the expert is basing that opinion on some real data. After identifying the data the opinion is based upon, obtain copies and document the source. Do not ask experts to estimate outside the bounds of their experience. Request copies of any referenced data. Ask for another expert for areas outside the scope of their expertise. Validate credentials of the expert and identify alternate experts for the same scope. Formulate consensus after individual consultation. There are several different approaches to expert opinion, which include
One-on-one interviews with experts: Request any documentation available on the subject. Iterate if possible. Round-table discussion: Multiple experts present all sides of an issue, and all the experts stay in a room until a consensus is reached. They document areas of risk in the estimate. Delphi technique: A group of experts provides their answers anonymously to avoid a single person influencing the results in a group environment. The results are summarized and sent back out for coordination/comments. This approach generates a range of opinions and generally results in convergence to a single number or at least tighter range of possible outcomes. This is a specific technique from Operations Research (OR). Using expert opinion or engineering judgment without an accompanying rationale is neither a valid technique nor sound estimating practice. Use insights of functional experts in conjunction with one of the three established costing techniques.
53
Costing Techniques Other Techniques Expert Opinion Application Use expert opinion only as a last resort, when other methods are not available or not valid for the situatio
It is hard to justify as a primary estimating method and it is difficult to run risk around an expert opinion. Research further because there is generally additional data that the opinion was based upon. Experts may have their own data, collected over many years and many projects, that were used explicitly or implicitly to develop the estimate. The key is to extract this information from the expert and document it. Expert opinion may be used to corroborate or adjust (as in the analogy technique) objective data. For example, an expert with many years of experience may be able to explain a data point which appears anomalous and help guide the analyst to appropriate treatment of an apparent outlier. It could also be used early in a program’s life cycle for a quick, high-level, Rough Order-of-Magnitude (ROM) estimate. Expert opinion is often used as a secondary method to offer a cross check for an estimate developed using established techniques. A sanity check with experts helps solidify a cost estimate. In some organizations, executives may be comforted knowing that the estimate was validated by a long-standing company expert
54
Costing Techniques Other Techniques Expert Opinion Considerations An advantage of expert opinion is that it can provide a sanity check or cross-check of an estimate produced using a combination of established techniques
Subject Matter Experts (SMEs) usually provide a different perspective and may point out things not previously considered. Interaction with experts allows for a better understanding of the product or process whose cost is being estimated. Remember that expert opinion is completely subjective and can be refuted using objective data. Expert opinion holds little, if any credibility, when not used in conjunction with one of the discussed costing techniques.
55
Costing Techniques Other Techniques Expert Opinion Uncertainty and risk As previously noted, expert opinion is subject to human mistakes in memory and judgment to which data-based techniques are immune
Humans are not great random number generators and they tend to understate both variance (spread) and randomness (lack of uniformity). Expert opinion is grounded in anecdotal experience. Unfortunately, faulty recollection of this experience tends to understate effort. The expert may recall getting a report done in a week and thus put down 40 hours, but then forget that they had to work overtime or that the report got returned with review comments that took another couple of days to resolve. In extrapolating from past experience, misremembered or not, experts may be excessively optimistic or conservative.
56
Using Costing Techniques Estimate Requirements Why are you doing this estimate? When any estimate is started, it is important to understand as much about the program, the purpose of the estimate, and the estimate requirements as possible. These details help determine what is needed to develop the estimate. How much detail is required? How soon is the estimate needed? What data is available? Answering these questions and others ensure that the task is well understood and gauges what cost estimating techniques are best to use. What is the purpose of developing this estimate? Will it be used for:
milestone decisions, budget development, ROM estimates, alternatives comparisons, or proposal development or evaluation? Several different types of estimates are listed above. An estimate for a milestone decision, for example, requires more detail than a ROM.
57
Using Costing Techniques Top-down vs. Bottom-up How is the estimate developed?
There are two main ways to structure a cost estimate, top-down an- bottom up. Although any of the costing techniques may be used with either estimating approach, top-down is generally associated with parametric or analogy. It involves known top-level requirements (e.g., weight or power) or parameters to develop an estimate for an entire system. Bottom-up involves working from information at the lowest level to develop an estimate for an entire system. Develop discrete estimates for each element by estimating required labor hours, materials, and other costs and applying direct and indirect rates, and then roll up these lower-level elements to arrive at an estimate for the entire system. Bottom up is most closely associated with the build-up technique, but again, multiple techniques may be used in an estimate.
58
Using Costing Techniques Cost Element Structure What is being estimated? In conjunction with determining the why and how of developing an estimate, determine what to estimate. A good way to break down the estimate into manageable pieces is to develop a Cost Element Structure (CES). The CES is the framework for an estimate in a hierarchical tree structure where the costs of children sum up to the cost of their parent or, conversely, the cost of the parent can be allocated to the children. The CES dictionary defines what is included in each element of the CES. There are several characteristics associated with cost elements, which are used to classify the costs. These include:
program phase (e.g., development, production, or O&S), appropriation (e.g., Research, Development, Test, and Evaluation (RDT&E), procurement, or Operations and Maintenance (O&M)), funding source, non-recurring or recurring costs, and direct or indirect costs.
59
Using Costing Techniques Technique Selection After establishing the purpose, scope, approach, and structure of the estimate, select costing techniques at each appropriate level. To select a technique:
Research which techniques are available from all potential sources. There may be several alternative techniques for any given element. Some customers may have specific models that they want you to use. In addition, many cost estimators maintain a library of costing techniques. Along with existing techniques, consider developing new ones. After researching potential techniques, compare the techniques against the requirements and against each other. To compare techniques look at the underlying data, do statistical comparisons, and make sure that nothing is overlooked or left out. In addition to finding the right techniques, be careful not to use outdated techniques. If a technique was developed based on old data, it may not accurately estimate new technology. Make sure that the technique is for the appropriate program phase. Select the technique most appropriate for the estimate or develop a technique that fills the needs. Remember that multiple techniques can be used in the same estimate. Each technique has strengths and weaknesses and can be applied at different times in the life cycle of a cost estimate. In addition to selecting a primary technique, select a secondary technique as a cross-check.
60
Using Costing Techniques Checking Results
To increase the credibility of an estimate, check the results using another method. A cross-check is a test of reasonableness. The cross-check may involve using an analogy to compare with the results of a parametric estimate. The cross-check does not have to result in the exact same number, but should be a similar number. At a minimum, it should be the same order of magnitude. Validation of results is more detailed than a cross-check. The intent is to get nearly the same result using different techniques. An example is using a commercially available software estimating package to check the results obtained using another software package. The sense of validation is an Independent Cost Estimate (ICE). Generally, cross-check, validation, and ICE reflect increasing levels of detail, fidelity, and persuasiveness in corroborating a cost estimate.
61
Using Costing Techniques Documentation
Documentation is necessary to show the ground rules, assumptions, data, and calculations used to develop an estimate. If there is any doubt about whether to include something, then include it. More information is better than less (within reason). All data used in the analysis should be included in the documentation, with sufficient detail that another analyst would be able to recreate the work based solely on the documentation. This replicability is the acid test for documentation. Documentation is another way to lend credibility to an estimate. A well-documented estimate convinces others that a thorough estimate was developed.
62
Life Cycle Applicability Fig. 2.10: Costing techniques comparison - life cycle applicability[5] Figure 2.10 shows when different costing techniques are commonly applied relative to the Department of Defense (DoD) program's life-cycle phase.
It shows the approximate proportional usage of costing techniques by phase. This figure helps provide an idea of the appropriate time to apply particular techniques. At the beginning of a program, there is more emphasis on using analogies and parametric relationships. In these early phases, gross estimates are expected as detailed estimates are not usually possible due to poor program definition, changing requirements, and scarce cost data. As the program matures, it becomes more defined, additional data is collected, and the estimates get more detailed. Engineering build-up and extrapolation from actuals are used more frequently in the production and deployment phase and O&S phase
63
Summary This module provided an overview of the costing techniques
including a definition of each technique, when to apply them, how to apply them, and the strengths and weaknesses of each technique. Remember that there are many different techniques, which can be applied in different situations. Carefully consider each technique analytically and select the best technique or techniques for the program.
64
Testable Topics
Definition, Examples, Advantages, Disadvantages, and Risk Implications of each technique Mapping of techniques to life-cycle phases Operating and Support (O&S) Estimating
65
Not Testable
Schedule Estimating Other advanced topic
66
COST ESTIMATING TECHNIQUES
The Analogy technique refers to comparing the cost of an item to be estimated to that of a similar item. The Parametric technique uses a mathematical relationship based on historical data to relate cost to one or more technical, performance, cost, or programmatic parameters. The Build-up technique involves estimating costs at the lowest definable level and typically applies to Industrial Engineering (IE). These techniques have different uses and degrees of applicability during a program's life cycle. Cost estimators use one or more of these techniques depending upon the level of fidelity required. Other techniques include expert opinion and extrapolation from actuals. The Expert Opinion technique uses subjective information from Subject Matter Experts (SMEs) to corroborate or adjust cost estimates.
67
EXTRAPOLATION FROM ACTUALS
Extrapolation from actuals is an umbrella term referring to the use of actuals from past or current items to predict future costs for the same item. It includes techniques such as learning curve, averages, and earned value management.  For example: contract actuals from a contract that has been in place for an extended period can be used to estimate the follow-on costs for the contract.  For another example: extrapolation from actuals should be used to predict follow-on unit or lot costs when estimating production costs
68
Expert opinion involves using an expert or group of experts to estimate the cost of a system. This is often known as engineering judgment. Expert opinion is generally viewed as too subjective. To alleviate this concern, continue delving into the opinion to determine if the expert is basing thaopinion on some real data. After identifying the data the opinion is based upon, obtain copies and document the source. Do not ask experts to estimate outside the bounds of their experience. Request copies of any referenced data. Ask for another expert for areas outside the scope of their expertise. Validate credentials of the expert and identify alternate experts for the same scope. Formulate consensus after individual consultation. There are several different approaches to expert opinion.
One-on-one interviews with experts: Request any documentation available on the subject. Iterate if possible. Round-table discussion: Multiple experts present all sides of an issue, and all the experts stay in a room until a consensus is reached. They document areas of risk in the estimate. Delphi technique: A group of experts provides their answers anonymously to avoid a single person influencing the results in a group environment. The results are summarized and sent back out for coordination/comments. This approach generates a range of opinions and generally results in convergence to a single number or at least tighter range of possible outcomes. This is a specific technique from Operations Research (OR). Using expert opinion or engineering judgment without an accompanying rationale is neither a valid technique nor sound estimating practice. Use insights of functional experts in conjunction with one of the three established costing techniques.
69
Uncertainty is the range of possible outcomes of the estimate. For estimates based on an analogy, there will be uncertainty in both the point of departure and the slope of the adjustment. The point of departure is the analogous system. There is uncertainty in whether or not the chosen point of departure is truly analogous to the new system.
For estimates based on an analogy, there are risks not included in the analogous system. This means that the new system may have risks that would not have been captured in the costs associated with the old system. Examples include risks associated with new technologies, risks associated with economic conditions (e.g., inflation), and risks associated with the labor environment. In this image, the solid gray line shows the true underlying cost driver relationship and the two dotted gray lines show plus or minus one standard deviation (sigma) of the associated error term. Note the fundamental assumptions for OLS regression for making appropriate comparisons using the parametric technique later. The green square represents an estimate that is based on an analogy. In this example, the analogous point is circled in red. The line between the estimate and the analogous point is linear and passes through the origin. The error associated with the estimate based on an average was included to determine the uncertainty surrounding the estimate based on an analogy.
70