5.1 Test Planning Flashcards
What purposes does a Test Plan serve?
- documents the means and schedule for achieving test objectives
- helps ensure performed test activities will meet established criteria
- means of communication with team members and stakeholders
- demonstrates testing will adhere to existing test policy and test strategy (or explains any deviations)
How does test planning benefit the test process and lifecycle?
Test planning guides thinking and forces testers to confront future challenges around risks, schedules, people, tools, costs, effort, etc. The planning process is a useful way to think through the effort needed to achieve test project objectives.
What is the typical content of a test plan?
- context (scope, test objectives, constraints, test basis)
- assumptions and constraints
- stakeholders (rolls, responsibilities, relevance to testing, hiring and training needs)
- communication (forms & frequency, doc templates)
- risk register (product risks, project risks)
- test approach (test levels, types, techniques, deliverables, data requirements, environments, deviations from policy, etc)
- budget and schedule
In iterative SDLCs, what two types of planning typically occur?
release planning and iteration planning
What does release planning involve?
Release planning looks ahead to the release of the product, defines and re-defines the product backlog, and may involve refining larger user stories into a set of smaller user stories.
How do testers participate in release planning?
Testers in release planning write testable user stories and acceptance criteria, help with project and quality risk analyses, estimate test effort for user stories, determine the test approach, and plan testing for the release.
What does iteration planning involve?
Iteration planning looks to the end of a single iteration and is concerned with that iteration’s backlog.
Testers involved in this participate in the detailed risk analysis of user stories, determine the testability of user stories, break down user stories into tasks (particularly testing tasks), estimate test effort for testing tasks, and identify and refine functional and non-functional aspects of the test object.
What is the definition of entry criteria, and what does it usually include?
Entry criteria define the preconditions for undertaking a given activity (like testing). Typical criteria for testing include:
- availability of resources (people, tools, environments, data, budget, time)
- availability of testware (test basis, requirements, user stories, test cases)
- initial quality level of the test object (eg all smoke tests pass
AKA Definition of Ready
What is the definition of exit criteria, and what does it usually include?
Exit criteria are the objectives required to declare an activity complete. Typical criteria for testing include:
- measures of thoroughness (coverage level, # unresolved defects, defect density, # failed test cases)
- completion criteria (all planned tests executed, static testing completed, all found defects reported, all regression tests automated)
Running out of time or budget can also be view as valid exit criteria, if stakeholders have accepted the risk of going live without further testing.
AKA Definition of Done.
Given the likelihood of estimation errors increases with the size of the task, how can one increase the accuracy of estimation for a large testing project?
By breaking the testing task down into smaller tasks and pieces, each of which can be estimated more accurately. It’s always important to make clear to stakeholders that test effort estimates are based on assumptions and always subject to potential error.
How does Estimation based on ratios work?
This is a metrics-based technique in which figures are collected from previous projects within the organization to derive standard “ratios” of development time to test time for similar projects. The ratios of an organizations own projects are generally the best source to use in the estimation process.
These standard ratios can be used to estimate the test effort for the new project. For example, if the previous project had a development to test effort ratio of 3:2, and the current project’s development effort is expected to be 600 person-days, test effort can be estimated at 400 person-days.
How does the Extrapolation estimation method work?
Another metrics-based technique in which measurements are made as early as possible in the current project to gather effort data. As observations are collected, the effort required for the remaining work can be approximated by extrapolating from this data (usually with a mathematical model).
This method is suitable to iterative SDLCs where both the dev and test work go through full cycles in each iteration. For example, the team may extrapolate test effort for an upcoming iteration as the averaged effort from the last three iterations.
How does the Wideband Delphi estimation technique work?
This is an iterative, expert-based technique in which SMEs make experience-based estimates. Each expert makes their estimation in isolation, results are collected, and the experts discuss any deviations that are out of agreed upon ranges / boundaries. Each expert then makes a new estimation based on that feedback (again in isolation). This process is repeated until consensus is reached.
Planning Poker is a variant of this widely used in Agile.
How does Three-Point Estimation work?
In this expert-based technique, the SMEs each make 3 estimations: (a) the most optimistic, (m) the most likely, and (b) the worst case scenario. The final estimate (E) is their weighted arithmetic mean.
In the most popular version of this technique, it is calculated as E = (a + 4*m + b) / 6.
The advantage of this technique is that the experts can calculate the measurement error: SD = (b - a) / 6.
What are the three most common test case prioritization strategies?
- risk-based, where test cases covering the most important risks are executed first
- coverage-based, where test cases achieving highest coverage are executed first. A variant is additional coverage prioritization, where each subsequent test case is the one that adds the most additional coverage.
- requirements-based, where the order of test execution is traced back to the requirement priorities established by the stakeholders
