Quality Assurance Software Testing Flashcards
What are the basic testing definitions?
Test cases, stubs, and drivers:
- Test cases provide
- a set of inputs and
- list a set of expected outputs
- Test stubs:
partial implementations of components
on which a component under test (CUT) depends
- Test drivers:
partial implementations of components
that exercise and depend on the tested component
Test Driver -> (dependsOn) CUT -> (dependsOn) Test Stub
What is the ugly truth about software testing?
Bugs are inescapable:
- It is impractical to test all possible uses
- Nonetheless, testing raises our confidence
that software has its desired behaviour
The impracticality of exhaustive testing:
- Large input space
- Large output space
- Large state space
- Large number of possible execution paths
- Subjective or imprecise requirements
What is exhaustive testing?
Testing all possible input combinations (but often infeasible). Some input spaces are finite. impractical or impossible (at worst),
What is an execution path?
Pssible flor of control of program
=sequence of executed statements
ex;
i = 2
j = 3
res = i * j
print(res) -> 1 paht
What is the execution path in this code?
i = i + j
if (i < 10):
print(“ok”)
else:
i = i/2
2 paths:
1:
i = i + j
print(“ok”)
2:
i = i + j
i = i/2
What is the execution path in this code?
for (i in [1..n]):
if (condition):
stmtA
else:
stmtB
2^n paths:
N = 1
Path 1:
stmtA
Path 2:
stmtB
N = 2
Path 1:
stmtA, stmtA
Path 2:
stmtA, stmtB
Path 3:
stmtB, stmtA
Path 4:
stmtB, stmtB
How many execution paths?
for (int i = 0; i < n; ++i) {
if (a.get(i) == b.get(i)) {
x[i] = x[i] + 100;
} else {
x[i] = x[i]/2;
}
}
2^n
->Loop Header-> +100 -> Loop Header
-> /2 -> Loop Header
(+100 or /2) then back to Loop Header
n Number of paths
1 2
2 4
3 8
10 1024
etc
What is a control flow diagram?
Representation of all execution paths:
Using graph notation
o Show where the flow is divided
o 1 node per block
oEdges to show different paths
o Predicate node: condition + several outgoing edges
o Terminal node
(see slide 17 for image)
Draw the execution path of:
i = i + j
if (i < 10):
print(“ok”)
else:
i = i/2
Path 1:
i = i + j
print(“ok”)
Path 2:
i = i + j
i = i/2
(see slide 18 for answer)
Draw the execution path of:
for (i in [1..n]):
if (condition):
stmtA
else:
stmtB
see slide 19 for answer
Why is this path infeasible? How many execution paths?
inputs: n and m
…
if (n > m)
x = n;
else
x = m;
if (x >= n)
y = 1;
else
y = 0;
…
There are two execution paths:
1. if (n > m) true, then x=n; so if (x >=n) always true since then n>=n always holds -> y=1
2. if (n>m) not true (so n<=m), then x=m; so since n<=m then if (x>=n) true since m=x which is x>=n.
So last else is never visited!
This means that this code has infeasible path executions.
Feasibility challenge: Even if we could test all
execution paths, some of them may not be feasible!
See slides 20-22 for more info!
What is the ugly truth about testing?
Lack of continuity.
Testing a bridge:
- If a bridge can sustain weight w1,
then it can also sustain w2 < w1
Testing software: Butterfly effect
- Small differences in operating
conditions can dramatically impact
software behaviour!
So… solution:
Striking a balance:
- A balance must be reached
between the cost of testing
and the risk of missing bugs
What is the truth about bugs…?
The truth about bugs:
- Not all known bugs are fixed
- Not enough time
- Too risky to fix (might create more problems)
- When is a bug really a bug?
- Time pressure may lead to downgrading bugs
to feature requests
- Since many people struggle to process criticism,
QA personnel are not the most popular people…
See slide 26 for info about oracles, verdicts and Test coverage.
What is a Test Coverage Criteria?
Measured in percentage: Tested / All
* Statement Coverage:
- executed statements / all statements
* Branch Coverage:
- both true and false branches of all conditions
- execution branches / all branches
* Path Coverage:
- Executed paths / all paths (see CFG)
How many test cases are
needed for 100% coverage?
inputs: m, n
int x=0, y=0, z=0
if (m > 0)
x = m;
if (n > 0)
y = n;
if (m < n)
z = n-m;
return (x+y)*z;
There are three if statements, each with two possible outcomes (the condition can either be true or false), making
2^3=8 paths in total. This is the full combination of all true/false outcomes for the three if conditions. (slide 29 for CFG). Every path is one test
Branch is 2 because you cover all true and false branches of all conditions. Doing all ifs or not.
Statement is 1 since you only need one test case that covers all the statements (statement is all lines/action tbd).
Recall:
Measured in percentage: Tested / All
* Statement Coverage:
- executed statements / all statements
* Branch Coverage:
- both true and false branches of all conditions
- execution branches / all branches
* Path Coverage:
- Executed paths / all paths (see CFG)
What are the testing stages?
Development testing:
- Tests that are typically written by the
development team
Release testing:
- Separate QA team produces tests to check
the system before it is released
User testing:
- Tests run by users to ensure that they will
be satisfied with the product
What are the three Granularities of development testing?
Unit Testing, Integration Testing, and System Testing.
What is Unit Testing?
Unit testing:
- Target: particular program unit (class)
- Focuses on testing the functionality of an
object or a method
Unit testing:
- Maximize coverage of the system while
minimizing execution time
- Our test cases should cover all the possible
paths through our methods
- But we want our tests to finish quickly so
that we do not delay the release process
What are the key phases of unit testing?
Key phases:
- Setup part: Prepare the system for
executing the test case
- Call part: Call the method under test
- Assertion part: Check that the system state
is in its expected form
- Teardown part: Clean up after a test
What are the goals within unit tests?
Goals in designing unit tests:
(1) Show that (when used as expected) the
unit does what it is supposed to do
(2) If there are defects in a unit, the test cases
should reveal them!
Need different types of unit tests!
What is equivalence partitioning?
- Equivalence partitioning:
- Inputs can be thought of as members of sets with
common characteristics - Rule of thumb
- Select test cases on the edge of the partitions
- Select a test case near the middle as well
- Can be applied to inputs or outputs
Many inputs could be lead to the same output ex: error if the bounds are incorrect.
Invalid <=50, Valid 50-90, Invalid >=90
see slide 38 and 39 for image.
Exercise equivalence partionning:
Password:
- At least 8 characters long, at least one
number, lower & upper case letters, no
special characters (!@#…)
Gender
GRE score for Quantitative Reasoning:
- Between 130 and 170, 1 point increments
Postal code:
- According to Canadian system
Valid:
Password:
- length >= 8 (1)
- at least one number (2)
- at least one lower case letter (3)
- at least one upper case letter (4)
- no special characters (5)
Gender:
- in {F, M} (11)
GRE score:
- 130 <= score <= 170 (13)
Postal code:
- length is 6 characters (16)
- start with letter (17)
- alternates between letters and numbers (18)
Invalid:
Password:
- length < 8 (6)
- no number (7)
- no lower case letter (8)
- no upper case letter (9)
- at least one special character (10)
Gender:
- not in {F, M} (11)
GRE score:
- score < 130 (14)
- score > 170 (15)
Postal code:
- length < 6 (19)
- length > 6 (20)
- does not start with letter (21)
- does not alternates between letters and numbers (22)
Combine as many valid equivalence classes into
one test case as possible:
- Assuming that there is one method that validates
password, gender, GRE score, and postal code at
the same time
- One test case covers all valid equivalence
classes (1–5, 11, 13, 16–18)
One test case for each invalid equivalence class:
- 12 test cases are needed (6–10, 12, 14, 15, 19–22)
What is Testing Integration?
Integration testing:
- Target: the combination of units
- Focuses on interface that is exposed to
others
- It’s all about dependencies and testing
order
What is a stub?
Stub:
Called from the software component
Stubs are used to simulate the behavior of lower-level modules or components that are not yet implemented or are unavailable for testing. If you are testing a higher-level module and it depends on a lower-level module to function, you would use a stub to represent the lower-level module. Stubs provide the necessary outputs when called upon by the higher-level module, and they are also known as “called programs.”
- Replaces a called module
- Stubs for input modules pass test data
- Stubs for output modules return test results
- Can replace whole components (e.g.,
persistence layer, network communication) - Must be declared/invoked like the real unit
- Same name
- Same parameter list and return type
- Same modifiers (e.g., static)
public static int someFunc(floatfThat) {…}
stub example:
public static int someFunc(floatfThat) {
int iDummy = 42;
System.out.println(“In method someFunc “ +
“Input float =” + fThat);
System.out.println(“Returning “ + iDummy +
“.”);
return iDummy;
}
