Part E: Testing Flashcards
- Testing definitions (1) (max 4 points)
What are Error, Fault, Failure and Incident?
● Error: human make while coding, design, in requirement
● Fault is a representation of an error, also called defect (e.g. in ISTQB glossary)
● Failure: software faults become software failures only when they are executed
● Incident: is the symptom associated with a failure that alerts the user to the occurrence of a failure.
313:
Error may lead to Fault may cause Failure may be observed as Incident
- Testing definitions (2) (max 5 points)
What is testing? (1p) What is your view of the goal of testing? (2p) What are the limitations of testing? (2p)
What is testing?
Testing is concerned with errors, faults, failures, and incidents. Test is to exercise software with test cases.
What is your view of the goal of testing? (2p)
A test has two distinct goals: to find failures or to demonstrate correct execution.
Limitations of testing?
● Testing cannot prove correctness
○ Testing can demonstrate the presence of defects
○ Testing cannot prove the absence of defects
● How much testing is enough, and how do we decide when to stop testing
- Testing levels (max 5 points)
Name and describe the different levels of testing.
Unit Testing
UNIT TESTING is a type of software testing where individual units or components of a software are tested. The purpose is to validate that each unit of the software code performs as expected. Unit Testing is done during the development (coding phase) of an application by the developers. Unit Tests isolate a section of code and verify its correctness. A unit may be an individual function, method, procedure, module, or object.
Component Testing
Component Testing is performed on each individual component separately without integrating with other components. Component testing is performed by testers. ‘Unit Testing’ is performed by the developers where they do the testing of the individual functionality or procedure. After Unit Testing is performed, the next testing is component testing. Component testing is done by the testers. It’s one of the most frequent black box testing types which is performed by the QA Team.
Integration Testing
INTEGRATION TESTING is defined as a type of testing where software modules are integrated logically and tested as a group. The purpose of this level of testing is to expose defects in the interaction between these software modules when they are integrated
System Testing System Testing (ST) is a black box testing technique performed to evaluate the complete system's compliance against specified requirements. In System testing, the functionalities of the system are tested from an end-to-end perspective. System Testing is usually carried out by a team that is independent of the development team in order to measure the quality of the system unbiased. It includes both functional and Non-Functional testing.
Acceptance Testing
Acceptance testing, a testing technique performed to determine whether or not the software system has met the requirement specifications. The main purpose of this test is to evaluate the system’s compliance with the business requirements and verify if it has met the required criteria for delivery to end users.
There are various forms of acceptance testing:
User acceptance Testing
Business acceptance Testing
Alpha Testing
Beta Testing
- Testing process (max 4 points)
What is a common process of doing testing according to ISTQB?
The ISTQB process consists of 4 steps:
● Analyze test basis: Analyzing the available specifications, requirements, user manuals, business process descriptions, etc. to gain an understanding of the domain.
● Identify test conditions: A test condition is an item or event that could be verified by one or more test cases, e.g. a function, transaction, feature, quality attribute, etc.
● Design and specify test cases: Self explanatory basically.
● Develop test procedure: A test procedure is a detail of how the tester will physically run the test, the physical set-up required, and the procedure steps that need to be followed.
- Testing challenges (max 4 points)
What are the common testing challenges?
● Not enough time to test properly, test well, test thoroughly
● Difficulty in determining the expected results of each test
● Nonexistent or rapidly changing requirements
● Management that either doesn’t understand testing or doesn’t care about quality
● No training in testing process
● No tool support
- White box & black box testing (max 4 points)
What is “white box testing”? (1p) What is “black box testing”? (1p) Why do we need both of them? (2p)
In white box testing, the tester knows the inner workings of the code, and can design lower level tests and unit tests. In black box testing, the tester does not know how the program works on the inside, and so only performs higher level testing to verify that the requirements are met.
Both are needed, as in white box testing technical problems can be identified, but black box testing gives a broader perspective and can verify that the requirements are met.
- Test cases (1) (max 4 points)
What is test case? (1p) Why is it important to prepare test data? (1p) When do we say a set of test cases are effective and efficient? (2p)
A test case is a set of inputs and a list of expected outputs.
It is important to prepare test data to ensure that all major types of values are tested, without becoming too intractable. For example, testing invalid data (too small), values on the border of invalidity, whole numbers, fractions, invalid data (too big), but not infinitely many of each type.
An effective test case detects more faults.
An efficient test case detects faults with less effort
- Test cases (2) (max 5 points)
What is test case? (1p) Name and describe a few techniques of test case design for back box testing. (4p)
s. 334:
A test case has a set of inputs and a list of expected outputs.
s. 336-380:
● Equivalence Class Partition
○ Divide the input domain into partitions that
■ Don’t overlap
■ Cover the entire input domain
○ e.g. negative and positive numbers (usually more partitions though)
○ Each value in a partition is assumed to be equally useful for testing
○ So we only need to test one of them
● Boundary-Value Analysis (BVA)
○ Programmers often make mistakes in processing values at and near the boundaries of partitions in the input domain.
○ Process:
■ 1. Partition the input domain
■ 2. Identify the boundaries for each partition
■ 3. Test values on & either each side of each boundary
● Pairwise Testing
○ Instead of testing all possible combinations of inputs (could be very many),
○ Construct a set of inputs such that every pair of 2 inputs occurs at least once.
○ There are tools available to generate such sets (time consuming combinatorial task to do by hand).
● Decision Table based Testing
○ Construct a decision table:
○ From this, each column becomes one test case, with the conditions as inputs, and actions become expected results.
● State Transition based Testing
○ Construct a state transition diagram:
○ Draw test cases as paths in the diagram, such that they cover
■ Every state
■ Every event
■ Every path
■ Every transition
● Use Case based Testing
○ 1. For each use case, generate a full set of use-case scenarios.
○ 2. For each scenario, identify at least one test case, probably more.
○ 3. For each test case, identify the input data values with which to test.
- Test coverage (max 6 points)
What is test coverage and why is it useful? (2p) Name and describe a few code coverages. (2p) Is a set of test cases with higher coverage are “better” test cases (discuss in terms of more effective and efficient)? (2p)
What is test coverage?
s. 387:
It measures how much of a program is executed when a particular set of test cases run.
Why is it useful? s. 389: Serves two purpose: ● Quantitative test measurement ● Plan your test case design (for white box testing)
Name and describe a few code coverages. s. 388: ● Lines of code ○ % of lines of code executed ● Decision outcomes ○ % of decision outcomes covered (edges in program graph). ● Branches ○ Same as decision outcomes?? ● Modules ○ ?? ● Menu options ○ ?? (Maybe as in a navigation menu, % of those visited?) ● Functions ○ (Probably) the % of code functions executed.
Is a set of test cases with higher coverage “better” test cases? (discuss in terms of more effective and efficient)
Källa: jag
Test cases with higher coverage are likely more effective, meaning they will detect more faults (because they execute more possibilities).
They are not necessarily more efficient though. It is possible to brute force your way to 100% coverage, by doing loads of tests.
- Static testing (max 5 points)
What is review/inspection/walkthrough (also called static testing in ISTQB)? (1p) Why is it important? (1p) What are the benefits of using this technique? (3p)
s. 430 -
What is review/inspection/walkthrough (also called static testing in ISTQB)?
It is to review any software artefacts to find faults, without executing the code.
Why is it important?
Because it catches faults at an early stage, which reduces costs.
What are the benefits of using this technique?
● Reduce cost and improve productivity
● Data from review are useful
● Other side benefits
○ Sharing understanding and knowledge
○ An effective forum for less experienced team members
- Regression testing (max 6 points)
What is the difference between testing and regression testing? (3p) What are the common regression testing challenges? (3p)
What is the difference between testing and regression testing?
Regression testing means testing a new version of the system, to make sure it is not less effective than before, whereas testing tests a new system.
What are the common regression testing challenges?
- No/outdated test plan
- During regression testing, the testers may not be the ones who developed/tested the previous versions
- Needs to be done after every modification
