Testing

Question 1

What is the famous saying regarding testing for errors

Answer 1

Testing can only show the presence of errors, not their absence

Question 1

What are the 2 things you are trying to do when testing software

Answer 1

1. Making sure all requirements are met
2. Squash bugs

Question 2

What is the famous wording for the difference between validation and verification

Answer 2

Validation: Are we building the right product?
Verification: Are we building the product right?

Question 3

What is V&V concerned with

Answer 3

Checking that the software is meeting specs/reqs and that the functionality works as expected

Question 4

What is verification (as opposed to validation)

Answer 4

Checking that the software meets functional and non-functional requirements

Question 5

What is validation (as opposed to verification)

Answer 5

Checking that the software meets the user’s and stakeholder’s expectations, which are not necessarily part of any requirements

Question 6

What is the goal of V&V

Answer 6

Establish confidence that the software system is “fit for purpose”

Question 7

What does it mean when a system is “fit for purpose”

Answer 7

The system must be good enough for its intended use

Question 8

What are the 3 things that determine how much confidence a system needs to have (when is it “good enough”)

Answer 8

1. Software Purpose (Criticalness)
2. User Expectations (Previous Experience + Company Maturity)
3. Marketing Environment (Other Competition)

Question 9

What should also be done for V&V besides testing, and what are the 3 advantages of doing it

Answer 9

Inspections/Reviews:
1. Errors are discovered that testing would have otherwise been hiding due to the lack of knowing if it came from a known error or a new one
2. You can discover errors before the system can even be ready to test, reducing development costs
3. You can discover a lack of quality in the code that testing cannot identify, such as compliance, portability, and maintainability

Question 10

What are the 3 things inspections NOT good for that testing could detect

Answer 10

1. Unexpected interactions between different parts of the system
2. Timing problems (race conditions)
3. System performance

Question 11

What is important to understand when it comes to making test cases

Answer 11

Test cases are impossible to automatically generate. You must be INVOLVED in understanding what the system is supposed to do in order to create test cases and the expected test results.

Question 12

What are the 3 major stages of testing for a software’s lifespan

Answer 12

1. Development Testing (during development)
2. Release Testing (finalizing)
3. User Testing (post-prod in non-dev environments)

Question 13

What is particularly impossible to automatically test

Answer 13

GUIs and unanticpated side effects

Question 14

What are the 3 stages of Development Testing

Answer 14

1. Unit Testing (objects/methods)
2. Component Testing (composite interfaces)
3. System Testing (whole, component interactions)

Question 15

What are the 3 parts of an automated test

Answer 15

1. Setup (initialize, inputs + outputs)
2. Call (the object)
3. Assertion (compare result)

Question 16

What can be used that would skip bottlenecks in automated tests, such as database requests

Answer 16

Mock objects

Question 17

What is unit test case “Effectiveness”

Answer 17

Making test cases that either show that the component is doing what it is supposed to do and that defects are revealed

Question 18

What are 2 effective types of strategies for developing test cases

Answer 18

1. Partition Testing
2. Guideline-based Testing

Question 19

What is Partition Testing

Answer 19

Identify and define all possible (separate) sets of inputs for a unit test as classes, and group them from their (supposedly correct) behavior (output) as classes (behavior of separate input classes can interleave), with these classes being called “equivalence partitions/domains”, and for all classes of input and output partitions, if the partitions are ranges, test the boundaries of them (along with the middle), or otherwise test the full sets. Also identify all possible INVALID input and output equivalence partitions and test that the system is correctly handling those errors.

Question 20

What is Guideline-based Testing

Answer 20

Use previous knowledge of common developer errors to include in all tests

Question 21

When is it appropriate to start performing Component Testing

Answer 21

Once all of the unit testing has been completed

Question 22

What are the 4 types of interfaces between components

Answer 22

1. Parameter Interfaces (direct data flow)
2. Shared Memory Interfaces (external)
3. Procedural Interfaces (call the methods from other)
4. Message-Passing Interfaces (pls)

Question 23

What are the 3 most common forms of interface errors

Answer 23

1. Interface Misuse (type)
2. Interface Misunderstanding (ordering, etc)
3. Timing Errors (race cond.)

Question 24

What is sometimes better to do for detecting interface errors

Answer 24

Inspections/reviews

Question 25

What are the 2 important differences of System Testing that separates it from Component Testing

Answer 25

1. You can be testing old components (or purchased off-the-shelf components) that have been previously developed and is now being integrated to the whole system 2. The teams that developed components separately join together to test the integration, or a new team (not involved in the development of all the components at all) is used

Question 26

What must be done to achieve Exhaustive Testing

Answer 26

It is impossible

Question 27

What are the 5 steps of TDD

Answer 27

1. Identify the next increment 2. Write an automated test 3. Run the test(s) to make sure the test is working 4. Implement the increment and re-run the test. Repeat this until the test passes. 5. Go back to step 1

Question 28

What are the 4 benefits of TDD

Answer 28

1. Code Coverage 2. Regression Testing 3. Simplified Debugging 4. System Documentation

Question 29

What type of system makes TDD ineffective

Answer 29

Multi-threaded systems

Question 30

What are the 2 things that make Release Testing different than System Testing

Answer 30

1. It should not be done by the developers 2. It focuses on meeting requirements, not squashing bugs

Question 31

What is Release Testing also known as

Answer 31

Functional Testing (concerned with functionality, NOT implementation)

Question 32

Regarding this topic, what should be considered when performing Requirements Engineering

Answer 32

Making sure each requirement is testable

Question 33

What are you NOT doing for Requirements-Based Testing

Answer 33

Defect Testing (you are instead validating that requirements are implemented)

Question 34

What is Scenario Testing

Answer 34

You write tests based on narratives and scenarios, in which each covers multiple requirements

Question 35

What are the 2 goals of Stress Testing

Answer 35

1. Checking that system failure due to overload causes it to "fail-soft" (not resulting in data corruption or unexpected loss of services) 2. Reveal the defects that only appear under overload

Question 36

What type of systems are important to Stress Test and why

Answer 36

Distributed systems that are based on a network of processors. This is because there is a point in which degredation occurs where processors wait on other processors, and this Stress Testing can identify when that happens to implement when transactions should be rejected to end this spiral before it begins.

Question 37

Why is User Testing essential

Answer 37

Because the developer's environment is artificial and will never replicate the actual working environment

Question 38

What are the 3 types of User Testing

Answer 38

1. Alpha Testing (internal users) 2. Beta Testing (closed and/or experimental release) 3. Acceptance Testing (custom systems)

Question 39

What are the 6 stages of Acceptance Testing

Answer 39

1. Define Acceptance Criteria (before contract signed) 2. Plan (budget and schedule) 3. Derive/Build Tests 4. Run/Execute (ideally in environment, with user training) 5. Negotiate (good enough?) 6. Reject/Accept (repeat if rejected)

Question 40

What typically happens even if Acceptance Testing fails

Answer 40

The system is used anyways because the customer already set up the environment to use it and would cost more to wait and pay for more development than to just work around the issues. In Agile development, usually this is the end of Acceptance Testing (move to alpha user stories).