What is a testing framework?
A testing framework, a.k.a testing automation framework is an execution environment for automated tests.
What are the purposes of test automation?
- It controls test execution including ordering and filtering tests based on criteria.
- It avoids repetitive tasks that are inherent with manual test execution.
What does a testing framework do?
- Defines how assertions are defined, which promotes maintainability.
- Controls and interacts with the tested application.
- Executes tests.
- Reports results including both individual and aggregate results.
What are test framework assumptions?
- What responsibilities are being delegated and how.
2. Focus of the developer should be on writing tests – not maintaining supported software.
What are test framework concepts?
- Structural consistency
- Tests – what a test does
- Results – a collection of successes and failures for analysis, and how to report
What are some of the test framework tools?
- Environment setup – both around the individual tests and groups of tests.
- Application control – selectively starting and stopping parts of it.
- Test data (some frameworks) – specify test ranges or external resources.
- Execution control – a subset of tests or setting thresholds of acceptable failure or duration.
Why use a testing framework?
- Lower maintenance costs
- Update tests, not the framework
- Reusability
- Delegate responsibility
What are the four phases of testing frameworks?
- Setup
- Execution
- Validation
- Cleanup
What is the ‘setup’ phase of a testing framework?
It’s the first phase thereof – the application is prepared for testing (may be as simple as importing a module), dependencies are replaced, and test data is prepared if it’s needed.
What is the ‘execution’ phase of a testing framework?
It’s the second phase thereof – the target behavior is performed and the output is captured including the response from an API and other sources.
What is the ‘validation’ phase of a testing framework?
It’s the third phase thereof – the expected result is compared with the actual to ensure correctness. This is when the assertions are used to compare the result with whatever was expected. The testing framework collects the results for reporting.
What is the ‘validation’ phase of a testing framework?
It’s the fourth and last phase thereof – the application state is restored in order to allow other tests to run without interference. Cleanup is only needed if there’s something that needs to be restored.
