2. Testing throughout the software life cycle

Question 1

What is verification testing?

Answer 1

Verification is concerned
with evaluating a work product, component or system to determine whether it
meets the requirements set. In fact, verification focuses on the question ‘Is the
deliverable built according to the specification?’.

Question 2

What is validation testing?

Answer 2

Validation is concerned with
evaluating a work product, component or system to determine whether it meets
the user needs and requirements. Validation focuses on the question ‘Is the
deliverable fit for purpose, e.g. does it provide a solution to the problem?’.

Question 3

What is Waterfall model?

Answer 3

The waterfall model was one of the earliest models to be designed. It has a
natural timeline where tasks are executed in a sequential fashion. We start at the
top of the waterfall with a feasibility study and flow down through the various
project tasks finishing with implementation into the live environment. Design
flows through into development, which in turn flows into build, and finally on
into test. Testing tends to happen towards the end of the project life cycle so
defects are detected close to the live implementation date. With this model it has
been difficult to get feedback passed backwards up the waterfall and there are
difficulties if we need to carry out numerous iterations for a particular phase.

Question 4

What is V-model?

Answer 4

The V-model pro-
vides guidance that testing needs to begin as early as possible in the life cycle. It also shows that testing is not only an execution-
based activity. There are a variety of activities that need to be performed before the end of the coding phase. These activities should be carried out in parallel with development activities, and testers need to work with developers and business analysts so they can perform these activities and tasks and produce a set of test deliverables. The work products produced by the developers and business analysts during development are the basis of testing in one or more levels.

Question 5

What are the common test levels of V-model?

Answer 5

• component testing: searches for defects in and verifies the functioning of
software components (e.g. modules, programs, objects, classes etc.) that are
separately testable;
• integration testing: tests interfaces between components, interactions to dif
ferent parts of a system such as an operating system, file system and hard
ware or interfaces between systems;
• system testing: concerned with the behavior of the whole system/product as
defined by the scope of a development project or product. The main focus of
system testing is verification against specified requirements;
• acceptance testing: validation testing with respect to user needs, require
ments, and business processes conducted to determine whether or not to
accept the system.

Question 6

What 2 major types of testing lie cylces?

Answer 6

sequential, iterative or incremential

Question 7

What is iterative testing life cycle approach?

Answer 7

A common feature of iterative approaches is that the delivery is divided into
increments or builds with each increment adding new functionality. The initial
increment will contain the infrastructure required to support the initial build
functionality. The increment produced by an iteration may be tested at several
levels as part of its development. Subsequent increments will need testing for
the new functionality, regression testing of the existing functionality, and inte-
gration testing of both new and existing parts. Regression testing is increasingly
important on all iterations after the first one. This means that more testing will
be required at each subsequent delivery phase which must be allowed for in the
project plans. This life cycle can give early market presence with critical func-
tionality, can be simpler to manage because the workload is divided into smaller
pieces, and can reduce initial investment although it may cost more in the long
run.

Question 8

Provide exampoles of iterative and incremental development models

Answer 8

prototyping, Rapid Application Development (RAD), Rational Unified Process (RUP) and
agile development.

Question 9

What is rapid application development?

Answer 9

It is formally a parallel development of
functions and subsequent integration. Components/functions are developed in parallel as if they were mini proj-
ects, the developments are time-boxed, delivered, and then assembled into a
working prototype.
The RAD development process encourages active customer feedback.
The customer gets early visibility of the product, can provide feedback on the design and can decide, based on the existing functionality, whether to proceed with the development, what functionality to include in the next
delivery cycle or even to halt the project if it is not delivering the expected value.

Question 10

What is Dynamic System Development Methodology?

Answer 10

It is a refined RAD
process that allows controls to be put in place in order to stop the process
from getting out of control.

Question 11

What is extreme programming?

Answer 11

It is currently one of the most well-known agile
development life cycle models. (See [Agile] for ideas behind this approach.)
The methodology claims to be more human friendly than traditional develop-
ment methods.

Question 12

What are the characteristics of XP?

Answer 12

• It promotes the generation of business stories to define the functionality.
• It demands an on-site customer for continual feedback and to define and
carry out functional acceptance testing.
• It promotes pair programming and shared code ownership amongst the
developers.
• It states that component test scripts shall be written before the code is
written and that those tests should be automated.
• It states that integration and testing of the code shall happen several times
a day.
• It states that we always implement the simplest solution to meet today’s
problems.

Question 13

What are the main characteristics of good testing?

Answer 13

• for every development activity there is a corresponding testing activity;
• each test level has test objectives specific to that level;
• the analysis and design of tests for a given test level should begin during the
corresponding development activity;
• testers should be involved in reviewing documents as soon as drafts are avail
able in the development cycle.

Question 14

What is component testing?

Answer 14

Component testing, also known as unit, module and program testing, searches 
for defects in, and verifies the functioning of software (e.g. modules, programs, 
objects, classes, etc.) that are separately testable. 
Component testing may be done in isolation from the rest of the system depend-
ing on the context of the development life cycle and the system. Most often stubs 
and drivers are used to replace the missing software and simulate the interface 
between the software components in a simple manner.
Usually it is done by progarmmers.

Question 15

What is a stub?

Answer 15

A stub is called from the

software component to be tested

Question 16

What is a driver?

Answer 16

a driver calls a component to be tested

Question 17

What is test-driven development?

Answer 17

One approach in component testing, used in Extreme Programming (XP), is
to prepare and automate test cases before coding. This is called a test-first
approach or test-driven development. This approach is highly iterative and is
based on cycles of developing test cases, then building and integrating small
pieces of code, and executing the component tests until they pass

Question 18

What is integartion testing?

Answer 18

It tests interfaces between components, interactions to dif-
ferent parts of a system such as an operating system, file system and hard-
ware or interfaces between systems. Note that integration testing should be
differentiated from other integration activities. Integration testing is often
carried out by the integrator, but preferably by a specific integration tester or
test team.

Question 19

What is component integration testing?

Answer 19

component integration testing tests the interactions between software com
ponents and is done after component testing

Question 20

What is system integarion testing?

Answer 20

system integration testing tests the interactions between different systems
and may be done after system testing. In this case, the developing organiza
tion may control only one side of the interface, so changes may be destabi
lizing. Business processes implemented as workflows may involve a series of
systems that can even run on different platforms.

Question 21

What is a ‘big-bang’ integration testing?

Answer 21

all components or systems are integrated simultaneously, after which everything is tested
as a whole. Big-bang testing has the advantage that everything is finished before integration testing starts. There is no need to simulate (as yet unfinished) parts. The major disadvantage is that in general it is time-consuming and difficult to trace the cause of failures with this
late integration.

Question 22

What is incremental integartion testing approach?

Answer 22

Another extreme is that all programs are integrated one by one, and a test is
carried out after each step. The incremental approach has the advantage that
the defects are found early in a smaller assembly when it is relatively easy to
detect the cause. A disadvantage is that it can be time-consuming since stubs
and drivers have to be developed and used in the test.

Question 23

What are the incremetal integration testing types?

Answer 23

• Top-down: testing takes place from top to bottom, following the control flow
or architectural structure (e.g. starting from the GUI or main menu).
Components or systems are substituted by stubs.
• Bottom-up: testing takes place from the bottom of the control flow upwards.
Components or systems are substituted by drivers.
• Functional incremental: integration and testing takes place on the basis of
the functions or functionality, as documented in the functional specification

Question 24

What is System testing?

Answer 24

System testing is concerned with the behavior of the whole system/product as defined by the scope of a development project or product. It may include tests
based on risks and/or requirements specification, business processes, use cases,
or other high level descriptions of system behavior, interactions with the operating system, and system resources. System testing is most often the final test on
behalf of development to verify that the system to be delivered meets the spec-
ification and its purpose may be to find as many defects as possible. System testing should investigate both functional and non-functional requirements of the system.

Question 25

What is non-functional testing?

Answer 25

Typical non-functional tests include performance
and reliability. Testers may also need to deal with incomplete or undocumented
requirements.

Question 26

What is functional testing?

Answer 26

System testing of functional requirements starts by using the
most appropriate specification-based (black-box) techniques for the aspect of
the system to be tested. For example, a decision table may be created for com-
binations of effects described in business rules. Structure-based (white-box) techniques may also be used to assess the thoroughness of testing elements such
as menu dialog structure or web page navigation

Question 27

What is acceptance testing?

Answer 27

When the development organization has performed its system test and has cor-
rected all or most defects, the system will be delivered to the user or customer
for acceptance testing. The acceptance test should answer questions such as:
‘Can the system be released?’, ‘What, if any, are the outstanding (business)
risks?’ and ‘Has development met their obligations?’. Acceptance testing is
most often the responsibility of the user or customer, although other stakehold-
ers may be involved as well. The execution of the acceptance test requires a test
environment that is for most aspects, representative of the production environ-
ment (‘as-if production’). The goal of acceptance testing is to establish confidence in the system, part of the system or specific non-functional characteristics, e.g. usability, of the system. Acceptance testing is most often focused on a validation type of testing,
whereby we are trying to determine whether the system is fit for purpose.
Finding defects should not be the main focus in acceptance testing.

Question 28

What are the two main types of acceptance testing?

Answer 28

The user acceptance test

Operational acceptance test

Question 29

What is the user acceptance test?

Answer 29

The user acceptance test focuses mainly on the functionality thereby validating the fitness-for-use of the
system by the business user?

Question 30

What is the operatinal acceptance test? (production acceptance test)

Answer 30

validates whether the system meets the

requirements for operation.

Question 31

What are the other types of acceptance testing?

Answer 31

contract acceptance testing

compliance acceptance testing

Question 32

What is alpha testing stage in acceptance testing?

Answer 32

This test takes place at the devel-
oper’s site. A cross-section of potential users and members of the developer’s
organization are invited to use the system. Developers observe the users and
note problems. Alpha testing may also be carried out by an independent test
team.

Question 33

What is beta testing stage in acceptance testing? (field testing)

Answer 33

sends the system to a cross-section of users
who install it and use it under real-world working conditions. The users send
records of incidents with the system to the development organization where the
defects are repaired.

Question 34

Tell about the test types please

Answer 34

Test types are introduced as a means of clearly defining the objective of a certain test level for a programme or project. We need to think about differ-ent types of testing because testing the functionality of the component or system may not be sufficient at each level to meet the overall test objectives.
Focusing the testing on a specific test objective and, therefore, selecting the appropriate type of test helps making and communicating decisions against
test objectives easier.
A test type is focused on a particular test objective, which could be the testing of a function to be performed by the component or system; a non-
functional quality characteristic, such as reliability or usability; the structure or architecture of the component or system; or related to changes, i.e. con-firming that defects have been fixed (confirmation testing, or re-testing) and looking for unintended changes (regression testing). Depending on its objec-
tives, testing will be organized differently. For example, component testing
aimed at performance would be quite different to component testing aimed
at achieving decision coverage.

Question 35

What is the function of a system?

Answer 35

The function of a system (or component) is ‘what it does’.

Question 36

What is a functional testing?

Answer 36

Functional testing considers the specified behavior and is often also referred
to as black-box testing. This is not entirely true, since black-box testing also
includes non-functional testing

Function (or functionality) testing can, based upon ISO 9126, be done focus-
ing on suitability, interoperability, security, accuracy and compliance.

Question 37

From which perspectives testing functionality can be done?

Answer 37

Testing functionality can be done from two perspectives: requirements-based
or business-process-based.

Question 38

What is a requirements-based perspective of functional testing?

Answer 38

Requirements-based testing uses a specification of the functional require-
ments for the system as the basis for designing tests. A good way to start is to
use the table of contents of the requirements specification as an initial test
inventory or list of items to test (or not to test). We should also prioritize the
requirements based on risk criteria (if this is not already done in the specifica-
tion) and use this to prioritize the tests. This will ensure that the most impor-
tant and most critical tests are included in the testing effort

Question 39

What is a Business-process-based perspective of functional testing?

Answer 39

Business-process-based testing uses knowledge of the business processes.
Business processes describe the scenarios involved in the day-to-day business
use of the system. For example, a personnel and payroll system may have a busi-
ness process along the lines of: someone joins the company, he or she is paid on
a regular basis, and he or she finally leaves the company. Use cases originate
from object-oriented development, but are nowadays popular in many develop-
ment life cycles. They also take the business processes as a starting point,
although they start from tasks to be performed by users. Use cases are a very
useful basis for test cases from a business perspective.

Question 40

What is non-functional testing?

Answer 40

A second target for testing is the testing of the quality characteristics, or non-
functional attributes of the system (or component or integration group). Here
we are interested in how well or how fast something is done. We are testing
something that we need to measure on a scale of measurement, for example
time to respond.
Non-functional testing, as functional testing, is performed at all test levels.
Non-functional testing includes, but is not limited to, performance testing, load
testing, stress testing, usability testing, maintainability testing, reliability testing
and portability testing.

Question 41

What is ISO?

Answer 41

The
International Organization for Standardization (ISO) has defined a set of
quality characteristics [ISO/IEC 9126, 2001]. This set reflects a major step
towards consensus in the IT industry and thereby addresses the general notion
of software quality.

Question 42

What is ISO 9126 standard?

Answer 42

The ISO 9126 standard defines six software quality characteristics
and the subdivision of each quality characteristic into a number of sub-characteristics.

The characteristics and their sub-characteristics are, respectively:

• functionality, which consists of five sub-characteristics: suitability, accuracy,
security, interoperability and compliance; this characteristic deals with func
tional testing as described in Section 2.3.1;
• reliability, which is defined further into the sub-characteristics maturity
(robustness), fault-tolerance, recoverability and compliance;
• usability, which is divided into the sub-characteristics understandability,
learnability, operability, attractiveness and compliance;
• efficiency, which is divided into time behavior (performance), resource uti
lization and compliance;
• maintainability, which consists of five sub-characteristics: analyzability,
changeability, stability, testability and compliance;
• portability, which also consists of five sub-characteristics: adaptability,
installability, co-existence, replaceability and compliance.

Question 43

What the major test types?

Answer 43

Functional, non-functional, strutctural, related ot changes

Question 44

What is structural testing (white-box)?

Answer 44

The third target of testing is the structure of the system or component. If we are
talking about the structure of a system, we may call it the system architecture.
Structural testing is often referred to as ‘white-box’ or ‘glass-box’ because we
are interested in what is happening ‘inside the box’.
It can occur at any test level, although is it true to say that it tends to be
mostly applied at component and integration and generally is less likely at
higher test levels, except for business-process testing.

Question 45

What code coverage measuring tools do?

Answer 45

Coverage
measurement tools assess the percentage of executable elements (e.g. state-
ments or decision outcomes) that have been exercised (i.e. covered) by a
test suite. If coverage is not 100%, then additional tests may need to be
written and run to cover those parts that have not yet been exercised.

Question 46

What is confirmation testing or re-testing?

Answer 46

When a test fails and we determine that the cause of the failure is a software
defect, the defect is reported, and we can expect a new version of the software
that has had the defect fixed. In this case we will need to execute the test again
to confirm that the defect has indeed been fixed. This is known as confirmation
testing (also known as re-testing).
When doing confirmation testing, it is important to ensure that the test is
executed in exactly the same way as it was the first time, using the same inputs,
data and environment.

Question 47

What is a regression testing?

Answer 47

The term ‘regression testing’ is something of a misnomer. It would be better
if it were called ‘anti-regression’ testing because we are executing tests with the
intent of checking that the system has not regressed (that is, it does not now
have more defects in it as a result of some change). More specifically, the
purpose of regression testing is to verify that modifications in the software or
the environment have not caused unintended adverse side effects and that the
system still meets its requirements

Question 48

What is a regression test

suite or regression test pack?

Answer 48

This is a set of test cases that is specifically used for
regression testing. They are designed to collectively exercise most functions (cer-
tainly the most important ones) in a system but not test any one in detail. It is
appropriate to have a regression test suite at every level of testing (component
testing, integration testing, system testing, etc.).

Question 49

What is maintanance testing?

Answer 49

Once deployed, a system is often in service for years or even decades. During
this time the system and its operational environment is often corrected,
changed or extended. Testing that is executed during this life cycle phase is
called ‘maintenance testing’.
Usually maintenance testing will consist of two parts:
• testing the changes
• regression tests to show that the rest of the system has not been affected by
the maintenance work

Question 50

What is impact analysis?

Answer 50

During impact analysis, together with stakeholders, a deci-
sion is made on what parts of the system may be unintentionally affected and
therefore need careful regression testing. Risk analysis will help to decide
where to focus regression testing - it is unlikely that the team will have time
to repeat all the existing tests

Question 51

What are the triggers for maintenance testing?

Answer 51

It is triggered by modifications, migration, or retirement of the system.
Modifications include planned enhancement changes (e.g. release-based), cor-
rective and emergency changes, and changes of environment, such as planned
operating system or database upgrades, or patches to newly exposed or discov-
ered vulnerabilities of the operating system. Maintenance testing for migration
(e.g. from one platform to another) should include operational testing of the
new environment, as well as the changed software. Maintenance testing for the
retirement of a system may include the testing of data migration or archiving, if
long data-retention periods are required.

Question 52

What are the two main types of modifictaions from the point of view of testing?

Answer 52

There are modifications in
which testing may be planned, and there are ad-hoc corrective modifications,
which cannot be planned at all. Ad-hoc corrective maintenance takes place
when the search for solutions to defects cannot be delayed. Special test proce-
dures are required at that time.

Question 53

What are the main types of planned modifications?

Answer 53

• perfective modifications (adapting software to the user’s wishes, for instance
by supplying new functions or enhancing performance);
• adaptive modifications (adapting software to environmental changes such as
new hardware, new systems software or new legislation);
• corrective planned modifications (deferrable correction of defects).

Question 54

What are ad-hoc corrective modifications?

Answer 54

Ad-hoc corrective modifications are concerned with defects requiring an imme-
diate solution, e.g. a production run which dumps late at night, a network that
goes down with a few hundred users on line, a mailing with incorrect addresses

Question 55

What is commercital off-the-shelf software (COTS)?

Answer 55

A software product that is developed for the general market, i.e. for a large
number of customers, and that is delivered to many customers in identical format.

Question 56

What is incremental development model?

Answer 56

A development lifecycle where a project is broken into a
F-AT series of increments, each of which delivers a portion of the functionality in the overall project
requirements. The requirements are prioritized and delivered in priority order in the appropriate
increment. In some (but not all) versions of this lifecycle model, each subproject follows a ‘mini Vmodel’
with its own design, coding and testing phases.

Question 57

What is test level?

Answer 57

A group of test activities that are organized and managed together. test level is linked to the responsibilities in a project. Examples of test levels are component test, integration
test, system test and acceptance test. [After TMap

Question 58

What is validation?

Answer 58

Confirmation by examination and through provision of objective evidence that the
requirements for a specific intended use or application have been fulfilled. [ISO 9000

Question 59

What is a functional requirement?

Answer 59

A requirement that specifies a function that a component or system must
perform. [IEEE 610]

Question 60

What is integration?

Answer 60

The process of combining components or systems into larger assemblies.

Question 61

What is operational testing?

Answer 61

Testing conducted to evaluate a component or system in its operational
environment. [IEEE 610]

Question 62

What is compliance testing?

Answer 62

The process of testing to determine the compliance of the component or system.

Question 63

What is robustness testing?

Answer 63

Testing to determine the robustness of the software product

Question 64

What is robustness?

Answer 64

The degree to which a component or system can function correctly in the presence of invalid inputs or stressful environmental conditions. [IEEE 610]

Question 65

What is test environment?

Answer 65

An environment containing hardware, instrumentation, simulators, software tools,
and other support elements needed to conduct a test. [After IEEE 610]

Question 66

What is interoperability?

Answer 66

The capability of the software product to interact with one or more specified
components or systems. [After ISO 9126]

Question 67

What is inetroperability testing?

Answer 67

The process of testing to determine the interoperability of a software product.

Question 68

What is load testing?

Answer 68

A type of performance testing conducted to evaluate the behavior of a component or
system with increasing load, e.g. numbers of parallel users and/or numbers of transactions, to
determine what load can be handled by the component or system. See also performance testing,
stress testing.

Question 69

What is maintainability?

Answer 69

The ease with which a software product can be modified to correct defects, modified
to meet new requirements, modified to make future maintenance easier, or adapted to a changed
environment. [ISO 9126]

Question 70

What is maintainability testing?

Answer 70

The process of testing to determine the maintainability of a software product.

Question 71

What is performance?

Answer 71

The degree to which a system or component accomplishes its designated functions
within given constraints regarding processing time and throughput rate. [After IEEE 610] See also
efficiency.

Question 72

What is performance testing?

Answer 72

The process of testing to determine the performance of a software
F-AT product. See also efficiency testing.

Question 73

What is portability testing?

Answer 73

The process of testing to determine the portability of a software product.

Question 74

What is portability?

Answer 74

The ease with which the software product can be transferred from one hardware or
software environment to another. [ISO 9126]

Question 75

What is reliability?

Answer 75

The ability of the software product to perform its required functions under stated conditions
for a specified period of time, or for a specified number of operations. [ISO 9126]

Question 76

What is reliability testing?

Answer 76

The process of testing to determine the reliability of a software product.

Question 77

What is security?

Answer 77

Attributes of software products that bear on its ability to prevent unauthorized access,
whether accidental or deliberate, to programs and data. [ISO 9126]

Question 78

What is security testing?

Answer 78

Testing to determine the security of the software product.

Question 79

What is a specification-based testing?

Answer 79

It is a black box testing.
Testing, either functional or non-functional, without reference to the internal
structure of the component or system.

Question 80

What is stress testing?

Answer 80

A type of performance testing conducted to evaluate a system or component at or
beyond the limits of its anticipated or specified workloads, or with reduced availability of resources
such as access to memory or servers. [After IEEE 610]

Question 81

What is test suite?

Answer 81

A set of several test cases for a component or system under test, where the post condition
of one test is often used as the precondition for the next one.

Question 82

What is usability testing?

Answer 82

Testing to determine the extent to which the software product is understood, easy to learn, easy to operate and attractive to the users under specified conditions. [After ISO 9126]