Software Testing Flashcards
What is Unit Testing?
UNIT TESTING is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. A unit is the smallest testable part of any software. It usually has one or a few inputs and usually a single output. In procedural programming, a unit may be an individual program, function, procedure, etc. In object-oriented programming, the smallest unit is a method, which may belong to a base/ super class, abstract class or derived/ child class. (Some treat a module of an application as a unit. This is to be discouraged as there will probably be many individual units within that module.) Unit testing frameworks, drivers, stubs, and mock/ fake objects are used to assist in unit testing.
Unit Testing Method It is performed by using the White Box Testing method.
When is it performed? Unit Testing is the first level of software testing and is performed prior to Integration Testing.
Who performs it? It is normally performed by software developers themselves or their peers. In rare cases, it may also be performed by independent software testers.
What is Integration Testing?
INTEGRATION TESTING is a level of software testing where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units. Test drivers and test stubs are used to assist in Integration Testing.
Analogy
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. For example, whether the cap fits into the body or not.
Method
Any of Black Box Testing, White Box Testing and Gray Box Testing methods can be used. Normally, the method depends on your definition of ‘unit’.
What is System Testing?
SYSTEM TESTING is a level of software testing where a complete and integrated software is tested. The purpose of this test is to evaluate the system’s compliance with the specified requirements.
Analogy
During the process of manufacturing a ballpoint pen, the cap, the body, the tail, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. When the complete pen is integrated, System Testing is performed.
Method
Usually, Black Box Testing method is used.
What is Acceptance Testing?
ACCEPTANCE TESTING is a level of software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system’s compliance with the business requirements and assess whether it is acceptable for delivery.
Analogy
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and unit tested separately. When two or more units are ready, they are assembled and Integration Testing is performed. When the complete pen is integrated, System Testing is performed. Once System Testing is complete, Acceptance Testing is performed so as to confirm that the ballpoint pen is ready to be made available to the end-users.
Method
Usually, Black Box Testing method is used in Acceptance Testing. Testing does not normally follow a strict procedure and is not scripted but is rather ad-hoc.
Who performs it?
Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the organization that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product Management, Sales and/or Customer Support.
External Acceptance Testing is performed by people who are not employees of the organization that developed the software.
Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software. [This is in the case of the software not being owned by the organization that developed it.]
User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the software. They can be the customers themselves or the customers’ customers.
What is Black Box testing?
BLACK BOX TESTING, also known as Behavioral Testing, is a software testing method in which the internal structure/design/implementation of the item being tested is not known to the tester. These tests can be functional or non-functional, though usually functional.
This method is named so because the software program, in the eyes of the tester, is like a black box; inside which one cannot see.
Techniques
Following are some techniques that can be used for designing black box tests.
Equivalence Partitioning: It is a software test design technique that involves dividing input values into valid and invalid partitions and selecting representative values from each partition as test data.
Boundary Value Analysis: It is a software test design technique that involves the determination of boundaries for input values and selecting values that are at the boundaries and just inside/ outside of the boundaries as test data.
Cause-Effect Graphing: It is a software test design technique that involves identifying the cases (input conditions) and effects (output conditions), producing a Cause-Effect Graph, and generating test cases accordingly.
Advantages
Tests are done from a user’s point of view and will help in exposing discrepancies in the specifications.
Tester need not know programming languages or how the software has been implemented.
Tests can be conducted by a body independent from the developers, allowing for an objective perspective and the avoidance of developer-bias.
Test cases can be designed as soon as the specifications are complete.
Disadvantages
Only a small number of possible inputs can be tested and many program paths will be left untested.
Without clear specifications, which is the situation in many projects, test cases will be difficult to design.
Tests can be redundant if the software designer/developer has already run a test case.
Ever wondered why a soothsayer closes the eyes when foretelling events? So is almost the case in Black Box Testing.
What is White Box Testing?
WHITE BOX TESTING (also known as Clear Box Testing, Open Box Testing, Glass Box Testing, Transparent Box Testing, Code-Based Testing or Structural Testing) is a software testing method in which the internal structure/design/implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming know-how and the implementation knowledge is essential. White box testing is testing beyond the user interface and into the nitty-gritty of a system.
This method is named so because the software program, in the eyes of the tester, is like a white/transparent box; inside which one clearly sees.
Example
A tester, usually a developer as well, studies the implementation code of a certain field on a webpage, determines all legal (valid and invalid) AND illegal inputs and verifies the outputs against the expected outcomes, which is also determined by studying the implementation code.
White Box Testing is like the work of a mechanic who examines the engine to see why the car is not moving.
Advantages
Testing can be commenced at an earlier stage. One need not wait for the GUI to be available.
Testing is more thorough, with the possibility of covering most paths.
Disadvantages
Since tests can be very complex, highly skilled resources are required, with a thorough knowledge of programming and implementation.
Test script maintenance can be a burden if the implementation changes too frequently.
Since this method of testing is closely tied to the application being tested, tools to cater to every kind of implementation/platform may not be readily available.
What is Gray Box Testing?
GRAY BOX TESTING is a software testing method which is a combination of Black Box Testing method and White Box Testing method. In Black Box Testing, the internal structure of the item being tested is unknown to the tester and in White Box Testing the internal structure is known. In Gray Box Testing, the internal structure is partially known. This involves having access to internal data structures and algorithms for purposes of designing the test cases, but testing at the user, or black-box level.
Gray Box Testing is named so because the software program, in the eyes of the tester is like a gray/semi-transparent box; inside which one can partially see.
Example
An example of Gray Box Testing would be when the codes for two units/modules are studied (White Box Testing method) for designing test cases and actual tests are conducted using the exposed interfaces (Black Box Testing method).
Levels Applicable To
Though Gray Box Testing method may be used in other levels of testing, it is primarily used in Integration Testing.
What is Ad hoc testing?
AD HOC TESTING, also known as Random Testing or Monkey Testing, is a method of software testing without any planning and documentation. The tests are conducted informally and randomly without any formal procedure or expected results.
The tester improvises the steps and arbitrarily executes them (like a monkey typing while dancing). Though defects found using this method are more difficult to reproduce (since there are no written test cases), sometimes very interesting defects are found which would never have been found if written test cases existed and were strictly followed. This method is normally used during Acceptance Testing.
The success of ad hoc testing depends on the creativity and tenacity of the tester (and, of course, luck).
What is smoke testing?
SMOKE TESTING, also known as “Build Verification Testing”, is a type of software testing that comprises of a non-exhaustive set of tests that aim at ensuring that the most important functions work. The result of this testing is used to decide if a build is stable enough to proceed with further testing.
The term ‘smoke testing’, it is said, came to software testing from a similar type of hardware testing, in which the device passed the test if it did not catch fire (or smoked) the first time it was turned on.
Elaboration
Smoke testing covers most of the major functions of the software but none of them in depth. The result of this test is used to decide whether to proceed with further testing. If the smoke test passes, go ahead with further testing. If it fails, halt further tests and ask for a new build with the required fixes. If an application is badly broken, detailed testing might be a waste of time and effort.
Smoke test helps in exposing integration and major problems early in the cycle. It can be conducted on both newly created software and enhanced software. Smoke test is performed manually or with the help of automation tools/scripts. If builds are prepared frequently, it is best to automate smoke testing.
As and when an application becomes mature, with addition of more functionalities etc, the smoke test needs to be made more expansive. Sometimes, it takes just one incorrect character in the code to render an entire application useless.
Advantages
- It exposes integration issues.
- It uncovers problems early.
- It provides some level of confidence that changes to the software have not adversely affected major areas (the areas covered by smoke testing, of course)
Levels
Smoke testing is normally used in Integration Testing, System Testing and Acceptance Testing levels.
What is functional testing?
FUNCTIONAL TESTING is a type of software testing whereby the system is tested against the functional requirements/specifications.
Functions (or features) are tested by feeding them input and examining the output. Functional testing ensures that the requirements are properly satisfied by the application. This type of testing is not concerned with how processing occurs, but rather, with the results of processing. It simulates actual system usage but does not make any system structure assumptions.
During functional testing, Black Box Testing technique is used in which the internal logic of the system being tested is not known to the tester.
Functional testing is normally performed during the levels of System Testing and Acceptance Testing.
Typically, functional testing involves the following steps:
Identify functions that the software is expected to perform.
Create input data based on the function’s specifications.
Determine the output based on the function’s specifications.
Execute the test case.
Compare the actual and expected outputs.
Functional testing is more effective when the test conditions are created directly from user/business requirements. When test conditions are created from the system documentation (system requirements/ design documents), the defects in that documentation will not be detected through testing and this may be the cause of end-users’ wrath when they finally use the software.
What is usability testing?
USABILITY TESTING is a type of testing done from an end-user’s perspective to determine if the system is easily usable.
Elaboration
Systems may be built 100% in accordance with the specifications. Yet, they may be ‘unusable’ when it lands in the hands of the end-users. For instance, let’s say a user needs to print a Financial Update Report, every 30 minutes, and he/she has to go through the following steps:
- Login to the system
- Click Reports
- From the groups of reports, select Financial Reports
- From the list of financial reports, select Financial Update Report
- Specify the following parameters
- Date Range
- Time Zone
- Departments
- Units
- Click Generate Report
- Click Print
- Select an option
- Print as PDF
- Print for Real
If that’s the case, the system is probably practically unusable (though it functions perfectly fine). If the report is to be printed frequently, wouldn’t it be convenient if the user could get the job done in a couple of clicks, rather than having to go through numerous steps like listed above? What if there was a feature to save frequently generated reports as a template and if the saved reports were readily available for printing from the homepage?
What is security testing?
SECURITY TESTING is a type of software testing that intends to uncover vulnerabilities of the system and determine that its data and resources are protected from possible intruders.
Focus Areas
There are four main focus areas to be considered in security testing (Especially for web sites/applications):
Network security: This involves looking for vulnerabilities in the network infrastructure (resources and policies).
System software security: This involves assessing weaknesses in the various software (operating system, database system, and other software) the application depends on.
Client-side application security: This deals with ensuring that the client (browser or any such tool) cannot be manipulated.
Server-side application security: This involves making sure that the server code and its technologies are robust enough to fend off any intrusion.
Example
This is an example of a very basic security test which anyone can perform on a web site/application:
Log into the web application.
Log out of the web application.
Click the BACK button of the browser (Check if you are asked to log in again or if you are provided the logged-in application.)
Most types of security testing involve complex steps and out-of-the-box thinking but, sometimes, it is simple tests like the one above that help expose the most severe security risks.
What is performance testing?
PERFORMANCE TESTING is a type of software testing that intends to determine how a system performs in terms of responsiveness and stability under a certain load.
Types
Load Testing is a type of performance testing conducted to evaluate the behavior of a system at increasing workload.
Stress Testing is a type of performance testing conducted to evaluate the behavior of a system at or beyond the limits of its anticipated workload.
Endurance Testing is a type of performance testing conducted to evaluate the behavior of a system when a significant workload is given continuously.
Spike Testing is a type of performance testing conducted to evaluate the behavior of a system when the load is suddenly and substantially increased.
Tips
Establish a test environment as close to the production environment as possible.
Isolate the test environment even from the QA or UAT environment.
Though there’s no perfect tool for performance testing, research and decide on the tool that best fits your purpose.
Do not rely on the results of one test. Conduct multiple tests to arrive at an average number. Be wary of any changes to the test environment from one test to the other.
What is Regression testing?
REGRESSION TESTING is a type of software testing that intends to ensure that changes (enhancements or defect fixes) to the software have not adversely affected it.
The likelihood of any code change impacting functionalities that are not directly associated with the code is always there and it is essential that regression testing is conducted to make sure that fixing one thing has not broken another thing.
During regression testing, new test cases are not created but previously created test cases are re-executed.
Levels
Regression testing can be performed during any level of testing (Unit, Integration, System, or Acceptance) but it is mostly relevant during System Testing.
Extent
In an ideal case, a full regression test is desirable but oftentimes there are time/resource constraints. In such cases, it is essential to do an impact analysis of the changes to identify areas of the software that have the highest probability of being affected by the change and that have the highest impact to users in case of malfunction and focus testing around those areas.
Due to the scale and importance of regression testing, more and more companies and projects are adopting regression test automation tools.
What is compliance testing?
COMPLIANCE TESTING, also known as conformance testing, regulation testing, standards testing, is a type of testing to determine the compliance of a system with internal or external standards.
Internal standards could be standards set by the company itself. For example, a web application development company might set the standard that all webpages must be responsive.
External standards could be standards set outside of the company. For example, Health Insurance Portability and Accountability Act (HIPAA) has set regulations for the healthcare industry.
Compliance testing could also be done by an external organization. This normally results in some sort of compliance certification.
The method and type of testing to be conducted during compliance testing depends on the specific regulation / standard being assessed.
The depth of compliance testing could range from a high-level audit on a sampling basis to a detailed scrutiny of each specified standard.
