|
Glossary of Software Testing Terms:
S
This glossary of software testing
terms and conditions is a compilation of knowledge, gathered
over time, from many different sources. It is provided “as-is”
in good faith, without any warranty as to the accuracy or
currency of any definition or other information contained
herein. If you have any questions or queries about the contents
of this glossary, please contact
Project Realms directly.
Safety Testing
The process of testing to determine the safety of a software
product.
Top
Sanity Testing
Brief test of major functional elements of a piece of software
to determine if it's basically operational. See also Smoke
Testing.
Top
Scalability Testing
Scalability Testing is designed to determine the ability of
a system to expand as the usage increases. It measures individual
components of the system to determine their performance characteristics
under increasing user loads. It also examines inter-component
and multiple component behavior.
Top
Schedule
A scheme for the execution of test procedures.
The test procedures are included in the test execution schedule
in the order in which they are to be executed.
Top
Scrambling
Data obfuscation routine to de-identify sensitive data in test
data environments to meet the requirements of the Data Protection
Act and other legislation.
Top
Scribe
The person who has to record each defect mentioned and any
suggestions for improvement during a review meeting, on a logging
form. The scribe has to make ensure that the logging form is
understandable.
Top
Script
When the user's actions are recorded, the output that is generated
and stored in a script.
Top
Security
The preservation of availability, integrity, and confidence
of information:
- Availability is ensuring that authorized users have access
to information and associated assets when required.
- Integrity is safeguarding the accuracy and completeness
of information and processing methods.
- Confidentiality is ensuring that information is accessible
only to those authorized to have access.
Top
Security Requirements
A specification of the required security for the system or software.
Top
Security Testing
This is a process to determine that an IS (Information System)
protects data and maintains functionality as intended.
The six concepts that need to be covered by security testing
are: confidentiality, integrity, authentication, authorization,
availability, and non-repudiation.
Confidentiality
A security measure which protects against the disclosure of
information to parties other than the intended recipient(s).
Often ensured by means of encoding, using a defined algorithm
and some secret information known only to the originator of
the information and the intended recipient(s) (a process known
as cryptography) but that is by no means the only way of ensuring
confidentiality.
Integrity
A measure intended to allow the receiver to determine that the
information which it receives has not been altered in transit
or by other than the originator of the information. Integrity
schemes often use some of the same underlying technologies as
confidentiality schemes, but they usually involve adding additional
information to a communication to form the basis of an algorithmic
check rather than encoding all of the communication.
Authentication
A measure designed to establish the validity of a transmission,
message, or originator. It allows a receiver to have confidence
that the information it receives originated from a specific
known source.
Authorization
The process of determining that a requester is allowed to receive
a service or perform an operation.
Availability
Assuring information and communications services will be ready
for use when expected. Information must be kept available to
authorized persons when they need it.
Non-repudiation
A measure intended to prevent the later denial that an action
happened, or a communication took place, etc. In communication
terms, this often involves the interchange of authentication
information combined with some form of provable time stamp.
Security Testing focuses on these key areas of security
- Application-level Security; including access to the Data
or Business Functions
- System-level Security; including logging into / remote
access to the system
Application-level security ensures that, based upon
the desired security, users are restricted to specific functions
/ use cases or are limited in the data that is available to
them. For example, every user may be permitted to enter data
and create new accounts, but only managers can delete them.
If there is security at the data level, testing ensures that
all users can see all customer information, including financial
data; however, managers can also see the demographic data
for the client.
System-level security ensures that only those users
granted access to the system are capable of accessing the
applications and only through the appropriate gateways.
Security testing is a broad category of testing. Based on
the two types listed above, there are some areas of focus
to consider:
- Application Based Security - These are home grown
solutions in which the programmers create and maintain users
and user security permissions within a given application.
This means the users, their security roles and their security
permissions are written in the application. Security could
be applied at a field level, screen level, feature level
or system level. One common subset of this type is to utilize
operating system based security (see below) and integrate
it with application based security. This means the basic
authorization for the user is managed at the operating system
level, but if authorized, then application specific permissions
are managed by the programmers. A review of the system design
and requirements should help define the testing effort for
these types of security systems
- Operating System Based Security - This security
is defined by the Operating System (OS) of the system. Microsoft
uses an OS based security that is managed by an internal
application called Active Directory (AD). AD allows system
administrators to create and manage users on the system.
It defines what level of access the user has to system objects
and Microsoft system hardware. It allows the customization
of user access by utilizing security groups and the definition
of custom user security properties. It is recommended that
you study Active Directory from Microsoft to understand
how to test this type of security
- Device Based Security - There are many hardware
devices on the market dedicated to security. Hardware firewalls,
Security Card Readers and biometric devices are just a few.
Each of these are some combination of a physical device
and software designed to interpret the information passing
through the physical device while applying security rules
to or based on the data flowing through the device. Testing
of these devices requires knowledge of what the device is
for and how it works. The device manufacturer will typically
make this information available, though sometimes they require
specialized training
- Web Based Security - This is a combination of all
three security types; Application, OS and Device based security.
As such it can be tested from various points of view. One
could test the browser security and the code presented in
the browser. Both efforts would be considered application
based security testing. One could also test the web server
security. This testing attempts to uncover vulnerabilities
at the web server tier. This is considered both application
and OS based security testing as web servers are typically
distributed with an OS. Another level of testing could be
external, intrusion testing. This probes for vulnerabilities
from outside the web based system. This will include testing
all three securities simultaneously. The security testing
of web systems is a discipline and a career for some people.
Top
Simple Subpath
A subpath of the control flow graph in which no program part
is executed more than necessary.
Top
Simulation
The representation of selected behavioral characteristics of
one physical or abstract system by another system.
Top
Simulator
A device, computer program or system used during software verification,
which behaves or operates like a given system when provided
with a set of controlled inputs.
Top
Smoke Testing
Smoke testing is a quick test (sometimes called a sanity check)
often used to validate a new build. It validates that the configuration
is available, functional, and no gross defects are apparent.
Key points to include in a smoke test are:
- If the System has a User Interface, then request each
screen once to:
- Validate the screen exists
- Validate key points on the screen to ensure you are
not getting an error message screen and that the screen
is usable. This is subject to interpretation, but the
guiding principal is gross functionality is available
- Do not validate all the data on the screen. This is
the job of other tests
- Request each system interface to verify the following:
- The connection is active
- The response was not an error response
- The response was timely
- The data is accurate for a single scenario. This can
occur via a User Interface or directly through the database
- Smoke test should be no longer than 30 minutes. They are
typically 5 minutes in duration
- Create a single Test Case, a group of Test Cases called
Smoke Tests or a check list identifying each component of
the Smoke Test to ensure repeatability
- Use automated tools whenever possible
Top
Soak Testing
Soak testing involves testing a system with a significant load
extended over a significant period of time, to discover how
the system behaves under sustained use.
For example, in software testing, a system may behave exactly
as expected when tested for 1 hour. However, when it is tested
for 3 hours, problems such as memory leads cause the system
to fail or behave randomly.
Soak tests are used primarily to check the reaction of a subject
under test under a possible simulated environment for a given
duration and for a given threshold. Observations made during
the soak test are used to improve the characteristics of the
subject under test further.
Top
Software Requirements Specification
This is a deliverable that describes all data, functional, and
behavioral requirements, all constraints, and all validation
requirements for software.
Top
Software Testing
The process used to measure the quality of developed computer
software. Usually, quality is constrained to such topics as
correctness, completeness, security, but can also include more
technical requirements as described under the ISO standard ISO
9126, such as capability, reliability, efficiency, portability,
maintainability, compatibility, and usability. Testing is a
process of technical investigation, performed on behalf of stakeholders,
that is intended to reveal quality-related information about
the product with respect to the context in which it is intended
to operate. This includes, but is not limited to, the process
of executing a program or application with the intent of finding
errors. Quality is not an absolute; it I a value to some person.
With that in mind, testing can never completely establish the
correctness of arbitrary computer software; testing furnishes
a criticism or comparison that compares the state and behavior
of the product against a specification. An important point is
that software testing should be distinguished from the separate
discipline of Software Quality Assurance (SQA), which encompasses
all business process areas, and not just testing.
Today, software has grown in complexity and size. The software
product developed by a developer is according to the System
Requirement Specification. Every software product has a target
audience. For example, a video game software has its audience
completely different from baking software. Therefore, when an
organization invests large sums in making a software product,
it must ensure that the software product must be acceptable
to the end users or its target audience. This is where Software
Testing comes into play. Software testing is not merely finding
defects or bugs in the software, it is the completely dedicated
discipline of evaluating the quality of the software.
There are many approaches to software testing, but effective
testing of complex products is essentially a process of investigation,
not merely a matter of creating and following routine procedures.
One definition of testing is "the process of questioning
a product in order to evaluate it", where the "questions"
are operations the test attempts to execute with the product,
and the product answers with its behavior in reaction to the
probing of the tester. Although most of the intellectual processes
of testing are nearly identical to that of review or inspection,
the word testing is also used to connote the dynamic analysis
of the product - putting the product through its paces. Sometimes
one therefore refers to reviews, walkthroughs
or inspections as "static testing".
Whereas actually running the program with a given set of test
cases in a given development stage is often referred to as "dynamic
testing", to emphasize the fact that formal review
processes form part of the overall testing scope.
Top
Specification
A description, in any suitable form, of requirements.
Top
Specification Testing
An approach to testing wherein the testing is restricted to
verifying the system/software meets an agreed specification.
Top
Specified Input
An input for which the specification predicts an outcome.
Top
Stabilization Period
A period of time after the system is promoted to production,
during which the project team may need to support post-deployment
patches. The amount of time will vary depending on each project,
but it may span a few days to a few weeks. It is wise to ensure
that testing personnel are available to support these patches
before moving on to the next testing project. This period of
time is sometimes called Code
Freeze.
Top
State Transition
A transition between two allowable states of a system or component.
Top
State Transition Testing
A test case design technique in which test cases are designed
to execute state transitions.
Top
Statement
An entity in a programming language which is typically the smallest
indivisible unit of execution.
Top
Statement Coverage
The percentage of executable statements in a component that
have been exercised by a test case suite.
Top
Statement Testing
A test case design technique for a component in which test cases
are designed to execute statements. Statement testing is a structural
or white box technique, because it is conducted with reference
to the code. Statement testing comes under dynamic analysis.
In an ideal world every statement of every component would be
fully tested. However, in the real world this hardly ever happens.
In statement testing every possible statement is tested.
Top
Static Analysis
Analysis of a program carried out without executing the program.
Top
Static Analyzer
A tool that carries out static analysis.
Top
Static Code Analysis
The analysis of computer software that is performed without
actually executing programs built from that software. In most
cases the analysis is performed on some version of the source
code and in the other cases some form of the object code. The
term is usually applied to the analysis performed by an automated
tool, with human analysis being called program understanding
or program comprehension's.
Top
Static Testing
This form of testing involves any activity that looks for defects
by inspecting, reviewing, and analyzing any static component
of the software development process. Multiple testing roles
should participate in the static testing exercise in order to
gather diverse points of view. Static components include code,
design documents, models, requirements, test plans, test cases,
etc. Investing time in static testing activities offers a high
return on that investment rather than finding defects and managing
change later in dynamic testing (i.e., Unit Testing, System
Testing, etc.).
Top
Statistical Testing
A test case design technique in which a model is used of the
statistical distribution of the input to construct representative
test cases.
Top
Storage Testing
Testing that verifies the program under test stores data files
in the correct directories and that it reserves sufficient space
to prevent unexpected termination resulting from lack of space.
This is external storage as opposed to internal storage.
Top
Stress Testing
Stress testing determines the stability of an application to
perform under low and/or excessive loads to ensure that critical
information and services are available when and how the end
user expects them. It identifies conditions under which the
system fails to continue functioning properly, including the
peak workload the System Under Test can handle. It involves
testing beyond normal operational capacity, often to a breaking
point, in order to observe the results. In addition, Stress
Testing is used to find defects that result from competition
for shared resources.
Top
Structural Coverage
Coverage measures based on the internal structure of the component.
Top
Structural Test Case Design
Test case selection that is based on an analysis of the internal
structure of the component.
Top
Structural Testing
See Structural Test Case
Design.
Top
Structured Basis Testing
A test case design technique in which test cases are derived
from the code logic to achieve % branch coverage.
Top
Structured Walkthrough
See Walkthrough.
Top
Stub
A skeletal or special-purpose implementation of a software module,
used to develop or test a component that calls or is otherwise.
Top
Subgoal
An attribute which becomes a temporary intermediate goal for
the inference engine. Subgoal values need to be determined because
they are used in the premise of rules that can determine higher
level goals.
Top
Subpath
A sequence of executable statements within a component.
Top
Suitability
The capability of the software product to provide an appropriate
set of functions for specified tasks and user objectives.
Top
Suspension Criteria
The criteria used to (temporarily) stop all or a portion of
the testing activities on the test items.
Top
Symbolic Execution
A static analysis technique used to analyze if and when errors
in the code may occur. It can be used to predict what code statements
do to specified inputs and outputs. It is also important for
considering path traversal. It struggles when dealing with statements
which are not purely mathematical.
Top
Symbolic Processing
Use of symbols, rather than numbers, combined with rules-of-thumb
(or heuristics),
in order to process information and solve problems.
Top
Syntax Testing
A test case design technique for a component
or system in which test case design is based upon the syntax
of the input.
Top
System
The application software and hardware related to the testing
effort. This includes application code, database software and
hardware, operating system software, server hardware and software,
network equipment and connectivity, load balancing software
and hardware, and third party software included in processing
of the simulated transactions. It does not include external
systems that are deemed in scope of the testing effort.
Top
System Integration Testing
System Integration Testing is the process of testing the system
as a whole with other applications, systems, and interfaces.
It ensures that the various pieces interact properly and function
together. System Integration Testing assumes that system testing
of the allocated features and functions has previously occurred
and that those features and functions operate correctly. If
certain features or functions are inoperable, then there must
be a practical and documented workaround. This type of testing
is sometimes called Integration Testing or End-To-End
testing.
Testing a collection of systems and ensuring that they communicate
with each other is a part of this type of testing. As this
is a communications test, the verifications should focus on
integration points. Verifications could include:
- Verifying successful transfer and processing of information
through a set of systems
- Verifying Error Condition behaviors when part of the system
chain is broken
- Determine the system behavior when immediate systems are
no longer connected
- Determine the system behavior when secondary systems are
no longer connected
Top
System Testing
System Testing is a suite of testing that consists of, but is
not limited to, Configuration
Testing, Security Testing,
Data
and Database Integrity Testing, User
Interface Testing, Failover
Testing, Functional
Testing, Installation
Testing, Recovery
Testing, and Stress Testing.
System Testing assumes that testing of the allocated features
and functions has occurred previously and that those features
and functions operate correctly. If certain features or functions
are inoperable, then there must be a practical and documented
workaround. System testing is aimed at proving that the application
or system meets the stated requirements and objectives.
Top
System Under Test (SUT)
See System.
Top
| Contact us for more info
|
|
