In the following article, we explore the concept of the performance test environment and the items that should be considered or asked when defining and building the environment to be used for performance testing. It will cover the following-
- Environment Definition
- Environment Architecture
- Environment Monitoring
- Environment Sizing
- Environment Resources
- Environment Data and
- Environment Scheduling
The term Environment on its own is ambiguous. What is generally being spoken about is either Environment Type or Environment Instance. However, when we state: ‘We need four environments.’ we are normally referring to the fact that we need the ability to run four simultaneous instances of one or more types of test environments. For example, development environments, system integration, user acceptance and performance/non-functional environments.
A test environment encompasses everything that is required to perform a certain phase of testing. Therefore, a performance test environment is likely to consist of:
- Operating system(s).
- Servers i.e. Web Server, App Server, DB Server, File Server etc.
- Clients i.e. Desktops, laptops etc.
- *Connectivity to the test assets.
- Test tools.
- Software products (including 3rd party software).
- Interfaces between systems and applications.
*Includes the networking aspects: firewall rules, IP settings, MACs, routers/switches, and so on.
What does the Architecture consist of?
Hardware Architecture – Document all the hardware items including versions to be deployed, this will make up the performance test environment, ideally, you want a structural design that gives enough detail to implement the solution, for an expel of this, see the table below.
Software Architecture — Document all the software items including versions etc. that will be installed onto the performance test environment.
See the table below for an example.
Refer to our article A High-Level Insight Into Performance Monitoring
What monitoring will be required on the environment – Document each component and the monitors required as can be seen in the example table below.
How is the environment sized? – is it representative of production?
Performance testing will normally require an environment that covers the same scope as production, whilst not necessarily having High Availability (HA).
The performance environment will ideally be as close to production as possible.
A performance testing in an environment similar to production is best practice as differences between elements can significantly affect performance. Note it is not always possible to conduct performance testing in a production-like environment.
There are advantages and disadvantages in the sizing for both full scale and scaled-down environments.
A full-scale environment would have the same hardware as production, for instance, the same number of processors, memory and so on.
A scaled-down would be a cut down version.
What resources are required to facilitate and support a performance test environment?
The following table suggests some of these key roles and responsibilities
now would be a good time to refer to our article around Data Considerations For Performance Testing
What data is required within the environment and should cover the following items:
- Database structures
- Uniqueness and other constraints.
- Method of data generation.
- Method of load and restore.
- Refresh requirements.
- Static and Dynamic data.
- Any data protection requirements with the use of live customer datasets.
What date(s) is the environment available for and how will the testing be scheduled? Is it shared?
The schedule should detail not only the time and period required but also the exact requirements of the environment. See the table below for an example.
In conclusion, it is important that the performance test environment items as above, are specified and considered prior to a performance test environment being built.
To see how SQA Consulting may assist your company in performance testing your applications, please contact us.