Test Resources

Testing Environment

The ideal environment for system testing purposes would be an instance of the database with the exact system configuration that will be used for operational purposes post go-live (or at least is as near to it as is known at the time of testing), but this may not always be feasible. It may be that some of the equipment to be used in the future will not have been installed when testing has begun, or the machine resources have to be shared with different systems to those that will be in existence during future live operation, including the old system that is being replaced. Consequently, some compromises may have to be made during system testing, however it is imperative these are documented so those involved in future testing iterations are aware of any factor that might influence results.

For some phases and types of testing, it may not be necessary to replicate the full system configuration (for example, if unit testing one isolated procedure in the Preliminary Phase) but for others, such as Volume/Performance/Capacity testing this is essential if you are to gain accurate results.

Test Data

Ideally, a copy of live system data would be used for system testing purposes, but to arrange for that to happen and to re-load data as each test is completed imposes a great workload on the technical project personnel or IT Services. Therefore, for Unit Testing and Preliminary Testing, it is perfectly acceptable for those conducting the tests to set-up their own artificial data, although this should be as realistic as possible.

However, for UAT and Final Testing purposes, a copy of live data should be taken and loaded into the test environment as often as necessary. This means that the data on which the significantly important tests are carried out will be entirely realistic and meaningful and it may be possible to compare the test results with the live data that still resides on the old system and more confidently reconcile any discrepancies. You should be aware that while this may be possible for some processes - e.g. comparing names on class lists - there are other functions for which it may be virtually impossible to exactly match the output from systems, particularly where calculations are involved - e.g. payroll systems performing complex calculations or systems outputting statistical agency returns. After all, one of the reasons for implementing a new system may well have been to improve calculation accuracy!

Test Scripts

Those staff identified with the role and responsibility should produce Test Scripts in such a form that they can be used by trained users for the UAT phase and signed-off with results recorded. They should reflect the requirements of the Data Input, Data Processing and Data Output Testing aspects of Functionality Testing and should include clear instructions for the users conducting the testing and anticipated results. These scripts can then be used for UAT purposes during future system upgrades or re-implementations.

Testing Tools

There are a number of testing tools available on the market to assist the various system testing activities. They are designed to generate large volumes of data to simulate input to the system by a large number of users and measure machine performance during simulated live operation. These are particularly useful for Volume and Performance testing, but they are costly and often difficult to set-up to match the specific sequences and formats of data to be input to the organisation's software or the precise characteristics of its system.

Test Documentation

The details of the plan for system testing, all of the tests to be used and their results should be fully documented to enable a comprehensive review of the testing procedures in the event of a major failure, investigation into and resolution of any particular issue and repeat testing in the future for system upgrades or re-implementations.

Testing Issue Resolution

The decision making framework developed and approved by the organisation's principal officers during the implementation planning stage will be particularly useful during the system testing phase to ensure that issues raised during testing are resolved in a swift and efficient manner or agreed to be resolved later.

All unresolved software errors and issues surrounding poor performance and volumes that can be handled by the new system should be recorded in detail on a centrally controlled list of issues, which can then be used to track progress towards resolution of each of the issues and to make a judgment regarding the readiness of the system to go-live. All changes made to the software, business processes or hardware configurations to resolve errors or issues during the UAT activities should be subject to the Change Control procedures described in this infoKit.