Regression is 'the phenomenon that the quality of a system as a whole declines as a result of individual adaptations' (TMap Next, 2006). To prevent damage as a result of these software errors, a regression test is performed. The purpose of regression tests is to determine whether all system parts continue to function properly after one or more changes have been implemented. It is also advisable to carry out a regression test after solving open test findings.
In short, a regression test set is a valuable tool to secure the continuity of the IT landscape.
the phenomenon that the quality of a system as a whole declines as a result of individual adaptations
An overview of ten important aspects when building a regression test set:
Lay down in the acceptance criteria of projects that the test ware created in the project is transferred to management after completion. This test ware contains a good basis for the regression test set of the delivered system (component). This saves a lot of time and prevents reinventing the well-known wheel.
The regression test focuses particularly on the relationship between the changed and unchanged parts, as this is where the risk of errors is greatest. Tests in which individual functionality is tested are less relevant.
Limited time and resources are by definition a challenge in any form of testing. That is why it is also important to compile a regression test set based on product risks. The components with the greatest potential damage (and error probability) are examined first and most thoroughly.
Although in projects both good and error paths are tested, it makes sense to limit the regression test entirely to the good paths (also called 'happy flows').
A major challenge in regression testing is keeping the test cases and test scenarios up-to-date. Therefore make sure that the regression test set is set up in a way that it is easy to maintain. For example, in Testersuite test scenarios are compiled on the basis of test cases. This way only the test case needs to be changed and the change is immediately applied to all test scenarios that use this test case.
Although compiling an initial regression test set is a time-consuming challenge, this invested time is recouped as the regression tests reuse this test set. This payback is, by the way, completely separate from the 'earned back' costs by preventing production disruption.
For large releases, the entire regression test set is often performed. However, it is not unusual for small ad hoc changes to cause major regression problems. The urge to solve a production problem quickly is often greater than the awareness to test the solution for regression in a structured way. By building the regression test set in a scalable way, it can also be planned as a subset. In this way, the test set can be used both for ad hoc changes and for planned major maintenance.
The storage of large numbers of test cases and test scenarios is an enormous challenge, especially since in addition to the storage of the tests, the tests performed (including test results) must also be retained. Content management of test cases and test scenarios on a network drive is a full-time job. Take care of overview and version management by central storage, for instance by using a test case management solution like Testersuite.
Who is the owner (read: guardian!) of the regression test set? This should preferably be someone with test expertise who does not work in the project organisation. This person can supervise the use and reuse of the regression test set, as well as taking care of updating the regression test set.
The more often the regression test is performed, the higher the quality and the more detailed the test set. This makes it possible to partially (!) automate tests that are executed repeatedly and do not change often. automate. This results in a considerable saving of time!
With small changes, the testing effort to exclude regression is many times greater than testing the small change itself. For this reason, it is wise to consider implementing changes as a release. A broader regression test can then be planned and the test effort is more in proportion to the number of changes.
Work towards mature regression testing
In addition to the ten principles mentioned above, the most important principle in regression testing is of course the continuity that is guaranteed towards the business (users, customers, suppliers, etc). Work towards a mature regression test in small manageable steps. We are happy to help guide this process or make it easier with our solutions.
