Do you know the difference between trying and testing? Rest assured, we are not going to have a semantic discussion here. But what we do want to say is that we see roughly two flavours when it comes to application testing. Namely, companies that try out applications and companies that test applications. And there is a world of difference!
Disturbing
It is actually serious to have to conclude that many organisations in our heavily automated society do not test software but try it out. Even more serious is the fact that higher management often has no idea about application testing. It is seen as a nuisance or too expensive. Only when something goes wrong are critical questions asked.
Testing applications requires ownership, process descriptions, planning & coordination, test knowledge, time, discipline and professional tooling. However, as soon as there is incidental testing of (new) software, reporting in spreadsheets and a lack of will and execution, there is talk of (trying out) applications.
Funnily enough, (trying out) applications still appears to be mostly the practice. We find this disturbing. The Testersuite team encounters this situation on a daily basis. And certainly not at the SME around the corner. On the contrary.
Even more worrying
A phenomenon that is even more disturbing is the sharing of test results. Here we see two or more organisations using the same application testing releases simultaneously. This encourages false security. How can you be sure that a module relevant to you has been properly tested by a fellow company? OK, you use the same modules of an application, but is the configuration exactly the same? Are the quality requirements on the same level? Are the testers of the calibre that you have trained yourself? And we can think of a few more questions like that.
Suppose that company A,B and C are testing the same release of an application at the same time. The test results are shared during test cycle. When estimating the risks, the focus is largely determined by the test results of fellow companies that are also testing. Modifications that have already been tested by fellow companies and that have not produced any bugs automatically receive less focus. This is where the risk of false security comes into play, which can lead to an incorrect focus. It may well be that the design of the application is different or that it has not been properly tested by fellow companies. A modification receives insufficient attention in test cycle and problems arise in production. Take it from us that we know of many practical examples where this has gone badly wrong.
Sensible sharing of content
We too see in practice that it can make sense to share content (e.g. test scenarios) and test results with fellow companies testing the same applications. Testersuite also provides functionality to do this. Just don't overestimate the added value. Every company will still have to do its own risk assessment and test modifications from a release. Watch out for the risk of false security!
Do not think lightly of application testing
Our message is 'don't take testing lightly'. Testing applications is a profession and not a matter of trial and error. Something they discovered in the aviation industry a long time ago. Fortunately, we speak to many test coordinators and test managers who know what they are talking about. We see their daily struggle against higher management. We would like to give them a shot in the arm.
