“You think you’re testing… until you actually have to test”
From Excel to Real Control Over Testing – A Conversation with Kevin Regts
Testing. Everyone does it. But few organizations have it truly well organized. In many organizations, it starts small: an Excel file, a checklist, a few people who “just take a quick look.”
Until things get tense. A major release. An ERP upgrade. Or a system that has become business-critical. And suddenly it turns out:
- You don't have an overview;
- You don't know what was tested;
- and you have no traction.
Sound familiar? In this episode of “Let’s Talk About Testing,” we speak with Kevin Regts, a test manager at BMN. Without a background in testing, he worked his way up to test manager—and was tasked with professionalizing the testing process for a complex ERP upgrade:
- From Excel to a structured format;
- From random checks to monitoring;
- From intuition to insight.
BMN is one of the largest building materials wholesalers in the Netherlands, with a large number of locations and customers. As a result, an ERP upgrade has a direct impact on daily operations.
Who is Kevin?
Let's start at the beginning. Who is Kevin Regts?
“My name is Kevin. I’m 30 years old, live in Heerenveen, and have been working at BMN for about six years now. I started out on the business side and later moved up to become a regional mentor—the link between business and IT.”
And outside of work?
“I play soccer and I bike a lot. I really enjoy road cycling, especially in the summer.”
Was becoming a test manager a conscious decision?
“No, not at all. I studied business economics, so it wasn’t covered in my program. And it wasn’t part of my work at first, either. Of course, you work with systems, but how does testing actually work? I wasn’t really thinking about that question.”
“I’d never really given software testing much thought. It’s not until you’re right in the thick of it that you realize just how much is involved.”
An ERP Upgrade: Why Testing Became Crucial
When did testing suddenly become a serious matter for you at BMN?
“That was during the upgrade of our ERP system. We use Microsoft Dynamics Business Central, but we were running an outdated version that was no longer supported.”
So there was no choice?
“No. We had to upgrade. But that did mean that everything under the hood changed.”
And that means testing, too?
“Yes. In that case, it’s no longer enough to just ‘check it quickly.’ You have to be absolutely sure that everything works—from start to finish.”
What made that so complicated?
“Our processes are intertwined. Any change you make in one process affects the other. So you need to test the entire chain, not just individual steps.”
Unstructured Testing: Why Excel No Longer Works with ERP Systems
How prepared was your organization for test cycle like this?
“Honestly? Not really.”
What do you mean?
“The knowledge was there. People knew how the processes worked. But it wasn’t documented in any central location.”
So, no standard testing approach?
“No. Everyone did it their own way—using Excel, Word, or just from memory. Everyone knew how it worked, but no one had actually documented it. That only became a problem when we had to retest everything.”
When Excel Is No Longer Enough
What specific issues did you run into?
“Lack of an overview. Especially when you have multiple rounds of testing and many stakeholders.”
What kinds of tests did you take?
“Keep in mind:
- user acceptance testing (UAT);
- cross-departmental chain testing;
- regression testing for new iterations;
- "and later also test it during the hypercare phase."
Didn't that fit in Excel anymore?
“No. Then you lose track of the big picture and don’t know where you stand anymore. That makes it difficult to report to management.”
“When it became clear that we weren’t going to be able to manage this with Excel, we realized we needed a structured approach.”
The Choice of Structure
When did you decide to do things differently?
“When it became clear that we weren’t going to be able to manage this with Excel, we realized we needed a structured approach.”
And then came Testersuite?
“Yes. That gave us one central place for everything related to testing.”
How BMN Testersuite for UAT, End-to-End Testing, and Regression Testing
What Testersuite do you use Testersuite for?
“For the entire testing process related to the ERP upgrade. That includes:
- documenting processes;
- creating test cases;
- conducting UAT;
- chain testing;
- regression tests per test round;
- "and later also in the hypercare unit to defects ."
"So, from start to finish. It gives us clarity: these are my processes, this is what I need to test, and this is the status. We didn't have that insight before."
Testing with 150+ testers
How did you actually go about organizing that?
“We deliberately started small. First, we wanted to get the setup right, then scale up. At our peak, we had about 150 to 200 testers.”
Are those testing professionals?
“No, mainly key users and process owners.”
Why did that work?
“Because it was concrete. People didn’t have to figure out what to do. It was all clearly laid out in Testersuite. The barrier to entry needs to be as low as possible. People don’t need to understand the tool—they just need to know what to test. That’s exactly how it works in Testersuite
"Despite the complexity and the hectic pace of the business, we made it. For me, that's the biggest milestone."
Result: A Successful Business Central Go-Live Thanks to Structured Testing
Looking back, what is the most important milestone you've achieved?
“That we went live with a fully functional system within the set timeframe.”
That sounds logical, but what makes it special?
“It was the combination. Business just kept going as usual. People were doing this on the side. And yet we still had everything under control.”
So, no chaos leading up to the launch?
“No. We had a clear overview, we were able to steer the process, and we knew where the risks lay. Despite the complexity and the hectic pace of the business, we pulled it off. To me, that’s the biggest milestone.”
As an added bonus, you now have a clear overview of the processes
“That’s right. The great thing is that the testing processes are now documented—in a structured way and easy to find in a handy tool. Every Testersuite can now access a clear and precise description of the processes.”
Lessons Learned from ERP Testing: From Happy Flow to Full Coverage
What are your most important lessons learned?
“Testing is much more complex than you think.”
You hear that a lot—but what exactly do you mean?
“You shouldn’t just test the ‘happy flow.’ It’s the exceptions that matter most. It’s also about:
- that you need to have your basics in order;
- that you need to document your processes;
- that you defects organize defects ;
- "and that regression testing is essential."
Why are regression tests essential for you?
“Because every change can affect something else. You often don’t realize that until after the fact. You think you’ve tested everything—until something changes and something else breaks. That’s when you realize: testing never ends.”
From defect Insight
What has changed in the way you handle defects?
“It suddenly became a structured process. Not just recording, but also:”
- categorize;
- prioritize;
- "analyze."
“Because this became much more robust, we were much better able to provide management with a well-founded picture of where we stood. That brought a lot of peace of mind.”
From testing during the project phase to testing during the operations phase.
Was Testersuite used Testersuite for projects?
“No. That’s when it gets interesting.”
What do you mean?
“In the maintenance phase, we Testersuite use Testersuite for releases and regression testing. The project phase provided us with a comprehensive test suite. And we continue to use and expand this test suite during the maintenance phase.”
So it's actually becoming more and more valuable?
“Yes. Because you keep building up your knowledge.”
"What's funny is that, at a certain point, I Testersuite my way around Testersuite better than Testersuite in our ERP system."
What value does Testersuite bring Testersuite BMN?
What specific benefits has Testersuite provided Testersuite with?
“Time, clarity, and control.”
Could you be more specific about that?
“Whereas it might take you a week to set everything up in Excel, we were able to do it in half a day—for 150 testers. What would take a week without a structure is now ready in half a day. That makes all the difference.”
“What’s funny is that at some point, I actually Testersuite my way around Testersuite better than I did our ERP system, hahaha.”
Optimizations for Testersuite
If you take a critical look at Testersuite where can we still improve?
“Especially the little things that make it easy to use.”
Like what?
“For example, navigating more quickly between defects test cases. Things like that.”
So, no fundamental changes?
“No. The basics are right. It’s more about optimization.”
Given what you know now, what would you do differently?
If you had to start this project over, what would you do differently, given your current knowledge and experience?
“More focus on un-happy flows.”
Why there, of all places?
“Because that’s where the real risks lie. Things that don’t happen often, but have a major impact.”
Advice from Kevin: How to Build a Solid Testing Process
Finally, what is your most important piece of advice for new test managers?
“Make sure you have a solid foundation.”
And what does that mean?
“Just to be clear:
- what you're testing;
- when you take the test;
- how to test;
- "and how to defects ."
That sounds simple.
“That’s exactly it. But it’s often overlooked. Without a solid foundation, you won’t have a firm grasp—no matter what your tools are.”
