Why regression testing in SAP is so important to your continuous delivery strategy

Plenty of us read articles online to keep up to date with the latest industry trends, as I often do, but nothing beats getting information straight from the horse’s mouth.

So one of my resolutions this year was to spend a lot more time meeting and talking with my customers so I could get some first-hand feedback on what’s really causing them pain at the moment.

It’s all about transformation

In the enterprise application world of SAP, there are some recurring topics that have been raised by pretty much everyone I’ve spoken to:

• We’re moving to cloud-based systems but we’re worried about security, uptime, and performance
• We’re adopting SAP’s in-memory HANA database but need to trim down to make it cost effective
• We’re struggling to make a business case for S/4HANA and then face the monumental task of actually doing the transition
• We need to move faster as we respond to digital challenges and initiatives
• We need to tie multi-platform applications together so systems of engagement run in harmony with back-end ERP

So it's all about transformation in one way or another - whether it be platform, applications, processes or even a combination of them all - and the need to be on top of all this is causing some major headaches.

One thing that’s clear is that the ability to be more agile is key as more and more companies are adopting continuous delivery to address some of these challenges.

The monolithic release cycles of the past are out.  In comes flexible, dynamic release strategies that enable delivery on-demand.

How do we keep up with the testing?

If we agree that speed and flexibility are essential in enabling transformation, how do we make sure that we’re not going to break stuff when we put change live, causing system downtime or business interruption?

When we release in big batches we expect to spend a good chunk of time doing regression testing. But that’s the problem: it takes time! This is because in many cases the testing is done manually, but even those that do automate this process have limited coverage - below 30% is far from unusual. To get around this, companies often only test what they think will be impacted or worse, they avoid regression testing all together, resulting in costly deployment issues.

If we’re now aiming to be more agile, and going to release more regularly or want to validate that applications still work on new platforms, how do we run those regression processes more quickly and frequently?

The commonly-used risk-based testing approach, where you define a limited regression test scope within achievable limits, is simply not going to cut it as the chances that something critical will be missed are too high.

So we need to be able to run full regression test cycles before we release anything to the business.  For that, we need an effective automation solution which…

1. Allows us to validate the to-be state with confidence
2. Provides as close to 100% coverage of day-to-day business processes as possible
3. Keeps itself up to date with a constant rate of change so it’s ready for the next drop

One of the customers I spoke to has implemented just that.

They have robust and automated controls over the delivery of change but they now also have fully automated regression testing with coverage exceeding 90%.

Pretty impressive.

Before using this automation technology, they used to have 20-30 critical issues per release.  Now they have virtually zero and the occasional ones that do get through are far easier to manage because there’s so much less noise.

So, when looking to adopt more agile application release processes for continuous delivery, don't underestimate the challenge that testing poses.  In SAP, where delivery processes have often lagged behind those seen in the wider application world, near to full test coverage is now possible through new revolutionary technology, so there’s no excuse not to deliver change and transformation at the pace your business demands.

If you want to find out more about how you can increase your regression test coverage to meet your continuous delivery goals, please get in touch for a chat. You can also find more information about Testimony, the solution that uses Robotic Test Automation technology, here.