Kosli - What the FCA found when analyzing 1 million production changes

What the FCA found when analyzing 1 million production changes

Bruce Johnston
Bruce Johnston
Published March 24, 2021 in news

A recent FCA report shows that the financial services industry needs to reimagine its approach to change management. By analyzing data from over 1 million production changes, they found out what works and what doesn’t work in the land of regulated change. Let’s dig in…🕵️‍♀️

On the 5th of February the Financial Conduct Authority (FCA) published its Implementing Technology Change report. It focuses on the way financial firms manage technology changes and the impact of failures.

And something leaps right off the early pages.

While most financial institutions award themselves a mature rating for change management capabilities, change management failures are the most common cause of incidents reported to the FCA.

In other words, the industry thinks it has change management licked when the exact opposite is true. 🤯

What is happening?

The reason for this contradiction is the misplaced trust the industry has in Change Advisory Boards (CABs). Institutions throw a ton of time and money at these boards whose job it is to approve changes. Here’s what the FCA has to say about them:

“One of the key assurance controls firms used when implementing major changes was the Change Advisory Board (CAB). However, we found that CABs approved over 90% of the major changes they reviewed, and in some firms the CAB had not rejected a single change during 2019. This raises questions over the effectiveness of CABs as an assurance mechanism.”

signed off in the cab ops problem now meme Kosli

When the regulator reaches for diplomatic phrasing like “raises questions” you can tell they’re a bit flustered. And that’s because CABs are an exercise in risk theatre designed to create the illusion of effective change management. It’s a powerful illusion because even the institutions believe in it.

This is an important finding: adherence to traditional change management processes doesn’t work to manage the risk of changes.

The science of DevOps backs this up. Here’s the unvarnished truth on external approvals and CABs based on research by Dr. Nicole Forsgren, Jez Humble, and Gene Kim in their 2018 book, Accelerate: Building and Scaling High Performing Technology Organizations.

“We found that external approvals were negatively correlated with lead time, deployment frequency, and restore time, and had no correlation with change fail rate. In short, approval by an external body (such as a change manager or CAB) simply doesn’t work to increase the stability of production systems, measured by the time to restore service and change fail rate. However, it certainly slows things down. It is, in fact, worse than having no change approval process at all.”

Worse than no change approval process at all.

So what does work then?

The FCA identified several practices that contributed to change success. Unsurprisingly, having well defined processes and a majority of the IT budget on delivering change will help.

But, in terms of actually delivering the software, they also found this:

“Frequent releases and agile delivery can help firms to reduce the likelihood and impact of change related incidents:

Overall, we found that firms that deployed smaller, more frequent releases had higher change success rates than those with longer release cycles. Firms that made effective use of agile delivery methodologies were also less likely to experience a change incident.”

The practices described here are the foundation of DevOps - frequent deployments, agile development, defined processes, and change as the normal way of working. So, if you’re already practicing DevOps, you’re well on the way to finding a better way to manage change. If you’re not - you should probably start. 😇

im listening meme Kosli

Winner winner chicken dinner

The thread that runs through all of Dr. Forsgren’s work is that DevOps maturity correlates very, very closely with business performance. If you’re going to corner your market and outpace the competition you need a strong DevOps game.

Some industry leaders have understood the importance of technological performance for quite a while. In 2014, Richard Fairbank, Capital One CEO, said “ultimately, the winners in banking will have the capabilities of a world class software company.” He might not have known it at the time, but he was describing DevOps organizations.

Being a mature DevOps outfit is especially effective in regulated verticals because you’ve got this annoying change management stuff that no one really wants to deal with. With DevOps you can automate your change and release control in the pipelines.

Imagine that. Compliant software on demand. And no CABs either. This is what the real winners in fintech and other regulated spaces will look like.

Now, can someone call a taxi for the CAB please? Peep peep peep! 🚕

Kosli yellow taxi


About this article

Published March 24, 2021 in news

About the author

Bruce Johnston

Bruce Johnston


ABOUT THIS ARTICLE

Published March 24, 2021, in news

AUTHOR
Bruce Johnston
Bruce Johnston
Live in Git Blame? Don’t spend hours searching for the change that broke your application! Query, search and discover all the changes in one place

Latest articles

Kosli announces Innovation Partnership with DNB and Firi

We are pleased to announce that Innovasjon Norge has awarded Kosli an innovation grant of 3.4 million NOK to pursue a R&D project with DNB and Firi. In this blog we’ll give you an overview of the …

The Ultimate Guide to git blame: A How To with Examples

Source control tools give users many powers and one of the big ones is traceability. With traceability tools you can know exactly who made each change and when they made it. In Git, you use the git …

Git Blame in VS Code: The 4 Best Options

Most production projects have a team collaborating on them, so even in a single file there can be multiple contributors. When things go wrong, it’s useful to understand how and why certain changes …

Sign up to our newsletter

We'll let you know about the Kosli launch, product news, features and updates
Kosli is committed to protecting and respecting your privacy. By submitting this newsletter request, I consent to Kosli sending me marketing communications via email. I may opt out at any time. For information about our privacy practices, please visit Kosli's privacy policy.
Kosli team reading the newsletter

Let’s chat!

Got a question about Kosli? An idea for a new feature? Join Kosli Slack and talk to us.

Join
Developers using Kosli