Kosli devops change management compliance system of record

How to design a DevOps Compliance System of Record

Mike Long
Mike Long
Published February 1, 2022 in technology

If you deliver software in a regulated industry you have to be able to show that you are following a defined process. And that means being able to produce a record of what’s going on in your DevOps workflows.

When we have conversations about DevOps compliance with regulated software teams this topic frequently comes up. And what these teams require is best described by Carl Nygard as a Compliance System of Record (CSoR).

But, how should you design such a record? What properties should it have? What is the best way of collecting and storing the necessary information?

Whether you build your own, recruit an existing internal system, or choose a DevOps Change Management system like Kosli, we believe there are some important implementation choices you can make to ensure success.

In this article we’ll set out the 5 essential properties needed for a DevOps Compliance System of Record.

Compliant System of Records Carl Nygard Kosli

1. Content addressable identities

All decisions should be based on secure and provable identities defined by the contents of those systems. This is the foundation of Artifact Binary Provenance and there’s lots of ways to do it. In Kosli, our approach is to use the SHA256 sum of any binaries that we care to track.

This provides the assurance that what we qualify is what we deploy, and we can easily track software regardless of labels and metadata.

2. Secure, Append-only/Immutable data

It should not be possible to modify any data. From a compliance perspective we have to be able to trust that the record of decisions, and the data they were made on, haven’t been changed.

That isn’t to say we shouldn’t be able to update information. But, if we do, it should be done in a fully traceable and non-destructive manner. In other words, the data should be append-only.

This is actually quite hard to implement in most databases due to their fundamental design around CRUD transactions. There are ways to do this at the application level, but this requires additional guarantees that data cannot be updated “outside” of the application.

One way we have solved this problem is by using git as the data repository with no-force-push remotes to ensure no rewrites/rebasing. Alternatively, Kosli offers no write “access” to the backing database for customers. Either way, using append-only data structures and merkle-trees is a big step towards ensuring immutability.

3. Pipeline-friendly

A CSoR is essential for audit purposes, but we should never forget the most frequent user: the DevOps pipeline.

Now, what makes a tool easy to use from a pipeline? 🤔

  • Small and efficient - no one wants a slow pipeline
  • Trivial to store and query data
  • Context aware - it should be easy to control via CI environment variables
  • Circuit breaker - it should be easy to move into “pass-through” mode in emergencies

In Kosli, we have gone through several iterations of client tooling. The very first was based on curl commands directly to our APIs. We then created a python client that was a big improvement in terms of usability. Our latest client is written in Golang which makes it very fast to use in pipelines with distributions for all the major platforms and architectures.

4. Service-friendly

An easily overlooked use case for DevOps compliance is self-notification events. It should be simple for an application to make attestations or queries on its own state.

Why would you need this?

Well, many compliance and security events occur outside of a DevOps pipeline. Monitoring systems might want to notify deployments, platform provisioning tools might need to create new assets to track, and sensitive services might want to record audit logs.

The easiest way to implement this is to expose a REST-style API to your Compliance System of Record.

5. Easy to navigate

The final consideration is also the most important - humans. How do we make it easy to reason about high levels of change in modern distributed architectures?

  • What’s in production?
  • What are the recent changes to staging?
  • Who approved this deployment?
  • When was this artifact in production?

Designing a user interface which meets the needs of developers, compliance officers, security personnel, and auditors isn’t easy, but it is extremely valuable.

Whether you build your own solution, or adopt a tool like Kosli, these 5 elements will ensure a CSoR that will satisfy the demands of any regulator or regulatory standard.

How to design a DevOps Compliance System of Record list Kosli


About this article

Published February 1, 2022 in technology

About the author

Mike Long

Mike Long


ABOUT THIS ARTICLE

Published February 1, 2022, in technology

AUTHOR
Mike Long
Mike Long
Live in Git Blame? Don’t spend hours searching for the change that broke your application! Query, search and discover all the changes in one place

Latest articles

Kosli announces Innovation Partnership with DNB and Firi

We are pleased to announce that Innovasjon Norge has awarded Kosli an innovation grant of 3.4 million NOK to pursue a R&D project with DNB and Firi. In this blog we’ll give you an overview of the …

The Ultimate Guide to git blame: A How To with Examples

Source control tools give users many powers and one of the big ones is traceability. With traceability tools you can know exactly who made each change and when they made it. In Git, you use the git …

Git Blame in VS Code: The 4 Best Options

Most production projects have a team collaborating on them, so even in a single file there can be multiple contributors. When things go wrong, it’s useful to understand how and why certain changes …

Sign up to our newsletter

We'll let you know about the Kosli launch, product news, features and updates
Kosli is committed to protecting and respecting your privacy. By submitting this newsletter request, I consent to Kosli sending me marketing communications via email. I may opt out at any time. For information about our privacy practices, please visit Kosli's privacy policy.
Kosli team reading the newsletter

Let’s chat!

Got a question about Kosli? An idea for a new feature? Join Kosli Slack and talk to us.

Join
Developers using Kosli