Why developers need a DevOps database

Mike Long
Mike Long
Published June 17, 2022 in news, features

Can you imagine developing software without version control? What if I told you that we were doing exactly the same thing with DevOps?

In this article I’ll explain why developers need a database for DevOps, and make the case that it should have similar semantics to a version control system.

Can you imagine life without version control?

Life without software version control would really suck. We’d lose all the history of the changes made to our software.

We wouldn’t know who’d changed our software or why they’d made that change. Comparing any two versions of our software would be a huge pain.

We wouldn’t have an easy way to share changes either. These days we take git push for granted, but there was a time when we would have to share zip files or copy to network drives.

And how would we label these versions? We’d no doubt use some clunky method like software_project_v1.latest.final.revision3.zip.

But what about Version Control for DevOps?

Today, all of the interesting information about our DevOps disappears instantly. Not only are we forgetting every event that happens, we have no means to navigate and understand changes as data. Our DevOps is in a state of amnesia.

Think of all the rich event data we throw away in our automation. The build binary provenance, the test results, the security scanning, and the deployment activities just vanish.

Understanding and navigating all this change requires the detective instincts of Sherlock Holmes with the technical expertise of Grace Hopper.

But what if we could preserve this data and query it?

Kosli

Imagine…

A way of understanding everything in production that is as easy as git status The ability to navigate the history of an environment that is as direct as git log. The power to diff two environments that is as simple as git diff.

$ 

 

Now imagine the sort of higher level tooling you could build on top of this memory. You could data mine your own DevOps, gather DORA metrics, create dashboards - who knows what else?

Designing a DevOps Version Control system

Ok, but what would a history of DevOps events in a version control system look like? Maybe we can learn a thing or two from git?

Git wasn’t the first version control system, but it popularized some very important features. It introduced a snapshot-based, content-addressable storage identity, Merkel trees to ensure a secure history, and powerful collaboration modes. What if these features were available for DevOps? What would we want it to look like?

  • Ideally we’d want to compare any runtime assets. We’d be able to understand all the environment changes - CDN’s, lambda functions, kubernetes containers, and configuration changes. Nothing should be out-of-band.
  • Our data would be content addressable. We should be able to identify things based on secure and tamper-proof identity.
  • We’d want to know that the change log is provable
  • We’d want it to come with a powerful command line interface

Can we do it with git?

Git is a great version control system for source code, but its strengths in source control make for weaknesses in devops:

  • It is great for recording changes in our software inputs, but not for storing outputs
  • Working with additional repos in our pipelines is super clunky. In our devops pipelines and our shell we want to use a command line client.
  • Git’s whole content system is based on files. This is too low level, doesn’t model the domain, and is hard to query.
  • It’s distributed. This is an advantage in a source control system, where you can craft changes locally decoupled from sharing them globally. However, in a system for recording devops events we want to commit to the global stream immediately.
  • It doesn’t have a developer experience and frontends to support these use cases
  • It doesn’t scale to multiple teams and services

Doesn't GitOps solve this?

GitOps has been extremely important in moving from imperative to declarative definitions of deployments and infrastructure. This helps, but only for recording desired states for certain cloud-native systems. GitOps still misses how and when these desired states get applied, and their connection through the DevOps lifecycle.

Kosli is The VCS for DevOps

We didn’t set out to build a VCS for DevOps, but it’s what we’ve ended up with. It took us some time to realize that Kosli actually gives every developer the power to commit, diff, log, and status every aspect of change in their DevOps automation. The data can be gathered automatically, and be navigated from the command line, slack, and in the web browser.

How does Kosli work?

The first step is to start monitoring your production systems. This can be done with a cron job in CI, or in your favorite workload scheduler. Once this is set up, Kosli records every change to your environment.

Environment recording

The next step is to log important pipeline events that you would like to capture, such as builds, code reviews, test, security scans, and deployments.

Pipeline events

What questions can you ask Kosli?

A version control system for DevOps is a powerful resource for developers, operators, auditors, and security teams who want to understand what’s happening in their pipelines and environments. With it, they can get quick and easy answers to all kinds of difficult questions

  • Deployment progress: what commits are in each environment?
  • Service Catalog: who owns this service?
  • Time Machine: what was in production on Thursday night?
  • Change Log: when did this service scale up?
  • Admission Controller: is this workload approved for this environment?
  • SBOM: What are the ingredients to this artifact?
  • Metrics: What is the delivery lead time across pipelines?
  • Compliance: What systems don’t have code review in place?

Free Tier Launching in summer

We are launching the beta of our free tier this summer. If you are interested in trying it out you can sign up for the waitlist here 👇👇

Sign up here

Pssst: we are hiring!


About this article

Published June 17, 2022 in news, features

About the author

Mike Long

Mike Long


More posts in news

Merkely is now Kosli!

Merkely is now Kosli! Hello! We would like to announce that we have renamed and rebranded as Kosli. A lot has happened over the last six months and it’s time for a new sign above the door and a …

Kosli 2021 - Making friends with change

A lot can happen in a year, and 2021 was no different. We want to help everyone make friends with change and in 2021 we made a few changes ourselves. Hit play on Eye of the Tiger 🐅 and roll the …

We're heading to DevOps Con Berlin!

Later this month Mike Long, our CEO, will give a talk on DevOps and Change Management at DevOps Con in Berlin. Here’s a little taste of what you can expect on June 14th. Is it possible to do DevOps in …

Get exclusive early access for FREE by signing up to our newsletter

We'll let you know about the Kosli launch, product news, features and updates
Kosli is committed to protecting and respecting your privacy. By submitting this newsletter request, I consent to Kosli sending me marketing communications via email. I may opt out at any time. For information about our privacy practices, please visit Kosli's privacy policy.
Kosli team reading the newsletter

Let’s chat!

Got a question about Kosli? An idea for a new feature? Join Kosli Slack and talk to us.

Join
Developers using Kosli