The Kosli CLI provides several attest commands, such as kosli attest snyk, kosli attest jira, etc. These attestations are “typed” - each one knows how to interpret its own particular kind of input. For example, kosli attest snyk interprets the sarif file produced by a snyk container scan to determine the true/false value for that individual attestation.

If you’re using a tool that does not yet have a corresponding kosli attest command then, until now, you’ve had to use the “untyped” kosli attest generic command, which can attest anything, but it cannot calculate a true/false compliance value for you. Generic attestations are a useful “escape hatch”, but have their drawbacks:

• It can take some effort to calculate a true/false value in some cases.
• It would be nice to split generic attestations into different types.
• Most importantly, many customers would prefer it if Kosli calculated all compliance values, as part of a zero trust model.

Based on this feedback we’ve implemented a new attest command called kosli attest custom. In a previous blog post we created a very simple custom attestation type and used it to migrate an existing generic attestation. In this blog post we’ll create a much richer custom attestation, with its own schema, and use it to check some very specific coverage metrics against thresholds.
We’ll:

• Define a custom attestation type (with schema) capturing several metrics with thresholds.
• Gather these metrics in the unit-test run.
• Add steps to our CI workflow to report the result to Kosli using the kosli attest custom CLI command.
• Update the type of the attestation in the Flow’s compliance template yaml file.

Create your own rich custom attestation type

We’ll create a new custom attestation type to check some very specific coverage metrics for the cyber-dojo differ microservice’s unit-test run.

• The type is called coverage-metrics.
• An attestation against this type will be true only if all its rules evaluate to true.
• It has four threshold rules, specified as jq expressions using the --jq flag:
  • .code.lines.total <= 500 to limit the size of our microservice
  • .code.lines.missed == 0 to enforce full statement coverage
  • .code.branches.total <= 75 to limit the complexity of our microservice
  • .code.branches.missed <= 1 to enforce (almost) full branch coverage
• It has a https://json-schema.org/draft/2020-12/schema schema defined in the file metrics-coverage.schema.json specifying the types of the names used in the --jq expressions, whether they are required, whether they have defaults, etc.
• It lives inside a CI workflow in its own git repository

We define this new custom attestation type using the kosli create attestation-type CLI command.

For example:

 kosli create attestation-type coverage-metrics
    --jq ".code.lines.total <= 500"
    --jq ".code.lines.missed == 0"
    --jq ".code.branches.total <= 75"
    --jq ".code.branches.missed <= 1"
    --schema=metrics-coverage.schema.json

Note: the --schema and the --jq rules are both optional. If you create a type with no --schema and no --jq rules then all custom attestations made for that type will have a compliance value of true.

Gather your custom data into a json file

The gathering step is often relatively simple. You may well be doing it already. In our example we:

• added an extra reporter to the coverage library (see example)
• implemented the JSONFormatter to save the required metrics data
  (see example)

When we run our unit-tests, the reporter generates a json file called coverage_metrics.json containing the required metrics values.

For example:

{  "code": {
     "lines": {
       "total": 352,
       "missed": 0
     },
     "branches": {
       "total": 60,
       "missed": 1
     }
  },
  ...
}

Note that the json format of these values exactly matches the --jq rules in our type.
For example:

• one of the --jq rules is .code.lines.total <= 500
• the value of .code.lines.total in the json file is 352
• 352 <= 500 is true
• so when Kosli processes the json file against our coverage-metrics type it will determine this --jq rule is true

Attest the custom result in your CI workflow

In our CI workflow we simply add a step calling the kosli attest custom command:

• The --type flag names the custom type (coverage-metrics)
• The --attestation-data flag names the file containing the generated data (coverage_metrics.json)
• The --name flag names the entry in the Flow’s template yaml file

For example:

on:
  ...

env:
  ...

jobs:
  ...
  unit-tests:
    ...
    steps:
      ...
      - name: Run unit tests
        run:
          make unit_test # Creates ./reports/server/coverage_metrics.json
      ...
      - name: Attest coverage metrics to Kosli
        if: ${{ github.ref == 'refs/heads/main' && (success() || failure()) }}
        run:
          kosli attest custom --type=coverage-metrics
            --attestation-data=./reports/server/coverage_metrics.json
            --name=differ.unit-test-coverage-metrics

The --name of our custom attestation is differ.unit-test-coverage-metrics which we add as a custom:coverage-metrics type to our Flow template yaml file.
For example:

version: 1

trail:
  ...
  artifacts:
    - name: differ
      attestations:
        ...
        - name: unit-test-coverage-metrics
          type: custom:coverage-metrics
        ...

Each kosli attest custom command adds its compliance evidence to a Kosli Trail.

For example:

Custom attestation types are versioned

At the top of the above screenshot you can see the coverage-metrics type used for the shown attestation is v8. This is a link to the UX page listing all the versions for the type. As always, everything in Kosli is append-only. When you create a v9 (by changing the rules or schema) it can never affect any existing attestations or compliance results.

For example

Summary

Kosli’s new custom attestations improve on generic attestations:

• You can define your own custom attestation types. These can be very simple and replace existing generic attestations, or arbitrarily rich, expressing rules with thresholds.
• These custom types can be reused across all Flows in your Kosli Org.
• Kosli can ensure each compliance calculation adheres to a provided schema.
• Most importantly, as part of a zero trust model, Kosli can now calculate the compliance values of all attestations in a Trail.