pint

pint is a Prometheus rule linter/validator.

Requirements

pint will run checks on Prometheus alerting & recording rules to detect potential problems with those rules. Some checks rely only on the rule itself and can be run “offline” - without talking to any Prometheus server. You can run pint in “offline” if you:

Don’t pass any configuration file to pint.
You pass configuration file to pint that doesn’t contain any prometheus definition.
You pass --offline flag to pint command.

Most checks included in pint will require sending queries to a running Prometheus server where those rules are, or would be, deployed. Those checks are enabled if you pass a configuration file to pint that includes at least one prometheus block. Checks might use various Prometheus HTTP API endpoints to retrieve extra information, for example Prometheus configuration or metrics metadata. If you run pint against a different service, like Thanos some checks might return problems due to API call errors, since not all Prometheus HTTP APIs are supported by it. In that case you might want to disable failing checks in pint configuration file.

Usage

There are three modes it works in:

CI PR linting
Ad-hoc linting of a selected files or directories
A daemon that continuously checks selected files or directories and expose metrics describing all discovered problems.

Pull Requests

Run it with pint ci. Git is currently the only supported VCS.

When pint ci runs it will find all files in the current working directory and try to parse them as Prometheus rules. Then it will look for all commits on the current branch that are not present in the parent branch and to decide which rules were modified. Checks are run only on modified rules but they require the full list of all rules to find any cross-rule dependencies.

Running pint ci doesn’t require any configuration but it’s recommended to add a pint config file with ci section containing at least the include option. This will ensure that pint validates only Prometheus rules and ignores other files.

Results can optionally be reported using BitBucket API or GitHub API to generate a report with any found issues.

Exit code will be one (1) if any issues were detected with severity Bug or higher. This permits running pint in your CI system whilst at the same you will get detailed reports on your source control system.

If any commit on the PR contains [skip ci] or [no ci] somewhere in the commit message then pint will skip running all checks.

GitHub Actions

The easiest way of using pint with GitHub Actions is by using prymitive/pint-action. Here’s an example workflow:

name: pint

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  pint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run pint
        uses: prymitive/pint-action@v1
        with:
          token: ${{ github.token }}
          # directory containing Prometheus rules
          workdir: 'rules'

To customise pint checks create a .pint.hcl file in the root of your repository. See Configuration for a description of all options.

If your repository contains other files, not only Prometheus rules, then tell pint to only check selected paths when running checks on a pull request:

ci {
  include    = [ "rules/dev/.*.yml", "rules/prod/.*" ]
}

When pint runs checks after a push to a branch (for example after a merge), then it will pass workdir option to pint lint, which means that all files inside rules directory will be checked.

Ad-hoc

Lint specified files and report any found issue.

You can lint selected files:

pint lint rules.yml

or directories:

pint lint path/to/dir

or both:

pint lint path/to/dir file.yml path/file.yml path/dir

Watch mode

Run pint as a daemon in watch mode:

pint watch rules.yml

By default it will start a HTTP server on port 8080 and run all checks every 10 minutes. This can be customised by passing extra flags to the watch command. Run pint watch -h to see all available flags.

Query /metrics to see all expose metrics, example with default flags:

curl -s http://localhost:8080/metrics

Or setup Prometheus scrape job:

scrape_configs:
  - job_name: pint
    static_configs:
      - targets: ['localhost:8080']

Available metrics:

pint_problem - exported for every problem detected by pint. To avoid exposing too many metrics at once pass --max-problems flag to watch command. When this flag is set pint will expose only up to --max-problems value number of pint_problem metrics.
pint_problems - this metric is the total number of all problems detected by pint, including those not exported due to the --max-problems flag.

pint problem metric can include owner label for each rule. This is useful to route alerts based on metrics to the right team. To set a rule owner add a # pint file/owner $owner comment in a file, to set an owner for all rules in that file. You can also set an owner per rule, by adding # pint rule/owner $owner comment around given rule.

Example:

# pint file/owner bob

- alert: ...
  expr: ...

# pint rule/owner alice
- alert: ...
  expr: ...

Here’s an example alert you can use for problems detected by pint:

- alert: Pint Problem Detected
  # pint_problem is only present if pint detects any problems
  # pint disable promql/series(pint_problem)
  expr: |
    sum without(instance, problem) (pint_problem) > 0
  for: 1h
  annotations:
    summary: |
      {{ with printf "pint_problem{filename='%s', name='%s', reporter='%s'}" .Labels.filename .Labels.name .Labels.reporter | query }}
        {{ . | first | label "problem" }}
      {{ end }}
    docs: "https://cloudflare.github.io/pint/checks/{{ $labels.reporter }}.html"

Control comments

There is a number of comments you can add to your rule files in order to change pint behaviour, some of them allow you to exclude selected files or line, see docs here for details.

There are a few requirement for any comment to be recognized by pint:

All comments must have a pint prefix.
All comments must have at least one space between # symbol and pint prefix.

Good comment examples:

# pint file/owner bob
#   pint file/owner bob

Bad comment examples:

# file/owner bob
#pint file/owner bob

Release Notes

See changelog for history of changes.

Quick start

Requirements:

Git
Go - current stable release

Steps:

Download a binary from Releases page or build from source:

git clone https://github.com/cloudflare/pint.git
cd pint
make

Run a simple syntax check on Prometheus alerting or recording rules file(s).
```
./pint lint /etc/prometheus/*.rules.yml
```
Configuration file is optional, but without it pint will only run very basic syntax checks. See configuration for details on config syntax. By default pint will try to load configuration from .pint.hcl, you can specify a different path using --config flag:
```
./pint --config /etc/pint.hcl lint /etc/prometheus/rules/*.yml
```

There are docker images available on GitHub. Example usage:

docker run --mount=type=bind,source="$(pwd)",target=/rules,readonly ghcr.io/cloudflare/pint pint lint /rules

License

Copyright (c) 2021-2023 Cloudflare, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    
    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.