sklp ci — pipeline reference

Configuration for `sklp run ci`, the lint + test runner that gates Pull Requests. Diff-aware step gating via scopes, deterministic tool installs, and live service dependencies (Postgres, NATS, …) materialised as sidecars in K8s or local processes/containers on a laptop.

For AI coding agents (Claude Code, Codex, Cursor)

This page is your single source of truth for authoring or editing a .sklp/pipelines/ci.yaml. The same kind: pipeline schema is used by _base.yaml (shared header) and any other task file that inherits from it (extends: _base).

Hard rules: every value in step.scope must be a key in pipeline.scopes. Every value in step.services must be a key in pipeline.services. A service is either run: (shell) or image: (OCI container), never both — same rule as devstack. oneshot: true cannot have ready:. Never invent ports — read the source or ask the user.

Mental model

A CI pipeline is a list of steps. Each step has a shell run: command and an optional scope: (which decides when it runs) and services: (which decides what infra is live during it).

• Scopes are named glob lists declared once in _base.yaml. A step's scope: [a, b] means "run me when at least one changed file matches scope a OR scope b".
• Services are long-running dependencies declared in a top-level services: map. A step's services: [postgres] means "make sure postgres is up and Ready before my run: starts".

The same YAML works in two execution contexts:

The runner detects K8s via the SKLP_RUNNER_MODE=k8s env var injected by the spawner. You don't change the YAML.

Quick start

# .sklp/pipelines/_base.yaml
version: 1

tools:
  go: "1.25.0"
  node: "22.19.0"
  pnpm: "10.27.0"

scopes:
  backend:
    - "apps/core/**"
    - "go.work"
  frontend:
    - "apps/web/**"
    - "pnpm-lock.yaml"
  shared:
    - ".sklp/**"
    - "Makefile"

# .sklp/pipelines/ci.yaml
version: 1
kind: pipeline
extends: _base
name: ci

services:
  postgres:
    image: postgres:16-alpine
    ports: ["5432:5432"]
    env:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: test
    ready:
      tcp: localhost:5432
      timeout: 60s

steps:
  - name: test-go
    scope: [backend, shared]
    tools: [go]
    run: make test-core

  - name: test-integration
    scope: [backend, shared]
    tools: [go]
    services: [postgres]
    run: make test-integration

  - name: build-web
    scope: [frontend, shared]
    tools: [node, pnpm]
    run: pnpm --filter @app/web run build

Run it:

sklp run ci                       # diff vs origin/main, dry-run by default
sklp run ci -base-ref HEAD~3      # explicit base
sklp run ci -dry-run=false        # actually execute mutating commands

Top-level fields

Step

Scope semantics

OR across named scopes        scope: [a, b] runs when a OR b matched
OR across globs in a scope    scopes[a] = [g1, g2] matches g1 OR g2
OR across changed files       any file that matches any glob → match

An unknown scope name in step.scope is a load-time error — no silent miss.

Running with no diff (git failure, missing base, empty changeset) defaults to run everything — safer than silent skip.

Overriding the diff

The diff decides what runs, but two flags take that decision back:

sklp run ci -resolve-only        # list what would run; execute nothing
sklp run ci -force               # run every step, whatever the diff says
sklp run ci -only frontend       # run only the steps scoped to `frontend`
sklp run ci -only frontend,shared  # comma-separated, OR semantics

They apply to any pipeline, so the same flags drive a release — sklp run publish -only core republishes one service from a clean tree, without forcing every image.

-force widens (every scoped step is in-scope), -only narrows (only the named scopes run, diff ignored). When both are passed -only wins — asking for one service must never publish three.

Two things worth knowing about -only:

• A step with no scope: is never selected. It names nothing, so no -only value can ask for it — the flag means "these scopes", not "these scopes plus whatever else runs unconditionally".
• An unknown scope is a hard error listing the declared names, not an empty selection. A typo must not report a green run that built nothing.

Neither flag bypasses the content-addressed cache: a step whose inputs are unchanged still serves from cache. Pair with -no-cache for a real rebuild.

-resolve-only honours both flags, so the printed plan matches what a real run would do.

Services semantics

• step.services: [postgres, nats] declares dependencies; both must be Ready before the step starts.
• Service definitions live in the top-level services: map (same shape as sklp dev).
• Unknown service names fail at load time.
• services: is independent of scope: — a step can be gated by scope AND require services. If the step is skipped (no scope match), its services are not started.
• The runner starts the union of all services used by at least one step that will run, once per pipeline invocation. Unused catalogue entries are not booted.

Service

Same schema as sklp dev services. Either run: (local process) or image: (OCI container).

Ready probe

Exactly one of http, tcp, log, file should be set.

Worked example — integration tests with Postgres

version: 1
kind: pipeline
extends: _base
name: ci

services:
  postgres:
    image: postgres:16-alpine
    ports: ["5432:5432"]
    env:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: test_db
    ready:
      tcp: localhost:5432
      timeout: 60s

  migrate:
    run: go run ./cmd/migrate up
    workdir: apps/core
    env:
      DATABASE_URL: postgres://test:test@localhost:5432/test_db?sslmode=disable
    depends_on: [postgres]
    oneshot: true

steps:
  - name: test-unit
    scope: [backend, shared]
    tools: [go]
    run: go test -short ./...

  - name: test-integration
    scope: [backend, shared]
    tools: [go]
    services: [migrate]    # depends_on: [postgres], so postgres comes up too
    env:
      DATABASE_URL: postgres://test:test@localhost:5432/test_db?sslmode=disable
    run: go test -run Integration ./...

What happens on sklp run ci -base-ref origin/main:

1. Diff computed: list of changed files vs origin/main.
2. Step gating: each step's scope: checked against the diff. Steps with no matching files are skipped.
3. Service union computed across the steps that will actually run. If test-integration is among them, migrate is requested; its depends_on: [postgres] pulls in postgres too.
4. Topo-sort + start: postgres boots first, the orchestrator waits for tcp: localhost:5432 (≤ 60s). Then migrate runs to completion (exit 0 = ready); failure aborts the pipeline.
5. Steps execute in order. test-integration sees DATABASE_URL pointing at the live Postgres.
6. Teardown on pipeline exit (success or failure, including SIGINT/SIGTERM): all services stopped, OCI containers removed.

Local vs K8s — what changes

The SKLP_RUNNER_MODE=k8s env var is set by the spawner; the runner uses it to skip the local orchestrator path entirely.

CPU budget — how much one run may use

Every CI run is given a CPU budget, a whole number of cores, decided by the spawner before the space starts. It is the single knob that bounds what one run can do to the host, and it is enforced at three levels so it holds whether or not cgroups can be applied:

Without the budget, -jobs and the Go toolchain both default to runtime.NumCPU(), which ignores cgroups: three concurrent runs on a 10-core host schedule 3 × 10 steps × 10 compiler threads ≈ 300 runnable threads. The env levels need no cgroup at all, so the parallelism collapses even where the cap cannot be applied — the macOS vz guest has no systemd and therefore no delegation.

The default, and how to change it

budget = SKLP_CI_CPUS            if set (whole cores, > 0)
       = max(2, NumCPU / 4)      otherwise, never above NumCPU

NumCPU is what the spawner sees — on macOS that is the vz guest's vCPUs (min(host cores, 8)), not the Mac's. The default assumes up to three concurrent spaces and keeps a quarter of the host free:

Override it with SKLP_CI_CPUS in the environment of the process that spawns the run (sklp run ci on your machine, the sklp-run-api unit on a CI node). A single run on an otherwise idle machine is deliberately slower under the default: raise it (SKLP_CI_CPUS=6) when you know you are the only run.

Inside the runner, SKLP_CPUS is the resolved budget. The scheduler reads it first, then the cgroup cpu.max chain it runs under, then NumCPU() — so a runner started outside sklp (a hosted CI Pod with a cgroup limit) still sizes itself correctly.

When the cap is a contract

In CI (CI=true or SKLP_CI set) the spawner sets RequireLimits: the cgroup cap is a contract, and a host that cannot apply it fails the run with the missing controller in the error rather than running unbounded. On a dev machine (CI unset) the cap is dropped silently and only the env levels bound the run.

How the cap is applied depends on who sklp runs as:

cat /sys/fs/cgroup/user.slice/user-$(id -u).slice/user@$(id -u).service/cgroup.controllers
# rootless: must list cpu pids (memory for memory caps) — else: sudo sklet host setup

The cgroupfs path is what makes the cap hold inside the vz guest: it bounds the runner's whole subtree, so tools that ignore GOMAXPROCS (tsc, vite, vitest) are capped too. See deploy resources — rootless delegation.

Common pitfalls

• Three runs feel like they eat the whole machine. Check the budget: SKLP_CI_CPUS unset gives 2 cores per run. If the host is still saturated, something in a step is ignoring GOMAXPROCS (a Node/pnpm build, cargo) — bound it in the step's env: (UV_THREADPOOL_SIZE, CARGO_BUILD_JOBS).

• Empty services: [] is the same as omitting the field. No service is started.

• Don't forget tools: — a step that uses pnpm but lists tools: [go] will fail with pnpm: command not found. Empty tools: [] means "no tool resolved" (use for pure-git / pure-shell steps).

• shared is an OR-extender, not a default scope. Add it to a step's scope: if you want .sklp/** changes to also trigger that step.

• ports: ["5432:5432"] — even in K8s where the host port is ignored, write both for parity with the local mode. The runner uses the container port for sidecar reachability.

• Volumes must be named. volumes: ["./data:/data"] is rejected. Use volumes: ["my-data:/data"].

Automatic failure diagnosis

When a pipeline step fails, sklp run can call the diagnose engine on the captured step logs and print a structured hypothesis + suggested fix before exiting. It is opt-in — pipelines without explicit opt-in keep the default behaviour (step fail → pipeline fail, no LLM call).

Three sources of intent, checked in order — first hit wins:

1. CLI flag: sklp run ci -diagnose-on-fail
2. Env var: SKLP_DIAGNOSE_ON_FAIL=1 (also accepts true, yes, on)
3. Default: off.

When opt-in is on but the API key env var for the configured provider is empty (e.g. OVH_KEY, ANTHROPIC_API_KEY, …), the runner logs a single skip notice and keeps going — diagnosis is best-effort, never a gate.

The diagnosis is rendered:

• In text mode: as a boxed block on stderr between the FAILURES section and the final banner.
• In JSON mode: attached to the failed step's StepResult.diagnosis field. Consumers that don't know the field (older spawner builds) keep parsing fine — it's omitempty.

Authoring tips for AI agents

1. Read _base.yaml first. Scope names you reference in ci.yaml come from there.
2. Reuse, don't duplicate. If the same Postgres definition is needed in both dev.yaml and ci.yaml, the user may want to move it to a shared file via extends:.
3. Service ↔ step coupling is by NAME ONLY. No magic — if a step says services: [postgres], the catalogue must have postgres:.
4. Don't add latest tags. Production registry policy forbids them; CI should match that. Pin postgres:16-alpine, not postgres:latest.
5. oneshot: true is for migrations and seeders. Long-running daemons are not oneshot.

Editor support

The repo ships a JSON Schema for .sklp/pipelines/*.yaml at pipeline/pipeline.schema.json, generated from the Go Pipeline struct (run go generate ./sklp/... to regenerate).

To get autocomplete + validation in VSCode, Cursor, Neovim, or any other editor that runs the YAML Language Server, add this comment as the first line of your pipeline file:

# yaml-language-server: $schema=../../sklp/pipeline/pipeline.schema.json
version: 1
kind: pipeline
extends: _base
# …

The path is relative to the YAML file (.sklp/pipelines/ is two levels deep, so ../../). Field descriptions in the schema are sourced from the Go doc comments on the struct fields — same source of truth as this page.

See also

• sklp dev — devstack reference — same services: / ready: / depends_on: semantics, applied to local development.