Launched this week
Kestrix
Repo-agnostic CI control plane with DAG-based triggers.
37 followers
Repo-agnostic CI control plane with DAG-based triggers.
37 followers
Kestrix defines CI/CD pipelines in HCL and resolves them into a DAG. A task runs only when the repositories its trigger names have changed. Live task logs in a React Flow UI, and remote runners that only make outbound HTTPS requests.
Free Options
Launch tags:Developer Tools
Launch Team / Built With
Atlassian LoomTalk it through in Loom. Ship with AI
Promoted
Hi Product Hunt!
I'm Damian, the author of Kestrix.
I built it for myself, to keep my DevOps projects separate from my coding projects. Kestrix treats repositories as inputs rather than as owners of the pipeline, so one pipeline definition can react to changes in several repositories.
What it does:
1. Pipelines in HCL: Pipelines are declared with variables, dependencies, and trigger expressions instead of per-repository YAML.
2. DAG per run: Kestrix resolves a Directed Acyclic Graph for every run. A task runs when its trigger matches the changed repositories, and tasks that don't match are skipped.
3. Runners: Remote runners make only outbound HTTPS requests, so they need no inbound ports. In Docker mode, tasks run with `--cap-drop ALL` and `no-new-privileges`.
4. Terraform / OpenTofu: The provider (`terraform-provider-kestrix`) manages Kestrix projects.
Kestrix is self-hostable: one binary plus PostgreSQL, or Docker Compose. The source is public under PolyForm Noncommercial 1.0.0. It is free for non-commercial use, and commercial use needs a paid plan.
Repository: https://gitlab.com/kestrix/kestrix (GitHub mirror: https://github.com/Draqun/kestrix). Docs: https://kestrix.gitlab.io/kestrix/
Do you think this will catch on? Am I the only one who prefers to keep DevOps projects separate from strictly software development projects?
The remote runners making only outbound HTTPS requests is the part I'd want to understand first — with no inbound ports, how do live task logs get back to the React Flow UI, and what happens to a run if that outbound connection drops mid-task?
@jcbuilds With only outbound HTTPS requests and no inbound ports, how do live task logs reach the React Flow UI, and what happens to a run if that outbound connection drops mid-task? How Live Logs Reach the React Flow UI The browser and the runner never talk directly to each other. Both communicate only with the central Kestrix server. 1. Job claim The runner starts on the Mac: kestrix runner --server https://kestrix.example.com --token It long-polls GET /api/runner-jobs/next. When a job is ready, the server returns the task definition, a job ID, and a claim epoch (such as epoch=1). 2. Capturing output The runner runs the task script through local bash or Docker. It wraps stdout and stderr in an internal traceWriter (cmd/kestrix/runner.go). Before sending bytes across the network, the runner redacts known pipeline secrets in memory with redact.NewWriter(). 3. Streaming chunks As the script prints output, traceWriter sends outbound HTTPS requests to the server: PATCH /api/runner-jobs/{job_id}/trace Body: {"epoch": 1, "offset": 0, "chunk": "Compiling source...\n"} Each request includes the runner bearer token. The runner tracks offset += len(chunk) for every accepted write. 4. Server persistence The server handler verifies the token, confirms the request epoch matches the active claim epoch in the database, and runs a second secret-redaction pass. In PostgreSQL (pgstore_remote_job.go:AppendJobTrace), the server appends the chunk to runs.output and remote_jobs.output. The task status transitions from claimed to running, and updated_at updates to now(). 5. Browser polling While a run has status running or queued, the web UI (ui/src/pages/Pipelines.jsx) polls every 1.5 seconds: GET /api/projects/{projectName}/runs/{runId} The response returns the updated aggregate output, the DAG definition, and the status of each node. React Flow paints the running task node orange, and the log viewer appends the new lines. When the script exits, the runner calls PUT /api/runner-jobs/{job_id}/complete. The server stores the final result, and the next UI poll paints the node green or red. What Happens When the Outbound Connection Drops Transient network errors If an individual PATCH request fails due to a network glitch, traceWriter logs the error to the runner's stderr and continues executing the task. A dropped log chunk does not terminate the build script. If the runner retries a chunk after a network timeout: - If offset < traceOffset, the database recognizes that this byte range was already stored. It returns success without duplicating text. - If offset > traceOffset, the server rejects the write as an out-of-order gap. Extended disconnect, system sleep, or runner crash The server tracks runner liveness via updated_at on the remote_jobs table. Every trace chunk and poll refreshes this timestamp. A background loop on the server (StartRemoteJobReclaimLoop in internal/web/server.go) checks for inactive jobs. When a job remains claimed or running without a trace append or complete call beyond KESTRIX_RUNNER_REMOTE_JOB_TIMEOUT: 1. Reclaim The server moves the job status back to pending and increments claim_epoch (for example, from 1 to 2). This makes the job available for another runner to claim. The prior output remains in runs.output, while the new claim starts with clean task output. 2. Abandonment Kestrix limits retries via KESTRIX_RUNNER_REMOTE_JOB_MAX_RECLAIMS (default: 1). If no runner claims the reclaimed job within the timeout, or if reclaims hit the maximum limit, the server marks the job failed with: abandoned by unresponsive runners Dependent tasks transition to skipped, and the run finishes in the failed state. 3. Epoch fencing If the disconnected runner finishes its task locally and attempts to submit results after being reclaimed: PUT /api/runner-jobs/{job_id}/complete {"epoch": 1, ...} The server checks the submitted epoch against the current claim_epoch in PostgreSQL. Because the epoch was incremented during recovery, the server rejects the request with HTTP 409 Conflict. The stale runner cannot overwrite data or alter a finished run.
My free GitHub Actions minutes ran out this month so my CI now runs on my Mac at home. Can a Kestrix runner work behind a home router like that? And if three merges land during one run, does it run once on the newest commit?
Yes, a runner on a Mac behind a home router works fine. It only makes outbound connections. It long-polls the Kestrix server (GET /api/runner-jobs/next), claims jobs, streams logs and reports results back. You don't need a public IP, port forwarding or DynDNS. GitHub Actions self-hosted runners work the same way.
With --mode local, jobs run directly in the macOS shell. With --mode docker, they run in containers through Docker Desktop or OrbStack. This assumes the server runs on a VPS or in the cloud, and only the runner is on your Mac. If you run kestrix serve on the Mac itself, GitHub has to send webhooks to it. Then you'd need a tunnel (Cloudflare Tunnel, Tailscale Funnel, ngrok) or port forwarding. As for three merges landing during one run: Kestrix runs all three. Each push creates its own run pinned to its SHA. The runs wait in a FIFO queue, and runner.max_concurrent_runs (default 2) caps how many run at once. Kestrix doesn't cancel superseded runs on its own. You can cancel them in the web UI or with POST /api/projects/{name}/runs/{id}/cancel. A queued run cancels instantly and uses no runner time. Would you want auto-cancel as a feature, similar to GitHub's cancel-in-progress? When a new commit lands on a branch, Kestrix would drop the older runs for that branch. If so, should it cancel only queued runs, or running ones too?
Seeing why a task was skipped would matter as much as its logs here. With several repositories in one pipeline, the trigger needs to be easy to inspect.
@oleksii_sekundant Great catch, and I completely agree with that — debugging a skipped task without knowing which condition evaluated to false (especially across multiple repos) is frustrating. I'll work on it soon: we can surface the trigger expression along with the skip reason (e.g., in the task details panel / tooltip) so it's immediately obvious why it didn't run. I'll update the PR once it's in place. Thanks!