hacktobeer's website
Table of contents

Developing OpenRelik with AI: My Herdr & Pi Setup

Introduction

Over the past months, AI coding assistants have evolved beyond simple inline autocompletions and one-off chat windows. When working on larger open-source projects with multiple moving parts, single-prompt chat interfaces quickly show their limits: context windows get polluted, small diagnostic loops burn through expensive tokens, and agents lack clear boundaries between architecture, coding, and integration testing.

In this post, I want to share my current setup for developing code and workers for OpenRelik using AI. My workflow pairs Herdr—a terminal workspace manager built specifically for AI coding agents—with Pi for tiered multi-agent orchestration.

Whether you are developing OpenRelik workers or building software in any modular monorepo, this pattern demonstrates how separating responsibilities across specialized agents keeps development fast, structured, and cost-effective.

What is OpenRelik?

OpenRelik is an open-source digital forensics and incident response (DFIR) workflow platform originally initiated by Google. It enables incident response teams to automate forensic analysis pipelines at scale.

A big part of OpenRelik development centers around workers—independent services responsible for specific forensic tasks like extracting EXIF metadata, calculating file hashes, scanning with YARA rules, or carving data.

The workers repository (openrelik-workers) is organized as a Python monorepo:

  • lib/openrelik_worker_common: Shared library package across all workers.
  • workers/<worker-name>: Individual worker packages (e.g. workers/hasher, workers/exif, workers/yara), each with their own pyproject.toml, source tree, and tests.

To build and maintain these workers reliably, you need strict dependency isolation, thorough unit test coverage, and end-to-end workflow verification against a running OpenRelik server.

The Tooling Stack

To tackle this efficiently with AI, I rely on three core tools:

1. Herdr (Terminal Workspace Manager)

Herdr is a terminal workspace manager designed from the ground up for running AI coding agents. Unlike generic terminal multiplexers, Herdr provides:

  • Named, persistent sessions that survive disconnections.
  • A socket API for programmatic workspace and pane management.
  • First-class Git worktree integration so agents can work in parallel branches without colliding.
  • Native agent integrations and notification helpers.

2. Pi.dev & Pi-Herdsman (Agent Harness & Orchestration)

Pi is an extensible, keyboard-driven AI coding harness. Combined with pi-herdsman, it allows defining custom agent roles, managing delegation hierarchies (Lead agent vs. child agents), and managing tool permissions with fine-grained control.

3. Modern Python Tooling (uv, ruff, pytest)

Instead of managing global virtual environments or dealing with slow dependency resolvers, every worker package is managed with:

  • uv: Extremely fast Python packaging. All commands run scoped via uv run inside the target worker directory.
  • Ruff: For instant linting and formatting (uv run ruff check and uv run ruff format).
  • Pytest: For isolated test execution (uv run pytest).

Getting Started: Installation & Repository Setup

If you want to set up this development workflow yourself, here is how to get everything installed and wired together:

1. Prerequisite: Local OpenRelik Server

Because our test agent (OpenRelik-Runner) validates workflows against an actual server on localhost, you will want a local OpenRelik instance running. Follow the official OpenRelik Installation Guide to get the server up and running via Docker Compose.

2. Install Herdr

Install Herdr via the official installer:

# macOS and Linux
curl -fsSL https://releases.herdr.dev/app/install.sh | sh

(Alternatively, if you use mise, run mise use -g herdr@latest).

3. Install Pi and Pi-Herdsman

Install the Pi CLI:

curl -fsSL https://pi.dev/install.sh | sh

Then install the pi-herdsman package for multi-agent coordination:

pi install npm:pi-herdsman

(Ensure you have Node.js 22.19+ and Python’s packaging tool uv installed via curl -LsSf https://astral.sh/uv/install.sh | sh).

4. Configure Model Credentials & Choose Models

Set your provider API key in your environment (for example, Gemini, Anthropic, or OpenAI):

export GEMINI_API_KEY="your-api-key-here"

You can list all models available to Pi at any time by running:

/models

Note on models: My setup uses gemini-4-argon for the Lead and gemini-3.8-flash for the child agents. If you want to use different models (such as Claude 3.5 Sonnet, GPT-4o, or others), simply update the model names in .pi/AGENTS.md and the frontmatter of each file in .pi/agents/*.md.

5. Fork and Clone openrelik-workers

  1. Fork the upstream repository https://github.com/openrelik/openrelik-workers to your GitHub account.
  2. Clone your fork locally:
git clone https://github.com/<your-username>/openrelik-workers.git
cd openrelik-workers

To keep agent definitions maintainable, version-controlled, and easy to reuse across projects, I keep my Pi and Herdr agent configurations in a dedicated GitHub repository: hacktobeer/pi-agents.

Clone that repository side-by-side with your openrelik-workers repo and symlink the worker configuration:

# Clone the pi-agents repository next to openrelik-workers
cd ..
git clone https://github.com/hacktobeer/pi-agents.git
cd openrelik-workers

# Symlink the OpenRelik workers agent configuration into .pi
ln -s ../pi-agents/openrelik-workers/ .pi

(You can fork hacktobeer/pi-agents if you want to tweak agent definitions, prompt templates, or role rules for your own workflow).

7. Install and Configure the OpenRelik CLI and Skill

Install the official OpenRelik CLI tool following the setup instructions in the OpenRelik CLI repository.

Before the CLI can talk to your local OpenRelik server, authenticate your client:

openrelik auth login

This ensures the CLI client is logged in and can successfully connect to the OpenRelik installation on localhost.

Next, rather than hand-crafting custom tool definitions or writing lengthy instruction prompts, OpenRelik’s CLI can generate its own native skill definition for Pi directly from its command tree:

openrelik skill install --local --harness pi --yes

When the OpenRelik-Runner agent initializes, it automatically discovers this generated skill and knows exactly what commands, parameters, and workflows are available on your OpenRelik installation.

8. Launch Herdr and Verify Agent Setup

Now, launch Herdr to start your persistent development session:

herdr

Inside a Herdr pane, start Pi in your openrelik-workers directory:

cd openrelik-workers
pi

Sanity Check: On startup, pi-herdsman automatically parses .pi/AGENTS.md and initializes your Lead profile. You can verify that the team is loaded by checking that Pi reports your role as Lead with child agents (python-coder, pytest-engineer, openrelik-runner) available for delegation.

Team Architecture: Separation of Roles

A key lesson from building with AI agents is that asking a single agent to plan, write, test, debug, and run end-to-end integration tests in one unbounded session usually leads to messy context and high token burn.

Instead, I configure a hierarchical team where each agent has strict boundaries, distinct model tiers, and explicit cost constraints.

1. The Lead Agent (Architect & Orchestrator)

  • Model: Frontier reasoning model (e.g. gemini-4-argon / high reasoning)
  • Responsibility: High-level architecture, task decomposition, reviewing completed work, and delegating assignments.
  • Strict Guardrail: As the most capable (and expensive) model, the Lead is strictly prohibited from running implementation code edits or repetitive diagnostic loops directly. Every execution task is handed off to specialized child agents.

2. Child Agent: Python-Coder

  • Model: Fast, cost-efficient model (e.g. gemini-3.8-flash with medium reasoning)
  • Allowed Tools: read, write, edit, bash
  • Responsibilities: Implements new features, refactoring, and bug fixes inside worker packages.
  • Strict Constraints:
    • Must run all commands via uv run inside the worker directory (e.g., cd workers/<worker-name> && uv run ...).
    • Never touches global pip or system Python.
    • Ensures uv run ruff format and uv run ruff check pass cleanly before reporting completion.
    • Cost optimization: Encouraged to make comprehensive, structural edits in one pass rather than small, iterative back-and-forth loops.

3. Child Agent: Pytest-Engineer

  • Model: Fast, cost-efficient model (gemini-3.8-flash)
  • Allowed Tools: read, write, edit, bash
  • Responsibilities: Writes comprehensive unit tests under workers/<worker-name>/tests/test_*.py, executes the test suite, verifies coverage, and ensures the worker Docker container builds cleanly (openrelik-worker-[NAME]).
  • Strict Constraints:
    • Must use uv run pytest inside the worker directory.
    • Read full stack traces and fix test regressions in targeted edits.

4. Child Agent: OpenRelik-Runner

  • Model: Fast, cost-efficient model (gemini-3.8-flash)
  • Allowed Tools: read, write, edit, bash
  • Responsibilities: End-to-end validation against a local OpenRelik instance running in Docker on localhost.
  • Strict Constraints:
    • Controls OpenRelik exclusively via the official openrelik CLI.
    • Strictly forbidden from modifying source code or connecting to raw internal APIs.
    • Verifies workflow outputs and inspects execution logs from an operator perspective.

Agent Configuration as Code

With the .pi directory linked to pi-agents/openrelik-workers/, the layout inside looks like this:

.pi/
├── AGENTS.md                 # Team roles, models, and delegation rules
├── CONTEXT.md                # Monorepo architecture, uv, ruff, & pytest rules
├── APPEND_SYSTEM.md          # Global instructions injected into agent sessions
└── agents/
    ├── python-coder.md       # Definition for Python-Coder
    ├── pytest-engineer.md    # Definition for Pytest-Engineer
    └── openrelik-runner.md   # Definition for OpenRelik-Runner

A Typical Development Workflow

Here is how developing a new OpenRelik worker feature looks in practice:

  1. Initiate with Lead: I present the requirements for the worker directly to the Lead in plain English:

    “I need a new worker under workers/magic_parser that uses python-magic to inspect input files and extract MIME types and human-readable descriptions into OpenRelik’s metadata output. Please build and verify this worker.”

    Because the agent team definitions and delegation boundaries are already configured in .pi/AGENTS.md, you do not need to explain the child roles or tell the Lead how to delegate. The Lead automatically decomposes the goal and coordinates the work.

  2. Scoping & Delegation: The Lead analyzes existing worker patterns in workers/ and crafts a clear, bounded task for Python-Coder.

  3. Isolated Coding in Herdr: The Python-Coder agent is spawned in a Herdr pane or an isolated Git worktree. It writes the worker implementation, adds dependencies via uv add <package>, and ensures ruff formatting passes.

  4. Automated Testing Hand-off: Once code implementation resolves, the Lead hands off to Pytest-Engineer. The test engineer writes edge-case unit tests, mocks external dependencies, runs uv run pytest, and verifies the container build.

  5. End-to-End Run: The Lead delegates verification to OpenRelik-Runner, which triggers an analysis task using openrelik workflow start on localhost and checks that the output data matches expectations.

  6. Final Review: The Lead reviews the combined changes, verifies all acceptance criteria are met, and delivers a concise summary back to me.

Context Management & Pi Compaction Settings

Developing workers involves running tests, analyzing tracebacks, and reading forensic tool output. In long multi-turn sessions, this command output quickly inflates the conversation context. Left unchecked, it blows through token budgets and causes prompt degradation.

Pi solves this with an intelligent compaction engine. When context approaches the model’s threshold, Pi replaces older message spans with a concise LLM-generated summary, while preserving recent history and system checkpoints intact.

You can customize this behavior globally in ~/.pi/agent/settings.json or on a per-project basis in .pi/settings.json:

{
  "compaction": {
    "enabled": true,
    "reserveTokens": 16384,
    "keepRecentTokens": 20000,
    "modelOverrides": {
      "google/gemini-3.8-flash": {
        "reserveTokens": 400000
      }
    }
  }
}

Key Compaction Parameters:

  • compaction.enabled (boolean, default: true): Enables or disables automatic compaction. Even if set to false, you can still compact manually.
  • compaction.reserveTokens (number, default: 16384): The buffer of free context tokens below which automatic compaction triggers. If your model’s window is nearly full and remaining tokens drop below this reserve, Pi initiates compaction before sending the next turn.
  • compaction.keepRecentTokens (number, default: 20000): The number of recent tokens retained verbatim without summarization. This ensures the agent does not lose recent error messages, diffs, or instructions that are currently being worked on.
  • compaction.modelOverrides (object): Allows overriding token budgets on a per-model basis. For frontier models with massive context windows (such as Gemini with 1M tokens), you can set a higher reserveTokens threshold (e.g. 400,000) so compaction triggers at an optimal point without blowing up request latency or API costs.

Manual Compaction (/compact)

In addition to auto-compaction, you can trigger compaction manually at any logical breakpoint during development using /compact [instructions]:

/compact Focus on the new YARA worker parser logic and recent pytest failures

By providing custom instructions, you direct Pi’s summarizer to retain critical architectural decisions and discard ephemeral command noise.

Key Takeaways & Cost Optimization

  1. Tiered Model Strategy: Using high-tier models (like Gemini 4 Argon) solely for orchestration while delegating repetitive tasks to fast models (like Gemini 3.8 Flash) cuts API costs significantly while actually increasing reliability.
  2. Strict Role Boundaries: Restricting what child agents can and cannot do (e.g. forbidding the runner from modifying code, forbidding the coder from running untracked global python) prevents accidental scope creep.
  3. Workspace Isolation with Herdr: Running background agents in independent Herdr panes and Git worktrees means agents don’t overwrite each other’s uncommitted work or lock up your primary terminal.
  4. Single-Turn Edits: Instructing agents to make structural, bulk edits instead of micro-iterative test-and-tweak loops saves considerable context tokens and execution time.

Conclusion

Combining Herdr’s workspace management with Pi’s multi-agent orchestration gives you a development environment that feels like managing a small, highly coordinated engineering team right from your terminal.

For complex projects like OpenRelik workers, where quality and testing rigor are paramount, this setup strikes the right balance between automation speed and engineering control.