Bryce Davidson

Hook or eval? Sorting the rules I give coding agents

Bryce Davidson, November 2026


Most of S4mple, the learning app I'm building, is written by Claude Code sessions. They work under rules I've written down, and every rule has to live somewhere. I use two places.

A hook is a script that runs during real work and blocks an action. An eval is a test session, graded afterwards, that measures whether the agent follows a written instruction.

The question that decides it: could a script, seeing only the action in front of it, make the right call every time without knowing what I wanted? If yes, it's a hook. If no, the rule stays in the instructions and gets an eval.

Here are six of my rules. Sort them before you read the answers.

Where does each rule live?

Pick hook if a script could enforce it, eval if it needs judgment.

  1. Don't merge until CI has passed on this exact commit.

    Hook.The answer is the same whatever I asked for. A script can look up the commit's CI status and refuse the merge.

  2. Stop and ask before anything that could reach the users, sessions or credentials tables.

    Eval.No list of commands can tell which query reaches those tables. The same database command is fine on a scratch copy and a disaster in production.

  3. Don't install a build on a test phone another session is using.

    Hook.Sessions claim the phone in a shared file. A script can read the claim and block the install.

  4. When I ask for research, build nothing.

    Eval.Merging is allowed in general. Only the way I framed the request says it's out of scope this time, and a script can't read that.

  5. Run the end-to-end tests before merging a change that touches them.

    Hook.Which files a change touches is a fact a script can check against a list of paths, so the merge is refused until the suite has passed.

  6. Say what you're aiming for before you start working.

    Eval.Nothing mechanical can tell whether the agent stated its intent before reaching for tools. This is the one rule that failed my first eval run: the agent said it only after six tool calls.

What the sorting teaches

The hooks have something in common: the right answer never depends on the request. That's what lets a script enforce them, and it's why they don't need measuring. They can't be broken.

The evals are the opposite. Each one depends on what I asked for, which database, or how I phrased something. A hook strict enough to catch them would block normal work.

Rules also move. Checking CI before a merge started as a written instruction with an eval case. Once I'd built the merge hook, there was nothing left to measure, and the case was retired. When a rule can become a hook, it should.