Notes on agents and harnesses
A harness, briefly put, is a runtime that connects a model to an environment and carries its state between calls. The model supplies its judgement, e.g. which action(s) to take, what a result means, and so on. The harness determines:
- which tokens and tools the model sees
- how tools execute and return results
- when the model runs again
- what gets persisted, compacted, branched, or forgotten
- what permissions, sandboxes, personas, memories, and other agents exist
- boring things, like how provider quirks and failures are repaired
The above is not necessarily an exhaustive list.
What is an agent
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators (Russell & Norvig, 2022).
At its most basic level an agent does two things:
- It receives information (the received information can be called a percept or observation).
- It chooses something to do (we call this an action).
For example, an agent might receive:
- a user message
- the contents of a file
- the result of a terminal command
And then act by reading another file, editing code, and so on.
Intermezzo: some formal definitions
A simple reflex agent:
read as f takes one current percept and returns one action. This works for a simple reflex agent where the correct action depends only on what it observes right now. This is insufficient for agents in worlds where the current observation may require different actions depending on what happened earlier.
A history-dependent deterministic agent:
read as the agent function f maps every possible finite percept history to an action. But this does not adequately describe probabilistic agents, e.g. an agent where, given the same history twice, may choose different actions.
A history-based probabilistic model:
read as the policy π maps every finite percept history to a probability distribution over possible actions. Policy here simply means a rule for choosing actions.
A state-based probabilistic model:
read as the new state is calculated from the old state, the new percept, and the previous action, and the agent can then choose an action from its state.
A belief-state probabilistic model:
read as the agent updates its previous belief using its previous action and newest observation. Its policy then assigns probabilities to possible actions based on that updated belief, from which an action is selected.
If we were to map the formal objects concretely:
| Formal Object | Possible correspondences |
|---|---|
| π (policy) | the action-selection behavior produced by the model, instructions, context, and harness constraints |
| u (state update) | the state-transition process implemented jointly by the harness and environment |
| A (action space) | tool calls, messages |
| P (percept space) | messages, files, tool results |
| st (internal state) | context, summaries, memories, task state |
The policy can be conditioned by things like instructions, skills, and memories.
A suitable plain-English definition of an agent could be:
What a general harness consists of
We can divide a harness into several components:
- Surface
- CLI, TUI, web
- Composition
- plugins, services, dependencies, lifecycle
- Control
- turn loop, scheduling, retries, steering, state
- Execution
- filesystem, shell, LSP, browser, sandbox
- Model
- persona, prompt, tools, provider messages
Connected to the model is the inference server/API.
