Agents that learn on the fly vs. agents you can govern

I live with two kinds of agents.

The first kind sits on my laptop. It remembers how I like my code formatted, which commands I run every morning and which shortcuts I never use. It gets a little better every week, and I like it more for it.

The second kind is the agent I design for work, the one that handles requests for thousands of clients. I want that one to behave “exactly” the same on Thursday as it did on Monday, unless someone has decided, with evidence, that it should change.

For a while I felt slightly inconsistent about this. I don’t any more. They are two different products, built on two different ways of learning.

The self-improving wave

Over the past year, self-improving agents have moved from research demos to tools many people use daily. Hermes Agent from Nous Research describes itself as a closed learning loop: it creates skills from experience, improves them as it uses them and builds a model of its user across sessions. Personal harnesses such as OpenClaw, or a coding agent with its own memory and skill files, follow the same idea. The agent writes down what worked and reuses it next time.

These tools are genuinely good, and I use them every day.

Two ways an agent can learn

A quick note on vocabulary first. In machine learning, online learning usually means updating a model’s weights as new data arrives. I’m using the terms more loosely here, for how an agent’s behaviour changes over time, whatever the mechanism: memory, skills, prompts or weights.

With online learning, the agent changes itself, in production, while it works. After a task it extracts a procedure, edits a memory file, refines a skill or adjusts its picture of the user. The next run is already different from the last. Nobody approved the change. The agent decided.

With offline learning, the agent in production is a frozen, versioned artefact: prompts, tools, graph, retrieval tables, model version. It never modifies itself. Its runs are logged and judged by people and by metrics, and that judgement is turned into a candidate new version outside production. The candidate ships only if it beats the current one on an evaluation suite.

Both learn from experience. What differs is who decides that a lesson is a good one, and when.

Why online learning is right for one person

For a personal agent, learning on the fly is the right trade-off, and the reasons are simple.

There is one user, and that user is also the judge. If the agent picks up a bad habit, I notice and correct it in my next message. When it gets something wrong, I lose a few minutes, and no customer complaint or regulatory finding follows. Personalisation is the whole point of the product, since an agent that knows my conventions is worth more every week. And I never have to wait for a release.

That short loop between experience and behaviour is what makes these tools feel a bit magical.

What happens when you scale it up

Now take the same mechanism and put it in front of thousands of clients. The picture changes quickly.

The first problem is traceability. If the agent changed itself between Tuesday and Wednesday, which version answered the client who complains on Thursday? “The agent learnt something” is an answer no auditor will accept, and I wouldn’t accept it either.

Then there is drift with no gate. A lesson learnt from one unusual case gets applied silently to thousands of ordinary ones. Online learning optimises for what happened recently, which is not always what is correct.

Then there is who gets to teach the agent. On my laptop, the only teacher is me. In production, the inputs come from clients, documents and emails, all of it untrusted content that can steer what the agent memorises. Putting private data, untrusted input and the ability to act in the same self-modifying loop is a combination I’m not willing to sign off.

Finally, regulated companies separate the people who build a system from the people who validate it. An agent that rewrites its own procedures plays both roles. Add the governance pieces these tools were never designed to provide, such as role-based access, signed and reviewed skills, and complete audit logs, and you have most of the work that stands between a clever prototype and something you can govern and audit.

None of this is a criticism of the tools. They were designed for a single trusted user, and for that user they are excellent.

The industrial pattern

What I build at work keeps the learning loop and moves it out of the hot path. In practice it comes down to seven habits.

1. Freeze and version everything you control: prompts, graph, tools, model, retrieval tables. Every production run points to an identifiable configuration.
2. Trace every step, not only the final answer: each node, its inputs and outputs, the passages retrieved, the tool calls and their responses, the exact model identifier, sampling settings, cost and latency.
3. Collect structured human judgement. Domain experts review real runs against explicit criteria, step by step, through review queues rather than thumbs-up buttons.
4. Turn feedback into data. Corrections become golden questions, regression cases and labelled examples.
5. Improve offline. Change a prompt, a chunking strategy, a tool or a model, then replay from the step that changed instead of re-running everything.
6. Gate with evaluation. The candidate has to beat the current version on the evaluation suite, and must not regress on the cases that matter, before it is promoted.
7. Keep evaluating in production. Sampled runs keep flowing into review, so drift has a good chance of showing up in the measurements before it shows up in complaints.

The agent still learns from every interaction. It learns through a process that can be audited and rolled back.

None of this makes an agent error-free, or even perfectly repeatable. A frozen configuration can still behave differently from one run to the next. Models are not fully deterministic, a hosted model can be updated by its provider, the documents behind retrieval change, and external tools return new answers. What the pattern guarantees is that every run is identifiable and that you have the evidence to explain it afterwards. An evaluation suite only catches what it was built to test. But when something does slip through, you can tell which configuration produced it, see what it received, fix it, and add the case to the suite.

Side by side

	Online learning (personal harness)	Offline learning + continuous evaluation (industrial)
Who changes the agent	The agent itself, during use	A team, through a release
Unit of change	A memory entry, a skill file	A versioned artefact
Who validates	The single user, implicitly	Domain experts and an evaluation suite, explicitly
Speed of improvement	Immediate	Days, by design
Traceability	Low: behaviour depends on accumulated history	High: every run maps to a configuration and a recorded trace
Blast radius of a bad lesson	One user	Designed to catch regressions before release
Best for	Personal productivity, exploration	Client-facing, regulated, high-volume processes

Where this is heading

So I don’t see the two approaches as rivals. They answer different questions. If you want an assistant that adapts to you, let it learn online. If you want an agent that thousands of people and a regulator can trust, let it learn offline, and keep evaluating it for as long as it runs, because its inputs and dependencies will keep changing even when its configuration doesn’t.

The part I find most interesting is the space in between: agents that propose what they have learnt, a new skill or a new memory, as a candidate change that goes through the same evaluation gate as any other release. Self-improvement with a reviewer in the loop. I suspect that is where the industrial agents of the next few years will end up, and it’s one of the reasons I think evaluation deserves to be treated as a battle worth fighting.