Article · Architecture

Building self-improving AI systems.

A system that repeats the same mistake every Tuesday is not learning, it is just running. Memory and reflection are what separate the two, and both are easier to describe than to build safely.

Architecture · · 12 min read

A system that makes the same mistake every Tuesday is not learning. It is simply running. The difference between the two is memory and reflection, and both are considerably easier to describe than to implement safely.

The appeal is obvious. A system improving from its own operation compounds in value rather than degrading. The risk is equally obvious and less discussed: a system that changes its own behaviour can change it for the worse, and can do so invisibly.

What memory actually means here

The word covers several distinct mechanisms that get conflated, which causes confusion in design discussions.

Conversational memory

Retaining context within an interaction. Largely a solved problem, handled by context windows and summarisation. Necessary, not interesting.

Episodic memory

Recalling specific past events. This customer raised this complaint in March; this supplier missed a deadline twice. Implemented as a searchable store of past interactions the agent can query.

The design difficulty is retrieval relevance. An agent that surfaces every past interaction drowns in its own history. One that surfaces too few misses the pattern that mattered.

Semantic memory

Accumulated knowledge abstracted from specific events. Not this invoice was rejected, but invoices from this supplier are consistently missing the PO reference.

This is where genuine improvement lives, and where the risk concentrates. Abstracting a general rule from limited examples is precisely the operation that produces confident overgeneralisation.

Procedural memory

Learned approaches to tasks. Which sequence works for this document type, which tool answers this class of question. Most valuable, hardest to validate, because a procedure that worked on the last forty cases may be wrong for the forty-first in ways nobody notices.

Reflection loops

Reflection means the system evaluating its own output before or after acting. In practice this is a second pass with a different prompt: check this draft against the source, identify anything unsupported, revise.

This measurably improves quality on tasks with verifiable properties. Did the summary include claims absent from the source, does the extracted data match the document, is the calculation arithmetically correct. Where correctness is checkable, reflection catches a meaningful proportion of errors.

It is considerably weaker where correctness requires judgement or external knowledge. A model reflecting on whether its reasoning was sound is using the same reasoning that produced the output. Self-critique does not escape a shared blind spot, and a confident error frequently survives reflection intact.

A practical rule: reflection against ground truth is valuable, reflection against itself is theatre. If the reflection step has access to the source document, the database record or the calculation, it can catch real errors. If it only has the model's own output, it mostly produces more confident versions of the same mistakes.

Where self-improvement quietly goes wrong

Learning from unrepresentative cases

Systems typically receive feedback on cases where something went visibly wrong. Cases handled correctly produce silence. A system learning primarily from complaints acquires a distorted picture, and the distortion is systematic rather than random.

Feedback loops that reinforce themselves

If a system learns that a certain enquiry type should be escalated, it escalates more of them. Those escalations become evidence that the type requires escalation. Nothing in the loop tests whether the original inference was correct, and confidence rises regardless.

Drift without a baseline

Behaviour changes gradually across many small updates. Each is individually reasonable. The cumulative effect can be substantial and nobody notices, because no single change looked wrong and there is no fixed reference to compare against.

Learned shortcuts

Systems optimise for the signal they are given, not the outcome you intended. If the measured signal is resolution speed, a system may learn that closing cases quickly scores well, whether or not the underlying issue was addressed.

Design patterns that contain the risk

Separate proposal from adoption

Let the system propose changes to its own behaviour, and require human approval before adoption. Slower, and it preserves the ability to answer why the system behaves as it does, which matters enormously in regulated contexts.

Maintain a frozen evaluation set

Hold a fixed set of cases with known-correct outcomes, and run it after every change. This is the only reliable defence against gradual drift. It requires curation effort and it is invariably worth it.

Version everything

Prompts, learned rules, memory contents. When behaviour changes unexpectedly, the first question is what changed, and answering it requires history. This also makes rollback possible, which is the difference between a bad afternoon and a bad quarter.

Bound what can be learned

Distinguish parameters the system may adjust from rules it must not. Retrieval strategy and phrasing preferences are reasonable candidates. Compliance rules, permission boundaries and escalation thresholds should be fixed, changed only through deliberate human process.

What is realistic today

Worth being clear about the current state. Production systems that genuinely improve autonomously are rare, and mostly operate in narrow domains with strong feedback signals.

What is achievable now is more modest and still valuable: systems accumulating episodic memory that makes them contextually aware, reflection against verifiable ground truth, and instrumentation good enough that humans can identify improvements and implement them deliberately.

That last point is the one most worth internalising. Most of the improvement attributed to self-improving systems in practice comes from humans reading good telemetry and making changes. The system is not learning; the team is. Building for that is less exciting than autonomous learning and considerably more likely to work.

The governance question underneath

A system that modifies its own behaviour raises a question that has no purely technical answer: what was this system approved to do, and is it still doing that?

In regulated environments this is decisive. If a system's behaviour today differs from the behaviour that was assessed and signed off, the approval arguably no longer applies. Organisations deploying adaptive systems in regulated contexts need a defensible answer, and the answer usually involves constraining adaptation more tightly than the technology strictly requires.

Thinking about this for your own operation?

A 30-minute call, no pitch. Describe the process and you leave with an honest written view of whether it is worth building.