Why AI governance has to meet in the middle

A few years ago, an unfinished technology landed in everyone’s hands. ChatGPT was released while OpenAI was still figuring out what the product was. Millions of people started using it, pushing it into strange corners and discovering failure modes its makers had not anticipated. The product evolved in public because the people building it were learning from what people did with it.

Governance works in almost exactly the opposite way.

Rules have to be written, reviewed, negotiated, approved and translated into controls before they can be applied. By the time that process is finished, the thing being governed may have changed shape entirely.

The EU AI Act has been running this experiment at continental scale. Most companies are now repeating it internally.

The European Commission published its proposal for the AI Act in April 2021. Nineteen months later, ChatGPT appeared. General-purpose AI then had to be fitted into legislation that was already well into negotiation.

The Act also depends on technical standards to make much of it operational. CEN and CENELEC were originally supposed to deliver those standards in April 2025. That deadline passed. So did the revised one.

The first standard written specifically in support of the Act, EN 18286, appeared in July 2026. It only covers AI quality-management systems: how organisations document processes, control versions, assign responsibilities and keep records. These are all useful things. But mostly the things that remain stable while everything else moves.

You can think the AI Act is a good law or a bad one, and this holds either way because the underlying problem is the same. Give a framework legal force, deadlines and hundreds of experts, and ask them to write it ahead of the technology. They can only be precise about the parts that hold still. The rest changes while they are writing.

Enterprises are now doing the same thing at a much smaller scale.

Inside enterprises, compliance teams are being asked to define how AI systems should be classified, which models may be used, what agents are allowed to do, how human oversight should work, and what evidence is required before anything reaches production.

Meanwhile, everyone else who wants to build with GenAI is waiting for the framework.

This sounds responsible, but it creates four problems.

The first is that most good rules are written backwards.

Look at any mature corporate security policy and you are looking at layers of past mistakes. One clause exists because somebody lost a laptop. Another because somebody clicked a phishing email. Another because client files ended up in a personal Dropbox account. The policy accumulated around actual failures.

You could not have written today's information-security manual in 1985, because most of the behaviour it regulates did not exist yet. If you write controls before the practice exists, you usually get one of two things: rules against hypothetical failures that never materialise, or a blanket prohibition.

The second is that the object being governed keeps changing.

In 2023, the object was mostly the model. The questions were bias, hallucinations, training data and accuracy. Then enterprise AI became retrieval. The model was often a fixed vendor component and the interesting failures moved somewhere else: permissions, stale documents, indexing, access control. Now the interesting object is an agent operating through an employee's identity.

The question is no longer only what the model said. It is what the system did, which tools it called, what data it touched, whose permissions it inherited and whether anyone can reconstruct the sequence afterwards. A governance process built around approving models is not much help when the model itself is a line in a configuration file.

The third problem is more subtle.

Written answers are becoming nearly worthless. Take a standard control question:

Has appropriate human oversight been implemented?

A team can answer yes in all of these cases:

• a person approves every action
• a person approves a batch of actions
• a person occasionally checks a dashboard
• a person can revoke access afterwards
• a person wrote the original prompt

These are radically different systems.

Historically, reviewers could learn quite a lot from how carefully a team described its controls. Now any team can produce an excellent-looking compliance answer in ninety seconds, using the technology being reviewed.

So the useful unit of governance shifts from explanation to evidence. Show me the eval. Show me the logs. Show me the failure cases. Show me what happened when permissions were wrong, the source document contained malicious instructions or the model was uncertain.

That evidence does not appear at the end of a project because somebody asks for it. It exists because somebody designed the system to produce it.

And then there is the fourth problem.

People do not stop building while governance is being written. They just stop telling governance. Someone has a Copilot licence. Someone else has an API key on a team credit card. A third person has connected an internal dataset to a model because they have a quarterly target and the prototype works. Six months later, the framework arrives and discovers an estate nobody has inventoried.

Its first practical effect is to declare a large amount of existing work non-compliant.

None of this means build first and write the paperwork later. The point is that both sides have to move toward each other.

Engineering teams need room to build real systems in controlled environments, with limited permissions and a contained blast radius. Governance teams need to move closer to the machinery, early enough to see what actually happens rather than reviewing a description of it afterwards.

They meet in the middle around evidence: traces, failures, permissions, logs and, most importantly, evaluations. An eval is simply a repeatable test of how the system behaves on cases that matter. An engineer sees a test; a compliance officer sees a control. Both are right.

If a governance requirement cannot be turned into something observable, it is difficult to enforce. And if engineers are testing things nobody in risk, legal or compliance cares about, they are probably testing the wrong things.

So the framework should not sit upstream waiting to be finished, and engineering should not disappear downstream until the product is done. They should converge as the system is built, with each side changing what the other does as the organisation learns what the technology actually does.

There is an obvious problem with all of this. ‘Meet in the middle’ is also what every engineering team says when it wants governance to leave it alone. Done badly, this becomes build now, document later. Only later never comes.

The useful test is whether the work keeps producing artefacts. At any point during development, can the team hand an examiner something produced by the system rather than a paragraph written about the system?

An eval run. A trace. A permissions matrix. A red-team result. An incident log. A versioned prompt. Intent is difficult to audit but last Tuesday's test run is not.

This model also asks more of both sides than the sequential one. Governance people have to spend time inside technical work while it is still messy. Engineering teams have to expose unfinished systems to people whose job is partly to ask awkward questions.

These are cultural changes disguised as process changes. Which is probably why organisations keep returning to the sequential model even after watching it fail.

A standard can tell you that responsibilities need to be documented. It can tell you to maintain records, manage versions and define oversight. It cannot tell you whether your sales agent should be allowed to send the email.

The auditor cannot tell you either. That answer depends on the system, the users, the consequences of getting it wrong and what your organisation learns when people actually use it.

Over the next few years, companies will accumulate thousands of answers like this. The interesting question is whether those answers become institutional knowledge, or whether every project discovers them again.

At Faktion, we work with enterprises that have to ship AI while the governance around it is still being written. In practice, this means the system and the framework have to produce evidence together.

One of the first things worth checking is very simple: are the people building the system and the people governing it looking at the same evals? In most organisations, they are not yet. That is a much smaller problem than writing another fifty-page AI policy.

AI governance is engineering work before it becomes a document.