Most arguments about AI start in the wrong place.
They start with the model. Is it capable? Is it safe? Is it cheap? Can it write, code, summarise, classify, predict, route, answer, persuade?
Those questions matter. They are not enough.
The harder question is what happens to the person on the other side of the system.
Does the tool make them more capable? More able to understand the work, make a decision, learn from a mistake, help another person, and take responsibility for an outcome?
Or does it make them smaller?
Smaller can still look productive. A person can send more messages, close more tickets, produce more drafts, and appear wonderfully efficient while exercising less judgment, learning less, and having less room to decide what the work is for.
That is the risk in treating AI as a productivity story alone. Productivity tells you that more happened. It does not tell you what happened to the human being doing the work.
Capability is not the same as output
Imagine two versions of the same assistant.
In the first, it helps a new employee understand a difficult task. It shows the relevant source material. It explains its recommendation. It lets the person test a decision against realistic cases. It makes the expert practice of the best people more available without pretending that the expert is no longer needed.
In the second, it gives the employee an answer, scores how quickly they act on it, and sends the result onward. The employee becomes a relay between an opaque system and a customer who assumes a human understood what happened.
Both systems may save time.
Only one is building a person.
This is not a sentimental distinction. It is operational.
The first system creates more people who can handle the next difficult case. The second creates a dependency: the work looks faster until the model is unavailable, wrong, or confronted with a situation it has not seen before. Then the organisation discovers that the human was never given enough understanding to take over.
The question behind “human in the loop”
“Human in the loop” sounds reassuring because it includes the word human.
But a person clicking approve after a system has made the meaningful decision is not much of a loop. They are a liability sponge with a login.
The real question is not whether a human touched the process.
It is whether the human has enough context, time, authority, and skill to disagree.
Can they see what the system relied on? Can they recognise when the answer is out of bounds? Can they override it without being punished for slowing the workflow? Can they explain the outcome to the person affected by it?
If not, the system has not augmented judgment. It has hidden judgment behind a faster interface.
That matters for trust. It matters for learning. It matters for the person whose name ends up on the decision when things go wrong.
There is no neutral implementation
AI systems distribute leverage.
They can give a junior person access to the patterns that used to be trapped with an expert. They can give a small team the ability to attempt work it could not previously afford. They can let someone with an idea make it concrete before they have permission, budget, or the confidence to explain it perfectly.
They can also concentrate leverage.
The people who get the best models, data, permissions, and time to experiment become more capable faster. The people who are given a restricted interface and a larger target become more measurable faster. Neither outcome is inevitable. Both are design choices.
That is why the question cannot be answered by a model benchmark or a quarterly usage chart.
The OECD's survey of more than 5,000 SMEs found that generative AI was commonly reported to improve performance, while respondents were more likely to report increased need for highly skilled workers than decreased need. The point is not that AI somehow rewards an elite. The point is that the human skills around the tool—interpretation, creativity, judgment—remain central, and the organisation decides who has the chance to build them. OECD
A better test before scaling a system
Before an AI system expands, ask five questions.
- What will this person understand after using it that they did not understand before?
- What judgment are they still expected to exercise—and are they being given the evidence to exercise it?
- What happens when they disagree with the system?
- Who becomes more capable if this works, and who becomes easier to measure or replace?
- What will the team be able to do without the system a year from now?
These are not anti-automation questions. They are the questions that make automation worth doing.
We will build systems that can produce more work than people can read. That part is not especially mysterious.
The more consequential choice is whether we use them to make people more thoughtful, more capable, and more able to own the world they are helping to change.
Or whether we use them to make people smaller, then call the result efficiency.