Your team calls the AI a black box. So how do you trust what it decides?

Someone on your team called the AI a black box last month. They were not wrong, and you approved its use anyway. So now it drafts the client note, scores the applicant, flags the risky invoice, and a quiet question sits under all of it. What is it actually doing when it decides? You can see the answer. You cannot see the reasoning. And you are the one who signs.

That gap is the real source of the unease. Not the fear that the model is stupid, it plainly is not, but the fact that you are being asked to trust a judgement whose working you cannot inspect. Every experienced leader knows the feeling from hiring: the confident answer with nothing behind it. You learned to ask people to show their reasoning. Until recently, you could not ask that of the machine.

Why has "it's a black box" been the honest answer?

Because it was true. A large language model (the kind of AI that writes and reasons in words) works through billions of numbers shifting inside it. You could read what went in and what came out. The middle was dark. So when a board asked whether an AI decision could be trusted, the careful answer was: we can test its outputs, but we cannot watch it think.

That is a real limit for anyone accountable. Auditors want the working. Regulators want the working. You want the working, because your name is on the call. It is one reason trust in AI runs so far behind its adoption. An answer you cannot interrogate is a decision you are taking on faith, and faith is a thin basis for a capital-grade choice.

What actually changed in 2026?

A frontier lab found a way to look inside. In July 2026, Anthropic (the AI company behind the Claude model) reported an emergent internal structure it named the J-space. Think of it as a small mental sketchpad: a few dozen concepts the model is holding, each tied to a word it might say next. It was not designed in. It formed on its own as the model learned.

What makes this useful is the second part. The team built a lens that reads that sketchpad, and it caught the model in the act. When the model fabricated a figure, the reading for "manipulation" lit up as it typed the false number. The output looked clean. The inner state did not. A separate researcher outside the company reproduced the core finding on open models, which is how a claim earns its keep.

There is a deeper echo here, and it is worth naming plainly. It points at the gap that quietly decides which AI strategies work, which sits in people and judgement more than in code. The internal workspace mirrors a leading neuroscience theory of how conscious attention works in people. It is called global workspace theory: the idea that the mind broadcasts a few selected thoughts to the whole brain at once. The researchers are careful: they are reading structure, not claiming the machine is aware. For a leader, the point is smaller and more practical. The inside of the model is turning from a rumour into something you can measure.

So what should a leader do with the black box question now?

Stop asking which model to buy, and start asking what each model will let you see. That is the better question, and almost no one arrives at it. Leaders come in asking which tool is safest, or which vendor is biggest, or how to cut the cost. Those are the low questions. The high one is this: for the decisions that carry weight, can I read why the machine chose what it chose?

You do not need to become a scientist to act on that. You need to treat readability as a control, the way you treat accuracy and cost. Build it in this order:

1. Sort your AI decisions by stakes. A drafted email needs no scrutiny. A credit call, a hiring screen, a safety flag does. Match the depth of inspection to the weight of the decision.
2. Ask vendors what they can show you of the reasoning, not just the output. The answer tells you how seriously they take the problem you are accountable for.
3. Keep a named human on the weighty calls. The tool proposes; a person owns. Interpretability supports that person; it does not replace them.
4. Write down what "good enough to trust" means for each class of decision, before you are in the room defending one.
5. Build the literacy first, then buy the next capability. The leaders who win the next decade are the ones who upgrade themselves first.

Where does this leave you a year from now?

Picture the next board meeting where an AI decision is questioned, one of the questions a board should be asking before it invests. Today you would defend it on results alone, hoping the pattern holds. Soon you will answer differently. You will be able to say what the model weighed, show where its attention sat, and point to the check that catches it drifting. Same technology. A completely different footing to lead from.

This is the quieter promise inside the noise. The models are levelling off into commodity, and everyone has the same ones. The edge moves to the leader who can see underneath the answer and decide, with reason rather than faith, whether to trust it. The black box is opening. The opportunity is to be the one who knows how to look, and to lead a business that has truly become AI-native, one that trusts its machines with its eyes open.

About the author

Thomas Green

British technology futurist, AI keynote speaker and advisor. Thirty years across enterprise technology and AI strategy, helping leaders navigate the future of work. The futurist who died.