Everything you trust is a black box

The most common objection to AI is that nobody can fully explain how it works. True. Now name one thing you use every day that you can.

By Ben Tannenbaum, Co-founder & CEO of AISO

Updated June 2026·Featured in Search Engine Land & Forbes·LinkedIn

There is a tidy argument going around that AI cannot be safe or reliable because it is a black box. We cannot trace every weight, we cannot narrate every step, therefore we should not trust the output. It sounds rigorous. It is also an argument that, taken seriously, would have you distrust almost everything you depend on, starting with your own brain.

You do not know how your memory retrieves a name. You do not know how aspirin stops a headache at the molecular level, and for most of the 20th century neither did the pharmacologists prescribing it. You board planes, swallow medicine, send money over networks, and run your own nervous system, all without a working model of the internals. What you have instead is something humans have always run on: a reliable enough map from input to output, and the intuition to act on it.

Bergson saw this coming a century ago

Henri Bergson spent his career on a distinction that turns out to be exactly the right tool here. He separated two ways of knowing a thing. One is analysis: you walk around the object, break it into parts, describe it from the outside in symbols and measurements. The other is intuition, by which he meant something precise and unmystical: knowing a thing from the inside, by direct acquaintance, the way you know how to ride a bicycle or read a friend's mood across a table.

His claim, and it ages well, is that you can possess deep, reliable, actionable knowledge of something through intuition while your analysis of it stays partial, clumsy, or absent. The cyclist cannot write the equations of balance. The native speaker cannot recite the grammar. The knowledge is real and it works. It simply does not live in the form of a complete external explanation.

Analysis multiplies endlessly the number of viewpoints on a thing. Intuition is a single act that puts you inside it. We have always lived inside far more than we can analyze.

Gilbert Ryle gave the same idea its cleaner English name two decades later: knowing-how versus knowing-that. You can know how to do something flawlessly without being able to state the rules that govern it, and stating the rules does not by itself give you the skill. Friedrich Hayek pushed it into economics: most of the knowledge that makes society function is tacit and distributed, held in habits and prices and practice, never written down, never centralized, and working fine precisely because no one needs the whole map to act on their corner of it.

Put the three together and the black-box objection starts to look backwards. The demand that we fully analyze a system before we trust it is not the normal condition of knowledge. It is the exception. The normal condition is reliable use without complete understanding. That is not a bug we tolerate. It is how knowledge has always mostly worked.

The question was never "do we understand it". It is "is the map reliable enough"

Once you stop demanding full transparency and start asking the real question, the criterion gets sharp and practical. You do not need to open the box. You need to know that the function from input to output is stable, tested, and accountable when it fails. That is a much more useful bar, and notice that it is exactly the bar we already apply to everything else.

We do not certify a drug by explaining its mechanism. We certify it with trials: same input, reliable output, side effects mapped, recourse when something goes wrong. We did not ground aviation until someone could derive lift from first principles; we built it on testing, redundancy, black-box recorders, and incident review. Reliability is established empirically and held accountable institutionally. Mechanism is a bonus, not the gate.

So the interesting move is not to win the philosophy argument. It is to notice what happens the moment a new black box becomes reliable enough that people start acting on it. Because that is happening right now, in a place most brands are not watching.

Which brings us to where this is no longer theoretical

People have started treating AI as exactly the kind of reliable-enough box they will act on without asking why. And nowhere is that clearer than in how they shop. They are not asking the model to explain its reasoning. They are asking it to decide.

We see this constantly in real, anonymized ChatGPT conversations on our consent-based panel. The pattern is not "explain the trade-offs so I can evaluate them myself." It is "you pick." People open with a request to be told the answer, and then they accept the named brands that come back without ever interrogating how the model got there. The recommendation lands as a verdict, not as an argument to be checked.

Here is one real conversation from the panel, lightly trimmed. A user wants a work laptop. Watch how fast the abstract question collapses into a list of specific products the user simply takes.

Now scale that. Across our panel, the brands that surface in these "just tell me the best" conversations are remarkably concentrated. A short list of names gets recommended again and again, and the user almost never asks for the reasoning. The table below is built from that pattern: the verticals where we see the most decision-by-deferral, and the kinds of brands that consistently get named.

Sit with what that means for a brand. The decision is increasingly made inside a box the buyer does not inspect and cannot interrogate. They asked for the best. The model named names. If yours was on the list, you won a sale you never saw coming. If it was not, you lost one you never knew you were in. No click, no impression, no line in your analytics. Just an answer, delivered as a verdict, in a conversation you had no visibility into.

The buyer treats the model's pick as reliable enough to act on. The only question left for you is whether the box says your name.

You cannot open the box. You can find out what it says about you

Here is the part that should feel less like resignation and more like an opening. The black-box objection is right that you cannot read the model's internal reasoning. But you do not need to. You can do the empirical thing, the same thing we do with every other reliable-enough black box: study the inputs and outputs at scale until the behavior becomes legible.

Ask a brand's real buying questions across the model, thousands of times, and the box stops being opaque in the way that matters. You see which competitors get named when yours does not. You see the phrasings where you are invisible and the ones where you lead. You see the moment in the conversation where the shortlist forms and whether you make it. You will still never see the weights. You will see something more useful: the verdict, and how to change it.

That is the whole game now. Not explaining the box. Reading what it says about you, and making it say something better.