Customer says no
I spent a few days again this week working through an idea with an AI. I’m sure I’m not alone in not writing prompts anymore, never mind clever, precise ones. I just talk to it in long, meandering voice notes, full of tangents, loosely connected thoughts and hypotheticals that would test the patience of all of my (remaining) friends. Alongside all that material, however, the most useful things I know I said were ‘nopes’: refusals, and fairly blunt ones at that.
“No, and if you read it again, you’ll see why.” “No, that makes no sense at all.” I didn’t need to give a reason. The model went back, found the problem or conflict, and the work improved more than it did when I asked for additions.
Popper described knowledge as growing through conjectures and refutations. A large language model is, in effect, a large conjecture engine. It produces plausible material faster than anyone can read it. And most of us work with it in long threads that keep adding what we want, or think we want. Those threads rarely turn to what we don’t want and to the power of negation. Just saying no would help.
What the model can’t reliably do is know when something is wrong for a specific situation. It’s trained to produce what is plausible, but the best it can ever do is plausible conjecture. It works by inclusion: given an idea, it finds material that fits and builds on it. It has also been trained partly on human approval, so it leans towards agreeing with whoever it’s working with. Put those together and you get confirmation bias at scale and a truckload more biases tbh. Get it to check its own work and it uses the same processes that produced the work in the first place, so it tends to find reasons the work is fine. And it has no contact with the situation itself, which is where the useful negation would come from.
Now read that paragraph again with ‘the enterprise’ in place of ‘the model.’ Does it still hold? Pretty much word for word. Organisations produce plausible plans and work by inclusion, building on what has already been agreed. They’re shaped by approval, from sign-offs to steering groups. They check their work through their own governance, which tends to find reasons it’s all fine. And the further work travels from stores and screens, the less contact it has with the customer’s situation. Many of the people shaping it have never met a customer, stood in the store or lived with the consequences of a bad decision. An enterprise is a large conjecture engine too, and putting AI inside one puts one conjecture engine inside another. That’s not just confirmation bias at scale, that’s hyper-confirmation-hyper-bias. Or something.
What’s good to know however is that the ‘no’ doesn’t need to be detailed to be useful. It doesn’t have to supply the right answer, or even explain what’s wrong. It only has to rule something out. That’s working by negation rather than inclusion, what philosophers call apophatic: knowing something by what it isn’t. And it’s the part the model struggles to do for itself. Actually, it’s not just a struggle, it simply can’t do it reliably for itself.
Most people using AI in the enterprise haven’t yet realised how useful and powerful a ‘no’ is. Once you have it, the large conjecture engine works better, whether you’re using it at home or at work.
But here’s the thing. Once you’re inside the enterprise, a “no, that’s crap” counts for far less than it does at home. Me saying no, or you saying no, to our instances of an AI model in the enterprise is still plausible conjecture. Nothing has been ruled out. We are still in the enterprise. And a steady flow of internal nos and yeses will run upstream to downstream for the next fifteen weeks of a sixteen-week programme before anything ships. All of that work is conjecture and always has been but now it’s on steroids.
The ‘nope’ that matters comes outside and comes from the customer. Customers don’t care how the enterprise works inside, they aren’t weighing up its strategy, and they aren’t interested in the steering groups and sign-offs. They just do or don’t. A nudge ignored, a substitution rejected, a loyal customer who stops coming: solvitur ambulando, the answer is given by walking, and it’s given all the time.
So the enterprise AI, this large conjecture engine, needs the one no it can’t produce for itself, customer-says-no. But don’t forget that it’s no good me or you just telling AI that the customer said no. Because that’s hearsay, with no provenance, and also relies on everyone remembering to say it, consistently, in every conversation they have with an AI. They won’t and can’t. The customer-says-no has to come from the source. So how does the AI get the customer-says-no, with provenance intact, at any and every moment it’s needed along the way?
That’s the problem I’ve been working on, and it’s this stratum.how