I wanted to do a double-click on one aspect of this week’s article for you, my paid subscribers.
In the article, we talked about the time that goes into alignment, which here means getting the business and IT to agree on what the thing should do. There’s the usual scope negotiation. And running underneath all of it is an argument about token cost, on the theory that more instructions means more tokens means more expense.
BUT. Nobody in that argument had produced a number. Not a benchmark, not a test run, not a per-call estimate. The token concern was operating as a belief.
Here’s why I want to call this out. Token cost is measurable, and you can find out in an afternoon.
I’ve watched this vocabulary before. In a previous life I sat on the business side of a lot of IT conversations, and I know what it sounds like when a technical constraint becomes a scope-control device. ‘That’s not in the architecture.’ ‘That would require a change request.’ ‘We can’t do that.’ Those sentences are sometimes true and always useful, because they end conversations without anyone having to say no.
(Early on in my career, I was told bolding text on a web page would take three weeks. 🤦🏼♀️ So, I’ll fully own my baggage here.)
Token budgets are the newest member of that family. Real enough to be unarguable, vague enough that nobody checks. And it arrived inside AI work, which was supposed to be the thing that dissolved this dynamic.
There’s a second area I want to highlight. Across months of arguing about how to write instructions for a language model, nobody put the question to the model. That’s a task these systems are good at. Draft it, critique it, compress it, tell me what this costs per call. An afternoon.
So why didn’t anyone? Because often, these discussions are being treated as a negotiation, and you don’t hand a negotiation to a tool.
The antidote to situations like this is two-fold.
