People occasionally ask why I am so unromantic about AI agents, given that designing and running them is what I do for a living now. The honest answer is that I have already watched a perfectly competent optimizer do real damage to an operation, from close range. The failure mode is old. It has better branding now, and it runs faster, but it is the same failure.
The three weeks I still think about
In 2012, a carrier deployed a “smart” dispatch system meant to optimize technician routing. It failed spectacularly within three weeks.
I spent roughly a decade routing technicians to cell towers and fiber nodes, and I was there for this one, ticket by ticket. Two things it did have stayed with me ever since. It sent diesel generator specialists to fiber splice jobs. And it routed techs past emergencies to handle routine tickets.
Neither of those is a subtle error. A generator specialist standing in front of a fiber splice is not a slightly suboptimal assignment — it is a wasted truck roll, a ticket that keeps aging while someone competent gets pulled off whatever they were already doing, and a technician whose day was spent proving that the assignment was impossible. Routing past an emergency is worse, because the routine ticket the system chose looked better on the scorecard. Fewer miles. Tighter sequence. Cleaner map. Every one of those decisions would have made a human dispatcher put the phone down and walk over to someone’s desk.
The optimizer was working exactly as designed
That is the part people tend to get wrong when I tell this story. They assume the system was broken. It wasn’t. It was doing precisely what it had been built to do, and doing it competently. The design simply did not know what the work was.
Nobody had encoded that “technician” is not a fungible unit of labor — that the word covers people with different certifications, different trucks, different tooling, and genuinely non-overlapping skills, and that some of those distinctions are not preferences but hard walls. Nobody had encoded that a ticket’s stated priority is not the same thing as its actual urgency, or that certain classes of work have to jump the queue regardless of what that does to the route.
So the system optimized the metric it was handed, and it got very good at the wrong thing. Three weeks is not long to lose the trust of a field organization, and once you have, the techs start working around the system — its own kind of failure, and much harder to see in a report.
Most AI agent deployments fail the same way
I now spend my days building agents that take real action inside real businesses, and I see this repeatedly. The model is competent. That is almost never where the problem lives. What is missing is that nobody wrote down which constraints are hard.
Which specialist can actually do which job. What separates an emergency from a routine ticket. Which actions can be taken in which order, and which ones can never be taken at all without a human in the loop. Absent that, the system optimizes the objective it was given and produces decisions no experienced operator would ever make — while reporting perfectly good numbers, because the numbers it was given to report do not measure the thing that just went wrong.
And this is not a model problem. A better model does not infer your escalation policy, or know that one customer sits under a contractual response window and another does not, or that an action available in the software is forbidden inside the organization. That knowledge lives in the head of whoever does the job today, and it has usually never been written down, because the people who hold it have never needed to.
What the dispatchers knew and never said out loud
The good dispatchers I worked with were not running better math than the optimizer. They were running worse math against a much better model of reality. They knew which two techs should not be sent to the same site. They knew that a particular tower was a two-person job regardless of what the ticket said. They knew when “routine” actually meant “this will become an outage tonight if we leave it.”
None of that was in a system of record. It surfaced in hallway conversations and in the pause before someone assigned a job. That tacit layer is what an automation project quietly deletes when it replaces the human step, and it is why so many deployments feel fine right up until they are not.
The list worth more than another tuning round
So here is the practice I have landed on, and it is deliberately unglamorous. Before deploying an agent, write down the calls a good operator would never make.
Not the goals. Not the happy path. The prohibitions. Sit with the person who does the work today and ask them what they would refuse to do, and why, and keep asking until they run out — because the first three answers are the obvious ones and the useful ones come after that. Write each as a hard boundary rather than a preference, because an optimizer treats a preference as something to trade away. Then separate the genuinely inviolable from the merely strong, and be honest about which is which — calling everything inviolable is another way of encoding nothing.
That list is your constraint set, and it is worth more than another model tuning round.
I will concede the obvious limit: your first constraint set will be incomplete. Mine always are. You discover the missing entries in production, which is why I would rather run an agent narrowly with a visible refusal path — where it stops and hands the decision back — than run it broadly and find out from a customer. Every refusal is a candidate constraint. Treat the list as something that grows and it becomes the most durable asset in the deployment; treat it as a one-time document and you are back to 2012 with a faster optimizer.
Where this work actually lives
This is the lens I bring to every engagement, delivered through the practice I built for this work: Interactive Intel. It is a small, Miami-based team, and we design, build, and run production AI agents for SMEs and healthcare practices — lead intake, scheduling, documentation, back-office reconciliation. Constraint elicitation is not a phase we bolt on at the end; it is most of the early work, and you can see how we frame it on our agentic AI consulting page.
The entry point is intentionally small: a fixed-price AI Opportunity Scan — one workflow, two weeks, the payback math in writing before you commit to anything larger. Every engagement is led by me personally. And if the constraints on your workflow turn out to be the kind a machine should not be making decisions inside of, I will tell you that plainly, because I have seen what it costs to find out the other way.
Related reading: AI for medical practices · AI consulting for small business