There is a shift playing out in enterprise AI, and most of the roadmaps I see are still pointed at the wrong part of it. Foundation models are widely available — GPT-5, Claude, Copilot, and the rest. Any company can reach them through an API or a platform with very little friction. The model is no longer the hard part. The hard part is integration: wiring those models into the workflows and systems where work actually gets done, in a way that creates business value rather than another impressive demo.
Put simply: the LLM is the interface. Integration is the product. And efficient integration is the business.
Most pilots can read. Few can act.
Most GenAI pilots do the same three things well. They read, they summarize, and they answer questions. That is real, and it is where almost everyone started, correctly. The pressure now is to move to agents and automated workflows — connecting steps across a CRM, a call center, an R&D pipeline — because that is where the return is and where things are clearly heading.
The problem is that the core systems these workflows have to touch were never built to work with modern AI. We keep trying to force old integration patterns into new workflows, and it breaks quickly. You end up with AI that can produce insight but cannot touch the systems that actually move work forward.
The model usually isn’t the problem. The ecosystem is.
Inside large organizations I keep seeing the same blockers, and almost none of them are about model capability:
- Legacy systems with limited or no usable APIs. The system of record can’t be reached in the way a modern workflow needs.
- Data scattered across ERPs, CRMs, and departmental platforms in incompatible formats, with no consistent metadata.
- Security controls that interrupt context and flow — necessary, but rarely designed with AI workflows in mind.
- No real-time paths into core systems — only batch jobs and aging middleware.
- AI allowed to observe but not act, because testing, trust, and model quality haven’t been resolved.
- Token costs that spike because the workflow was never engineered for scale.
Most GenAI failures I’m called into trace back to architecture, not to the model. Executives want automation and decision support. You don’t get either without deeper integration into systems of record — and that integration is engineering work, not a prompt.
The token trap is real, and it’s an architecture problem
There is a specific failure I’ve watched play out more than once, usually late in a pilot, when a team is getting ready to scale from a handful of users to real production volume. Someone finally asks: “What do we do to cap our token usage?” And too often the team isn’t prepared for the question.
The enthusiasm for agents and orchestration makes sense — chaining steps across systems is genuinely powerful. But every step that involves an LLM call consumes tokens, and cost can scale exponentially as you chain multiple agents together or give them too much autonomy. This isn’t hypothetical. Some AutoGPT users reported that even modest 50-step runs pushing near-maximum context could produce significant cost per run. Enterprise pilots with real workflows have hit the same wall, with token costs ballooning past budget mid-pilot. Token burn is a blind spot in the “AI workflow as product” thinking, and it shows up exactly when you’re trying to prove the thing works at scale.
The companies that win won’t be the ones that integrate the most agents. They’ll be the ones that integrate efficiently — hybrid architectures that pair discriminative models with generative ones, caching layers, routing, and the discipline to not send everything to the largest model. Integration is the product. Efficient integration is the business.
Where leaders should focus
If your AI pilots are stalling, start with the integration layer, not the model:
- Modernize API access for the systems that matter most.
- Build a stable data layer with consistent metadata, so high-value data is reusable instead of trapped.
- Integrate security and compliance into AI development from the start, rather than bolting them on at the end.
- Use hybrid model patterns to control cost, and fine-tune with your own data where it genuinely makes sense — I’ve built those models in biopharma, healthcare, and telecom, and when the use case fits, they change the economics.
- Design AI to take action, not watch from the sidelines. Observation is where pilots live; action is where value is.
The organizations making real progress aren’t chasing the next model. They’re fixing the architecture around it. That’s the unglamorous work, and it’s the work that decides whether enterprise AI ever gets past the pilot.