A man in British Columbia asked Air Canada’s website chatbot about bereavement fares. The bot told him he could book at full price and claim the discount afterward. He did. The airline then refused the refund, because its real policy said otherwise. In February 2024 a tribunal ruled in Moffatt v. Air Canada that the airline answers for what its own chatbot says, and awarded him $650.88 CAD plus interest and fees. Air Canada had offered a $200 coupon first.
The sum is tiny. The precedent isn’t. Anyone working with a Generative AI development company to build an agent that talks to customers has just learned who gets blamed when it’s wrong.
What Klarna and Gartner Say About the Hype
Klarna makes a handy example, and not a flattering one. In 2024 it announced that its AI assistant was doing the work of 700 customer service agents and handling about 75% of chats. By 2025 its CEO was telling Bloomberg that the push had gone too far, that quality had slipped, and the company started hiring human agents again. The 2024 numbers were company-reported. The reversal came from living with the thing.
Gartner sounds a similar note at a larger scale. It expects over 40% of agentic AI projects to be canceled by the end of 2027, mostly over cost, unclear value and weak risk controls. It also coined “agent washing” for vendors who relabel chatbots as agents. Worth knowing before you sign anything.
Why Agents Are Harder Than Chatbots
A chatbot answers. An agent acts: it calls tools, makes a string of decisions and can change things in other systems. One wrong step carries on through everything after it.
Ordinary unit tests miss the usual failures. An invented tool call, say. Or a model update that quietly makes things worse. Testing has to follow the agent step by step, and most first attempts don’t do that. And why would they? A demo only needs to work once.
The Tooling, Briefly
Most agent projects start from a framework. Comparison articles credit LangGraph with checkpointing and durable execution, and warn about its learning curve. CrewAI gets called the quickest route to a multi-agent prototype. The OpenAI Agents SDK is the minimal one, easiest with OpenAI models. The Model Context Protocol is becoming the usual way to connect tools. Almost all of that comes from vendors and agencies, with no independent benchmarks behind it, so read it as opinion. And none of it removes the testing problem.
When to Bring in a Generative AI Development Company
A few situations keep coming up. The prototype works and nobody can say how to make it dependable, because production needs evaluation sets, logging and guardrails that a first-time team hasn’t built. The agent will move money or touch customer records, which is the Air Canada position. Or a deadline is close, and a Generative AI Development Company that has shipped agents already knows which framework choices hurt later.
The catch is cost. Outside help isn’t cheap, and it only pays off if the work is scoped tightly.
When You Probably Don’t
Gartner’s own analysts said many use cases labeled agentic don’t need an agent. A lot of Generative AI Software Development is a fixed workflow with one language model call in the middle, which is cheaper and simpler to test. If the steps never change, an agent adds risk and nothing else. A team that already ships language model features can also manage a first low-risk agent, such as sorting internal tickets.
What Generative AI Software Development Looks Like for Agents
Teams that cope well start small. They pick one task and write down what “done” means. A person approves the risky actions, at least at the start. The test set gets built before the agent does, and every step gets logged, because nobody can fix a failure they can’t replay.
Testing won’t catch everything, though. One survey found about half of agent projects stuck at pilot stage, and another found few teams happy with their monitoring tools. Both were vendor-sponsored and small, so treat them as a mood, not a measurement.
Timing
When? Once a prototype exists, and before customers see it. The problem has a shape by then, and getting it wrong is still cheap. BiztechCS, which delivers AI/ML Development and GenAI Solutions for mid-market businesses, tends to start from the workflow and the point where a human signs off, then choose models and frameworks. That order suits Generative AI Software Development for agents better than the reverse.
