Insights

The Case Against Complex AI Pipelines (And What We Built Instead)

The Short Answer

We built a monolithic prompt system, then a four-stage pipeline with classifiers, routers, and quality gates. Both failed. The system that worked had one AI call, no routing, and took an afternoon to build. Complexity made every failure harder to diagnose and every success more fragile. If your AI system is getting harder to understand, that's a signal — not a milestone.

Sonia Ribas has been a fertility coach for over 15 years. She's helped 700+ families have babies. Her clients come through Instagram and WhatsApp DMs — someone sends a message, a conversation begins, and if it goes well, they book a discovery call.

The problem: message volume was growing faster than her team could handle. Most conversations followed familiar patterns. Grief. Failed IVF. A PCOS diagnosis. A partner who isn't on board. Someone who has tried everything and lost hope.

Sonia knew exactly what to say in each situation. She'd said it hundreds of times. She had a point of view, a voice, a way of making people feel heard and then gently guided toward what was actually possible.

The question was whether an AI could do what she does. Not a chatbot. Not a FAQ. A real conversation — empathetic, specific, expert — that ends with the right people booking a discovery call.

We said yes. Then we built three different systems before one actually worked.

The First System: Just Tell It What to Do

The first instinct seemed obvious. Write a detailed system prompt. Tell the AI exactly how to behave.

We told it: be warm but not a therapist. Never give medical advice. Ask at most one question per reply. Don't open with "Hi, I'm glad you reached out." Reference what the person has already shared. Match the emotional weight of what they're saying. Don't push toward booking too early. Use short, conversational sentences — this is Instagram DMs, not email.

We ran test conversations. The AI did things we didn't want. We added more rules. It did other things we didn't want. We added more rules for those too.

By the time we were done, the system prompt was 600 words. And it still wasn't working.

The AI would follow a rule for a few turns, then drift. It would say "I'm sorry to hear that" — a phrase we'd explicitly banned — because it had processed the ban as text but hadn't internalized the reason behind it. It would give generic empathy when someone shared something devastating, check all the technical boxes, and still feel completely hollow.

The hardest lesson: the more rules we added, the worse it got. The model was cognitively overloaded — simultaneously trying to write warmly, avoid 15 banned phrases, classify the scenario, time the booking invitation, ask exactly one question. Somewhere in all of that, Sonia's voice got lost.

A 600-word system prompt isn't a sign of thoroughness. It's a sign you don't yet understand the problem.

The Second System: Break It Into Stages

If the monolithic approach was failing, the solution seemed to be breaking it apart. Classic engineering thinking.

We designed a four-stage pipeline:

  • A classifier that reads the message and decides what kind of response is needed
  • A strategy engine that determines the goal for this turn
  • A writer that only writes — no classification, no strategy
  • A quality gate that reviews the reply before it goes out and fixes any rule violations

On paper it was elegant. Each stage had one job. Each job could be optimised independently.

We built it. We tested it. The conversations were worse than before.

The voice felt fragmented and incoherent. When you compose a reply through four different AI calls, you accumulate small errors and lose the thread. And when something went wrong — which it still did — diagnosing it meant tracing through all four stages to find where the breakdown happened. The whole system became opaque.

We scrapped the entire thing in one afternoon.

What We Were Getting Wrong

Somewhere in the middle of all this, we stopped and asked a different question.

Not "how do we tell the AI what to do?" but "how does the AI actually learn?"

These models weren't trained on rulebooks. They were trained on enormous quantities of human writing — conversations, stories, articles — and they learned by absorbing the patterns in that writing. Not the rules. The patterns.

We had been treating the AI like a new employee — handing it a policy document and expecting compliance. But it doesn't work that way.

Two principles changed everything:

Examples over rules. Show the model the behavior; don't describe it. A model that has seen 13 examples of how Sonia handles a grief conversation will handle grief conversations like Sonia. A model that has read a rule about grief conversations will follow that rule until something disrupts the pattern — and then it's lost.

Reasoning over instruction. Tell the model why, not what. A model that understands Sonia's reasoning — her belief that the body isn't broken, just not fully supported; her instinct to listen before advising; her understanding that trust has to come before any invitation to book — will apply that reasoning to situations you never anticipated. A model that has memorised rules will fail the moment the situation doesn't exactly match.

What We Built Instead

We wrote 13 conversations in Sonia's voice. Real scenarios: a miscarriage, a failed IVF cycle, a PCOS diagnosis, a partner who isn't on board, someone who has tried everything and given up hope. Each one was a complete example of how Sonia actually talks — the rhythm, the warmth, the way she holds someone's pain and then gently shifts the frame.

These went into every AI call. Not as instructions. As demonstrations. You can read more about how we structured those example conversations and why this technique works at the level of how LLMs actually learn.

We also rewrote the core prompt as a portrait of Sonia's thinking — not a list of dos and don'ts. Her belief that the body isn't broken, just not fully supported. Her instinct to listen before advising. Her understanding that trust has to come before any invitation to book a call.

The final system: one clear description of who Sonia is, 13 example conversations, one AI call per message. No routing. No classifiers. No stages.

One piece of logic we kept in application code, not in the prompt: the pricing reveal sequence. First time someone asks about price — redirect warmly to the discovery call. Second time — give the range ($1,500–$10,000 depending on level of support) and return to fit and personalisation.

This belongs in code because it's state-dependent. That distinction — what belongs in the model versus what belongs in code — is worth more than most architectural decisions.

What This Cost and What It Replaced

The client is a high-ticket fertility coaching brand with a large Instagram following. The system handles initial DM conversations at scale — grief, fear, skepticism, exhaustion — and moves the right people toward a discovery call in Sonia's voice.

A project of this scope at US AI engineering rates typically runs $8,000–$15,000 depending on conversation complexity, the number of example scenarios written, and integration requirements. This build sat at the lower end of that range — because the final architecture was genuinely simple.

Three Things We Take Into Every Build Now

1. If the system prompt is over 300 words, we have a prompt problem, not a model problem.

2. State-dependent logic belongs in code, not in language model instructions.

3. The diagnostic question for a failing AI system isn't "what rule are we missing?" — it's "what example haven't we shown it?"

Frequently Asked Questions

What should a good AI pipeline design include?

A good AI pipeline design should include only as many stages as the problem actually requires — which is usually fewer than you think. The most common mistake is adding infrastructure (classifiers, routers, quality gates) to compensate for a poorly designed prompt or insufficient examples. Start with the simplest possible system, measure where it fails, and add complexity only when subtraction has been exhausted.

When would you choose a single agent over a multi-agent approach?

Choose a single agent when the task can be fully resolved in one context window and doesn't require parallel execution or independent verification. Multi-agent architectures add coordination overhead, compounding error rates, and significantly more complex failure modes — the benefits only outweigh those costs when tasks are genuinely parallelisable or require specialised sub-agents that can't share context effectively.

What is the difference between single agent and multi-agent AI systems?

A single agent handles a task within one model call, with full context available throughout. A multi-agent system distributes the task across multiple model calls — each agent sees a subset of context and passes outputs to the next stage. Multi-agent systems can handle larger or more parallelisable workloads, but they introduce coordination complexity and make failures harder to trace. For most production use cases, a well-designed single agent outperforms a fragmented multi-agent pipeline.

How do I know if my AI system is too complex?

The clearest signal is that failures are hard to diagnose. If something goes wrong and you can't tell which stage caused it, the system has more layers than your current understanding justifies. A second signal: output quality hasn't improved proportionally to the infrastructure added. Complexity should earn its place. If the simpler version would produce comparable results, it's the right version.

Building an AI system that isn't behaving the way you expect?

If you're building an AI system that isn't behaving the way you expect, we've been through the complicated version. Book a call and we'll tell you what we'd actually do.

Book a call →