Key takeaways
- 58% of tier-1 tickets resolved without a human after phased rollout.
- Citation-backed RAG reduced “bot loop” complaints versus macro-only bots.
- Shadow mode de-risked launch before full traffic.
- Eight weeks included policy design—not just model tuning.
Northline SaaS came to us after a pricing change doubled support volume. Macros were stale, contractors could not access internal runbooks safely, and leadership wanted deflection without damaging CSAT.
We scoped a single outcome: resolve tier-1 billing and how-to tickets with cited answers, escalate everything else with full context. No billing changes without human approval—non-negotiable from week one.
Weeks one and two were knowledge audit, not coding. We indexed 340 help articles, removed contradictory macros, and built an evaluation set from real transcripts. Policy matrices defined PII handling, refund boundaries, and when confidence scores must route to people.
Weeks three through five shipped the agent, RAG pipeline, and Zendesk bi-directional sync. Every reply included source citations so agents and customers could verify answers. A QA harness replayed the eval set on every deploy.
Weeks six through eight were shadow mode, then 25% → 100% traffic with weekly reviews on live conversations. We tuned prompts from won escalations—not from synthetic demos.
Results at launch: 58% auto-resolution, 72% faster first reply, CSAT held at 4.6★. The project finished in eight weeks because discovery locked scope early and integrations were prioritized over UI polish.
If you are planning a similar agent, copy the sequence: audit content, define permissions, integrate where work already happens, eval before scale, and measure deflection plus satisfaction—not chat volume alone.
Ready to build?
Turn the idea into an AI system your team can operate.
We help companies design agents, automate workflows, integrate existing tools, and ship with testing and governance built in from day one.
More from the blog
All articles- Software Development
Vibe-Coded App Security Checklist Before You Launch
Exposed keys, weak auth, and open databases break AI-built apps in production. Use this checklist before real users arrive.
- AI Strategy
ChatGPT Built My App—Now What?
Your AI-generated app runs in demo. Here is how to reach production without exposing keys, users, or revenue.
- AI Strategy
How Much Does AI Automation Cost in 2026?
Realistic pricing for agents, workflow automation, and integrated software—without agency fluff.