You’ve probably felt the panic. Your AI feature is live, it’s saving your team hours of manual work every day, and then suddenly—it tells a customer the wrong refund policy. Or worse, it hallucinates a pricing tier that doesn’t exist.
Your first instinct? We need a bigger, smarter, more expensive model.
You’re wrong.
I recently finished a 30-day enhancement on a travel OTA AI system. The system was already working—handling channel operations, dynamic pricing, and competitor analysis in seconds. But one sub-feature, an “AI Operations Assistant,” was a massive liability. It ran on external public cloud APIs. If it hallucinated, operations staff would take that wrong answer and use it to change prices or approve refunds, directly amplifying business risk.
Most teams would push for a total system rebuild. I refused. We didn’t need a new system; we needed to fix one specific feature. We scoped it tightly: enhance accuracy, control, and resilience. That’s it.
This wasn’t just a win on paper. Alibaba’s travel platform, Fliggy, sent their team to study how we built and ran this system. They didn’t care about our cool models. They wanted to know how we made AI actually work in a live, high-stakes operations chain.
Here is the playbook that made it happen.
Stop Blaming the Model, Start Fixing the Context
Stakeholders kept saying the AI “wasn’t smart enough.” As a product manager, your first job is to translate that vague frustration into measurable gaps. I ran a blind test and found a 40% error rate.
But when I analyzed the failures, I realized the model wasn’t dumb. It was retrieving the wrong documents. Asking about pricing was pulling up marketing copy just because the word “commission” matched.
RAG doesn’t fail because the model is small; it fails because you fed it the wrong context.
We didn’t upgrade the model. We split the knowledge base into four strict domains: Channel Rules, Pricing, Marketing, and FAQs. We told the system to only look for pricing answers in the pricing domain. Hit rate jumped from 2.7% to 100% instantly. Zero model changes required.
The Rerank Trap: Why I Turned It Off
Everyone thinks Rerank—the process of re-sorting retrieved documents by relevance—is the magic bullet for AI accuracy. I tested it. My top-3 relevance score jumped 28.6%. Sounds great, right?
But when I dug into the data, some questions still scored zero. Why? The knowledge base literally didn’t have the answer. Rerank sorts garbage; it doesn’t create missing knowledge.
A 28.6% improvement in accuracy means nothing if the underlying knowledge base is missing the answer entirely.
So I made a counterintuitive call: I turned Rerank off by default. Instead of celebrating a vanity metric, we focused on filling the actual knowledge gaps. Attributing the root cause of the failure was infinitely more valuable than the temporary bump in the relevance score.
The “Private Deployment” Lie
To fix the control gap, we moved from external APIs to a localized open-source model (Qwen2.5-7B) running on a single 8GB RTX 4060. We built a dual-deployment system with automated failover.
Every vendor in the industry loves to slap “Private Deployment” on their slide deck. They promise your data never leaves your intranet. But here is the reality: architecture diagrams don’t prove safety. Process-level network audits do.
“Data doesn’t leave the intranet” is a lie until proven by a network connection audit, not a deployment diagram.
I didn’t just trust the setup. I ran process-level network audits, capturing every external connection the AI service attempted. The result? Zero public connections. Only then could I look stakeholders in the eye and say the system was truly controllable.
Draw the Boundaries, Don’t Sell the Magic
When it was time to go live, I didn’t just show off a flawless demo. I built an L0-L5 acceptance checklist with 21 specific scenarios. We tested infrastructure health, API contracts, AI quality, dual-link switching, and performance ceilings.
We explicitly documented where the AI would fail, why it would fail, and what the fallback behavior would be. We told stakeholders exactly what the system couldn’t do.
That is the real value of a product manager in the AI era. It’s not about chasing the shiniest tech or building the most expansive system. It’s about ruthless problem decomposition, scoping tightly, and setting boundaries.
Professionalism isn’t about building an AI that knows everything; it’s about knowing exactly where it will fail.
Stop trying to rebuild your entire tech stack. Find the riskiest feature, isolate the variables, and prove its boundaries. That’s how you build AI that businesses can actually trust.
FAQ
Q: If the model isn't the problem, why do AI products still fail to answer basic questions?
A: Because teams treat RAG as a magic wand instead of a structured retrieval process. If your knowledge base mixes marketing fluff with hard operational rules, the model will retrieve the wrong context. You have to isolate your data domains first.
Q: What's the practical takeaway for a PM managing an AI feature right now?
A: Stop chasing vanity metrics. A 28% boost in reranking scores is useless if the underlying knowledge base lacks the correct answer. Trace errors back to their root cause—missing data vs. poor sorting—before you touch a single model parameter.
Q: Isn't 'private deployment' just a marketing buzzword at this point?
A: Absolutely. Anyone can claim data stays on the intranet. Unless you have process-level network connection audits proving zero outbound calls to public APIs, your 'private' setup is just a theoretical promise. Verify the connections, don't trust the architecture diagram.