Your agent is live — now it's time to make it better. Improving AI answer quality is a process, not a one-time task. This guide gives you the exact playbook to diagnose weak answers and fix them systematically.
Diagnose before you fix
Before making changes, understand what's actually going wrong. There are three types of weak answers, and each has a different root cause:
| Symptom | Root cause | Fix |
|---|---|---|
| Agent says "I don't know" | Missing info in knowledge base | Add the missing content |
| Answer is vague or rambling | Source content is thin or unclear | Rewrite source content with specifics |
| Answer is factually wrong | Conflicting or outdated info in KB | Remove old content, add correct version |
| Answer is off-topic or weird | System prompt too loose | Tighten the system prompt scope |
| Answer is too long | No length instruction in prompt | Add "Keep to 2–3 sentences" to prompt |
The improvement loop
The highest-impact changes
If you're short on time, these three actions will produce the biggest improvement in the shortest time:
1. Add real pricing
Pricing questions are the most common reason agents underperform. If your knowledge base says "contact us for pricing", add real numbers. Even a starting price or range is dramatically better than nothing.
2. Write 20 manual Q&A pairs
Identify the 20 questions your customers ask most often. Write a clear, specific, complete answer to each one as manual entries in the knowledge base. This alone can transform your agent from mediocre to excellent.
3. Add a clear fallback and CTA to your prompt
Make sure your system prompt includes: (a) what to say when the agent doesn't know something, and (b) a call to action to push toward (booking link, email, next step). These two additions directly impact conversion.