Background
A global data, analytics, and technology leader aimed to improve customer service efficiency by automating responses to frequently asked questions. With millions of queries spanning financial products, identity protection, and credit services, the company required a secure, scalable, and intelligent chatbot solution. The goal was to reduce customer support workload, ensure reliable and safe responses, and create an auditable system for compliance and governance.
Challenges
The organization faced significant obstacles in scaling its customer FAQ support:
- High Query Volumes: Manual support channels were overwhelmed, creating delays and inconsistent customer experiences.
- Accuracy Concerns: Generic chatbots often hallucinated, providing incomplete or misleading responses.
- Security & Safety Risks: Responses needed to comply with strict content safety, regulatory, and audit requirements.
- Limited Transparency: Lack of visibility into document lineage and retrieval processes reduced trust in outputs.
- Tuning Complexity: Optimizing retrieval and generation pipelines required flexibility without complex coding.
Solution
Narwal designed and implemented an Advanced RAG Chatbot on Google Cloud Platform using Vertex AI, GCS/BigQuery, and ipywidgets:
- Unified Platform Architecture
- Leveraged Vertex AI (Gemini 2.x) with LangChain, keeping embeddings, ranking, and generation inside GCP for unified IAM, billing, and observability.
- Flexible Data Ingestion & Metadata Management
- Supported ingestion of PDFs, CSVs, and Excel files, capturing metadata and lineage in a notebook-first interface for transparency.
- Configurable Optimization
- Exposed tuning parameters (chunk sizes, retrieval settings, temps, token limits) for grid search without requiring code edits.
- Safety by Design
- Centralized SafetySettings blocked harmful or high-severity content across all model calls, ensuring compliant and trustworthy responses.
- Hybrid Retrieval & Ranking
- Combined FAISS semantic recall with BM25 lexical scoring, MMR diversity, deduplication, and Vertex AI Ranking (cross-encoder) for re-scoring and explainability.
- Disciplined Prompting & Validation
- Enforced extract-from-context behavior using templated prompts and validator loops, with LLM-as-a-Judge and reflection cycles improving completeness and accuracy.
- Interactive Debugging & Auditability
- Built ipywidgets-based chat with retriever debugger showing source documents, chunk scores, and audit trails for enterprise compliance.
Outcomes
The FAQ bot delivered measurable improvements across support operations:
- Increased Accuracy: LLM-as-a-Judge with reflection cycles improved precision and completeness of answers.
- Safer Responses: Centralized safety filters ensured that no harmful or non-compliant content reached customers.
- Faster Tuning: Configurable optimization knobs enabled rapid iteration and parameter search without developer overhead.
- Transparency & Trust: Built-in retriever debugger and explainability logs increased auditability and stakeholder confidence.
- Operational Efficiency: Automated answers reduced manual support volumes, accelerating customer query resolution at scale.
Conclusion
Narwal’s advanced RAG chatbot transformed the client’s customer FAQ operations by delivering accurate, safe, and transparent responses. With hybrid retrieval, disciplined prompting, and safety-first governance, the solution reduced manual support workload while building customer trust. Designed for scalability and auditability, this AI-powered system provides a strong foundation for future expansion across broader customer engagement use cases.
Partner with Narwal today to implement secure, transparent, and AI-powered customer engagement solutions.
