Automated Customer Support

I've tested over 20 AI agent frameworks in the past 2 years. Building with them, breaking them, trying to make them work in real scenarios. Here's the brutal truth: 99% of them fail when real customers show up. Most are impressive in demos but struggle with actual conversations. Then I came across Parlant in the conversational AI space. And it's genuinely different. Here's what caught my attention: 1. The Engineering behind it: 40,000 lines of optimized code backed by 30,000 lines of tests. That tells you how much real-world complexity they've actually solved. 2. It works out of the box: You get a managed conversational agent in about 3 minutes that handles conversations better than most frameworks I've tried. 3. Conversation Modeling Approach: Instead of rigid flowcharts or unreliable system prompts, they use something called "Conversation Modeling." Here's how it actually works: 1. Contextual Guidelines: ↳ Every behavior is defined as a specific guideline. ↳ Condition: "Customer wants to return an item" ↳ Action: "Get order number and item name, then help them return it" 2. Controlled Tool Usage: ↳ Tools are tied to specific guidelines ↳ No random LLM decisions about when to call APIs ↳ Your tools only run when the guideline conditions are met. 3. Utterances Feature: ↳ Checks for pre-approved response templates first ↳ Uses those templates when available ↳ Automatically fills in dynamic data (like flight info or account numbers) ↳ Only falls back to generation when no template exists What I Really Like: It scales with your needs. You can add more behavioral nuance as you grow without breaking existing functionality. What's even better? It works with ALL major LLM providers - OpenAI, Gemini, Llama 3, Anthropic, and more. For anyone building conversational AI, especially in regulated industries, this approach makes sense. Your agents can now be both conversational AND compliant. AI Agent that actually does what you tell it to do. If you’re serious about building customer support agents and tired of flaky behavior, try Parlant.

Power Automate Work Queues are not built for scale! That's a fact. When you think about scalability in Power Automate, one thing that will definitely come to mind at some point is queues and workload management. While you might be able to survive without them in some event-based transactional flows that only process a single item at a time, but whenever you process tasks in batches, or when RPA gets involved, you'll need queues. Power Automate comes with Work Queues out of the box. And you would think that's your go-to queueing mechanism for scaling. After all, it's at scale that you really need those queues - to de-couple your flows and make it easier to maintain, support, debug them, as well as make them more robust and efficient. Queues is a must even at medium scale. Heck, we use them even in small scale implementations. But the surprising thing about Power Automate Work Queues is that they are not fit for high scale implementations. And that is by design! The docs themselves (link in the comments) explicitly state that if have high volumes or if you dequeue (pick up work items from the queue for processing) concurrently, you should either do it within moderate levels or use something else. If you try and use Power Automate Work Queues for high scale implementations (more than 5 concurrent dequeue operations or hundreds/thousands of any type operations involving the queues), you'll get in trouble. There can be all sorts of issues that could happen - your data may get duplicated, you may accidentally deque the same work item in multiple concurrent instances, or your flows might simply get throttled or even crash. This is because of the way they're build and the way they utilize Dataverse tables for storing work items and work queue metadata. So, if you do want to scale, it's best to use an alternative. And, obviously, Microsoft wouldn't be Microsoft if they didn't have an alternative tool to do that. The docs themselves recommend Azure Service Bus Queues for high throughput queueing mechanisms. Another alternative could also be Azure Storage Queues, but that only makes sense if the individual work items in your queue can get large (lots of data or even documents) or when you expect your queue to grow beyond 80GB (which is possible in very large scale implementations). Otherwise, Azure Service Bus Queues are absolutely perfect for very large volumes of small transactions. On top of that, they have some very advanced features for managing, tracking, auditing and otherwise handling your work items. And, of course, there's a existing connector in Power Automate to use it. So, while I do love Power Automate Work Queues, I'll only use them in relatively small scale implementations. And for everything else - my queues will go to Azure. And so should yours.

I just got quoted in Forbes ☺️ ... in a piece that explores the rise of conversational video AI - a market projected to reach $82.46B by 2034 - where agents can now see your face, hear your tone, and respond in real time. But instead of obsessing over avatars & realism, we focused on the part of ML that’s far messier and far more important... Epistemic Humility! Right now, most of the industry is racing to make AI _look_ more human: • Better facial expressions • Smoother voice inflections • Real-time visual & emotional cues ... but imo true intelligence isn't just about bluffing coherence. It’s about: → Explicitly tracking uncertainty → Admitting “I don’t know” when data is missing → Reasoning through ambiguity instead of faking confidence → Moving from surface mimicry to joint attention, shared context, intent, and understanding As I shared with Forbes, the next leap isn’t hyper-realism. It’s AI that can maintain joint attention, reason under uncertainty, and participate in conversation like a human - not just perform like one! Key takeaways for anyone building or deploying AI systems: • Overconfident models are riskier than uncertain ones • Human-like isn’t a UX problem - it’s a modeling & evaluation problem • Reliability matters more than realism in real-world systems • The ability to say I DON'T KNOW is a feature, not a failure The future of conversational AI isn’t about perfect mimicry. It’s about integrity. Systems that are honest about their limits & designed to handle uncertainty responsibly. Honored to be featured alongside brilliant researchers & leaders in this space. Thank you, Forbes, for featuring my voice ✨ #ConversationalAI #AgenticAI #TechTrends #Forbes

Conversational AI is transforming customer support, but making it reliable and scalable is a complex challenge. In a recent tech blog, Airbnb’s engineering team shares how they upgraded their Automation Platform to enhance the effectiveness of virtual agents while ensuring easier maintenance. The new Automation Platform V2 leverages the power of large language models (LLMs). However, recognizing the unpredictability of LLM outputs, the team designed the platform to harness LLMs in a more controlled manner. They focused on three key areas to achieve this: LLM workflows, context management, and guardrails. The first area, LLM workflows, ensures that AI-powered agents follow structured reasoning processes. Airbnb incorporates Chain of Thought, an AI agent framework that enables LLMs to reason through problems step by step. By embedding this structured approach into workflows, the system determines which tools to use and in what order, allowing the LLM to function as a reasoning engine within a managed execution environment. The second area, context management, ensures that the LLM has access to all relevant information needed to make informed decisions. To generate accurate and helpful responses, the system supplies the LLM with critical contextual details—such as past interactions, the customer’s inquiry intent, current trip information, and more. Finally, the guardrails framework acts as a safeguard, monitoring LLM interactions to ensure responses are helpful, relevant, and ethical. This framework is designed to prevent hallucinations, mitigate security risks like jailbreaks, and maintain response quality—ultimately improving trust and reliability in AI-driven support. By rethinking how automation is built and managed, Airbnb has created a more scalable and predictable Conversational AI system. Their approach highlights an important takeaway for companies integrating AI into customer support: AI performs best in a hybrid model—where structured frameworks guide and complement its capabilities. #MachineLearning #DataScience #LLM #Chatbots #AI #Automation #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gj6aPBBY    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/gFjXBrPe

How can businesses get the most from conversational and agentic AI? Both are reshaping how organizations work and serve customers, but they deliver impact in different ways. The opportunity for leaders is knowing where each shines and how to combine them for maximum ROI.  🔹 Conversational AI thrives in the moment. It understands and responds naturally during interactions to answer questions, guide customers to the right resources, and gather details in real time. 🔹 Agentic AI takes it further. Built with skills like memory, reasoning, and autonomous action, it can recognize signals, predict needs, and trigger workflows without manual input. Picture a support call: conversational AI greets a customer, identifies the issue, and provides initial guidance. Agentic AI detects urgency in their tone, escalates the case, and updates records across systems instantly. When organizations pair the responsiveness of conversational AI with the autonomy of agentic AI, they create interactions that are more personalized, efficient, and impactful. At RingCentral, we’re building on two decades of voice expertise to make this pairing even more powerful with solutions like our AI Receptionist and RingSense, so every conversation can become an engine for long-term growth.

A junior reached out to me last week. One of our APIs was collapsing under 150 requests per second. Yes — only 150. He had tried everything: * Added an in-memory cache * Scaled the K8s pods * Increased CPU and memory Nothing worked. The API still couldn’t scale beyond 150 RPS. Latency? Upwards of 1 minute. 🤯 Brain = Blown. So I rolled up my sleeves and started digging; studied the code, the query patterns, and the call graphs. Turns out, the problem wasn’t hardware. It was design. It was a bulk API processing 70 requests per call. For every request: 1. Making multiple synchronous downstream calls 2. Hitting the DB repeatedly for the same data for every request 3. Using local caches (different for each of 15 pods!) So instead of adding more pods, we redesigned the flow: 1. Reduced 350 DB calls → 5 DB calls 2. Built a common context object shared across all requests 3. Shifted reads to dedicated read replicas 4. Moved from in-memory to Redis cache (shared across pods) Results: 1. 20× higher throughput — 3K QPS 2. 60× lower latency (~60s → 0.8s) 3. 50% lower infra cost (fewer pods, better design) The insight? 1. Most scalability issues aren’t infrastructure limits; they’re architectural inefficiencies disguised as capacity problems. 2. Scaling isn’t about throwing hardware at the problem. It’s about tightening data paths, minimizing redundancy, and respecting latency budgets. Before you spin up the next node, ask yourself: Is my architecture optimized enough to earn that node?

OpenAI doesn’t measure support the way you do. They’re not chasing CSAT or time-to-close. They rebuilt support — and what they came up with changes everything. Here’s the shift: A ticket opens. It gets solved. It closes. And most of the knowledge dies there. OpenAI saw that model couldn’t scale. Support wasn’t just a volume problem — it was an engineering and operational design problem. So they built something different: a system where every interaction improves the next. It starts with three building blocks: 🔲 Surfaces. Where support lives: chat, email, voice, and increasingly embedded directly in-product. 🔲 Knowledge. Not static docs, but living guidance that evolves with real conversations, policies, and context. 🔲 Evals & classifiers. Shared definitions of quality built by humans + software, continuously running to steer the system. These pieces form a loop. A pattern spotted in an enterprise chat updates the knowledge base. An eval created for one case trains the model for thousands more. And because the same primitives power every channel, improvements scale automatically. And here’s what really struck me: At OpenAI, reps aren’t just responding to tickets. They flag interactions that should become test cases. They propose new classifiers. They even prototype lightweight automations to close workflow gaps. Training shifts too — from just “policies” to spotting structural gaps and feeding improvements back. The result? Support isn’t measured by throughput, but by its capacity to evolve. And the loop doesn’t stop there. Each interaction compounds: Evals turn daily conversations into production tests. They codify what “great” means: not just solving, but solving politely, clearly, consistently. Patterns flow back into knowledge, automation, and product design. Every resolution strengthens the system. Every pattern spotted improves future answers. Every classifier scales across channels. And the org itself learns alongside the AI — reps shape classifiers, contribute datasets, and watch quality improve in real time through observability dashboards. What does all this point to? A blueprint for the future of support. Glen Worthington put it best: “Support has never really been about replying to just tickets. It’s about whether people get what they need, whether it actually serves them well.” That’s the profound shift: Support specialists are recognized not just for solving problems, but for refining knowledge, improving models, and extending the system itself. The future isn’t support as a destination. It’s support as an action — woven into every product surface. Here’s the uncomfortable question for every support leader 👇 If you look at your last 100 tickets… How many made tomorrow’s support better than today’s? Because in the future, the answer needs to be: all of them. Jay Patel Shimul Sachdeva

A workflow at 90% accuracy handling 1,000 requests produces 100 exceptions. That same workflow at 95% accuracy handling 10,000 requests produces 500 exceptions. Accuracy went up by 5 percentage points. Exception volume went up by 5x. This is a bottleneck most operations teams do not plan for when a workflow moves from pilot to production volume. Teams can usually report seats, prompt counts, or active users. Fewer can show exception volume, median resolution time, or cost per case for a live workflow. In claims intake or invoice processing, that usually means the team cannot show how many cases required human review yesterday or how long those cases sat before resolution. When production stalls, the usual assumption is that the AI is the problem. More often, the problem is that exception handling was never scoped for production conditions. A pilot with curated inputs and extra attention can absorb exceptions through effort. A production workflow with variable inputs and volume cannot. Exception flow is not an edge case to handle after the workflow ships. It is the architecture the operation should be designed around from the start. #AIOperations #ExceptionManagement #AIWorkflows #OperatingModel #BusinessProcessOperations

Your architecture isn't tested by traffic. It's tested by unpredictable traffic. And that’s where most systems quietly break - not because they couldn’t scale, but because they scaled the wrong layer at the wrong time. From real-world systems, one thing becomes clear: scaling is not one strategy - it’s a combination of patterns working together. This breakdown covers the ones that actually matter 👇 ➞ Queue-Based Scaling Decouple workloads and process asynchronously. Critical when handling spikes without overwhelming systems. ➞ Microservices Scaling Scale only what needs scaling - not the entire application. Efficiency comes from isolating bottlenecks. ➞ Vertical vs Horizontal Scaling More power vs more machines. Good engineers know when to use each - great engineers combine both. ➞ Auto Scaling Let systems respond to real-time demand automatically. No manual intervention, no guesswork. ➞ Load Balancing Distribute traffic intelligently to avoid single points of failure. ➞ Caching Strategy Reduce repeated computation and database load. Often the simplest way to get massive performance gains. ➞ CDN (Content Delivery Network) Move content closer to users - latency drops instantly. ➞ Database Scaling Replication, sharding, partitioning. Because most scaling problems eventually become data problems. ➞ Serverless Scaling Event-driven, auto-managed scaling without worrying about infrastructure. Here’s what consistently goes wrong: → Teams rely only on auto scaling and ignore architecture → Compute scales, but databases become the bottleneck → Asynchronous patterns are introduced too late That’s why systems fail exactly when demand increases. Real scalability isn’t about handling more traffic. It’s about handling uncertainty without breaking. If your system had 10x traffic tomorrow, which layer would struggle first? 👇