Explainable AI Tools

Can LLMs improve product recommendations with re-ranking? Fascinating new paper from Meta on applying LLMs to recommendation systems. The domain discussed in the paper is content re-ranking, but I don't see why this couldn't be applied to ads. Re-ranking takes a ranked list of candidate items (following retrieval and, sometimes, pre-ranking) and updates the ordering to better optimize some objective function (eg., purchase). The authors describe how they utilize an LLM to re-rank candidates with a number of novel innovations: - Instead of building the LLM vocabulary from item embeddings, which would likely be too large to be useful, they decompose each item embedding into a sequence of "tokens" produced from a K-stage quantization process (with RQ-VAE). This process accepts the item embedding at k=1, calculates the residual vector from the nearest of C learned centroids for that step (called "codebooks"), and passes that output to k=2, and so on to K. This produces a Semantic ID (SID) of length K. - Because re-ranking must be done quickly, it requires a smaller LLM (the paper uses 8B parameters). So in training, they prompt a large model (Qwen-32B) with the user's history, the candidate items produced in ranking, the SIDs, and instructions to reason through its process of re-ranking these items. That model produces a reasoning trace and a re-ranked list. The authors use rejection sampling to retain only the outputs (reasoning traces + rankings) for which the ground-truth item is ranked sufficiently high. The 8B student model is fine-tuned via SFT on that distribution, learning P( reasoning trace + ranking | prompt ). - Finally, the authors fine-tune this model with RL on the outcome, using the ground truth's location in the list as the reward. This aligns the model's policy with the reward, enabling more thorough comparison across candidates (versus reasoning collapse). The authors make the point that LLMs can introduce additional product context, scalability, and "world knowledge" to RecSys, turning ranking into a structured reasoning task rather than a pure scoring task. The paper is quite dense but worth reading in full; link below.

Can your AI system explain why it rejected benefits claims? If not, it's not going into production. Multiple central government departments are now requiring new AI systems, especially those used in decision making like benefit fraud detection. This is being done to pass a formal AI Explainability gate before production rollout. This elevates technical risk and transparency from a compliance checklist to a crucial delivery step. The change responds to both the EU AI Act's ripple effects and local GenAI pilot failures that exposed the risks of deploying black box systems in public facing services. This is the governance maturity the sector needs. Building AI systems is straightforward. Explaining how they reach decisions is significantly harder. The timing matters. Too many pilots have failed not because the technology didn't work, but because nobody could justify the decisions it made. What this means for delivery: → Explainability must be designed in from day one → Technical teams need to document decision logic in plain language for non-technical colleagues → Procurement specifications must now include explainability requirements upfront The departments getting this right are treating explainability as a core architectural requirement, not a final compliance hurdle. This gate will slow some projects initially. But it prevents the far costlier problem of deploying AI systems that make decisions nobody can defend. How is your organisation building explainability into AI systems from the start? #AI #PublicSector #AIGovernance

Product recommendations play a critical role in helping customers discover relevant items and drive engagement and conversion. Even small improvements in recommendation quality can compound at scale, especially in large retail platforms. In a recent tech blog, data scientists from CVS Health shared how they enhanced their existing “You May Also Like” recommendation module by integrating large language models into the system. - At a high level, the recommendation approach is grounded in “similarity”. For any given product, the goal is to identify other products that are similar based on rich product attributes. The overall workflow follows a familiar pattern: generate product embeddings, measure similarity between products, and surface recommendations accordingly. - The key challenge, however, lies in data quality. Not every product comes with a well-written title or detailed description that can be used to generate meaningful embeddings. Some products have extremely short or uninformative metadata, which limits the effectiveness of traditional embedding-based methods. - This is where LLMs add value. For products with sparse or low-quality text, the team leveraged ChatGPT to generate a roughly 200-word product summary, expanding on the product’s purpose, usage, ingredients, and key attributes. These enriched summaries provide higher-quality inputs for embedding generation, improving both coverage and recommendation accuracy. While rebuilding an entire recommendation engine around GenAI can be appealing, this case is a good reminder that incremental, well-targeted improvements often deliver real impact. Integrating LLMs to strengthen weak points in existing systems can be both practical and powerful—and this example is a nice one to keep in mind. #DataScience #MachineLearning #GenAI #LLM #ChatGPT #Recommendation #IncrementalImprovement #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gFYvfB8V    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/g7Xv3tG4

Exciting Research Alert: LLM-powered Agents Transforming Recommender Systems! Just came across a fascinating survey paper on how Large Language Model (LLM)-powered agents are revolutionizing recommender systems. This comprehensive review by researchers from Tianjin University and Du Xiaoman Financial Technology identifies three key paradigms reshaping the field: 1. Recommender-oriented approaches - These leverage intelligent agents with enhanced planning, reasoning, and memory capabilities to generate strategic recommendations directly from user historical behaviors. 2. Interaction-oriented methods - Enabling natural language conversations and providing interpretable recommendations through human-like dialogues that explain the reasoning behind suggestions. 3. Simulation-oriented methods - Creating authentic replications of user behaviors through sophisticated simulation techniques that model realistic user responses to recommendations. The paper introduces a unified architectural framework with four essential modules: - Profile Module: Constructs dynamic user/item representations by analyzing behavioral patterns - Memory Module: Manages historical interactions and contextual information for more informed decisions - Planning Module: Designs multi-step action plans balancing immediate satisfaction with long-term engagement - Action Module: Transforms decisions into concrete recommendations through systematic execution What's particularly valuable is the comprehensive analysis of datasets (Amazon, MovieLens, Steam, etc.) and evaluation methodologies ranging from standard metrics like NDCG@K to custom indicators for conversational efficiency. The authors highlight promising future directions including architectural optimization, evaluation framework refinement, and security enhancement for recommender systems. This research demonstrates how LLM agents can understand complex user preferences, facilitate multi-turn conversations, and revolutionize user behavior simulation - addressing key limitations of traditional recommendation approaches.

A regulator asked a bank to explain its AI agent's last 100 decisions. The bank showed them a confidence score. The regulator shut it down. This is happening more than anyone admits. "Explainable AI" has become the most misleading phrase in enterprise software. Every vendor checks that box. Almost none of them can produce what a regulator actually needs: Which rule fired. What data was examined. What the agent decided. Why. With evidence. For every single action. Not "the model was 92% confident." That tells a regulator nothing. They want to see: "Section 3.1(a) requires site verification for draws over $250K. The inspection report was dated Feb 15. The draw was $420K. Verification was confirmed within the 30-day window. Approved." That's the difference between a confidence score and an evidence chain. I've started calling this the Why-Trail. Not because it's clever, but because "explainability" has been diluted to the point where it means nothing. A Why-Trail is deterministic. It traces the exact policy, the exact data, and the exact logic path. It's reproducible. You can hand it to an auditor and they can follow it like a receipt. The EU AI Act hits full enforcement August 2, 2026. Article 14 mandates human oversight for every high-risk AI system. Credit scoring, loan approvals, insurance underwriting: all classified high-risk. 81% of leaders say human-in-the-loop is essential. Only 20% have mature governance to support it. That gap is where the next wave of regulatory enforcement will land. Here's the test: if your AI agent made a decision five minutes ago, could you pull up the full reasoning chain right now? Not a summary. Not a probability. The actual rule, the actual data, the actual logic. If you can't, you don't have explainability. You have a marketing page that says you do. For anyone deploying AI in regulated industries: what does your audit trail actually look like today?

What if your app actually understood your data structure before searching? We built it into Elysia, and it turned out to have a HUGE impact on the quality of context and responses. Most RAG systems work blind. They perform vector searches without really understanding the structure, content, or meaning of what they're searching through. It's like asking someone to find a quote in a book they've never read before - they might get lucky, but they're basically guessing. Elysia takes a completely different approach. Before you ever ask a question, an LLM analyzes your data and the structure of your collections. Here's how: 1️⃣ 𝗦𝗮𝗺𝗽𝗹𝗶𝗻𝗴 𝗮𝗻𝗱 𝗦𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀: The LLM examines your data by sampling it and checking fields to understand what information you actually have available. 2️⃣ 𝗦𝘂𝗺𝗺𝗮𝗿𝘆 𝗮𝗻𝗱 𝗠𝗲𝘁𝗮𝗱𝗮𝘁𝗮 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻: It creates summaries and generates metadata about your collections and properties within those collections. 3️⃣ 𝗗𝗶𝘀𝗽𝗹𝗮𝘆 𝗧𝘆𝗽𝗲 𝗥𝗲𝗰𝗼𝗺𝗺𝗲𝗻𝗱𝗮𝘁𝗶𝗼𝗻𝘀: Based on this analysis, it recommends the most appropriate ways to display results (tables, product cards, documents, charts, etc.). Turns out, this upfront analysis is pretty essential for handling complex queries. Traditional systems like our OG RAG library Verba often failed with ambiguous data because of their blind search approach - they'd just hope for relevant results without understanding context. Elysia's approach means it can: • Handle complex queries across multiple data types • Generate intelligent filters based on actual data structure • Choose appropriate display formats for different content • Understand relationships between different collections Take a query like "Show me products under $100 in the electronics category." A naive RAG system might struggle with this because it doesn't know your data has a price field or a category field. Elysia already analyzed your product collection and knows 𝘦𝘹𝘢𝘤𝘵𝘭𝘺 how to 𝗰𝗼𝗻𝘀𝘁𝗿𝘂𝗰𝘁 𝘁𝗵𝗮𝘁 𝗾𝘂𝗲𝗿𝘆 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗿𝗶𝗴𝗵𝘁 𝗳𝗶𝗹𝘁𝗲𝗿𝘀. The system also 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰𝗮𝗹𝗹𝘆 𝗿𝗲𝗰𝗼𝗺𝗺𝗲𝗻𝗱𝘀 𝗱𝗶𝘀𝗽𝗹𝗮𝘆 𝘁𝘆𝗽𝗲𝘀 based on this analysis. E-commerce data gets product cards, conversation data gets message displays, documents get document viewers - all chosen based on the 𝘢𝘤𝘵𝘶𝘢𝘭 𝘴𝘵𝘳𝘶𝘤𝘵𝘶𝘳𝘦 of your data. You can see all this analysis in 𝗘𝗹𝘆𝘀𝗶𝗮'𝘀 𝗱𝗮𝘁𝗮 𝗲𝘅𝗽𝗹𝗼𝗿𝗲𝗿. The system shows you unique values within fields, min/max ranges for numeric data, and all the generated metadata. You can even edit it if the LLM got something wrong (because let's be honest, LLMs aren't perfect 😅). Check out the blog post: https://lnkd.in/eQJAWaM4 Or the full video: https://lnkd.in/ePCdm5tC

This paper, “From Logs to Language: Learning Optimal Verbalization for LLM-Based Recommendation in Production,” was featured in a Substack I follow on top IR papers of the week. Of all the strong papers in this week’s newsletter, this one stood out for a subtle but powerful reason. It reframes something we usually take for granted: how we translate structured logs into language for LLMs. Most LLM-based recommendation systems rely on hand-crafted templates to turn user interaction logs into prompts. Logs go in, text comes out, and the model does the rest. This paper asks a different question: What if verbalization itself were learnable? Instead of hard-coding templates, they train a verbalization agent to transform interaction logs into optimized natural language context for downstream recommendation. What I found especially interesting were the emergent behaviors that surfaced during this process: - Syntax normalization — the model standardizes structure to make signals clearer. - Noise and ID filtering — it learns that raw identifiers and cluttered metadata are not useful semantic signals. - Abstraction of item-level details — it summarizes granular interactions into higher-level preference patterns. In other words, the system is not just learning what to recommend. It is learning how to represent user behavior in language. That feels like a data-centric shift. The improvement does not come from a bigger model, but from a better representation of behavioral logs. Another interesting detail: they evaluate using Recall@1, not Precision@1. That choice makes sense in a discovery setting. Unlike search, where precision at the top result is critical, recommendation is about surfacing at least one highly relevant item from a large candidate pool. Recall@1 is a better indicator of how well the system can identify previously unseen content that aligns with user preferences, which is essential for meaningful content discovery. This paper reinforces a broader pattern: Strong LLM-based systems are often limited not by the model, but by how we structure and present data to it. Sometimes the biggest gains come from teaching the model how to read. https://lnkd.in/ep2z9cyz

Your recommender guesses. Mine thinks twice. I just read a fresh agent recipe that turns recommendation into a two speed brain. It ranks fast. Then it reflects slow. It explains why it picked each item, learns from the miss, rewrites what it knows about you, and tries again. That loop is the unlock for thin data and cold start. Today most LLM recs feel like vibes. They match patterns and pray. This agent self audits in plain English, so you can see the why and fix drift fast. The training plan is simple and strong. First learn from a better reasoning model to anchor good habits. Then run light RL that rewards real ranking wins. Result. Better picks with a tiny slice of data and small models that still ship on modest hardware. This is why I am excited for Main Street use. Shops, clinics, firms with short histories can still get smart recs on day one. You can demo this on a single screen. Left side fast list with short reasons. Right side reflection note after feedback. Click once. Watch the memory update. Run the next list. Cleaner picks in minutes. If you build with n8n, Lovable, or your own stack, here is the playbook I would test this week. Keep preferences as a readable note, not a black box vector. Add a slow loop that rewrites that note after each click. Tie rewards to rank, not vanity metrics. Think fast, then think slow, then ship. This arXiv report breaks down a two step recommender that ranks fast, then reflects to update a simple text profile of the user.

Most recommendation systems today are a patchwork of specialized models — one for search, one for collaborative filtering, one for user understanding. Spotify's NEO paper at KDD 2026 asks a genuinely uncomfortable question: what if that architectural fragmentation is the problem, not the solution? The paper introduces NEO, a decoder-only LLM adapted into a tool-free, catalog-grounded generator that handles recommendation, search, and user understanding within a single sequence model. The key mechanism is language-steerability: text prompts control the task, the target entity type, and the output format — IDs, free text, or mixed — while constrained decoding guarantees that generated items are always valid catalog entries. Items are represented as Semantic IDs (SIDs), treated as a distinct modality and integrated via staged alignment and instruction tuning. The system was evaluated on a real-world catalog of over 10 million items across multiple media types. Why does this matter to a CTO or Head of Research at a MarTech company? Three reasons. First, the orchestration tax of multi-model pipelines is real — every handoff between a retrieval model, a ranker, and a user understanding module introduces latency, error propagation, and optimization barriers. NEO demonstrates that a single model can achieve cross-task transfer, meaning that training on recommendation improves search performance and vice versa. Second, language-steerability is not a UX feature — it is an architectural primitive. The ability to condition a generative model on natural language instructions at inference time is precisely what makes AI systems adaptable to the kind of long-tail, context-specific targeting that platforms like SOMIN have been pursuing through Omni-Sourced User Profiling. Third, the SID-as-modality framing is a meaningful contribution to the broader question of how to ground LLMs in structured, domain-specific entity spaces without sacrificing generative flexibility — a problem directly relevant to anyone building on top of large catalogs. From my own research trajectory — starting with multi-source user profile learning at ICMR 2015, through cross-domain recommendation via multi-layer graph clustering at SIGIR 2017, and into the explainability work we published at ACM MM 2023 and 2025 — the direction NEO takes feels like a natural convergence point. The challenge was always that user signals are fragmented across modalities and tasks. What NEO proposes is not just a model architecture but a reframing: treat the user's intent as a language-expressible instruction, and let a single grounded generator handle the rest. The question of whether this holds at the scale and heterogeneity of real marketing catalogs — where items span creative assets, audience segments, and campaign objectives — remains open. Key takeaways: - A single language-steerable generative model can outperform task-specific baselines on recommendation, search, and user understanding simultane

You’re hired as a GRC Analyst at a fast-growing fintech company that just integrated AI-powered fraud detection. The AI flags transactions as “suspicious,” but customers start complaining that their accounts are being unfairly locked. Regulators begin investigating for potential bias and unfair decision-making. How you would tackle this? 1. Assess AI Bias Risks • Start by reviewing how the AI model makes decisions. Does it disproportionately flag certain demographics or behaviors? • Check historical false positive rates—how often has the AI mistakenly flagged legitimate transactions? • Work with data science teams to audit the training data. Was it diverse and representative, or could it have inherited biases? 2. Ensure Compliance with Regulations • Look at GDPR, CPRA, and the EU AI Act—these all have requirements for fairness, transparency, and explainability in AI models. • Review internal policies to see if the company already has AI ethics guidelines in place. If not, this may be a gap that needs urgent attention. • Prepare for potential regulatory inquiries by documenting how decisions are made and if customers were given clear explanations when their transactions were flagged. 3. Improve AI Transparency & Governance • Require “explainability” features—customers should be able to understand why their transaction was flagged. • Implement human-in-the-loop review for high-risk decisions to prevent automatic account freezes. • Set up regular fairness audits on the AI system to monitor its impact and make necessary adjustments. AI can improve security, but without proper governance, it can create more problems than it solves. If you’re working towards #GRC, understanding AI-related risks will make you stand out.