AI Language Processing

I shared a new tutorial + experiments on finetuning LLMs for classification efficiently. In this video, I explain how to convert a decoder-style LLM into a classifier. Many business problems are text classification problems, and if classification is all we need for a given task, using "smaller" and cheaper LLMs makes a lot of sense! (But, of course, also always run a simple logistic regression or naive Bayes baseline to determine if you even need a small LLM.) 🧪 In addition, I also ran a series of 19 experiments to answer some "what if" questions around finetuning pretrained LLMs for classification. Here, I kept things simple and small (e.g., GPT-2 on a toy binary classification task): Here's a snapshot summary of some of the interesting ones: 1) As would be expected, training on the last token yields much better performance than the first 2) Training the last transformer block is way better than just the last layer 3) LoRA performs on par or better than full finetuning—while being faster and more memory-efficient 4) Padding to full context length hurts performance 5) No padding or smart position selection leads to consistently higher accuracy 6) Surprisingly, training from random weights isn't much worse than using pretrained 7) Averaging embeddings over all tokens can improve performance slightly with little cost The full video is available here: https://lnkd.in/gcfqR2mH PS: If you are wondering why GPT instead of BERT? Well, you can of course also use BERT. Based on experiments on the 50k Movie Review dataset It's interesting though that this 3x smaller LLM performs on par (actually slightly better) than BERT. (ModernBERT then again is 2% better.)

Large Language Models (LLMs) are powerful, but how we 𝗮𝘂𝗴𝗺𝗲𝗻𝘁, 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲, 𝗮𝗻𝗱 𝗼𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗲 them truly defines their impact. Here's a simple yet powerful breakdown of how AI systems are evolving: 𝟭. 𝗟𝗟𝗠 (𝗕𝗮𝘀𝗶𝗰 𝗣𝗿𝗼𝗺𝗽𝘁 → 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲)   ↳ This is where it all started. You give a prompt, and the model predicts the next tokens. It's useful — but limited. No memory. No tools. Just raw prediction. 𝟮. 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻)   ↳ A significant leap forward. Instead of relying only on the LLM’s training, we 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗲 𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝗳𝗿𝗼𝗺 𝗲𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝘀𝗼𝘂𝗿𝗰𝗲𝘀 (like vector databases). The model then crafts a much more relevant, grounded response.   This is the backbone of many current AI search and chatbot applications. 𝟯. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗟𝗟𝗠𝘀 (𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 + 𝗧𝗼𝗼𝗹 𝗨𝘀𝗲)   ↳ Now we’re entering a new era. Agent-based systems don’t just answer — they think, plan, retrieve, loop, and act.   They: - Use 𝘁𝗼𝗼𝗹𝘀 (APIs, search, code) - Access 𝗺𝗲𝗺𝗼𝗿𝘆 - Apply 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗰𝗵𝗮𝗶𝗻𝘀 - And most importantly, 𝗱𝗲𝗰𝗶𝗱𝗲 𝘄𝗵𝗮𝘁 𝘁𝗼 𝗱𝗼 𝗻𝗲𝘅𝘁 These architectures are foundational for building 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗔𝗜 𝗮𝘀𝘀𝗶𝘀𝘁𝗮𝗻𝘁𝘀, 𝗰𝗼𝗽𝗶𝗹𝗼𝘁𝘀, 𝗮𝗻𝗱 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗺𝗮𝗸𝗲𝗿𝘀. The future is not just about 𝘸𝘩𝘢𝘵 the model knows, but 𝘩𝘰𝘸 it operates. If you're building in this space — RAG and Agent architectures are where the real innovation is happening.

One of the biggest deals of yesterday's Meta's announcements is Llama Stack The way I describe it is the operating system for AI with Llama. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗟𝗹𝗮𝗺𝗮 𝗦𝘁𝗮𝗰𝗸? One of the biggest pain points for developers using Llama is how to use it, get started, and make it more developer-friendly overall. With that, Meta creates a stack that standardizes the building blocks needed to bring generative AI applications to market. These blocks span the entire development lifecycle: - model training - fine-tuning - ai product evaluation - building and running AI agents in production and more 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗶𝗻𝗰𝗹𝘂𝗱𝗲𝗱 𝗶𝗻 𝘁𝗵𝗶𝘀 𝗿𝗲𝗹𝗲𝗮𝘀𝗲? - Llama CLI (command line interface) to build, configure, and run Llama Stack distributions - Client code in multiple languages, including python, node, kotlin, and swift - Docker containers for Llama Stack - Distribution Server and Agents API Provider 𝗟𝗹𝗮𝗺𝗮 𝗦𝘁𝗮𝗰𝗸 𝗰𝗼𝗺𝗲𝘀 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗳𝗼𝗹𝗹𝗼𝘄𝗶𝗻𝗴 𝗰𝗼𝗿𝗲 𝗰𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀: - PromptStore: For storing and managing prompts for Llama models. - Batch Inference: A tool for making request on batches of data. - Continual Pretraining: A method for continuously pretraining Llama models on new data. - Realtime Inference: A tool for making predictions in real-time. - Quantized Inference: A method for optimizing inference performance through quantization. - Evals: A component for evaluating the performance of Llama models. - Finetuning: A method for fine-tuning Llama models on specific tasks. - Reward Scoring: A tool for scoring rewards for Llama models. - Synthetic Data Generation: A method for generating synthetic data for Llama models. - Data: A component for managing and processing data for Llama models. - Models: A component for managing and deploying Llama models. - Hardware: A component for managing and optimizing hardware resources for Llama models. - Accelerators: A type of hardware accelerator for Llama models. - Storage: A component for managing and storing data for Llama models. - Safety: A component for ensuring the safety and reliability of Llama models. 𝗟𝗹𝗮𝗺𝗮 𝗦𝘁𝗮𝗰𝗸 𝗶𝘀 𝗮𝘃𝗮𝗶𝗹𝗮𝗯𝗹𝗲 𝗶𝗻 𝗠𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗱𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻𝘀: - Single-node Llama Stack Distribution via Meta internal implementation and Ollama - Cloud Llama Stack distributions via Cloud providers - On-device Llama Stack Distribution on iOS implemented via PyTorch ExecuTorch - On-prem Llama Stack Distribution supported by Dell Another excellent gift for developers. I cannot wait to see what's built on top of Llama Stack and how the community embraces it.

If you’re an AI engineer, understanding how LLMs are trained and aligned is essential for building high-performance, reliable AI systems. Most large language models follow a 3-step training procedure: Step 1: Pretraining → Goal: Learn general-purpose language representations. → Method: Self-supervised learning on massive unlabeled text corpora (e.g., next-token prediction). → Output: A pretrained LLM, rich in linguistic and factual knowledge but not grounded in human preferences. → Cost: Extremely high (billions of tokens, trillions of FLOPs). → Pretraining is still centralized within a few labs due to the scale required (e.g., Meta, Google DeepMind, OpenAI), but open-weight models like LLaMA 4, DeepSeek V3, and Qwen 3 are making this more accessible. Step 2: Finetuning (Two Common Approaches) → 2a: Full-Parameter Finetuning - Updates all weights of the pretrained model. - Requires significant GPU memory and compute. - Best for scenarios where the model needs deep adaptation to a new domain or task. - Used for: Instruction-following, multilingual adaptation, industry-specific models. - Cons: Expensive, storage-heavy. → 2b: Parameter-Efficient Finetuning (PEFT) - Only a small subset of parameters is added and updated (e.g., via LoRA, Adapters, or IA³). - Base model remains frozen. - Much cheaper, ideal for rapid iteration and deployment. - Multi-LoRA architectures (e.g., used in Fireworks AI, Hugging Face PEFT) allow hosting multiple finetuned adapters on the same base model, drastically reducing cost and latency for serving. Step 3: Alignment (Usually via RLHF) Pretrained and task-tuned models can still produce unsafe or incoherent outputs. Alignment ensures they follow human intent. Alignment via RLHF (Reinforcement Learning from Human Feedback) involves: → Step 1: Supervised Fine-Tuning (SFT) - Human labelers craft ideal responses to prompts. - Model is fine-tuned on this dataset to mimic helpful behavior. - Limitation: Costly and not scalable alone. → Step 2: Reward Modeling (RM) - Humans rank multiple model outputs per prompt. - A reward model is trained to predict human preferences. - This provides a scalable, learnable signal of what “good” looks like. → Step 3: Reinforcement Learning (e.g., PPO, DPO) - The LLM is trained using the reward model’s feedback. - Algorithms like Proximal Policy Optimization (PPO) or newer Direct Preference Optimization (DPO) are used to iteratively improve model behavior. - DPO is gaining popularity over PPO for being simpler and more stable without needing sampled trajectories. Key Takeaways: → Pretraining = general knowledge (expensive) → Finetuning = domain or task adaptation (customize cheaply via PEFT) → Alignment = make it safe, helpful, and human-aligned (still labor-intensive but improving) Save the visual reference, and follow me (Aishwarya Srinivasan) for more no-fluff AI insights ❤️ PS: Visual inspiration: Sebastian Raschka, PhD

Shipping AI agents into production without governance is like deploying software without security, logs, or controls. It might work at first. But sooner or later, something breaks - silently. As AI agents move from experiments to real decision-makers, governance becomes infrastructure. This framework breaks AI Governance into the core functions every production-grade agent system needs: - Policy Rules Turn business and regulatory expectations into enforceable agent behavior - defining what agents can do, must avoid, and how they respond in restricted scenarios. - Access Control Limits agents to approved tools, datasets, and systems using identity verification, RBAC, and permission boundaries — preventing accidental or malicious misuse. - Audit Logs Create a full activity trail of agent decisions: what data was accessed, which tools were called, and why actions were taken — making every outcome traceable. - Risk Scoring Evaluates agent actions before execution, assigns risk levels, detects sensitive operations, and blocks unsafe decisions through thresholds and safety scoring. - Data Privacy Protects confidential information using PII detection, encryption, consent management, and retention policies — ensuring agents don’t leak regulated data. - Model Monitoring Tracks real-world agent performance: accuracy, drift, hallucinations, latency, and cost - keeping systems reliable after deployment. - Human Approvals Adds human-in-the-loop controls for high-impact actions, enabling escalation, overrides, and sign-offs when automation alone isn’t enough. - Incident Response Detects failures early and enables rapid containment through alerts, rollbacks, kill switches, and post-incident reporting to prevent repeat issues. The takeaway: AI agents don’t just need intelligence. They need guardrails. Without governance, agents become unpredictable. With governance, they become enterprise-ready. This is how organizations move from experimental AI to trustworthy, compliant, production systems. Save this if you’re building agentic systems. Share it with your platform or ML teams.

Few-shot Text Classification predicts the label of a given text after training with just a handful of labeled data. It's a powerful technique for overcoming real-world situations with scarce labeled data. SetFit is a fast, accurate few-shot NLP classification model perfect for intent detection in GenAI chatbots. In the pre-ChatGPT era, Intent Detection was an essential aspect of chatbots like Dialogflow. Chatbots would only respond to intents or topics that the developers explicitly programmed, ensuring they would stick closely to their intended use and prevent prompt injections. OpenAI's ChatGPT changed that with its incredible reasoning abilities, which allowed an LLM to decide how to answer users' questions on various topics without explicitly programming a flow for handling each topic. You just "prompt" the LLM on which topics to respond to and which to decline and let the LLM decide. However, numerous examples in the post-ChatGPT era have repeatedly shown how finicky a pure "prompt" based approach is. In my journey working with LLMs over the past year+, one of the most reliable methods I've found to restrict LLMs to a desired domain is to follow a 2-step approach that I've spoken about in the past: https://lnkd.in/g6cvAW-T 1. Preprocessing guardrail: An LLM call and heuristical rules to decide if the user's input is from an allowed topic. 2. LLM call: The chatbot logic, such as Retrieval Augmented Generation. The downside of this approach is the significant latency added by the additional LLM call in step 1. The solution is simple: replace the LLM call with a lightweight model that detects if the user's input is from an allowed topic. In other words, good old Intent Detection! With SetFit, you can build a highly accurate multi-label text classifier with as few as 10-15 examples per topic, making it an excellent choice for label-scarce intent detection problems. Following the documentation from the links below, I could train a SetFit model in seconds and have an inference time of <50ms on the CPU! If you're using an LLM as a few- or zero-shot classifier, I recommend checking out SetFit instead! 📝 SetFit Paper: https://lnkd.in/gy88XD3b 🌟 SetFit Github: https://lnkd.in/gC8br-EJ 🤗 SetFit Few Shot Learning Blog on Huggingface: https://lnkd.in/gaab_tvJ 🤗 SetFit Multi-Label Classification: https://lnkd.in/gz9mw4ey 🗣️ Intents in DialogFlow: https://lnkd.in/ggNbzxH6 Follow me for more tips on building successful ML and LLM products! Medium: https://lnkd.in/g2jAJn5 X: https://lnkd.in/g_JbKEkM #generativeai #llm #nlp #artificialintelligence #mlops #llmops

If you write content, then you have to pay attention. Linkedin has just published the do’s and don’ts for staying visible. In the future, content that does not show up inside AI-generated answers will effectively not exist. Here is my summary of the Linkedin guide: 𝟭. 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 𝗺𝘂𝘀𝘁 𝘀𝗲𝗿𝘃𝗲 𝘁𝘄𝗼 𝗮𝘂𝗱𝗶𝗲𝗻𝗰𝗲𝘀 𝗮𝘁 𝗼𝗻𝗰𝗲: human readers and AI models. Plain, accessible language, a neutral authoritative tone, and sections that make sense on their own increase the chance content is trusted, quotable, and surfaced in AI-generated answers. 𝟮. 𝗖𝗹𝗲𝗮𝗿, 𝘄𝗲𝗹𝗹-𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲𝗱 𝘀𝗲𝗻𝘁𝗲𝗻𝗰𝗲𝘀 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗼𝗳 𝗔𝗜 𝘃𝗶𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆: use complete declarative statements, keep sentences under 20 words, add inline context, and use factual language over metaphors so both humans and models understand meaning quickly. 𝟯. 𝗟𝗟𝗠 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗰𝗮𝗻𝗻𝗼𝘁 𝗿𝗲𝗽𝗹𝗮𝗰𝗲 𝗦𝗘𝗢 𝗳𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹𝘀: structure, depth, site performance, and technical foundations still determine visibility. 𝟰. 𝗛𝗲𝗮𝗱𝗶𝗻𝗴𝘀 𝗻𝗼𝘄 𝗮𝗰𝘁 𝗮𝘀 𝗮 𝗿𝗼𝗮𝗱𝗺𝗮𝗽 𝗳𝗼𝗿 𝗔𝗜: use descriptive main titles that state a full idea, break pages into question-driven subheadings, avoid vague labels, and maintain a logical broad-to-specific flow. 𝟱. 𝗟𝗲𝗮𝗱 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗮𝗻𝘀𝘄𝗲𝗿: start major sections with a short, definitive answer (30–80 words), then expand with detail - don’t bury the main point in explanation. 𝟲. 𝗜𝗻𝘁𝗲𝗿𝗻𝗮𝗹 𝗹𝗶𝗻𝗸𝘀 𝘁𝗲𝗮𝗰𝗵 𝗔𝗜 𝗵𝗼𝘄 𝘁𝗼𝗽𝗶𝗰𝘀 𝗰𝗼𝗻𝗻𝗲𝗰𝘁: use descriptive link text, connect related themes, organize content around central pages with supporting articles, and avoid overwhelming pages with excessive links. 𝟳. 𝗠𝗮𝗸𝗲 𝗰𝗼𝗻𝘁𝗲𝗻𝘁 𝗲𝗮𝘀𝗶𝗲𝗿 𝗳𝗼𝗿 𝗔𝗜 𝘁𝗼 𝗲𝘅𝘁𝗿𝗮𝗰𝘁: clearly define key terms, use lists and numbered steps, add real FAQ sections, and clearly show that the page is regularly updated with visible “last updated” dates. 𝟴. 𝗦𝗽𝗲𝗮𝗸 𝘁𝗵𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗼𝗳 𝗔𝗜: use schema markup (machine-readable labels added to pages) to describe what the content is about, who published it, and what is being offered so models understand context, trust the source, and surface accurate answers. 𝟵. 𝗦𝗲𝗻𝗱 𝗰𝗹𝗲𝗮𝗿 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗮𝗹 𝘀𝗶𝗴𝗻𝗮𝗹𝘀 𝘁𝗼 𝗔𝗜: use clean, accessible page structure and semantic markup (HTML tags that label page sections) along with natural-language titles, descriptions, and URLs so models can correctly interpret meaning and match content to real user questions. 𝟭𝟬. 𝗩𝗶𝘀𝘂𝗮𝗹𝘀 𝗶𝗻𝗳𝗹𝘂𝗲𝗻𝗰𝗲 𝗔𝗜 𝘃𝗶𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆: images and videos need captions, descriptive alt text (text descriptions for images), clear filenames, optimized metadata, and transcripts so AI can understand what they show and when to surface them. Source: How to optimize your owned content for AI search – Linkedin 𝐒𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 𝐭𝐨 𝐦𝐲 𝐧𝐞𝐰𝐬𝐥𝐞𝐭𝐭𝐞𝐫: https://lnkd.in/dkqhnxdg

One of the persistent challenges in using large language models (LLMs) is getting them to follow instructions reliably — especially when the instructions are subtle or domain-specific. DeepMind’s latest research introduces Symbol Tuning, a simple yet powerful fine-tuning method that significantly improves an LLM’s ability to follow symbolic prompts (e.g., bullet points, XML, Markdown, or code-like instructions) in zero-shot and few-shot settings. https://lnkd.in/gzKDdHQ2 Why this matters: 🔹 Improves instruction following in GPT-class models 🔹 Works with tiny amounts of data (just 100K tokens!) 🔹 Boosts performance in math, code, and reasoning-heavy tasks 🔹 Enhances models' ability to generalize across symbolic formats This has massive implications for building enterprise agents, RAG pipelines, and developer copilots that need high-precision, structured interaction with users or data. A great reminder: sometimes, small, well-targeted innovations create massive gains. #LLM #InContextLearning #SymbolTuning #PromptEngineering #DeepMind #GenAI #AIResearch #InstructionFollowing #EnterpriseAI #DeveloperTools

Building useful Knowledge Graphs will long be a Humans + AI endeavor. A recent paper lays out how best to implement automation, the specific human roles, and how these are combined. The paper, "From human experts to machines: An LLM supported approach to ontology and knowledge graph construction", provides clear lessons. These include: 🔍 Automate KG construction with targeted human oversight: Use LLMs to automate repetitive tasks like entity extraction and relationship mapping. Human experts should step in at two key points: early, to define scope and competency questions (CQs), and later, to review and fine-tune LLM outputs, focusing on complex areas where LLMs may misinterpret data. Combining automation with human-in-the-loop ensures accuracy while saving time. ❓ Guide ontology development with well-crafted Competency Questions (CQs): CQs define what the Knowledge Graph (KG) must answer, like "What preprocessing techniques were used?" Experts should create CQs to ensure domain relevance, and review LLM-generated CQs for completeness. Once validated, these CQs guide the ontology’s structure, reducing errors in later stages. 🧑⚖️ Use LLMs to evaluate outputs, with humans as quality gatekeepers: LLMs can assess KG accuracy by comparing answers to ground truth data, with humans reviewing outputs that score below a set threshold (e.g., 6/10). This setup allows LLMs to handle initial quality control while humans focus only on edge cases, improving efficiency and ensuring quality. 🌱 Leverage reusable ontologies and refine with human expertise: Start by using pre-built ontologies like PROV-O to structure the KG, then refine it with domain-specific details. Humans should guide this refinement process, ensuring that the KG remains accurate and relevant to the domain’s nuances, particularly in specialized terms and relationships. ⚙️ Optimize prompt engineering with iterative feedback: Prompts for LLMs should be carefully structured, starting simple and iterating based on feedback. Use in-context examples to reduce variability and improve consistency. Human experts should refine these prompts to ensure they lead to accurate entity and relationship extraction, combining automation with expert oversight for best results. These provide solid foundations to optimally applying human and machine capabilities to the very-important task of building robust and useful ontologies.

𝗘𝘃𝗲𝗿𝘆 𝗹𝗼𝗻𝗴-𝗿𝘂𝗻𝗻𝗶𝗻𝗴 𝗮𝗴𝗲𝗻𝘁 𝗲𝘃𝗲𝗻𝘁𝘂𝗮𝗹𝗹𝘆 𝗿𝘂𝗻𝘀 𝗼𝘂𝘁 𝗼𝗳 𝗰𝗼𝗻𝘁𝗲𝘅𝘁. A bigger context window does not fix this, attention quality degradation almost always happens in the range of 100-200 thousand tokens. The longer a session runs, the more tool outputs, retries, and history is added. Context compressions is what we implement as AI Engineers to handle inefficiencies caused by too many tokens in the context window. There are mainly three larger groups of how we can implement this. Everything you have read about context engineering in a sense of context compression is one of these three, or a combination. 𝟭. 𝗗𝗿𝗼𝗽 𝗶𝘁 Remove low-value tokens. Some of the strategies: ▪️ Sliding window over message history: keep the most recent turns. ▪️ Hard truncation at a token limit, head or tail. ▪️ Pruning stale tool outputs once they have been acted on. ▪️ Deduplicating repeated content and re-retrieved chunks. ▪️ … ✅ Cheapest option with close to zero added latency since it is encoded in deterministic application code. ❗️ Risk of dropping high value information when it is not. 𝟮. 𝗦𝘂𝗺𝗺𝗮𝗿𝗶𝘇𝗲 𝗶𝘁 Replace a long span of context with a shorter representation that stays in the window. ▪️ Compaction: fold older turns into a running recap, on a trigger (recursively, when the history is itself too big for one pass). ▪️ Compressing a large tool result down to the few fields that matter. ▪️ Distilling a chain of reasoning into the decision it reached. ▪️ Rolling state: keep a compact running summary of facts and decisions, updated as you go. ▪️ … ✅ You keep the signal and reduce the token count. ❗️ The cost is an extra model call and the chance that the summary quietly loses a detail you will miss three steps later. 𝟯. 𝗢𝗳𝗳𝗹𝗼𝗮𝗱 𝗶𝘁 Move content out of the window into external storage, then pull it back only when it is needed. ▪️ Retrieval over a vector store or document store. ▪️ Long-term memory the agent queries on demand. ▪️ Sub-agent delegation: hand a subtask to a fresh context, keep only the result. ▪️ Reference by pointer: store the blob, pass an ID or path instead of the content. ▪️ … ✅ Nothing is lost and the window stays small. ❗️ Requires complex retrieval infrastructure and risks fetching the wrong thing back in. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗱𝗼𝗻𝗲 𝗶𝗻 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻. Real systems rely on a mix of different strategies. E.g. 👉 Drop the obvious noise 👉 Summarize the middle history 👉 Offload anything that must survive the whole session. Pick per type of content, not once for the whole agent. The hard part? Each application benefits from different strategy combinations :) Which of the three is doing the heavy lifting in your agent today? 👇