Prompt Engineering Applications

Last week, I described four design patterns for AI agentic workflows that I believe will drive significant progress: Reflection, Tool use, Planning and Multi-agent collaboration. Instead of having an LLM generate its final output directly, an agentic workflow prompts the LLM multiple times, giving it opportunities to build step by step to higher-quality output. Here, I'd like to discuss Reflection. It's relatively quick to implement, and I've seen it lead to surprising performance gains. You may have had the experience of prompting ChatGPT/Claude/Gemini, receiving unsatisfactory output, delivering critical feedback to help the LLM improve its response, and then getting a better response. What if you automate the step of delivering critical feedback, so the model automatically criticizes its own output and improves its response? This is the crux of Reflection. Take the task of asking an LLM to write code. We can prompt it to generate the desired code directly to carry out some task X. Then, we can prompt it to reflect on its own output, perhaps as follows: Here’s code intended for task X: [previously generated code] Check the code carefully for correctness, style, and efficiency, and give constructive criticism for how to improve it. Sometimes this causes the LLM to spot problems and come up with constructive suggestions. Next, we can prompt the LLM with context including (i) the previously generated code and (ii) the constructive feedback, and ask it to use the feedback to rewrite the code. This can lead to a better response. Repeating the criticism/rewrite process might yield further improvements. This self-reflection process allows the LLM to spot gaps and improve its output on a variety of tasks including producing code, writing text, and answering questions. And we can go beyond self-reflection by giving the LLM tools that help evaluate its output; for example, running its code through a few unit tests to check whether it generates correct results on test cases or searching the web to double-check text output. Then it can reflect on any errors it found and come up with ideas for improvement. Further, we can implement Reflection using a multi-agent framework. I've found it convenient to create two agents, one prompted to generate good outputs and the other prompted to give constructive criticism of the first agent's output. The resulting discussion between the two agents leads to improved responses. Reflection is a relatively basic type of agentic workflow, but I've been delighted by how much it improved my applications’ results. If you’re interested in learning more about reflection, I recommend: - Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023) - Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023) - CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024) [Original text: https://lnkd.in/g4bTuWtU ]

When working with multiple LLM providers, managing prompts, and handling complex data flows — structure isn't a luxury, it's a necessity. A well-organized architecture enables: → Collaboration between ML engineers and developers → Rapid experimentation with reproducibility → Consistent error handling, rate limiting, and logging → Clear separation of configuration (YAML) and logic (code) 𝗞𝗲𝘆 𝗖𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀 𝗧𝗵𝗮𝘁 𝗗𝗿𝗶𝘃𝗲 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 It’s not just about folder layout — it’s how components interact and scale together: → Centralized configuration using YAML files → A dedicated prompt engineering module with templates and few-shot examples → Properly sandboxed model clients with standardized interfaces → Utilities for caching, observability, and structured logging → Modular handlers for managing API calls and workflows This setup can save teams countless hours in debugging, onboarding, and scaling real-world GenAI systems — whether you're building RAG pipelines, fine-tuning models, or developing agent-based architectures. → What’s your go-to project structure when working with LLMs or Generative AI systems? Let’s share ideas and learn from each other.

Few Lessons from Deploying and Using LLMs in Production Deploying LLMs can feel like hiring a hyperactive genius intern—they dazzle users while potentially draining your API budget. Here are some insights I’ve gathered: 1. “Cheap” is a Lie You Tell Yourself: Cloud costs per call may seem low, but the overall expense of an LLM-based system can skyrocket. Fixes: - Cache repetitive queries: Users ask the same thing at least 100x/day - Gatekeep: Use cheap classifiers (BERT) to filter “easy” requests. Let LLMs handle only the complex 10% and your current systems handle the remaining 90%. - Quantize your models: Shrink LLMs to run on cheaper hardware without massive accuracy drops - Asynchronously build your caches — Pre-generate common responses before they’re requested or gracefully fail the first time a query comes and cache for the next time. 2. Guard Against Model Hallucinations: Sometimes, models express answers with such confidence that distinguishing fact from fiction becomes challenging, even for human reviewers. Fixes: - Use RAG - Just a fancy way of saying to provide your model the knowledge it requires in the prompt itself by querying some database based on semantic matches with the query. - Guardrails: Validate outputs using regex or cross-encoders to establish a clear decision boundary between the query and the LLM’s response. 3. The best LLM is often a discriminative model: You don’t always need a full LLM. Consider knowledge distillation: use a large LLM to label your data and then train a smaller, discriminative model that performs similarly at a much lower cost. 4. It's not about the model, it is about the data on which it is trained: A smaller LLM might struggle with specialized domain data—that’s normal. Fine-tune your model on your specific data set by starting with parameter-efficient methods (like LoRA or Adapters) and using synthetic data generation to bootstrap training. 5. Prompts are the new Features: Prompts are the new features in your system. Version them, run A/B tests, and continuously refine using online experiments. Consider bandit algorithms to automatically promote the best-performing variants. What do you think? Have I missed anything? I’d love to hear your “I survived LLM prod” stories in the comments!

In the last three months alone, over ten papers outlining novel prompting techniques were published, boosting LLMs’ performance by a substantial margin. Two weeks ago, a groundbreaking paper from Microsoft demonstrated how a well-prompted GPT-4 outperforms Google’s Med-PaLM 2, a specialized medical model, solely through sophisticated prompting techniques. Yet, while our X and LinkedIn feeds buzz with ‘secret prompting tips’, a definitive, research-backed guide aggregating these advanced prompting strategies is hard to come by. This gap prevents LLM developers and everyday users from harnessing these novel frameworks to enhance performance and achieve more accurate results. https://lnkd.in/g7_6eP6y In this AI Tidbits Deep Dive, I outline six of the best and recent prompting methods: (1) EmotionPrompt - inspired by human psychology, this method utilizes emotional stimuli in prompts to gain performance enhancements (2) Optimization by PROmpting (OPRO) - a DeepMind innovation that refines prompts automatically, surpassing human-crafted ones. This paper discovered the “Take a deep breath” instruction that improved LLMs’ performance by 9%. (3) Chain-of-Verification (CoVe) - Meta's novel four-step prompting process that drastically reduces hallucinations and improves factual accuracy (4) System 2 Attention (S2A) - also from Meta, a prompting method that filters out irrelevant details prior to querying the LLM (5) Step-Back Prompting - encouraging LLMs to abstract queries for enhanced reasoning (6) Rephrase and Respond (RaR) - UCLA's method that lets LLMs rephrase queries for better comprehension and response accuracy Understanding the spectrum of available prompting strategies and how to apply them in your app can mean the difference between a production-ready app and a nascent project with untapped potential. Full blog post https://lnkd.in/g7_6eP6y

Small variations in prompts can lead to very different LLM responses. Research that measures LLM prompt sensitivity uncovers what matters, and the strategies to get the best outcomes. A new framework for prompt sensitivity, ProSA, shows that response robustness increases with factors including higher model confidence, few-shot examples, and larger model size. Some strategies you should consider given these findings: 💡 Understand Prompt Sensitivity and Test Variability: LLMs can produce different responses with minor rephrasings of the same prompt. Testing multiple prompt versions is essential, as even small wording adjustments can significantly impact the outcome. Organizations may benefit from creating a library of proven prompts, noting which styles perform best for different types of queries. 🧩 Integrate Few-Shot Examples for Consistency: Including few-shot examples (demonstrative samples within prompts) enhances the stability of responses, especially in larger models. For complex or high-priority tasks, adding a few-shot structure can reduce prompt sensitivity. Standardizing few-shot examples in key prompts across the organization helps ensure consistent output. 🧠 Match Prompt Style to Task Complexity: Different tasks benefit from different prompt strategies. Knowledge-based tasks like basic Q&A are generally less sensitive to prompt variations than complex, reasoning-heavy tasks, such as coding or creative requests. For these complex tasks, using structured, example-rich prompts can improve response reliability. 📈 Use Decoding Confidence as a Quality Check: High decoding confidence—the model’s level of certainty in its responses—indicates robustness against prompt variations. Organizations can track confidence scores to flag low-confidence responses and identify prompts that might need adjustment, enhancing the overall quality of outputs. 📜 Standardize Prompt Templates for Reliability: Simple, standardized templates reduce prompt sensitivity across users and tasks. For frequent or critical applications, well-designed, straightforward prompt templates minimize variability in responses. Organizations should consider a “best-practices” prompt set that can be shared across teams to ensure reliable outcomes. 🔄 Regularly Review and Optimize Prompts: As LLMs evolve, so may prompt performance. Routine prompt evaluations help organizations adapt to model changes and maintain high-quality, reliable responses over time. Regularly revisiting and refining key prompts ensures they stay aligned with the latest LLM behavior. Link to paper in comments.

I recently went through the Prompt Engineering guide by Lee Boonstra from Google, and it offers valuable, practical insights. It confirms that getting the best results from LLMs is an iterative engineering process, not just casual conversation. Here are some key takeaways I found particularly impactful: 1. 𝐈𝐭'𝐬 𝐌𝐨𝐫𝐞 𝐓𝐡𝐚𝐧 𝐉𝐮𝐬𝐭 𝐖𝐨𝐫𝐝𝐬: Effective prompting goes beyond the text input. Configuring model parameters like Temperature (for creativity vs. determinism), Top-K/Top-P (for sampling control), and Output Length is crucial for tailoring the response to your specific needs. 2. 𝐆𝐮𝐢𝐝𝐚𝐧𝐜𝐞 𝐓𝐡𝐫𝐨𝐮𝐠𝐡 𝐄𝐱𝐚𝐦𝐩𝐥𝐞𝐬: Zero-shot, One-shot, and Few-shot prompting aren't just academic terms. Providing clear examples within your prompt is one of the most powerful ways to guide the LLM on desired output format, style, and structure, especially for tasks like classification or structured data generation (e.g., JSON). 3. 𝐔𝐧𝐥𝐨𝐜𝐤𝐢𝐧𝐠 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠: Techniques like Chain of Thought (CoT) prompting – asking the model to 'think step-by-step' – significantly improve performance on complex tasks requiring reasoning (logic, math). Similarly, Step-back prompting (considering general principles first) enhances robustness. 4. 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 𝐚𝐧𝐝 𝐑𝐨𝐥𝐞𝐬 𝐌𝐚𝐭𝐭𝐞𝐫: Explicitly defining the System's overall purpose, providing relevant Context, or assigning a specific Role (e.g., "Act as a senior software architect reviewing this code") dramatically shapes the relevance and tone of the output. 5. 𝐏𝐨𝐰𝐞𝐫𝐟𝐮𝐥 𝐟𝐨𝐫 𝐂𝐨𝐝𝐞: The guide highlights practical applications for developers, including generating code snippets, explaining complex codebases, translating between languages, and even debugging/reviewing code – potential productivity boosters. 6. 𝐁𝐞𝐬𝐭 𝐏𝐫𝐚𝐜𝐭𝐢𝐜𝐞𝐬 𝐚𝐫𝐞 𝐊𝐞𝐲: Specificity: Clearly define the desired output. Ambiguity leads to generic results. Instructions > Constraints: Focus on telling the model what to do rather than just what not to do. Iteration & Documentation: This is critical. Documenting prompt versions, configurations, and outcomes (using a structured template, like the one suggested) is essential for learning, debugging, and reproducing results. Understanding these techniques allows us to move beyond basic interactions and truly leverage the power of LLMs. What are your go-to prompt engineering techniques or best practices? Let's discuss! #PromptEngineering #AI #LLM

When it comes to building truly reliable AI agents, I’ve realized that prompting isn’t just about giving instructions, it’s about crafting intentional conversations that guide the model with clarity, structure, and context. These prompt engineering techniques have shaped the way we should think about deploying LLM-powered systems in the real world. The goal isn’t just output, it’s precision, traceability, and contextual awareness baked into every generation It starts with being hyper-specific and detailed—think of your LLM like a new team member. The clearer you are about their task, constraints, and tone, the better they perform. Pair that with persona prompting to set the right expectations, and suddenly your LLM behaves more like a domain expert than a chatbot. From there, you outline the task and give it a plan, making even the most complex workflows feel digestible for the model. Structuring the prompt with bullet points, Markdown, or even XML-like tags makes the output predictable and parseable, especially when dealing with automation pipelines. I often add few-shot examples directly in the prompt to guide the model with real-world context. These examples anchor behavior and dramatically reduce misunderstanding. Things really start to scale with prompt folding and dynamic generation. In multi-stage flows, I let earlier outputs shape the next prompt. It’s how you make agents more adaptive. Still, I always include an escape hatch—asking the LLM to admit when it doesn't know something. It’s a small tweak that prevents hallucinations and builds trust. For deeper insight, I include debug info or thinking traces. Asking the LLM to explain its logic is like reading the footnotes of its thought process—great for debugging and refinement. But the real crown jewel? Your eval suite. Prompting without evaluation is like flying blind. Having test cases lets you track improvements, regressions, and stability across iterations. Finally, LLM personalities and distillation matter more than people think. Some models need more hand-holding; others just “get it.” I often use a bigger model to refine prompts and then distill them down for faster, cheaper inference with smaller models. Building reliable AI agents, don’t overlook the prompt. Get intentional, get structured.

Most people treat AI like a search engine. That’s the problem. Most people type a question and hope for the best. That’s not prompting. That’s guessing. The difference between a weak AI result and a strong one is almost never the tool. It is the instruction and context. Here’s a structured prompting workflow that works: 1. Set the role — Give AI a clear job. Coach, editor, strategist. 2. Define the goal — Name the outcome, audience, and success criteria. 3. Add context — Share background, limits, and examples of what good looks like. 4. Ask for a restatement — Let AI repeat the task back. Catch misunderstandings early. 5. Work in steps — Start with an outline. Then details. Then edits. 6. Use reflection loops — Ask what might be missing. Then ask it to improve. 7. Save repeatable chats — Keep useful threads as ongoing projects for future work. Stop treating AI like a Google search. Start treating it like a competent junior teammate who needs a good brief. The better your instructions, the better your results. ♻️ Share if this resonates ➕ Follow Jason Moccia for more insights on AI and leadership.

I’ve spent thousands of hours prompting LLMs (many times badly lol), but recently carved out space to get more systematic about it. I brushed up on LLM fundamentals from a course from DeepLearning.AI. It focused on prompt structure, demystifying “perfect prompt” myths, and testing outputs through small API prototypes. Got me to move beyond one-off prompts and think more systematically about how models behave, and how the principles behind how to engage them echo what we already know from building software. A few things stuck out: - Avoid short prompts. Detailed, explicit instructions guide the model better. Treat them like product specs instead of strings of chat. Or Tweets -  LLMs tend to rush answers! If you force the model to “work it out first,” you get fewer brittle answers. Same as giving a person time to think. Otherwise, it rushes wrong answers - Iteration is KEY. Prompt design mirrors debugging or ML training: draft → test → refine. There is no “perfect prompt”!!!!!!! - Delimiters (``` or <tags>) act like syntax for the model. They set the boundaries so the system knows exactly what text to work with, reducing ambiguity - Hallucinations are verrry real but can be reduced. Ask the model to cite from source text before answering, so claims can be traced more easily. Think about designing “evidence-first” prompts So while prompting might look or feel like "chatting," under the hood you’re engaging a system that encodes meaning, relationships, and logic. Treat it accordingly.

Most companies overcomplicate AI implementation. I see teams making the same mistakes: jumping to complex AI solutions (agents, toolchains, orchestration) when all they need is a simple prompt. This creates bloated systems, wastes time, and becomes a maintenance nightmare. While everyone's discussing Model Context Protocol, I've been exploring another MCP: the Minimum Complexity Protocol. The framework forces teams to start simple and only escalate when necessary: Level 1: Non-LLM Solution → Would a boolean, logic or rule based system solve the problem more efficiently? Level 2: Single LLM Prompt → Start with a single, straightforward prompt to a general purpose model. Experiment with different models - some are better with particular tasks. Level 3: Preprocess Data → Preprocess your inputs. Split long documents, simplify payloads. Level 4: Divide & Conquer → Break complex tasks into multiple focused prompts where each handles one specific aspect. LLMs are usually better at handling a specific task at a time. Level 5: Few Shot Prompting → Add few-shot examples within your prompt to guide the model toward better outputs. A small number of examples can greatly increase accuracy. Level 6: Prompt Chaining → Connect multiple prompts in a predetermined sequence. The output of one prompt becomes the input for the next. Level 7: Resource Injection → Implement RAG to connect your model to relevant external knowledge bases such as APIs, databases and vector stores. Level 8: Fine Tuning → Fine tune existing models on your domain specific data when other techniques are no longer effective. Level 9 (Optional): Build Your Own Model → All else fails? Develop custom models when the business case strongly justifies the investment. Level 10: Agentic Tool Selection → LLMs determine which tools or processes to execute for a given job. The tools can recursively utilise more LLMs while accessing and updating resources. Human oversight is still recommended here. Level 11: Full Agency → Allow agents to make decisions, call tools, and access resources independently. Agents self-evaluate accuracy and iteratively operate until the goal is completed. At each level, measure accuracy via evals and establish human review protocols. The secret to successful AI implementation isn't using the most advanced technique. It's using the simplest solution that delivers the highest accuracy with the least effort. What's your experience? Are you seeing teams overcomplicate their AI implementations?