Optimizing Technology Spending

Businesses don’t care if you built your DAG with Airflow or Mage or Databricks workflows! They care that it produces a high return on investment! ROI can be measured on two sides: - how much it impacts the business? What decisions are informed with this data? What products are enhanced with this data? How much revenue is generated or cost saved? - how expensive it is to maintain and operate? How often are data quality issues and other on call issues impacting engineering time? How difficult is it to add more fields and enhance the data sets? How much cloud costs is the pipeline using? If you can maximize the first number and minimize the second, you’ll be well on your way to creating massive impact for businesses!

You don't need a 2 trillion parameter model to tell you the capital of France is Paris. Be smart and route between a panel of models according to query difficulty and model specialty! New paper proposes a framework to train a router that routes queries to the appropriate LLM to optimize the trade-off b/w cost vs. performance. Overview: Model inference cost varies significantly: Per one million output tokens: Llama-3-70b ($1) vs. GPT-4-0613 ($60), Haiku ($1.25) vs. Opus ($75) The RouteLLM paper propose a router training framework based on human preference data and augmentation techniques, demonstrating over 2x cost saving on widely used benchmarks. They define the problem as having to choose between two classes of models: (1) strong models - produce high quality responses but at a high cost (GPT-4o, Claude3.5) (2) weak models - relatively lower quality and lower cost (Mixtral8x7B, Llama3-8b) A good router requires a deep understanding of the question’s complexity as well as the strengths and weaknesses of the available LLMs. Explore different routing approaches: - Similarity-weighted (SW) ranking - Matrix factorization - BERT query classifier - Causal LLM query classifier Neat Ideas to Build From: - Users can collect a small amount of in-domain data to improve performance for their specific use cases via dataset augmentation. - Can expand this problem from routing between a strong and weak LLM to a multiclass model routing approach where we have specialist models(language vision model, function calling model etc.) - Larger framework controlled by a router - imagine a system of 15-20 tuned small models and the router as the n+1'th model responsible for picking the LLM that will handle a particular query at inference time. - MoA architectures: Routing to different architectures of a Mixture of Agents would be a cool idea as well. Depending on the query you decide how many proposers there should be, how many layers in the mixture, what the aggregate models should be etc. - Route based caching: If you get redundant queries that are slightly different then route the query+previous answer to a small model to light rewriting instead of regenerating the answer

Indian IT Stock Selloff: Strong Deal Wins Suggest Overdone Correction Indian IT stocks have tumbled 10-20% in the past month on concerns that artificial intelligence will disrupt the sector's business model. However, a closer look at recent deal wins data suggests this pessimism may be excessive. If AI were truly decimating demand for IT services, it would be reflected in slowing order books. Yet the numbers tell a different story. TCS, the sector bellwether, secured $9.4 billion in Q1 FY26, $10.0 billion in Q2, and $9.3 billion in Q3—demonstrating remarkable consistency well above the $9 billion mark each quarter. Infosys showed accelerating momentum, with deal wins growing from $3.8 billion in Q1 to $4.8 billion in Q3, its strongest quarter. HCL Tech's trajectory was even more impressive, nearly doubling from $1.8 billion in Q1 to approximately $3.0 billion by Q3. Wipro opened the fiscal year strong with $5.0 billion in Q1 bookings, while LTIMindtree posted its highest-ever quarterly TCV of $1.69 billion in Q3. These figures represent actual client commitments—enterprises signing multi-year contracts for digital transformation, cloud migration, and modernization work. Notably, all five companies emphasized that AI-led transformation deals were among their key growth drivers, not detractors. The paradox is clear: if AI were cannibalizing traditional IT services demand, deal pipelines would be contracting, not expanding. Instead, companies are reporting robust bookings with strong net-new business components. Infosys's Q3 deals were 57% net new, while 67% of its Q2 wins represented fresh business. The market appears to be pricing in a doomsday scenario that current operational data doesn't support. While AI will undoubtedly reshape the industry, the transition is creating as much new work as it displaces. With valuations now reflecting deep pessimism despite solid fundamentals, the recent correction looks overdone. Smart investors might view this as a buying opportunity rather than a signal to flee.

If you’re an AI engineer trying to optimize your LLMs for inference, here’s a quick guide for you 👇 Efficient inference isn’t just about faster hardware, it’s a multi-layered design problem. From how you compress prompts to how your memory is managed across GPUs, everything impacts latency, throughput, and cost. Here’s a structured taxonomy of inference-time optimizations for LLMs: 1. Data-Level Optimization Reduce redundant tokens and unnecessary output computation. → Input Compression:  - Prompt Pruning, remove irrelevant history or system tokens  - Prompt Summarization, use model-generated summaries as input  - Soft Prompt Compression, encode static context using embeddings  - RAG, replace long prompts with retrieved documents plus compact queries → Output Organization:  - Pre-structure output to reduce decoding time and minimize sampling steps 2. Model-Level Optimization (a) Efficient Structure Design → Efficient FFN Design, use gated or sparsely-activated FFNs (e.g., SwiGLU) → Efficient Attention, FlashAttention, linear attention, or sliding window for long context → Transformer Alternates, e.g., Mamba, Reformer for memory-efficient decoding → Multi/Group-Query Attention, share keys/values across heads to reduce KV cache size → Low-Complexity Attention, replace full softmax with approximations (e.g., Linformer) (b) Model Compression → Quantization:  - Post-Training, no retraining needed  - Quantization-Aware Training, better accuracy, especially <8-bit → Sparsification:  - Weight Pruning, Sparse Attention → Structure Optimization:  - Neural Architecture Search, Structure Factorization → Knowledge Distillation:  - White-box, student learns internal states  - Black-box, student mimics output logits → Dynamic Inference, adaptive early exits or skipping blocks based on input complexity 3. System-Level Optimization (a) Inference Engine → Graph & Operator Optimization, use ONNX, TensorRT, BetterTransformer for op fusion → Speculative Decoding, use a smaller model to draft tokens, validate with full model → Memory Management, KV cache reuse, paging strategies (e.g., PagedAttention in vLLM) (b) Serving System → Batching, group requests with similar lengths for throughput gains → Scheduling, token-level preemption (e.g., TGI, vLLM schedulers) → Distributed Systems, use tensor, pipeline, or model parallelism to scale across GPUs My Two Cents 🫰 → Always benchmark end-to-end latency, not just token decode speed → For production, 8-bit or 4-bit quantized models with MQA and PagedAttention give the best price/performance → If using long context (>64k), consider sliding attention plus RAG, not full dense memory → Use speculative decoding and batching for chat applications with high concurrency → LLM inference is a systems problem. Optimizing it requires thinking holistically, from tokens to tensors to threads. Image inspo: A Survey on Efficient Inference for Large Language Models ---- Follow me (Aishwarya Srinivasan) for more AI insights!

Having a lot of data isn’t the same thing as having high-value data. If you’re having a hard time explaining that to executive leaders, try a different approach. Teach them how to put a dollar value on the business’s data. Every curated dataset creates new opportunities for the business, and that’s the connection between data and profit. The simplest data valuation method is called ‘With & Without’. The business thinks that every dataset creates the same value, so I run an early experiment to disprove that assumption. I turn off access to datasets that stakeholders believe are high value and wait for the complaints to roll in. In most cases, no one notices. Three months later, I propose putting the dataset into cold storage. Business leaders push back, saying their teams would grind to a halt without access to those datasets. I tell them about the experiment. Now I can start a rational conversation about connecting data to use cases and putting a dollar value on each dataset. Data doesn’t create value for two reasons: 1️⃣ It’s incomplete. The data required to support the use case isn’t being gathered holistically. Sometimes that’s an accessibility issue. Other times, the use case, workflow, and outcomes aren’t understood well enough to know what data is necessary. 2️⃣ It lacks context. Data points aren’t enough to support use cases. Context about the process, product, person, intent, and outcome is required. Until data is gathered contextually, its value creation is limited. Connecting datasets with opportunities creates the justification for changing how the business gathers and leverages data. Putting a dollar value on contextual datasets quantifies the ROI of information architecture and engineering initiatives. That’s the shortest path to getting budget and buy-in. Quantify value in terms that business leaders care about and show them a clear connection with outcomes they believe are essential.

AI adoption and usage is skyrocketing, but usage bills are running up fast. The good news? There’s a bunch of ways to cut costs without sacrificing quality. Here are ten practical ways to optimize your (and your team’s) AI spend: 1. Right model, right task - You don't need a frontier model to summarize an email or parse a PDF. Routing simple tasks to smaller models can cut costs by 40% to 85% with a negligible drop in quality. Different models are better at different tasks. 2. Mind the token asymmetry - Output tokens are usually 3x to 4x more expensive than input tokens. Be explicit about expected outputs - asking for concise summaries, structured JSON or a specific output format can cut down costs and rework. 3. Pre-process first - Don't waste expensive model context windows on raw formatting noise eg convert PDFs or PowerPoints to Markdown using standard software before involving an LLM. This is especially important for very large data sets or documents. 4. Use single-turn alignment - Back and forth chat resends the whole conversation history with every prompt, compounding input costs. Try to put all your requirements, constraints, and output formats in one clear initial prompt. 5. Maximize token allowances - Schedule non-urgent batch tasks for off-peak hours or near the end of a billing cycle to use up existing capacity. Share unused tokens across team members. 6. Clean up inactive seats - Research shows 30% to 50% of enterprise SaaS licenses go underutilised. Regularly audit user accounts, reclaim seats from inactive staff, and right-size plans before renewals. 7. Track usage - You can't manage what you don't measure. Implement basic monitoring to spot runaway queries early, and identify high-value internal use cases. 8. Share the learning - Trial-and-error across a team burns through API budgets fast. Share best practices and firm wide knowledge, build internal prompt libraries to share best practices so employees aren't reinventing the wheel. 9. Have options - Have a backup in case one tool hits an unexpected bottleneck. Some providers let you carry forward unused tokens across billing cycles, and there are some free options too. 10. Ask for help - You can ask some models to help you plan an activity and get it to spin up cheaper subagents to keep the cost down and quality right. Managing AI costs isn't about being cheap - it's about being smart and operational maturity. If the model providers do keep increasing the cost of their frontier model then we’ll all have to be smarter in how we use their tools. What is your team’s favorite strategy for keeping AI costs under control?

Is your Shopify store a maze of apps? Simplify to amplify. Less is more when it comes to optimizing for conversion! Imagine walking into a store with products scattered everywhere. Confusing, right? Your online store isn't so different. Too many apps can clutter your digital shelves, slow down your site, and frustrate potential customers. Here's how to streamline your Shopify store for maximum impact: ✔ Audit your apps → List all the apps you're currently using. → Identify which ones are truly essential. → Remove the ones that don’t directly contribute to your core goals: conversion and customer satisfaction. ✔ Evaluate functionality over quantity → Opt for apps that provide multiple functions. → This reduces complexity and improves site speed. ✔ Test site speed regularly → Use tools like Google PageSpeed Insights. → Monitor how each app impacts your speed. → Prioritize apps that are lightweight and optimized. ✔ Continuous review → The #ecommerce landscape evolves rapidly. → Regularly reassess your app suite to ensure you're aligned with the latest trends and customer expectations. Remember, it's not about how many apps you have. It's about having the right ones. Simplify your digital storefront to amplify your sales. What app has made the biggest impact on your #Shopify store?

𝗔𝗿𝗲 𝘆𝗼𝘂 𝗽𝗿𝗼𝗮𝗰𝘁𝗶𝘃𝗲𝗹𝘆 𝗺𝗮𝗻𝗮𝗴𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 𝗦𝗼𝘂𝗿𝗰𝗲-𝘁𝗼-𝗣𝗮𝘆 𝘁𝗲𝗰𝗵𝗻𝗼𝗹𝗼𝗴𝘆 𝗰𝗼𝘀𝘁𝘀? If not, why let savings from smart Procurement slip away due to outdated technology or suboptimal use? S2P technology plays a central role in cost management, yet many companies lack a strategic approach to continuously assess and optimise their tech stack. Companies can adopt Bain & Co’s "𝗥𝗲𝗱𝘂𝗰𝗲, 𝗥𝗲𝗽𝗹𝗮𝗰𝗲, 𝗮𝗻𝗱 𝗥𝗲𝘁𝗵𝗶𝗻𝗸" model to continuously evaluate their technology infrastructure and costs, ensuring a more optimised and sustainable cost profile. Here is the model in action for Source to Pay technology cost optimisation: ▪️ 𝗥𝗲𝗱𝘂𝗰𝗲 to recover 10 to 20% of costs through short-term actions such as - adjusting licenses to match actual usage and adoption patterns - discontinuing features or functionalities that add little value - switching off modules where business capabilities have not yet caught up Avoid over-licensing by matching user access to actual needs, ensuring modules align with Procurement’s readiness. ▪️ 𝗥𝗲𝗽𝗹𝗮𝗰𝗲 to yield 20 to 30% of savings by - transitioning to cost-optimal, flexible solutions and getting out of lock-ins - switching subscription models when premium offerings are unnecessary - consolidating overlapping tools that offer similar features For example, merge multiple eSourcing tools into a primary platform and adopt a tender-based pricing for niche auction needs. This helps to adjust the cost profile of your Source to Pay technology with the actual needs. ▪️ 𝗥𝗲𝘁𝗵𝗶𝗻𝗸 to realise up to 40% cost optimisation by: - reimagining the architecture with a modular, composable design - automating and orchestrating processes and integrating new digital tools - reevaluate the mix of best-of-breed solutions vs integrated suites A new Procurement strategy requires a fresh look at the S2P tech stack to ensure it adapts and supports growth cost-effectively, while offering flexibility through additional digital levers like AI and automation. 𝗢𝗽𝘁𝗶𝗺𝗶𝘀𝗶𝗻𝗴 𝗦𝟮𝗣 𝘁𝗲𝗰𝗵𝗻𝗼𝗹𝗼𝗴𝘆 𝗶𝘀 𝗮 𝗰𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗷𝗼𝘂𝗿𝗻𝗲𝘆, 𝗻𝗼𝘁 𝗮 𝗼𝗻𝗲-𝘁𝗶𝗺𝗲 𝗲𝗳𝗳𝗼𝗿𝘁, especially with contractual commitments, sunk costs, and change management challenges. Rather than following IT preferences and standards, it’s about keeping technology fresh and aligned with business needs as they evolve. ❓How do you manage your S2P technology to adapt to changing business needs while maintaining cost efficiency.

It surprises me how many e-commerce brands pretend to offer a personalized storefront, but show the same store to everyone. The attached visual that shows what a modern storefront actually looks like behind the scenes, which is a simple system that reacts in real time. Thought it would be useful to break this down into three stages with the recommended tech stack below: Stage 1: Signals (data in) You capture (live) what’s already happening the moment someone arrives. How they got there, what they’re doing, what device they’re on, and whether they’ve bought before. Typical stack: • Segment or RudderStack for event capture • Shopify events and customer data • Google Tag Manager • Meta / TikTok UTMs for paid context Focus on clean, real-time signals without overengineering identity. Stage 2: Decisions (what to show) Those signals get turned into a simple decision immediately. Which message, which products, which path makes sense for this visitor right now. If it’s not fast enough to change the first screen, it doesn’t count. Typical stack: • Dynamic Yield or Nosto • Vercel edge logic • Cloudflare Workers • Simple rules or light models, not heavy AI Remember, speed beats sophistication. Stage 3: Experience (what changes) The storefront responds on arrival. The hero, first product grid, and primary CTA change instantly so the site feels relevant from the first moment. Typical stack: • Shopify Hydrogen or native Shopify sections • Contentful or Optimizely • Server-side or edge-rendered changes, not client-side flicker Important, personalize above the fold first. A returning high-value customer sees new arrivals and a faster path to checkout. A first-time visitor from paid sees a clearer offer and fewer choices. A deal-driven shopper sees bundles and savings upfront. Everything else comes later. If you want to start without overengineering: • Pick the two audiences that matter most • Personalize only the hero and first product grid • Measure lift on conversion rate and revenue per session • Add complexity only after this works Start simple: focus on one working example that proves the storefront can adapt in real time in a way customers actually feel.

Today’s The Times coverage makes clear that Innovate UK, the UK government’s innovation agency, is embarking on a significant strategic shift in how it deploys its £1.1 billion budget, moving away from broad‑based grant support for hundreds of thousands of innovators each year toward concentrating resources on a smaller group of high‑potential early‑stage technology companies. Over recent years Innovate UK’s provided a wide range of grants and programmes; under the new approach, the agency intends to focus on several thousand companies with clear prospects to scale significantly and deliver major economic impact. The emphasis will be on sectors deemed strategically important, such as advanced manufacturing, life sciences, digital technologies including artificial intelligence, semiconductors, and quantum computing. This recalibration is designed to incubate “future industry giants” and bolster the UK’s competitiveness in key global technology arenas. Innovate UK plans to discontinue or repurpose legacy grant streams, such as the well‑known Smart Grants, and reallocate those resources toward more targeted, sector‑specific support. Another notable change highlighted in the article is the repositioning of the Women in Innovation grant to focus on female‑led high‑growth tech enterprises, signalling an intention to align innovation funding more closely with both strategic sector goals and broader inclusion objectives. In addition, programmes such as the Business Growth Advice service and support for Catapult centres will be realigned to place stronger emphasis on company‑level impact and scaling outcomes. New initiatives are also being introduced, including “Velocity,” a concierge‑style service intended to help high‑growth firms navigate early‑stage challenges, and an expanded Growth Catalyst scheme offering sizeable, strategic grants. The new strategy fosters closer engagement with private capital aiming to leverage its technical expertise to provide credible due diligence bridging public funding with private investment. By doing so, the agency intends to lower the barriers to private capital for emerging firms and create clearer pathways for later‑stage financing. The reporting underscores a broader shift in the UK’s innovation funding ecosystem: public support is being refocused toward fewer but deeper bets. Reactions from founders, ecosystem practitioners and commentators illustrate a nuanced picture, there is concern that narrowing the funding aperture too far risks excluding viable innovators that don’t yet meet rigid “high‑growth” definitions. The tension between strategic concentration of funding for maximum impact and the risk of leaving promising early‑stage innovators behind is interesting. The test of the new strategy will be how effectively it navigates these tensions in implementation, maintaining broad ecosystem vitality while driving deeper impact through focused support. #UKRI #innovateuk #innovation #HMtreasury #startups