Retail Sales Forecasting Methods

Modeling something like time series goes past just throwing features in a model. In the world of time series data, each observation is associated with a specific time point, and part of our goal is to harness the power of temporal dependencies. Enter autoregression and lagging -  concepts that taps into the correlation between current and past observations to make forecasts.  At its core, autoregression involves modeling a time series as a function of its previous values. The current value relies on its historical counterparts. To dive a bit deeper, we use lagged values as features to predict the next data point. For instance, in a simple autoregressive model of order 1 (AR(1)), we predict the current value based on the previous value multiplied by a coefficient. The coefficient determines the impact of the past value on the present one only one time period previous. One popular approach that can be used in conjunction with autoregression is the ARIMA (AutoRegressive Integrated Moving Average) model. ARIMA is a powerful time series forecasting method that incorporates autoregression, differencing, and moving average components. It's particularly effective for data with trends and seasonality. ARIMA can be fine-tuned with parameters like the order of autoregression, differencing, and moving average to achieve accurate predictions. When I was building ARIMAs for econometric time series forecasting, in addition to autoregression where you're lagging the whole model, I was also taught to lag the individual economic variables. If I was building a model for energy consumption of residential homes, the number of housing permits each month would be a relevant variable. Although, if there’s a ton of housing permits given in January, you won’t see the actual effect of that until later when the houses are built and people are actually consuming energy! That variable needed to be lagged by several months. Another innovative strategy to enhance time series forecasting is the use of neural networks, particularly Recurrent Neural Networks (RNNs) or Long Short-Term Memory (LSTM) networks. RNNs and LSTMs are designed to handle sequential data like time series. They can learn complex patterns and long-term dependencies within the data, making them powerful tools for autoregressive forecasting. Neural networks are fed with past time steps as inputs to predict future values effectively. In addition to autoregression in neural networks, I also used lagging there too! When I built an hourly model to forecast electric energy consumption, I actually built 24 individual models, one for each hour, and each hour lagged on the previous one. The energy consumption and weather of the previous hour was very important in predicting what would happen in the next forecasting period. (this model was actually used for determining where they should shift electricity during peak load times). Happy forecasting!

Time Series Residuals Analysis 101 👇🏼 When evaluating a time series forecasting model, residual analysis is one of the most important steps. Residuals—the differences between the actual values and the model’s predictions—help us understand whether the model has captured the underlying patterns or if important structure remains unexplained and what features are missing. In a good model, the residuals should be white noise (no patterns left) and normally distributed (required for model inference and reliable prediction intervals) 🎯 Here’s what each diagnostic plot helps us check: 🔹 Actual vs. Fitted Plot This plot shows how closely the model’s fitted values track the actual observations. It helps you visually spot systematic under- or over-prediction, missed trends, or structural breaks that the model failed to capture. 🔹 Residuals Plot (over time) Plotting residuals across time shows whether they fluctuate randomly around zero. Patterns such as trends, clusters, or seasonal waves indicate that the model has not fully captured the time-dependent structure. 🔹 Residuals ACF (Autocorrelation Function) The ACF plot checks whether residuals are correlated with their own past values. Significant autocorrelation at any lag suggests the model left some temporal structure unmodeled and could be improved. 🔹 Q–Q Plot (Residual Normality Check) The Q–Q plot compares the distribution of residuals to a theoretical normal distribution. Deviations from the diagonal line signal non-normality, which can affect inference and the validity of prediction intervals. 🔹 Residual Density Plot This shows the overall distribution of residuals. A symmetric, bell-shaped curve centered at zero indicates the model errors behave as expected; skewness or heavy tails may highlight model misspecification or outliers. Pro tips: 🔹 Overlay the residual standard deviation on the Actual vs. Fitted plot. I use a range of ±2σ to ±3σ (orange) and bands above ±3σ to immediately spot points where the model’s errors are unusually large, making it easier to diagnose poor fit or outliers. 🔹 Highlight seasonal lags in the residuals ACF. Marking seasonal lag positions (e.g., lag 7, 12, 24, 168—depending on your frequency) in a different color makes it simple to see whether any seasonal structure remains in the residuals, signaling that the model may not have fully captured seasonality. #timeseries #forecasting #datascience

Every retail function should have 3 core metrics they live and breathe. Here are the ones that matter most and how to calculate and influence them. 🧭 C-Suite 1. EBITDA (Earnings Before Interest, Tax, Depreciation & Amortisation) Formula: Revenue – (COGS + Operating Expenses) To influence: Improve gross margin, optimise operating costs, reduce discounting, and increase efficiency in marketing and fulfilment. 2. Cash Conversion Cycle (CCC) Formula: Days Inventory Outstanding + Days Sales Outstanding – Days Payables Outstanding To influence: Shorten inventory holding times, collect receivables faster, and negotiate longer payment terms with suppliers. 3. Cash Runway Formula: Current Cash ÷ Average Monthly Operating Expenses To influence: Build cash reserves, manage spend discipline, and forecast expenses against revenue cycles to extend runway. 💻 Digital / eCommerce Teams 1. New Customer Growth Formula: (New Customers This Period – New Customers Last Period) ÷ Last Period × 100 To influence: Test new acquisition channels, refine paid media targeting, and optimise onboarding flows. 2. Returning Revenue Growth per Customer Formula: (Revenue from Returning Customers ÷ Number of Returning Customers) To influence: Strengthen retention campaigns, loyalty programmes, and personalised offers to lift AOV and frequency. 3. Contribution Profit per Order Formula: (Revenue – Variable Costs) ÷ Number of Orders To influence: Improve product mix, raise AOV through bundling or upsells, and reduce fulfilment or ad costs. 🛍️ Buying / Planning Teams 1. Inventory Turnover Rate Formula: COGS ÷ Average Inventory Value To influence: Buy closer to demand, phase replenishment more frequently, and liquidate slow-moving SKUs early. 2. Days/Weeks on Hand Formula: (Average Inventory ÷ COGS) × 365 (or × 52 for weeks) To influence: Tighten buy quantities, align intake with sell-through velocity, and manage replenishment cadence. 3. Sell-Through Rate Formula: Units Sold ÷ Units Received × 100 To influence: Optimise initial buys, use price elasticity analysis, and manage markdown cadence proactively. The best-performing retailers make these metrics visible, actionable, and owned by the teams that influence them. Curious which metrics your team is prioritising most right now?

A forecast that’s right half the time isn’t a sales execution problem. It’s a system signal — the model producing the number doesn’t know what it’s producing. At $8M–$30M ARR, the instinct is to tighten stage definitions, add scrutiny to late-stage deals, introduce a second opinion on every forecast call. Those moves produce small improvements. They don’t fix the underlying issue. Because 55% accuracy isn’t noise. It’s signal. Three mechanical causes, in order of prevalence: 1. ICP drift. The accounts converting today aren’t the accounts your conversion rates were calibrated against. Your historical close rates are predicting a buyer profile that’s changed underneath you. 2. Stage definitions that describe activity, not commitment. “Proposal sent” tells you what your team did. It doesn’t tell you what the buyer did. Forecasts built on activity stages will always oscillate. 3. Pipeline carrying its own history. 20–40% of most pipelines at $8M–$30M ARR are deals that should have been disqualified two quarters ago. They’re distorting every ratio the forecast depends on. None of these are sales problems. They’re architecture problems. Which is why adding a second sales review doesn’t fix them — and why tightening your CRM workflow makes the symptom worse by hiding the break deeper in the data. What it means for the board conversation: If you’re reporting forecast accuracy under 70%, the defensible board narrative isn’t “we’re working on sales discipline.” It’s “we’ve identified that our forecast model is calibrated against assumptions that need to be re-validated — here’s the work underway.” The first framing sounds like you don’t control the outcome. The second sounds like you understand the system. One question to bring to your next exec team meeting: When was the last time our conversion rates were recalibrated against the accounts we’re actually closing today — not the ones we were closing 18 months ago? If no one has an answer, the forecast isn’t wrong. The model underneath it is. — Forecast Fridays #01

Machine learning beats traditional forecasting methods in multi series forecasting. In one of the latest M forecasting competitions, the aim was to advance what we know about time series forecasting methods and strategies. Competitors had to forecast 40k+ time series representing sales for the largest retail company in the world by revenue: Walmart. These are the main findings: ▶️ Performance of ML Methods: Machine learning (ML) models demonstrate superior accuracy compared to simple statistical methods. Hybrid approaches that combine ML techniques with statistical functionalities often yield effective results. Advanced ML methods, such as LightGBM and deep learning techniques, have shown significant forecasting potential. ▶️ Value of Combining Forecasts: Combining forecasts from various methods enhances accuracy. Even simple, equal-weighted combinations of models can outperform more complex approaches, reaffirming the effectiveness of ensemble strategies. ▶️ Cross-Learning Benefits: Utilizing cross-learning from correlated, hierarchical data improves forecasting accuracy. In short, one model to forecast thousands of time series. This approach allows for more efficient training and reduces computational costs, making it a valuable strategy. ▶️ Differences in Performance: Winning methods often outperform traditional benchmarks significantly. However, many teams may not surpass the performance of simpler methods, indicating that straightforward approaches can still be effective. Impact of External Adjustments: Incorporating external adjustments (ie, data based insight) can enhance forecast accuracy. ▶️ Importance of Cross-Validation Strategies: Effective cross-validation (CV) strategies are crucial for accurately assessing forecasting methods. Many teams fail to select the best forecasts due to inadequate CV methods. Utilizing extensive validation techniques can ensure robustness. ▶️ Role of Exogenous Variables: Including exogenous/explanatory variables significantly improves forecasting accuracy. Additional data such as promotions and price changes can lead to substantial improvements over models that rely solely on historical data. Overall, these findings emphasize the effectiveness of ML methods, the value of combining forecasts, and the importance of incorporating external factors and robust validation strategies in forecasting. If you haven’t already, try using machine learning models to forecast your future challenge 🙂 Read the article 👉 https://buff.ly/3O95gQp

A sales leader told me "Our forecast is always off by 20-30%. I don't know what's real anymore." I looked at his pipeline. Every deal in "proposal stage" had an 80% close probability. I asked him one question: "Has an executive at the buyer's company authorized solving this problem?" He had no idea. Here's the problem: His CRM stages were measuring seller activity. Not buyer commitment. Discovery meant "we had a discovery call." Not "they acknowledged a costly problem." Demo meant "we showed them the product." Not "multiple stakeholders agreed this needs to be solved." Proposal meant "we sent pricing." Not "an executive authorized budget to fix this." So his forecast was always wrong. Because he was tracking the wrong things. Here's what we did: We rebuilt his qualification framework around buyer stages instead of seller activities. The ADVANCED framework: Acknowledged problem Documented issue Validated by team Authorized by executive Narrowed to external Chosen as vendor Established timeline Deal terms finalized These are buyer commitments. Not seller activities. When we ran his pipeline through this framework, reality hit hard. Most of his "80% deals" were actually at 25%. They had acknowledged a problem but nothing was documented. No executive sponsorship. No validation from multiple stakeholders. 𝗪𝗶𝘁𝗵𝗶𝗻 𝗼𝗻𝗲 𝗾𝘂𝗮𝗿𝘁𝗲𝗿, 𝗵𝗶𝘀 𝗳𝗼𝗿𝗲𝗰𝗮𝘀𝘁 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝘄𝗲𝗻𝘁 𝗳𝗿𝗼𝗺 65% 𝘁𝗼 93%. Not because his team started working harder. Because they started tracking what actually predicts if deals close. BTW: When you can forecast within 3%, you can predict your income. You can plan for your family. You can budget for that house or wedding or kids' school. When your forecast is always off by 20%, you're guessing. Your compensation is unpredictable. Your future is uncertain. This isn't just about making your boss happy. This is about controlling your financial future. Track buyer commitment. Not seller activity. That's how you build forecast accuracy. — Sales Leaders! Your sales team doesn’t need more training. it needs a revenue operating system: https://lnkd.in/ghh8VCaf

PepsiCo released its paper for AI based pricing and promotions. I summarized the process, the finding and what you should know.. They have two different systems: - PromoAI: For 16 promo calendar refreshes - PricingAI: To optimize MSRP by geo Both rely on Pepsico connecting the following pricing and promotion layers: Demand prediction. Elasticity estimation. Mathematical optimization. Configurable business constraints. Planner-controlled scenario workflows. PROMO AI: Creates an executable promo calendar - Which product should be promoted? - In which week? - With what mechanic? - At what depth? - Under what retailer constraints - What margin constraints - What trade spend constraints - What timing constraints PepsiCo first forecasts promotional demand using historical sales, pricing, promotion, product, seasonality, and retailer data. That forecast is then optimized using constraints that are fed to the system. For example you can only promote once a quarter, minimum 16 weeks between promos, can't be cheaper than smaller size.. etc etc. Then the system runs an optimization model to search across millions of possible product-promotion-week combinations and recommend a calendar that works commercially. 85% of the recommendations were accepted. PricingAI: This AI system optimizes base prices across the portfolio. A base price move on one SKU can shift volume to another pack size, another brand, a value tier, or a competitor. So PepsiCo estimates both own-price elasticity and cross-price elasticity at various level (size, sku, brand, flavor etc). Then PricingAI uses those elasticity estimates to recommend base price changes while respecting price ladders, margin targets, revenue goals, volume limits, market share, and psychological price thresholds. Additional leverage: It then leverages humans in the loop. Planners can run scenarios, change constraints, override recommendations, and see the financial impact. Now, most planners have some opinion on what each promotion does, and that knowledge combined with AI generated reco is extremely powerful. This is is the kind of decision support system that is available using AI. I have explained everything in more detail in the newsletter. Comment, share and Like if you find it useful. Find me here at #PricingTribe

“Don’t Just Stock It—Make It Count: The MBQ , Availability and Fill Rate Advantage.” In retail, the key to success lies not just in stocking products, but ensuring that the products are adequately stocked is critical for driving sales. Understanding critical inventory metrics—Minimum Base Quantity (MBQ), Availability, Fill Rate—is essential to optimizing sales and meeting customer expectations. Further to truly make an impact, these metrics must be applied at the assortment level. 🔹Minimum Base Quantity (MBQ): Units required on the shelf to maintain product visibility and meet customer demand. It’s not enough for a product to be “available”—it needs to be sufficiently stocked to catch the customer’s eye and drive purchases. It’s a critical measure, its true value is realized when applied to specific assortments that cater to different customer preferences. Ensuring each assortment meets its MBQ helps guarantee that the diverse needs and choices of customers are adequately addressed. 🔹Availability: Is more than just having a product on the shelf—it’s about meeting a set percentage of the MBQ (e.g., 75%) to ensure the product is impactful. If the stock falls below this threshold, the product might as well be considered “unavailable,” as its impact on sales diminishes sharply. Customers expect a range of options within an assortment, and if one part of the assortment is understocked, it can lead to a perception of unavailability even if other products are present. Focusing on availability at the assortment level ensures that all customer needs are met, not just those for the most popular items. 🔹Fill Rate: This measures how well we are meeting customer demand from your current stock, Overall fill rates can be misleading if they don’t reflect the availability of specific products within an assortment that address different customer needs and preferences. By monitoring and optimizing fill rates within each assortment, you can ensure that every customer finds what they’re looking for. 🔸Loss of Sale: When availability falls below the MBQ threshold at the assortment level, you’re not just risking a poor shelf presence—you’re also risking direct financial loss. The loss of sale is calculated by comparing potential sales (based on historical average daily sales) with actual sales on days when availability was low. This analysis can help in minimizing missed revenue opportunities. 🔸Backend Operations: A robust backend—covering vendor fill rates, warehouse stocking, and supply chain speed—is essential for sustaining high MBQ fill, availability, and fill rates at the front end, directly impacting your ability to meet demand and minimize loss of sales. 🔸Benchmarks: Varies between a fashion rand a grocery retailer due to differences in product types, demand patterns, purchase frequency and customer expectations - 1. Fashion Retailer • Availability: 85-90% • Fill Rate: 90-95% 2. Grocery Retailer • Availability: 95-99% • Fill Rate: 98-99%

Forecasts are worthless if they don’t drive action. This document shows how to turn forecast errors into insights: # 1 - Compare Forecast vs Actual Pattern, Not Just Values Look for trend breaks: promotions, seasonality shifts, competitive actions Insight: shows whether the model or the business behavior changed # 2 - Separate Volume Error from Mix Error Your total forecast may be right but SKU mix is wrong Insight: points to cannibalization, launches, or customer preference shifts # 3 - Slice the MAPE (forecast error) MAPE at total level hides the real problem; slice by SKU, region, channel, and planner Insight: find where the system is breaking, not the average # 4 - Track Bias Consistently MAPE shows how much you miss; bias shows how you think Insight: positive bias = optimism; negative bias = fear of stockouts # 5 - Connect Error Spikes to Events Overlay error trend with business events; launches, stockouts, price changes and map everything Insight: turns disconnected numbers into cause-and-effect stories # 6 - Use FVA (forecast value added) to Check If Adjustments Helped or Hurt Measure whether human overrides improved or worsened accuracy Insight: helps remove emotional adjustments from the process # 7 - Build an Error Heatmap One view showing where the biggest misses are by SKU, month, region Insight: quickly identifies where planning attention is needed # 8 - Weekly Error Deep Dive Pick the top 5 SKUs with the biggest misses; ask: “what changed?” and “who owns the correction?” Insight: makes forecasting a feedback loop, not a ritual Any others to add?

Demand Planning Mistake: Treating Last Year as the Future!! One common forecasting mistake is assuming last year will repeat itself. It rarely does. Last year’s sales data is useful, but it only tells us: • What happened? • Not always why it happened? • Not always whether the same pattern will happen again? A planner may say: “Last Season we sold 15,000 units, so this year we should plan the same.” But this year may be different. 🔸 Prices changed 🔸 Competitors launched promotions 🔸 Customers shifted behavior 🔸 Supplier lead times changed 🔸 Some SKUs had availability issues 🔸 Sales focus moved to another category So the forecast may look logical, but still fail. The problem is not using historical data. The problem is using historical data without challenging the assumptions. 📊 A Simple Example Last year’s demand: 15,000 units This year’s actual demand: 10,500 units Forecast gap: 4,500 units Why did the gap happen? ⚠️ Competitor promotion ⚠️ Price increase ⚠️ Supplier delays ⚠️ Customer shift to cheaper alternatives ⚠️ Different sales priorities The forecast missed because the business copied the past instead of reading the present. ✅ Better Approach Use history as a baseline, not the final answer. Before finalizing the forecast, ask: • What changed in the market? • Were last year’s sales affected by promotions or stockouts? • Are customers behaving differently? • Are competitors influencing demand? • Do sales, supply, procurement, and finance agree on the assumptions? Good demand planning is not about copying last year. It is about: 🔹 Understanding the past 🔹 Reading current signals 🔹 Preparing for what may happen next The best planners do not ask only: “What did we sell last year?” They ask: “What changed since then?” #DemandPlanning #Forecasting #SupplyChainPlanning #SOP #SupplyChain #InventoryManagement #LeanSixSigma #BusinessPlanning #SupplyChainAnalytics