Using Data to Improve Student Outcomes

You've just launched a reskilling program aimed at boosting digital literacy across your organization. Now, the big question is: how do you measure its success? To answer that, a combination of hard data and real-world feedback is key. Take the example of AT&T, which famously invested $1 billion in reskilling its workforce for the digital age. They tracked success through KPIs like training completion rates and skill acquisition. Post-training, they saw a marked increase in employees' ability to handle new technologies, evidenced by improved performance metrics. But metrics only tell part of the story. Gathering qualitative feedback is equally important. IBM, for instance, uses surveys and pulse checks to gauge how employees feel about their upskilling efforts. This feedback allows them to tweak programs in real-time, ensuring that learning remains relevant and engaging. Lastly, consider long-term evaluation. Adobe ties reskilling outcomes to annual performance reviews, allowing them to see if the new skills are leading to sustained improvements. This holistic approach—combining KPIs, feedback, and long-term tracking—ensures that reskilling initiatives not only deliver immediate results but also contribute to lasting change. Are you ready to measure the true impact of your reskilling efforts? #hr #chro #reskilling #datainsights #employeedevelopment #employeeskilling

Your project started without a baseline? Welcome to 90% of real-world Monitoring and Evaluation. Most programmes launch with urgency, political pressure, or donor timelines, not perfect data systems. That doesn’t mean you can’t measure change. It just means you need to reconstruct the “before” using the tools seasoned evaluators rely on: 🔹 Start with what already exists Intake forms, early reports, planning documents, grant proposals, even if they weren’t created for MEL, they often contain reference points you can extract. 🔹 Use recall methods strategically Ask participants and staff to describe conditions before the intervention, but anchor their memory to major events: ↳ “Before the school opened…” ↳“Before the water point was installed…” This reduces bias and increases accuracy. 🔹 Pull secondary data to fill the gaps Census tables, ministry surveys, NGO assessments, anything close in geography and timeframe can provide a credible reference. 🔹 Triangulate relentlessly Never rely on one source. Cross-check community recall with government data, staff insights, and documentation. Retrospective baselines aren’t shortcuts. They’re structured, defensible methods for rebuilding the past and they’re what experienced evaluators use when perfection isn’t possible (which is most of the time). 🔥 If you want more practical MEL techniques like this with no jargon, no theory-only talk, join my mailing list for weekly insights that will sharpen your practice. #Baseline

I wish we talked more about ROI in the nonprofit world. Funding is limited, so we have to ensure it creates the greatest possible impact. Right? But I still often see vague metrics like: We “reached” or “served” X number of people. That’s an output, not an outcome. It’s fine to measure outputs: number of trainings, meetings, distributions, etc. But it’s more important to measure the outCOMES: How did people’s lives measurably improve as a result of this work? I get that it can be hard to measure outcomes, especially in areas that don’t lend themselves to quantitative impact (governance, capacity building, etc.) But if the goal is to improve people’s economic power, then that’s what we should measure. And we should do it in a way that: ➡️ 1) Shows the ROI based on program cost ➡️ 2) Calculates attribution–i.e., how much revenue change is attributable to your organization’s work, vs. external market factors? ➡️ 3) Doesn’t assume unrealistic lasting impact. (For instance, TechnoServe assumes continued revenue improvement as a result of our work for THREE years afterward--no more. We often see indications that the impact lasts longer. But until we have more data from post-project evaluations--too rarely funded--we use a conservative, low-end estimate.) 🔸🔸🔸 So here's a quick breakdown of TechnoServe's 2025 ROI: ♦️ This past year, the people TechnoServe worked with gained an average $5.70 in additional revenue for every $1 we spent working with them 👇 ♦️ The ROI of all our closed projects last year ranged from over 20-to-1 to less than 1. ♦️ For very low-ROI projects: Some were simply failures. Others were pilots that we hope will attain a positive ROI as they scale. ♦️ For very high-ROI projects: Some may not reflect true cost-effectiveness--e.g., they might have had unusually low starting points due to COVID. Others may be truly impactful projects, where we’ll seek to replicate successful elements where we can. But we’ll be looking at the WHY behind all these scores to see what we can improve or scale. 👉 I think the development world owes it to the people fighting poverty worldwide to find what works and fix what doesn't. And that starts with working as hard as possible to measure things right.

Monitoring and Evaluation (M&E) systems form the backbone of program accountability, learning, and improvement. This document, Developing a Monitoring and Evaluation Plan, offers a step-by-step guide for creating robust and responsive M&E frameworks tailored to the complexities of humanitarian and development programs. It emphasizes the importance of aligning indicators, data collection methods, and reporting processes with program goals to ensure reliable and actionable insights. The content covers critical components of an effective M&E plan, including defining SMART indicators, setting baselines and targets, and establishing data acquisition and reporting methods. Humanitarian professionals will benefit from its practical focus on data quality, emphasizing validity, reliability, and timeliness as essential criteria for ensuring the credibility of findings. Additionally, the guide explores various data collection techniques, from surveys to focus group discussions, offering strategies to select the most appropriate methods for different contexts. This document serves as a comprehensive resource for M&E practitioners committed to optimizing program performance. By mastering the tools and principles presented, professionals can design M&E systems that drive evidence-based decision-making, enhance program accountability, and foster meaningful impact in humanitarian interventions.

How to Quantify Your Impact Like a Professional A lot of people are doing incredible work but struggle to communicate it in a way that is measurable, credible, and compelling. If you want to stand out in applications, interviews, or leadership opportunities, you must learn to translate your work into data. Here is a simple guide to help you start. 1.    Weak vs. Strong Impact Statements Weak: “I worked on a literacy program that reached some students in different communities. We partnered with a few groups and distributed some books. Reading time improved.” Strong: “In 12 months, I expanded a literacy program to reach 1,450 students across 22 communities, mobilized 102 volunteers, built 12 partnerships, delivered 4,200 books, and increased reading time by 68%.” This is the standard you should aim for. Numbers matter. 2. What You Should Measure Start tracking clear, countable indicators like: • People reached • Workshops/events held • Partnerships built • Funds raised • Resources distributed • Social media reach • Volunteer hours • Pre/post assessment results These are your core impact metrics. 3.    Turn Activities Into Data Vague: “Trained many students.” Specific: “Trained 350 students across 12 schools in Jos, Nigeria, in 3 months.” Specificity turns your work into evidence. 4.     Use Before & After Metrics Show growth by comparing where you started vs. where you are now: • Volunteers: 15 → 52 • Partnerships: 0 → 7 • Retention rates: +47% Before-and-after numbers make progress visible and undeniable. 5.    Use Percentages + Timeframes Percentages show the scale of change: • Engagement increased 47% • Comprehension improved 40%+ • Attendance rose 50%+ Timeframes add clarity: • in 6 months • in one academic year • Jan–Sept 2023 Always anchor numbers in time. 6.     Visualize Your Impact When writing reports or applications, use visuals such as: 📊 Bar charts 📈 Growth lines 🔢 Big bold numbers 📍 Maps of communities served Visuals help your data tell a memorable story. 7.     Build a “Metrics Bank” Keep one simple file that tracks: • Outreach numbers • Volunteers mobilized • Partnerships • Media features • Annual summaries Then reuse these metrics in applications: • “Reached 6,200+ students.” • “Mobilized 52 volunteers contributing 5,000 hours.” • “Raised ₦4.2M in community donations.” Your work matters. Your numbers prove it.

Day 2: What I’d Do as an Analyst – Measuring Amazon’s Loyalty Program Success Hi Everyone! Welcome to Day 2 of my 7-day series, “What I’d Do as an Analyst.” Today, I’m tackling a scenario where Amazon launches a new loyalty program for frequent shoppers. The Scenario Amazon wants to reward frequent shoppers with a loyalty program. The question is: What KPIs would I track to measure success, and how would I evaluate its impact on revenue? Step 1: Understanding the Problem Loyalty programs aim to drive retention and revenue. Key questions include: - Are shoppers buying more often or spending more? - What is the short-term vs. long-term value of the program? Step 2: Key KPIs to Track To measure the success of the loyalty program, I’d focus on: 1️⃣ Customer Retention Metrics - Repeat Purchase Rate: Are loyalty program members shopping more frequently? - Churn Rate: Has the program reduced the percentage of customers leaving Amazon? 2️⃣ Engagement Metrics - Enrollment Rate: How many eligible customers are signing up for the program? - Program Engagement: Are members actively redeeming rewards or benefits? 3️⃣ Revenue Impact - Average Order Value (AOV): Are loyalty members spending more per transaction? - Incremental Revenue: How much additional revenue is directly tied to loyalty members? 4️⃣ Customer Lifetime Value (CLV) - Are loyalty members showing a higher CLV compared to non-members over time? 5️⃣ Program Costs - Are the costs of running the program (e.g., discounts, rewards) sustainable relative to the revenue it generates? Step 3: The Solution Approach Here’s how I’d evaluate the program’s impact: 1️⃣ Segment and Compare - Create separate customer segments (e.g., loyalty members vs. non-members) and compare key metrics like AOV, repeat purchase rate, and CLV. - Use cohort analysis to track how customer behavior changes over time. 2️⃣ Track Behavior Changes - Monitor if loyalty members are increasing their purchase frequency, spending more, or trying new product categories. - Analyze redemption behavior—are members redeeming rewards in ways that drive repeat purchases? 3️⃣ Run Controlled Experiments - Implement A/B testing by offering the program to a test group and comparing their behavior to a control group. - Evaluate the program’s incremental impact on revenue while controlling for external factors like seasonality. 4️⃣ Evaluate Long-Term Sustainability - Use predictive modeling to estimate the program’s long-term impact on revenue, factoring in retention improvements and increased CLV. - Monitor program costs to ensure a healthy ROI. Step 4: Expected Outcome - Retain more customers and increase their lifetime value. - Drive higher revenue through increased purchase frequency and basket sizes. - Ensure the loyalty program remains profitable and scalable over time. What KPIs would you prioritize to measure success? Share your thoughts below! 👇 #DataAnalytics #KPIs #BusinessGrowth #EcommerceInsights

𝗠𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴 𝘁𝗵𝗲 𝗜𝗺𝗽𝗮𝗰𝘁 𝗼𝗳 𝗬𝗼𝘂𝗿 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗣𝗿𝗼𝗴𝗿𝗮𝗺 📚 Creating a training program is just the beginning—measuring its effectiveness is what drives real business value. Whether you’re training employees, customers, or partners, tracking key performance indicators (KPIs) ensures your efforts deliver tangible results. Here’s how to evaluate and improve your training initiatives: 1️⃣ Define Clear Training Goals 🎯 Before measuring, ask: ✅ What is the expected outcome? (Increased productivity, higher retention, reduced support tickets?) ✅ How does training align with business objectives? ✅ Who are you training, and what impact should it have on them? 2️⃣ Track Key Training Metrics 📈 ✔️ Employee Performance Improvements Are employees applying new skills? Has productivity or accuracy increased? Compare pre- and post-training performance reviews. ✔️ Customer Satisfaction & Engagement Are customers using your product more effectively? Measure support ticket volume—a drop indicates better self-sufficiency. Use Net Promoter Score (NPS) and Customer Satisfaction Score (CSAT) to gauge satisfaction. ✔️ Training Completion & Engagement Rates Track how many learners start and finish courses. Identify drop-off points to refine content. Analyze engagement with interactive elements (quizzes, discussions). ✔️ Retention & Revenue Impact 💰 Higher engagement often leads to lower churn rates. Measure whether trained customers renew subscriptions or buy additional products. Compare team retention rates before and after implementing training programs. 3️⃣ Use AI & Analytics for Deeper Insights 🤖 ✅ AI-driven learning platforms can track learner behavior and recommend improvements. ✅ Dashboards with real-time analytics help pinpoint what’s working (and what’s not). ✅ Personalized adaptive training keeps learners engaged based on their progress. 4️⃣ Continuously Optimize & Iterate 🔄 Regularly collect feedback through surveys and learner assessments. Conduct A/B testing on different training formats. Update content based on business and industry changes. 🚀 A data-driven approach to training leads to better learning experiences, higher engagement, and stronger business impact. 💡 How do you measure your training program’s success? Let’s discuss! #TrainingAnalytics #AI #BusinessGrowth #LupoAI #LearningandDevelopment #Innovation

8 metrics run your maintenance program, yet most teams only track 3. Typically, the 3 chosen are those that make the monthly report look good: PM compliance, schedule compliance, and cost. However, each metric in isolation can be misleading. For instance, 95% PM compliance seems impressive until you examine MTBF and discover that the same repairs are being performed on the same equipment every 6 weeks. You're completing work orders, but not effectively fixing issues. Low maintenance costs may appear efficient until asset availability drops below 85%, resulting in production losses. This isn’t efficiency; it’s underinvestment dressed up for the budget meeting. Fast MTTR may indicate responsiveness, but it often means that the team is repeatedly fixing the same failures. The most effective maintenance teams I’ve encountered monitor 8 metrics together: - PM compliance - Schedule compliance - Work order backlog - Mean time to repair (MTTR) - Mean time between failures (MTBF) - Asset availability - Maintenance cost as % of RAV - Emergency work percentage These metrics are analyzed as a system rather than a scoreboard. Execution metrics (PM compliance, schedule compliance, backlog) indicate if the team is performing the work. Reliability metrics (MTTR, MTBF, availability) reveal if the work is effective. Financial metrics (cost, emergency %) assess if spending is appropriate or merely reactive. When PM compliance is high but MTBF remains flat, it signals that PMs require revision. When costs are low but emergency work exceeds 15%, it suggests maintenance is being deferred under the guise of savings. If availability appears satisfactory but backlog is increasing, it indicates future borrowing. Every metric has a partner that ensures accountability. Teams that excel in this area don’t possess superior data; they ask better questions. Keep this in mind for your next planning meeting. It’s crucial when someone presents a single KPI as progress. What combination of metrics provides the clearest insight into your program's health? I would genuinely like to hear what you track.