Developing Training for New Technologies

Most L&D professionals learned the Kirkpatrick Model early on. Fewer have seen it applied beyond Level 1. Here's what each level can actually look like when you put it into practice, not just the textbook definition. ✨ Level 1: Reaction 🔹 Textbook version: Did learners find the training engaging and worth their time? ✅ In practice: Instead of "Did you enjoy this session?", ask "Was this relevant to the work you do?" and "Could you apply this right away?" ✅ Metric to track: Relevance and applicability ratings, not just satisfaction scores. ✨ Level 2: Learning 🔹 Textbook version: Did learners gain the intended knowledge or skills? ✅ In practice: Replace recall-based quizzes with scenario-based checks. Can the learner apply the concept to a situation they'd actually face? ✅ Metric to track: Pre/post assessment scores on scenario-based questions, not just "did you pass the quiz." ✨ Level 3: Behavior 🔹 Textbook version: Are learners applying what they learned on the job? ✅ In practice: 30/60/90-day check-ins, manager observations, or peer feedback on whether the new behavior is showing up in real work. ✅ Metric to track: % of participants demonstrating the target behavior, based on manager or peer input, not self-reported confidence. ✨ Level 4: Results 🔹 Textbook version: Did the training impact business outcomes? ✅ In practice: Pick one business metric the program was meant to influence, before you build it, not after, and track the change. ✅ Metric to track: Movement in that specific KPI (error rates, time-to-productivity, conversion rates, retention) compared to a baseline. Most programs are measured thoroughly at Level 1 and barely at all beyond it. But Levels 3 and 4 are where the "did this actually matter" conversation happens, and they are also where L&D earns a seat at the table. Which level does your organisation measure consistently, and which one do you wish you could measure better? #LearningAndDevelopment #LnD #KirkpatrickModel #TrainingEvaluation #InstructionalDesign #LearningMeasurement #TrainingAndDevelopment #LnDStrategy

Kirkpatrick is often criticized. But rarely fully understood. Let's change this 👇 The model is simple. It describes four levels of evaluating learning impact: Level 1 — Reaction How participants experience the learning. Level 2 — Learning What knowledge and skills they acquire. Level 3 — Behavior How their on-the-job behavior changes. Level 4 — Results What organizational outcomes improve. That’s it. Four levels. And yet, it is frequently dismissed as outdated or simplistic. Why? Because we often treat it as a measurement checklist, instead of a design framework. Kirkpatrick is not just about evaluating training. It’s about thinking in cause-and-effect logic. Instead of asking, “Was the training good?” we should be asking a sequence of strategic questions. When designing: – What business outcome must change? – What behavior must shift to deliver that outcome? – What knowledge and skills are required? – What learning experience will enable mastery? And when evaluating: – How did participants evaluate the experience? – How well did they acquire the knowledge and skills? – How did behavior change at work? – What changed in the targeted business indicators? Planning must start from the top (Results). Measurement must begin from the bottom (Reaction). Think forward. Measure backward. Of course, the model has nuances - leading and lagging indicators, performance environment, manager accountability, isolation factors. But beneath the complexity lies a simple and powerful logic. The pyramid is not a hierarchy of surveys. It’s a chain of impact. That’s why I created this visual, to show the model not as theory, but as a practical thinking framework. How do you approach Kirkpatrick in your projects? #designforclarity #LearningAndDevelopment #InstructionalDesign #LearningStrategy #Kirkpatrick #LearningImpact #LXD #CorporateLearning

Most training programs create excitement. Very few create measurable business impact. A few months ago, I worked with an organization that had a very specific challenge. Their frontline teams were attending workshops, feeling motivated, taking notes but when it came to actual performance on the field, their sales conversion was very low. Great energy. Poor execution. Something was missing. So before designing the learning intervention, I asked one simple question: “What’s the real context in which your people operate daily?” Not the role. Not the job description. Not the competencies. The context. What pressures do they face? What conversations are toughest? Where do deals collapse? Who influences decisions? What behaviours matter most on the ground? The organization opened up. We mapped real scenarios. We shadowed calls. We watched interactions. We decoded customer psychology. We understood the reality behind the numbers. Only then did we build the training journey. Not generic content. Not textbook concepts. Not motivational theory. But a program designed exactly around their on-ground realities. The impact. Over the next eight weeks, something changed. Sales conversations became sharper. Objections were handled with more confidence. Teams spoke value, not price. Managers reinforced learning consistently. The conversion saw a huge jump and this was created not by more training, but by the right training. The lesson is simple: Content informs. Context transforms. Workshops don’t create results. Relevance does. When learning mirrors the real world, people don’t just listen they apply. When they apply, organizations grow. What’s one area in your team where you feel content is high but context is missing? If your organization wants training that delivers real, measurable outcomes let’s talk.

As Instructional Designers, we often track training completion in spreadsheets. But rows and columns rarely show us the real shape of a learning culture. So I used Gephi to model a sample organizational training network. 🔵 Blue nodes: Training topics 🟣 Purple nodes: Employees Each connection represents actual participation, not just assignment. When the data turned into a network, the story became much clearer: 🔹 Hidden silos appeared immediately. A group of employees clustered only around Health & Safety, completely disconnected from core digital topics like Data Security. They are compliant — but isolated. 🔹 “Super Learners” stood out naturally. Employees like Emp #7 emerged as bridges between technical and soft skills. These are not just learners — they are potential mentors, knowledge carriers, and internal champions. 🔹 Core vs. Edge became visible. While Data Security sits at the heart of the learning culture, Leadership training appears at the fringe, signaling a possible disconnect between strategic development and daily learning behavior. This reminded me of something important: Instructional Design is not only about creating content. It is about revealing gaps, breaking silos, and intentionally designing connections. Spreadsheets show who completed what. Networks show who is truly connected to learning. How do you currently look at your training data: as a list — or as a living system?

OpenAI just dropped something last week that should be on every L&D professional's radar. They've introduced the Learning Outcomes Measurement Suite (LOMS) — a framework designed to track how AI use affects student learning over time. Not just whether learners like using AI. Not just short-term recall scores. But deeper cognitive outcomes: persistence, motivation, creative problem-solving. It monitors model behaviour, how learners interact with it, and which cognitive outcomes change over time. And here's the line that stopped me: "What really matters is whether the gains and associated productive behaviours remain durable." Yes! Because that's the real question -  not whether learning happened in the moment, but whether it stuck. Limited studies show AI tutoring offers short-term recall gains, but there's little insight into lasting effects. We're seeing early signals that it can go deeper. A learner working with an AIReady™ AI coach at a healthcare client told us: "I really enjoyed the cases because they provided a realistic setting in which to apply the material being presented." Realistic application is where transfer begins. And we're starting to see it in the numbers too. One Higher-Ed client has seen desired behaviours nearly double after AI-enabled training implementation — measured through concrete actions, not just self-reported satisfaction. Anecdotal? Yes. But worth paying attention to. OpenAI's framework is a step toward metrics on learning with AI. But until the long-term data is in, we — the designers, the facilitators, the people who actually build learning experiences — are the ones responsible for holding that standard.That is why I am a big fan of learning teams building AI interactions themselves.  What are you doing to measure real outcomes in your AI-integrated programs? Image: An annotated version of OperAI’s LOMS framework. (I will write a detailed blog on this soon.) #LearningAndDevelopment #AIinEducation #InstructionalDesign #AIReady #AIAccelerator #eLearning

A new study examining the long-run effects of the One Laptop per Child program in rural #Peru provides important evidence for digital education policy. The program substantially increased students’ computer skills over time, but it did not translate into measurable improvements in academic achievement, school completion, or later educational transitions. The results underline a central consideration for digital transformation agendas: access to devices on its own is unlikely to generate sustained learning gains. Effective integration of technology requires complementary investments in teacher training, pedagogical support, and curriculum alignment. As countries expand digital initiatives, it is important to consider how technology can be leveraged more strategically to strengthen teaching and learning. Full study here: https://lnkd.in/eV-_73z7 National Bureau of Economic Research Santiago Cueto Caballero Diether Beuermann Julian Cristia Ofer Malamud  Francisco Pardo Pajuelo Sonia Suarez Enciso Gabriela Gambi Elena Arias Ortiz

There’s a reason training impact feels so hard to measure. It’s not because impact isn’t there. It’s because we look for it at the wrong time. Training impact doesn’t show up all at once. It unfolds in stages. Right after training, you won’t see behavior change yet. But you can see early signals: Do people understand it? Do they feel confident applying it? Do they see why it matters? These signals don’t prove impact. But they predict whether it’s even possible. A few weeks later, different things become visible: Early application Intent to use Where people get stuck This is where learning starts to show up at work. Months later, real change follows: Behavior shifts Adoption increases New habits form And only much later does it make sense to ask: Did this improve performance? Did it move the business? Was there ROI? Most training is evaluated far too early to see business impact. Good evaluation is about measuring the right things at the right time.

Is just conducting training enough? or should we measure ROI? I’ve delivered 100s of corporate workshops & I can tell you... = Training without measurement is a shot in the dark. You wouldn’t invest in marketing without tracking leads. You wouldn’t launch a new product without monitoring sales. So why invest in training without checking its impact? Real ROI in training = Lasting behavioral change + tangible business results. <> Did communication improve after the session? <> Are teams collaborating better? <> Has productivity actually gone up? <> Did leadership skills translate to lower attrition rates and higher motivation? If you don’t answer these, you won’t know: <> Where your training dollars went <> What genuinely moved the needle <> Where further investment is needed How do you measure ROI? 1/ Pre & post assessments 2/ Feedback forms tied to actual performance, not just “did you enjoy the session?” 3/ 30-60-90 day follow-ups with real data 4/ Tracking specific KPIs (attrition, productivity, engagement metrics) Just conducting training is like planting seeds and never checking if they grew. Strong organizations don’t just train... they track, tweak & transform. Is your training driving change or just ticking a calendar box? Tag an HR or L&D leader who cares about impactful outcomes, not just agendas. Let’s build cultures where learning leads to real, measurable growth.

Hello #Leaders 👋🏽 Especially my fellow #HR and #L&D professionals. Lately, we've been hearing a lot of messages like: 👉 "Don't focus on measuring attendance." 👉 "Don't focus on reporting training hours." Instead... "Measure #impact." While I agree with the intention, I think we're missing part of the picture. We should #measure all of it. The difference is understanding what each measure tells us. 📊 Outputs – What did we do? These are your activity measures. ✔ Number of participants ✔ #Training hours ✔ Completion rates ✔ Courses delivered ✔ Attendance They tell us whether learning happened which is critical for logistical planning. 📈 Outcomes – What changed? Now we're asking different questions. ✔ Did #knowledge improve? ✔ Did #behaviourschange? ✔ Are #managers leading differently? ✔ Are #teams applying what they learned? ✔ Are employees working differently? Now we're beginning to understand whether #learning has transferred into the #workplace. 🚀 Impact – Why did it matter? This is where the #business starts paying attention. Did learning contribute to: 📈 Higher productivity? 😊 Better #employeeengagement? 💰 Reduced costs? 🎯 Improved quality? 📉 Fewer customer complaints? ⚡ Faster project delivery? This is where L&D earns a #strategic seat at the table. Think of it this way. 📊 Outputs tell us what we delivered. 📈 Outcomes tell us what changed. 🚀 Impact tells us why it mattered to the organization. And here's the reality. When L&D cannot demonstrate value... #Learning is often seen as a cost rather than an #investment. ▫️Budgets become harder to defend. ▫️Training becomes one of the first things leaders question. ▫️And L&D risks becoming an administrative function instead of a #strategicbusinesspartner. Simply because we haven't shown value. This is one of the conversations we spent unpacking in our recently concluded L&D #MentorshipCohort. Helping L&D practitioners understand that our role goes far beyond coordinating #training. We don't just discuss theory. Together, we work through real #workplace challenges, from conducting #learningneeds analyses, designing interventions, measuring learning, writing strategic reports, and positioning L&D as a true business partner. The goal is simple: That you leave with practical tools, frameworks and documents you can immediately use in your #organization. If you've been looking for a mentorship programme that covers the complete L&D journey from strategy to implementation to measurement. I would love to have you in our next cohort. Comment or DM and I will send you details on upcoming dates. But before submitting your next L&D report, ask yourself: 💭 Have I only reported activity? 💭 Have I shown what changed? 💭 Have I demonstrated why it mattered? Because leaders don't just want to know what you delivered. They want to know what your learning intervention achieved. #PeopleandCulture #LeadershipDevelopment #TalentManagement

𝗘𝘅𝗽𝗲𝗰𝘁𝗮𝘁𝗶𝗼𝗻: "I'll harness data and AI for smarter decisions" 🌟 𝗥𝗲𝗮𝗹𝗶𝘁𝘆: "My data is a mess" "Data collection is such a manual chore" 😣   Drawing from my experience as a Machine Learning Engineer in Cambridge and management consultant at McKinsey helping CEOs drive strategic insights, I've seen the power of data.   When I founded GuruLab, I was determined to integrate data analytics into our core operations and make that our competitive advantage.   In the next 3 posts, I will be sharing a few examples of how our data initiatives drive outcomes and the invaluable learnings we gained. --- 🔍 𝗛𝗼𝘄 𝗚𝘂𝗿𝘂𝗟𝗮𝗯 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗲𝗱 𝗣𝗿𝗼𝗴𝗿𝗲𝘀𝘀 𝗥𝗲𝗽𝗼𝗿𝘁𝗶𝗻𝗴 𝗯𝘆 𝟭𝟬𝗫 Traditionally, parents awaited report cards for an update on their child's performance, often just once every six months.   We made continuous feedback without overwhelming our educators possible by building a streamlined data workflow. This allowed us to send progress updates to >500 students easily, boasting an impressive 80% open rate with each report being revisited ~7 times!   Here's how we did it: 1️⃣ Automate Backend Metrics The one-off investment in tech setup enabled us to collect clean, accurate, and continuous data without manual work moving forward. We used existing SaaS tools like PostHog and built bespoke trackers. For example, attendance is tracked by join and leave class clicks, engagement by patterns of reward points earned. Soon, we can also monitor if students remain focused in our live class. No more attendance marking or noting down which students struggled, improved, or enjoyed the lessons.   2️⃣ Integrate Data Collection into Existing Workflows Let's be real, if your users don't currently do an action, they are not gonna do it for you to collect data.    When we got tutors to write student feedback, they were slow and reluctant with the additional chores.   Instead, we built a seamless process to extract scores from classwork and translate a wall of numbers into parent-facing comments that tutors could easily verify.    With generative #ai on the rise, we are also exploring opportunities in this area 🚀   3️⃣ Collaborative Approach between Tech and Student Success Purely tech-driven data systems risk losing touch with the actual user needs. On the flip side, non-tech teams might not envision the full technological potential (you don't know what you don't know 😉)   The report you see below has data automatically pulled and populated - and parents resonated with the content. We achieved this by engaging our developers. We learned how to communicate context clearly and to come prepared with a manual workflow, which forces the non-tech team to think through each step of what they need.   --- How does your organisation automate data collection? Would love to exchange notes ☕ Stay tuned next week on how we analyse the data at GuruLab! Make sure to hit that 🔔 on my profile to get notified.