Your videos have been so helpful! I have an ID interview tomorrow! I'm a Learning and Development specialist but haven't focused on the ID side of things. Thank you so much for helping me to prepare for the interview in understanding how I can add to my learning.
This is one area where I think a lot of educators would have some experience, especially if there were in any area of curriculum develop or accreditation studies, but it could be helpful to have some more real world examples. It seems to be hard to find those in the courses I've done on LinkedIn and such. Thanks again for the refresher!
Thanks for the vid. Briefly,... Level 3 & 4 are needed for just in time training, however they don't immediately apply to capacity building training, which rely mostly on level 1 & 2 evals.
Thanks for the summary. Useful content and I do agree that many instructional designers avoid Levels 3 and 4, but this is where true value lies. It's not as complicated as it may seem. We should be included in planning and done right you're able to measure training impact - leading to improved learning interventions. It's also important to periodically measure your evaluation methods so that you continually improve them.
00:00 Kirkpatrick model for evaluation 1:13 Level 1 : Reaction Overall satisfactions and engagement Survey, interviews, focus group 2:05 Level 2 :Learning New Knowledge, skills or attitude Quizzes / tests, interviews, focus groups, discussions, observation, and more. 3:09 Level 3 :Impact Are people performing the critical tasks? Surveys, observation, work review, KPls 4:30 Level 4 :Results Which results do we aim to achieve? What do people need to do differently? What knowledge and skills do they need? How do we design an attractive intervention? 8:33 Conclusion
Thanks @Devlin for knowledge in only 9 minutes You have been able to summarize a model perfectly. Thanks! I continue having a little doubt. How I could show that level 3 and 4 It is a consequence of a training? when It has runned time for example 6 months or 1 year . Thanks in advance!
Good question, Cynthia! I would suggest looking into the success case method for a program that has already been running for a while. This model works best when baked into the strategy and design from the beginning 😃. You could also look at performance and metrics both before the training was implemented and after it was implemented.
@@DevlinPeckYT thank for your answering.. Before test and post test...I am exploring it more. Thanks for your information and congrats for great conversation with Cathy Moore. I am reading this remarkable book... I follow you!! greetings from Peru
This content is very timely as this is our topic in my Masters Degree in Human Resource Management. Thank you very much for sharing your personal thoughts on the planning perspective. Will subscribe and hoping to learn more from you. Keep rocking!
This old model lacks human capital factors, for instance, level four should also include employee retention (as an example: The customer received high level service and needs are met for the employee - Maslow hierarchy of needs)
Hi...great explanation and so helpful.can you tell ACTIVITIES for this each 4 level that helps to language learning trainees or teacher? Thanks in advance.
@@sindhuvelmurgan361 Level 1 pertains to how the audience reacted to the learning experience, level 2 pertains to how much they learned, level 3 pertains to whether or not they're using what they learned in a real-world context, and level 4 pertains to the impact that their newfound langauge skills are having on the organization that they're working for :D
Thank u for the sharing Devlin. I'm wondering any other factors influencing training effectiveness score..for example virtual learning vs physical learning, virtual 2 hours vs full 8 hours..do u have any view? I've been asked to this question a lot by the leaders..
Hello hello! Thanks for checking out the video. Those items would be great things to ask your intended audience (and collect data about how they’re received, how they impact learning, whether people use those new skills on the job at the same rate, etc).
@@DevlinPeckYT thank u for the idea..would be a good start to do survey..the learning platform have just swift from physical to virtual since COVID pandemic. thanks again yaa!
@@DevlinPeckYT How about content on the AECT standards? My portfolio for my MS in LDT will be judged against it this semester for graduation but NO employer ever mentions it in the job listings which mean they do not know it or its skill over credential or both.
Hi Devlin! Is the designing part in the planning process (the 4th point: designing attractive intervention) related to the reaction level (i.e. Level 1)? If yes, then how?
Level 3 actually requires additional time from SME persons. They should later send the results of whether something has changed in terms of user behavior. In most situations they are very busy and this is rarely delivered unfortunately. In most cases, you will not get these results because of the organization you are in. This also applies to the entire level.
I am moving to the USA. I have experience as trainer and university professor. Do you recommend to offer to design a course to a local business, in my home country, for free in order to build my portfolio?
That could work, Fabian! But you could also create a concept project so that you do not need to rely on unavailable SMEs, unreasonable requests from clients, etc.
Great videos. It's beyond me as an educator, why we can't graduate better knowledgeable, skilled, and intelligent students with all the tools we have in the United States. This then tells me that it has to be a discipline issue.
Devlin -- I've worked on projects where there's an interest in knowing an ROI to justify the cost of say, narration or video vs just Powerpoint, but there's NO interest in looking at level 2 or 3. Maybe it's compliance or other training where the ROI is a little squishier. Management is only interested in justifying the cost and cares less about the long-term results. How would Kirkpatrick address this? Perhaps that kind of ROI is better evaluated with just infographics on the value of eLearning? Thx.
I think that Kirkpatrick would say that those efforts are unlikely to produce any real results. If the stakeholders are not interested in improving business metrics or actually changing human behavior, then there are few cases where the efforts will make a difference for the audience or the organizaton.
@@DevlinPeckYT Thanks for the reply. I think that for compliance and other BAU training there’s no interest in metrics.. only in compliance. No behavior change if the employees are already in compliance, so it comes down to evaluating the return on the training investment every year to keep the status quo.
Hi Travis! I appreciate the support. I referenced this book when I was making this vid to make sure I didn't miss / misrepresent anything: www.kirkpatrickpartners.com/product/Kirkpatricks-Four-Levels-of-Training-Evaluation/
I love how this guy seems annoyed by his own presentation. I could hear him sigh...... hahahaha dude, we're not forcing you to do this, you want to present this information to us 😅🤣🤣🤣🤣🤣
👀👀👀. I must have caught myself on a bad day with this one haha. Definitely happy to present the info but it can be rough some times…especially in these earlier days when I was newer to RUclips 😂
I liked how you talked about working backwards from Level 4 to Level 1. It made so much more sense.
Me too, working backwards makes so much more sense as I'm working on Behavioral competency e-learning project
You explain in less than 10 mins what others take hours to. Thank you.
You’re welcome! Thanks for checking out the video 😃
Thank so much! I work in professional development and this 10min video has given me so many ideas as to how we can improve our needs analysis!!
Wonderful! I’m glad to hear it 😃
wow tthis was so clear and precise it helped me form my training and development poa
Glad to hear it! Thanks Shubham 😃
Kirkpatrick's Model of Evaluation,..
Checked!! Thank you.. off to the next video...
Keep up the good work, Jackie :D
Your videos have been so helpful! I have an ID interview tomorrow! I'm a Learning and Development specialist but haven't focused on the ID side of things. Thank you so much for helping me to prepare for the interview in understanding how I can add to my learning.
You’re welcome! Good luck and please let us know how it goes 😃
This is one area where I think a lot of educators would have some experience, especially if there were in any area of curriculum develop or accreditation studies, but it could be helpful to have some more real world examples. It seems to be hard to find those in the courses I've done on LinkedIn and such. Thanks again for the refresher!
You're welcome! Thanks for the insight, Jessie. Glad that you can relate to evaluation as well.
Excellent information and explained clearly. Thanks.
Glad it was helpful!
Great refresher, and yes many times we stop at level 2 but should try to get to levels 3 or 4.
For sure. Thanks for checking out the video :)
Thanks for the vid.
Briefly,... Level 3 & 4 are needed for just in time training, however they don't immediately apply to capacity building training, which rely mostly on level 1 & 2 evals.
You’re welcome! Thanks for commenting 😃
Thanks for the summary. Useful content and I do agree that many instructional designers avoid Levels 3 and 4, but this is where true value lies. It's not as complicated as it may seem. We should be included in planning and done right you're able to measure training impact - leading to improved learning interventions. It's also important to periodically measure your evaluation methods so that you continually improve them.
This was very crisp and informative. Also, thank you for sharing the amazing list of books :)
You’re welcome! Thanks for watching, Pooja 😊
Great content, thanks! I'm a fresh Instructional Designer and your videos are super useful for quick overviews!
That’s wonderful! Glad to hear it 😃. If there’s other content that you think would be helpful (but don’t see here on the channel), please let me know!
Hi Devlin, I have done an ID course but this provides greater clarity to the theory.
I’m glad that it helped!
So true, easily the best evaluation model extant
So says Randall Kirkpatrick
It’s a good one! 😃
00:00 Kirkpatrick model for evaluation
1:13 Level 1 : Reaction
Overall satisfactions and engagement
Survey, interviews, focus group
2:05 Level 2 :Learning
New Knowledge, skills or attitude
Quizzes / tests, interviews, focus groups, discussions, observation, and more.
3:09 Level 3 :Impact
Are people performing the critical tasks?
Surveys, observation, work review, KPls
4:30 Level 4 :Results
Which results do we aim to achieve?
What do people need to do differently?
What knowledge and skills do they need?
How do we design an attractive intervention?
8:33 Conclusion
Thanks for sharing this, Andres!
This video helped me understand evaluation and the Kirkpatrick model so much.
Thank you so much!! I just realized that I evaluated only the level one!
That’s very common! The higher up the levels that you get with the evaluation, the more valuable that the data will be 😃
Thank you, Devlin. Awesome.
You’re welcome! Thanks for watching 😃
This was SO helpful. I was trying to find the best way to summarize evaluations for my students and this video is perfect. Thank you!!
You're welcome! Glad to hear that it helped :)
Thank you for a wonderful presentation. Very well explained. 👍
You're very welcome! Thank you for checking it out :)
You have given the essence of the model. Great video! Looking forward to a video on SAM model for both simple and complex projects
Thanks so much, Lakshmi! Glad to hear that you’re interested in a SAM video.
Next is Philips ROI - Level 5. Information that truly demonstrates value to an organization and what executives will want to hear.
Super clear and crisp . Thank you :-)
You’re welcome! Thanks for watching and commenting 😃
Explained so well. Thank you so much.
Thanks so much for this summary!
You're welcome! Thanks for watching :)
Hope you can add information on criticisms and limitations of the model
Thanks for the suggestion, Patricia!
Thanks.
You’re welcome!
Learning just got easier
Glad that you found your way here!
Great Summary!
Thanks!
Thanks @Devlin for knowledge in only 9 minutes You have been able to summarize a model perfectly. Thanks! I continue having a little doubt.
How I could show that level 3 and 4 It is a consequence of a training? when It has runned time for example 6 months or 1 year . Thanks in advance!
Good question, Cynthia! I would suggest looking into the success case method for a program that has already been running for a while. This model works best when baked into the strategy and design from the beginning 😃. You could also look at performance and metrics both before the training was implemented and after it was implemented.
@@DevlinPeckYT thank for your answering.. Before test and post test...I am exploring it more. Thanks for your information and congrats for great conversation with Cathy Moore. I am reading this remarkable book... I follow you!! greetings from Peru
@@cynthiareid6416 yes, and good idea! Thanks for supporting the content 😊
Makes me proud.
Haha because you have the same last name? I’d be proud too 😃
Really helpful summary, thank you
This content is very timely as this is our topic in my Masters Degree in Human Resource Management. Thank you very much for sharing your personal thoughts on the planning perspective. Will subscribe and hoping to learn more from you. Keep rocking!
This old model lacks human capital factors, for instance, level four should also include employee retention (as an example: The customer received high level service and needs are met for the employee - Maslow hierarchy of needs)
Thanks John! I can see how that would be an important organizational factor to consider :D
well explained.
Thanks 😃
Hey man, really good production here. Congratz and thanks.
You're welcome. Thanks for checking it out :)
Any weaknesses of Kirkpatrick model when applying to prepare a Training for nurse learners? Thanks
Amazing
Thanks Priyanka!
Sometimes the BEST videos are the ones with less views and likes.
Thanks Fernando!
thanks for your presentation. can you help me how to get that book?
Hi...great explanation and so helpful.can you tell ACTIVITIES for this each 4 level that helps to language learning trainees or teacher? Thanks in advance.
Thanks so much! And I am not sure I understand the question. Are you asking about how to assess language learning?
@@DevlinPeckYT thanks for the reply and yes, like how to assess and example that might suits for language learners.
@@sindhuvelmurgan361 Level 1 pertains to how the audience reacted to the learning experience, level 2 pertains to how much they learned, level 3 pertains to whether or not they're using what they learned in a real-world context, and level 4 pertains to the impact that their newfound langauge skills are having on the organization that they're working for :D
@@DevlinPeckYT Thanks alot.
@@sindhuvelmurgan361 you’re welcome 😃
Thank u for the sharing Devlin. I'm wondering any other factors influencing training effectiveness score..for example virtual learning vs physical learning, virtual 2 hours vs full 8 hours..do u have any view? I've been asked to this question a lot by the leaders..
Hello hello! Thanks for checking out the video. Those items would be great things to ask your intended audience (and collect data about how they’re received, how they impact learning, whether people use those new skills on the job at the same rate, etc).
@@DevlinPeckYT thank u for the idea..would be a good start to do survey..the learning platform have just swift from physical to virtual since COVID pandemic. thanks again yaa!
@@MsNedy you’re welcome! 😊
Grab the FREE interactive "Become an ID" checklist: www.devlinpeck.com/become-an-id-checklist
Ch15 right here.
Haha I'm not sure I understand this reference, Faizan :D
@@DevlinPeckYT Lol that was brief. Its from the textbook Leaving ADDIE for SAM.
@@flagshipbuilds ok, good to know. Going to record some SAM content soon!
@@DevlinPeckYT How about content on the AECT standards? My portfolio for my MS in LDT will be judged against it this semester for graduation but NO employer ever mentions it in the job listings which mean they do not know it or its skill over credential or both.
@@flagshipbuilds Good suggestion! Thank you :D
Hi Devlin! Is the designing part in the planning process (the 4th point: designing attractive intervention) related to the reaction level (i.e. Level 1)? If yes, then how?
Hi! Yes, exactly. If people find that it's appealing / attractive / engaging, then that's a level 1 reaction :D
@@DevlinPeckYT Thank you, Devlin!
Level 3 actually requires additional time from SME persons. They should later send the results of whether something has changed in terms of user behavior. In most situations they are very busy and this is rarely delivered unfortunately. In most cases, you will not get these results because of the organization you are in. This also applies to the entire level.
Thanks for the insights! Gathering information for Level 3 can be tricky for sure.
I am moving to the USA. I have experience as trainer and university professor. Do you recommend to offer to design a course to a local business, in my home country, for free in order to build my portfolio?
That could work, Fabian! But you could also create a concept project so that you do not need to rely on unavailable SMEs, unreasonable requests from clients, etc.
Great videos. It's beyond me as an educator, why we can't graduate better knowledgeable, skilled, and intelligent students with all the tools we have in the United States. This then tells me that it has to be a discipline issue.
Thanks so much! And I'm sure everyone is trying their best :D
Devlin -- I've worked on projects where there's an interest in knowing an ROI to justify the cost of say, narration or video vs just Powerpoint, but there's NO interest in looking at level 2 or 3. Maybe it's compliance or other training where the ROI is a little squishier. Management is only interested in justifying the cost and cares less about the long-term results. How would Kirkpatrick address this? Perhaps that kind of ROI is better evaluated with just infographics on the value of eLearning? Thx.
I think that Kirkpatrick would say that those efforts are unlikely to produce any real results. If the stakeholders are not interested in improving business metrics or actually changing human behavior, then there are few cases where the efforts will make a difference for the audience or the organizaton.
@@DevlinPeckYT Thanks for the reply. I think that for compliance and other BAU training there’s no interest in metrics.. only in compliance. No behavior change if the employees are already in compliance, so it comes down to evaluating the return on the training investment every year to keep the status quo.
@@catboy721 Yeah, compliance training is notoriously one of those "checklist" training items. "Did employees complete it? Then we're in the clear!"
Yes, the same doubts! Thanks for displaying it!
@@cynthiareid6416 you’re welcome!
Did you make your intro on AE?
Good question. I actually got a template from Envato and then just added my logo to it in AE 😃. And got a separate music file from Envato as well.
@@DevlinPeckYT Nice!
@@flagshipbuilds thanks Faizan!
I liked and subscribed please share youre references. espicially google scholar
Hi Travis! I appreciate the support. I referenced this book when I was making this vid to make sure I didn't miss / misrepresent anything: www.kirkpatrickpartners.com/product/Kirkpatricks-Four-Levels-of-Training-Evaluation/
Hi
Hi there! Thanks for watching :)
I love how this guy seems annoyed by his own presentation. I could hear him sigh...... hahahaha dude, we're not forcing you to do this, you want to present this information to us 😅🤣🤣🤣🤣🤣
👀👀👀. I must have caught myself on a bad day with this one haha. Definitely happy to present the info but it can be rough some times…especially in these earlier days when I was newer to RUclips 😂