This project is the first phase of a safety e-learning program for a company named Western Manufacturing. This scope of this topic includes a short overview on why the safety program is needed, what the company hopes to accomplish by teaching this, and then touches on mandatory protective gear and facility warning signage, and three common workplace hazards. The below highlights the project description, learning objectives, how I went about testing my learning objectives with users of the safety training system, and detailed results.
About the Project
Client Created at
Personal Work
DePaul University
The purpose of this safety program and the context in which it could be used is for a basic introduction for new hires to the company, for existing employees who change positions, and as a yearly reminder for all employees.
It is never too early to begin safety instruction, especially at a manufacturing facility, since according to OSHA.gov (Occupational Safety and Health Administration), “4,836 workers were killed on the job in 2015—on average more than 93 a week or more than 13 deaths every day.” Out of this number, 21.4% were in the construction/manufacturing industry. Because of this, many companies have put into place a mandated safety program which may also need to follow city or state legal requirements. These safety programs and a goal of “zero accidents” are now the highest priority of most companies, even over production of products. However, many companies only have a very basic safety program which may include the reading of long text documents followed by the requirement of a signature at the end. In creating an interactive eLearning application for a company safety program, it will not only help employees better consume and understand the safety topics, but may also help in preventing future accidents or death.
The learning objectives are based on “A Revision of Bloom’s Taxonomy: An Overview” by David R. Krathwohl (2002).
Multimedia – Use words and graphics instead of only words
Contiguity – Align printed and spoken words to graphics
Modality – Use audio narration instead of on-screen text
Redundancy – Explain visuals with audio or text, but not both
Coherence – No extraneous words/sounds/graphics present
Personalization – Use conversational style, polite wording, and human voice
Segmenting – Break lesson into parts
Practice – Use practice questions to achieve learning objectives
Program Control – Use pacing control, but limit learner control for novice learners
Feedback – Have immediate and helpful feedback
These were the questions that were included on both the pre-test and post-test. All learners were requested to take the pre-test, then go through the safety training course, then take the post-test. Their answers were then compared to evaluate learning of the material. The (X) next to the answers highlights the correct choices.
Other miscellaneous questions:
On Pre-Test Only:
On Post-Test Only:
Number of participants tested: 12
Number of identical questions on pre-test and post-test: 9
Questions correct out of 9 for each user:
Other Findings:
Most learners had previous workplace safety training experience: 10 out of 12
Comments/Feedback From Learners:
Results Comparison
Looking at an average of all scores, post-test scores were over 40% higher than pre-test scores.
100% of learners scored higher on the post-test compared to their pre-test scores
Pre-Test:
Mean Score: 3.5, 95% Confidence Interval: 2.993 thru 4.007
Post-Test:
Mean Score: 7.25, 95% Confidence Interval: 6.268 thru 8.232
The p value is 0.000 and is less than 0.05.
Therefore, the probability of this result, assuming the null hypothesis, is 0.000.
Effect size is a quantitative measure of the strength of a phenomenon.
Result of effect size calculation is 3.208 and since it is greater than 0.5 it would be considered a large statistically significant effect. Learning scores were increased from pre-test to post-test by more than 3 standard deviations.
(post-test mean minus pre-test mean) / (average standard deviation)
(7.25 – 3.50) / (1.54 + 0.798) / 2 =
3.75 / 1.169 = 3.208
Overall I feel this initial implementation and learner testing was a success. The fact that all learners had a higher score on the post-test than on the pre-test and the effect size was well over the ideal of 0.5, indicates that at least in the short term, I was able to help users gain knowledge about something they did not know when they started the course.
This is positive, however many questions remain. Would these learners be able to retain this knowledge in their long-term memory for an extended amount of time? Would they be able to transfer the knowledge to their everyday experiences and jobs? Would taking this course actually help them prevent an injury or death in the future?
I would make the assumption that the answer to these questions would be most likely no, but this course was also not intended to be the only element of a company safety training program. It is meant to be the first introductory eLearning course of many that will also be used alongside other learning opportunities that would make for a better chance of far transfer of learning, such as in-person interactive demos, monthly safety meetings focused on single topics, and one-on-one discussions.
Thinking long-term, if I were to implement a full safety training program based on the single course I have started, I could then test and evaluate how each element would affect long-term retention of knowledge, far transfer of knowledge, and also see how each would hopefully lower the amount of accidents and injuries after program implementation.
In the short-term however, I could make changes to the course I have created. I would most likely get into more detail about the personal protective equipment (PPE) and facility warning signs (learning objectives 2 and 3), as almost all learners answered those questions correct on the pre-test. Since it appears they already have knowledge transfer from previous experience, I could dive a little deeper and experiment with more difficult questions. The other way I could approach this would be to seek out learners with zero background in workplace training, since 10 out of 12 learners that took my course had previous experience, that could also explain why most got those questions correct.
Another aspect I would change would be the table about which type of fire extinguisher to use on which type of fire (learning objectives 4 and 5). Many learners did answer the questions about this incorrect on both the pre-test and post-test. I feel that a possible cause of this was there being too much information on one page, and learners had to use more cognitive thought to study and focus on the table themselves. While creating it, I was unsure what the best way was to present this information. I felt adding audio to it would make it even more confusing since there was text and a lot of detail. The ideal choice would have probably been to spread out the information on that one page to three separate pages—one for each type of fire, but I was also worried about the time constraints for the assignment.
Again, I do feel the creation and testing of this implementation was successful overall and a great starting point if I were to take this further.
