Certainty Principle: How Do We Know Training Improves Performance?
Many organizations ask for feedback at the end of a training class – was the training relevant, was the instructor interesting and what information will you apply on the job? However, few organizations measure the impact of training on employee performance. This is usually because it takes too much time or is too expensive. As knowledge leader in employee development, our staff at Anne Arundel Community College’s Center for Workforce Solutions (CWS) challenged ourselves to respond to this market need by creating a cost-effective, easy-to-use learning outcomes tool which could be contextualized for any organization or development need.
Author: Veronica Townsend, Solutions Developer, Center for Workforce Solutions, Anne Arundel Community College
There are four levels of learning measurement that professional training standards include:
- Participant reaction (did I like it?)
- Knowledge gain (what new knowledge was acquired?)
- Behavior (what changed on the job?)
- Results (what was the impact on organizational measurements?)
Since we recognize the level of impact that accurately measuring on-the-job behavior change can have on training programs and participants, our staff worked to develop a simple process that provides standardized scoring methods while taking minimal client time. Clients spend thousands of dollars on training, and CWS wanted to show the value of this investment by comparing participant performance before and after the training.
Our formula for training success used a three-step approach:
- Use focus groups of participants, managers and clients to define the current performance and desired future performance of the participants.
- Provide a series of training sessions spaced over time
- Involve the supervisors of the participants in reinforcing the new desired behaviors
The Pilot
In the pilot program, our staff partnered with Goodwill Industries of the Chesapeake, using about 30 participants, to introduce this process in a real-world environment. Industries of the Chesapeake, using about 30 participants, to introduce this process in a real-world environment. Research indicates achieving the greatest change in behavior depends on reinforcement by supervisors. We therefore made sure our plan provided training highlight sessions for supervisors of the participants as well as tips to help supervisors lead on-the-job reinforcement activities, follow-up session for participants to share their stories of achievement and rubric identifying beginning, developing, accomplished and exemplary performance to measure the results. A section of the rubric is shown below.
|
Behaviors to Achieve Success |
Beginning 1 |
Developing 2 |
Accomplished 3 |
Exemplary 4 |
|
% of time behavior occurs: |
Less than 20% |
26-50% |
51-75% |
>75% |
|
Listening: Listens to understand what the resident/ employee says they need as well as their underlying needs. Asks questions to clarify. |
|
|
|
|
|
Open to Feedback & Change: |
|
|
|
|
To fine-tune the rubric and automate the survey tool, we turned to Nassim Ebrahimi, Ph.D., Director of Learning Outcomes Assessment, who integrated the CWS approach into AACC’s college-wide emphasis on measuring learning outcomes.
“Learning is at the heart of AACC’s mission,” says Ebrahimi. “Documenting proof of learning and using the information to enhance our offerings improves learning opportunities and better prepares your employees for success in the workplace.”
Participants completed the rubric as a self-evaluation, while supervisors of each participant evaluated observations over three stages: before the training, three months after the last training activity, and again five months later. All the participants and their supervisors needed to do was check the appropriate box.
CWS staff then delivered both an in-depth data analysis and a trend analysis to Goodwill’s management team, along with practical and objective feedback. In the pilot program, the average ratings for each of the seven performance competencies improved between 26% and 53%. The normal bell-shaped curve of the overall “before training” rating - about 50% in the lower “beginning” and “developing” ratings - was transferred to 90% of the participants in the top two rating of “accomplished” and “exemplary” in the “after training” ratings. The improved ratings were sustained between the three- and five-month measurements, indicating sustained behavior change was observed by the supervisors on-the-job.
Our staff was excited to work with Goodwill in providing one of our key learning philosophies - that training success is not just about having one great learning event in the classroom. We have found that employee
performance improves through a series of trainings, reinforcements and evaluations. When this learning is linked to observable change, the value of the investment in employee training is apparent to all.
As a competitive differentiator, rubrics are simple and effective way to visually demonstrate the impact that training has on performance improvement. Following the success of this learning outcomes pilot, our CWS staff developed an online survey process to accommodate larger groups of participants. As a knowledge leader both locally and nationally, CWS is committed to providing employee development that delivers measurable results and will continue to reach out to other organizations to develop a similar program, custom-designed to achieve exceptional performance improvement in any competency in any organization.