Overcoming Evaluation Obstacles

Ask nearly any Learning and Development (L&D) professional about his/her efforts to evaluate training and the responses can be described as dismal, at best. L&D and organizational leaders alike recognize the value in collecting data to determine whether learning supports business results. Who hasn’t heard a leader ask for the business impact of learning? Yet organizations’ L&D often faces barriers that prevent them from delivering data-driven results.
​
Association for Talent Development’s (ATD’s) 2016 research report, Evaluating Learning: Getting to Measurements, surveyed 199 talent development professionals and found that only:

There’s room for improvement. So, what gets in the way? The same report asked participants a similar question.

Identifying and Overcoming Barriers

Participants reported challenges isolating the impact of learning on results and a lack of:

• Access to critical data (e.g., financials, performance records)
• Learning management system (LMS) functionality
• Resources (e.g., staff time and required tools to conduct evaluations)1

These barriers may be of no surprise. Still, what simple steps can an organization take to overcome them?

Let’s look at each barrier and lay out some tips to address them.

Access Critical Data

• Know C-level suite and leader language. Identify:
  • How do they speak?
  • What terminology do they use?
  • What are their individual communication styles?
  • What steps can you take to communicate more effectively with them?
• Develop financial and business acumen. Assess your skill level, and ask yourself:
  • What are my knowledge gaps? Am I well-versed in basic financial skills such as cash and accrual accounting? What about reading and understanding basic financial statements, preparing budgets, analyzing variances, and financial analysis to support initiatives?
  • What data do I need?
  • What information exists?
  • How is it collected, and where is it located?
• Build a business case to access information across the organization and secure a seat at the table. Or, better yet, just do it! Being proactive communicates a strong measure to leadership and those with whom you want to collaborate.

LMS Functionality

Make your product work for you! Once you identify what you want to measure, connect with your provider to discuss solutions. If the LMS has limited capabilities, what technology and systems exist in-house to help you achieve your goal? For example, is there an existing dashboard that could capture required information? Rethink current analytic processes and brainstorm on expanding functionality.

Resources

Evaluation is as much of a mindset as it is a strategy. Develop a two-pronged approach that aligns with the Kirkpatrick-Phillips model and apply that strategy at the onset of every project discovery discussion and beyond.

Yes, data collection often poses challenges, but don’t overthink it. Look for ways to simplify it. You don’t want or need ROI for every course or effort.

Consider some of the examples below, or use them to spark ideas for other methods to evaluate learning opportunities:

1. Create an evaluation plan that sets standards and expectations, and then share it with stakeholders and internal clients. Encourage feedback, revise as needed periodically, and build buy-in. This communicates a convincing, unified purpose, and that evaluation is part of the company culture.
2. Establish role responsibility and gain commitment from your team and project members. Everyone has a stake in implementing evaluation as part of the company culture. What does that look like? How is it executed?
3. Seek evaluation capacity-building opportunities. When L&D is entrenched throughout the organization, conversations organically arise for input and collaboration. For example, the Sales Department is discussing and vetting new technology to support sales analytics. This is a perfect opportunity to identify and lead with a proactive evaluation mindset.
4. Design course objectives that support business needs and quantify outcomes.
5. Distribute pulse surveys to learners, their managers, and other relevant stakeholders. Pulse surveys can inform you about the training content value and applicability. They may also reveal additional areas of improvement for the learner or unpredicted outcomes.
6. Gauge application on the job. Develop and implement an observation component that evaluates application in 30, 60, or 90 days to assess learning integration.
7. Assess learning by implementing tests and assessments during and after the training.
8. Invite stakeholders and learners to participate in focus groups to assess skill application of the program.
9. Require learners to create action plans during training while the learning experience is still fresh. This drives skill application and integration, identifies learning metrics on the job, and holds learners accountable for action.
10. Monitor business performance and operational data and performance records to determine improvement and assess any unpredicted impacts.

Achieving Results

Maximizing resources, improving business impact, and creating value for learners are some of the outcomes an organization can achieve when evaluation obstacles are removed. It starts by creating an evaluation mindset, commitment, and culture. An organization’s efforts don’t have to be costly or time-consuming to accomplish evaluation goals that link to business outcomes. Developing an evaluation strategy can guide and automate efforts and set an organization on the right path.
 
Stay tuned for an upcoming blog post where we introduce Net Promoter Scores (NPSs) as an evaluation tool.

[1] Ho, M. (2016, April 07). Evaluating the Business Impact of Learning: Why Aren't More Organizations Doing It? Retrieved November 15, 2017, from https://www.td.org/Publications/Blogs/L-and-D-Blog/2016/04/Evaluating-the-Business-Impact-of-Learning