Training Effectiveness: How to Start with a Metric

In a 2023 study carried out in North America, only 33% of those responsible for learning and development measured the impact of training on financial results (Canadian HR Reporter). In many companies the question "what did the training deliver?" is still asked today, and the answer is often a headcount of participants and an average survey score. A survey alone, however, is not an evaluation of training effectiveness.

None of this means training does not work. It means something else: the effect may be there, but if nobody measures it, the business will not see it. A salesperson may have started handling conversations better, and a manager may have started assessing the team more rigorously. If nobody wrote down how things looked before the training, all that is left afterwards is an opinion.

What companies measure after training, and what they skip

The 2018 survey by the CLO Business Intelligence Board covered 1,500 people working in learning and development. Participants' reaction to training was measured, or planned to be measured, by 93% of companies. Impact on work results was measured by around half, and 26% had calculated a return on investment in the year of the survey. These figures are not directly comparable: the first includes companies that were only planning to measure, and the last counts only those that actually did. In many companies, measuring training effectiveness stops at a survey.

The 2018 study is old by now, but a more recent one, from 2023, shows a similar pattern. It was carried out by D2L together with Training Industry and covered 329 people responsible for learning and development in North America. Of them, 33% measured the impact of training on financial results, and 46% said that demonstrating a return on investment was a challenge.

Both studies show the same picture: almost every respondent measured, or planned to measure, participants' reaction, about half checked what had changed at work (2018), and a third measured the effect on financial results (2023). Practitioners say so themselves: Training Industry writes that many organisations still struggle to measure behaviour and results, the third and fourth levels of evaluation. An evaluation survey tells you whether participants liked the training, that is, it rates the quality of training in their eyes rather than its effectiveness. The board asks about something else.

The four levels of the Kirkpatrick model

The best-known model for evaluating training, the Kirkpatrick model, has four levels: reaction, learning, behaviour and results. You can check the first two while participants are still in the room, and the last two only at work.

reaction: whether participants liked the training,
learning: whether participants gained knowledge and skills,
behaviour: whether they apply them at work,
results: what changed in the company's numbers.

Assessing training effectiveness therefore takes data from work, not from the classroom.

In the 2018 study, roughly half of companies did not check what had changed at work, and in the 2023 study two in three did not measure the financial impact. Yet the effect is decided exactly there, after the training and with the participant's manager.

Learning transfer: training is an event, application is a process

A training course is usually a one-off event: it often starts and ends within a day or two. Applying what the participant learned is a process that can take weeks. It should be spread over time and tied to everyday tasks: after the training, the participant gets a concrete task at work that uses the new skill, comes back to it in a conversation with their manager, and you check the metric only after several weeks. The training alone rarely moves the metric. What moves it is what happens afterwards.

Why completion reports do not answer the board's question is covered in our article on nine things companies must finally change in learning and development. Here we deal with the moment before: what to do before training is ordered at all.

Start with the metric, not the course content

The usual order goes like this: someone raises a topic, HR picks a provider, sets a date, and sends an evaluation survey after the training. The metric appears, if at all, at the end, when the board asks about results. By then it is too late for a baseline, because nobody wrote down how things were before. With soft-skills training the metric is harder to name, which is all the more reason to set it at the start.

Kirkpatrick Partners, the firm that develops the model, advises reversing the order: start from level four and design the training backwards from the result the company needs. First the metric, then the behaviour that is meant to move it, and only at the end the format: a course, a workshop, microlearning, coaching or something else entirely. The training programme is built on that basis.

Three questions before ordering training

Which one metric should change?A good metric is one the company already tracks and the board knows: sales per salesperson, turnover among new hires in the first six months, time to proficiency, number of complaints. If you cannot name the metric in one sentence, the training does not have a goal yet.
What is it today, and by when should it change?Without a baseline you cannot show a change. Training Industry points out that evaluation at the results level requires measurement before and after the training. Record the figure before the first day of training, not after.
What should participants do differently, and who will see it?Behaviour lies between the training and the metric. If nobody can describe it and nobody will observe it, the result will be left to chance.

A quick test: does your training have a metric?

Take a course you have planned for this quarter and answer "yes" or "no" to five statements. The test does not apply to mandatory training such as health and safety.

Four or five "yes" answers mean you can commission the course. With two or three, fill in the missing points before you sign the order. With one or none, you have a course without a metric, and evaluation after the fact will be guesswork.

Only then choose the format

Once the metric and the behaviour are described, choosing the format gets easier, because it follows from the training needs, that is, from what people have to learn, and not from what happens to be in a provider's offer. Sometimes it turns out that training is not the answer at all. If salespeople know the arguments and still do not use them, another course on argumentation will not help. What will help is checking whether goals and the way performance is assessed reward something else.

Robert Brinkerhoff's Success Case Method draws attention to this difference. It studies extreme cases: a few people for whom the training produced a result, and a few for whom it did not. Conversations with the first group show what role the work environment played in the result. Conversations with the second reveal the obstacles that stopped people from using the training. The full method is more elaborate. Our simplified version: after every larger course, spend a quarter of an hour talking to three people for whom it worked and three for whom it did not.

Involving managers: three short conversations

It is the manager who sees whether someone works differently after a training, and the manager can reinforce it or kill it. Support for participants when they return to work depends mainly on their manager. In the Kirkpatrick model, as Kirkpatrick Partners describes it, this stage is a package of accountability and support that includes, among other things, follow-up conversations with the manager, observation of work and job aids for applying knowledge in everyday tasks (source). Without this, evaluation never gets beyond the classroom.

You will not win a manager over with a slogan about developing people. You win them over with their own metric: their team's sales, turnover in their team, the time it takes their new hire to start working independently. Start the conversation with their number, not with the training.

Three short conversations are enough. Do this for large programmes, not for every course.

Three conversations with participants' managers

Before the training, 10 minutesAsk what behaviour the manager wants to see in the team and how they will recognise it. Write the answer down in their own words.
A week after the trainingAsk the manager to ask the participant what they are already changing in their daily work. One sentence is enough; what matters is that the question is asked.
A month or two laterCheck the metric together and agree what is still getting in the way.

An invitation to the first conversation might sound like this: "We want the sales conversation training to lift sales per person in your team. I need ten minutes from you to talk about what should change in your team's work."

What to show the board: a result, not attendance

When assessing training programmes, the board does not need a training report. It needs a basis for a decision: continue, change or stop. One slide is enough if it has four elements.

One slide for the board

The metric that was meant to change
Its value before the training
Its value after the agreed time
Everything else that could have affected it in that time

The fourth point makes the difference to credibility. The Phillips model (ROI), which adds a fifth, financial level to the four, requires separating the impact of training from other factors, for example by comparing with a group that did not take the training. Training Industry lists changes in the market and in the way the company operates among such factors (source). You do not need to run a full analysis. It is enough to state honestly that new targets started or a price list changed during the same period, and not to credit the training with the whole change. A report that points out its own limits earns more trust.

Converting the result into money, that is, valuing the training in financial terms, is a separate step and, as Training Industry notes, takes more work and rigour. Keep it for programmes where the stakes justify it: large budgets, high training costs and large groups.

The board understands a number that describes a change in the work of the company, not the number of participants. We write about arguments for the board, using employee onboarding as an example, in How to convince the board to invest in onboarding.

Start with one course from your plan

Take a course you have planned for this quarter and go through the test in this article. If a metric is missing, add it before ordering, not after the training. The first conversation with participants' managers will take ten minutes.

FAQ - training effectiveness and evaluation

What is training evaluation?

Training evaluation is checking what a course changed: in participants' opinions, in the development of employees' skills, in behaviour at work and in company results. It most often relies on the four levels of the Kirkpatrick model. A full picture comes only from measuring the last two levels, behaviour and results.

How do you assess training effectiveness if you have no business data?

Start with a metric the company already tracks, even if it does not belong to HR: sales, quality, number of complaints, time to complete a task. If none fits, measure behaviour. A manager can rate on a simple scale how often a participant uses the new way of working, before and after the training.

How do you plan an effective training?

Start with the metric that should change and the behaviour that will move it. Then choose the format, record the baseline value and decide which managers will watch for the change at work. Only at the end, choose the provider.

How do you identify training needs?

Training needs follow from what participants have to learn for the metric to move, not from the provider's offer. If people know the arguments and still do not use them, the problem may be something other than lack of knowledge, for example goals and the way performance is assessed.

What causes low training effectiveness?

Causes of ineffective training include: no metric and no baseline value, no manager support after the training, and training ordered as an answer to a problem it does not solve. Evaluating after the fact on the basis of a survey alone will not reveal these causes.

How do you check a change in behaviour after a training?

Ask the manager to ask the participant, a week after the training, what they are already changing at work, and a month or two later check the metric together. You can also rate the change in behaviour on a simple scale: how often the participant uses the new way of working, before and after the training.

Do you always have to calculate a return on investment in training?

For most training, it is enough to show a change in a metric the board knows. Converting it into money makes sense where the budget and the scale of the programme are large, because it takes more work, including isolating the impact of training from other factors. For small courses, a reliable change in a metric is better evidence than an estimated return.

Do the studies in this article apply to my market?

The studies are a few years old and the 2023 one covers North America, so they show a direction, not a benchmark for your market. That is why the best reference point is the value of the metric in your own company before the training. If sales or finance holds the data, ask for it before the training, not after.

Training Effectiveness the Business Will Notice: Start with the Metric, Not the Course Content