Using e‑learning analytics to improve training outcomes and learner retention in compliance programmes

If you coach a sports team and never look at the score, the GPS data or the heart rate, you’re guessing. It’s the same with e‑learning. Most organisations still “run the session”, tick the box, and hope people remember enough to stay compliant.

Hope is not a strategy. Data is.

In this article, we’ll look at how to use e‑learning analytics the same way a good coach uses match stats: to improve performance, fix weak spots, and stop people dropping out of your compliance programmes.

What e‑learning analytics actually is (in real life, not in theory)

E‑learning analytics is just this: collecting and using data about how people learn online, so you can improve both the training and the results.

Forget the buzzwords. Think in simple questions, like a coach:

• Who is showing up?
• Who is dropping out?
• Where are they struggling?
• How fast do they progress?
• Does performance in training translate to performance on the job?

Your LMS or e‑learning platform already tracks most of this. The problem is that many organisations either never look at it, or they only use it for reporting (“94% completed the course… great, next topic”). That’s like celebrating possession stats when you lost 3–0.

The goal here is different: use analytics to actually improve training outcomes and retention over time.

Why compliance training loses people (and what the data usually shows)

Let’s be honest: most compliance training is treated like admin, not like performance training. The result?

• People click as fast as possible.
• They retain the minimum necessary.
• They repeat the same mistakes on the job.
• They dread the annual refresher.

When you look at the analytics from these programmes, you typically see the same patterns:

• High drop‑off at the start: Many learners stop in the first 10–20% of the course.
• Quiz “retry spam”: Learners fail the first quiz, then pass on the second or third attempt with almost no extra time spent. Translation: guesswork and clicking through answers, not learning.
• Huge time variation: Some learners “finish” a 40‑minute module in 8 minutes. They haven’t trained. They’ve clicked.
• Low score on scenario‑based questions: People know the policy sentence, but not what to do in a real situation.

If you ignore those signals, nothing changes. People complete the module. The LMS shows a nice completion rate. But in reality, your compliance risk is still high.

The key metrics that actually matter

Let’s keep this simple. You don’t need 40 dashboards. Start by tracking these core metrics for each compliance course:

• Enrolment vs completion rate (how many start, how many finish?)
• Time‑on‑task (how long do they actually spend in the module?)
• Drop‑off points (where do they stop or disappear?)
• Quiz performance
  • Average score on first attempt
  • Number of attempts per quiz
  • Time spent per attempt
  • Questions with the highest failure rate
• Return rate (how many learners come back later to review or check content?)
• Device and access time (desktop vs mobile, office hours vs off hours)

Once this is stable, you can add “performance” metrics that connect training with workplace results, such as:

• Number of incidents or near‑misses before vs after training
• Number of reported breaches or policy violations
• Audit findings related to the trained topics

Think like a coach: training data is good, but competition data (real‑world behaviour) is better.

Using analytics before, during, and after a compliance programme

The mistake most organisations make is to look at data only at the end: “Did people complete? Did they pass?” That’s too late. Instead, use analytics across three phases: before, during, and after.

Before: diagnose the real needs

Before you build or launch a new course, look at your existing data for that topic:

• Which modules have the lowest completion rate?
• Which questions are most often failed or skipped?
• Which teams or locations have the most incidents linked to that compliance area?

Example: You’re updating your Health and Safety induction. Analytics shows:

• New starters consistently fail questions on manual handling and PPE.
• Time‑on‑task is high on the section about fire safety, but scores are fine.

Action plan:

• Shorten the fire safety content (it’s understood, but maybe too long).
• Build more scenarios and practice around manual handling and PPE.
• Add a short pre‑assessment to identify at‑risk learners and teams.

Just like in sport: you don’t design the same training week for a team that lacks endurance as for one that lacks strength. The data tells you what to prioritise.

During: adjust while people are still learning

Once the programme is live, don’t wait until next year’s review. Track what’s happening in real time, especially in the first 2–4 weeks:

• Are people opening the course promptly after enrolment?
• Where do they pause or drop?
• Which questions are repeatedly failed on first attempt?

Here’s a simple weekly “coaching review” you can run with your compliance or L&D team:

• List the three slides or sections with the biggest drop‑off.
• List the five questions with the worst first‑attempt scores.
• List the teams or departments with the lowest completion rate.

Then decide on quick interventions for the next week:

• Send targeted reminders to low‑completion teams, with a clear deadline.
• Clarify wording on the most failed questions.
• Add a short explainer video to a difficult section.

In sport, you don’t wait until the end of the season to change tactics. You adjust on the fly. Same logic here.

After: connect training to behaviour and risk

When the first wave of learners is done, look at how training performance relates to real‑world behaviour:

• Do teams with lower quiz scores have more incidents?
• Are repeat offenders (policy breaches, safety violations) the ones who rushed the course or failed multiple times?
• After a refresher module, do audit findings improve in that area?

If you find, for example, that most data protection breaches come from staff who completed the GDPR module in under 10 minutes and barely passed, you have a clear signal: completion alone is not enough. You need to redesign how you train that high‑risk group.

Turning analytics into action: simple changes that move the needle

Let’s get practical. Here are concrete ways to use analytics to improve both outcomes and retention.

Use time‑on‑task to spot fake learning

If your module is designed for 30–40 minutes, but 25% of learners “finish” in under 10 minutes, you have a problem.

Operational steps:

• Define a realistic minimum completion time for each course (e.g. 25 minutes).
• Flag any completion under that time as “non‑valid learning”.
• Automatically re‑assign the module or an extra quiz to those learners.
• Communicate this clearly up front: “Rushing through will trigger a re‑take. Take the time to do it once, properly.”

Yes, you’ll be less popular. You’ll also be more compliant.

Use quiz data to rebuild weak content

If a particular question has a repeated failure rate above, say, 40% on first attempt, something is off. Either:

• The content doesn’t explain it clearly.
• The question is badly worded.
• The scenario is unrealistic.

Action plan for each “problem question”:

• Review the related content slide: is the key idea obvious, or buried in text?
• Ask 2–3 learners to explain the concept in their own words.
• Rewrite the question to focus on a single idea, with a clear scenario.
• Test the new version with a small group and compare scores.

You don’t guess. You test, adjust, and re‑test. Exactly like changing a drill when it doesn’t build the skill you want.

Use drop‑off points to redesign the learning journey

When analytics shows that 40% of learners leave between slide 8 and 12, that’s a red flag. Something in that segment is either:

• Too long
• Too boring
• Too complex
• Technically buggy

To fix it:

• Cut long explanations into short, focused segments (2–3 minutes max per concept).
• Insert a quick interaction or scenario after 3–5 slides.
• Replace dense policy text with:
  • “In this situation, you must…”
  • “Never do X when…”
  • “Always report Y within Z hours.”
• Test the updated flow with 5–10 learners and check if drop‑off decreases.

Think of it like conditioning work: you don’t give a 40‑minute continuous monologue and expect focus. You break it into intervals.

Use cohort comparison to target support

Analytics lets you compare results between groups:

• Department A vs Department B
• New starters vs experienced staff
• Office staff vs field staff

If one group is clearly underperforming (lower scores, higher drop‑out, more incidents), don’t just blame them. Ask:

• Do they have less time to take the training?
• Is the content less relevant to their reality?
• Do they face technical access issues (devices, bandwidth)?

Then adapt:

• Offer shorter modules for field staff, with offline access.
• Include scenarios taken from their specific environment.
• Plan dedicated time slots with manager support for completion.

Same sport logic: you don’t coach your goalkeeper like your striker. The foundations are the same, but the application is different.

Building a simple analytics “playbook” for compliance

You don’t need a big transformation programme to start. You need a clear routine. Here’s a basic monthly playbook you can run with your team.

Every month, for each key compliance course:

• Check:
  • Enrolment rate
  • Completion rate
  • Average time‑on‑task
  • Average first‑attempt score
• Identify:
  • Top 3 drop‑off sections
  • Top 5 most‑failed questions
  • 2–3 teams with lowest completion or scores
• Decide on:
  • One content change (shorten, clarify, add scenario)
  • One communication change (better reminder, message from leadership, clearer deadline)
  • One support action for at‑risk teams (Q&A session, manager briefing, alternative format)

Track the impact over the next 1–3 months. If:

• Completion goes up
• Drop‑off goes down
• Scores improve
• Incidents reduce

… then keep the change and move to the next weak point.

Common mistakes when using e‑learning analytics

A few traps to avoid if you don’t want to drown in data or send the wrong message.

• Chasing vanity metrics only. High completion with low engagement and no behaviour change is a false win.
• Over‑penalising low performers. Use data to support and coach first, not just to punish.
• Ignoring context. A low score from a team with poor equipment or no protected learning time tells a different story than a low score from a well‑supported team.
• Making reports no one reads. If your analytics report doesn’t result in at least one concrete action, it’s just decoration.
• Changing too many things at once. Adjust 1–2 variables, then measure. Like in training: if you change volume, intensity, and frequency at the same time, you won’t know what worked.

How to get started this month (simple 2‑week plan)

If you want immediate impact without overhauling your whole system, try this focused plan.

Week 1:

• Pick one critical compliance course (e.g. Health and Safety, Data Protection, Safeguarding).
• Pull basic data:
  • Enrolment and completion rate
  • Average time‑on‑task
  • Average first‑attempt quiz score
  • Top 5 failed questions
  • Sections with biggest drop‑off
• Choose:
  • One question to rewrite
  • One section to shorten or make more interactive
  • One team or group to support more closely

Week 2:

• Implement those changes in the course.
• Send a clear, specific message to the target team:
  • Why the training matters (link to real risk or incidents)
  • What is expected (deadline, time needed)
  • What support is available (contact, Q&A, manager help)
• Track:
  • Completion rates for that team
  • Scores on the updated question
  • Drop‑off on the revised section

If you see even a modest improvement (5–10% better completion, fewer failures, more stable time‑on‑task), you have proof that analytics‑driven changes work. Then you scale the approach to other courses.

From “tick‑box” to performance mindset

Compliance will never feel like game day at Wembley. But it can shift from “necessary evil” to “effective risk training” if you treat it like performance work, not paperwork.

Use your e‑learning analytics like a coach uses stats:

• Define the right metrics.
• Review them regularly.
• Make small, targeted changes.
• Check the impact on real‑world behaviour.

In sport, the teams that improve season after season are not the ones who shout the loudest. They’re the ones who measure, adjust, and repeat. Apply that same logic to your compliance programmes, and you’ll not only keep people compliant – you’ll keep them learning, engaged, and on the pitch instead of in the risk report.