Qualification Achievement Rates - Calculations

Introduction

Qualification Achievement Rates (QAR) are essential metrics for evaluating the effectiveness of training programs. Our QAR calculations are based on the official specifications Published by the ESFA each year. These calculations involve dividing the number of achieved programme aims by the number of leavers for a given hybrid end year. This article aims to help you understand how we calculate the QAR metrics shown in our reporting dashboard, using a number of scenarios.

Overview of the calculation

The QAR calculation, at its core, is straightforward. It involves dividing your overall number of achieved programme aims by your overall number of leavers for any given hybrid end year. This document details the various elements used to calculate the two key variables: Overall Achiever and Overall Leaver.

The elements that go into the calculation are;

Cross year matching

Data submitted via our ILR Upload Portal over the last five years is used to create a QAR dataset for reporting. This data is ingested into our data warehouse to process your QAR calculations.

Many learners take more than one year to complete, meaning the same aim for a learner can appear in multiple ILR files. And so, a process of cross-year matching is used to determine which record is included. We use combinations of variables to ensure we match the correct records across years, following ESFA guidance.

The cross year matching process utilises the UKPRN, Unique Learner Number, Programme type and Framework code or Standard code together with the following combinations, in the following sequence:

  • Learning start date, Learning planned end date and Learning actual end date
  • Learning start date and Learning planned end date
  • Learning start date and Learning actual end date
  • Learning start date

Once we have matched programme aim records, we discard those from previous years, retaining only the latest information for each programme aim in the QAR dataset.

Note: If you have not uploaded all 5 years worth of ILRs in our upload tool QAR calculations may differ substantially to what the ESFA calculate. Where a year in the middle of the sequence is missed, we will only calculate back to the latest consistent upload. For example, by submitting years 1, 2, 3 and 5 we will only run calculations back to years 1-3.

 

Hybrid end year

We base the overall QAR on the hybrid end year, calculated as follows. It is the later of:

  • the achievement year of an apprenticeship
  • the actual end year of an apprenticeship
  • the planned end year of an apprenticeship
  • the reporting year

Exceptions occur when the learner is flagged as overdue continuing or an overdue planned break. In these instances, the hybrid end year is calculated as follows:

Overdue continuing aim

  • One year after the reporting year in which the learner last appeared.

Overdue planned breaks

  • The year after the actual end year or planned end year of the aim, whichever is later

Exclusions

There are various reasons why an aim may be excluded from the QAR calculations, as defined by the ESFA. Within the provided Bud reports, we include exclusion reason codes to help you understand why an aim may not be included and to assist with reconciliations to ESFA data. These codes are shown in the table below:

Code Description Detail
1 Transferred to new aim at the same provider Withdrawal Reason = 40 AND A corresponding aim exists starting within 120 days of the Actual End Date
2 Aim has transferred to a new provider - ESFA intervention Withdrawal Reason = 7
3 Aim has transferred to a new provider - specific government strategy OR Aim has transferred to a different provider as a result of a merge Withdrawal Reason = 41 OR 47
4 Traineeships with a positive outcome This exclusion reason is not currently coded into the calculations
5 Breaks - not overdue Learning aims on a break (completion status = 6)
Aim is not an overdue planned break
6 Unemployed learner claiming Universal Credit or JSA or ESA Unemployed learners claiming Universal Credit, Job Seeker’s Allowance or Employment and Support Allowance who cannot continue their learning as they gained employment
7 ESF aim European Social Fund funded learning aims recorded under Funding model 70 (‘ESF’)
8 Community Learning aim Community Learning funded learning aims recorded under Funding model 10 (‘Community Learning’)
9 Aim withdrawn before the qualifying period Where a learner, withdraws without achievement, within the respective funding qualifying period (QP) for their planned programme duration.
10 Learning generates no funding Aim Funding Model = 99
11 Innovation Code aim Innovation Code learning aims (references ZINN0001 to ZINN0006 and Z0004474 to Z0007833)
12 Unitisation qualification This exclusion reason is not currently coded into the calculations
13 Work Experience or Placement aim Aim Reference is Z0007834 - 38 or ZWRKX001 - 9
14 Specific LDM codes LDM Code = 332, 337, 341, 348, 034, 383,  332, 341, 351, 320, 321, 386
15 Overdue Continuer Where a learner has previously been flagged as an overdue continuer and counted against a previous QAR

 

Overall Leaver

This field indicates that a programme aim has ended, is an overdue continuing learning aim or an overdue continuing planned break. The value is set to 1 where;

  • Learning actual end date is present
  • OR achievement date is present
  • OR Overdue_Continuing = 1
  • OR Overdue_Planned_Break = 1

Overall Achiever

For the QAR calculation, any programme aim is deemed to be an Overall_Achiever if the aim has an Outcome of 1 (Achieved).

 

Overdue Breaks

Generally, programme aims on a planned break are excluded from the QAR as defined in the above exclusion reasons table. If aims have not returned from their break timely, they are treated as a withdrawn learner. An overdue_break flag is set to 1, Overall_Leaver is set to 1, and the hybrid end year is adjusted to the year after the actual end year or planned end year of the aim, whichever is later.

The steps to identify these aims are as follows;

  • Completion Status = 6
  • The aim actual end year is 1 or more years before the QAR year or earlier. Aims with an actual end year in the QAR year have until the end of the next year to return.

We then identify the restart aims to determine if they are overdue;

To be considered a restarted aim, there must be a corresponding aim to the break aim where:

  • The aim reference = break aim reference
  • The aim start date > break aim actual end date
  • The aim original start date = break aim start date or break aim original start date if there are multiple break aims
  • The restart indicator is 1

If the aim actual end year is 2 years or more before the QAR year, there must be a restarted aim with a start date before the aim actual end year + 2 years.

For example; for QAR Year 2022-23: If an aim ended in 2020/21, and there isn't a restart by 31st July 2022 (R12 2021-22), the aim is an overdue planned break.

 

If the aim actual end year is in the year before the QAR year, there must be a restarted aim with a start date before R04 of the QAR year +1.

For example; for QAR Year 2022-23: If an aim ended in 2021-22 and there isn't a restart by 30th November 2023 (end of R04 2023-24) the aim is an overdue planned break.

 

Overdue Continuing Aims

Continuing learning aims (with a completion status of 1) in the R14 ILR return for a funding year, that do not have a corresponding record in the following funding year, are treated as a withdrawal for the overall QAR calculation.

For example, if you submitted the learning aim as completion status 1 in the 2022-23 R14 ILR and there is no corresponding record in 2023-24 ILR return;

  • The hybrid end year is set to one year after the learner last appeared (in this example 2023-24)
  • Overall_Leaver is set to 1
  • Overall_Achiever is set to 0

If you subsequently return the record in the ILR in any of the years after, the aim is excluded as per the above exclusions table so the aim does not count towards two different QAR years.