Methodology

Further education skills index

Published

Introduction

This guide sets out the details of the methodology and data sources for the ‘Further Education Skills Index’. It explains the concepts and methods used to calculate estimates of value added, and provides an overview of the data sources and other relevant information.

To assess the full impact of the FE system, DfE periodically produces full Net Present Value estimates compliant with the HM Treasury Green Book. The Further Education Skills Index adds to this by providing a simpler, more tractable measure that we can use to monitor changes in the value-added of the FE system over time. Though the rate of productivity growth is influenced by a number of factors, a country’s skills level is a major component, as giving people valuable knowledge, skills and behaviours boosts their productivity - see UK Skills and productivity in an international context, BIS (2015)

The Further Education Skills Index takes the increases in earnings attributable to different types of FE training and aggregates these to estimate a total impact on productivity. This assumes that in a well-functioning labour market, an individual’s earnings reflect their productivity. This approach is well-established in academia and public policy analysis. See, for example, Becker (1975) and Mincer (1974), and HMT (2018), The Green Book.

Overall approach

The Skills Index covers funded adult Further Education achievers and all apprenticeship achievers in England. The estimation process takes into account the provision type, level and sector subject area of the qualification achieved. Training funded through the Adult Community Learning budget is not included. The Skills Index starts in 2012/13, which is the base year (=100) for all indices included.

Calculation of value-added for different achievement groups

Value-added is calculated separately for each provision type, level and sector subject area by multiplying together estimates of: 

  1. The number of learners that achieved qualifications in each academic year. Where learners achieved more than one qualification, their highest and latest qualification is taken. 
  2. The proportion of learners expected to be employed after achieving their qualification. Static values are used to ensure the Skills Index reflects changes in the value of FE over time rather than short-term changes in economic conditions.
  3. The expected median real earnings for employed achievers. Static values consistent with the expected percentage earnings returns.
  4. The expected percentage earnings returns associated with achieving a qualification, as a percentage increase relative to non-achievers. These are static values, calculated at Sector Subject Area Tier 1 where possible and at provision/level type where a SSA breakdown cannot be produced. See Annex for details of derivation. 

Calculation of the main Skills Index

The value-added totals for each sub-group are added together to create the total value-added for the entire FE system for the year. The total value-added therefore reflects changes in the number of achievers and the provision mix over time. Annual total value-added is indexed to 2012/13 and an annual change figures are calculated. Some subjects with very few achievements do not have an estimated value due to small sample sizes and are not included in the calculation of total value-added.

In simple terms, the value-added is essentially measured as: 

Value-added = FE achievers * employment rate for achievers in that qualification * estimated additional earnings from achieving the qualification

As the employment rate and estimated additional earnings are static values, an increase (or decrease) in the Skills Index would be caused by one or more of: 

  1. An increase (decrease) in the number of learners; 
  2. An increase (decrease) in achievement rates; 
  3. A shift towards (away from) more economically valuable training, through more (less) learning being undertaken in qualifications with higher additional earnings.

Calculation of the value-added per learner index

The total value-added is divided by the total number of achievers for the year, to determine average value-added per learner. This measure reflects changes in the provision mix but is not affected by changes in the total volume of achievements. Annual average value-added per learner is indexed to 2012/13 and annual change figures are calculated. 

Limitations of the Skills Index

The Skills Index is not intended to be: 

  • A full assessment of the total value generated by the FE system. Training also delivers economic value not captured by earnings returns, such as increased profits to employers, benefits to the Exchequer in the form of greater tax revenue and lower welfare spending, and wider benefits to society rooted in greater and improved products and services.
  • A full assessment of the productivity impact over a learner’s lifespan. The Skills Index is based on the increase in annual earnings attributable to training over 3 to 5 years and does not capture the total increase in earnings over a lifetime.
  • A timely measure for evaluating specific policy changes. This is because it takes a minimum of six years after a learner completes training before we can start estimating the earnings returns of their qualification.
  • A method for tracking changes in the quality of qualifications delivered. The Skills Index monitors changes in the provision mix, i.e. the distribution of qualifications by level and subject. It does not currently track changes to the value of qualification levels and subjects, which are based on one set of returns estimates and effectively assumed to remain fixed over time.

Definitions

The key measures used in reporting on the Skills Index are defined as follows.

AchievementsThe number of learners who successfully complete a learning aim in an academic year.
Value-added per learnerThe value-added attributable to each learner that achieved a particular qualification type.
Total value-addedThe total increase in earnings generated by the FE system each year. 
Skills IndexAnnual total value-added is indexed to the estimate for 2012/13 in order to create the Skills Index.
Annual change in the Skills IndexThe percentage change in the Skills Index compared with the previous year.
Average value-added per learnerThe average value-added attributable to each learner that achieved a qualification.
Value-added per learner indexAnnual value-added per learner is indexed to the estimate for 2012/13 in order to create the value-added per learner index.
Annual change in the Value-added per learner indexThe percentage change in the Value-added per learner index compared with the previous year.
Index of achiever numbersAchiever numbers in scope of the estimates, indexed to 2012/13 in order to provide context to changes in the overall FE Skills Index and the Value-added per learner index.

Other definitions used in the Skills Index to classify achievements are as follows.

Provision typeIn this publication, Further Education achievements are classified into Apprenticeships and Classroom-based learning. Achievements in Community Learning provision are not in scope of the FE Skills Index.
Apprenticeships Apprenticeships are paid jobs that incorporate on-the-job and off-the-job training leading to nationally recognised qualifications. As an employee, apprentices earn as they learn and gain practical skills in the workplace. 
Classroom-based learningIn the FE Skills Index, adult further education that is not classed as an apprenticeship or community learning is described as classroom-based learning.
Sector Subject AreaOfqual's subject classification for Further Education qualifications.

Further details of inputs are set out in the following section. 

Data inputs

Achievement volumes 

  • The number of funded learners that achieved qualifications in each academic year is derived from Individualised Learner Record (ILR) data.
  • Records for all ages Apprenticeship achievers are combined with those for funded adult (19+) achievers on classroom-based learning, excluding Community Learning. 
  • Learners are then counted only once each year at their highest level of achievement.
  • Headline numbers of achievements by provision type and level are regularly published on gov.uk while the specific numbers used for the Skills Index are included in the annual publication.
  • The totals used in the Skills Index span the achievements reported in the annual Further Education and Skills and Apprenticeships and Traineeships statistics, with some differences.
  • The apprenticeship achievement figures in the Apprenticeships and Traineeships publication count the number of aims achieved rather than the number of unique achievers used in the Skills Index.
  • The Further Education and Skills learner achievement volume figures include Community Learning and learning at an unknown level, both of which are excluded from the Skills Index.

Employment rates  

  • The proportions of learners expected to be employed after achieving their qualification are  based on the sustained employment rate measures from DfE’s Outcome-Based Success Measures statistics.
  • The Skills Index uses the OBSM rates for the 2013/14 cohort as reported in the 2017/18 publication, applying the specific rates for the provision type, level and, where relevant Sector Subject Area.
  • Static rates are used to ensure the Skills Index reflects changes in the value of FE over time rather than short-term changes to economic conditions.
  • Details of the values used are set out in the following section.

Expected percentage earnings returns

  • The percentage increase in earnings of achievers relative to non-achievers, is derived from a regression analysis using data from the Longitudinal Education Outcomes dataset.
  • These are static values calculated from LEO data describing learners who exited learning between academic years 2008/09 and 2013/14, with an average taken of returns over 3 to 5 years after the qualification.
  • Details of the values used are set out in the following section and the Annex provides an outline of the methods used to derive them.

Expected median earnings  

  • The median earnings for achievers of the relevant qualifications, derived from the same Longitudinal Education Outcomes dataset used to calculate the percentage earnings returns.
  • These are therefore static values, based on learners exiting learning between academic years 2008/09 and 2013/14, with an average taken over 3 to 5 years after the qualification. 

Employment rates, median earnings and earnings returns

The following tables set out the inputs relating to each type and level of qualification and, where relevant, Sector Subject Area. Median earnings and value-added per learner are in 2016/17 prices. Further detail of the derivation of the median earnings and the earnings returns can be found in the Annex. 

Within the Sector Subject Area breakdowns, some subjects with very few achievements do not have an estimated value-added due to small sample sizes. As a result these achievements are not assigned a value in the calculation of total value-added.

Employment rates, median earnings and earning returns for classroom-based qualifications

Qualifications that were reclassified in 2016/17 from Full Level 2 to Other Level 2, or from Full Level 3 to Other Level 3 are still assigned the value-added per learner rates for Full Level 2 and Full Level 3 qualifications in all years. This is because earnings impacts were estimated using the classifications in use prior to 2016/17. Further details can be found in the “Changes to presentation of reclassified Full Level 2 and Full Level 3 qualifications” section.

Provision type and levelSector subject areaEmployment ratesMedian earnings (£s)Earning returnsValue-added per learner  (£s)
Classroom-based –Below level 2All49% 13,3305% 300
Classroom-based – Other Level 2All61% 15,610 1% 120
Classroom-based – Full Level 2Health, Public Services and Care84% 14,190 14% 1,470
Agriculture, Horticulture and Animal Care69% 19,100 14% 1,610
Engineering and Manufacturing Technologies74% 20,520 6% 860
Construction, Planning and the Built Environment81% 24,510 16% 2,720
Information and Communication Technology (ICT)62% 16,430 4% 390
Retail and Commercial Enterprise60% 15,780 13% 1,090
Leisure, Travel and Tourism78% 16,120 5% 600
Arts, Media and Publishing53% 13,360 11% 700
Education and Training73% 10,150 7% 480
Preparation for Life and Work75% 15,050 10% 1,030
Business, Administration, Finance and Law71% 19,150 8% 1,010
Classroom-based – Other Level 3All79% 16,550 3% 420
Classroom-based – Full Level 3Health, Public Services and Care72% 14,440 13% 1,200
Science and Mathematics53% 14,080 11% 740
Agriculture, Horticulture and Animal Care69% 17,430 18% 1,850
Engineering and Manufacturing Technologies73% 27,560 12% 2,160
Construction, Planning and the Built Environment74% 28,280 19% 3,340
Information and Communication Technology (ICT)46% 17,070 14% 970
Retail and Commercial Enterprise67% 17,620 15% 1,540
Leisure, Travel and Tourism66% 16,7208% 820
Arts, Media and Publishing50% 12,920 12% 700
History, Philosophy and Theology55% 12,900 12% 770
Education and Training80% 13,360 13% 1,230
Business, Administration, Finance and Law72% 20,89013% 1,720
Classroom-based – Level 4+All87% 21,58011% 1,870

Employment rates, median earnings and earning returns for apprenticeships

The earnings outcomes used in the Skills Index all relate to frameworks, even if the apprentice trained on an apprenticeship standard, because of the period the earnings returns relate to (see the Annex for details). Standards are of higher quality and have additional training hours compared to frameworks; this may lead to changes in value-added in future once specific outcomes data for standards comes through.

Provision type and levelSector subject areaEmployment ratesMedian earnings (£s)Earning returnsValue-added per learner (£s)
Apprenticeship – IntermediateHealth, Public Services and Care91% 14,050 18% 1,940
Agriculture, Horticulture and Animal Care85% 16,500 13% 1,610
Engineering and Manufacturing Technologies92% 21,050 22% 3,480
Construction, Planning and the Built Environment86% 18,770 20% 2,690
Information and Communication Technology (ICT)91% 20,740 27% 4,020
Retail and Commercial Enterprise88% 14,480 12% 1,370
Leisure, Travel and Tourism86% 15,790 5% 650
Business, Administration, Finance and Law91% 17,140 11% 1,550
Apprenticeship – AdvancedHealth, Public Services and Care92% 14,010 10% 1,180
Agriculture, Horticulture and Animal Care88% 19,520 15% 2,240
Engineering and Manufacturing Technologies92% 27,680 37% 6,910
Construction, Planning and the Built Environment91% 24,460 19% 3,560
Information and Communication Technology (ICT)89% 20,300 28% 3,940
Retail and Commercial Enterprise90% 16,300 9% 1,210
Leisure, Travel and Tourism81% 17,900 7% 950
Business, Administration, Finance and Law92% 19,710 11% 1,800
Apprenticeship – HigherAll93% 23,84023% 4,160

Changes to presentation of reclassified Full Level 2 and Full Level 3 qualifications

In Further Education Skills Index: 2020 to 2021, the presentation of reclassified Full Level 2 and Full Level 3 qualifications was changed but not the value assigned to these qualifications.

  • In 2016/17 a number of Full Level 2 and Full Level 3 qualifications were reclassified by the ESFA for the 19-23 entitlement, to align with the 16-19 offer and recommendations in the Wolf Review of Vocational Education.
  • In previous editions of the Skills Index, these qualifications were still included according to their original classification (i.e. Full Level 2 and Full Level 3) even after 2015/16. This was consistent with the added-value estimates applied in the Skills Index, which were unchanged.
  • From the 2020 to 2021 edition, to be coherent with other Official and National Statistics reporting on Further Education achievements, the reclassified qualifications were counted against their new categories (i.e. Other Level 2 and Other Level 3) and the breakdowns back to 2016/17 were revised.
  • The reclassified qualifications are still assigned the same value-added as Full Level 2 and Full Level 3 qualifications, using the same Sector Subject Area level estimates, because the earnings impacts of all qualifications were estimated using the classifications in use prior to 2016/17. 
  • This change therefore has no impact on the overall Skills Index and Value-added per Learner outputs, but it does mean the breakdowns are presented differently. 

The annual contribution of the reclassified qualifications to total value added is shown below. These contributions are included within the Other Level 2 and Other Level 3 values presented in the annual publication. 

 2016/172017/182018/192019/202020/21
Previously Full Level 215%17%20%17%16%
Previously Full Level 30%0%4%4%4%

 

Annexes

Methodology for estimating earnings returns

Background

The methodology for estimating earnings returns in FE was established and refined by a series of research projects led by the University of Westminster during 2012 through 2015. The estimates used in previous editions of the Skills Index were taken from:

These earnings returns have now been updated internally, applying the same modelling methodology defined by Bibby et. al. (2014) to a more recent set of learners. The latest data spans those who had completed courses between academic years 2008/09 and 2013/14 and their earnings up to the financial year 2016/17. 

Overview of returns estimation methodology

The methodology established by Bibby et. al. (2014) to estimate earnings returns uses a multiple regression model to isolate the effect on earnings for those who achieve a qualification, compared with those who start but do not achieve. The regression model controls for other observable characteristics and is run to estimate effects on (logged) earnings in each of years 3, 4 and 5 after completion, which are then averaged to provide a final value.

The controls used are: sex; age; region; ethnicity; Index of Multiple Deprivation (IMD); prior attainment, duration of study; number of previous FE learning spells; sector subject area; the number of days an individual was on active benefits in the year before learning; whether an individual has an inactive benefit spell in the year before learning; number of days in sustained (6 months) employment an individual has just before learning. OLASS learning and academic qualifications are also excluded from the dataset.

The estimates are presented as percentage increase in earnings that occur as a result of achieving a qualification. This is an average effect across those achieving a qualification at that level for the first time, and those who may have retrained or extended their knowledge at the same level.

This returns estimation approach has been validated through both the detailed work in the original research programme and in subsequent research. Chapter 6 of Bibby et. al. (2014) provides an assessment of the robustness of the achiever/non-achiever comparison to derive the returns. In “Settling the counterfactual debate”, the Centre for Vocational Education Research (CVER) has also compared this counterfactual against a method looking at learners in possession of qualifications at the level below, including the impact on earnings differentials.

Updated administrative dataset

The multiple regression modelling uses a dataset constructed by joining a longitudinal view of the Individualised Leaner Record (ILR) to relevant data from the Longitudinal Educational Outcomes (LEO) study. 

  • Learning activity recorded on the ILR is combined into “spells”, so that a learner’s full activity can be connected (including across summer holidays) to allow identification of the highest and latest learning aim for use in the regression. 
  • Data on earnings, employment and benefits is then taken from LEO data to derive a range of outcome measures at yearly intervals after the completion of that highest and latest learning aim. Within this, earnings are converted to constant prices (2017 in the current dataset).

The original research by Bibby et. al. (2014) pre-dated the creation of the full LEO methodology and a significant part of the research was focussed on the construction of a prototype database. This linked ILR with the same benefit information (from Department for Work and Pensions data) and PAYE employment and earnings histories (from HMRC data) that are now used in LEO. This dataset spanned learners completing during 2004/05 to 2008/09, with earnings up to the financial year 2011/12.

To produce updated estimates we have refreshed the dataset construction process, to make use of the LEO datasets. We have also used more recent data spanning learners who had completed courses between academic years 2008/09 and 2013/14 and their earnings up to the financial year 2016/17. 

Other changes implemented as part of the refreshed dataset construction approach include: extending the age coverage of the apprenticeships cohort to include 16-18 year olds; including qualifications at Level 4 and 5 for the first time; refining the method for defining learning spells; and (via the now established LEO process) improved linkage between education, and benefits and tax records, with improved data cleansing.

Changes in the earnings returns

The Skills Index uses returns estimates associated with different levels of learning and, where available, different Sector Subject Areas (SSA). 

In order to produce statistically significant returns estimates, the regression methodology requires a sufficiently large cohort of non-achievers and these are not available at detailed SSA level for all types and levels of courses. As a result, we only produce more detailed SSA Tier 1 estimates for Full Level 2 and Full Level 3 learning (for both Classroom-based and Apprenticeship courses). Estimates for Other Level 2 and 3 Classroom-based learning have not yet been updated, as further work is required on refining the data structures to properly isolate these qualifications within learning spells. For these types of courses the Skills Index still uses the 2014 estimates.

The below table shows how the updated estimates compare with the 2014 estimates, although not all values in the table are used to calculate the Skills Index. 2014 estimates for apprenticeships were based on adults aged 19+ whereas the updated estimates cover the whole 16+ apprenticeship cohort.  No estimates are available where cells are marked [x]. 

Provision typeLevel2014 estimateUpdated estimateDifference
Classroom-based (19+)Below level 22%5%+3ppt
Other Level 21%[x][x]
Full Level 211%9%- 2ppt
Other Level 33%[x][x]
Full Level 39%16%+7ppt
Level 4/5[x]11%[x]
Apprenticeships (16+)Intermediate11%14%+3ppt
Advanced16%17%+1ppt
Higher[x]23%[x]

The subject-level estimates from Cerqua et. al. (2015) were based on a subject classification different to Sector Subject Area, so the below table only shows estimates from the updated process. No estimates are available where cells are marked [x].

Sector Subject Area (Tier 1)Intermediate ApprenticeshipAdvanced ApprenticeshipClassroom-based Full Level 2Classroom-based Full Level 3
Health, Public Services and Care18%10%14%13%
Science and Mathematics[x][x][x]11%
Agriculture, Horticulture and Animal Care13%15%14%18%
Engineering and Manufacturing Technologies22%37%6%12%
Construction, Planning and the Built Environment20%19%16%19%
Information and Communication Technology (ICT)27%28%4%14%
Retail and Commercial Enterprise12%9%13%15%
Leisure, Travel and Tourism5%7%5%8%
Arts, Media and Publishing[x][x]11%12%
History, Philosophy and Theology[x][x][x]12%
Education and Training[x][x]7%13%
Preparation for Life and Work[x][x]10%[x]
Business, Administration, Finance and Law11%11%8%13%