BetaThis is a new service – your feedback will help us to improve it.
Methodology
Further education skills index
Published
Introduction
This guide sets out the details of the methodology and data sources for the ‘Further Education Skills Index’. It explains the concepts and methods used to calculate estimates of value added, and provides an overview of the data sources and other relevant information.
To assess the full impact of the FE system, DfE periodically produces full Net Present Value estimates compliant with the HM Treasury Green Book. The Further Education Skills Index adds to this by providing a simpler, more tractable measure that we can use to monitor changes in the value-added of the FE system over time. Though the rate of productivity growth is influenced by a number of factors, a country’s skills level is a major component, as giving people valuable knowledge, skills and behaviours boosts their productivity - see UK Skills and productivity in an international context, BIS (2015).
The Further Education Skills Index takes the increases in earnings attributable to different types of FE training and aggregates these to estimate a total impact on productivity. This assumes that in a well-functioning labour market, an individual’s earnings reflect their productivity. This approach is well-established in academia and public policy analysis. See, for example, Becker (1975) and Mincer (1974), and HMT (2018), The Green Book.
The Skills Index covers funded adult Further Education achievers and all apprenticeship achievers in England. The estimation process takes into account the provision type, level and sector subject area of the qualification achieved. Training funded through the Adult Community Learning budget is not included. The Skills Index starts in 2012/13, which is the base year (=100) for all indices included.
Calculation of value-added for different achievement groups
Value-added is calculated separately for each provision type, level and sector subject area by multiplying together estimates of:
The number of learners that achieved qualifications in each academic year. Where learners achieved more than one qualification, their highest and latest qualification is taken.
The proportion of learners expected to be employed after achieving their qualification. Static values are used to ensure the Skills Index reflects changes in the value of FE over time rather than short-term changes in economic conditions.
The expected median real earnings for employed achievers. Static values consistent with the expected percentage earnings returns.
The expected percentage earnings returns associated with achieving a qualification, as a percentage increase relative to non-achievers. These are static values, calculated at Sector Subject Area Tier 1 where possible and at provision/level type where a SSA breakdown cannot be produced. See Annex for details of derivation.
Calculation of the main Skills Index
The value-added totals for each sub-group are added together to create the total value-added for the entire FE system for the year. The total value-added therefore reflects changes in the number of achievers and the provision mix over time. Annual total value-added is indexed to 2012/13 and an annual change figures are calculated. Some subjects with very few achievements do not have an estimated value due to small sample sizes and are not included in the calculation of total value-added.
In simple terms, the value-added is essentially measured as:
Value-added = FE achievers * employment rate for achievers in that qualification * estimated additional earnings from achieving the qualification
As the employment rate and estimated additional earnings are static values, an increase (or decrease) in the Skills Index would be caused by one or more of:
An increase (decrease) in the number of learners;
An increase (decrease) in achievement rates;
A shift towards (away from) more economically valuable training, through more (less) learning being undertaken in qualifications with higher additional earnings.
Calculation of the value-added per learner index
The total value-added is divided by the total number of achievers for the year, to determine average value-added per learner. This measure reflects changes in the provision mix but is not affected by changes in the total volume of achievements. Annual average value-added per learner is indexed to 2012/13 and annual change figures are calculated.
Limitations of the Skills Index
The Skills Index is not intended to be:
A full assessment of the total value generated by the FE system. Training also delivers economic value not captured by earnings returns, such as increased profits to employers, benefits to the Exchequer in the form of greater tax revenue and lower welfare spending, and wider benefits to society rooted in greater and improved products and services.
A full assessment of the productivity impact over a learner’s lifespan. The Skills Index is based on the increase in annual earnings attributable to training over 3 to 5 years and does not capture the total increase in earnings over a lifetime.
A timely measure for evaluating specific policy changes. This is because it takes a minimum of six years after a learner completes training before we can start estimating the earnings returns of their qualification.
A method for tracking changes in the quality of qualifications delivered. The Skills Index monitors changes in the provision mix, i.e. the distribution of qualifications by level and subject. It does not currently track changes to the value of qualification levels and subjects, which are based on one set of returns estimates and effectively assumed to remain fixed over time.
The key measures used in reporting on the Skills Index are defined as follows.
Achievers
The number of learners who successfully complete a learning aim in an academic year.
Value-added per learner
The value-added attributable to each learner that achieved a particular qualification type.
Total value-added
The total increase in earnings generated by the FE system each year.
Skills Index
Annual total value-added is indexed to the estimate for 2012/13 in order to create the Skills Index.
Annual change in the Skills Index
The percentage change in the Skills Index compared with the previous year.
Average value-added per learner
The average value-added attributable to each learner that achieved a qualification.
Value-added per learner index
Annual value-added per learner is indexed to the estimate for 2012/13 in order to create the value-added per learner index.
Annual change in the Value-added per learner index
The percentage change in the Value-added per learner index compared with the previous year.
Index of achiever numbers
Achiever numbers in scope of the estimates, indexed to 2012/13 in order to provide context to changes in the overall FE Skills Index and the Value-added per learner index.
Other definitions used in the Skills Index to classify achievements are as follows.
Provision type
In this publication, Further Education achievements are classified into Apprenticeships and Classroom-based learning. Achievements in Community Learning provision are not in scope of the FE Skills Index.
Apprenticeships
Apprenticeships are paid jobs that incorporate on-the-job and off-the-job training leading to nationally recognised qualifications. As an employee, apprentices earn as they learn and gain practical skills in the workplace.
Classroom-based learning
In the FE Skills Index, adult further education that is not classed as an apprenticeship or community learning is described as classroom-based learning.
Sector Subject Area
Ofqual's subject classification for Further Education qualifications.
Further details of inputs are set out in the following section.
The number of funded learners that achieved qualifications in each academic year is derived from Individualised Learner Record (ILR) data.
Records for all ages Apprenticeship achievers are combined with those for funded adult (19+) achievers on classroom-based learning, excluding Community Learning.
Learners are then counted only once each year at their highest level of achievement.
Headline numbers of achievements by provision type and level are regularly published on gov.uk while the specific numbers used for the Skills Index are included in the annual publication.
The apprenticeship achievement figures in the Apprenticeships and Traineeships publication count the number of aims achieved rather than the number of unique achievers used in the Skills Index.
The Further Education and Skills learner achievement volume figures include Community Learning and learning at an unknown level, both of which are excluded from the Skills Index.
Employment rates
The proportions of learners expected to be employed after achieving their qualification are based on the sustained employment rate measures from DfE’s Outcome-Based Success Measures statistics.
The Skills Index uses the OBSM rates for the 2013/14 cohort as reported in the 2017/18 publication, applying the specific rates for the provision type, level and, where relevant Sector Subject Area.
Static rates are used to ensure the Skills Index reflects changes in the value of FE over time rather than short-term changes to economic conditions.
Details of the values used are set out in the following section.
Expected percentage earnings returns
The percentage increase in earnings of achievers relative to non-achievers, is derived from a regression analysis using data from the Longitudinal Education Outcomes dataset.
These are static values calculated from LEO data describing learners who exited learning between academic years 2008/09 and 2013/14, with an average taken of returns over 3 to 5 years after the qualification.
Details of the values used are set out in the following section and the Annex provides an outline of the methods used to derive them.
Expected median earnings
The median earnings for achievers of the relevant qualifications, derived from the same Longitudinal Education Outcomes dataset used to calculate the percentage earnings returns.
These are therefore static values, based on learners exiting learning between academic years 2008/09 and 2013/14, with an average taken over 3 to 5 years after the qualification.
Employment rates, median earnings and earnings returns
The following tables set out the inputs relating to each type and level of qualification and, where relevant, Sector Subject Area. Median earnings and value-added per learner are in 2016/17 prices. Further detail of the derivation of the median earnings and the earnings returns can be found in the Annex.
Within the Sector Subject Area breakdowns, some subjects with very few achievements do not have an estimated value-added due to small sample sizes. As a result these achievements are not assigned a value in the calculation of total value-added.
Employment rates, median earnings and earning returns for classroom-based qualifications
Qualifications that were reclassified in 2016/17 from Full Level 2 to Other Level 2, or from Full Level 3 to Other Level 3 are still assigned the value-added per learner rates for Full Level 2 and Full Level 3 qualifications in all years. This is because earnings impacts were estimated using the classifications in use prior to 2016/17. Further details can be found in the “Changes to presentation of reclassified Full Level 2 and Full Level 3 qualifications” section.
Provision type and level
Sector subject area
Employment rates
Median earnings (£s)
Earning returns
Value-added per learner (£s)
Classroom-based –Below level 2
All
49%
13,330
5%
300
Classroom-based – Other Level 2
All
61%
15,610
1%
120
Classroom-based – Full Level 2
Health, Public Services and Care
84%
14,190
14%
1,470
Agriculture, Horticulture and Animal Care
69%
19,100
14%
1,610
Engineering and Manufacturing Technologies
74%
20,520
6%
860
Construction, Planning and the Built Environment
81%
24,510
16%
2,720
Information and Communication Technology (ICT)
62%
16,430
4%
390
Retail and Commercial Enterprise
60%
15,780
13%
1,090
Leisure, Travel and Tourism
78%
16,120
5%
600
Arts, Media and Publishing
53%
13,360
11%
700
Education and Training
73%
10,150
7%
480
Preparation for Life and Work
75%
15,050
10%
1,030
Business, Administration, Finance and Law
71%
19,150
8%
1,010
Classroom-based – Other Level 3
All
79%
16,550
3%
420
Classroom-based – Full Level 3
Health, Public Services and Care
72%
14,440
13%
1,200
Science and Mathematics
53%
14,080
11%
740
Agriculture, Horticulture and Animal Care
69%
17,430
18%
1,850
Engineering and Manufacturing Technologies
73%
27,560
12%
2,160
Construction, Planning and the Built Environment
74%
28,280
19%
3,340
Information and Communication Technology (ICT)
46%
17,070
14%
970
Retail and Commercial Enterprise
67%
17,620
15%
1,540
Leisure, Travel and Tourism
66%
16,720
8%
820
Arts, Media and Publishing
50%
12,920
12%
700
History, Philosophy and Theology
55%
12,900
12%
770
Education and Training
80%
13,360
13%
1,230
Business, Administration, Finance and Law
72%
20,890
13%
1,720
Classroom-based – Level 4+
All
87%
21,580
11%
1,870
Employment rates, median earnings and earning returns for apprenticeships
The earnings outcomes used in the Skills Index all relate to frameworks, even if the apprentice trained on an apprenticeship standard, because of the period the earnings returns relate to (see the Annex for details). Standards are of higher quality and have additional training hours compared to frameworks; this may lead to changes in value-added in future once specific outcomes data for standards comes through.
Changes to presentation of reclassified Full Level 2 and Full Level 3 qualifications
In Further Education Skills Index: 2020 to 2021, the presentation of reclassified Full Level 2 and Full Level 3 qualifications was changed but not the value assigned to these qualifications.
In 2016/17 a number of Full Level 2 and Full Level 3 qualifications were reclassified by the ESFA for the 19-23 entitlement, to align with the 16-19 offer and recommendations in the Wolf Review of Vocational Education.
In previous editions of the Skills Index, these qualifications were still included according to their original classification (i.e. Full Level 2 and Full Level 3) even after 2015/16. This was consistent with the added-value estimates applied in the Skills Index, which were unchanged.
From the 2020 to 2021 edition, to be coherent with other Official and National Statistics reporting on Further Education achievements, the reclassified qualifications were counted against their new categories (i.e. Other Level 2 and Other Level 3) and the breakdowns back to 2016/17 were revised.
The reclassified qualifications are still assigned the same value-added as Full Level 2 and Full Level 3 qualifications, using the same Sector Subject Area level estimates, because the earnings impacts of all qualifications were estimated using the classifications in use prior to 2016/17.
This change therefore has no impact on the overall Skills Index and Value-added per Learner outputs, but it does mean the breakdowns are presented differently.
The annual contribution of the reclassified qualifications to total value added is shown below. These contributions are included within the Other Level 2 and Other Level 3 values presented in the annual publication.
The methodology for estimating earnings returns in FE was established and refined by a series of research projects led by the University of Westminster during 2012 through 2015. The estimates used in previous editions of the Skills Index were taken from:
These earnings returns have now been updated internally, applying the same modelling methodology defined by Bibby et. al. (2014) to a more recent set of learners. The latest data spans those who had completed courses between academic years 2008/09 and 2013/14 and their earnings up to the financial year 2016/17.
Overview of returns estimation methodology
The methodology established by Bibby et. al. (2014) to estimate earnings returns uses a multiple regression model to isolate the effect on earnings for those who achieve a qualification, compared with those who start but do not achieve. The regression model controls for other observable characteristics and is run to estimate effects on (logged) earnings in each of years 3, 4 and 5 after completion, which are then averaged to provide a final value.
The controls used are: sex; age; region; ethnicity; Index of Multiple Deprivation (IMD); prior attainment, duration of study; number of previous FE learning spells; sector subject area; the number of days an individual was on active benefits in the year before learning; whether an individual has an inactive benefit spell in the year before learning; number of days in sustained (6 months) employment an individual has just before learning. OLASS learning and academic qualifications are also excluded from the dataset.
The estimates are presented as percentage increase in earnings that occur as a result of achieving a qualification. This is an average effect across those achieving a qualification at that level for the first time, and those who may have retrained or extended their knowledge at the same level.
This returns estimation approach has been validated through both the detailed work in the original research programme and in subsequent research. Chapter 6 of Bibby et. al. (2014) provides an assessment of the robustness of the achiever/non-achiever comparison to derive the returns. In “Settling the counterfactual debate”, the Centre for Vocational Education Research (CVER) has also compared this counterfactual against a method looking at learners in possession of qualifications at the level below, including the impact on earnings differentials.
Updated administrative dataset
The multiple regression modelling uses a dataset constructed by joining a longitudinal view of the Individualised Leaner Record (ILR) to relevant data from the Longitudinal Educational Outcomes (LEO) study.
Learning activity recorded on the ILR is combined into “spells”, so that a learner’s full activity can be connected (including across summer holidays) to allow identification of the highest and latest learning aim for use in the regression.
Data on earnings, employment and benefits is then taken from LEO data to derive a range of outcome measures at yearly intervals after the completion of that highest and latest learning aim. Within this, earnings are converted to constant prices (2017 in the current dataset).
The original research by Bibby et. al. (2014) pre-dated the creation of the full LEO methodology and a significant part of the research was focussed on the construction of a prototype database. This linked ILR with the same benefit information (from Department for Work and Pensions data) and PAYE employment and earnings histories (from HMRC data) that are now used in LEO. This dataset spanned learners completing during 2004/05 to 2008/09, with earnings up to the financial year 2011/12.
To produce updated estimates we have refreshed the dataset construction process, to make use of the LEO datasets. We have also used more recent data spanning learners who had completed courses between academic years 2008/09 and 2013/14 and their earnings up to the financial year 2016/17.
Other changes implemented as part of the refreshed dataset construction approach include: extending the age coverage of the apprenticeships cohort to include 16-18 year olds; including qualifications at Level 4 and 5 for the first time; refining the method for defining learning spells; and (via the now established LEO process) improved linkage between education, and benefits and tax records, with improved data cleansing.
Changes in the earnings returns
The Skills Index uses returns estimates associated with different levels of learning and, where available, different Sector Subject Areas (SSA).
In order to produce statistically significant returns estimates, the regression methodology requires a sufficiently large cohort of non-achievers and these are not available at detailed SSA level for all types and levels of courses. As a result, we only produce more detailed SSA Tier 1 estimates for Full Level 2 and Full Level 3 learning (for both Classroom-based and Apprenticeship courses). Estimates for Other Level 2 and 3 Classroom-based learning have not yet been updated, as further work is required on refining the data structures to properly isolate these qualifications within learning spells. For these types of courses the Skills Index still uses the 2014 estimates.
The below table shows how the updated estimates compare with the 2014 estimates, although not all values in the table are used to calculate the Skills Index. 2014 estimates for apprenticeships were based on adults aged 19+ whereas the updated estimates cover the whole 16+ apprenticeship cohort. No estimates are available where cells are marked [x].
Provision type
Level
2014 estimate
Updated estimate
Difference
Classroom-based (19+)
Below level 2
2%
5%
+3ppt
Other Level 2
1%
[x]
[x]
Full Level 2
11%
9%
- 2ppt
Other Level 3
3%
[x]
[x]
Full Level 3
9%
16%
+7ppt
Level 4/5
[x]
11%
[x]
Apprenticeships (16+)
Intermediate
11%
14%
+3ppt
Advanced
16%
17%
+1ppt
Higher
[x]
23%
[x]
The subject-level estimates from Cerqua et. al. (2015) were based on a subject classification different to Sector Subject Area, so the below table only shows estimates from the updated process. No estimates are available where cells are marked [x].
