Methodology

A level and other 16 to 18 results

• Published: 1 February 2024
• Last updated: 2 February 2023
  See all notes (3)

  1. 2 February 2023

     Updated with revised data for the 2021/22 academic year.

  2. 26 May 2022

     Updated to accompany the 2021 retention data update to 'A level and other 16 to 18 results'. Extends impact analysis of the methodology change to cover 2020/21 retention data

  3. 4 November 2021

     Updated to accompany the 2021 update to 'A level and other 16 to 18 results'. Describes methodology change from 2020/21 determining when a student has reached the end of 16 to 18 study.

Introduction

This document provides an overview of the data used in the production of the ‘A level and other 16 to 18 results’ statistical publication which complements the 16-18 school and college performance tables. 

It outlines the main performance data included in the statistical release (with a link to the more comprehensive description of methodology in the separate technical guide for accountability headline measures). 

Additionally it also provides information on the data sources, coverage, quality, and on how the data is validated and processed. This document is based on the Office for National Statistics’ guidelines for measuring statistical quality.

Changes introduced in the 2022/23 release

Deciding when a student is at the end of 16-18 study

Students are eligible to be included in 16 to 18 performance measures if they are of academic age 16, 17, 18 at the start of the 2022/23 academic year (31 August 2022) and at the end of their 16-18 studies. They are identified as being at the end of 16-18 study when they first meet at least one of the following criteria:

• they have entered for at least 2 qualifications, each of which is at least the size of an A level or they have entered for at least 1 qualification the size of at least 2 A levels, in the reporting year; OR 
• they have entered for both a T Level core and T Level occupational specialism in any year during 16-18 study (T Level attainment will not be included in performance data until 2023/24, but this rule is applied a year early to produce a normal cohort when reporting begins), OR
• they are 18 at the start of the reporting year and have not been reported in 16 to 18 performance measures at their current allocated school or college.

The new rule specific to T Level component qualifications is to ensure T Level students are ‘triggered’ for reporting in the most appropriate reporting year.

Schools or colleges will still be able to defer students who meet criteria a) or b) but who are continuing their 16-18 study, as part of the checking exercise, as long as the student has not reached age 18 before the start of the reporting year. 

Note, a student eligible for reporting in 16 to 18 performance measures will still also need to meet criteria for each performance measure to be included in that measure. For example, to be included in the average point score per A level entry expressed as a grade, the student must have been entered for at least an AS qualification.

Performance measure methodology

Otherwise, the methodologies for performance measures data in 2022/23 are unchanged from 2021/22.

Data for 2022/23 continues to exclude results of qualifications awarded between January 2020 and August 2021, including AS and A level qualifications taken as part of the extraordinary autumn 2021 examination series. 

This means that where students have taken three years to complete 16-18 study, and have also entered a qualification during their first year – the 2020/21 academic year – these results will not count towards 2022/23 performance measures.

Incorrectly included institutions

Two Higher Education Institutions (HEIs) that also deliver some 16-18 further education were included in the provisional data  for 2022/23 in error.  The impact on Local Authority and National figures of including these HEIs is minimal and is corrected in revised data.

Changes introduced in the 2021/22 release

Data excluded from 2021/2022 measures due to alternative assessment arrangements in 2020 and 2021

We have committed not to include results of qualifications achieved between January 2020 and August 2021 in school and college performance measures on Find Schools and College Performance Data website, given the alternative assessment arrangements used to award those grades.

For consistency, when calculating 16 to 18 performance measures for 2021/2022 in this statistical release, data will only include entries and grades from qualifications awarded in the 2021/2022 academic year, with the exception of AS and A level qualifications taken as part of the extraordinary autumn 2021 examination series which are also excluded.

However, results from qualifications routinely offered in the autumn of 2021 that would usually be approved for reporting, such as vocational and technical qualifications taken in autumn 2021, will be included even if some of the VTQ constituent unit-level grades were awarded between January 2020 and August 2021 via Centre Assessment Grade, Teacher Assessed Grade, or another process.

The table below shows the impact at national level of excluding these data, by comparing the number of students and headline attainment measures for each cohort.

The impact on A level student numbers is modest. It is greater for students entering vocational and technical qualifications (VTQs), who are more likely to enter qualifications over multiple years. It should be noted that students may have results from multiple years including 2021/22. In this case the student is included in this data but some of their results are excluded. 

The technical certificate cohort has been substantially impacted by this change, as students are often not triggered for reporting in the same year as they complete the qualification.

Smaller cohorts in 2021/22, do not necessarily translate into impacts on attainment. Nationally, the impact on A level Average Point Score (APS) is minimal (1/200th of a grade) with slightly larger changes for the level 3 vocational and technical qualifications. 

		A  level	Applied  general	Tech level	Technical certificate
Number of students	Published data	284,380	119,853	28,535	1,404
	Inc TAG and CAG	289,260	133,439	33,592	8,580
Average Point Score (APS)	Published data	38.87	31.98	30.56	5.88
	Inc TAG and CAG	38.82	31.38	29.84	5.76
Impact of the change	Number of students	-2%	-10%	-15%	-84%
	APS per entry	+0.06	+0.60	+0.72	+0.12

Checking exercise

The 2021/22 academic year saw the return of the school and college checking exercise, after this was suspended in the two previous years. This updated release is based on revised data, meaning that any approved result amendments requested have been applied, ‘late’ results from some Awarding Organisations added, and some students removed from a school or colleges results (for example because the student wasn’t on roll, or has not yet completed 16-18 study) or added (where flagged by school or college as having completed 16-18 study despite not being triggered).

 

Other changes for the 2021/22 release

English and maths progress measures will not be published in this statistical release until these measures return to Find school and college performance data website (likely 2024/25, as set out in 16 to 18 accountability headline measures: technical guide, due to the impact of excluding CAG/TAG grades on this performance measure).

Similarly, 16 to 18 value-added measures, which would rely on KS4 prior attainment including some data from summer 2020, continue not to be published in 2021/22. Value-added measures will return as soon as possible, which will be for the 2023/24 academic year at the earliest; further details are included in the technical guide.

Breakdowns of performance measures by prior attainment, usually published in Spring with the Retention update, have been brought forward to the provisional release along with other breakdowns by student characteristics. This is because we distinguish between the use of prior attainment data nationally, , and its use in calculating  value-added and progress measures that will be published for individual schools and colleges on Find School and College Performance Data.

SEN Provision contains additional breakdowns by SEN types for the first time in national level data at 16 to 18. 

Additional ‘bivariate’ data giving a breakdown of headline attainment and disadvantage status in combination with other characteristics are available for the first time. 

 

Changes continuing from the 2020/21 release

Characteristic breakdowns usually published in the revised update have been brought forward to the provisional release. This is a permanent change.

The rules for determining when students reach the end of their 16-18 study changed in 2020/21 and students can no longer be triggered solely due to spending two years in the same school or college. The impact of this change on 2020/21 data was to reduce the number of students in each cohort, with vocational and technical qualifications impacted most. In 2021/22, student numbers returned to pre-rule change levels  as expected.

Changes introduced in the 2020/21 release

Deciding when a student is at the end of 16 to 18 study

Students are included in this statistical release when they reach the end of 16 to 18 study. The so-called ‘trigger’ rules for deciding when this happens changed in 2020/21. From 2020/21 students are included in performance measures, if;

a)       They enter at least 2 qualifications, each of which is at the size of an A level, or they have entered at least 1 qualification the size of at least 2 A levels, in that year

OR

b)       They are 18 at the start of the year and have not previously been reported.

Previously there was a third way a student could be triggered for reporting, which was to attend (and so be ‘allocated’) to the same school or college for the last two years.

This two-year rule has been removed from 2020/21 to reduce the number of students being triggered too early. That had created a burden on schools and colleges to tell us about those students through a ‘checking exercise’, where students could be removed from the data in between provisional and revised versions of statistics (but reported the following year when the student was 18).

The impact is to reduce the number of students reported in accountability measures in 2020/21, but numbers will return to normal levels from 2021/22 (when the students no longer being triggered by the two-year rule in 2021/22 are offset by those students whose reporting was delayed in 2020/21).

The table below demonstrates the impact of the change in trigger rules by comparing provisional data for 2020/21 in headline attainment measures, and the English and maths progress measures for data produced with and without the change in trigger rules. 

2020/21		Potential 16-18  students	A level  APS per entry	Applied general APS per entry	Tech level APS per entry	Technical certificate APS per entry	English progress	maths progress
Number of students	with trigger change	479,879	271,072	85,832	20,213	3,308	53,524	76,844
	without trigger change	596,422	274,097	98,775	26,002	8,484	107,833	141,735
Performance measure	with trigger change	n/a	41.60	32.82	31.76	5.64	0.159	0.242
	without trigger change	n/a	41.53	31.91	30.56	5.74	0.422	0.392
Impact of trigger change	Number of students	-20%	-1%	-13%	-22%	-61%	-50%	-46%
	Performance measure	n/a	+0.07	+0.91	+1.20	-0.10	-0.26	-0.15

Other changes continuing from the 2019/20 release

Given the alternative process to award grades in 2020 and 2021 the government announced that it would continue not to publish any school or college level performance data based on grades awarded in 2021, as per 2020, nor would attainment data from either year be used by DfE or others to hold schools and colleges to account.

The data used in this statistical release continues to be adapted as a result:

• The checking exercise was cancelled; as such, statistics labelled as ‘provisional’ will not be updated to ‘revised’ data in January;
• However, characteristic breakdowns usually published in January have been brought forward to the provisional release;
• Level 3 value-added data continues not be created.
• The points scale for the English and maths progress measure changed in 2019/20 to reflect the change in GCSE grading from A&-G to 9-1 (see impact on data in Annex B: Derived variables)

 

Retention data update in May 

The update in May to include retention statistics in 2019/20 and 2020/21 in part uses ‘final’ data on examination entries and results; compared to ‘provisional’ data it includes both late results, and results from the exception Autumn series for A and AS levels the following November for both years (for students unhappy with their CAG and/or TAG grade awarded in summer 2020 and 2021 respectively). In the period 2015/16 to 2018/19 examination data was based on ‘revised’ data following the checking exercise.

Exam data in 2020/21 following the trigger change and prior attainment

In 2019/20 for the first time, and again in 2020/21, retention statistics include a breakdown based a student's prior attainment. Prior attainment is a measure of the average points attained per qualification at Key Stage 4. Simple discounting rules apply, with the best result in any subject used when calculating the Average Point Score (APS) at KS4. Any GCSE resits or additional qualifications gained during 16 to 18 are ignored. For the vast majority, prior attainment is between 0-9 reflecting the range of possible grades at GCSE (where 9 is the highest grade).

This data is used to classify students into groups based on whether the KS4 APS per entry was:                                           below grade 4, between grade 4 and below grade 7, and grade 7 and above.

The following table updates the analysis of the impact of the trigger change on the number of students entering exams at A level, applied general, tech level, and technical certificate students with final data,  also breaking that down by prior attainment, showing the impact of the trigger change is bigger for students with lower KS4 prior attainment.

Students entering exams in 2020/21 final data	KS4 prior	A level	Applied general	Tech level	Technical certificate
with trigger change	0 to < 4	9,239	19,673	5,696	2,168
	4 to < 7	178,451	66,227	14,409	1,028
	7+	76,167	2,300	451	20
	unknown	7,614	1,584	373	139
	Total	271,471	89,784	20,929	3,355
impact of trigger change	0 to < 4	-5.4%	-29.5%	-39.7%	-64.6%
	4 to < 7	-1.2%	-5.3%	-12.1%	-51.4%
	7+	-0.3%	-2.6%	-5.5%	-35.5%
	unknown	-1.3%	-10.1%	-15.0%	-35.3%
	Total	-1.1%	-11.9%	-21.8%	-60.5%

Impact of the trigger change on headline retention rates

The impact of the trigger change is to reduce the reported number of students in scope for headline retention across all exam cohorts in 2020/21, with the biggest impact on vocational and technical cohorts (VTQs).

The impact of the trigger changes is to increase retention rates slightly in level 3 cohorts (A level, applied general, tech level) than would have been seen otherwise. However, the impact on the level 2 technical certificate cohort is a large decrease in headline retention.

Students in scope for headline retention measure in  2020/21		A level	Applied general	Tech level	Technical certificate
with trigger change	in scope for measure	221,180	66,005	19,923	4,879
	% retained	94.1%	89.9%	90.6%	82.1%
impact of trigger change	in scope for measure	-1.8%	-15.9%	-26.7%	-50.5%
	% retained	+0.8ppts	+1.0ppts	+0.9ppts	-5.2ppts

 

Changes introduced in the 2019/20 release

• The school and college checking exercise was cancelled in 2020 and 2021. This means that whilst the statistics are labelled as ‘provisional’ there will no update with ‘revised’ data as would typically happen later in January following checking by schools and colleges. This release includes all characteristics breakdowns that are usually published in January’s ‘revised’ release.
• The department decided it is not appropriate to publish value-added measures. These are normally calculated by comparing a student’s actual results to a set of expected results produced by a model based on national averages. The difference between the estimated results and the actual results are used to calculate value-added data for an institution (students in an institution making more or less progress than expected). However, in 2020 and 2021 the vast majority of grades awarded were those submitted by schools and colleges. The difference between a result submitted by the centre to a result estimated by a model would have very little meaning. 
• In Autumn 2020 and 2021 there have been exception exam series for GCSE and A Level, to provide another opportunity for pupils who are unhappy with the grade given to them in the summer, and for pupils who were not able to have a grade awarded. The results of the A level series were published by the Joint Council for Qualifications (JCQ) in December, and the results from GCSE English and maths in January 2021. Given the relatively low take up of the autumn exams in 2020, attainment data from November 2020 was not updated in May to reflect these exams (when retention data for 2019/20 was published).
• Characteristic breakdowns usually published in January have been brought forward to the provisional release (and will continued to be published with provisional data in the future);
• The points scale for the English and maths progress measure changed in 2019/20 to reflect the change in GCSE grading from A&-G to 9-1 (see impact on data in Annex B: Derived variables)

Data sources

16-18 qualification entries and attainment 

Awarding organisations (AOs) deliver examination entries and results for all qualifications reported in this statistical publication and 16-18 performance tables for students at the age of 16-18. The AOs hold information on their qualifications, the students who have entered them and the results awarded. The department collects data from AOs each year via an external contractor.

The COVID-19 pandemic has not changed flow of data from AOs to contractor to the department; what has changed is how some 2020 and 2021 results in the AO data have been awarded.

Key stage 4 prior attainment

The department collects key stage 4 attainment for pupils at the end of key stage 4 each year, also from AOs. The information is used to calculate the English and maths progress measures, for those students who did not achieve 9-4 by the end of key stage 4. Most students at the end of 16-18 study have key stage 4 (KS4) prior attainment from two years earlier, although for a significant minority of students reported at age 18 it can come from 3 years earlier.

Student characteristics at the end of key stage 4

Student characteristics, such as ethnicity and free school meal eligibility are not routinely or consistently collected at key stage 5. This publication makes use of the characteristic information recorded for a pupil at the end of key stage 4[1].

Data is broken down by the following student characteristics: 

• disadvantage status; 
• free school meal (FSM) eligibility; 
• ethnicity; 
• first language; 
• special educational needs (SEN)
• KS4 prior attainment

Note, in order to be classed as disadvantaged a student must attend a state-funded institution (and so disadvantage breakdowns are only presented for state-funded institutions nationally in the statistical release).

More information on disadvantaged status at the end of key stage 4 can be found in the Quality and Methodology document for the revised GCSE and equivalent results publication.

[1] Key stage 4 data, from which student characteristics information is sourced, is compiled by the department from key stage 4 results, school census records and qualification entries and results collected from awarding bodies

Student level information 

Student information such as enrolment status and learning aim is collected through the department’s school census and the Individualised Learner Record (ILR) for state-funded schools and colleges. The school census is a statutory termly data collection, during which schools/local authorities (LAs) send the department details of the pupils on their roll. 

The following school types participate in the pupil-level school census:

• Local authority-maintained schools: community, foundation, voluntary-aided, voluntary-controlled schools, Pupil Referral Units (PRUs) and special schools (including hospital schools)
• Academies, including alternative provision (AP) academies
• Free schools, including AP free schools
• Studio schools
• University Technical Colleges (UTCs)
• Non-maintained special schools

Further details are available from the following link: school census guidance 

The ILR, an ongoing data collection conducted by the Education and Skills Funding Agency (ESFA), gathers data on learners and their learning from providers in the Further education (FE) and skills sector. The ESFA is also responsible for funding the provision of 16 to 19 education and receives relevant ILR funding returns. More information is available from the following link: ILR guidance

Independent school pupil data is taken from awarding organisation data rather than collected from student level returns.

Institution characteristics 

Information on school or college type and phase of education is taken from ‘Get Information about Schools’ (GIAS). This is a register of educational establishments providing compulsory, higher and further education in England and Wales, maintained by the Department for Education. More information is available in the GIAS guidance.

The school and college types that are reported in statistical publications are taken from their status at the start of the reporting year. Any schools or colleges converting to an academy before 12 September of that year are reported as an academy. Those that have converted on or after this date have been treated as their predecessor school or college type.

Information on where students studied

In the 16-18 statistical publication and performance tables, each student is assigned to a school or college where they are recorded as being on roll for the purpose of school or college funding each year. The examination results collected from awarding organisations provide information on which institution a student took exams at, not whether the student was also recorded as being on roll. In order to assign students, the spring school census is used for schools and, since 2016, the ILR for colleges. These tell us where students were on roll in each year of their 16-18 studies.

For the purposes of the 16-18 statistical publication and performance tables, the ESFA’s June ILR snapshot (SN10) is used to identify where students are on roll at a college. More detail on the ILR is available here.

 Where a student is not recorded on either the school census or the ILR, usually because they are attending an independent school, the awarding organisation data is used to assign them to a school. This is usually based on where the highest volume of a student’s exams were taken.

Condition of funding exemptions 

For pupils who did not achieve GCSE 9-4 or equivalent in English or maths by the end of key stage 4, English and maths attainment during 16-18 studies are calculated using key stage 4 and 16-18 attainment data submitted from awarding organisations. However, in order to align with the ESFA’s condition of funding rules, the department also uses funding data from the autumn school census and the ILR that identifies students who are exempt from the requirement to retake English and maths during 16-18 study. This includes students who have learning difficulties and/or disabilities and those who have overseas qualifications that are equivalent to GCSE grade 4 or above.

Student learning aims

The 16-18 performance tables use information on student 16-18 learning aims from the school census and ILR to calculate retention measures. Retention measures are typically published in March due to when data is available (compared to exam results from Awarding Organisations). However, in 2019/20 data was further delayed until May to co-ordinate with the availability of data from the Autumn 2020 exception exam series. Similarly, retention data for 2020/21 will also be delayed until May 2022.

How the statistics are created (and differences due to COVID-19 in 2019/20 and 2020/21)

The entries and awards for qualifications at key stage 4 and 16-18 are submitted by awarding organisations to an external contractor, who processes the data and completes a set of quality checks on behalf of the department. This includes checks to ensure coverage is as expected and the data does not contain inaccurate or missing values. 

Once the latest entries and results data has been submitted to the contractor in August, data is matched to students’ characteristics and students’ prior attainment information, to calculate performance measures for each student. These student level performance measures are then aggregated to give the overall results for England, with some measures broken down also by region and local authority. Several performance measures and other derived variables (see Annex B) are added during this process. This data is then usually used to produce the provisional statistical publication. As with 2020, the data for 2021 is being released in November; this partly reflects the timing of when data is available, and partly to allow more detailed data including characteristic breakdowns to be published earlier (these data are normally produced in ‘revised’ data in January).

Typically, the data is checked with schools and colleges during the 16-18 performance tables checking exercise, which runs between September and October, allowing schools and colleges to request amendments to their data. This could include adding any missing results, supplying information on grade amendments following successful reviews of results, or deferring students who are not yet at the end of 16-18 study. 

However, the checking process did not take place in 2019/20 nor 2020/21 as a consequence of the alternative process for awarding results, and following the announcements that 2020 and 2021 exam results would not be published at school or college level in accountability data, which included sharing through the checking site. Consequently, there was no publication of ‘revised’ data in January 2021 and January 2022  taking into account changes agreed through the checking. The checking processes returned in 2021/22, and revised’ data was published in February 2023.

Relevance, punctuality and timeliness

Relevance

This section describes the degree to which the statistics meet current and potential needs of the users.

Key strengths and limitations of the data

The key strength of the data is that it is derived from an administrative data collection and quality assured. This means that it can supply accurate data down to small geographical areas and school and college levels. In addition, the data is merged with other administrative data held by the department (the school census and ILR) to provide detailed information on sub-groups of the school/college population. 

However, the student counts for some groups, particularly at local authority level and some ethnicity categories can be small. Care should be taken when comparing performance measures for small groups. 

Punctuality

Punctuality refers to the time lag between the actual and planned dates of publication.

The proposed month of publication is announced on gov.uk at least twelve months in advance and precise dates are announced in the same place at least four weeks prior to publication. In the event of a change to the pre-announced release schedule, the change, and the reasons for will be announced.

Timeliness

Timeliness refers to the lapse of time between the period to which the data refers and the publication of the statistics.

The department quality assures the data, produces statistics, and publishes the provisional ‘A level and other 16 to 18 results in England’ statistical publication after receiving the provisional data from the contractor. This process normally takes around four weeks.

A more detailed production timetable is shown below (based on the awarding process in 2021/22):

May and June 

Main A level and AS level examinations are taken (Vocational and technical qualifications can be taken throughout the year)

July to September

The Joint Council for Qualifications (JCQ) collects results data from the awarding bodies, publishing data on A level results day in mid-August . There is no single ‘results day’ for VTQ qualifications.

The contractor matches the latest attainment data with other departmental data sources (school census and ILR)

October to December

The department receives unamended (provisional) attainment data from the contractor. 

The 16 to18 Autumn checking exercise takes place from early October based on unamended data for schools and colleges.

Amendment requests from school and colleges are scrutinised against a set of criteria following the checking exercise.

The department normally receives two sets of revised data during November and December, which take account of changes made during the checking exercise, and any late results.

The statistical release (A level and other 16 to 18 results) containing national and local authority level data broken down by characteristics is published based on unamended data (described as provisional)

January and February

Statistical release (A level and other 16 to 18 results) is updated based on revised data. School and college accountability data based on revised data is published on Find School and College Performance Data website.

The errata period commences for school and colleges. We share revised data with schools and colleges, and in exceptional circumstances some amendments may be requested to this data.

The autumn school census learning aims data becomes available. This is matched alongside learning aims data from an autumn ILR snapshot, in order to calculate retention measures.

March 

The statistical release is further updated to add data on retention measures (also based on revised data). School and college data on retention added to Find School and College Performance Data website.

 April

The department receives final (post errata) data from the contractor and updates Find School and College Performance Data.

Accuracy and reliability

Accuracy

Accuracy describes the closeness between an estimated result and the (unknown) true value.

Data coverage

The source data covers the eligible population for the statistical publication and 16-18 performance tables, as the department collects annual data on all results achieved by students. The eligible population for the qualification data collection is all 16 to 18 year old students who by the end of the reporting year completed 16-18 studies in schools and colleges in England. 

Note, this means the eligible population does not include all 16 to 18 year olds in England; students not studying in schools and colleges are excluded from the statistics, which will include: students studying through private training providers, most students enrolled in traineeship and apprenticeships, and young people in employment (unless also enrolled in a school/college).

Students who are not at the end of 16-18 studies, or are re-sitting exams, are excluded from the calculations. As these students cannot be identified centrally, the department applies a set of rules to identify students at the end of 16-18 study, namely:

1. they have entered for at least 2 qualifications, each of which is at least the size of an A level or they have entered for at least 1 qualification the size of at least 2 A levels, in the reporting year; 

OR 

1. they are 18 at the start of the reporting year and have not been reported in the performance tables at their current allocated school or college

During the September checking exercise, schools and colleges may apply for students to be removed from their figures for several reasons; for example, the student is not at the end of 16-18 study, if the student is an external candidate and not on roll, or if the student is deceased. This is normally reflected in updated statistics published in January.

Note, due to the cancellation of the checking exercise in 2019/20 and 2020/21, this did not happen for those cohorts of students.

Measurement error

Measurement error is the difference between the actual value of a quantity and the value obtained by a measurement. 

Examination results

Note: these steps just apply to results based on externally assessed exams (i.e this does not apply to AS/A level results in 2020 and 2021 that due to COVID-19 were awarded by an alternative process)

The following steps are taken to minimise measurement error in examination results.

Clear guidance is provided to schools and colleges regarding the administration of examinations, including instructions for keeping the exam materials secure prior to the exams and storage of completed scripts until they are collected for marking.

Once schools and colleges have completed 16-18 exams such as AS and A levels, they are required to send their exam materials and attendance registers to the awarding organisations for external marking and data capture.

The exam papers are externally marked by the awarding organisations to ensure that marking is consistent between institutions.Exam boards follow the principle that if the cohort of students taking a subject is similar to that of previous years, then the proportions of students at each grade will be similar. A key piece of evidence in determining if the cohort is the same is prior attainment at GCSE. 

After marking, results are returned to schools and colleges. Data is then passed from the awarding organisations to the department’s contractor for use in the statistical publications and 16-18 performance tables. If schools and colleges have concerns over the marking of an exam paper, they may request for it to be re-marked. Outcomes of reviews are not reflected in the provisional statistical publication data but are included in the revised data and performance tables. 

Further information on the general regulations for schools and colleges and how exams are conducted and marked can be found on the JCQ website: Joint Council for Qualifications (JCQ) website.

Qualifications

Ofqual sets the standards and rules that awarding organisations need to meet when they design, deliver and award regulated qualifications. Awarding organisations and qualifications are also monitored to maintain standards. OfQual are a risk-based regulator and concentrate on where resources are most needed. For example, Ofqual may discover that an awarding organisation’s processes need updating or that there are risks associated with a qualification taken by thousands of students every year. Further guidance on the role of Ofqual can be found at their website.

School census

The school census is a statutory data collection, which means that schools are obliged to provide the department with data on their pupils. School census returns are quality assured by the department to ensure that they contain data that is as accurate and complete as possible. The department acts to ensure the quality of this data in the following ways:

• Detailed technical guidance is provided to schools to support them in submitting full and accurate data to the department;
• Management Information System suppliers build validation into the systems that they provide to schools. The department provides guidance on the types of validation it expects for school census purposes;
• The department’s COLLECT web-based data collection tool automatically validates the data that schools and local authorities supply, providing error and query reports so that these can be investigated/resolved.

More information on the school census can be found on gov.uk.

Individualised learner record (ILR)

The ILR is managed by the Education and Skills Funding Agency (ESFA), who undertake various activities to ensure the data returned by colleges is accurate and high quality:

• Detailed technical guidance and specifications are published by the ESFA to support colleges in providing the data that they must return
• Data submitted through the web-based data collection system is validated to check for errors or issues. Detailed validation guidance is available to colleges.
• As part of assurance work, the data reported through the ILR is monitored by the ESFA to ensure it is high quality and meets their published funding rules. A sample of colleges are also selected for external funding audit, part of which checks that ILR returns accurately reflect the learners on roll and the learning they receive.

More information on the ILR can be found on gov.uk.

Validation and quality assurance of source data

The department applies a set of internal quality protocols to ensure the published data is accurate. First, the data production process is subject to a ‘dry run’ during the summer. This involves producing a dummy dataset, which conforms to how the current year’s data will be supplied (e.g. if new subjects have been introduced, then dummy data would be added to last year’s data to simulate these). This dummy dataset is used to test the contractor’s systems and the department’s checking processes. This allows potential problems to be resolved prior to receiving the live data.

In addition, at every stage in the ‘live’ data cycle, the department checks all calculations used in the production of the figures by independently dual-running the methodologies at institution, local authority and national levels. Further quality assurance is then carried out on the data produced for the statistical publication (see examples below). Any discrepancies in the data are discussed and resolved prior to publication.

Examples of additional checks Comparisons with previous figures to identify any large changes Comparisons with provisional data to identify any large changes (revised/final data only) Check totals are consistent across tables Check patterns in the data are as expected Check figures against those produced by our contractor for the performance tables Check the data format against specifications

Disclosure Control

The Code of Practice for Official Statistics requires us to take reasonable steps to ensure that our published or disseminated statistics protect confidentiality. We assess our statistics with reference to the National Statistician’s Guidance on Confidentiality of Official Statistics and guidance from the Information Commissioner’s Office (ICO) to establish the risk of disclosure and its associated impact and suppress the data accordingly. In our statistical releases the following conventions are applied: ‘c’ for confidential, ‘z’ for not applicable, ‘:’ for not available.

Reliability

Reliability is the extent to which an estimate changes over different versions of the same data.

Change between releases

The figures published in the statistical publication may change between provisional and revised releases; however there tends to be a slight change in the headline national performance measures rather than any significant fluctuation in the statistics. This is due to the combined effect of removals of students that should not be included and re-marking and submission of late results by awarding organisations. 

Changes over time 

In 2016, there were several changes to 16-18 attainment measures due to educational policy reforms. As such, there are difficulties with making comparisons to data published in 2015 or earlier in the headline performance measures. The impact of the 2016 reforms was assessed in the 2016 A level and other 16-18 results provisional release.

Comparability of the statistical publication and performance tables

The statistical publication and performance tables use the same data sources. Where they both reference the same performance measures, they use the same methodology for calculation. In addition, the statistical publication includes further A level and other examination results based on a single year of academic study. The statistical publication also includes further tables on English and maths progress for those students who did not achieve 9-4 or A*-C by the end of key stage 4. These use the same methodology as the performance tables measure.

 

Other statistics from the Department for Education

The statistics shown below are produced by the Department for Education but provide different information to that published in the 16-18 statistical publication and performance tables.

Destination measures	Figures for young people who went into education, employment or training destinations the year after they completed key stage 4 or key stage 5. The department publishes statistics including information from a data source which gives more complete coverage of participation in employment than the earlier ‘experimental’ destinations data.
Level 2 and 3 attainment at 16-18	Statistics on the attainment of young people aged 19, based on matched administrative data.
Participating in education, training and employment by 16 18 year olds in England	Estimates of participation in education, training and employment and those who are not in education, employment or training (NEET) for 16, 17 and 18 year olds in England.
Key stage 4	GCSE and equivalent results for key stage 4.
Key stage 2	National curriculum assessments and review outcomes at key stage 2 (KS2), including measures of progress between KS1 and KS2.
Key stage 1	National curriculum assessments at key stage 1 and phonics screening check results.

Joint Council for Qualifications (JCQ) and wider UK education data

Joint Council for Qualifications (JCQ)

The figures in the statistical publication and 16-18 performance tables differ from those published by JCQ on results day, whose figures relate to the outcome of all students in England, Wales and Northern Ireland in the latest academic year, regardless of their age. The examination results published in this statistical publication include the performance of students aged 16-18 taking examinations in England only. We also publish statistics that relate to students who are at the end of 16-18 study. This will include results of qualifications taken by these students in previous academic years.

Wider UK education data

Wales

The Welsh Government publishes the results of external examinations taken by students aged 15 or 17, available at Welsh government statistics and research 

Northern Ireland

The Department for Education Northern Ireland (DENI) publishes AS and A level headline statistics at Department for Education Northern Ireland (DENI)

Scotland

The publication, ‘Summary statistics for attainment, leaver destinations and healthy living’, is published by the Scottish Government and is available at The Scottish Government website

Other statistics from the Department for Education

The statistics shown below are produced by the Department for Education but provide different information to that published in the 16-18 statistical publication and performance tables.

Destination measures	Figures for young people who went into education, employment or training destinations the year after they completed key stage 4 or key stage 5. The department publishes statistics including information from a data source which gives more complete coverage of participation in employment than the earlier ‘experimental’ destinations data.
Level 2 and 3 attainment by young people aged 19	Statistics on the attainment of young people aged 19, based on matched administrative data.
Participating in education, training and employment	Estimates of participation in education, training and employment and those who are not in education, employment or training (NEET) for 16, 17 and 18 year olds in England.
Key stage 4	GCSE and equivalent results for key stage 4.
Key stage 2	National curriculum assessments and review outcomes at key stage 2 (KS2), including measures of progress between KS1 and KS2.
Key stage 1	National curriculum assessments at key stage 1 and phonics screening check results.

Got a query? Like to give feedback?

 

If from the media	Press Office News Desk, Department for Education, Sanctuary Buildings, Great Smith Street, London SW1P 3BT. 020 7783 8300
If non-media	Email: Attainment.STATISTICS@education.gov.uk

Annexes

Annex A: Further details on institution characteristics

This section provides further detail on the school/college characteristics.

School Admission Basis

Schools can be grouped based on whether they select their pupils by ability. State-funded school admission basis has historically been taken from ‘Get information about schools’ as per the following definitions:

Comprehensive schools	Takes all pupils, usually regardless of their ability, aptitude or whether they have been selected for a place at a selective school. Includes schools operating pupil banding admission arrangements.
Modern schools	Takes pupils regardless of their ability or aptitude, including those who have not been selected for a place at a local selective school.
Selective schools	Admits pupils wholly or mainly with reference to ability. These schools are formally designated as grammar schools.

However, this information is self-declared by each school and may not necessarily be a true reflection of a school’s admissions policy.

Institution Type

The institution types are taken from Get Information about Schools (GIAS) and are given at 12th September at the start of the academic year. They are defined as follows:

Academy Sponsor Led	Sponsored academies are all-ability, state-funded schools established and managed by sponsors from a wide range of backgrounds, including high performing schools and colleges, universities, individual philanthropists, businesses, the voluntary sector, and the faith communities.
Academy Converter	Schools that have chosen through Governing Body Resolution and application to the Secretary of State to become an academy under the Academies Act 2010.
Free School	Free schools are funded by the government but are not run by the local council. They have more control over how they do things. ‘All-ability’ schools, so cannot use academic selection processes. Free schools can set their own pay and conditions for staff, change the length of school terms and the school day and they do not have to follow the national curriculum.
Sixth form colleges	Administered through the ILR rather than the school census. Students aged 16-19 typically study for level 3 qualifications
Other Further Education (FE) sector colleges	Administered through the ILR rather than the school census. ‘Other FE sector colleges’ includes general FE, tertiary and specialist colleges; deliver range of full and part-time vocational and academic provision at all ages, and all levels.
Studio School	Studio schools are all-ability and mixed sex state funded schools, independent of LAs. An innovative model of educational provision, delivering mainstream qualifications through practical project-based learning. They are not extensions or conversions from existing provision, but are new 14-19 academies, typically with around 300 pupils.
University Technical College (UTC)	UTCs are all-ability and mixed sex state funded schools, independent of local authorities. UTCs specialise in subjects that need modern, technical, industry-standard equipment, such as engineering and construction, and teach these disciplines alongside business skills and a broad, general education. Pupils integrate academic study with practical learning, studying core GCSEs alongside technical qualifications.
City Technology College (CTC)	CTC’s are independent all-ability, non-fee-paying schools offering students and pupils the opportunity to study a curriculum that, with the help of private sector sponsors, is geared towards the world of work.
LA maintained school	Schools fully or partially under LA control that are state-funded, mainly by the Dedicated Schools Grant. These include community schools, foundation schools, voluntary aided school, voluntary controlled schools and LA maintained special schools.
Registered independent school	Any school which provides full time education for 5 or more students which is not state-funded or a non-state-funded special school.
Independent special school	Approved by the Secretary of State for Education. They are run on a not-for-profit basis by charitable trusts and normally cater for children with severe and/or low incidence special educational needs.
State-funded special schools	Includes LA maintained special schools, academy sponsor led special schools, academy special schools and special free schools.
Non-maintained special schools (NMSS)	NMSS are schools for children with special educational needs that the Secretary of State for Education has approved under section 342 of the Education Act 1996. They: teach students with special educational needs are independent of local authority control operate on a not-for-profit basis
Pupil referral unit (PRU)	A PRU is established and maintained by a local authority which is specially organised to provide education for children who are excluded, sick or otherwise unable to attend mainstream school and is not a special or other type of school
Alternative provision (AP)	Education arranged by local authorities for pupils who, because of exclusion, illness or other reasons, would not otherwise receive suitable education; education arranged by schools for pupils on a fixed period exclusion; and pupils being directed by schools to off-site provision to improve their behaviour.

In the statistical release institution types are commonly grouped together for reporting purposes as follows:

FE sector colleges	Includes sixth form colleges and ‘other FE sector colleges’
State-funded schools	LA maintained schools, academies, free schools, city technology colleges, studio schools, UTCs, and state-funded special schools (excluding hospital schools, PRUs, APs, and independent schools).
State-funded institutions	Includes state-funded schools and FE sector colleges
All independent schools	Includes independent schools and independent special schools.
All schools	Includes all independent schools and state-funded institutions (including hospital schools, PRUs, and APs)
All institutions	Includes all schools and FE sector colleges
State-funded mainstream schools	Includes LA maintained mainstream schools, academies, free schools, city technology colleges, studio schools, and UTCs (excludes all special schools, pupil referral units, alternative provision and independent schools). These institution types are included in retention measures.

 

Annex B: Points, qualifications reported and discounting, and performance measures

Several derived variables are produced in response to changes in education policy and statistical methodology. The following derived variables are added to calculate student level results in the 16-18 statistical publication and performance tables.

Examination point score and size 

Points are allocated to each of a student’s 16-18 exam results using a scoring system. You can find full detail on point scores in the discounting guidance, and how they are calculated in the Practical guide. 

The point scores for level 3 qualification are assigned for each qualification by calculating GCE A level size equivalence using the Guided Learning Hours (GLH) from the Ofqual register and the base points for each grade from the level and grade structure. 

This enables the calculation of average level 3 point scores for 16 to 18 year old students at the end of advanced level studies. 

New performance points were developed for the vocational qualifications at level 2, reported for the first time in 2017 16-18 performance tables. The performance points at level 2 have been designed to have the following properties: 

• as per level 3 performance points, larger qualifications attract more points (size is measured relative to 1 GCSE equivalent);
• differences in reported headline attainment measures (average point score per entry, average grade) make intuitive sense; for most qualifications an improvement of one grade throughout an institution would translate into an improvement of one unit (APS per entry) or one grade (average grade)

Point scores – English and maths progress measures

The English and maths progress measures use a distinct point score system, used to assign points to both a student’s prior attainment at key stage 4 and their attainment by the end of 16-18 study, for the purpose of calculating the progress they have made.

Following a review and consultation with stakeholders the points used 2017-19 have been updated for 2019/20 as follows:

• Increasing the relative progress points for attaining Functional Skills Level 1 and Level 2 qualifications so schools and colleges can further support students to take the level 2 qualification which is most appropriate for them.
• Reflecting the current 9-0 GCSE grading system and making the points consistent by moving from the old 8-point scale to the new 9-point scale, with each grade worth one point.
• Recognizing progress made between each entry level. Previously attainment of any entry level qualifications receives 0.4 progress points. From 2020, Entry Level 1 will receive 0.25, Entry Level 2 0.5 and Entry Level 3 0.75 points.

Left hand table: points used in 2016/17, 2017/18 and 2018/19

                                                                                                                                             Right hand table: points used since 2019/20

An impact analysis was carried out on the ‘technical’ impact of the change in the points table by re-creating provisional data from 2017/18 with the new 2019/20 points scheme [note, it is not appropriate carry out the analysis in retrospect, i.e. to re-run 2019/20 data with the 2017/18 points scheme; this is because in addition to the technical change, the revised points are expected to lead to behaviour change as the relative incentive to enter GCSE, reformed Functional Skills qualifications, and other qualifications is altered. This analysis is based on 2018 data published before the change in the points scheme was publicly announced].

• Overall the impact is small but increases the average progress in English and maths nationally by 0.05 and 0.02 points respectively;
• The impact on males and females is the same nationally, so the change in points is not expected to bias the estimates of the gender gap in progress in English and maths;
• Consistent impacts are seen for students with SEN vs. non-SEN, likewise for disadvantaged vs. non-disadvantaged students;
• The relative increase in points for L2 Functional Skills and distinguishing Entry level grades is expected to benefit FE sector colleges more than schools overall simply because more of these qualifications are taken in FE colleges, though the impact on the gap in progress is still small (narrowing it by 0.05 pts or 1/20th of a grade for maths).

	English progress			Maths
	October 2018	Re-run with 2020 points	Difference	October 2018	Re-run with 2020 points	Difference
England	0.08	0.13	+0.05	0.07	0.09	+0.02
Male	0.06	0.11	+0.05	0.06	0.08	+0.02
Female	0.13	0.17	+0.05	0.07	0.09	+0.02
All state-funded schools	0.40	0.43	+0.03	0.27	0.26	-0.02
FE sector colleges	-0.07	-0.01	+0.05	-0.08	-0.05	+0.03
SEN	-0.11	-0.06	+0.05	-0.03	0.01	+0.03
Non-SEN	0.17	0.23	+0.06	0.09	0.11	+0.03
Disadvantaged	-0.06	-0.01	+0.05	-0.06	-0.02	+0.03
Not disadvantaged	0.19	0.24	+0.05	0.15	0.17	+0.02

Qualifications reported

Level 3 Academic: Results reported only cover qualifications accredited by Ofqual and approved under Section 96 of the Learning and Skills Act (2000). Other qualifications or learning provisions such as Graded Exams or work-based learning are not included.

Technical and applied: From 2016 onwards, the level 3 vocational and technical qualifications that can count are restricted to those that are on the approved list of applied general or tech level qualifications. Lists are published on gov.uk. This change is the result of policy reforms to encourage schools and colleges to offer qualifications that help prepare students for further study or employment. The background to the reforms can be found in the Wolf review of vocational education. As a result, the ‘vocational’ cohort measures published in the statistical publication and performance tables have been broken down into the ‘applied general’ and ‘tech level’ cohorts, reflecting the approved qualifications included in these categories.

Below level 3 English and maths: The 16-18 statistical publication and performance tables have reported attainment in below level 3 qualifications in the English and maths progress measures since 2016. Qualifications included are only those English and maths qualifications that are approved for either prior attainment or progress purposes, according to the condition of funding rules.

You can find details of the qualifications included in the condition of funding on gov.uk

Level 2 vocational qualifications: From 2019, only technical certificates will be recognised in the 16-18 performance tables. In 2017 and 2018 statistical releases all level 2 vocational qualifications of size equivalent to at least two GCSEs (minimum 145 guided learning hours) were reported to give time for institutions to transition towards these qualifications. 

Technical certificates: The lists of eligible qualifications can be found here: performance tables: technical and vocational qualifications.

The statistical publication and performance tables include statistics for different cohorts of pupils, depending on the type of study programme they took. Since 2016, students need to have entered a qualification at least the size of 0.5 of an A level or equivalent to be included in a cohort.

• A level students: students entered for an AS or A level, applied single award AS or A level or applied double award AS or A level during 16-18 study
• Academic students: students entered for an advanced level academic qualification at least the size of 0.5 of an A level during 16-18 study. The academic cohort includes A level cohort students
• Applied general students: students entered for an approved applied general qualification at least the size of 0.5 of an A level during 16-18 study
• Tech level students: students entered for an approved tech level qualification at least the size of 0.5 of an A level during 16-18 study
• Level 2 vocational students: students entered for level 2 vocational qualifications of size equivalent to at least two GCSEs during 16-18 study (2016 to 2018 performance tables)
• Technical certificate students: students entered for an approved technical certificate qualification during 16-18 study

Institutions reported

The performance of schools and colleges is in scope for reporting purposes but other education and training provider types, such as independent training provides, are not (see annex A for further information).

In the 16-18 performance tables, special schools are not automatically reported but they may opt to have their results published. However, for aggregated results at national level in both the performance tables and statistical publication, special schools are included. 

Discounting flags

Discounting is used to ensure that, when a student takes two or more qualifications in the same subject area and where there is significant content overlap, the performance measures only give credit once for teaching a single course of study. For example, where a student has gained an AS level but then went onto complete an A level in the same subject, only the A level result will count (i.e. the AS is ‘discounted’ as it is in the same subject as the A level, and the smaller qualification). However, if a student fails to obtain a pass grade at A level, the AS pass grade would be reported in the statistical publication. For further information on discounting see the technical guide.

By applying discounting, the number of examination entries and results reported as A levels in this statistical publication are different from the figures reported by JCQ.

1. Allocation of students to institutions
2. Since 2016, students are allocated to institutions on an annual basis, for the reporting year and the two years prior to this. This means they can be reported at up to three institutions if they moved during 16-18 study. Previously, a student was reported against a single institution only, regardless of whether they had moved during their 16-18 studies.
3. Results will continue to be published at the end of the 16-18 study phase; usually when the student is academic age 17 (year 13). However, students can be reported up to academic age 18, which might mean that some results are published three years after the end of key stage 4.
4. The allocation methodology ensures that the results more accurately reflect where a student has studied and the patterns of movement between different schools and/or colleges.
5. Handling of students in institution-type breakdowns
6. The change to allocation rules in 2016 has an impact on the calculations for the national institution-type breakdowns reported in the statistical publication. In 2015 and earlier years, these statistics were calculated by aggregating school and college results by type of institution e.g. academy converter, FE college, and students could only be reported once in the table.
7. With the change to allocation rules students can now potentially be counted twice (or three) times at institution level if they attend different institutions in different academic years during their two or three years of 16-18 study.
8. Students continue to be reported once in the totals for ‘All schools and FE-sector colleges’, and ‘All state-funded schools and colleges’.
9. Students can also be reported multiple times in LA reporting (where they move between LAs) in different academic years.

Performance measures

The following headline measures are published in the A level and other 16-18 results statistical publication:

• Attainment: The attainment measure shows the average point score per entry, expressed as a grade and average points. Separate grades are shown for level 3 academic (including a separate grade for A level), Applied General, Tech Level and Tech Cert.
• English and maths progress (for those students who have not achieved A*-C in 2016 and earlier years, or 9-4 from 2017): This measure shows the average change in grade separately for English and maths, with students who do not enter an English or maths qualification as applicable, being given a score of -1. The methodology for the measure is closely aligned with the condition of funding rules, which means that students that do not achieve the standard by the end of key stage 4 are required to continue to study English and/or maths during the 16-18 phase. 
• Retention: As the participation age has increased to 18, it is increasingly important that all young people are given suitable education and training opportunities that they see through to completion. The retention measure therefore shows the proportion of students who are retained to the end of their main programme of study (typically published in March)

The following additional attainment measures are also created alongside the headline measures (although perhaps just published in underlying data):

• Best 3 A levels
• AAB including at least 2 facilitating subjects 
• Tech Bacc
• % entering approved vocational qualifications at L2 (technical certificate)
• % entering approved vocational qualifications at L3 (applied general, tech level)
• Level 3 maths
• Returned and retained for a second year (typically published in March)
• Retained and assessed (typically published in March)

More details on how the performance measures are calculated can be found in the technical guide. Note this guidance has not been updated in 2020 and 2021 as it is designed to support 16-18 accountability measures in performance tables, which were cancelled in both years (though, new information was published in 2021 showing the numbers of exam entries for specific subjects approved for reporting in performance tables). However, the fundamental methodologies underpinning performance measures are unchanged.

Note that data reported against a particular cohort just includes qualifications approved for reporting in the relevant year (which updates even if performance tables are cancelled for that year, to ensure the continuity of reporting). The list of qualifications approved for reporting in performance tables are published on gov.uk:

16 to18 qualifications, discount code and point score

Help and support

Contact us

If you have a specific enquiry about A level and other 16 to 18 results statistics and data:

Attainment statistics team

Email: Attainment.STATISTICS@education.gov.uk
Contact name: Michael Greer

Press office

If you have a media enquiry:

Telephone: 020 7783 8300

Public enquiries

If you have a general enquiry about the Department for Education (DfE) or education:

Telephone: 037 0000 2288

Opening times:
Monday to Friday from 9.30am to 5pm (excluding bank holidays)