Accuracy
Accuracy describes the closeness between an estimated result and the (unknown) true value.
Data coverage
The source data covers the eligible population for the statistical publication and 16-18 performance tables, as the department collects annual data on all results achieved by students. The eligible population for the qualification data collection is all 16 to 18 year old students who by the end of the reporting year completed 16-18 studies in schools and colleges in England.
Note, this means the eligible population does not include all 16 to 18 year olds in England; students not studying in schools and colleges are excluded from the statistics, which will include: students studying through private training providers, most students enrolled in traineeship and apprenticeships, and young people in employment (unless also enrolled in a school/college).
Students who are not at the end of 16-18 studies, or are re-sitting exams, are excluded from the calculations. As these students cannot be identified centrally, the department applies a set of rules to identify students at the end of 16-18 study, namely:
- they have entered for at least 2 qualifications, each of which is at least the size of an A level or they have entered for at least 1 qualification the size of at least 2 A levels, in the reporting year; OR
- they have entered for both a T Level core and T Level occupational specialism in any year during 16-18 study (T Level attainment will not be included in performance data until 2023/24, but this rule is applied a year early to produce a normal cohort when reporting begins), OR
- they are 18 at the start of the reporting year and have not been reported in 16 to 18 performance measures at their current allocated school or college
During the September checking exercise, schools and colleges may apply for students to be removed from their figures for several reasons; for example, the student is not at the end of 16-18 study, if the student is an external candidate and not on roll, or if the student is deceased. This is normally reflected in updated statistics published in January.
Note, due to the cancellation of the checking exercise in 2019/20 and 2020/21, this did not happen for those cohorts of students.
Measurement error
Measurement error is the difference between the actual value of a quantity and the value obtained by a measurement.
Examination results
Note: these steps just apply to results based on externally assessed exams (i.e this does not apply to AS/A level results in 2020 and 2021 that due to COVID-19 were awarded by an alternative process)
The following steps are taken to minimise measurement error in examination results.
Clear guidance is provided to schools and colleges regarding the administration of examinations, including instructions for keeping the exam materials secure prior to the exams and storage of completed scripts until they are collected for marking.
Once schools and colleges have completed 16-18 exams such as AS and A levels, they are required to send their exam materials and attendance registers to the awarding organisations for external marking and data capture.
The exam papers are externally marked by the awarding organisations to ensure that marking is consistent between institutions. Exam boards follow the principle that if the cohort of students taking a subject is similar to that of previous years, then the proportions of students at each grade will be similar. A key piece of evidence in determining if the cohort is the same is prior attainment at GCSE.
After marking, results are returned to schools and colleges. Data is then passed from the awarding organisations to the department for use in the statistical publications and 16-18 performance tables. If schools and colleges have concerns over the marking of an exam paper, they may request for it to be re-marked. Outcomes of reviews are not reflected in the provisional statistical publication data but are included in the revised data and compare school and college performance data. (opens in a new tab)
Further information on the general regulations for schools and colleges and how exams are conducted and marked can be found at JCQ: Joint Council for Qualifications (JCQ) website (opens in a new tab).
Qualifications
Ofqual sets the standards and rules that awarding organisations need to meet when they design, deliver and award regulated qualifications. Awarding organisations and qualifications are also monitored to maintain standards. Ofqual are a risk-based regulator and concentrate on where resources are most needed. For example, Ofqual may discover that an awarding organisation’s processes need updating or that there are risks associated with a qualification taken by thousands of students every year. Further guidance on the role of Ofqual (opens in a new tab).
School census
The school census is a statutory data collection, which means that schools are obliged to provide the department with data on their pupils. School census returns are quality assured by the department to ensure that they contain data that is as accurate and complete as possible. The department acts to ensure the quality of this data in the following ways:
- Detailed technical guidance is provided to schools to support them in submitting full and accurate data to the department;
- Management Information System suppliers build validation into the systems that they provide to schools. The department provides guidance on the types of validation it expects for school census purposes;
- The department’s COLLECT web-based data collection tool automatically validates the data that schools and local authorities supply, providing error and query reports so that these can be investigated/resolved.
More information on the school census can be found on gov.uk (opens in a new tab).
Individualised learner record (ILR)
The ILR is managed by the Education and Skills Funding Agency (ESFA), who undertake various activities to ensure the data returned by colleges is accurate and high quality:
- Detailed technical guidance and specifications are published by the ESFA to support colleges in providing the data that they must return
- Data submitted through the web-based data collection system is validated to check for errors or issues. Detailed validation guidance is available to colleges.
- As part of assurance work, the data reported through the ILR is monitored by the ESFA to ensure it is high quality and meets their published funding rules. A sample of colleges are also selected for external funding audit, part of which checks that ILR returns accurately reflect the learners on roll and the learning they receive.
More information on the ILR can be found on gov.uk (opens in a new tab).
Validation and quality assurance of source data
The department applies a set of internal quality protocols to ensure the published data is accurate. First, the data production process is subject to a ‘dry run’ during the summer. This involves producing a dummy dataset, which conforms to how the current year’s data will be supplied (e.g. if new subjects have been introduced, then dummy data would be added to last year’s data to simulate these). This dummy dataset is used to test the department’s systems and checking processes. This allows potential problems to be resolved prior to receiving the live data.
In addition, at every stage in the ‘live’ data cycle, the department checks all calculations used in the production of the figures by independently dual-running the methodologies at institution, local authority and national levels. Further quality assurance is then carried out on the data produced for the statistical publication (see examples below). Any discrepancies in the data are discussed and resolved prior to publication.
Examples of additional checks - Comparisons with previous figures to identify any large changes
- Comparisons with provisional data to identify any large changes (revised/final data only)
- Check totals are consistent across tables
- Check patterns in the data are as expected
- Check the data format against specifications
|
---|
Disclosure Control
The Code of Practice for Official Statistics requires us to take reasonable steps to ensure that our published or disseminated statistics protect confidentiality. We assess our statistics with reference to the National Statistician’s Guidance on Confidentiality of Official Statistics and guidance from the Information Commissioner’s Office (ICO) to establish the risk of disclosure and its associated impact and suppress the data accordingly. In our statistical releases the following conventions are applied: ‘c’ for confidential, ‘z’ for not applicable, ‘:’ for not available.
Reliability
Reliability is the extent to which an estimate changes over different versions of the same data.
Change between releases
The figures published in the statistical publication may change between provisional and revised releases; however there tends to be a slight change in the headline national performance measures rather than any significant fluctuation in the statistics. This is due to the combined effect of removals of students that should not be included and re-marking and submission of late results by awarding organisations.
Changes over time
In 2016, there were several changes to 16-18 attainment measures due to educational policy reforms. As such, there are difficulties with making comparisons to data published in 2015 or earlier in the headline performance measures. The impact of the 2016 reforms was assessed in the 2016 A level and other 16-18 results provisional release (opens in a new tab).
Comparability of the statistical publication and performance tables
The statistical publication and performance tables use the same data sources. Where they both reference the same performance measures, they use the same methodology for calculation. In addition, the statistical publication includes further A level and other examination results based on a single year of academic study. The statistical publication also includes further tables on English and maths progress for those students who did not achieve 9-4 or A*-C by the end of key stage 4. These use the same methodology as the performance tables measure.
Other statistics from the Department for Education
The statistics shown below are produced by the Department for Education but provide different information to that published in the 16-18 statistical publication and performance tables.
Destination measures | Figures for young people who went into education, employment or training destinations the year after they completed key stage 4 or key stage 5. The department publishes statistics including information from a data source which gives more complete coverage of participation in employment than the earlier ‘experimental’ destinations data. |
---|
Level 2 and 3 attainment at 16-18 | Statistics on the attainment of young people aged 19, based on matched administrative data. |
---|
Participating in education, training and employment by 16 18 year olds in England | Estimates of participation in education, training and employment and those who are not in education, employment or training (NEET) for 16, 17 and 18 year olds in England. |
---|
Key stage 4 | GCSE and equivalent results for key stage 4. |
---|
Key stage 2 | National curriculum assessments and review outcomes at key stage 2 (KS2), including measures of progress between KS1 and KS2. |
---|
Key stage 1 | National curriculum assessments at key stage 1 and phonics screening check results. |
---|