6.1 Weighting the main SCEYP
Weighting was used to ensure that the final achieved samples were representative of early years and childcare providers in England. Survey weights were designed separately for the three provider types[15] (opens in a new tab) to correct for unequal selection probabilities and non-response bias. Grossing weights were created in order to ensure that the weighted achieved samples gross up to the population of early years and childcare providers in England.
[15] (opens in a new tab) Survey weights were designed for the three overall provider types (group-based, school-based and childminders). In the official statistics report there are five provider types reported on separately. However, private and voluntary are sub-groups of the overall group-based provider type (and were sampled in the same way), and maintained nursery schools and school-based providers offering nursery provision are sub-groups of the overall school-based provider type (and were sampled in the same way).
6.1.1 Main SCEYP: Weighting for group-based providers
Population totals
The achieved sample of group-based providers was weighted to be representative of all active group-based childcare providers (GBP) in England that were eligible for the study. However, the sampling frame did not allow for the exclusion of all ineligible institutions prior to sampling, therefore the eligible population size and profile were estimated using information about institutions subsequently found to be ineligible.
The total eligible population was estimated at 22,924 establishments – 98.1% of the sample frame.
Weighting process
Design weights
Design weights were first calculated to correct for unequal selection probabilities arising from the fact that establishments in the North East were oversampled to allow robust figures for the region to be calculated.
Calibration weighting
Calibration weighting was used to remove the (measurable) bias introduced through non-response to the main Survey of Childcare and Early Years Providers (SCEYP) and align the profile of the achieved sample to the profile of the eligible population defined by: region, register type, ownership type, and deprivation band based on Income Deprivation Affecting Children Index (IDACI).
Questionnaire variant weights
Questionnaire variant weights were calculated for the analysis of providers allocated to each of the three GBP variants of the questionnaire. The final overall weight served as a pre-weight for adjustments to the responding cases and their profiles were grossed up to the eligible population profile.
Day of the week weights
The selected sample was allocated systematically within each variant of the questionnaire to a specific day of the week (Monday to Friday), about which they were asked certain questions. However, providers were able to complete the survey about a different day. The profile and the overall population size for each day were estimated from the main SCEYP (weighted by overall interview weight), using the data from questions about which days of the week GBPs were open in 2022, as this question was not asked in 2023. Subsamples of each day were then calibrated to the population profile.
Staff weights
In addition to calculating weights to make the sample representative of GBPs in England, weights for analysis of information on staff in such settings were also calculated. Staff weights were needed to account for the fact that settings employing more than a certain number of staff at a given qualification level were only asked to give information about one or two members of their team chosen at random. A design weight to take account of this selection was calculated and then multiplied by the provider-level interview weights to obtain staff level weights for each case. Additional calibration was applied to ensure that the weighted staff profile (by level of education) was in line with the data collected about the number of staff working.
Some data were excluded from the staff-level datasets prior to weighting. This included cases where information on the staff member was not provided or the respondent reported an overseas qualification. Providers for whom the number of paid staff for whom qualifications were recorded was greater than +/- 2 the total number of paid staff reported have been excluded.
6.1.2 Main SCEYP: Weighting for school-based providers
Population totals
To create a sampling frame of school-based providers the Schools’ Census database from June 2022 was used, enhanced with further information from a Get Information About Schools (GIAS) extract. During the interviews some institutions were found to be ineligible to take part in the survey: schools that had closed down and schools that did not offer nursery provision. As for group-based providers, establishments coded as “duplicates” and those that had shut down were treated as eligible on the assumption that any school that had opened in the intervening period that would therefore not be included in the sampling frame, would be roughly analogous to those that had subsequently closed. The total eligible population was estimated at 9,943 – 99.9% of the initial sample frame.
Weighting process
Design weights
Design weights were calculated to correct for disproportionate sampling of different types of schools.
Calibration weighting
Calibration weighting was used to remove the (measurable) bias introduced through non-response to the main SCEYP and align the profile of the achieved sample to the profile of the eligible population on the following variables: school type (main stratum), region, type of establishment, quintile of number of places registered.
Questionnaire variant weights
Questionnaire variant weights were calculated for the analysis of school-based providers allocated to each of the two variants of the questionnaire. The responding cases and their profiles were grossed-up to the eligible population profile.
Day of the week weights
As with GBPs, although respondents were allocated a random day of the week (Monday – Friday) to answer certain questions about, they were able to complete the survey about a different day. As in the previous four waves, an assumption was made that all schools were open every week day and the sample of schools which answered about each day of the week was weighted to the overall profile of the eligible population.
Staff weights
In addition to calculating weights to make the sample representative of SBPs in England, weights for analysis of information on staff in such settings were also calculated. Staff weights were needed to account for the fact that settings employing more than a certain number of staff at a given qualification level were only asked to give information about one or two members of their team chosen at random. A design weight to take account of this selection was calculated and then multiplied by the provider-level interview weights to obtain staff level weights for each case. Additional calibration was applied to ensure that the weighted staff profile (by level of education) was in line with the data collected about the number of staff working.
Some data were excluded from the staff-level datasets prior to weighting. This included cases where information on the staff member was not provided (the staff loop was empty) or the respondent reported an overseas qualification. Providers for whom the number of paid staff for whom qualifications were recorded was greater than +/- 2 the total number of paid staff reported have been excluded.
6.1.3 Main SCEYP: Weighting for childminders
Population totals
The childminders’ data was weighted to be representative of the eligible population of childminders in England as of July 2022. Some childminders sampled for the main survey were found to be ineligible in that they were no longer offering provision. As in the previous waves, it was decided not to redefine the population and to assume that the population was stable in its characteristics.
As in the previous four waves, providers with zero registered places were included on the assumption that at the time of the interview, information about the number of registered places might have already been outdated. It was decided, as in previous waves, that cases who reported zero registered places during the interview would be re-coded as ineligible, and responding childminders would be weighted to the profile of population excluding cases with zero registered places in the sample frame. Following data cleaning, the total eligible population was 28,262.
Weighting process
Design weights
Design weights were first calculated to correct for unequal selection probabilities arising from the oversampling of smaller regions.
Calibration weights
Calibration weighting was used to remove measurable bias introduced through non-response to the main SCEYP and to align the profile of achieved sample to the profile of the population. The population targets used for calibration weighting included: region, whether on all three registers (Early Years Register, Compulsory Childcare Register and Voluntary Childcare Register), registration year, and deprivation band based on IDACI. Questionnaire variant weights were calculated for the analysis of childminders allocated to each of the two childminder variants of the questionnaire. The responding cases and their profiles were grossed-up to the eligible population profile.
Day of the week weights
The selected sample was allocated systematically within each variant of the questionnaire to a specific day of the week (Monday to Friday) about which they were asked certain questions. The overall population size for each day were estimated from the main SCEYP (weighted by overall interview weight) using 2022 data which asked childminders which days of the week they were open for. Subsamples of each day were then calibrated to population estimates.
Staff weights
Respondents that answered questionnaire Variant 1 were asked several follow up questions about the assistants they employ. There were up to two loops in the questionnaire for assistants, so for respondents with more than two assistants design weighting was required to compensate for this. The final staff level weight was re-scaled so that the total weighted number of assistants was in line with an estimate of the total number of assistants.
6.2 Weighting the short SCEYP
Data from the short SCEYP was combined with data from the financial variant of the main SCEYP where variables were available in both, i.e., the fees and funding questions.
The weighting strategy was designed to create a weight to be used in analysis of the combined dataset comprising data from the short SCEYP and the financial variants of the main SCEYP. Three separate weights were created for SBPs, GBPs and childminders as set out below.
The weighting strategy was the same for all provider types taking part in the short SCEYP or the financial variant of the main SCEYP. Data from the two surveys were combined for use in financial analysis and calibrated to the same variables and population totals as used for the main SCEYP.
A combined weight was also created for analysis of all groups together. Combined weights for SBPs, GBPs and childminders were scaled so that the weighted profile by provider type was aligned with the percentage distribution of provider types in the population (modelled estimates).