The future of small area population statistics

Downloads

• 1 Calder and Teague - Beyond 2011 (.pdf)
• 2 Dorling - Cost of the census (.pdf)
• 3 Dugmore - The future of small area population Statistics (.pdf)
• 4 Fransham - A local authority perspective (.pdf)
• 5 Finney and Simpson - Census and ethnicity (.pdf)
• 6 Williamson - Potential role of small area estimation (.pdf)
• 7 Diamond - Area statistics meeting (.pdf)
• 8 Rees - Area statistics for health (.pdf)
• 9 Martin - Responding to the consultation (.pdf)

 

The UK Office for National Statistics consulted on the census and the future provision of population statistics in England and Wales from September - December 2013. The options under consideration had profound implications for the future availability of data on population counts and characteristics for small areas (at and below the local authority level).

The Society, the RGS-IBG Population Geography Research Group (PopGRG), Demographics User Group and British Society for Population Studies held this meeting to understand more fully the key issues and implications to promote informed responses to the consultation, and to identify concerns about the future of small area population statistics.

 

Event report

Themes

The themes identified below are those highlighted by the presentations and the discussion and were intended to be promoted over the following weeks to help statistics users to formulate their responses to the ONS consultation before their deadline of December 13 2013.

1. In 2014 Parliament will be making what is easily arguable as its biggest ever decision on the UK statistical system. The Office for National Statistics is consulting on a major shift in the provision of statistics which would end the decennial census of population statistics begun in its modern form in 1841 and underpinning the whole conduct of population estimation and social surveys. The options being offered for consultation are a modernised census, intended to be conducted online as far as possible (the ‘online census’), or an option combining systematic use of linked administrative data to provide annual population estimates and a 4% annual survey to provide sub-national statistics with greater regularity but fewer social characteristics and with much less geographical detail. We refer to this as the ‘survey option’, although ONS rather confusingly refer to both as census options.

2. The two options on the table have been fairly characterised by ONS as “great statistics once every ten years, or good statistics more frequently”. Under the ‘survey option’, basic population counts by age and sex could be provided annually to a fair quality standard, even for the smallest current census output areas (OAs –mean 300 persons). However, social characteristics would no longer be available for units of this size but would be available as three-year and five-year averages in restricted form for neighbourhoods (LSOAs approx. 1500 population and MSOAs approx. 5,000 population), and annually with very limited detail for local authorities. Without these small area social characteristics it would not be possible to continue with present approaches to small area classification –such as the ONS Output Area Classification which has been so welcomed by users of 2001 and 2011 censuses.

ONS provide detail of the types of data that could be available, for example a small count of 800 in an average local authority would be known from one year’s survey as between 480 and 1120 with 95% confidence. It does not appear that interaction datasets (migration and travel to work, currently available at small area level from the census) could be provided at all under the ‘survey option’ and the ONS consultation offers no solutions to this loss, particularly of significance to transport planners, local government and migration researchers. ONS are confident that the ‘online census’ option could be delivered, although there are likely to be increased challenges of public resistance, low response rates, increased need for expensive follow-up and social and geographical bias in the resulting data compared to census 2011. The ’online census’ offers provision of data, including social characteristics for the smallest census OAs, at ten-year intervals but with no provision to capture data in between.

3. Parliamentary decision-making timescale. For a decision to be made in this parliament, for either option or for some intermediate hybrid, the UK Statistics Authority will need to make a recommendation in spring 2014 for a parliamentary decision perhaps in September 2014.

4. Cost effectiveness is inevitably a major driver in the political decisions to be made. If users believe that it is essential to retain small area data, discussions must establish clear requirements for and a strong case in support of small area data and the relative acceptability and cost of alternative solutions. Otherwise, the political decision may well be made simply on overall cost. This meeting took the discussion of alternatives further but there is some way to go before establishing the feasible alternatives. Speakers clearly identified the dangers associated with a discussion based solely on cost, but which has incomplete information on uses, benefits and risks.

 

Issues/concerns and rationale

5. More regular local statistics are needed to govern the UK in the 21stCentury. The pace of social change and the fragmentation of local service management demand more rather than fewer local statistics, and more frequently. Various examples were presented from the perspectives of health service resource allocation (Phil Rees), cultural integration (Nissa Finney), and land use planning (Mark Fransham). What came across clearly at the meeting was the substantial breadth and depth of the ways in which the small area data were being used in support of policy, practice, and public and economic benefit, and by a wide range of individual users and institutions.

6. The use of survey-sourced statistics for small areas raises serious challenges for statistical education and analysis. The production of statistics with different time frames (single years and various multiple-year averages), accuracy differing between areas (dependent on the area’s population size), and with changing political boundaries on a frequent basis in Britain, risks losing or confusing a large section of those who use census products transparently, and would require an extensive new suite of analytical approaches that have not yet been developed. Evidence from the USA where the American Community Survey (ACS) has been implemented since census 2000 and shares the important feature of pooling estimates across multiple years, suggests that this is very difficult for users to interpret. This is of major concern. The loss of the smallest areas greatly reduces the flexibility to assemble statistics for non-standard areas such as those generated analytically to describe service catchment areas (e.g. schools, shops), areas prone to environmental hazards (e.g. flooding) or future administrative units.

7. Administrative data can and should provide some small area statistics. The developments for population and household estimation that have proven possible should be generally welcomed and should certainly be adopted as part of ONS standard practice for annual population estimates. It would be unreasonable not to implement the lessons from this research.

8. Some believe there to be far greater potential for the generation of social characteristics from administrative data sources than has yet been explored. ONS recognise the potential to derive social characteristics for small areas from administrative sources. This was acknowledged in the ONS presentation but has not yet been explored in any depth and is not part of the options under consultation. This is a major weakness, given that these data might fill a gap that is of enormous concern to users –very little evidence has been presented on which to base a decision, yet this could be a feasible way to produce social characteristics for smaller areas.

9. The barriers to achieving routine and high-quality statistics from administrative data are significant. These barriers include the need for new legislation to allow sharing between government departments and to minimise the impact of administrative changes on statistical products, and methodological challenges to validate the relationship between administrative data and the population. An RSS group has been commissioned to review some of the methodological research undertaken by ONS but its report has not been published in time to inform users in the first half of the present consultation period.

10. Quality criteria for statistics from a survey require debate with specific examples. The suggested minimum quality criterion is a 95% confidence interval of 60%-140% of the estimated value. In such a situation, it may not be possible to identify a significant difference between a percentage value in two areas even though one was more than four times the other. Confidently distinguishing a narrowing from a widening inequality trend or ranking areas reliably on key indicators under such a system are major issues which have not been resolved. The impact of accepting such quality criteria needs to be tested with specific examples.

11. The option to maintain a census could be developed much further than has been so far discussed. By 2021 a census designed to be online might be the basis for regular sample follow-ups to provide more regular social statistics. This would be one hybrid solution, combining both of the consultation proposals.

12. The dynamic nature of the census should not be underestimated. Far from being unchanged for centuries, the census has developed. Its development over the past 40 years has focused on the identification of groups that are otherwise hard to count –ethnicity and other cultural characteristics, illness and disability, and smaller areas. These characteristics are difficult to measure through administrative records and the smaller areas are not provided well by surveys. Recognition of this development helps to clarify what may be lost.

13. Considerable support exists for developing a “transition option”’ between the known advantages of census methods and new methods. This support is not on the basis of “we want everything”, which is accepted as unrealistic and inefficient but rather that there is widespread concern that the work done by ONS to date, although promising, simply does not deliver sufficient evidence that the proposed options could be delivered as planned, nor does it provide sufficient information on which to evaluate the quality of the likely outputs. There is reasonable concern that the proposed survey option implemented for 2021 will not have the appropriate environment (see barriers above) and that the optimum result may be a hybrid solution that could only emerge from a continued census. It was noted by ONS and several speakers that other countries moving to administratively based solutions have taken several decades and also have the advantage of a pre-existing population register; hence data linkage starts from a much stronger base.

14. A high quality register of addresses underlies positive developments in both options. The address register developed for the 2011 Census allowed targeted enumeration and was key to both quality assurance of the results and their final acceptance. A maintained address register needs to be stipulated as part of the options, as does a clear articulation of whether and how individuals would be grouped into households and whether household-based statistics could be produced. Households are currently an important focus of policy but are not easily or accurately captured by lists of matched individuals.

 

Next steps

15. The uncertainty attached to the proposed changes requires careful management. There is a sense that on the one hand current proposals are too great a change to implement in the timescale up to 2021, but on the other that a transition to a radically different system of population statistics is nonetheless indicated by technological and social changes. To find a way through to successful change in the current consultation requires firm management. The professional societies could help by continuing to work together and with ONS to find acceptable proposals, positively involving politicians and the media.

16. Discussion on the options should be promoted urgently, including use of the Stats User Net community on Beyond 2011. To develop commentary on the proposals and to develop the proposals themselves, users of small area statistics will need to communicate concerns and ideas in professional forums, the media and among politicians. We have identified the ‘Beyond 2011 community’ on http://www.statsusernet.org.ukto encourage as a suitable forum of discussion across professional groupings.

17. The case of small area statistics is made with specific examples. Case studies presented at the meeting were particularly useful in highlighting what would be possible and not possible under each option. Realistic examples are likely to include multi-dimensional characteristics, comparing one area with another, and monitoring change over time. There is a continued need to garner effective case studies of the use of small area data. Please submit ideas for case studies to David Martin (D.J.Martin@soton.ac.uk) by November 15; he and the RGS-IBG are working together to accumulate and publish a short and succinct policy briefing for parliamentarians that demonstrates the breadth and depth of use of small area data in informing and shaping policy, practice, economic benefit andpublic benefit.

 

Presentations

Context

• Alistair Calder and Andy Teague, Office for National Statistics - “Beyond 2011” proposals and the future provision of small area data  

Use cases of small area data

• Professor Danny Dorling, University of Oxford - health and equality

• Keith Dugmore, Demographic User Group - commercial

• Mark Fransham, Oxford City Council - local authority

• Dr Nissa Finney, University of Manchester - ethnicity

• Dr Paul Williamson, University of Liverpool - role of small area estimation

Responding to the consultation

• Professor Sir Ian Diamond - Principal, University of Aberdeen - Response

• Professor Phil Rees, British Academy - Why we need area statistics

• Professor David Martin, University of Southampton (on behalf of the Beyond 2011 Independent Working group) - Response

 

Additional resources

Presentations from this event are available to download on the right.

You can learn more about this issue via our case study on Professor David Martin's work on census Output Areas