Project 4β supports Warwick University research on UK housing supply barriers

Oxylabs’ pro bono initiative, Project 4β, partnered with Dr. Nikhil Datta, Assistant Professor in the Department of Economics at Warwick University, on a project investigating how local planning regulations affect housing supply across the UK.

The research focuses on one of the country’s most pressing policy challenges: why new housing supply continues to fall short of demand. By collecting and analyzing data from publicly available planning registers, the project aims to identify the barriers that delay, stall, or prevent housing development across the UK.

Research with public planning data

The UK’s housing affordability crisis is closely linked to a persistent shortage of new homes. While planning restrictions are often seen as a major factor, there is still limited systematic evidence showing which specific rules, processes, and local conditions create the biggest obstacles to housing delivery.

This study seeks to close that gap by building a planning-application-level dataset from local authority planning portals. The goal is to better understand how planning outcomes vary across councils and what drives approvals, rejections, and delays in the decision-making process.

The research will help answer questions such as which planning-related constraints most often slow down new housing supply, how these barriers differ across local authorities, how long different types of applications take to reach a decision, and what policy interventions could help increase housing delivery where it is most needed.

Why public web data access matters

A key part of the project involves gathering information from public planning registers on local authority websites. These registers are an essential source of data for understanding how the planning system works in practice and are meant to be publicly accessible as part of the UK planning process. 

However, collecting this information at scale is not straightforward. Planning portals are spread across many council websites, often using different systems and structures, and some can restrict repeated access, making large-scale data collection difficult. For a research project that depends on broad geographic coverage and consistent historical data, these barriers can significantly limit the size and representativeness of the dataset.

Supporting academic research at scale

Through Project 4β, the research team gained access to dedicated ISP proxies, enabling more reliable and scalable data collection across local planning portals. With Oxylabs’ infrastructure, the team has been able to collect substantially more data from public planning application registers than would otherwise have been possible.

According to the researchers, access to proxy infrastructure enabled the collection of tens of thousands more planning applications, strengthening both the depth and representativeness of the dataset.