The dearth of accurate information about roadworks has been an issue for policymakers, but big data analysis of Elgin’s streetworks database could be about to change that, says Shane O’Neill.

Roadworks undertaken either because of emergency or longer-term replacement necessity will continue to be a major annoyance to motorists and pedestrians alike, and be a communications challenge for the industry.

Unlike many sectors, though, the streetworks industry has very little hard analytical data upon which to compare trends, measure performance and underpin policymaking and legislation.

One Department for Transport official formerly closely involved in policymaking said in 2013: “We are entirely reliant on a Halcrow engineer’s report from over ten years ago to inform our policy approach.”

No doubt this is hyperbole arising out of frustration, but perhaps not that far from the truth in an industry where over 200 public bodies hold the statutory records and each utility and contractor naturally guards its internal records with appropriate security. More than two million roadworks take place annually; more than 6,000 new works begin every day; and their impact on traffic congestion and the working lives of the populace has been but barely measured and analysed.

All that may be about to change, however. The Department for Transport has recently taken delivery of a streetworks data report which has the potential to introduce “big data” analysis into national roadworks data for the first time.

In August 2015, a research grant was awarded to specialist boutique consultancy TJH Consulting with a particular expertise in roadworks data under the department’s Transport-Technology, Research and Innovations Grant scheme. The task was to mine the Elgin national roadworks.org data, which has achieved national coverage over the past five years and which represents an aggregated database of millions of roadworks undertaken during that period.

The initiative was given added momentum by transport minister Patrick McLoughlin’s plans to announce changes in seven day working practices. There was an overriding need to understand some of the industry’s basic metrics.

Lead consultant Tobias Hemmings, who has worked closely with the industry for several years, was set the challenge of validating the quality of Elgin roadworks.org against source datasets. Elgin harvests every few minutes roadworks information from more than 168 of the 177 local highway authorities in England and Wales and serves up more than 14 million look-ups on these from its Amazon cloud-based servers.

Hemmings takes up the story: “Our first task was to take an extract of the Elgin data and map this against local highway authority datasets of different types. This was a painstaking process of trying to understand the data in depth and ensure we were comparing apples with apples and that the department and the industry could trust the data and at least understand that where there were variables we understood why.

“One of the wonderful things about having access to such a unique national dataset is that we were able programmatically to add additional information by correlating the data with other datasets and introduce validation routines to improve the data quality”

The second criterion set by the department was for the data to be rendered into a high-powered visualisation and analytics software. TJH Consulting choose Tableau, which can quickly transform complex queries across millions of lines of data into attractive, easy-to-understand visuals.

However, there needs to be caution. We are in early days: the data needs to be analysed and outputs compared and the industry itself needs to come up with a code of practice by which this complex specialist information can be used. The output provided to the department was seasonal data only, and has already been succeeded by a 12 months data extract and been subjected to further validation routines.

Nonetheless, it looks as if the streetworks industry is beginning to take a giant step towards placing policymaking and performance analysis on a data-supported approach through the use of big data.

Shane O’Neill, chairman, Elgin

There will be a free to enter keynote conference on streetworks at Utility Week Live on 17 May. See: www.utilityweeklive.co.uk

Think big

Bob Gallienne, chief executive of NJUG, commenting on big data in support of evidence-based policy making.

“The challenge that utilities have ahead of them is clear: there is an unprecedented investment programme to maintain, replace and extend their networks, while at the same time dealing with a significantly increased housing programme and a very active level of commercial development. All of this has to be delivered in a situation of a forecasted rise in the amount of traffic and population.

“The use of big data, like the kind Elgin has produced, allows for the ability to analyse performance and areas of activity, which is vital in regards to being innovative in the way works are delivered in order to safely and effectively minimise road occupation and communicate with the public.

“NJUG’s own Step Change in Information project, which was completed last year, further supports this. It contains information on the current and future cost of undertaking streetworks in England and Wales. It is only with effective data that the industry can appropriately support government in regards to proposed policy changes.”