We can learn a lot from the Scots about data readiness

As the English water sector prepares for non-domestic competition, Charles Vincent explores lessons in data readiness from the Scottish market, including confusion over occupancy status.

Open Water has released a second iteration of its Market Architecture Plan (MAP 2) before the opening of the English non-domestic market. It highlights the importance of market data integrity.

Scotland completed its initial data migration over seven years ago, so we can look back to see what lessons can be learned.

In Scotland, data has caused issues – and significant expense – for all participants in the market. Both Scottish Water and Business Stream have invested multiple millions of pounds in projects, post market opening, to resolve data problems. These included:

•    occupancy status of properties (vacants/voids),

•    properties missing from the market (gaps),

•    alignment of data among market participants,

•    accuracy of meter data, and

•    matching of data to an external database.

These projects were initiated because of problems inherent in the available data before market opening. Retailers indicate that the majority of customers that have transferred have had problems with their data – anything from missing address elements to missing sites or incorrect services. These problems were compounded by structural changes to the data during market opening and subsequent operation.

A key example is the management of void (vacant) properties and gap sites. Occupancy status data in Scotland (and under MAP 2 in England) sits with the retailer. So the retailer has the ability to switch off the wholesalers’ revenue stream for individual properties if it believes the property should not be charged.

Historically, the retailer could send speculative bills to the occupier. The only cost of doing so was in raising and sending the bill, so the cost of incorrectly sending bills to vacant or duplicate properties was negligible.

However, under competition arrangements, if the retailer does this it will incur the full wholesale charge. This drives different behaviours and retailers become instinctively more conservative about speculative billing. They also become keen to remove properties that should not be in the market.

Conversely, the wholesaler is incentivised to maximise both the number and occupancy of properties in the market, resulting in a focus on identifying gap sites and properties incorrectly marked vacant. But there is zero incentive to remove inaccurate properties.

These competing objectives have resulted in the introduction of complex incentive and administration charge schemes.

This drove the market to implement a link to an external reference database held by the Scottish Assessors. The Scottish Matching Exercise is designed to ensure that all eligible premises are included in the market and any non-eligible premises are removed. This helps resolve various issues:

•    assessing ongoing eligibility for non-domestic charging;

•    services being split/merged across supply points;

•    ability for competitors to identify and switch supply points;

•    mismatching of property and meter addresses/locations;

•    accurate charging on RV-based tariffs;

•    duplicates (both existing and addition of new gaps);

•    vacancy (void) management; and

•    communication between wholesaler and retailer, including ability to accurately verify services.

The key lesson from the Scottish Matching Exercise suggests that achieving a match of 99 per cent or more between an external database and supplied services/properties helps to prevent or reduce data-related issues.

A good match is essential, because unmatched properties cause most of the issues with matched properties. An example is duplicate properties, where an unmatched property is a duplicate of a matched property.

Also, the early Scottish market was plagued by data alignment issues between the wholesaler, market operator and retailer.

The Scottish market operator made it possible to integrate wholesale and retail systems with market systems, removing the need for human intervention. A website for manual updating was also provided.

However, none of the organisations was integrated at market opening, so system users started to introduce errors manually or failed to make all the necessary updates. From that point, data alignment between the wholesaler, market operator and retailer began to drift. This resulted in inaccurate wholesale and/or retail charging.

Correcting this proved difficult because participants had to establish the data set that was correct for misalignments, often one at a time. This frequently required retrospective adjustment of customer bills.

The final lesson from Scotland is that fixing individual issues is always more difficult after market opening because the retailer and wholesaler are affected in different ways.

Also, not all stakeholders have access to the same information. For example, the wholesaler does not have all customer information and the retailer does not have graphical information system data. This is worse in a “switch” where issues must be dealt with by four parties (wholesaler, incoming retailer, outgoing retailer and customer) rather than two (wholesaler and customer).

Where additional customer charging is involved (historically an agreement may have been reached between the customer and wholesaler), there is no mechanism for this to happen, because settlement calculations (the calculation of what the retailer owes the wholesaler) have no flexibility. This means retailers either charge customers full back-charge amounts, or incur a direct loss by paying for services that they will not charge for.

Where these issues are highlighted by switching and result in increased or back-charging, this can provide a negative attitude by customers to the switch process.