Case study: optimising water supplies

How Affinity Water teamed up with Talend, AWS and Datalytyx to better understand its assets and meet changing regulatory and customer demands

Employing 2,000 people, Affinity Water supplies 900 million litres of water daily to 3.6 million domestic and business customers in three English regions. The largest, Central, extends from Bedfordshire and Buckinghamshire into Hertfordshire, Greater London, Essex and Surrey. The smaller East region covers parts of Essex, while the Southeast takes in coastal areas of Kent. Key activities focus on the abstraction, treatment and delivery, and billing of clean water services.

The challenge: Optimising water supplies

“We’re part of the national infrastructure: our product is essential to life,” says David Clifton, Enterprise Solutions Architect. “The key issue is maintaining supplies in the most effective ways. There are many challenges in this industry that we can only overcome using data collection and analysis. We have to deal with ever-increasing demand while combating water scarcity and drought: we operate in the UK’s driest regions. Data and analytics can help ensure enough water for the future.”

This is where Amazon Web Services (AWS), Talend technology and Datalytyx expertise come in.

“The water industry is regulated and we’re an asset-centric organisation,” Clifton continues. “We must submit a business plan to regulators in five-year cycles on how we intend to spend our money and manage those assets.

“This Asset Maintenance Plan includes commitments to regulators and customers. To do this, we need to analyse and model assets in various ways. For example, hydraulic modelling of the pressure required to feed water to properties, and models to examine bursts – looking at our pipe network, different ages, are they in a wet or dry environment. Analysing all that data is essential to how we operate.”

The solution

“We made a strategic decision to move from traditional IT delivery to the cloud,” Clifton says, “providing the ability to provision infrastructure services without high capital investment.” AWS was selected following a competitive tender: “Its services and cost model are hard to beat, at a time when it’s increasingly difficult for individual IT people to stay abreast of all the new services and opportunities. Our future business operations will depend on the cloud: the ability to analyse data at any scale with a click gives us unparalleled advantage that we could never achieve using a traditional self-hosting IT model.”

He says it was also essential “to standardise data integrations: to remove swivel-chair manual integrations and legacy script-based file transfers.” As part of this, Talend was selected, initially to provide more batch-centric integration, although this is changing: “We’re also exploring the Data Quality side of Talend, not least in light of future Data Governance programmes and impending cyber and data security commitments. Master Data Management will become one of our most valuable future initiatives.”

With Talend recommending Datalytyx, Clifton says the Datalytyx team: “Brings the right skills in Talend and other applications such as SharePoint and K2. Datalytyx provided a ‘product support wrapper’, allowing us to trade-off in-house skills. The largest Datalytyx project to date has been developing our new centralised meter reading solution.” Previously, information was held in a variety of system including asset management, meter reading, billing and web portals. The solution collects data from multiple sources including Internet-connected smart- meters, loading it into a data lake structure for ease of management and analytics. New data sources can be added easily.

“It’s early days but the solution already allows Affinity Water to capture this data in one place, and is enabling us to meet our commitments to the new open market in the commercial customer space. A recent project focused on helping the company trade with other supply businesses efficiently for the fast and accurate settlement of bills.

“Going forward, this is the kernel repository that will allow us to analyse and understand  the demands of customers. With automated meter reading and smart metering we can start understanding real-time events on our delivery network, to trace and fix leaks faster and reduce customer bills.”

The benefits

Clifton adds; “Without this underlying technology we’d struggle to operate and meet our business commitments, and also risk financial penalties from regulators and other areas, including requirements such as the EU General Data Protection Regulation (GDPR). We’re certainly seeing gains from AWS and Talend. It’s a little early to assess those in monetary terms but we’re certainly seeing benefits in lead times to deliver change.”

In telemetry, for example, data was previously analysed and reported from an old RDBMS, which wasn’t optimised. “Using Talend and AWS Redshift means our business teams can now execute reports in minutes rather than hours or even days.

“We had some reports we couldn’t execute for years due to the size of the database; these run in less than one hour. And with AWS Redshift provided ‘as a service’, a large amount of the IT operations needed to maintain underlying systems is taken care of, while we also have disaster recovery and restore options. And without Talend, transmitting the data would be expensive and complex.”

• More effective IT operations that are ‘cost-neutral’
• Enabling the company to operate successfully and meet business commitments, avoiding financial penalties from regulators
• Delivering new business insights and speeding up report delivery from days to minutes

The future

In terms of future plans, Clifton responds with “Data, data and data. By that, I mean looking at Master Data Management and Data Governance, and analysing customer meter data to better understand demand, to develop our real-time operations and situational awareness, and increase customer channels.”