If at first you don’t succeed…

Everybody agrees that customer engagement is essential to a properly functioning market, and the only way to get it is to experiment and find out what works, says James Harvey.

Few are surprised by one conclusion of the Competition and Markets Authority’s (CMA) energy market investigation: some retail customers are “disengaged” and competition would be fiercer if they were “engaged”.

Competition authorities and sector regulators find “the disengaged” on an almost annual basis, particularly in pipes and wires markets, such as utilities and financial services – where suppliers differentiate themselves through service quality rather than product features.

The story is always the same: the disengaged could save hundreds of pounds every year if only they spent a few minutes on a price comparison website, but they don’t bother and remain zombie-like with their current supplier. They get asked: “Why don’t you search and switch?” and they reply with statements such as: “I’m too busy”; “It’s too complicated”; and “there’s no point, all suppliers are the same”.

One typical regulatory prescription for the disengaged is to give them more information so that they can work out what’s best for them. The diagnosis is that a lack of information, or it being too difficult to use, is to blame (and is also consistent with what the disengaged say).

Yet the same consumers successfully complete more time-consuming and complex buying tasks to, say, save £20 on a new tablet or choose the right summer holiday for the whole family. The diagnosis and prescription sit uncomfortably with how these customers actually behave, and are probably wrong.

Experimenting on customers

The question is: how can we make better diagnoses and prescriptions?

One answer to this question is to follow the CMA’s proposed remedy to “…establish an ongoing programme to identify, test and implement measures to provide domestic customers with different or additional information with the aim of promoting engagement in the domestic retail energy markets, including… randomised controlled trials…”.

In this context, a randomised control trial would involve Ofgem working with suppliers to establish a control group who are given a retail placebo (that is, current bill presentation) and one or more treatment groups who are given a retail drug (new bill presentations). You then watch what each group does to see if the drug has any effects (in terms of switching rates).

This approach contrasts with relying on what you think consumers might do or what consumers say they would do in response to different interventions and so departs from the standard regulatory approach. The recommendation implies that the CMA thinks the standard regulatory approach to understanding and increasing customer engagement will (at least in this case) be subject to error.

It is consistent with its observation: “There are many potentially plausible but divergent arguments about the way in which information should be provided to domestic customers to facilitate understanding and engagement. Without adequate testing it is not possible to know which approach will work best in practice. Further, even if testing is conducted ex ante, changes in technology and cultural practices are likely to mean that what works changes over time.”

Our pipes and wires versus tablet anecdote suggests that there is some force in the CMA’s argument. Dumping more information on consumers who have shown themselves to be capable of completing other complex buying tasks seems unlikely to engage them. More likely, we need to find a way of finding and pressing the same buttons that buying a tablet or holiday presses (within reason, of course).

Would consumers be more motivated to shop around if they knew that other people like them had done so and saved money? What about if they were told that they had missed out on a weekend mini-break by not switching? Does it make a difference if the sums involved are framed as a potential saving to make or a loss incurred? When is the best time to alert a customer to the potential savings he or she could make? Academic studies in economics and psychology suggest that how a sum of money is presented or “framed” is as important a driver of behaviour as its size.

Implications for policy makers, regulators and companies

The CMA’s recommendation has potentially far-reaching practical and policy implications to energy and other utility retailers.

The practical implications include the following:

•    Retailers will have to become conversant with new methods for understanding customer behaviour. The workhorse of customer surveys will be complemented and in some cases replaced by randomised controlled trials or lab-based experimental techniques.

•    Difficult decisions will need to be made  in relation to what changes should be tested; how they should be tested (with trials or lab-based experimental techniques or surveys); which companies and customers; and how frequently and for how long?

The CMA’s recommendation also raises numerous policy questions:

•    What type of evidence should be used to judge whether a retail market is suitable for being opened up to competition (such as the household water market)? Should we be sceptical of what customers say, given the experiences in other sectors?

•    Where will Ofgem and others draw the line between encouraging companies to make pro-competitive changes and getting overly involved in the details of how services should be designed and presented to consumers?

James Harvey, director, Economic Insight