Opinion polls, focus groups, executive interviews, mail panels, mall intercepts, customer contact evaluations, Internet surveys, and website feedback mechanisms: each of these methods will yield information about what your customers may be thinking, but they are far from equal. All have their uses, but they differ widely in both accuracy and applicability. This article will review some of the more common applications of opinion research, how companies can make use of the results, some of the pitfalls to avoid, and review some of the recent work by the Canadian Electricity Association (CEA) indicating the principal concerns of today’s electricity customers.
Value of Research
Not everyone is convinced as to the value of opinion research, so let me state my bias up front. For the past two decades, in the electricity sector and in other fields I have been both a user and designer of opinion research. Having been involved in all aspects of research, from concept, design, fielding, data compilation, analysis and reporting of opinion polls, focus groups, executive interviews, media studies and readership survey, I am among those who believe in the value of research. At the same time, I have also seen poor research, and recognize that without integrity with respect to the objectives, design, implementation, analysis and reporting, misleading or inaccurate conclusions can result in potentially negative consequences if business decisions are made based on flawed research.
So how do you avoid the garbage-in/garbageout trap? The answer is simply through credible and actionable research as detailed below.
Methodology
It starts with the definition of the problem. While all steps are important, this one is absolutely critical. The first principle is defining what information your need, what is the critical issue upon which you need further input in order to make a decision. Once the problem has been clearly articulated, and you are able to determine that information on the customers’ view of the issue would aid in decision-making, you can formulate the goals and objectives of your research. In some cases, opinion research projects should in fact come to halt at this stage, because some issues simply to not lend themselves to research, either because the customer has no knowledge of the issue, or the customers’ view is irrelevant in the decision–-making process. In these instances, knowing the customers’ opinion would be of little value.
Once the goals and objectives are clearly understood and articulated, and you have determined that some form of opinion research would assist in decision-making, the next step is to match goals and objectives to the specific method of seeking the customers’ view.
At the outset, a number of methods that companies use to gauge customer opinion have been identified. Each has different characteristics, and it is at this stage that matching goals and objectives to methods should take place. Among the characteristics you need to take into consideration are accuracy, level of detail, cost, and timing.
Types of Research
Generally, opinion research methods can be quantitative or qualitative, and a simple way of understanding the difference is to think of quantitative research as being able to tell what customers think, and qualitative research as giving you an understanding of why customers hold the opinion they do. While each can be used for the other purpose as well, this short-hand is an indication as to what method of research is best suited to certain types of goals and objectives.
The most frequently used methods of quantitative research include opinion polls, mail panels and Internet polls. The opinion poll often is the most accurate, and depending upon the sample, can yield results with a very high degree of confidence. It is most helpful in determining customers views, from general to very specific, and can be employed to identify changes over time.
CEA uses polling as the anchor for its annual Public Attitudes Research study. As part of that study, we measure customer views on a broad range of issues, from service satisfaction to perceptions of environmental performance. Because we are seeking maximum accuracy, we opt for an opinion poll of a random sample of Canadians, which delivers a high degree of confidence that results are representative of the population as a whole.
The need for accuracy is the reason we do not use other quantitative methods of inquiry. For example, while an Internet-based poll may provide results that suggest how customers feel, it is only accurate in so far as it is representative of Internet users, and specifically those Internet users who respond to the survey. Internet usage is higher in certain demographic groups, and can depend upon age, income and level of education. As a result, an Internet poll may be a simple and cost effective way to get a general indication of how people view a certain issue, but the results can not be extrapolated to the population as a whole.
Combining Methods and Thorough Analysis
Quantitative and qualitative research is often used together as part of an opinion research project. For example, a survey may indicate a reduction in the level of confidence in the industry, and while an analyst may speculate as to the cause, qualitative research can be used to better understand the reason for the views the customer holds. The most frequently used method of qualitative research is the focus group. In focus groups, 8-12 participants are gathered together, and with the guidance of a moderator, have an indepth discussion on a predetermined issue.
Many companies use focus groups as part of their marketing efforts in order to understand customer motivations, and test both product offerings and marketing campaigns. CEA has also used focus groups to gain a better understanding of customer views on public policy issues, to test receptivity to various policy options, and to test communications materials. I highly recommend attending focus groups to company decisionmakers. Often, focus groups facilities are equipped with one-way mirrors so clients can observe the proceedings. You certainly get a great deal more out of focus groups if you are able to hear what customers have to say, in their own words, than simply reading the results in a report. As well, a report cannot capture tone, attitude and body language.
One pitfall to be mindful of is drawing the wrong conclusions due to either inaccurate or a lack of depth in data analysis. An example of this can be found in work CEA undertook at the request of one of its members a number of years ago. We were asked to provide an assessment of the effectiveness of their bill stuffers. The first pass at the poll data suggested less than a third of customers were aware of a significant campaign being promoted in their bill stuffers. The initial conclusion drawn by the researcher was that bill stuffers were not an effective way of communicating with customers. However, a more detailed analysis of the data showed that a large majority of one segment of customers was aware of the campaign, and that segment was customers who were most supportive of and favorably disposed towards the company. Thus the correct conclusion was not that bill stuffers were a poor communications vehicle, but that they were in fact a very effective communications vehicle for communicating information to one of the most important customer segments. This conclusion also provided guidance as to what type of information should be provided in bill stuffers.
Current Customer Concerns
In late April and early May of this year, CEA surveyed the Canadian public as part of its annual Public Attitudes Research project, aimed at providing an indication of customer views on a broad range of issues, and to pinpoint how those views have changed over time. While the detailed report is available only to CEA Corporate Utility members, the following summary of those results provides insight into current customer concerns.
The first and perhaps most important conclusion drawn from the research, and this has been the case for the past few years, is the price, while very important, is not the most important issue for customers. We track more than a dozen service attributes each year, both in terms of importance and satisfaction, and more than half of those are viewed as more important than price to customers. The most important service attributes are those which relate to the accuracy of billing and reliability, and include ensuring a sufficient supply for the foreseeable future, and speed in restoration when an outage occurs.
In a direct trade-off between reliability, environment and price, when customer must choose between them as to which is more important, reliability is the most important. Price in fact comes in third in this equation, less important than both reliability and ensuring environmental protection.
Environmental considerations are once again a growing concern for electricity customers, and the interest of customers in electricity from “green” sources continues to be significant. When asked what they would value most from an electricity supplier, electricity from alternative sources such as wind and solar was seen as more valuable than rates that were 10 percent lower.
In terms of overall measures of satisfaction, satisfaction is highest with respect to servicerelated issues, less so with respect to the perception of the environmental performance of the industry, and lowest overall when rating the communication efforts of companies. Clearly, more effort needs to be placed on effectively communicating with customers.
In conclusion, while many companies use opinion research in the development and delivery of their marketing efforts, there is a role for research in many areas of industries operations. This could include assisting communications, corporate planning, public policy, rate cases and facility site selection and approval. The key is ensuring the goal and objectives are conducive to opinion research, the methodology is appropriate for the issue being research, the process is conducted with integrity, and the analysis is thorough and without bias.
Value of Research
Not everyone is convinced as to the value of opinion research, so let me state my bias up front. For the past two decades, in the electricity sector and in other fields I have been both a user and designer of opinion research. Having been involved in all aspects of research, from concept, design, fielding, data compilation, analysis and reporting of opinion polls, focus groups, executive interviews, media studies and readership survey, I am among those who believe in the value of research. At the same time, I have also seen poor research, and recognize that without integrity with respect to the objectives, design, implementation, analysis and reporting, misleading or inaccurate conclusions can result in potentially negative consequences if business decisions are made based on flawed research.
So how do you avoid the garbage-in/garbageout trap? The answer is simply through credible and actionable research as detailed below.
Methodology
It starts with the definition of the problem. While all steps are important, this one is absolutely critical. The first principle is defining what information your need, what is the critical issue upon which you need further input in order to make a decision. Once the problem has been clearly articulated, and you are able to determine that information on the customers’ view of the issue would aid in decision-making, you can formulate the goals and objectives of your research. In some cases, opinion research projects should in fact come to halt at this stage, because some issues simply to not lend themselves to research, either because the customer has no knowledge of the issue, or the customers’ view is irrelevant in the decision–-making process. In these instances, knowing the customers’ opinion would be of little value.
Once the goals and objectives are clearly understood and articulated, and you have determined that some form of opinion research would assist in decision-making, the next step is to match goals and objectives to the specific method of seeking the customers’ view.
At the outset, a number of methods that companies use to gauge customer opinion have been identified. Each has different characteristics, and it is at this stage that matching goals and objectives to methods should take place. Among the characteristics you need to take into consideration are accuracy, level of detail, cost, and timing.
Types of Research
Generally, opinion research methods can be quantitative or qualitative, and a simple way of understanding the difference is to think of quantitative research as being able to tell what customers think, and qualitative research as giving you an understanding of why customers hold the opinion they do. While each can be used for the other purpose as well, this short-hand is an indication as to what method of research is best suited to certain types of goals and objectives.
The most frequently used methods of quantitative research include opinion polls, mail panels and Internet polls. The opinion poll often is the most accurate, and depending upon the sample, can yield results with a very high degree of confidence. It is most helpful in determining customers views, from general to very specific, and can be employed to identify changes over time.
CEA uses polling as the anchor for its annual Public Attitudes Research study. As part of that study, we measure customer views on a broad range of issues, from service satisfaction to perceptions of environmental performance. Because we are seeking maximum accuracy, we opt for an opinion poll of a random sample of Canadians, which delivers a high degree of confidence that results are representative of the population as a whole.
The need for accuracy is the reason we do not use other quantitative methods of inquiry. For example, while an Internet-based poll may provide results that suggest how customers feel, it is only accurate in so far as it is representative of Internet users, and specifically those Internet users who respond to the survey. Internet usage is higher in certain demographic groups, and can depend upon age, income and level of education. As a result, an Internet poll may be a simple and cost effective way to get a general indication of how people view a certain issue, but the results can not be extrapolated to the population as a whole.
Combining Methods and Thorough Analysis
Quantitative and qualitative research is often used together as part of an opinion research project. For example, a survey may indicate a reduction in the level of confidence in the industry, and while an analyst may speculate as to the cause, qualitative research can be used to better understand the reason for the views the customer holds. The most frequently used method of qualitative research is the focus group. In focus groups, 8-12 participants are gathered together, and with the guidance of a moderator, have an indepth discussion on a predetermined issue.
Many companies use focus groups as part of their marketing efforts in order to understand customer motivations, and test both product offerings and marketing campaigns. CEA has also used focus groups to gain a better understanding of customer views on public policy issues, to test receptivity to various policy options, and to test communications materials. I highly recommend attending focus groups to company decisionmakers. Often, focus groups facilities are equipped with one-way mirrors so clients can observe the proceedings. You certainly get a great deal more out of focus groups if you are able to hear what customers have to say, in their own words, than simply reading the results in a report. As well, a report cannot capture tone, attitude and body language.
One pitfall to be mindful of is drawing the wrong conclusions due to either inaccurate or a lack of depth in data analysis. An example of this can be found in work CEA undertook at the request of one of its members a number of years ago. We were asked to provide an assessment of the effectiveness of their bill stuffers. The first pass at the poll data suggested less than a third of customers were aware of a significant campaign being promoted in their bill stuffers. The initial conclusion drawn by the researcher was that bill stuffers were not an effective way of communicating with customers. However, a more detailed analysis of the data showed that a large majority of one segment of customers was aware of the campaign, and that segment was customers who were most supportive of and favorably disposed towards the company. Thus the correct conclusion was not that bill stuffers were a poor communications vehicle, but that they were in fact a very effective communications vehicle for communicating information to one of the most important customer segments. This conclusion also provided guidance as to what type of information should be provided in bill stuffers.
Current Customer Concerns
In late April and early May of this year, CEA surveyed the Canadian public as part of its annual Public Attitudes Research project, aimed at providing an indication of customer views on a broad range of issues, and to pinpoint how those views have changed over time. While the detailed report is available only to CEA Corporate Utility members, the following summary of those results provides insight into current customer concerns.
The first and perhaps most important conclusion drawn from the research, and this has been the case for the past few years, is the price, while very important, is not the most important issue for customers. We track more than a dozen service attributes each year, both in terms of importance and satisfaction, and more than half of those are viewed as more important than price to customers. The most important service attributes are those which relate to the accuracy of billing and reliability, and include ensuring a sufficient supply for the foreseeable future, and speed in restoration when an outage occurs.
In a direct trade-off between reliability, environment and price, when customer must choose between them as to which is more important, reliability is the most important. Price in fact comes in third in this equation, less important than both reliability and ensuring environmental protection.
Environmental considerations are once again a growing concern for electricity customers, and the interest of customers in electricity from “green” sources continues to be significant. When asked what they would value most from an electricity supplier, electricity from alternative sources such as wind and solar was seen as more valuable than rates that were 10 percent lower.
In terms of overall measures of satisfaction, satisfaction is highest with respect to servicerelated issues, less so with respect to the perception of the environmental performance of the industry, and lowest overall when rating the communication efforts of companies. Clearly, more effort needs to be placed on effectively communicating with customers.
In conclusion, while many companies use opinion research in the development and delivery of their marketing efforts, there is a role for research in many areas of industries operations. This could include assisting communications, corporate planning, public policy, rate cases and facility site selection and approval. The key is ensuring the goal and objectives are conducive to opinion research, the methodology is appropriate for the issue being research, the process is conducted with integrity, and the analysis is thorough and without bias.
