A Rising Tide Lifts All Boats

By Patrick Glaser, CMOR Dir. Respondent Cooperation

The nature of the research process includes tradeoffs. A smart researcher realizes this, and also knows that every decision has an influence on a project’s costs and validity. One of CMOR’s areas of focus, respondent cooperation, reflects some of these balances and tensions. For example, how does a research firm balance quality results and cost effectiveness when poor response rates are a potential sign of invalid data?

On any survey, higher response rates may give some evidence that a study has valid results. At the same time, obtaining those response rates may mean the research carries a higher price tag. It may also involve other, less tangible consequences. What, for argument’s sake, is the cumulative effect of a thousand research firms each slightly irritating just a single sample member in order to get a “complete?”

Michel Rochon, President of ASDE Survey Sampler, addresses this problem in the June 2005 issue of MRA’s monthly magazine, Alert! Rochon posits that a “tragedy of the commons” is, and has been, occurring where respondents are overused as a free resource in our industry.

In Rochon’s words:

“The overexploitation of the resource means only that it will disappear over time. The public won’t disappear over time, the phone lines won’t disappear; the public will not ask to have its phones disabled. Rather pressure upon politicians will bring about legal restraints on an industry, which cannot restrain itself. That is how society dealt with air and water as public goods. The polluters were fined out of existence. Closer to home that is how legislators have dealt with the inconvenience of telemarketing polluting the telephone.”

Many researchers view respondent cooperation as the “nitty-gritty,” the type of thing that End Users are aware of, but prefer to leave to the researchers to worry about.

This is an idea that falls outside the margin of error. As a resource for the industry, I’ve received a number of questions from End Users regarding cooperation trends. Like many other methodological issues, respondent cooperation has become an increasingly difficult problem to alleviate and is a cause for concern for everyone involved in the research process.

If it is true that the researcher’s existence hangs in the balance because of declining response rates and regulation, then it is equally true that End Users are threatened with the prospect of losing a vital decision-making tool. Likewise, when researchers face the anguish of completing the desired amount of interviews within a certain timeframe and for a certain budget, clients of research firms face both the threat of compromised validity and the financial anguish of costly data.

As Michel Rochon states, part of the theory of “The Tragedy of the Commons” demonstrates that no single individual firm has enough incentive to correct the problem on their own. No firm wants to be the first, or only, organization to bear the costs necessary to improve respondent cooperation. It’s dubbed as the classic “free-rider” problem. Everyone benefits, but only one entity pays.

At CMOR, we believe in, and advocate, a host of options for improving respondent cooperation. For example, management and survey design decisions can optimize a firm’s individual respondent cooperation while respecting the public. In this manner, research firms employing these methods, and ultimately their End User clients, receive an individual benefit. At the same time, the survey research industry benefits as a whole.

This type of solution is the crux of Rochon’s position, and it’s one that should be well taken by the industry. Individual firms can make a difference in the industry and still receive a lion’s share of individual benefit. Still, industry associations such as CMOR are proof that if the right mechanisms are in place there are also collaborative solutions. Firms can work together and develop best practices promoting respondent cooperation.

An important lesson in improving respondent cooperation is understanding that no single group of stakeholders can successfully claim ownership of the challenges the industry faces. Survey researchers can take practical steps in designing respondent-friendly studies and survey researchers and End Users alike can provide feedback to respondents by conveying the value of participation in their public communications.

Perhaps a problem in survey research has been the view of respondent cooperation being the researcher provider’s ‘private domain.’ Many efforts throughout the industry focus solely on researcher-based solutions. Going forward, many of the greatest gains in improving respondent cooperation may be made through finding practical ways and benefits for End Users to leverage their strength and to get involved.