Why Testing is a Good Idea, but so is also Throwing Away the Results
Last week's Business Week included an article by Steve McKee warning that pretesting advertising campaigns can kill good ideas. ("Beware the Advertising Pretest" at
http://www.businessweek.com/smallbiz/content/dec2007/sb2007127_693450.htm?campaign_id=rss_daily ).
Many successful campaigns, argues McKee, would never have passed traditional pre-testing with focus groups. The reasons are not hard to find, but the most important is the disconnect between the real-life reaction of consumers and what they say when you ask for an opinion.
Predictably when you ask someone to look at an ad and comment they feel obliged to find some flaw; they feel that to say honestly that it looks ok is somehow not enough to pay for the free food and other considerations they were given. And in any case most people simply don't have the knowledge or the experience to provide meaningful comments.
You get the same problem when you test new brand names on non-professional audiences. Many now valuable brands would never have passed any systematic testing regime. Sony's Walkman, for example, was initially rejected in many countries and for a time it was called the "Soundabout" or "Stowaway". "Walkman sounds funny" was the complaint. But in the beginning all names sound funny. I am sure Coca Cola would have been rejected, too.
In my own experience I don't recall a single name that was ever accepted without comment from someone. In one case the name "Tosca" was opposed by someone whose explanation was that his customers were not fans of opera. By the same logic only an astronomer can drive a Saturn.
Testing new ads, new names and other ideas on samples of the public is not in itself a bad idea -- it can catch some potentially expensive errors -- but it must be done carefully. Your tests should try to capture a natural reaction and not some spurious comments motivated only by the desire to give feedback. And once you have the feedback you have to know when to listen to it and when to throw it away.
http://www.businessweek.com/smallbiz/content/dec2007/sb2007127_693450.htm?campaign_id=rss_daily ).
Many successful campaigns, argues McKee, would never have passed traditional pre-testing with focus groups. The reasons are not hard to find, but the most important is the disconnect between the real-life reaction of consumers and what they say when you ask for an opinion.
Predictably when you ask someone to look at an ad and comment they feel obliged to find some flaw; they feel that to say honestly that it looks ok is somehow not enough to pay for the free food and other considerations they were given. And in any case most people simply don't have the knowledge or the experience to provide meaningful comments.
You get the same problem when you test new brand names on non-professional audiences. Many now valuable brands would never have passed any systematic testing regime. Sony's Walkman, for example, was initially rejected in many countries and for a time it was called the "Soundabout" or "Stowaway". "Walkman sounds funny" was the complaint. But in the beginning all names sound funny. I am sure Coca Cola would have been rejected, too.
In my own experience I don't recall a single name that was ever accepted without comment from someone. In one case the name "Tosca" was opposed by someone whose explanation was that his customers were not fans of opera. By the same logic only an astronomer can drive a Saturn.
Testing new ads, new names and other ideas on samples of the public is not in itself a bad idea -- it can catch some potentially expensive errors -- but it must be done carefully. Your tests should try to capture a natural reaction and not some spurious comments motivated only by the desire to give feedback. And once you have the feedback you have to know when to listen to it and when to throw it away.
Comments