speaking in clicks

[February 5, 2010]

On the face of it, testing creative concepts in a focus group before developing them fully might make sense. In theory, you should save oodles of time and money and guarantee that nothing you produce will turn off your customers. As I see it, however, the premise is false for two reasons. First, you’re not actually testing the finished product. Second, you’re gauging audience reactions in a completely unrealistic setting.

As with every other form of human communication, creative ideas take their meaning in large part from their intended context. At the same time, asking consumers to evaluate a rough mockup assumes they have both the training and the creative intuition to fill in the blanks. You might as well ask medical patients to evaluate their own surgery. Whether they feel better or worse, their evaluation is meaningless in medical terms.

“Artificial Intelligence.”
Besides, there’s no correlation between how consumers will react to creative on their own, unobserved, and how they will react in a focus group. Sitting in a fishbowl, knowing their responses are being monitored, the temptation to grandstand is irresistible. Goaded on by a moderator’s leading questions, people in groups are prone to seek a group consensus. After all, nobody wants to be known as the loser who can’t see the emperor’s new clothes.

At the other end of the spectrum are the classic egomaniacs focus groups attract. They’ve learned long ago that the best way to grab attention is to buck the trend of the room. They’re all too eager to usurp the role of a creative innovator without the requisite expertise and without accepting the concomitant risk to their reputations.

Questionable Answers. 
Finally, let’s consider the impact of interrupting the creative process by putting a teams’ incomplete work up for evaluation. Once “the feedback’ has been absorbed, that process is forever disrupted, as the creative team is urged to ignore its instincts, honed over years of experience, and “go with the data.”

Now, don’t get me started on whether this “data” is even remotely analogous to true scientific data. The samples are too small and the conditions aren’t tightly controlled. Science? I don’t think so.

So as far as I’m concerned, creative concepts modified to conform with doubtful “research findings” are always suspect. Tainted goods, they’re potentially toxic to a brand’s relationship with its audience—as the sour taste of poorly conceived messaging slowly undermines consumer appetite for the product.

Lessons Lost. 
Marketing to consumers means accepting risk. Giving anxious clients false reassurance with doubtful “clinical trials” is irresponsible and bad for business. The only way to know what works is to try it out on the big stage. Anything less cheats agency and brand alike of the invaluable lessons we can only learn from unfiltered failure and honest success.