When conducted by qualified professionals, properly executed and reported market research is a great tool. But, unfortunately, a lot of research is pure bunk. And that’s seriously dangerous. Especially if the results will inform a major decision. If you want to learn about your staff’s favorite snacks, by all means, fire up a poll on Facebook. Otherwise, keep an eye out for these pitfalls:
Management Objectives MIA. Every research project needs a management objective. It’s a clear, simple declaration of the what decisions you would like to make based on the research. Find out what features to add to a patio grill, or decide what’s most meaningful in your customer care program. Decisions may include adding or eliminating features, discontinuing the product, or modifying style. Translation: if you don’t know what you will do with the research, save your money.
Fake Focus Groups. You know a focus group is fake if you’re conducting it yourself; the focus group knows who’s sponsoring the research; or the participants don’t represent your target market. For honest feedback, hire a trained moderator and book a facility that doesn’t give away your company’s identity. A prime example is a home building client inviting realtors to lunch to elicit information about homebuyers. How accurate is that feedback when the guy “moderating” the discussion is picking up your tab? And you should be talking to homebuyers, not the people that sell to them.
The No Confidence Level. Quantitative research is meaningless unless reported with a confidence level. The confidence level is the percentage of the population that you can expect to duplicate any given result. For example, you’ve enlisted 100 male and 100 female car owners to survey paint colors. You might discover that 10% of men prefer their rides pre-rusted. Chill, it’s not a new market niche.
Without a confidence level, you might infer that 10% of men are rust aficionados. That would be ever so wrong, because the sample size is small, which leads to a very low confidence level. In general, confidence level goes up the larger the sample, and plunges when it’s smaller. Without a confidence level, your information is cooked.
Bad recruit. The usefulness of quantitative or qualitative data depends on how accurately your focus group members or survey sample represents your customers. For example, you don’t want to talk to Florida residents about snow tires. The best respondents are those that are randomly selected from the population you want to research. Not your mailing list, extended family, or former sorority sisters. Define your customer segments, and task your research company with finding a random sample of respondents. Generally, the more detailed the recruiting profile, the more expensive the recruit. But it’s worth it if you value confidence in your results.
DIY Questionnaire Design. The answers you get depend on how you ask the question. “Do you agree that Michael Jordan is the greatest basketball player ever?” [ ] Yes [ ] No. That’s a perfect example of a biased question.
It should be crafted like this. “Which, if any, of the following basketball players would you consider the greatest ever?” [ ] Michael Jordan [ ] Shaq [ ] Kobe Bryant [ ] LeBron James [ ] None of the above
You need a certain type of mojo to write and design an effective survey.
There are more ways to create bias in asking questions than combinations of deli meats to put on a sandwich. Questions and answers need to be rotated. The right types of rating scales need to be used, and so on. Again, if you want serious questions answered, hire a professional.
Conducting Research to Support the Desired Result. We already showed you how to play with confidence levels to manipulate results, possibly to protect a pet project, but usually it happens at a lower level of consciousness. You and your boss hope for a certain result, and it expresses itself in a bias when you ask a question. It’s more than enough to invalidate a study.
- Hire a competent research firm, especially if a program or people’s jobs hang in the balance.
- Avoid DIY focus groups. They’ll tell you what you want to hear.
- Recruit the right people.
- Insist on confidence level reporting in your quantitative research.
- Filter out your personal bias by removing yourself from the tactical phase of research.