Focus Groups are Dead

In his book, Originals, Adam Grant¹ discusses why focus groups are largely ineffective and unreliable. He uses the pilot tests for the sitcom, Seinfeld, as his case study to support this argument. Broken down into four bullet points, Grant’s reasoning behind his theory was as follows:

1. When people are placed in focus groups, they know that they are meant to be critical. This means that you’re not getting Susan’s natural reaction, since she has now been predisposed to be judgmental.
2. Individuals in a focus group are not in their natural environments. John’s reaction to to a dirty joke on Instagram when he’s out with his friends may be different than his reaction to the same Instagram joke when he’s at home with his wife. In any case, a focus group environment will not directly mimic the environment that consumers will be in when using the product. Therefore, the reactions are going to be different.
3. People are generally pleasers. Humans like to ensure that everyone is happy. Thus, when individuals are asked to evaluate something in a focus group, they become exceptionally critical. This is because they’re worried that if they miss something, someone may not be happy as a result.
4. The greatest reason that Grant gives for focus groups being unreliable is actually that they are too knowledgeable. If your study participants know too much about the industry or realm that your product resides in, then they are going to judge it by how it compares to other successes in the same realm. This completely disregards any truly innovative or new products since they are, essentially, untested in the market and therefore cannot be judged by the standard criteria.

Make the Holidays Work For You

The holidays are a great time to test out my theory with at-home focus groups. In full disclosure, the information that follows is pure theory and has not, as of yet, been put to the test. This next section contains my hypothesis that,

Using holiday gatherings as an opportunity for a focus group will create more honest results than doing a traditional focus group study.²

The holidays [may] create for a more effective focus group because of the following factors:

1. The holidays are time when you can bring together large groups of people simply for the purpose of spending time with one another. Gatherings may include friends, family, and even extended family, which creates for a diverse group in both age and gender.
2. Since stress is high and families begin spending more time together, people tend to get more opinionated about things. They also look for discussion topics when the conversation lags or falls on a heated topic. These are great ways to insert a comment about the product/service that you’re testing.
3. It’s a great opportunity for product placement, allowing you to either bring your product as a host gift or put it out in the open for people to engage with. Throughout the evening, you can watch how people interact with it, note any problems that arise, and capture both photos and videos of people using your product.
4. Since your test subjects are going to be people close to you, you have a better chance of getting a reply if you send a follow-up email or text accompanied by a quick survey and an explanation about the research you were doing.

Proving or Disproving The Hypothesis

Since I do not have the man power or the funds to do this test on my own, I’d love it if you could help me prove or disprove this theory.

Use your own product or service and try to follow these six steps:

1. Bring your product to a holiday gathering as a host gift or decoration item. Do NOT say anything about the product being related to you or your work; it’s just something you picked up. (This avoids any bias)
2. Put your product in a popular hangout spot and watch from a distance. You want to be in stealth mode because you don’t want to alter the results of your “study”.
3. Have your phone or a notepad nearby to jot down observations, problems, and any other information you deem pertinent.
4. Take photos and videos of people interacting with the product.
5. Send a follow-up email to everyone at the party with a brief survey and a quick explanation. Make sure that you do not lead them to any conclusions or opinions by your words anywhere in your survey.
6. Post your conclusion: Proved or Disproved, to social media or comment on this blog post so we can all benefit from the results. Use #AHolidayStudy if posting on social media so that we can find your results!

This won’t be a bullet proof study with concrete findings but you’ve got to admit, it will be interesting.

Happy Testing!

 https://embed.ted.com/talks/adam_grant_the_surprising_habits_of_original_thinkers

Post Notes:

¹ Purchase book on Amazon

² Adam Grant does offer a better way to test ideas, products, and services by using different parameters to define the test group audience. My hypothesis is solely my own, and I strongly recommend reading his book, Originals.