Measure Concept Engagement, Not Just Purchase Intentions

A lot is on the line to ensure that you advance the ideas and concepts with the greatest potential through your innovation pipeline.  For data-driven organizations, the mainstay has been the traditional concept-test and purchase intent survey; however, these methods often lead to results that are inconclusive and may not reflect true purchase behavior.

To create a more effective tool, we decided that, instead of measuring purchase intent, we would look at Concept Engagement. The idea of Concept Engagement was inspired by a client who sought a way to capture richer and more conclusive data in concept tests.  They sought tools that were more predictive of consumer behavior and felt like a simulated shopping exercise would be a step in the right direction.  The client considered e-commerce transaction tests but was very concerned about revealing too much to the competition.

Concept Engagement models the familiar online shopping experience and measure respondents’ behaviors, giving you more reliable information to pick the right opportunities.  Important metrics in evaluating concepts include “first look” (the concept selected first upon entering an e-commerce simulator), the item is chosen, time to make a choice, and total time spent reviewing a concept. There is growing evidence that actions equate to interest.  According to a study commissioned by Honda Motor Company, there is a strong correlation between the features researched on a manufacturer’s website and the features in the automobile purchased.  In a Stanford University study, researchers concluded that online purchasers engage in higher-effort actions than non-purchasers.  A study by a digital marketing agency concluded that there is a 0.6 correlation between time spent on a website and conversion rate and a 0.25 correlation between pages per session and conversion rate.

To prove out the idea that measuring Concept Engagement could yield  “richer and more conclusive data”, we conducted an experiment with a manufacturer of pet training products. Test A was a traditional monadic concept test using purchase intent as the primary metric. Test B was based on measuring Concept Engagement in the e-commerce simulator. From the standard concept test, results for the Client Concept were promising.  When examining Top 2 box scores, the concept outperformed the competition.  As often seen in concept tests, the results are harder to decipher when focusing on “definitely would buy.”  Here, Competitor B is viewed as a strong contender.

For the e-commerce simulator, we measured First Look, Product Chosen, and Time Spent as metrics of Concept Engagement.  Collectively, this data paints a more robust picture of how consumers will shop and make decisions in this product category.

First Look represents the number of times each concept was clicked first on the landing page. While Client Concept had an edge on First Look, some preference shifted to Competitor B as respondents compared options. These shifts are important in understanding how choice is ultimately segmented in the target population. Client Concept was the most heavily researched, telling us consumers were indeed interested in buying this “new to world” innovation. As previously discussed, Time Spent is an important metric as a resonance of a concept is determined by the amount of time people spend reviewing it.

Another benefit of this approach is that we can ask very specific follow-up questions on why concepts were chosen or not.  In this case, follow-up questions revealed that preference for Client Concept was driven by a segment of consumers who favored the product’s mode of action. Those shying away from Competitor A feared household members would accidentally injure themselves during use.  Those preferring Client Concept vs. Competitor B perceived B’s mode of action as “punishment.”  Preference for Competitor B increased between First Look and Product Chosen for those that had an acute problem and believed its mode of action was the most effective.

The Client Concept, however, was perceived as the “least effective.”  While field tests have been extremely positive, the results of the simulation tell us the importance of strong substantiation as part of the product’s eventual e-commerce presence:  customer testimonials, videos demonstrating product efficacy, and more.  And, as a result of one study, the client now has the confidence to move forward with product development and insight on how best to position the product against competitors.

For our industry, these are indeed exciting times as we witness a proliferation of techniques to add to our toolkit.  Measuring Concept Engagement is gaining ground as clients (and consumers) appreciate that products are being tested in a realistic environment.  For decisionmakers and non-researchers, the data is far more relatable and actionable.  Also encouraging is the tremendous response from consumers.  To date, 96% of study respondents rate this survey experience extremely or very positive.

Original Source: greenbook.org