Using Incrementality Testing With Predictive Analytics

Marketers are under pressure to show the value of their work. As a result, marketing managers are using incrementality testing to measure the impact of their efforts. The results are an essential guide for their decisions about where to increase marketing efforts and how to shape marketing strategies.

Incrementality testing is a cornerstone measurement approach that allows executive and marketing teams to scale initiatives while proving value and driving results in the process. It also complements predictive analytics by helping marketers assess the results of changes to marketing efforts.

What is incrementality testing?

Incrementality testing, also called A/B testing, is a testing methodology that measures the actual lift of a marketing initiative or ad campaign. For example, instead of measuring how many conversions result from a campaign, incrementality testing reveals how many additional conversions you generated with the campaign that would not have otherwise occurred.

Incremental testing lets you compare a new element with a controlled one.

Reviewing an example of an incrementality test on Facebook ads can illustrate the process. Measuring Facebook ads’ effectiveness can be accomplished by comparing the performance of a treatment group (i.e., the group that was exposed to the Facebook ad) to a control group (i.e., a group that was not exposed to the ad).

Note that it’s crucial to first align on the key performance indicators (KPIs) that the test will assess.

Here’s a seven-step overview of the testing process.

Define your goal or hypothesis for the test: Determine precisely what meaningful process and the outcome you are seeking to measure.
Define the objective: Clearly define the objective of the Facebook ad campaign and the metric that will be used to measure the campaign’s performance (e.g., click-through rate, conversion rate, etc.).
Create the control group: Create a control group of users who will not see the ad but are similar to the treatment group in terms of demographics, interests, etc.
Create the treatment group: Use Facebook’s targeting options to create a treatment group of users who will see the ad.
Run the campaign: Run the Facebook ad campaign for a limited period. Make sure to avoid scheduling it around any big announcements or holidays. Depending on your business and conversion windows, a test might run anywhere from a week to a month. Collect data on the performance of the treatment group and control group.
Analyze the data: Compare the performance of the treatment group and control group to calculate the lift, which is the incremental impact of the Facebook ad campaign. Ideally, results will be statistically significant to provide a meaningful, actionable insight about the campaign’s impact.
Make data-driven decisions: Use the results of the lift study to make data-driven decisions about how to optimize the Facebook ad campaign and improve performance.

Of course, Facebook ads are shown within a complex ecosystem of content. Keep in mind that many factors that can affect performance, including the makeup of the test and control groups. Conducting a lift study is just one way to help understand the effectiveness of a Facebook ad campaign. However, it’s a reliable way to get a sense of how much the ad is affecting the conversion rate, sales, or another vital KPI.

How should you set up audiences for incrementality tests?

Setting up audiences for incrementality tests requires a bit of finesse. Aim to create treatment and control groups that reflect similar demographics, interests, and behavior:

Use similar audience targeting: When creating the treatment group, use Facebook’s audience targeting options to select users similar to the target audience. For the control group, use the same targeting options to select a group of users who are similar to the treatment group but will not be exposed to the ad.
Use a random sampling method: Alternatively, use a random sampling method to select users for each group. This method reduces the chances of bias while helping to make each group representative of the target audience.
Use the same time period: To ensure that the results of the incrementality test are accurate, run the test over the same period for treatment and control groups. You can then make a fair comparison of the results.
Use a similar ad format: To avoid format bias, deploy the same ad format to the treatment and control groups (e.g., video, images, copy, and so on).
Monitor group size: Groups should be a similar size to reduce the risk of bias.

Careful setup of your groups and maintaining a high degree of similarity between them will provide accurate results. In turn, you’ll have better insights into the ad campaign’s effectiveness.

Why is incrementality testing important?

Research has shown that adding four or more channels to an integrated campaign could increase results by 300%. Every day, many new marketing services, tools, and online platforms are available, enabling marketers to reach their target audience.

But which of these channels is worth investing time and resources in? It’s difficult for marketers to understand which partners should be given the green light and added to an ad campaign or a user acquisition initiative.

The primary goal of marketing is to drive results while controlling spending. Incrementality testing allows marketers to test these new and emerging media channels to accomplish this goal. With incrementality testing, marketers can test new partners, analyze performance, and measure the incremental results of a given marketing initiative.

As a result, marketers can be more strategic with ad budgets and test new campaigns and ad partners with small budgets to prove their ability to drive results. Once results are proven, marketing can grow these tests in larger programs that create net new revenue streams.

How can incrementality tests help marketing campaigns?

Incrementality testing helps data-driven marketers answer strategic questions on the impact of their campaign, channel, or ad partner test. Below are a few questions that incrementality testing can help marketers answer.

Did a campaign, channel, or ad partner generate net new revenue?
Did the new campaign, channel, or ad partner have a positive return on ad spend (ROAS)?
Is the campaign, channel, or partner driving new buyers, more buyers, higher purchase frequency, or higher purchase orders?
Did the new audience segment create new revenue?

How has the initiative impacted market share? Or share of wallet?

How does incrementality testing complement predictive analytics?

Incrementality testing and predictive analytics are fantastic tools for measuring marketing campaigns’ effectiveness and driving data-focused decisions. They complement each other well in the marketer’s toolbox.

As you’ve seen above, incrementality testing helps you measure the incremental impact of a specific marketing campaign or strategy on a particular metric. As a result, you can assess the actual effect of your campaigns on your outcomes of interest. Equipped with that information, you can better allocate your resources.

Predictive analytics, on the other hand, is a technique that uses statistical models and machine learning algorithms to analyze historical data and make predictions about future outcomes. With predictive analytics, you can find patterns in data that are too complex for humans to identify. Those patterns and trends can be used to segment and target audiences.

Incrementality testing and predictive analytics can be a powerful duo for a complete understanding of marketing efforts. Marketers can use predictive analytics to identify patterns and trends and segment audiences. Then, they can apply incrementality testing to measure the impact of specific campaigns on those segments. This combination helps marketers optimize their marketing strategies and use their limited marketing budgets more effectively.

In summary, incrementality testing helps to understand how a specific campaign contributes to the outcome of interest, while predictive analytics helps to understand the underlying patterns in the data and make predictions about future results. Together they can give a more complete picture of the marketing efforts and inform better decisions.

Use predictive analytics + incrementality testing for a marketing boost

If you’d like to use predictive analytics to improve your marketing efficiency, we’re here to help. You’ll be ideally positioned to increase and demonstrate the value of your campaigns with incrementality testing and other measurement methods.