Showing posts with label Split Testing. Show all posts
Showing posts with label Split Testing. Show all posts

Wednesday, May 2, 2018

Best practices for split testing

Best practices for split testing

Split Testing lets you test different versions of your ads so you can see what works best and improve future campaigns.
https://www.facebook.com/business/help/290009911394576
When you create a split test, we divide your audience by ad sets and test one variable (for example: different target audiences) to see if one ad set performs better. We will then determine a winning version based on which one had the lowest cost per result.
You can use these best practices to create a split test with clearer and more conclusive results.

Test one specific factor for more conclusive results

You'll have more conclusive results for your test if your ad sets are identical except for the variable that you're testing.
Testing a single variable is important when you're experimenting with different creatives. If your ad sets have several creative aspects that vary (for example: different images, different headlines and different text), then when the winning ad set is declared, you won't know which factor to credit for this result.
For this reason, after you create your first ad set, we autofill the selections that you made in the first ad set to your second ad set. You can then easily change one factor, while keeping everything else the same. Your creative test will then have only one creative aspect that varies (for example: a different image) while every other aspect in the two ads will be identical. When your test is complete, you will know your results were due to the one varying aspect.
Keep in mind that testing multiple creative variables can still yield valuable results and learnings for future campaigns.

Focus on a measurable hypothesis

Once you figure out what you want to test or what question you want to answer, craft a testable hypothesis that will enable you to improve future campaigns.
For example, if let's say you're wondering: Do I get better results when I change my delivery optimizations? Should I optimize for link clicks, impressions or landing page views?
Using this question, you can create a measurable hypothesis such as: My cost per result will be lower when I optimize for landing page views.

Use an ideal audience for the test

Your audience should be large enough to support your test, and you shouldn't use this audience for any other Facebook campaign that you're running at the same time.
Overlapping audiences may result in delivery problems and contaminate test results.

Use an ideal time frame

We recommend 4-day tests for the most reliable results, and if you aren't sure about an ideal time frame, you can start with 4 days.
In general, your test should run for at least 3 days and no longer than 14 days. Tests shorter than 3 days may produce insufficient data to confidently determine a winner, and tests longer than 14 days may not be an efficient use of budget since a test winner can usually be determined in 14 days or sooner.
For this reason, we recommend a test between 3-14 days for tests created in the API. When creating a split test in Ads Manager, you must create a test with a schedule between 3 and 14 days.
Your ideal time frame (within the 3 to 14-day time period) may depend on your objective and business vertical.
For example, let's say you're running a split test with the conversion objective, and you want to drive product sales on your website. You know that people tend to take longer than 7 days to convert after seeing an ad, so you'll want to run your test for at least 7 days.

Set an ideal budget for your test

Your split test should have a budget that will produce enough results to confidently determine a winning strategy. You can use the suggested budget that we provide if you're not sure about an ideal budget.
Learn more about the basics of Facebook split testing or learn about creating your first split test.
Create a Split Test
Was this information helpful?

Yes

Facebook Split Testing : https://www.facebook.com/business/help/1738164643098669


https://www.facebook.com/business/help/1738164643098669

About split testing

Split testing lets you test different versions of your ads so you can see what works best and improve future campaigns. For example, you can test the same ad on two different audiences to see which ad performed better. Or, to test two delivery optimizations to determine which selection yields better results.
To get started, navigate to Ads Manager and create a split test. Use this guide to understand the basics of split testing, including variables, budget and scheduling.

-1:30
HD

How split testing works

Facebook's split testing feature allows advertisers to create multiple ad sets and test them against each other to see what strategies produce the best results. Here's how it works:
  • Split testing divides your audience into random, non-overlapping groups.
  • This randomization helps ensure the test is conducted fairly because other factors won't skew the results of the group comparison. It also ensures each ad set is given an equal chance in the auction.
  • Each ad set tested has one distinct difference, called a variable. Facebook will duplicate your ads and only change the one variable you choose.
  • To get the most accurate results from your split test, you'll only have the opportunity to test one variable at a time. For example, if you test two different audiences against each other, you can't also test two delivery optimizations simultaneously because you wouldn't know for sure which change affected the performance.
  • Split Testing is based on people, not cookies, and gathers results across multiple devices.
  • The performance of each ad set is measured according to your campaign objective and is then recorded and compared. The best performing ad set wins.
  • After the test is complete, you'll get a notification and email containing results. These insights can then fuel your ad strategy and help you design your next campaign.

Objectives available for split testing

Facebook split testing supports the following business objectives:
  • Traffic
  • App installs
  • Lead generation
  • Conversions
  • Video views
  • Catalog sales
  • Reach
  • Engagement

Variables available for split testing

Advertisers will have the option to test one of the following variables. You can test 5 different strategies with one of these variables.
  • Target audience
  • Delivery optimization
  • Placements
  • Creative
Below are some examples of variables you could split test.
Audiences:
  • Women, Age 21-30 versus Women, Age 31-40 versus Women, Age 41-50
  • People living in London versus people living in Paris versus people living in New York City
Delivery optimizations:
  • Optimize for conversions versus optimize for link clicks
  • Optimize for conversions with a conversion window of 1 day versus optimize for conversions for a conversion window of 7 days versus optimize for link clicks
Placements:
  • Automatic placements versus customized placements
  • Mobile placements versus desktop placements
Creative:
  • Ad with one image versus ad with different image versus ad with different image
  • Ad with a video versus ad with single image
Learn more about the variables you can test.

Setting a budget and schedule

Your split test should have a budget that will produce enough results to confidently determine a winning strategy. You can use the suggested budget that we provide if you're not sure about an ideal budget.The budget and audience will then be divided between the ad sets. You can choose to divide it evenly or weigh one more than the other(s), depending on your preference.
We recommend 4-day tests for the most reliable results, and if you aren't sure about an ideal time frame, you can start with 4 days.In general, your test should run for at least 3 days and no longer than 14 days. Tests shorter than 3 days may produce insufficient data to confidently determine a winner, and tests longer than 14 days may not be an efficient use of budget since a test winner can usually be determined in 14 days or sooner.
For this reason, we recommend a test between 3-14 days for tests created in the API. When creating a split test in Ads Manager, you must create a test with a schedule between 3 and 14 days.

Next steps

When the test is over, you'll receive a notification in Ads Manager and get an email with the results. Learn more about how the winning ad set is determined. Once you receive your split test results, you can review them to discover insights about the best performing ad set. These insights can help you determine your ad strategy and design your next campaign.
Create a Split Test
Was this information helpful?

Yes

Split Testing - https://developers.facebook.com/docs/marketing-api/guides/split-testing

https://developers.facebook.com/docs/marketing-api/guides/split-testing

Split Testing


Test different advertising strategies on mutually exclusive audiences to see what works. The API automates audience division, ensures no overlap between groups and helps you to test different variables. Test the impact of different audience types, delivery optimization techniques, ad placements, ad creative, budgets and more. You or your marketing partner can create, initiate and view test results in one place. See Ad Study Reference.

Guidelines


  • Define KPIs with your marketing partner or internal team you create a test.
  • Confidence Level Determine this before creating a test. Tests with larger reach, longer schedules, or higher budgets tend to deliver more statistically significant results.
  • Select only one variable per test. This helps determine the most likely cause of difference in performance.
  • Comparable Test Sizes When you test for volume metrics, such as number of conversions, you should scale results and audience sizes so both test sizes are comparable.

Test Restrictions


  • Max concurrent studies per advertiser: 100
  • Max cells per study: 100
  • Max ad entities per cell: 100

Variable Testing


While you can test many different types of variables, we recommend you only test one variable at a time. This preserves the scientific integrity of your test, and helps you identify the specific difference that drives better performance.

For example, consider a split test with ad set A and ad set B. If A uses conversions as its delivery optimization method and automatic placements, while B uses link clicks for delivery optimization *and *custom placements, you cannot determine if the different delivery optimization methods or the different placements drove better performance.

In this example, if both ad sets used conversions for delivery optimization, but had different placements, you know that placement strategy is responsible for differences in performance.

To setup this test at the ad set level:
curl \
-F 'name="new study"' \
-F 'description="test creative"' \ 
-F 'start_time=1478387569' \
-F 'end_time=1479597169' \
-F 'type=SPLIT_TEST' \
-F 'cells=[{name:"Group A",treatment_percentage:50,adsets:[<AD_SET_ID>]},{name:"Group B",treatment_percentage:50,adsets:[<AD_SET_ID>]}]' \
-F 'access_token=<ACCESS_TOKEN>' \ https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies

Testing Strategies


You can test two or more strategies against one another. For example, do ads with the conversion objective have a greater impact on your direct response marketing than a website visits objective? To setup this test at the campaign level:
curl \
-F 'name="new study"' \
-F 'description="test creative"' \ 
-F 'start_time=1478387569' \
-F 'end_time=1479597169' \
-F 'type=SPLIT_TEST' \
-F 'cells=[{name:"Group A",treatment_percentage:50,campaigns:[<CAMPAIGN_ID>]},{name:"Group B",treatment_percentage:50,campaigns:[<CAMPAIGN_ID>]}]' \
-F 'access_token=<ACCESS_TOKEN>' \ https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies

Evaluating Tests


To determine the test that performs the best, chose a strategy or variable that achieves the highest efficiency metric based on your campaign objective. For example, to test the conversions objective, the ad set that achieves the lowest cost-per-action (CPA) performs the best.

Avoid evaluating tests with uneven test group sizes, or significantly different audience sizes. In this case, you should increase the size and results of one split so that it is comparable to you other tests. If your budget is not proportionate to the size of the test group you should consider the volume of outcomes in addition to efficiency.

You should also use an attribution model that makes sense for your business, and to agree upon it before initiating a split test. If your current attribution model needs reevaluation, contact your Facebook representative to run a lift study. This can show the true causal impact of your conversion and brand marketing efforts.

Budgeting


You can use custom budgets with your split tests, and choose to test different budgets against each other. However, budget directly impacts reach for your test groups. If your test groups result in large differences in reach or audience size, you increase budget to improve your results and make your test comparable