Personalization – A/B Testing

Owned by Bettina Lippisch
Last updated: Aug 02, 2024 by Katie Robilotta
4 min read

Knowledge Base Home

Search the Omeda Knowledge Base

The A/B Testing tool in Personalizations allows you to optimize your website’s user experience by comparing different versions of personalization messages. With A/B testing, you can easily determine which design, content, or call-to-action resonates best with your audience, driving higher engagement and conversions. Whether you’re aiming to capture more leads, promote events, or enhance user interaction, our A/B testing feature provides valuable insights to help you make data-driven decisions and achieve your business goals.

Key Features

 

Creation of Two Message Versions:

  • Users can create two different versions of the personalization message (Version A and Version B).

Split Traffic Distribution:

  • Roughly half of the visitors will see Version A, while the other half will see Version B.

Rotational Viewing:

  • Visitors who initially see Version A will see Version B on their next visit, and vice versa.

Detailed Reporting:

  • The personalization reporting tool provides insights on total impressions and clicks for both versions.

Automated Winner Selection:

  • Users can optionally set up automation to choose a winner based number of clicks.

  • The losing version will automatically deactivate once a winner is chosen.

  • The user who created the A/B test will receive an email notification with information about the winning personalization message.

  • After the winner is chosen, visitors will only see the winning version.

Getting Started

When creating a new personalization, you will be asked if the personalization will be used for A/B testing on the Basic Information step. Select Yes:

image-20240712-145801.png

NOTE: Once you select Yes and save, you will not be able to update this response.

Win Criteria

If you prefer to setup automation to automatically pick a winner and deactivate the losing version, select the Win Criteria.

Number of clicks by date:

If you select the “# of clicks by date”, you will be given a date picker to select the end date. The test will complete at the end of the day of the date selected. The user that created the personalization will receive an email including information about the winning personalization the night of the selected end date.

Total Number of clicks:

If you select the “Total # of clicks”, you will be given a number field to select the number of clicks. The test will complete once a version reaches the selected number of clicks. The user that created the personalization will receive an email including information about the winning personalization once the winner has been declared.

The email will look like the below:

image-20240712-145811.png

NOTE: If you rather continuously rotate the two versions without automatically selecting a winner instead, skip the Choose Winner questions and press the Save and Continue button.

Message Content Versions

Continue to create the A/B personalization as you would normally (in Step 2 and Step 3). Once you land on step 4, you will be able to create the two different versions of message content. The first thing you may notice is the Version A and Version B toggle. This will allow you to switch between the two versions and add in the desired content.

Version A and Version B must share the same Personalization Type, but the other settings can differ (embed different forms, use different presentation types, etc.

Testing

Once both versions are completed and ready to be tested, a modal will ask which version you would like to test after selecting any of the three testing options; Quick Test, Test with Customer Id, and Test on My Site.

Once you have tested both versions and are happy with the way they display and you’ve completed the scheduling step of the personalization, you are ready to activate the personalization!

Reporting

Check back to view how the different message contents are trending by selecting the View Report action on the personalization. The impressions and clicks of Version A will display on the left and the impressions and clicks of Version B will display on the right.

Manually Deactivate a Version

If you did not select a Win Criteria and decide you want to deactivate one version manually because the other version is getting more attention, navigate back to the Message Content step of the personalization and select the Enable this Version toggle. By selecting this, the message version will deactivate and visitors will no longer see this version when viewing your web page.

Considerations

  • Once a Personalization has been created as an A/B testing personalization, the personalization cannot be reverted back to a non a/b testing personalization. This also means if an A/B testing personalization is cloned, the clone will also be an A/B testing personalization. Though, if there is a message version you no longer want to use, you can deactivate the unwanted version.

  • When testing an A/B test after it has been activated, and testing as an anonymous visitor, you may notice you see one version a few times before seeing the other version. This is normal and expected. The calculation is done at a grander scale and although we aim for a 50% split distribution, the amount of impressions between the two version may vary slightly.