EN |

A/B test

test and measure before shipping

Index

Versions: create an A / B Test

Filters

Shipping forms

A/B Test Statistics

The A / B Test is a technique used by marketing and communication teams to optimize the content of a campaign before impacting the entire user base.

It consists of sending the same campaign with different formats to a small group of users (control group). After some time (approximately 24 hours) the results of all formats are compared, and the most successful one is sent to the entire user base (audience). Avoid sending the control group so that they do not receive the same campaign twice.

The indigitall platform offers this functionality so that this process is very simple and comfortable , so that you can use this A / B test technique regularly in your day to day, and that in this way you can optimize your campaigns based on real data of user response.

The creation form is very similar to creating a normal push campaign. The difference are:

You must upload at least 2 different creatives for the same campaign.
In the shipping methods tab you can find:
- Smart shipping . Automatic method
- Manual shipping . You must specify the number of devices that will be part of the control group. After a while (about 24 hours), you must enter the console to see which format has been more successful to send it to the rest of the unaffected audience.

Versions

Create an A / B Test

The creation of each notification is the same as when you create a new campaign.

You can find more detailed information on how to create push campaigns at this link .

In the usage panel you will see that there are two options, A and B, although you have up to 5 options / versions C, D and E that you can use at the same time in the same campaign, just by clicking on the plus icon / symbol . To delete a version, the one you want to delete must be selected and then click on the minus symbol

During the test each device receives only one notification, regardless of the number of versions. Some users will receive notification A, others B,…, but in no case two versions of the same campaign.

Versions can be shipped via manual delivery or smart send.

Filters

With the filters you can segment to whom to send notifications in a personalized way.
Choose the country or region, if they are users who have not entered your website or app for a long time, or by the type of device that subscribed to your notifications.

Filters affect the number of devices that will receive the final notification.

Exception:In the case of campaigns that are sent automatically, the counter will not show a result of the number of devices to which it will be sent

In the following video-tutorial we will show you how to apply these filters:

Groups of interest

You can find detailed information about the Stakeholders tool at this link .

Devices subscribed to : the campaign will be sent to devices that are subscribed to a specific topic.
Devices that are not subscribed : the campaign will be sent to devices that are not subscribed to the specific topics.

External applications

You can find more information about the External Applications tool at this link .

Send campaigns to devices that have applications such as Facebook, Twitter or Spotify installed.

List of devices

You can establish a list of device IDs, so the campaign will be sent to the list made up of these devices.

You must upload a .csv file with the word deviceld in the first line and a deviceld for each line.

DeviceIds are device identifiers of end users. It is an anonymous data, generated by indigitall when a new device is registered. It looks like this: xxxxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx

Find out how to get these identifiers with this video:

Example:

list.csv

deviceId
6abcf613-e4e6-xxxx-a5d1-9533fe655e61
69b26254-aacd-xxxx-98c3-525fc7bff440
89b99158-2144-xxxx-b1df-d298649f934b
...

You can also use your own identifiers (for example, email, user ID in your CRM, etc) to create the list in .csv. In this case, you must load the .csv file with externalId as header (first line) and your identifiers in each of the other lines.

Example:

list.csv

externalId
user1@email.com
user2@email.com
user3@email.com
...

Geographic filters

This filter allows you to send geolocated notifications. You will send the campaign to the devices that are within the specified areas.

In this video tutorial we explain how to make shipments by selecting the areas:

Frequency of use

This option allows you the possibility of sending the campaign to devices that have not visited the application in a certain period of time. Also to users who have done so.

Type of device

When you use the device type filter, this option will allow you to select between mobile phone, tablet and desktop computer. You can also select in which browsers you want your campaign to be shown (Safari, Opera, Edge, Firefox, Chrome).

Delivery methods

Smart Shipping

Smart shipping is a unique product from indigitall that optimizes the testing process for different versions of the same campaign using our machine learning algorithm, Smart Test .

Our algorithm will progressively and automatically send one of the versions to all devices. Depending on the success of the versions, the one with the highest conversion rate will send more users.

The longer the test lasts, the better the math model will learn and send the most successful version to more devices. Notifications will be sent in a sale of 2 to 7 days in a constant and gradual way in daytime, depending on the number of total devices and hourly filters.

A/B Test Statistics

By clicking on the statistics button in the section [Campaign> Push Test A / B] (https://console.indigitall.com/tools/testab), you will be able to check the notifications sent and how many have been clicked by each version and device : web, android, iOS as the total.

Statistics with Smart Shipping

On the left is the detail by campaign and in the last line the total of the campaign:

Received : Number of devices that have received it. It can be less than those sent if there has been an error in the reception of a sent push.
Received : Percentage of received with respect to the total received of the campaign.
Clicks : Number of devices that have clicked.
Clicks : Percentage of clicked with respect to the total received.

On the right is the total for the campaign:

Devices : Total number of devices
Sent : Total and percentage of devices sent out of the total devices in the campaign.
Clicks : Total and percentage of devices that have clicked with respect to the total number of devices in the campaign.

When clicking on a version, a pop-up window appears with the details for each device (web, android, iOS) and the total CTR (Click Through Rate), number of notifications clicked among those received.

Statistics with Manual shipping

If you click on the statistics button, you will see the campaigns that have been sent and the number of clicks that have been made in each of them. You will be able to see which of the campaigns you have used for testing are the ones that have obtained the best results. Through CTR (Click Through Rate) with this metric we will be able to know how many clicks have been obtained from the total impressions launched and consequently which has been the campaign with the best data.