EN |
Intelligent A/B test
automatically test your notifications
Index
The A/B Test is a technique used by marketing and communication teams to optimize the content of a campaign. It consists of testing different versions of a campaign to see which one is the best.
The indigitall platform offers a 'intelligent' version based on Machine Learning of this functionality so that this process is very simple and comfortable , so that you can use this A / B test technique regularly in your day to day day, and in this way you can optimize your campaigns based on real data of user response.
Versions: Create an A/B Test
The creation of each notification is the same as when you create a new campaign.
You can find more detailed information on how to create push campaigns at this link .
In the usage panel you will see that there are two options, A and B, although you have up to 5 options / versions C, D and E that you can use at the same time in the same campaign, just by clicking on the plus icon/symbol . To delete a version, the one you want to delete must be selected and then click on the minus symbol.
During the test each device receives only one notification, regardless of the number of versions. Some users will receive notification A, others B,…, but in no case two versions of the same campaign.
How Intelligent Send Works
The method we use is known as Multi Armed Bandit testing, MBA, inspired by the algorithm of Tompson Sampling.
While A/B tests with manual send are based on exploration, collecting data to see which is the best version and making a decision, MBAs are designed for exploitation, showing the version that seems best while testing continues .
Our algorithm progressively and automatically sends one of the versions to all devices. During the first hour we will make four shipments with all the versions and thus be able to analyze which one seems better. From the second hour every fifteen minutes and depending on the success of the versions, the one with the highest conversion rate will be sent to more users.
The longer the test lasts, the better the math model will learn and send the most successful version to more devices. Notifications will be sent in a sale of 1 to 7 days in a constant and gradual way in daytime, depending on the number of total devices and hourly filters.
A/B Test Statistics with Intelligent Send
By clicking on the statistics button in the section [Campaign> Push Test A/B] (https://console.indigitall.com/tools/testab), you will be able to check the notifications sent and how many have been clicked by each version and device : web, android, iOS as the total.
You will get a view like the following:
On the left is the detail by campaign and in the last line the total of the campaign:
- Received : Number of devices that have received it. It can be less than those sent if there has been an error in the reception of a sent push.
- % Received : Percentage of received with respect to the total received of the campaign.
- Clicks: Number of devices that have clicked.
- % Clicks: Percentage of clicked with respect to the total received.
On the right is the total for the campaign:
- Devices : Total number of devices
- Sent : Total and percentage of devices sent out of the total devices in the campaign.
- Clicks: Total and percentage of devices that have clicked on the total number of devices in the campaign.
When clicking on a version, a pop-up window appears with the details for each device (web, android, iOS) and the total CTR (Click Through Rate), the number of notifications clicked among those received.
Status of an A/B Test
Created tests can be in one of these states:
-
Completed - Has been sent to all devices in the campaign.
-
Sent: Manual send of all versions has been completed.
-
In progress : The campaign is being sent by intelligent sending.
-
Canceled : The test has been canceled.
-
Error: An error has occurred.
-
Pending : The test has been created successfully but the intelligent send has not started yet.
Resources
Intelligent Test: Artificial intelligence to increase your sales