Split Testing for Autoresponder Emails in Arigato PRO Intelligence Module

This feature requires at least Arigato PRO 2.5.1.5 and Intelligence module 1.0.8.

It allows you to split test different versions of emails and see which perform best. The best place to figure this out is the new “Advanced stats” page introduced recently.

To enable the feature select the following checkbox in the Add / Edit Email Message page:

split-testing

The feature is not available for newsletters.

Few things to note:

If you select “Enable split testing” the email will be included in split testing with other emails that have ALL of the following satisfied:

  • They are in the same autoresponder campaign
  • They have “Enable split testing” selected
  • They are of the same type (i.e. all emails are of type “Send X days after registration”, or all are of type “Send every X days” etc)
  • They have to be sent in the same day
  • The “time of the day” setting is the same

Emails based on individual user events (enabled by custom profile fields) can not be included in split testing.

The split test will deliver each email to approximately equal number of subscribers.

How exactly does it work:

When the autoresponder runs it will select all autoresponder emails that have to be sent today. If there are 8 receivers and 3 split testing emails for the day, they will be spread approximately equal between the 8 receivers so the first 3 receivers will receiver email A, the next 3 – email B, and the last 2 – email C.

Split testing does not guarantee equal number of receivers for each email in the split test over time.  For example if there are 3 emails in a split test but there is only one receiver today, he will receive email A. If the same happens tomorrow, the receiver will also receive email A, and not B or C. Split testing takes effect only on the given day so it’s recommended for larger mailing lists where at least several members are expected to receive given email in a given day.