A/B testing is a great way to figure out which of different subscribe form designs perform best by attracting most subscribers per impression. And since the visual designer in Arigato Gozaimasu provides you a lot of possible designs – regular inline forms, responsive pop-ups, sticky top or bottom bars, or animated slide-ins – it’s great to know what works best for you. Now you can do it, with the A/B testing tool that the module provides.

Creating A/B Tests

To create A/B tests for a mailing list you need to create at least 2 form designs for that list using the Gozaimasu visual designer. You can access the visual designer through your Mailing Lists page under column Subscribe Form. The link is just under the simple subscribe form shortcode:

Once you have created and saved at least two designs you will see the A/B tests link next to the Save Design button:

It will also be shown on the Mailing Lists page next to the Visual designer link:

Clicking on the link will take you to the page with A/B tests created for that mailing list. A/B tests are always specific to a mailing list. Initially you will see no tests (because you have to create them first). Creating a test is very simple, it just requires name and selecting which of the existing designs you want to test. Include at least two designs:

That’s it. Save your test and when you go to the previous page you will see it along with its shortcode:

The shortcode of the test is what you need to publish on a page or inside a widget on your site to get the test running. Important: stats will not be recorded for the A/B if you use the individual design shortcodes provided on the Visual designer page. You should use the A/B test shortcode.

It ensures equal rotation of all the designs included in it and calculates stats that will show you which design performs best.

Viewing and Interpretation of Results

Once your A/B test start getting impressions you will see a table showing the performance of the different designs tested:

Since the tool ensures equal number of impressions for each tested design, the most important metrics you need to look at are the percentages – signups / impressions and active / impressions. Here “active” means the subscribers who have confirmed their email addresses so it’s probably correct to assume it the most important stat you need to know. The design that has the highest percentage of active / confirmed subscribers per 100 impressions is the best performing design.

We would recommend waiting for at least few hundred impressions per design before making conclusions. Once you decide on the best design you can replace the A/B test shortcode on your pages with the specific design shortcode.