Difference between revisions of "Split A-B Testing"
(Created page with " This page will describe a specific type of site/tour redirect, that can be used to accomplish "A/B" testing. Here is the scenario - suppose on one of your sites you have a...") |
TmmStephen (talk | contribs) |
||
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
| − | + | {{NATS4 Manual | |
| + | | show_sites_admin_section = true | ||
| + | }} | ||
This page will describe a specific type of site/tour redirect, that can be used to accomplish "A/B" testing. | This page will describe a specific type of site/tour redirect, that can be used to accomplish "A/B" testing. | ||
| Line 17: | Line 19: | ||
This will prevent any affiliates from grabbing any link codes that go directly to this tour. Instead, they will only be able to get linkcodes for the tour A, the traffic for which then will be split 50-50 "behind the scenes" with tour B, which will be accomplished by the redirect we are going to set up next. | This will prevent any affiliates from grabbing any link codes that go directly to this tour. Instead, they will only be able to get linkcodes for the tour A, the traffic for which then will be split 50-50 "behind the scenes" with tour B, which will be accomplished by the redirect we are going to set up next. | ||
| + | '''NOTE: Please ensure that the new tour created is allowed into all programs that may use the redirect.''' | ||
The third step is, you will need to go to your Sites Admin link, and then click the action icon for "Configure Redirects": | The third step is, you will need to go to your Sites Admin link, and then click the action icon for "Configure Redirects": | ||
Latest revision as of 15:01, 30 November 2015
This page will describe a specific type of site/tour redirect, that can be used to accomplish "A/B" testing.
Here is the scenario - suppose on one of your sites you have a tour that seems to be working pretty well for you, but you would like to know if a different tour may work even better. The way to accomplish this would be to create a redirect that sends 50% of the traffic from the one tour (tour A) to the another new tour (tour B), and then after letting both tours run a for a little while, check your stats to see which tour had performed better for you/your affiliates. (NOTE: The second tour does not actually need to be a new tour, but it will make it much easier to calculate the stats if it is a brand new tour. Likewise, making a new tour for use as tour A will also make it easier to calculate the stats). If either the intended tour A or tour B already exists (or if they both already exist), you can copy those tours to new tours (with the same tour URLs), and use these new copies of those tours for the split A / B testing.
Since we want all of the tour B traffic to only come from what is being split by the 50-50 redirect, the first thing we need to do is make that tour B hidden. To hide your second tour (i.e. tour B), go to the sites admin and edit the tour B that we will be hiding the link codes for. Under the "Tour Details" section, click the orange link for "(Show/Hide Advanced Details)":
Next, check the Hide Tour checkbox to hide that tour's link codes from affiliates.
This will prevent any affiliates from grabbing any link codes that go directly to this tour. Instead, they will only be able to get linkcodes for the tour A, the traffic for which then will be split 50-50 "behind the scenes" with tour B, which will be accomplished by the redirect we are going to set up next.
NOTE: Please ensure that the new tour created is allowed into all programs that may use the redirect.
The third step is, you will need to go to your Sites Admin link, and then click the action icon for "Configure Redirects":
Once you click the above icon, you will see any previous redirects (in the example below there already is one) that you have already created for this site, as well as a form to "Add New Redirect":
Setting the correct values in the above "Add New Redirect" form
This is what you will set in the "Add New Redirect" form above for each of the following fields:
- New(Site) Tour: Set this field to the tour name that you are using as tour B (i.e. this is the DESTINATION of the redirect).
- New Program: Leave this set to "No Change " (If you really wanted to, you could change the program also, but all of the same affiliates would have to be enabled in the new program as in all programs that send traffic to tour A, cascade and join option rules would have to allow these programs as well, and the affiliates may get upset that they are receiving a different payout than what they expected with the tour A program).
- Redirect: Click Use New (Site) Tour checkbox.
- Active for: Set this to 50% of traffic.
- Track redirect: Check this box, because you will want to track the traffic that is going through this redirect.
- Enable by default: DO NOT Check this box, because that would mean that the redirect would be enabled by default, which in turn would mean that half of any traffic to this site, from any tour, would go tour B (e.g. You could have 10 other tours for this site, but you only want to split the traffic between tours 11 and 12 as the 50-50 A/B test). So instead, leave it unchecked and we will use rules in the next step to allow this redirect to work only for traffic that comes from tour A.
- Click "Add This Redirect" once you are done filling out the required fields.
NOTE: As of NATS release 4.1.10.1, there will also be available an extra check box on the above redirect form for "Disable Original Hit", which if checked, will disable recording a hit on a tour that is being redirected away from. Without this new feature being checked, the total number of hits get counted on tour A. For example, without this new feature being checked, if 100 visitors go to tour A, and half of those end up going to tour A but the other half end up being redirected to tour B, the stats will record this as 100 hits on tour A and 50 hits on tour B. But with this new feature being checked, it ignores the total number of hits that were initially for tour A, and only records those that actually ended up there after the redirect. For example, with this new feature being checked, if 100 visitors go to tour A, and half of those end up going to tour A but the other half end up being redirected to tour B, the stats will record this as 50 hits on tour A and 50 hits on tour B.
Now that you have configured your new redirect, you will then have to add a rule for this newly added redirect. We now will need to add a Rule to make it redirect traffic, but only from Tour A. Click the rules icon for this new redirect:
Once you clicked on the rules icon for this redirect, it will open a Rules form as shown below. Since this redirect was not active by default, we are going to set an "Enable" rule, but only for traffic that comes from tour A (everything else in this Rule will will leave set to ALL):
Click the "Add" button on the far right to save this rule. That's it, you're done! Once you have done this, you will be left with just the one affiliate link code for tour A, and this redirect you have created will now randomly redirect 50% of your surfers from tour A to tour B.
And then after you have allowed your tours to split up the traffic equally on both tours for a while, you will be able to see the individual stats for each tour or site in the reporting admin. This can help with seeing which of your tours is better at converting referred surfers into members, as well as seeing what surfer/member behaviors are associated more with each tour or site.
(Please note, there also other ways to accomplish A/B testing, such as with different billers, where you could set up multiple cascades with the same exact description, and if they both have the same weight, the cascades will be split 50-50. And there are also Program Redirects available).
