ABConvert provides you with the ability to conduct real-time split testing of your store's shipping rates and free shipping thresholds. We use Shopify third-party carrier services to generate these shipping rates and this will be automatically be created when you launch your first test. The rates displayed at checkout will depend on the customer's test group, cart value, and delivery country. In addition, ABConvert rates can be shown alongside other dynamic services such as USPS or DHL, as well as any shipping rates that have been configured in your Shopify shipping zones.
🎯 In order to run shipping tests we utilize third-party carrier rates. This requires your store must be subscribed to the Advanced Shopify or Plus plan. If you are currently on the Basic or Shopify plan, you can opt to pay Shopify a monthly fee to add this feature or switch to yearly billing to have it enabled for FREE. If this situation applies to you, please contact Shopify Support for assistance.
If you're unable to verify carrier rates within ABConvert and you're confident that you have third-party carrier rates activated please contact [email protected]
Step 1: Activate Carrier Rates
To begin a new shipping test in ABConvert, click on "Create New Test" located at the top right of the screen. When running the shipping test for the first time, a message will appear at the top, prompting you to check if your Shopify store supports carrier rates. Please refer to the above information for further details about carrier rates.
After the carrier rate check is complete, your store will be able to begin running Shipping Tests.
Make sure you double check if your carrier rate is successfully enabled in the Settings -> Shipping and Delivery.
📌 We normally recommend you remove the origin rate during the testing period in order to prevent conflict.
However, if you have multiple rate such as Standard v.s. Express. You just have to remove the rate that you are testing.
Remember, do name the Shipping rate different than the existing rate (the rate you have not removed), otherwise Shopify will always return the rate with lower price if there are rates with the same name.
Step 2: Create Your Test
Once in the Shipping Test page you're presented with all the options required to get your shipping test up and running. We'll walk through the different blocks available for configuration in this section.
Test Information
This block contains the test name which is used to easily reference this test and analytics in the future as well as the hypothesis (description) for what you're hoping to achieve. This cannot be changed once a test has started.
Traffic Percentage
This section allows you to configure what % of traffic will see which variant.
We recommend leaving this on 50/50 to get the most accurate results.
🎯 Initially you might notice a significant disparity in the amount of traffic between the two variants, but over time, it will even out to within 1 - 2% of the predetermined percentage.
Shipping Test Type
ABConvert offers two types of tests currently, namely the Free Shipping Threshold and Flat Rate Shipping.
Free Shipping Threshold test is useful when you want to compare the performance of different thresholds, such as whether offering FREE Shipping over $120 results in better AOV and unchanged CVR compared to offering FREE Shipping over $80.
On the other hand, Flat Rate Shipping is useful when you want to test the maximum shipping prices that you can charge while maintaining the same CVR. This test can also be combined with a free shipping threshold to keep the customer experience unchanged from your current setup.
Variants
In the variants section, you can specify the free shipping threshold for each group if you choose the free shipping threshold test type, or set the flat rate shipping rates if you choose the flat rate shipping test type.
The values are defined in your store's default currency.
Shipping Countries
The shipping countries section is where can you choose which country your test will be active for. Please note that all checkouts with products in a selected shipping country will be part of the shipping test.
🎯 Although you can conduct multiple shipping tests concurrently, each active test is restricted to one test per country. This ensures that we obtain the most precise outcomes for every test and country you're evaluating.
✨ Need to test individual shipping profiles & zones instead? Reach out to support using the chat button on the bottom right to join the wait list for this feature
Shipping Configuration
Before launching your shipping test, the Checkout Configuration section needs to be set up. This section allows you to customize how the rate is displayed at checkout by choosing a shipping rate name and creating a personalized shipping description. In addition, ABConvert is the ONLY shipping test app to offer the unique feature of setting a custom transit time to measure the effects of shorter or longer shipping times.
At the bottom of this step, a checkout preview is available to show you precisely what will be displayed during checkout and what rates might appear depending on the test group.
Here is an example configuration:
That's it! Your shiny new test is all setup and ready to go. Just hit the next button at the bottom and we'll start providing shipping rates. If there are any errors we'll also let you know prior to the test going live.
Delete shipping rate
Shopify currently returns only one rate with the lowest price if there are multiple rates with the same name. You will need to delete the rate you are testing to ensure accurate shipping.
You can either delete your rate manually in your "Shipping and Delivery" settings in Shopify and add the rate back after you end the test, or you can select the rate to delete in our final review modal. Our app will delete the rate and recover it for you after you close the test.
⚠️ Multiple rates might be displayed, or your test rate might not appear if you do not delete the rate you are testing.
Preview the test
After create a shipping test, you can navigate to the analytics page and click summary to preview the test. Or you can follow the below instruction.
🎯 Not seeing the right experiment on your store? To provide a fast and seamless experience for your customers we cache experiment data for a period of up to 5 minutes. This won't affect customers' checkout capabilities. If you'd like to test it yourself without waiting, simply append ?preview=0&country=<coutry code> or ?preview=1&country=<coutry code> to your URL bar, allowing you to preview the ongoing experiment.
🏁 Ready to level up your shipping test?
Check out these resources: