π° Price test use cases
The goal of price testing is to find a product's optimal price point for maximum revenue.
Should I price my products higher or lower?
The most basic question you might want to answer with A/B testing is if you can:
Increase the price without hurting demand
Decrease the price and increase enough sales volume to profit.
You can simply run a price test by increasing and decreasing a product's price by a certain amount or percentage to verify this.
Example scenario
Group | Product A price |
Control group | $ 100 |
Test group 1 (-10%) | $ 90 |
Test group 2 (+10%) | $ 110 |
Potential outcomes and actions
The main metric to focus on is the Revenue and Conversion Rate (CVR). Consider choosing the price point with the highest Revenue. You can also observe how change in price will influence the amount of add to carts, checkouts and orders.
If you do not see a clear change in CVR, this might mean that customers are less price sensitive to your product. Thus, you can try experimenting with larger price changes. For instance (+- 30%).
Should I match my competitor's price?
If your product is entering a competitive marketspace, you can research the competing product's price. After understanding competitor prices, run a price test to experiment with:
Pricing above competitors to aim for a higher profit margin.
Pricing below competitors to aim for capturing more market share.
Price matching competitors to defend market share or build brand awareness.
Example scenario
You have 2 main competitors who prices their products at $100 and $120. You want to test if you can capture more market share by pricing lower or matching competitors.
Group | Product A price |
Control Group | $ 80 |
Test Group 1 | $ 100 |
Test Group 2 | $ 120 |
Potential outcomes and actions
The main metric to reference for decision making is Conversion Rate. Additionally, monitoring the Average Order Value and Profit per Visitor can provide insights into the financial impact of each pricing strategy, allowing you to balance market share and profitability.
Price anchoring
By setting up different "compare at" prices, you can test if displaying a higher anchoring price point alongside the actual price will influence consumers' perceived value to increase conversion.
Example scenario
You want to see if a higher "compare at price" will also bring higher Order %. Also, you want to find a balance where the "compare at price" is high enough to create perceived value but not so high that it appears unreasonable or deceptive.
Product Price | $ 100 |
Control Compare at Price | $ 120 |
Test Compare at Price | $ 150 |
Potential outcomes and actions
The main metric to reference for decision making is Conversion Rate (Order %) .
If a higher "compare at price" increased Order %, consider adjusting to the test price.
If a higher "compare at price" decreases Order %, it may be perceived as unreasonable. Lower the anchor to align better with customer expectations and market standards.
Remember to run price tests for a sufficient duration to gather statistically significant data. ABConvert recommends a minimum of 10,000 visitors in total and 200 orders across all test groups for reliable results. Always consider the broader impact of price changes on customer perception, brand positioning, and overall business strategy.
β
π¦ Shipping test use cases
The goal of shipping testing is to optimize shipping strategies to maximize conversion rates, average order value (AOV), and overall revenue.
Determining free shipping threshold
You might want to find the optimal free shipping threshold that encourages customers to increase their cart value without significantly impacting your profit margins. This test can help you understand how different thresholds affect customer behavior.
Example scenario
Group | Free shipping threshold |
Control group | Free shipping for orders over $30 |
Test group 1 | Free shipping for orders over $50 |
Test group 2 | Free shipping for orders over $75 |
Potential outcomes and actions
The main metrics to focus on are Conversion Rate, Average Order Value, and Profit per View.
Lower threshold:
Outcome: Higher conversion rate but potentially lower AOV.
Action: If the increase in conversions offsets the reduced shipping revenue, consider implementing this threshold.
Medium threshold:
Outcome: Balanced conversion rate and AOV.
Action: This might be the optimal point if it maximizes overall revenue while maintaining a good conversion rate.
Higher threshold:
Outcome: Lower conversion rate but higher AOV.
Action: If the increase in AOV significantly boosts overall revenue despite lower conversions, this threshold could be beneficial.
Optimizing flat rate shipping
Testing different flat rate shipping prices can help you find the highest price customers are willing to pay without negatively impacting conversion rates.
Example scenario
You want to test if increasing your flat rate shipping price impacts your conversion rate and overall revenue.
Group | Flat rate shipping |
Control group | $5.99 |
Test group 1 | $7.99 |
Test group 2 | $9.99 |
Potential outcomes and actions
Focus on Conversion Rate, Profit per View, and the Rate Seen metric (which shows how often each shipping rate is viewed by customers).
If conversion rates remain stable across higher prices:
Consider implementing the higher shipping rate to increase profit margins.
If conversion rates drop significantly at higher prices:
Find the balance between increased shipping revenue and maintaining conversion rates.
If the Rate Seen metric is high but conversion rates are low:
Customers frequently see a higher shipping rate but don't complete the purchase, it might indicate that the rate is too high for their liking. Consider lowering the rate to potentially increase conversion.
Improving profitability with advanced shipping rate
For businesses with varied product weights or order values, testing with advanced shipping rates can help optimize shipping costs and maintain profitability.
Example scenario
You want to test different shipping rate structures based on weight.
Group | Shipping rate |
Control group | Flat $9.99 for all orders |
Test group | $5.99 for orders under 5lbs, $9.99 for orders over 5lbs |
Potential outcomes and actions
Monitor Conversion Rate, AOV, and overall Revenue. Also, pay attention to the distribution of order weights or values.
If the Test Rate better aligns shipping costs with actual shipping expenses:
This could help maintain margins on heavier items while keeping shipping attractive for lighter orders.
If the control performs best:
The simplicity of a flat rate might be preferred by customers. Consider whether the operational simplicity outweighs potential gains.
Remember to run tests for a sufficient duration to gather statistically significant data. Always consider the broader impact on customer experience and operational costs when implementing changes based on test results.
β
π§© Template test use cases
The goal of template testing is to find the optimal layout and design for specific pages, such as: the home, product, or collections page in your Shopify store to maximize conversions.
Optimizing product information
A/B testing the amount of information or the layout to convey the information on your product page can assist you in deciding the best product page template for maximum conversion.
Example scenario
You want to decide whether to use the Multicolumn or the Multirow section to showcase your product images and descriptions. You also want to optimize the copywriting and text format.
Group | Template |
Control group | Multicolumn |
Test group 1 | Multicolumn |
Test group 2 | Multirow |
Test group 3 | Multirow |
Potential outcomes and actions
Compare the Conversion Rate, especially the Add to Cart Rate between the test groups. Consider picking the combination with the highest performance in this metric. Also, refer to Avg. Staying Time to observe if engagement is increasing conversion or creating friction.
Deciding homepage layout
You want to test if adding more sections to your homepage, like featured products, testimonials, or a video, impacts engagement and conversions compared to a simpler layout.
Example scenario
Group | Template |
Control group | Image slideshow |
Test group 1 | Image slideshow |
Test group 2 | Image slideshow |
Potential outcomes and actions
Compare the Conversion Rate, especially the Add to Cart Rate between the test groups. Consider picking the combination with the highest performance in these two metrics. The clickthrough rate of different sections and CTA buttons should also be monitored to determine whether further testing on the copywriting or format of specific sections are needed.
Adjusting collection templates for product discovery
Example scenario
You want to test different collection page layouts to see which one leads to better product discovery and higher click-through rates to product pages.
Group | Template |
Control group | Number of columns on desktop = 4 |
Test group 1 | Number of columns on desktop = 6 |
Test group 2 | Number of columns on desktop = 6 |
Potential outcomes and actions
Compare the Conversion Rate, especially the Add to Cart Rate between the test groups. Also reference AOV to see if better product discovery leads to larger average order values. Consider picking the combination with optimal performance in CVR and AOV.
Remember to control the number of variables you include in each test to better determine the effect of each variable. Always consider user experience and the loading times of your online storefront when implementing the changes based on test results.
β
π¨ Theme test use cases
The goal of theme testing is to find the optimal design and layout for your Shopify store to maximize user engagement and conversions.
Consider running a theme test when:
You want to launch a big theme update and need to ensure that the update does not impact your business goals and customer experience.
You want to inspect the stability of a new third party app like PageFly, which you used their advanced elements to design the landing pages and product pages.
You need to confirm if a new mobile-optimized layout with improved navigation can increase the conversion rates on the mobile platform.
Simply choose the updated or adjusted theme as a variant to test against your original theme.
Potential outcomes and actions
Compare the Conversion Rate and Average Order Value (AOV) between the test groups. Observe if the variant theme can bring better performance in the Add to Cart > Checkout > Order funnel, and if the variant theme leads to better product discovery to increase AOV. Consider picking the theme with the highest performance in these two metrics.
Focus on maintaining a cohesive brand identity when testing theme changes. Ensure that any updates align with your overall aesthetic and messaging. Also, prioritize user experience and maintain optimal loading times to enhance customer satisfaction and drive conversions.
πͺ Visual editor test use cases
The goal of Visual Editor testing is to optimize text, images, and HTML elements directly on your live pages without touching the theme code or creating any duplicate.
Optimizing call-to-action (CTA) copy
Testing different CTA button text can significantly impact click-through and add-to-cart rates.
Example scenario
Group | Button Text |
Control group | Add to Cart |
Test group 1 | Buy Now |
Test group 2 | Get Yours Today |
Testing product descriptions and headlines
You can change text elements to see which messaging resonates best with your audience.
Example scenario
You want to test if a benefits-driven headline works better than a feature-driven one.
Group | Headline Text |
Control group | 100% Organic Cotton T-Shirt |
Test group 1 | Experience the Softest T-Shirt You'll Ever Wear |
Swapping images or banners
Visual Editor allows you to replace image URLs, letting you test different hero images or product photos without changing your theme.
Potential outcomes and actions
Monitor Conversion Rate and Engagement (Click-throughs). If a lifestyle hero image generates more clicks to product pages than a plain product image, consider adopting the lifestyle imagery across your store.
The Visual Editor is ideal for quick, high-impact changes. Ensure that any visual changes remain responsive and look good on both desktop and mobile devices.
β
π³ Checkout test use case
The goal of checkout testing is to improve conversion (Checkout Completed). You can hide, rename, or reorder the payment / delivery options to reduce checkout friction.
Renaming delivery methods
Testing different names for delivery methods can help improve clarity and customer satisfaction during the checkout process.
Example scenario
Group | Delivery method naming |
Control group | Standard Shipping |
Test group 1 | Economy Delivery |
Test group 2 | Fast & Secure Delivery |
Reordering payment options
Testing the order of payment options can help identify which arrangement leads to higher conversion rates and improved customer satisfaction.
Example scenario
Group | Payment order |
Control group | Credit Card, PayPal, Apple Pay |
Test group 1 | PayPal, Credit Card, Apple Pay |
Test group 2 | Apple Pay, PayPal, Credit Card |
Dynamic content block testing (Shopify Plus plan only)
Testing the impact of dynamic content blocks in the checkout process can help optimize user experience, increase conversion rates, and improve customer satisfaction.
Example scenario
You want to test which type of dynamic content block (e.g., testimonial, benefit, or trust badge) placed in the checkout summary section improves customer trust and increases conversion rates.
Group | Checkout block |
Control group | No additional dynamic content is displayed in the checkout summary section. |
Test group 1 | A testimonial block showcasing customer reviews is added to the summary section. |
Test group 2 | A benefits block highlighting free shipping and easy returns is added. |
Test group 3 | A trust badge block displaying secure payment icons is added. |
Potential outcomes and actions for checkout modifications
Focus on Conversion Rate, especially Payment Info Submitted and Checkout Completed to evaluate the effectiveness of changes in delivery method names and payment option order.
If changes lead to increased conversions or preferred selections:
Implement the more effective names or order across your store.
Monitor feedback to ensure the changes align with customer preferences.
If there is no significant change or negative feedback:
Analyze customer feedback for insights into any confusion or inconvenience.
Consider further testing with alternative names or orders, or revert to the original setup if necessary.
If changes reduce cart abandonment:
Highlight popular options prominently.
Ensure clear communication and seamless functionality for all options.
Prioritize clear communication and intuitive navigation throughout the checkout process to enhance user experience. Regularly review customer feedback to identify areas for improvement, ensuring that any changes lead to a smoother and more efficient checkout, ultimately reducing cart abandonment.
β
π URL redirect test use cases
The goal of URL redirect testing is to optimize page content, layouts, and user journeys by comparing different versions of pages through URL redirection. This allows you to test variations without modifying your existing pages.
Testing product detail pages (PDP)
You might want to test different product page layouts, content structures, or features to improve product understanding and conversion rates.
Example scenario
Group | PDP page content |
Control group | Standard product page with traditional layout |
Test group 1 | Enhanced page with video demonstrations |
Test group 2 | Simplified page with focused content |
Trigger conditions
Test group settings
Test 1: /products/product-name-variant-1Test 2: /products/product-name-variant-2
Potential outcomes and actions
Monitor key metrics like Conversion Rate and Average Order Value (AOV).
Enhanced Video Version:
Higher conversion but slower load times
Action: If conversion increase outweighs performance impact, implement video features
Simplified Version:
Faster conversions but lower average order value
Action: Consider implementing for mobile users or specific segments
Landing page optimization
You might need to test different landing page variations for marketing campaigns to maximize conversion rates and ROI.
Example scenario
Group | Landing page |
Control group | Standard landing page |
Test group 1 | Benefits-focused landing page |
Test group 2 | Social proof-centered landing page |
Trigger conditions
Test group settings
Test 1: /landing/summer-sale-benefitsTest 2: /landing/summer-sale-social
Potential outcomes and actions
Monitor key metrics like Conversion Rate and Revenue. Choose to implement the landing page version with the best performance of the metrics you are looking to improve.
Properly configuring trigger conditions and destination URLs will help ensure that redirects occur as intended. Additionally, leveraging URL matching patterns effectively can enhance the accuracy of your tests, allowing you to compare different page versions accurately. Monitoring key metrics such as conversion rates and average order value will provide insights into the effectiveness of the changes being tested.
β
βοΈ Audience filter use cases
ABConvert provides various audience filters for you to fine-tune your experiment design.
Do you want your test to run only in specific countries?
This option can restrict your test to specific countries. Consider using this option when:
You want to test localized pricing strategies Example: Test different pricing strategies in Canada and Australia to accommodate local purchasing power without affecting the U.S. market.
You want to optimize for cultural preferences Example: Run separate tests in Japan and Germany to tailor product page descriptions and promotions to cultural preferences to see if it impacts conversion and revenue.
You have a main market that you don't want to affect Example: Conduct tests separate to your main market to confirm if results are positive before launching changes to minimize interference with sales.
Do you want to set advance option to display test price based on audience traffic source?
This option can let you assign which variant visitors from different traffic sources will see. Consider using this option when:
You want to find the more effective ad campaign Example: Test variants for users coming from a Facebook ad campaign versus a Google ad campaign to determine which platform yields better ROI.
You want to measure social media engagement Example: Offer a special discount for users arriving from Instagram to assess the impact on conversion rates compared to other social channels.
You want to optimize email marketing Example: Compare purchase behavior of users from an email newsletter against those from organic search traffic to optimize email marketing strategies.
Do you want to set condition for new and returning visitors?
This option allows you to show different variants based on visitor status. Consider using this option when:
You want to optimize acquisition Example: Test variants specifically for first-time visitors to increase new customer conversion rates.
You want to test retention strategies Example: Offer special pricing to returning visitors to encourage repeat purchases and increase customer lifetime value.
Do you want to show your test on all devices, or desktop or mobile only?
This option lets you run device-specific tests. Consider using this option when:
You want to optimize for device-specific behavior Example: Test different prices on mobile devices where customers might be more price-sensitive due to comparison shopping.
You want to enhance device-specific layouts Example: Test a single-column product layout theme on mobile for better scrolling experience, while maintaining a multi-column grid on desktop where screen space is abundant.
You want to improve device-specific conversions Example: Test a streamlined checkout template on mobile with larger buttons and simplified forms, while keeping detailed shipping options visible on desktop.
ABConvert's advanced testing options provide powerful tools for creating highly targeted and controlled experiments. When implementing these features, it's important to consider your testing strategy holistically. While granular targeting through country filters, device types, and traffic sources can yield valuable insights, avoid over-segmenting your audience as this may extend the time needed to reach statistical significance.
For optimal results, we recommend focusing on one or two key variables per test and ensuring sufficient sample sizes in each segment. Remember that each targeting option you enable effectively reduces your total test audience, so balance the benefits of precise targeting against the need for quick, reliable results.

