Data tracking is a big part of A/B testing. As a user-focused app, ABConvert implements a well-designed tracking system to help our merchants track their A/B testing results without placing any code.
How we track data
Currently, we track data in the following ways:
Webpixel
ABConvert script
Webhook
WebPixel
Shopify has provided us with an API to collect its events, and you can find the list here.
In ABConvert, we collect four events in WebPixel:
page_viewedproduct_added_to_cartcheckout_startedcheckout_completed
Our analytics are session-based for price test / shipping test / checkout test, and visitor-based for URL redirect test / template test / theme test.
The session will be the same as Shopify's, and it will last 30 minutes. Each event will only be recorded once in a session.
The visitor is counted once for a purchasing journey. Each event will only be recorded once for a visitor.
The WebPixel will be created once a test is created, and it will remain as is if there are active tests.
You can find the WebPixel in Shopify admin → Settings → Custom Events → ABConvert
ABConvert script
In addition to WebPixel, we use a script to track data that we can't get from WebPixel.
For URL redirect test, template test and theme test, we use scripts to track events such as sessions / products views, staying time, bounce events, click through.
Webhook
We use Webhook to track orders for all test types.
Why is my analytics different than Shopify’s data
A common question from users is: Why is the numbers in the ABConvert analytics dashboard different from Shopify? While it may seem like data should match across platforms, the reality is that tracking methods differ significantly, which lead to discrepancies. Below, we outline the primary reasons for these differences.
1. Path-wise data in Shopify vs. store-wide data in ABConvert
This mostly happens in the URL redirect test. Some users will take a look at the session and other data in a certain path in “Sessions by landing page” and compare it with ABConvert’s analytics dashboard.
However, the comparison might not be meaningful since Shopify records sessions only on the landing page, while our analytics tracks the following user behavior.
2. Differences in Visitor and Session Definitions
Definition | ABConvert | Shopify |
Visitor |
| A visitor is defined as a unique individual who interacts with the store. Visitors are deduplicated across sessions and devices. |
Session | A session lasts 30 minutes. It resets after 30 minutes (no matter if there is activity or not) or when a new browser session begins. | Sessions are defined as 30 minutes of continuous activity, resetting after inactivity or new browser sessions. |
3. Differences in Event Tracking Methods
Definition | ABConvert | Shopify |
Event Recording | Events (e.g., add-to-cart, checkout) are recorded at most once per session to prevent double counting. Each visitor or session can only attribute one event per funnel step. | Events are recorded based on Shopify’s predefined tracking logic, which may include multiple events within a single session. |
Data Collection Methods | Uses web pixels, JavaScript scripts, and webhooks to collect data directly from user interactions during tests. | Relies on its analytics infrastructure to collect data from various store interactions and channels. |
Data Collection Limitation | Checkout event tracking: if customers use shop pay or express checkout, the events cannot be tracked in ABConvert due to the exclusion of web pixel event. | N/A |
4. Inclusion/Exclusion of Bot Traffic
ABConvert employs anti-bot measures to ensure only real visitor data is recorded. Events like product views or add-to-cart actions are only captured if they are tied to genuine user interactions.
5. Privacy Settings and Browser Restrictions
Modern browsers like Safari block third-party cookies and certain JavaScript trackers by default. Additionally, privacy tools like ad blockers can prevent analytics scripts from firing. This will result in discrepancies in conversion metrics.
ABConvert uses first-party scripts and web pixels for tracking but may still encounter issues with blocked events in some environments (e.g., TikTok’s in-app browser).
Why is there order discrepancy between ABConvert analytics and Shopify’s report
Difference in attribution logic:
ABConvert: Attributes orders to a test if the visitor interacted with a tested product page at any point during their session. This includes cases where the final purchase contains products that were not part of the test. The logic is based on the idea that the tested experience influenced the visitor's behavior during their shopping journey.
Shopify: Attributes orders directly to specific products or collections included in the purchase. It does not consider whether the visitor interacted with a tested product page earlier in their session unless that product is part of the final transaction.
For example:
If a visitor views a tested product page but purchases an unrelated item later in their journey, ABConvert attributes that order to the test group because the tested experience influenced their behavior.
Shopify would not attribute this order to the tested product unless it was directly included in the purchase.
Multiple sales channels contribute to orders
ABConvert only records analytics if it happens on the Online Store sales channel.
For orders from shop, subscription renew, Pinterest or any other non-online store source, it won’t be recorded. Moreover, if there are post-purchase upsell apps, we might not record them as well.
Recommendations for Users
To minimize confusion when comparing analytics:
Understand how each platform defines key metrics like sessions, visitors, and orders.
Use ABConvert’s analytics dashboard to focus on metrics specifically designed for A/B testing performance evaluation, such as: AOV and conversion rates.
For deeper validation, export raw data from both platforms (e.g., order IDs) and compare manually.
If you encounter discrepancies that seem unusual or unexplained, feel free to contact our support team for assistance via the messenger!