Frequently Asked Questions

A/B testing (also called split testing) compares two versions of a page element to determine which performs better. Visitors are randomly assigned to see version A (control) or version B (variant), and their behavior is measured to find a statistically significant winner.

No. TestLift is built for merchants, not developers. Its step-by-step test creation wizard and dashboard let you create and launch tests without editing theme code.

TestLift installs on standard Shopify plans. For theme compatibility, Dawn is the validated baseline and official support is limited to Shopify's free themes. Other themes may work, especially with custom selectors, but they are not officially supported.

Open TestLift in Shopify admin and click Create Test. Choose a buy button or a homepage slideshow campaign, set Variant B, split traffic, and activate. Then enable the TestLift Tracking app embed in your theme customizer.

No. TestLift does not compare Shopify product variants or change product catalog offerings. Price tests, layout tests, and product-gallery reorder are not included.

Use TestLift's full promotion guardrail set before picking a winner: at least 1,000 unique exposures, at least 7 days of runtime, expected loss below 2.00%, and probability to win at or above 95%.

TestLift uses a theme app embed and Web Pixel extension to count a visitor only after the tested element is actually present (buy button or homepage slideshow), then attribute add-to-carts and purchases to that sticky assignment. Storefront tracking uses pseudonymous visitor identifiers rather than named customer profiles.

Probability to Win is a Bayesian metric representing the likelihood that a variant is the true best performer. A value of 95% means there's a 95% chance the variant is genuinely better than the alternatives. It's more interpretable than p-value-style significance and pairs with TestLift's other promotion guardrails.

Run tests until every promotion guardrail is met: 1,000+ unique exposures, 7+ days live, expected loss below 2.00%, and 95%+ probability to win.

TestLift adds a small, deferred storefront script (about 8KB brotli across two files) plus a Web Pixel. That is extra storefront work, not zero cost. Image tests do not hide the section first, so visitors may see the original image for about 0.5–0.9 seconds before Variant B appears. We do not claim zero storefront cost.

Look at four decision inputs together: (1) Probability to Win at or above 95%; (2) Uplift vs control; (3) Expected loss below 2.00%; and (4) enough data/time (1,000+ unique exposures and 7+ days).

TestLift is built for the tests that move a Shopify store: buy-button copy and homepage slideshow campaigns. It is not a full CRO suite — no price tests, layout tests, or product-gallery reorder, and no conversion-lift guarantee.

When Shopify sends the uninstall webhook, TestLift deletes shop-owned app records from its database and clears cached subscription state. Shopify may also send a follow-up privacy redaction webhook, which runs the same cleanup path again safely. Theme extension code is managed through Shopify, so if you need the storefront state verified after uninstall, confirm it in your theme/app-embed settings as part of your uninstall process.

Yes. TestLift offers a free plan with up to 3 active tests, Bayesian reporting, and automatic guardrails. Upgrade to Pro ($29/month) for unlimited active tests and priority email support.

Add to cart, Checkout, and custom purchase-button copy, plus homepage slideshow campaigns: a new image plus optional heading and copy, as one test and one assignment. Slideshow is the primary homepage path.