Test a Shopify theme change¶

Test a change to your Shopify theme by running a Two URLs experiment: set your live page as Control page (A) and a preview of the changed theme as Variant page (B). Squoosh sends synthetic shoppers to both and reports which converts better. Squoosh does not generate the variant — you supply the changed theme.

The run wizard has no theme picker. You compare two URLs, so the work is to get a stable preview URL for the changed theme, then point an experiment at it.

Before you start¶

• A Shopify theme that contains the change you want to test, kept unpublished so your live store is unaffected.
• A preview or share URL for that theme. In Shopify, duplicate your live theme, edit the copy, and use Preview or Share preview to get a URL that loads the changed theme. Confirm the URL opens the modified page in a browser before you use it.
• Your live page URL, for the control.

Run the test¶

1. In the sidebar, click Experiments.
2. Click New experiment.
3. For how you ship variants, keep Two URLs.
4. In Control page (A), enter your live page URL — the page as customers see it now.
5. In Variant page (B), enter the preview URL for the changed theme.
6. Set the conversion goal: Add to Cart or Checkout.
7. Set the traffic profile: Organic / Direct or Paid Ad (PDP).
8. Leave the audience on Standard (1,000 synthetic shoppers). This is the recommended sample.
9. Review the credit cost and your balance after the run, then click Launch experiment.

Squoosh runs both pages, compares the shopper outcomes, and files a verdict. See How Squoosh decides a winner.

Read the result¶

A run ends on its own once a winner holds at Squoosh's confidence bar, and never before part of the sample has finished, so you don't set a stop time. When the run finishes, the verdict is one of three outcomes:

Keep the original is a real result. If the theme change costs you conversions, Squoosh tells you to keep the live theme rather than framing the loss as a win. Squoosh never publishes or applies a theme — rolling out the change in Shopify is your decision. For how to read the numbers, see Reading lift and confidence.

Comparing two templates within one theme¶

If your change swaps the template a page renders — for example a default product template versus a long-form one — you can compare two templates within the same theme using each template's ?view= URL.

This is an opt-in feature in early validation. Turn it on under Settings → Features → Shopify template tests, then run a Two URLs experiment with one template's ?view= URL as the control and the other's as the variant. For most theme changes, the standard preview-URL approach above is the proven path.

Connect Shopify to calibrate shoppers¶

Connecting Shopify is optional for running the test, but it makes the result more accurate. With a store connected, Squoosh calibrates the synthetic shoppers against your real customer behavior so the outcome better reflects your audience.

1. In the sidebar, click Integrations.
2. On the Shopify row, click Install app.
3. In the Connect Shopify dialog, enter your Store domain — a store slug or a full *.myshopify.com domain — then click Install app. Squoosh redirects you to Shopify.
4. In Shopify, approve the requested permissions and install the app. Squoosh requests read access to read theme metadata and generate preview URLs for experiments.
5. Back in Squoosh, the Shopify row shows Connected to your store.

After connecting, you can switch the calibration source to Shopify Analytics from the connected Shopify row. A property uses one calibration source at a time: enabling Shopify Analytics disconnects Google Analytics, and enabling Google Analytics disconnects Shopify Analytics.

Synthetic shoppers stay out of your analytics. By default, Squoosh hides synthetic traffic from tools like Shopify Analytics and GA4, so your real conversion data stays clean. You can review this under Settings → Analytics & Privacy.

Troubleshooting¶