Theme CI: testing a Shopify theme before it ships

A theme is not an application, and pretending otherwise produces a pipeline nobody trusts. There is no unit test for a section rendering correctly at 375px with a 40-character product title, and there will not be one. What CI can do for a theme is narrower and still worth a great deal: it can catch the errors a linter catches, it can put a working preview in front of a human on every change, and it can stop a change that makes the store measurably slower.

That is the whole scope. Everything else — does this look right, does the merchandising make sense, does the mobile layout survive a long title — is a person looking at a page. CI’s job is to make sure that person is looking at the right page, quickly, without asking anyone to build them one.

It also has a second job that only becomes obvious after the first incident: it stops the store being deployed to by hand. Most theme disasters we are called about are not bad code. They are a theme push run from a laptop against the wrong store, or a live theme overwritten with a branch that was three weeks behind.

Before any pipeline, settle the awkward fact underneath Shopify theme development: the repository is not the only thing writing to the theme.

Shopify’s GitHub integration connects one branch to one theme, and the sync runs both ways. Pushing commits updates the theme; editing the theme in the admin — theme editor, code editor, or a theme app — produces commits back onto the branch, made by a Shopify bot, with edits saved within about ten seconds of each other batched into a single commit. That sync cannot be disabled. Only folders matching the default theme structure sync, and anything else in the repository is ignored.

The consequence is a genuine ordering problem. Two writers, no locking. The documentation is blunt about who wins in a conflict: in the code editor, "the version of the file in the code editor overwrites the GitHub version of the file", and a commit coming from Shopify can be rejected by GitHub as outdated when both sides move at once.

One more thing worth knowing before you start: a branch cannot be reconnected to a theme after it has been disconnected. Reconnecting adds it as a new theme instead. Plan the branch-to-theme mapping once rather than experimenting with it on a live store.

So choose the model deliberately. Either the integration is on and you accept that merchandisers commit to your repository, or it is off and CI is the only thing that writes to the store. Both work. Discovering which one you have during an incident does not.

This is the stage that changes how a team reviews, and it is the least work of the three.

shopify theme push --unpublished creates a new unpublished theme and pushes to it. Add --json and the command returns the result as JSON, including the preview URL, which your workflow can post as a pull request comment. Reviewers open a link and look at the store. Nobody sets up a local environment to check a spacing change.

Authentication is the part people get stuck on. Shopify documents the CI path clearly: generate a password from the Theme Access app and pass it as SHOPIFY_CLI_THEME_TOKEN, set the store in SHOPIFY_FLAG_STORE, and set SHOPIFY_FLAG_FORCE=1 if the step is timing out because the CLI is waiting for a prompt that will never come. The documentation is explicit that the password should be masked or stored as a secret — it is a credential with write access to your storefront.

One habit worth adopting early: give the throwaway theme a name that includes the branch and the build number, and delete it when the pull request closes. A store carrying forty abandoned preview themes is its own small problem, and the theme list is where a nervous person looks during an incident.

Shopify publishes a Lighthouse CI action for exactly this. It runs Lighthouse audits against a theme on pull requests and reports back as a status check, and it takes a store, credentials, and optional product_handle and collection_handle so the audit covers a product and a collection page rather than only the home page — which is the version of this check that actually tells you something, because the home page is rarely the template a customer lands on.

The thresholds are the point. lhci_min_score_performance defaults to 0.6 and lhci_min_score_accessibility to 0.9, both on a 0 to 1 scale, and both are the numbers to argue about as a team. A budget nobody agreed to is a budget somebody will disable the first time it goes red. Authenticate with the action’s client_id and client_secret inputs rather than a legacy custom app token, and note that the tokens it fetches are valid for 24 hours, so this is not a credential to cache.

Two honest caveats. Lighthouse in CI is a lab measurement on a machine that is not a phone on a train, so treat it as a regression detector rather than a score to optimise. And it will not see the apps that only load on the live theme, which is a large share of real-world storefront weight. The field data in Shopify’s own web performance reports — Largest Contentful Paint, Interaction to Next Paint and Cumulative Layout Shift from your real traffic — is the check on whether the lab number means anything.

One more tool worth knowing even though it does not belong in CI: shopify theme profile profiles Liquid performance on a given page. When the Lighthouse number moves and nobody knows why, that is where to start, because it distinguishes a slow template from a slow page.

If you have none of this today, build it in this order: preview on pull request, then lint, then the performance budget. That is deliberately not the order of increasing rigour. Previews change reviewer behaviour on day one and cost an afternoon; lint produces a backlog of findings on an inherited theme that somebody has to triage; the performance budget is only useful once the first two have stopped the obvious regressions.

Leave visual regression testing alone unless you have a specific, repeated failure it would have caught. Screenshot diffing a storefront where prices, stock badges and recommendations change hourly generates false positives faster than anyone will triage them, and a check people routinely override is worse than no check, because it teaches the team that red is normal.

The place this stops being a theme problem is when the pipeline needs something the CLI does not do: seeding a test store, reconciling data between the theme and a back office, or exercising an integration that the storefront depends on. That is application work with its own deployment story, and it belongs in a [custom app or back end](/services/integrate/custom-shopify-apps-and-back-ends) with a real test suite rather than bolted onto a theme workflow. Knowing where the boundary is saves a lot of time spent trying to make a theme pipeline behave like a product one.

• Custom Shopify apps and back ends
• Liquid performance: the loops that cost you a second
• Theme architecture: settings, blocks and the limits of each