How to Decide Your Rollback Path Before a Web Release Ships

How to Decide Your Rollback Path Before a Web Release Ships

Release day goes smoothly for most teams, most of the time. The days that are remembered are the other ones: the deploy that looked fine in testing and began throwing errors under real traffic, with half the team asking whether to push a fix forward or go back. This how-to gives you a practical way to decide your fallback before the release, so that when something goes wrong the switch is a short, calm procedure instead of a debate. The steps suit a small web team as well as a larger one.

Step 1: Name what the release depends on

Start with a plain list of the things this particular release touches or relies on. For a typical web change it will include the application code, the database schema, a configuration value, one or two external services and the content delivery or caching layer. Do not aim for completeness in an architectural sense. You want the dependencies that, if they behave unexpectedly, could hurt users.

Mark the ones that cannot easily be undone. A schema change that drops a column is a very different animal from a code change that can be redeployed in minutes. Those hard-to-reverse items are where the fallback matters most.

Step 2: Choose the fallback for each risky item

For every item on the list, write the specific alternative. Examples of fallbacks in a web release include:

  • Redeploy the previous build from the artefact store, with the exact version recorded.
  • Switch a feature flag off, leaving the new code present but inactive.
  • Route traffic back to the old environment if you run two side by side.
  • Serve a cached or static version of a heavy page if the backend struggles.
  • Use a second provider for an outside service, or disable the feature that calls it.

When a change cannot be undone in place, such as a data migration, the fallback may be to make it additive first: add the new column, keep the old one, and remove it in a later release. That way, going back remains possible.

Step 3: Write a trigger that is measurable

A fallback without a trigger is a good intention. The trigger should be something you can read from a dashboard, a log or a user report without interpretation. Examples:

  • The error rate for a key endpoint stays above an agreed level for ten minutes.
  • Response times on the checkout path pass an agreed ceiling for five minutes.
  • Two or more support reports about the same failure arrive within half an hour of release.
  • A health check fails on more than one server after the rollout.

Agree the numbers with product and support colleagues beforehand. People are surprisingly willing to accept a threshold when it is set in a quiet meeting, and surprisingly reluctant to accept one set in the middle of an incident.

Step 4: Decide who calls it

One person, named in advance, can order the switch. Make a second person the deputy in case the first is unavailable. The rule is not that they must act alone; it is that nobody wastes five minutes asking who has authority. Put both names in the release note next to the trigger.

Step 5: Check the fallback path itself works

A rollback that has never been exercised is a theory. Before the release, confirm that the previous build is still stored and deployable, that the feature flag really turns the feature off for users, that database credentials needed for a revert are available to the person who might use them, and that monitoring will show you whether the fallback fixed the symptom. This takes ten minutes on a calm morning and removes the worst surprises. For ideas about the kinds of questions worth settling before a build begins, the guide to what a team should ask itself prior to building a bespoke web app makes a handy companion.

Step 6: Release in a way that keeps the fallback cheap

Smaller releases make plan B simpler, because there are fewer moving parts to unwind. If the rollout can start with a thin slice of visitors, the trigger fires on a limited audience and the fallback affects only them. Time of day matters too: a release at the start of the working day, when the people who understand the system are at their desks, makes a calm switch more likely than a late-evening push.

Automation helps here, particularly for the repetitive parts. An overview of how workflow automation trims wasted time and mistakes covers the general idea; the point for rollback is that a scripted revert is far less prone to slips than commands typed by a tired person after hours.

Step 7: Tell people what the fallback is

Put a short note in the place where the team watches the release: the trigger, the fallback, the owner and the way to confirm the result. Support and customer-facing colleagues should know too, so that they can describe a brief reversal as planned behaviour rather than a mystery.

Step 8: Review the switch, whether or not you used it

After the release, take five minutes to ask whether the trigger was right, whether the fallback would have worked, and whether anything should change next time. If you did switch, record how long it took and what slowed it down. If you did not, note that the safety path was ready, so the team keeps trusting the process. Over several releases, the plans improve almost on their own.

A worked example

Imagine a small team shipping a new search feature that calls a third-party service and adds an index to the database. Their list has three risky items: the new service, the index build, and the front-end change.

For the service, the fallback is a feature flag that hides the new search box and restores the old one. For the index, they build it a day earlier, outside the release, so it cannot surprise them. For the front end, the fallback is redeploying the previous build, which they confirm is stored. The trigger: if the search page returns errors for more than a small fraction of requests for ten minutes, the flag goes off. The release manager decides, and the lead developer is the deputy.

On the day, the service slows under load. Nine minutes in, the numbers are borderline. At ten, the flag goes off and the old search returns. The postmortem takes five minutes: the threshold was reasonable, the service needs a timeout, and the next release will include one. Nobody had to improvise.

Common mistakes to avoid

  • Treating “we can always redeploy” as a plan without checking that the old build exists.
  • Setting a trigger that depends on someone’s mood, such as “if it feels bad”.
  • Choosing a fallback that needs the same database or service that just failed.
  • Forgetting that configuration and data changes may not roll back with the code.

A note for the team away from the keyboard

Release weeks are long, and people sometimes plan their downtime as carelessly as their deployments. If your team wants something to do together afterwards, such as a workshop or an evening out, MyEventNow is an events-listing website with pages for upcoming events and venues, plus a blog and a forum, so it is a quick place to browse options, and the habit of picking a second option is the same one you used for the release.

Quick reference

  1. List dependencies and flag the ones that are hard to undo.
  2. Pick a concrete fallback for each risky item.
  3. Write a measurable trigger and agree on it with colleagues.
  4. Name the person who decides, and a deputy.
  5. Exercise the fallback path before the release.
  6. Prefer small, staged rollouts.
  7. Share the fallback with everyone who might be asked about it.
  8. Review afterwards, even when everything went well.