Meridian Cargo is a fictional company, but every detail in this account is drawn from patterns that real platform teams describe almost word for word. Meridian runs a freight-booking platform: eleven microservices, three clusters, a twelve-person engineering group, and a delivery pipeline assembled over five years by sensible people in a hurry. This is the story of the quarter they spent moving that pipeline to GitOps — what pushed them, what they weighed, what broke, and what the dashboards said afterwards.
The finding that started Meridian’s GitOps conversation
The trigger was not an outage. It was a credential review. A security audit walked the auditors through the build server and found that the continuous-integration system held long-lived credentials for all three clusters — production included — because deployment meant the pipeline pushing changes outward from outside. The audit report quoted a line from NIST’s SP 800-204C that the team had read before and never fully absorbed: when clusters are changed by pushing from CI, the pipeline effectively holds the keys to production, and the standard goes as far as calling such push-based models insecure. Nobody had stolen anything. The exposure simply existed, patiently, on a build box that ran untrusted workloads every day. The engineering lead put it plainly in the retrospective: “Our security perimeter includes Jenkins.” That sentence allocated the quarter’s budget.
Weighing the options before adopting GitOps
Three architectures went onto the whiteboard. Option one: keep pushing from CI, tighten the credentials. Option two: adopt the pull model, with an agent that fetches declared state from the repository, but keep every sync as a deliberate human click. Option three: pull with automatic reconciliation — the agent not only fetches but continuously corrects the cluster toward the record. Rather than argue from taste, the team scored the options against weighted criteria — security exposure, auditability, rollout speed, and operational burden — using a comparison format published for exactly this decision. Their multi-team, ten-cluster scenario reproduced the published arithmetic almost exactly: pull with self-heal scored 4.05, pull with manual sync 3.70, push from CI 2.80. The lead also flagged the honest footnote: for a four-person startup, the same comparison lands in a near tie — 3.55, 3.60, and 3.50 — because a tiny team’s coordination costs differ from a twelve-person group’s. Meridian was not a startup. The self-healing pull model won, and the consultants they pulled in to sanity-check the plan agreed, on the condition that deletion would be guarded from day one.
The pilot: two sprints in staging
The pilot ran on the staging cluster and produced a small list of lessons that no document had fully conveyed. Bootstrapping came first: the agent was installed so that it managed its own configuration, meaning upgrades to the reconciler itself flowed through the same reviewed process as everything else. The team ran the first fortnight with syncs applied manually, watching drift alerts arrive and learning which differences were legitimate — autoscaler decisions, webhook-filled fields — and which were genuine divergence. They wrote their first architecture decision record in years, choosing the repository layout, the secrets approach, and a drift policy per resource class. By the end of the second sprint the agent had been granted the least access that let it function, scoped through application projects and impersonated service accounts rather than blanket cluster-admin, and the pipeline’s cluster credentials had been deleted — an act the security team described as the single best line item of the year.
What GitOps changed in the numbers
Meridian measured a full week of production traffic after the cutover, and their figures landed close to the published lab example that had shaped their plan: merged pull requests reached healthy workloads in a median of about 3.75 minutes, while the p90 landed at 26 minutes — the tail dominated by a database migration that, everyone agreed, had no business being fast. More revealing were the second-order effects. Out-of-band changes, previously invisible, now surfaced as alerts within one polling cycle, and each one either got reverted or got a ticket. The repository’s history became the first document consulted in every incident review. Two engineers independently reported that they had stopped keeping private notes about “how things are actually set up”, because the actual setup was, for the first time, the written one.
One scope decision deserves recording, because it saved the quarter from ballooning. Meridian deliberately left release strategy out of the migration. Canaries, feature flags, and staged rollouts remained owned by their existing release tooling, since GitOps answers “what should run?” while progressive delivery answers “how should a new version reach users?” — adjacent questions with separate toolkits such as Flagger and Argo Rollouts. Blurring them had been proposed in an early design session and quietly dropped once someone asked who would be paged when the flag store and the reconciler disagreed. Deciding what a migration will not attempt turned out to be as valuable as deciding what it would.
The incident that taught the team about suspension
Weeks later, a storage incident put the new model under real pressure. While the on-call engineer worked inside the cluster to stabilise the database, the reconciler kept trying to restore declared fields the responder had deliberately changed. The team’s runbook covered this: reconciliation was suspended — in their Flux-based setup the suspension survives precisely because it is not declared in Git — the responder worked, and afterwards the manual changes were entered through pull requests so the record caught up with reality. The lesson they added to the runbook afterwards concerned Argo CD colleagues at another business unit: there, a suspended parent application will happily undo a manual fix unless the changed fields are marked ignored, so “suspend” alone is not a complete instruction. Automation is obedient to a fault; responders need explicit, rehearsed ways to borrow authority from it briefly and hand it back.
What Meridian would tell a team considering GitOps
The quarter ended with a retrospective whose headings transfer well beyond freight booking:
- The security argument alone justified the migration, before any velocity benefits arrived.
- The weighted comparison is worth running with your own numbers — the arithmetic shifts with team size, and the footnote matters as much as the headline.
- Pilot on staging, sync manually at first, and let drift alerts teach you where your documentation ends.
- Guard deletion before enabling it; prune disasters are cheap to prevent and expensive to explain.
- Rehearse suspension and re-entry during daylight, not during an outage.
Those five headings now hang, laminated, beside the desk of the engineer who wrote “Our security perimeter includes Jenkins” — a sentence nobody on the team expects to say again.
Teams planning their own migration can go far deeper with xOps School’s GitOps handbook entry, which is where Meridian’s lead sent every engineer on the team before the first design session. The broader context also helped them: the same cloud patterns that let the platform absorb heavy web traffic through modern cloud infrastructure made the reconciler deployment straightforward, their earlier work on automation that saves time and reduces errors had already built the review habits GitOps formalises, and the security team framed the credential cleanup as an extension of protecting revenue and trust through application security. Fictional company, real playbook — and a delivery pipeline that now holds no secrets anyone can steal.
