{"id":1266,"date":"2026-11-04T08:40:00","date_gmt":"2026-11-04T08:40:00","guid":{"rendered":"https:\/\/cotocus.cn\/blog\/error-budget-questions\/"},"modified":"2026-11-04T08:40:00","modified_gmt":"2026-11-04T08:40:00","slug":"error-budget-questions","status":"publish","type":"post","link":"https:\/\/cotocus.cn\/blog\/error-budget-questions\/","title":{"rendered":"Error Budget Questions Teams Ask, Answered With Stories"},"content":{"rendered":"<p>Sooner or later, every engineering room produces the same uncomfortable scene. The release calendar is on the screen, somebody has just apologised for the third outage of the quarter, and a voice from the back asks the question everyone has been circling for months: how much failure are we actually allowed? That question has a name \u2014 the error budget \u2014 and the teams who learn to answer it stop having that awkward meeting altogether. What follows are the questions engineering teams really ask about error budgets, answered the way they are best understood: through the stories of teams who asked them first.<\/p>\n<h2>What is an error budget, really \u2014 beyond the textbook line?<\/h2>\n<p>A payments team we will call the Monday Crew learned it the hard way. They had promised their business stakeholders &#8220;maximum reliability,&#8221; which in practice meant every incident triggered a tribunal and every engineer learned to deploy as rarely as possible. Then a new lead reframed the deal. The service would aim to succeed 99.9% of the time, measured monthly \u2014 and the missing 0.1% was not shame, it was property. Theirs to spend. Roughly forty minutes of trouble a month could be invested in bold releases, experiments, even scheduled risk. The room changed that day. The budget was simply the gap between the target they chose and perfection, treated as a resource instead of a sin. Nothing about the system changed that afternoon; everything about the decisions did.<\/p>\n<h2>Why would we set a target below 100% on purpose?<\/h2>\n<p>Because the Monday Crew&#8217;s old promise was a lie, and everyone knew it. No system humans build succeeds every time, so a 100% target does not produce perfection \u2014 it produces hiding. Their incident reports before the change read like legal defences; afterwards, they read like engineering. There is also arithmetic underneath the honesty. Each extra fraction of dependability costs disproportionately more \u2014 redundant everything, slower releases, weekends sacrificed \u2014 while users below a certain threshold cannot even perceive the difference. Somewhere there is a level of reliability that your users genuinely need and your business can genuinely afford. Naming it out loud is not lowering the bar. It is the first time the bar has ever been real.<\/p>\n<h2>How do we pick the number without guessing?<\/h2>\n<p>A retail client&#8217;s story answers this best. Their first instinct was to copy a big platform&#8217;s figure, five nines, because it sounded professional. Their consultant asked one question: what does a minute of checkout failure cost you at peak, and what does it cost at 3 a.m.? The answers differed by a factor of a hundred. So the target followed the money \u2014 strict during trading hours for the checkout path, relaxed for the recommendation widget nobody would miss for an hour. The craft of choosing such targets, and writing them into service-level objectives a business will actually sign, is teachable; it sits at the centre of what <a href=\"https:\/\/sreschool.com\/\">SRESchool&#8217;s consultants and corporate training programmes<\/a> work through with teams, precisely because the number has to be negotiated between engineering and business rather than decreed by either. The honest method is always the same: start from user pain and revenue, not from what sounds impressive.<\/p>\n<h2>What happens when the budget runs out?<\/h2>\n<p>This is where most teams flinch, and where one logistics team became a legend in their company. Their agreement said: budget exhausted, feature releases stop, everyone works on stability until the measure recovers. In November, their busiest season, a string of bad deploys spent the whole allowance by the 9th. The product manager demanded an exception. The engineering lead pointed at the signed agreement and \u2014 this is the part people remember \u2014 the chief executive backed the agreement. Two weeks of hardening followed: retries fixed, alerts pruned, one flaky dependency replaced. December shipped more features than any month that year, because the platform finally held still under them. A budget with no consequence is a poster. The consequence is the mechanism.<\/p>\n<h2>Doesn&#8217;t this slow down delivery?<\/h2>\n<p>The teams who measure find the opposite, and the reason is visible in any crowded deploy pipeline. Fear is slower than any process. Before explicit budgets, the Monday Crew shipped monthly because every release was a trust fall. With a healthy budget on the wall, they shipped daily \u2014 small changes, cheerfully, because a stumble was affordable and the rollback was rehearsed. Automation amplifies the effect: once routine deployment and checks run without hands, each release spends less of the budget, a dynamic familiar to anyone who has read about <a href=\"https:\/\/cotocus.cn\/blog\/streamline-your-workflow-how-digital-automation-saves-time-and-reduces-errors\/\">taking manual steps out of workflows to reduce mistakes<\/a>. The budget does not ration speed. It rations recklessness, and the distinction is the whole game. There is even a tell for which side of the line a team is on: ask how they feel about Friday afternoon deployments. Fear of Fridays means releases are trust falls. A glance at the remaining budget before answering means the team has replaced superstition with accounting.<\/p>\n<h2>We run on cloud infrastructure \u2014 isn&#8217;t reliability the provider&#8217;s job?<\/h2>\n<p>A startup founder asked exactly this, gesturing at his provider&#8217;s impressive uptime promises. His consultant pulled up the incident log: of the last ten user-facing failures, nine originated in the startup&#8217;s own code, configuration or release process. The provider&#8217;s platform had kept its promises; the application dancing on top of it had not. Cloud platforms give you remarkable raw material \u2014 anyone who has studied <a href=\"https:\/\/cotocus.cn\/blog\/managing-massive-web-traffic-through-modern-cloud-infrastructure\/\">how modern cloud setups ride out enormous traffic surges<\/a> knows how much heavy lifting they do \u2014 but your users experience your service, not your vendor&#8217;s data centre. The error budget measures the whole journey, which is why it catches what infrastructure promises miss.<\/p>\n<h2>Is this only for big companies with dedicated reliability teams?<\/h2>\n<p>The smallest team in these stories was four people building a custom web application for a single client. They wrote one sentence into the project agreement: the booking flow succeeds 99.5% of the time each month, and if it does not, new feature work pauses until it does. No tooling beyond a scheduled check and a spreadsheet. That sentence did the work of a reliability department \u2014 it set expectations with the client, settled every &#8220;should we ship this risky thing on Friday&#8221; argument, and made the maintenance retainer renew itself. Teams scoping such projects do well to raise the question early, alongside the usual list of <a href=\"https:\/\/cotocus.cn\/blog\/questions-to-ask-before-building-a-custom-web-application-the-complete-guide\/\">things worth asking before a custom application gets built<\/a>. The practice scales down beautifully, because the practice is a decision, not a platform.<\/p>\n<h2>Where do we start on Monday?<\/h2>\n<p>The same place every team in these stories started: one service, one measure a user would recognise, one target you can defend in front of the person who pays for the system, and one written consequence for exhaustion. Track it for a month before you automate anything. Expect two surprises \u2014 the measure will be worse than you believed in one place and better in another \u2014 and expect the first budget-exhaustion argument to be the moment your team discovers whether the agreement was real. The teams that hold the line once rarely have to hold it again. After that, the uncomfortable scene from the opening of this piece simply stops occurring \u2014 not because outages end, but because they arrive pre-negotiated. Somebody still breaks something on a Tuesday. The difference is that the room already knows what it costs, who decides what happens next, and that the answer was agreed while everyone was calm. That, in the end, is all an error budget is: the argument you had once, in writing, so you never have to have it during an incident again.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What is an error budget, who picks the number, and what happens when it runs out? Real questions from engineering teams, answered through their stories.<\/p>\n","protected":false},"author":4,"featured_media":1267,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[330],"tags":[],"class_list":["post-1266","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"_links":{"self":[{"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/posts\/1266","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/comments?post=1266"}],"version-history":[{"count":0,"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/posts\/1266\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/media\/1267"}],"wp:attachment":[{"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/media?parent=1266"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/categories?post=1266"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cotocus.cn\/blog\/wp-json\/wp\/v2\/tags?post=1266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}