We had 12 experiments running at once. None of them shipped. Each was half-built and still got the odd hour on a Friday, so nothing ever properly died. The bets weren't the issue — we just never let one fail on purpose.
So we added a rule: before starting anything, write down what would prove it worth keeping. A metric, a threshold, a date. It sounds small. It cleared out eight of the twelve inside a quarter.
The bar goes down before the work starts
The fix is a decision made before the work, not after. Before writing a line, you write down what would prove the experiment worth continuing, in three concrete parts:
Experiment: in-app booking flow
Bar: >= 40 completed bookings/week from >= 15 distinct users
Deadline: 2026-08-01 (90 days)
Kill rule: below bar on the date -> pause, capture learning
The bar has to be able to fail. "See if users like it" was one of our twelve, and it's not a bar — there's no world where it misses, so it never got killed, it just sat there consuming Fridays. "40 bookings a week from 15 users by August 1st" can miss. That's the point of writing it that way.
Setting the number itself is where people flinch, because a number you can defend feels arbitrary and a number that's obviously safe defeats the purpose. We used a rule: the bar is the level at which the experiment is worth more engineering time than the next-best thing we could build instead. For the booking flow, below 40 completed bookings a week the feature wasn't earning its maintenance cost, let alone new investment. That's not a marketing target, it's the break-even where continuing beats stopping. Pick the number where the honest answer to "would we start this today knowing it performs at exactly this level?" flips from yes to no.
We ran two of the twelve without a real deadline at first — "we'll check in a month or so" — and both slid. "A month or so" is not a date; it's a deferral with a friendly face. The ones with a hard calendar date got decided on that date. The ones with a soft date got extended, every time, for exactly the reason the next section covers.
The clock is what forces the call
Without a date, "let's give it more time" wins every review, forever. There's always a reason next month will be different. We proved this to ourselves twelve times over.
The deadline removes the debate. On the date you check the number against the bar. Above it, keep going. Below it, it pauses or winds down. The call was made 90 days earlier, before anyone was attached to it, and now you just execute it. A test set up front, with no sunk cost in the room, is a fairer judge than you standing over three weeks of your own code.
Capture the learning on the way out
A killed experiment isn't wasted if you extract what it taught you first. The wind-down was a short write-up: what we bet, what the number actually was, why it missed, what we'd do differently. Then the code got archived and the surface area went away.
Here's the real one from the booking flow:
Outcome: 12 bookings/week, 7 users — well below the 40/15 bar
Why: discovery worked, but checkout friction killed completion
Keep: the discovery flow tested well, reuse it
Kill: this checkout — wrong payment integration for the market
That failed experiment still produced something durable. The next attempt started from "we know checkout friction is the wall" instead of rediscovering it, and it reused the discovery flow wholesale. Killing something without writing up why is the real waste. You already paid for the lesson, so keep it.
The write-ups compound across experiments in a way we didn't expect. Three of the eight we killed had a Why line that came down to some version of "we built for a user who wasn't the one who showed up." Reading three write-ups in a row surfaced that pattern, and it changed how we scoped the next batch — we validated who the actual user was before building the flow, not after. One write-up is a post-mortem. Eight in a folder is a map of how this team specifically tends to be wrong, and that's worth more than any single experiment that generated an entry.
The archive step matters as much as the write-up. A killed experiment that stays in the codebase, half-wired, still shows up in the router, still has a nav entry someone might click, still breaks a build when a shared dependency changes under it. We move the code out of the tree entirely — it lives in git history if we ever want it — and delete the routes and nav entries the same day. A kill that leaves the surface area behind hasn't actually stopped costing you; the dead code still breaks builds and still eats the Fridays you were trying to reclaim. Deleting it the same day is what makes the kill real.
Simple counters miss the tail
One trap in picking the metric: a raw count hides its own shape. We nearly kept a different experiment on the strength of "200 bookings this month," which sounded like it cleared the bar. Then we broke it down and 180 of the 200 came from 3 accounts, with the other 20 spread across everyone else. Three superfans and no traction. The count passed; the reality failed.
So we baked distribution into the bar — not "40 bookings a week" but "40 from at least 15 distinct users." The second number catches the tail the first one hides. Whenever the metric is a sum, ask what a healthy distribution behind it looks like and put that in the bar too. A raw total almost always flatters the tail, so make the bar check both.
There's a second way a raw count lies, and it's the opposite direction: a metric that passes the bar on borrowed traffic. One experiment cleared its number the week we linked it from the homepage. Pull the homepage link and the number would have collapsed, because the traffic was borrowed, not earned. We added a rule alongside the distribution check — the bar has to be met under normal conditions, not during a one-time push. If a launch email or a homepage feature is carrying the number, extend the window and measure the week after the boost fades. What you want to know is whether the thing holds up on its own, and a spike from a promotion tells you nothing about that.
The general shape: any single number can be gamed by its own composition. Sums hide concentration, spikes hide dependence on a push, week-of-launch numbers hide novelty decay. For every bar that's a single metric, we now ask what a passing number could be hiding, and add the second check that would catch it. It's usually one extra clause — "from N distinct users," "sustained over two weeks," "excluding referral traffic" — and it's the clause that keeps a flattering number from buying another 90 days.
The kill switch is what lets you ship
Setting a date on which you might kill your own work feels harsh. In practice it's the reason the team ships at all — the not-working experiments get cleared and the attention concentrates on the few that are moving. We went from 12 half-built experiments to 4 real ones, and two of those shipped.
The switch is a fair test agreed up front. You clear a number you set yourself, or you stop and keep the learning. Either way you're not babysitting a half-built thing on Fridays for the rest of the year.