Every regulation that reduces one hazard creates or fails to prevent another, and the second effect is systematically less visible than the first, because the people harmed by an intervention that did not happen are statistical and the people harmed by one that did are individuals with names. Drug regulators have known this for fifty years under the heading of the invisible graveyard: a cautious approval process produces identifiable victims of approved drugs and unidentifiable victims of the delay, and the incentives facing the regulator are wholly asymmetric, because nobody is ever fired for a drug that was not approved. Nuclear regulation raised the cost of reactors and, over four decades, extended the operating lives of coal plants whose particulate emissions kill people continuously and unphotogenically. There is a serious literature on this and its conclusions are uncomfortable in every direction.
The AI version is already here, and it is not hypothetical. When a developer declares a model to have critical offensive cyber capability and restricts access to vetted testers, the immediate effect is that attackers with no such scruples — and with access to capable open-weight systems trailing the frontier by months — face defenders who have been denied the best available tool. [...] That is a genuine cost, borne by real organisations with real vulnerabilities, and it is invisible in exactly the way the drug-lag deaths are invisible.
The same structure recurs throughout. Restricting biological capability restricts the people doing vaccine work alongside the people doing the other thing. Slowing deployment in medicine has a mortality cost that nobody counts. Concentrating development in a few secured institutions, which is the safety-motivated configuration, means a world with less scrutiny and fewer independent researchers able to examine what is being built.
And now the reason this argument is usually deployed in bad faith: it is almost never quantified. It is produced as a conversational move — but have you considered the costs of not building it? — by people who have not attempted to estimate either side, and its function is to convert a demand for evidence into a stalemate between two unmeasured quantities. The honest version requires you to put numbers on both arms, which nobody does, and the dishonest version requires only that you gesture at the second arm and stop.
The right posture, I think, is the one drug regulators arrived at after their own decades of this: accept that both arms are real, insist that both be estimated, and prefer interventions whose costs are recoverable to those whose costs are not. A delayed approval can be granted later. A disclosure requirement that turns out to be unnecessary can be repealed. A capability released to everyone cannot be recalled, and a concentrated industry cannot easily be dispersed. That asymmetry — reversibility, rather than magnitude — is the only principled ground I know for asymmetric caution, and it is available to both camps, and neither uses it much.
