The Promise Nobody Made

A builder warns that a machine can act beyond its permissions. A prospective buyer may hear extraordinary capability—and imagine that capability taking orders from them. The warning has acquired two claims it did not establish: that the machine can perform the buyer’s work, and that it will remain under the buyer’s control.

When capability is hard to measure, you may use possible damage as a substitute for demonstrated performance. A hazard label becomes a horsepower rating.

An admission of danger may seem more credible than a boast because it appears to cost the speaker something: sales, trust, or permission to release the product. That credibility can extend to claims the speaker never made. No secret marketing plan is required. The builder may be sincere, the risk real, and the resulting interest unwelcome.

A credible warning can genuinely provide evidence of capability. A system that breaches sophisticated defenses has demonstrated skills that could also be valuable in other tasks. But evidence of one ability is not proof of competence at another, and capability alone does not establish that a purchaser can control it.

A system may exceed its permissions because its controls are poor, not because its abilities are extraordinary. An employer may destroy a job simply by trusting inadequate software. Eliminating a department and doing its work are different achievements.

In politics, a voter may treat an opponent’s warning that a candidate could wreck institutions as proof that the candidate knows how to build better ones.

Someone expecting to suffer the damage may demand restrictions; someone imagining themselves in command may request a price. One difference is who expects to direct the danger and who expects to bear its costs.

A builder’s rising prestige need not mean the warning was ignored. Listeners may have accepted the danger and invented the guarantee: useful power, reliably under their command.