Publishing Dangerous Ideas

Two Stanford researchers have written and published a paper suggesting that it would be easy for terrorists to injure or kill hundreds of thousands of people by putting botulism in our milk. The paper was published in the Proceedings of the National Academy of Sciences after extensive review and consideration. It's accompanied by an editorial written by Academy president Bruce Alberts explaining why it was published. This link points to both the article and the editorial.

Alberts argues that the article doesn't contain any technical information that's not readily available through Google; that it's very useful to let other scientists check the work and think of solutions to the problem; and that publication of the dangers can show people how to avoid them.

In a sense, this is playing a deadly serious game called "How smart are the terrorists?" If they have never thought about putting botulism in our milk, or have wished they could do it but haven't thought through how little effort it might require or how many people it might kill, then perhaps we shouldn't be giving them ideas. (By that argument, Forbes's article discussing the problem might in fact be compounding the problem by spreading the idea.)

On the other hand, if we do nothing, eventually they will think of it, and we will be unprepared because we have done nothing. (I agree with Alberts that keeping reports like this in secret government files is likely to result in poor decision-making.) If we publish the idea, then we are engaging in a contest of wits, and on our own terms.

A few commentators on our blog have questioned whether CRN should be publicizing the detailed technical information that we do, explaining exactly why and how molecular manufacturing could lead to massively disruptive advances in technical capability. But talking about molecular manufacturing does not have the same potential short-term downside; no hostile power can act on the idea quickly. So we have more time for the contest of wits. And the key technical ideas were published more than a decade ago. Doing nothing is essentially guaranteed to lead to the implementation of the idea -- but without any chance to prepare for the impact of the technology.

We will continue to question, for each piece of technical research we do, whether it is wiser to publicize it or keep it quiet. But so far, we have always decided that publishing it is our most responsible option.

Chris Phoenix

Comments

[michael vassar]

As I pointed out at the WTA meeting, modeling terrorists as simply evil will fail to predict their actual actions. A look at historical data shows that anti-utilitarians don't actually exist, yet many transhumanists, policy makers, and scientists continue to evaluate the dangers of technological possibilies as if they did. No-one is trying to maximize fatalities. People are playing silly little games based on primate politics shaped by the media. Big explosions! Not little germs.
A look at the ineptitude with which a few groups have attempted to use germs should highlight this.
So should a consideration of the actual motivations that might lead to terrorism (especially suicide terrorism). 70 virgins whether you take out 1 evildoer or 1,000,000. Same feeling of self-rightous anger and group membership too.

[Chris Phoenix, CRN]

Well, there are several kinds of terrorists. Terrorists are people who kill random civilians outside of the rules of war. But it's not always only fueled by the motivations you list (anger, membership, virgins). If someone wanted to seriously hurt the U.S., poisoning a few hundred thousand people with a warm-and-cuddly food like milk would certainly do the job.

Hm... though it would also lead to a severe crackdown on illegal immigration and undocumented farm workers, which would make it harder to sneak in thousands of hostile people as is currently happening. It's nice to think that terrorists don't really want to hurt us, but ask yourself: what do they need 10,000 hostiles for that they couldn't do with 1,000? Is it only to spread their ideology to the grass roots in the US? Or are they planning something really big and really nasty?

Chris