Bill Joy was interviewed recently by the New York Times about the risk of plagues started either by carelessness (such as the recent SARS outbreak from a Chinese lab) or by malice (he suggests that plague and flu genomes should not be publicized). He's also worried about 'gray goo', the possibility of small, self-contained, foraging, free-floating molecular manufacturing robots that could in theory eat the biosphere.
CRN shares some of Joy's concerns, but we respectfully disagree with his focus on runaway nanotech replicators. There are several reasons for this. First, the development of molecular manufacturing is perhaps our best defense against biological plagues. Cheap medical sensors and cellular-scale robotics can detect and treat disease in ways that no other technology can. And second, gray goo will be very hard to design and build. It won't happen by accident. And it doesn't even make a very good weapon: a tiny self-contained foraging/manufacturing system would be very inefficient compared to a larger specialized factory producing non-replicating weapons.
This brings us to the biggest danger we see in molecular manufacturing: its potential as a weapons manufacturing system, which can only be called spectacular. And this leads straight to an arms race -- one which, for many reasons, will be less stable than the nuclear arms race. This cannot be prevented by science policy. Scientists will generally build what they're told to build, if not in the U.S., then elsewhere. And anyway, scientists have frequently built destructive weapons for idealistic reasons.
So we believe that molecular manufacturing presents massive risks, but that these must be addressed at a social and political level, not at a technical level. We also believe that molecular manufacturing will mitigate other major risks and harms, so any retarding of the basic technology must be considered very cautiously.
We will be contacting Bill Joy to propose a public debate or discussion of these points.