Dangerous Knowledge

O futile humans! Why does your folly teach skills innumerable, and search out manifold inventions still? But there is one knowledge you do not gain and have never sought it: to implant a right mind where no wisdom dwells. - Theseus

Myths abound about the folly of humans in meddling with forces beyond their control. Think of Prometheus, Pandora, Daedulus and Icarus...and that's just the Greeks! Closer to our own time, of course, is the story of Dr. Frankenstein.

Today, in the age of hyper-modern technology, we are forced to grapple with ever more serious challenges.

One such is the deadly potential of biotechnology when used as a weapon:

There is growing scientific consensus that biotechnology -- especially, the technology to synthesize ever larger DNA sequences -- has advanced to the point that terrorists and rogue states could engineer dangerous novel pathogens. . . We live in a world where gene-sequencing equipment bought secondhand on eBay and unregulated biological material delivered in a FedEx package provide the means to create biological weapons.

"The Loss of Biological Innocence" is how Jason Pontin, editor of MIT's Technology Review, describes his dilemma in deciding whether to publish a frightening story (quoted above) about the perilous availability of plague-like bioweapons.

Remember last year when the US Department of Health and Human Services published the full genome of the 1918 influenza virus on the Internet? Some hailed it as a step forward in openness, while others, most notably Ray Kurzweil and Bill Joy, decried it as "extremely foolish."

Here at CRN, we've had our own qualms about publishing technical information that might be abused.

Can dangerous knowledge ever be contained? Once opened, might Pandora's Box be shut again? Those questions lie at the heart of an ongoing debate over the necessity and application of precaution in the deployment of new technologies.

"Reversibility" is a concept that Jamais Cascio proposes as a wiser alternative to the Precautionary Principle or the lesser-known Proactionary Principle:

A cornerstone of the open future concept is that we should be striving towards a world that maximizes our flexibility in response to challenges. We will never have perfectly free choices when problems arise, but we are more likely to come up with good solutions under less-constrained conditions than we would if we were limited to a handful of options. The choice to pull back and say "let's try something different" is an option that we should strive to maintain.

But tech analyst George Dvorsky worries that Cascio's idea may be too idealistic:

There is no precedent yet in human history where the pursuit of certain weapons technologies have been abandoned due to their potential risks. It is the nature of the military to be in a perpetual search for the most sophisticated technologies. . . Worse, once a military force gains possession of a weapon, it will never relinquish it.

Dvorsky's arguments against the feasibility of Cascio's proposal are quite well reasoned. That's a pity, though, because it seems that something must be done. As Dvorsky says:

[W]ith non-state actors increasingly threatening to acquire dangerous weapons, societies are increasingly become more police-like in their approach to surveillance and control. Our social and legal infrastructure is being moulded by technological and geopolitical pressures -- something that is clearly beyond reversibility.

For more than two thousand years, we've heard warnings about our lack of preparedness or ability to handle dangerous knowledge. It would be a shame if, having come this far, we end up like poor Icarus.

Mike Treder

Tags: nanotechnology nanotech nano science technology ethics weblog blog

Comments

[JimboNC]

A better metafore for this essay would be Adam plucking a nuclear-symbol apple from the tree of knowledge.

Aren't we always curious enough to take chances, to try the forbidden, even if we get burnt? Nothing ventured nothing gained. That's why we have stoves instead of cooking over open fires. Although, the outdoor grill has brought that back for many.

Dr. Frankenstein was yesteryear's cloning scientist.

[Letalis]

Meanwhile the UN, ever vigilant, ever timely, strains at the gnat of the perceived over-proliferation of small arms.

[Rik]

There's a wonderful essay on spiked-online, about this problem: the increasing inability to take risks. In the case of MM or biotech everyone clearly recognizes that the payoff might be huge, but the same people also see the risks. Sort of, but not quite: to stop one man from using a hammer to kill people, we have to ban all hammers.
You can come up with all sorts of moratoria and treaties, but there will always be rogue states. Just think of the Khan nuclear network and Pakistan selling the secrets he stole.
The doom-mongerers think that their own moral standards apply to everyone, everywhere, forever. That's just plain stupid.
As for being wiped out by technology, I'd worry more about Yellowstone...

[Brian Wang]

Small arms are a valid issue. One of many issues.

http://hei.unige.ch/sas/Yearbook%202005/summaries/2005%20press%20kit-chap.9.pdf

60-90% of conflict deaths are from small arms.
60,000+ deaths per year.
Finding ways to reduce those numbers would be good.

There many problems and different people should be working on ways to make each situation better.

[Letalis]

Rather than small arms being an issue, some responsible people believe the problem is the failed states in which those conflicts almost always take place. Other than occasionally sending in "peace keepers" who seem to spend most of their time abusing the locals for fun and profit, the UN has (as usual) done nothing.

[Brian Wang]

I am not claiming that any attempts to make the problem of excess deaths from small arms better were working or that they were the right approach.

I think deaths from bioweapons, guns, disease, and whatever cause are not good things. It is valid to figure ways to make it better and to try to implement solutions.

Most attempts at fixes do not work or work that well. Sometimes the results can be improved by working on the implementation details. Sometimes the entire framing of the problem is flawed. An unbiased detailed look and analsys of the issues and past attempts and results needs to be done to determine what might work better. I am in favor of many different trials to more quickly find better systems.

On the UN. The UN has a lot of flaws. However, I think Peace keeping are in general worthwhile efforts. I see them as lending some extra law enforcement. People can also site police abuse. Does that mean that the police should be disbanded and the concept of police is flawed if there are incidents of police abuse?

[sbw]

"Dangerous Knowledge prompted me to dig back into Jacob Bronowski and write Turning point.

"The idea that someone can claim control over the life of another by virtue of magic is bankrupt. ..."

[L. Blanck]

The story of Icarus is a WARNING to humanity, but, as usual, humanity will not heed the warning! Humanitarians believe that humanity can solve humanities problems through their sciences and technologies, but, as usual, the solutions to those problems always lead to more problems, and an increase in DANGEROUS KNOWLEDGE! Eventually, the increasing availability of dangerous knowledge, the complexity of the knowledge distribution systems, and the problems associated with the spread of knowledge in general, will overwhelm humanity. Signs of this are already apparent.

[Concrescent]

The ideas expressed in the Precautionary, Proactionary, and Reversibility principles are laudible, but futile. It is not possible to stop, slow, redact, relinquish,redirect, guide, or otherwise 'manage' technological development in any given field, on a global basis. This is wishful thinking [or mental masturbation] run amok.

The nuclear nonproliferation effort has failed to prevent Pakistan - and even today fails to prevent Iran and North Korea - from producing nuclear weapons, and that technology requires rare, expensive material that must be extensively refined and purified requiring massive industrial efforts. As noted by Pontin, above, biotechnology is far more egalitarian; in the near future, any of thousands of medical labs, high school science classrooms, and anyone who can cook up a batch of Meth in their kitchen will be able to do genetic sequencing and create custom organisms. And, not just here, but around the world. With nanotech, the manufacturing is more difficult, but only for a first generation; later generations of nanotech will get easier and easier to make with the tools that each provides. The incentives to make proliferant nanotech - such as peak oil, global warming, etc. will overwhealm any moral suasion invoked against it as just meaningless drivel.

Exactly what 'force' do - can - any of these restraining principles propose to marshall by way of enforcement? The answer is that there is None. Government cant stop technologies from being developed and introduced, any more effectively than it can fight the [failed] 'war on drugs'. In fact, unless the targeted tech just happens to require some particular ingredient (i.e. uranium or pseudophedrine), there is no 'choke point' by which any level of totalitarian oppression can even hope to affect the outcome. Since Adenine, Cytosine, Guanine, Thymine, Carbon, and Silicon all occur plentifully in nature, there is no hope of choking off 'maverick' geneticists (nor, ultimately, nanotechnicians), through material control, as you can [hope to] do with nuclear technology. Any government intrusive and invasive enough to monitor and control all human behavior would be as nightmarish as any technocalypse we can imagine. Does anyone propose 'mass ignorance' as the answer? Too late; if you padlocked every college science classroom tomorrow - and burned all the books - there are already millions of people walking around who can sort out biotech or nanotech, should they desire to do so.

Nothing short of Mao's "Cultural Revolution", [which killed upwards of 150,000,000 people, most for commiting the crime of receiving an education] applied diligently, worldwide, will have any meaningful effect upon the creation and spread of 'dangerous knowledge'.

But it gets worse. There are other world changing technologies on the horizon, that will require nothing more than discrete electronic components to assemble.

It is unrealistic to think that we have the power to collectively manage our future by any sort of planned approach to technological development. The memes are in control of things. Technology developes and evolves chaotically, around vast numbers of individual nucleation sites (garage entrepreneurs, research inventors, engineering professors, government institutions, grad students, corporate labs, etc.) which cannot be sytematically regulated in any real world that anyone would want to live in.

Without absolute universal enforcement - which is impossible and undesireable in its own right - there is no plausible mechanism to prevent things from going wrong. They WILL. Its inevitable. Sooner or later. Get used to the idea.

[Janessa Ravenwood]

Concrescent: At last, someone else gets it.

[Phillip Huggan]

I'm not commenting here on whether regulation is good or not, just the logistics of it.
Concrescent, of course government labs and (to a lesser degree) grad students utilizing public facilities can be regulated. There are technologies evolving along decentralized individual nucleation sites, but also some have evolved along a concerted single big effort: the internet, the A-bomb, most space endeavours. Sure proliferation is an issue once the technology has been invented, but in terms of inventing the technology bigger is better. An anology for meth labs is that Oklahoma has reduced the number of Meth labs in operation by 80% merely by restricting cough medicine sales. If similiar vital necessities to MNT are restricted (possibly extreme coolants), the deck can be stacked in society's favour. Certainly nanotechnology innovations are a bottom-up process, but it is far from certain if MNT isn't top-down.
You can't surveil for other A-bombs with an A-bomb (though you can detroy them, a path the US considered against the USSR in the early '50s). You can surveil for MNT if you have MNT, like it or not.

[Mike Deering]

I concur with Concrescent and Janessa. I would repeat everything Concrescent wrote, but it just isn't necessary since it has already been written.

[DT]

Since Adenine, Cytosine, Guanine, Thymine, Carbon, and Silicon all occur plentifully in nature,

--------------------------------

Yes but extracting and processing them for use is one hell of a challenge. There is much much more to a product than some raw material, there is several types of chemicals that make a product into what it is...flexible,stainless,nonstatic,smooth etc. Its not like you mix water and silicon from your garden and voila you get a semiconductor. Not to mention how many types of fluids the 'MM' machine has to use to run smoothly. Some guy inventing or making a molecular manufacturing machine is equal to one guy trying to build a boeing 777 from scratch, for many reasons-not undestood by people who aren't in the manufacturing,materials and chemistry Industy- personal manufacturing machines are a dream.

Making up your own machines in your head isn't going to make it so either, so please don't respond back with something like "in the future there'll be a machine that'll make carbon and different chemicals with air costing only .0001 cent" or "you'll throw in a toaster and it'll come out a computer" cause those such things are just funny fantasies.

[Brian Wang]

Nuclear non-proliferation slowed the spread significantly. N Korea got the weapons 60 years after the USA.

How much effort is it now for say a major government to track the people and labs? It is not millions as DT noted. Maybe 1000-10,000 really might do something significant. Even if it was millions, technology is allowing a wider level of automatic monitoring.

Financial records analysis. Tracking key equipment sales. Gene sequencers. STMs.

Research tracking.

Datamine the monitoring of phones, cellphones, email, internet. Advanced spyware or changes to the protocols, monitoring at the ISPs. An article about making the monitoring the internet
http://www.technologyreview.com/read_article.aspx?ch=specialsections&sc=security&id=16360

Electronic bugging. Which is getting cheaper and more powerful with MEMS and NEMS. Smart dust. RFIDs - 7 cents each in high volume.

Hire some of the people to work for the government. Either in research or to provide intelligence.

On the "war on drugs". Don't get confused by a political slogan. There was not a full scale, no-hold barred effort on that issue.

I would not bet against oppression or purges if the stakes get high enough. Guantanamo can get easily expanded with some wings for dissident scientists. People (even many people) can get killed to take dangerous knowledge with them to their graves. So better solutions should be found.

I think significant/maybe complete loss of privacy in exchange for the necessary security will be that solution. Hopefully there will be a velvet glove around the iron fist of absolute enforcement.

[Brian Wang]

Widescale monitoring now.
Looking to use RFID to track 13 billion chickens.
http://eetimes.com/news/latest/showArticle.jhtml?articleID=183702439

RFIDs are going into Chinese ID cards.

Even without nano...moore's law for RFIDs. 20yrs they are 1000 times more powerful for the same price or 1000 times cheaper.

[Janessa Ravenwood]

Can't speak for those of you outside the US, but did the rest of you miss the Supreme Court decision invalidating blanket surveillance a while back (police using IR cameras driving down the street looking at houses)? Just because it will be possible to make a zillion nano AV cameras and put them everywhere does not mean it will be legal for the (US) government to do so. And people WILL rebel against that, make no mistake. Traffic cameras? Grumbling, but acceptance. Real-time *involuntary* AV feeds from people's living rooms, bathrooms, and bedrooms? An uprising like you've never seen before. Not going to happen, no matter much you David Brin acolytes might desperately want it. Massive constitutional violation – would be struck down by *any* US court INSTANTLY.

[Brian Wang]

Janessa

Your position seems to be near absolutist against losses of privacy. Even if the alternative is abuse of nanotechnology or biotechnology or other technology that is not privacy related.

Concrescent and you seem to say - let us throw up our hands and not try to stop technology abuse because we are not willing to take the steps to do something about it. Are there lesser of two or many evils?

You are willing to allow monitoring of public spaces for law enforcement. Traffic cameras. But there is more than traffic cameras. Keyhole spy satellites can see down to 15cm resolution or better. Airplanes and blimps have cameras too.

If someone is performing some illegal activity in their bedroom, you do not seem to want anything to be done to try to detect it. What is your position on that?

Someone was making a conventional bomb. But it was in his bedroom. So what do we do? Wait until he moves it out of his house and try to intercept it? If he blows it up in his apartment in a high rise, then what?
What if he is making a dirty bomb, a bio weapon?

I do not desperately want the loss of privacy. But I recognize that it already seems to be happening, that the trends are for more loss of privacy, that it will seem to be necessary to deal with the future issues.

I think the legalistic efforts to try to stop it will not be successful in a meaningful way. There are laws against spam too, my email inbox still seems to have spam. The electronic trail online that people leave also seems open to those that want it. There may even be technical solutions to some block some kinds of monitoring but I think the tidal wave of tech is that monitoring will be easy and widespread.

I think the IR rule just means that police and joe citizen can still use the IR cameras but that their might be some means to cause trouble if they get caught. Catching them will be rare. The opponents would probably need a lot of spy tech to find the abuse. I think it IR and other monitoring will still happen and is happening but that for law enforcement they would have to get evidence obtained using it in another way, which is easier if they know what to look for.

What kind of rebellion do you foresee? A large scale violent one? Or maybe posted rules like they have at gyms saying "Please don't use camera phones or digital cameras or you could be ejected or banned from the gym". With the implied if you get caught.

[Janessa Ravenwood]

Brian:

Inside my home, yes, my position *is* absolutist on privacy. *Thou shalt not* have involuntary AV feeds watching/recording me in my home. No exceptions, no matter *what*. Attempts to do this to me will be fought tooth and nail, by *any and all* available methods, legal or not.

Don’t agree? Fine. YOU sign up first. Please provide us a link to the cameras/microphones you will place in every room of your house (make sure the angles are good and we don’t expect a fee for access). Please also provide us with your credit card #’s (complete with expiration dates and exact name spellings), a copy of your DL and SSN cards, your bank account #’’s, all online account user ID’s and passwords, your phone #’s, your address, and any other private information you can think of. Not going to do that? Then maybe a complete and total loss of all privacy everywhere isn’t such a good idea, no?

IR cameras? Some IR-opaque paint can take care of them (you can coat your windows to foil laser listeners as well, if you want to go that far). As for my rebellion, I mean a citizen/political one. Think the (CA) uproar that caused Proposition 13 was bad? If anyone tries this, well, then you ain’t seen nothin’ yet. And illegally-gathered evidence is inadmissible in court, if the police are still engaging in such practices.

[Brian Wang]

Janessa

As I said, I think it is a pity that we are losing our privacy. I am not actively trying to give mine up. I don't like spam but I don't realistically think it will be completely eliminated. This does not mean that I go to hacker and spammer sites and sign up. I believe that I and others can be spied upon by those who are motivated. I am not going to help them but I am not going take the measures that you describe. I am following the Indiana Jones strategy of being one conforming box among many in the big warehouse.

Good luck with your fight for privacy.

[Mike Deering]

DT you need to do some more reading before you pass judgement on what is possible with MM.

[DT]

DT you need to do some more reading before you pass judgement on what is possible with MM.

--------------------------

On paper anything is possible, when I was 12 on paper I was superman,a trillionaire and king of the world, but in reality I'm not even close, 'MM' is just that, theories written on paper. So I can pass as many judgements on MM as you can claim what MM can do.

That aside what parts do you disagree or believe its 'false'?

[Mike Deering]

A diamondoid nanofactory is not as complicated as a digital computer, and many of those were designed and assembled by individuals from off-the-shelf parts, back in the day. I expect the same of diamondoid nanofactorys. The first MM will be used by Intel and others to make computer circuitry. It will be small parts of huge industrial machinery. Individual molecular robot arms will be in use in University labs for research and teaching. These will be the off-the-shelf components used by some clever students to make the first desktop diamondoid nanofactory.

[DT]

I'm sorry but what? a machine that assembles products on a atomic scale with robotic arms the size of bacteria doing perfect percision isn't as complicated as a digital computer? apparently you don't know much or anything about manufacturing machinery. The simplest thing you could've thought of is we still can't even build something remotely close to the "proposed" MM today or in the next 10 years yet we made digital computers a few years back and thriving, so if its more simpler to build why not have 18 yearolds at home build personal molecular manufacturing machines like personal pcs. Its comments like this that make Molecular Manufacturing sound like ridiculous dreams of sci-fi enthusiasts with not even a ounce of knowdledge about actual manufacturing technology or chemistry.

[Mike Deering]

Actually, the robotic molecular scale arms are much smaller than bacteria. The simplicity comes from the massive redundancy. Every one of the billions of robotic arms are exactly alike. Their arrangement in the machine is a perfectly regular pattern. Likewise the other components of the machine, the internal control system, the carbon delivery system, the cooling system, are duplicated identically billions of times.

The reason we don't have eighteen year olds building nanofactories in their garages yet is not because they are too complicated, but because the components are not yet available.

[Chris Phoenix, CRN]

DT, which is more complicated: a modern CPU, or a scanning probe microscope? A CPU, or an automobile (minus the embedded CPU's)? A CPU, or an industrial robot? I think the CPU wins all those comparisons.

I'm not sure whether a modern CPU, with its hundreds of millions of transistors, will be more or less complex than a nanofactory. But they're in the same ballpark. A nanofactory is not much more than a bunch of robot arms, computers, and passive structure.

We don't have the tools to build a nanofactory yet. We will. Look how fast DNA synthesis technology is advancing, and how rapidly DNA machines are being developed. DNA is one of several possible pathways to MM.

Chris

Ps. I'm engaging you respectfully, if belatedly. Please start doing the same, or you'll be ignored and may have your messages deleted.