Down with goo, up with AI!

In the early years of speculation about risks and benefits of nanotechnology, a primary concern was so-called gray goo, hypothetical runaway self-replicating nanomachines that could, if unchecked, consume the planet. In more recent years, worries about such goo have receded, at least among the cognoscenti, and have been replaced by other fears.

Today, CRN and other investigators are more prone to highlight the danger of a nano-fueled arms race, which could be unstable and spiral out of control. Other issues of concern include environmental damage caused by overproduction of cheap products, ill-considered massive engineering projects with unforeseen disastrous consequences, social disruption from ubiquitous intrusive surveillance, and economic upheavals from the nearly overnight collapse of numerous industries. All of these are worthy of study today and perhaps of action at an appropriate time.

Another danger which has not received much attention, at least outside of a limited circle, is the possibility of runaway artificial intelligence. The projected ability of molecular manufacturing to build supercomputers smaller than a grain of sand, and to enable networking of these vastly powerful computers, could provide a substrate for a recursively self-improving "intelligent" software product. In theory, if this potentially smarter-than-human AI was not designed carefully in the first place, it might go off in a direction unfriendly to humans, or even to human existence.

This AI worry is one that sounds so much like science fiction that CRN has been reluctant to discuss it. However, as Stephen Hawking has said, "Today's science fiction is often tomorrow's science fact." We don't yet have an answer to this perplexing problem; indeed, no one has. But it surely must be added to the list of matters to be studied diligently as we head into the nano era.

Mike Treder

Comments

[Michael Vassar]

You can go farther than that. Steven Hawking has strongly asserted on multiple occasions that he does expect superhuman AI to be developed within 50 years, and that there is a significant chance that it will end human life. Other people who take this possibility seriously include the following list, first compiled by Michael Annisimov. If you know any of these people, or know anyone who knows any of them, please contact me to discuss building a singularity aware network/discussion group. Chris, does this list belong on Wise Nano under the heading "potential singularitarian discussion group"?

Arthur C. Clarke
Futurist, visionary, author of 2001: A Space Odyssey and Profiles of the Future
www.lsi.usp.br/~rbianchi/clarke/ - 225,000 results
John Robb
Former President and CEO of Userland Software
http://jrobb.mindplex.org/ - 70,900 results
Bill Joy
Software pioneer, formerly Chief Scientist at Sun Microsystems
http://www.sun.com/aboutsun/media/ceo/mgt_joy.html/ - 69,900 results
John von Neumann, Ph.D
Computing pioneer, information theorist, theoretical physicist, decision theorist
http://ei.cs.vt.edu/~history/VonNeumann.html - 66,400 results
Stanislaw Lem
Science fiction author, philosopher
http://www.cyberiad.info/english/main.htm - 50,100 results
Freeman Dyson, Ph.D
Former professor of physics at the Institute for Advanced Study in Princeton
http://www.sns.ias.edu/~dyson/ - 30,700 results
Marvin Minsky, Ph.D
Professor of Computer Science at MIT, author of Society of Mind
http://web.media.mit.edu/~minsky/ - 30,500 results
Ray Kurzweil, Ph.D
Entrepreneur, author of The Age of Spiritual Machines and the forthcoming The Singularity is Near
www.kurzweiltech.com/ - 28,100 results
Vernor Vinge, Ph.D
Professor of Mathematics at San Diego University, author of A Deepness in the Sky
www.ugcs.caltech.edu/~phoenix/vinge - 27,700 results
William Calvin, Ph.D
Professor of Neurobiology at Washington University, author of A Brain for All Seasons
http://williamcalvin.com/index.html - 20,900 results
K. Eric Drexler, Ph.D
Father of nanotechnology, author of Engines of Creation and Nanosystems
www.foresight.org/FI/Drexler.html - 20,600 results
Sir Martin Rees
Professor of Astronomy and Cosmology, author of Our Final Hour
http://www.ast.cam.ac.uk/IoA/staff/mjr/ -16,300 results
Douglas Engelbart, Ph.D
Computing pioneer, inventor of the mouse, professor of Computer Science
www.bootstrap.org/ - 15,800 results
Jaron Lanier
Software pioneer, Chief Scientist of Eyematic Interfaces, Inc.
www.well.com/user/jaron/ - 14,500 results
Ben Goertzel, Ph.D
Professor of Mathematics, AGI researcher, entrepreneur, author
http://www.goertzel.org/ - 14,400 results
Eliezer S. Yudkowsky
AGI researcher, Research Fellow of the Singularity Institute for AI
http://yudkowsky.net/ - 14,200 results
Greg Egan
Science fiction author, computer programmer
http://gregegan.customer.netspace.net.au/ - 13,600 results
Max More, Ph.D
Professor of Philosophy, Chairman of the Extropy Institute, business consultant, futurist
http://www.maxmore.com/ - 12,600 results
Hans Moravec, Ph.D
Roboticist at Carnegie Mellon University, author of Mind Children
http://www.frc.ri.cmu.edu/~hpm/ - 12,200 results
Eugen Leitl
Chemist, transhumanist writer and speaker
http://www.lrz-muenchen.de/~ui22204/.html/eleitl-cv.html - 11,500 results
Anders Sandberg, Ph.D
Professor of Computer Science, neurobiologist, transhumanist writer and speaker
http://www.nada.kth.se/~asa/cv.html - 10,200 results
Damien Broderick, Ph.D
Futurist, author of The Last Mortal Generation and The Spike
http://www.panterraweb.com/the_spike.htm - 9,840 results
Peter Voss
AGI researcher, entrepreneur
http://www.optimal.org/peter/peter.htm - 9,400 results
Stephen Hawkings, Ph.D
Theoretical physicist, author of A Brief History of Time
http://www.hawking.org.uk/ - 9,270 results
Gregory Stock, Ph.D
UCLA bioethicist, author of Metaman and Redesigning Humans
http://research.mednet.ucla.edu/pmts/Stock.htm - 8,880 results
John Smart
Systems theorist, futurist, author of SingularityWatch.com
http://www.singularitywatch.com/bio_johnsmart.html - 8,460 results
Hal Finney
Futurist, engineer, cryptographer
http://www.finney.org/~hal/ - 7,120 results
Samantha Atkins
Software architect
http://www.singinst.org/comments/statements.html - 5,920 results
Robin Hanson, Ph.D
Professor of Economics at George Mason University
http://hanson.gmu.edu/ - 5,810 results
Bart Kosko, Ph.D
Professor of Electrical Engineering, author of Heaven in a Chip
http://sipi.usc.edu/~kosko/ - 5,580 results
Christopher Phoenix
Director of Research, Center for Responsible Nanotechnology
http://xenophilia.org/ - 4,790 results
Christine Peterson
President of the Foresight Institute, nanotechnology spokesperson
www.foresight.org/FI/Peterson.html - 4,630 results
Nick Bostrom, Ph.D
Professor of Philosophy at Cambridge University
http://www.nickbostrom.com/ - 4,620 results
Frank Tipler, Ph.D
Professor of Mathematics at Tulane University, author of The Physics of Immortality
http://www.math.tulane.edu/~tipler/ - 4,410 results
James Hughes, Ph.D
Director of the World Transhumanist Association, author of the forthcoming Cyborg Democracy
http://www.changesurfer.com/Hughes.html - 4,200 results
Hugo de Garis, Ph.D
AI researcher, neural network pioneer, futurist, author
http://www.cs.usu.edu/~degaris/ - 3,440 results
Francis Heylighen, Ph.D
Research professor, author of the Principia Cybernetica Project
http://pespmc1.vub.ac.be/HEYL.html - 3,340 results
I.J. Good, Ph.D
Computing pioneer, probability theorist, polymath, futurist
http://ei.cs.vt.edu/~history/Good.html - 2,970 results
Alex J. Champandard
Programmer, AI researcher, webmaster
http://ai-depot.com - 2,790 results
Brian Atkins
Venture capitalist, Director of the Singularity Institute for AI
http://www.posthuman.com - 2,580 results
Pei Wang, Ph.D
Professor of Computer Science, AGI researcher
http://www.cis.temple.edu/~pwang/ - 2,260 results
Natasha Vita-More
President of the Extropy Institute, transhumanist writer, artist
http://www.natasha.cc/ - 2,240 results
Bruce J. Klein
Life extension activist, founder of the Immortality Institute for Infinite Lifespans
http://imminst.org - 1,600 results
J. Storrs Hall, Ph.D
Professor of Computer Science, nanotechnology pioneer
www.imm.org/HallCV.html - 1,490 results
Michael Anissimov
Futurist, Singularity writer, co-directs the Immortality Institute for Infinite Lifespans
http://www.acceleratingfuture.com/michael - 1,200 results
Mike Treder
Executive Director, Center for Responsible Nanotechnology
www.kurzweilai.net/bios/bio0219.html - 1,030 results
Stephen Reed, Ph.D
Professor of Computer Science, AI researcher for Cycorp
http://www.sl4.org/archive/0205/3754.html - 919 results
Joseph N. Pelton, Ph.D
Research Professor for the Institute for Applied Space Research
http://www.clarkeinstitute.com/lecture8.html - 851 results
Eric Boyd, Ph.D
Professor of Computer Science, entrepreneur, software architect
http://www.eecs.umich.edu/PPP/eric-boyd.html - 820 results
Gordon Worley
Programmer, Singularity advocate, transhumanist
http://homepage.mac.com/redbird/ - 800 results
James N. Gardner
Complexity theorist, science essayist, cosmologist, author of The Selfish Biocosm Hypothesis
http://www.biocosm.org/ - 500 results
Mike LaTorra
Director of the World Transhumanist Association, teacher, author of The Modern Tao
http://www.transhumanism.org/speakersbureau.htm#LaTorra - 300 results
Lyle Burkhead
Nanotechnology critic, futurist
http://www.geniebusters.org/ - 200 results
Phil Bowermaster
IT and Telecommunications company manager, blogger
http://www.speculist.com/ - 180 results

[Chris Phoenix, CRN]

Yes, nano-enabled AI would seem to be a valid topic for Wise-Nano.

"Singularitarian" is too jargon-y. I'd pick a more obvious title, something like "Massive disruptive runaway change."

Chris

[Mike Deering]

What method do you use to get the results numbers? I put "John Robb" into Google and got 219,000 results. Your number of 70,900 seems a little low.

[micheal vassar]

Michael Annisimov put together the list months ago. Google's probably more powerful now than then.

[Paul]

I believe one SF author (maybe Asimov or Clarke) once commented that, contrary to exciting stories of evil AIs run amok, there'no reason why AIs couldn't turn out to be more moral, more good, and more wise than humans.

[Michael Vassar]

Of course there isn't. There is obviously no reason at all to expect AIs to be evil. There is every reason to expect them to do exactly what they are told to do, whatever the intended or unintended consequences. That's where the evil comes in.