A Takeoff into Nightmare?

Posted today at KurzweilAI.net, and available for discussion through their MindX forum:

Is AI Near a Takeoff Point?
By J. Storrs Hall

Computers built by nanofactories may be millions of times more powerful than anything we have today, capable of creating world-changing AI in the coming decades. But to avoid a dystopia, the nature (and particularly intelligence) of government (a giant computer program -- with guns) will have to change.

Singularities and Nightmares
By David Brin

Options for a coming singularity include self-destruction of civilization, a positive singularity, a negative singularity (machines take over), and retreat into tradition. Our urgent goal: find (and avoid) failure modes, using anticipation (thought experiments) and resiliency -- establishing robust systems that can deal with almost any problem as it arises.

Tags: nanotechnology nanotech nano science technology ethics weblog blog