lukeprog

I'll interpret your first question as: "Suppose you created superhuman AI: What would you use it for?"

It's very risky to program superhuman AI to do something you think you want. Human values are extremely complex and fragile. Also, I bet my values would change if I had more time to think through them and resolve inconsistencies and accidents and weird things that result from running on an evolutionarily produced spaghetti-code kluge of a brain. Moreover, there are some serious difficulties to the problem of aggregating preferences from multiple people — see for example the impossibility results from the field of population ethics.

if it is super intelligent, it will have its own purpose.

Well, it depends. "Intelligence" is a word that causes us to anthropomorphize machines that will be running entirely different mind architectures than we are, and we shouldn't assume anything about AIs on the basis of what we're used to humans doing. To know what an AI will do, you have to actually look at the math.

An AI is math: it does exactly what the math says it will do, though that math can have lots of flexibility for planning and knowledge gathering and so on. Right now it looks like there are some kinds of AIs you could build whose behavior would be unpredictable (e.g. a massive soup of machine learning algorithms, expert systems, brain-inspired processes, etc.), and some kinds of AIs you could build whose behavior would be somewhat more predictable (transparent Bayesian AIs that optimize a utility function, like AIXI except computationally tractable and with utility over world-states rather than a hijackable reward signal). An AI of the sort may be highly motivated to preserve its original goals (its utility function), for reasons explained in The Superintelligent Will.

Basically, the Singularity Institute wants to avoid the situation in which superhuman AIs' purposes are incompatible with our needs, because eventually humans will no longer be able to compete with beings whose "neurons" can communicate at light speed and whose brains can be as big as warehouses. Apes just aren't built to compete with that.

Dr. Neil DeGrasse Tyson mentioned that if we found an intelligence that was 2% different from us in the direction that we are 2% different [genetically] from the Chimpansees, it would be so intelligent that we would look like beings with a very low intelligence.

Yes, exactly.

How does your group see something of that nature evolving and how will we avoid going to war with it?

We'd like to avoid a war with superhuman machines, because humans would lose — and we'd lose more quickly than is depicted in, say, The Terminator. A movie like that is boring if there's no human resistance with an actual chance of winning, so they don't make movies where all humans die suddenly with no chance to resist because a worldwide AI did its own science and engineered an airborn, human-targeted supervirus with a near-perfect fatality rate.

The solution is to make sure that the first superhuman AIs are programmed with our goals, and for that we need to solve a particular set of math problems (outlined here), including both the math of safety-capable AI and the math of aggregating and extrapolating human preferences.

Obviously, lots more detail on our research page and in a forthcoming scholarly monograph on machine superintelligence from Nick Bostrom at Oxford University. Also see the singularity paper by leading philosopher of mind David Chalmers.

Perhaps you're asking about which factors are causing AI progress to proceed more slowly than it otherwise would?

One key factor is that much of the most important AI progress isn't being shared, because it's being developed at Google, Facebook, Boston Dynamics, etc. instead of being developed at universities (where progress is published in journals).

Yes — we don't want superhuman AIs optimizing the world according to parochial values such as "what Exxon Mobile wants" or "what the U.S. government wants" or "what humanity votes that they want in the year 2050." The approach we pursue is called "coherent extrapolated volition," and is explained in more detail here.

Maybe 30%. It's hard to estimate not just because it's hard to predict when superhuman AI will be created, but also because it's hard to predict what catastrophic upheavals might occur as we approach that turning point.

Unfortunately, the singularity may not be what you're hoping for. By default the singularity (intelligence explosion) will go very badly for humans, because what humans want is a very, very specific set of things in the vast space of possible motivations, and it's very hard to translate what we want into sufficiently precise math, so by default superhuman AIs will end up optimizing the world around us for something other than what we want, and using up all our resources to do so.

"The AI does not love you, nor does it hate you, but you are made of atoms it can use for something else" (source).

I have a pretty wide probability distribution over the year for the first creation of superhuman AI, with a mode around 2060 (conditioning on no other existential catastrophes hitting us first). Many AI people predict superhuman AI sooner than this, though — including Rich Sutton, who quite literally wrote the book on reinforcement learning.

Once AI can drive cars better than humans can, then humanity will decide that driving cars was something that never required much "intelligence" in the first place, just like they did with chess. So I don't think driverless cars will cause people to believe that superhuman AI is coming soon — and it shouldn't, anyway.

When the military has fully autonomous battlefield robots, or a machine passes an in person Turing test, then people will start taking AI seriously.

Amusing note: Some military big-shots say things like "We'll never build fully-autonomous combat AIs; we'll never take humans out of the loop" (see Wired for War). Meanwhile, the U.S. military spends millions to get roboticist Ronald Arkin and his team to research and write the book Governing Lethal Behavior in Autonomous Robots. (One of the few serious works in the field of "machine ethics", BTW.)