Who's steering the ship?

Once, the fate of the human race might have been in the hands of the well-adjusted, the social ones, the masters of emotion. Now all that is getting upgraded.

[Richard Powers]
The Overstory, p. 121

It is very easy to fall into the notion that if the new is viable, then there must have been something wrong with the old. This view, to which organisms already suffering the pathologies of over-rapid, frantic social change are inevitably prone, is, of course, mostly nonsense.

What is always important is to be sure that the new is not worse than the old.

It is still not certain that a society containing the internal combustion engine can be viable or that electronic communication devices such as television are compatible with the aggressive intraspecies competition generated by the Industrial Revolution. 

Other things being equal (which is not often the case), the old, which has been somewhat tested, is more likely to be viable than the new, which has not been tested at all.

[Gregory Bateson]
Mind and Nature, p. 194-5

Are we heading toward a future where the barriers between linear and interactive entertainment come down?

Hmm, I don't know, I've not given it a lot of thought. I suppose you will get some form of convergence when the act of watching games is sufficiently fun. And we're getting closer to that. Games like LA Noire and Max Payne are pretty fun and spectacular to watch. A lot of people have said to me that they love playing LA Noire with their partners – that is a baby step toward convergence. Certainly there are areas of the multiplayer mode that are moving in a similar direction.

But again, we're so focused on what we have to do this week, this month, we don't have the time to think about that.

My job is to get the bloody game done and survive in the process. People often ask why don't you go to games conferences – we don't have the bloody time! This is relentless!

Grand Theft Auto 5: Rocktar's Dan Houser on Los Santos and the future

Innovations become irreversibly adopted into the on-going system without being tested for long-time viability; and necessary changes are resisted by the core of conservative individuals without any assurance that these particular changes are the ones to resist.

Individual comfort and discomfort become the only criteria for choice of social change and the basic contrast of logical typing between member and the category is forgotten until new discomforts are (inevitably) created by the new state of affairs.

Fear of individual death and grief propose that it would be 'good' to eliminate epidemic disease and only after 100 years of preventive medicine do we discover that the population is overgrown. And so on.

[Gregory Bateson]
Mind and Nature, p. 238

 Levy has very little time for jokes. Or, it turns out, for philosophy. 

“Are humans machines?” I ask him. He tells me he’s learned not to try to answer philosophical questions.

Ethics, however, he’s interested in. “People ask: is it cheating? Only if women using vibrators are cheating. Will sex workers be put out of business? It’s possible.” What about bigger issues though – what about sex and empathy? And: can a robot consent? “When AI advances, robots will exhibit empathy. People will feel towards them as they do towards animals.”

Sex, love and robots: is this the end of intimacy?

The people who say that artificial intelligence is not a problem tend to work in artificial intelligence.

Many prominent researchers regard Bostrom’s basic views as implausible, or as a distraction from the near-term benefits and moral dilemmas posed by the technology—not least because A.I. systems today can barely guide robots to open doors.

Last summer, Oren Etzioni, the C.E.O. of the Allen Institute for Artificial Intelligence, in Seattle, referred to the fear of machine intelligence as a “Frankenstein complex.” Another leading researcher declared,

“I don’t worry about that for the same reason I don’t worry about overpopulation on Mars.”

The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?


“He was ultra-consistent,” Daniel Hill, a British philosopher who befriended Bostrom while they were graduate students in London, told me. “His interest in science was a natural outgrowing of his understandable desire to live forever, basically.”

Bostrom has written more than a hundred articles, and his longing for immortality can be seen throughout. In 2008, he framed an essay as a call to action from a future utopia. “Death is not one but a multitude of assassins,” he warned. “Take aim at the causes of early death—infection, violence, malnutrition, heart attack, cancer. Turn your biggest gun on aging, and fire. You must seize the biochemical processes in your body in order to vanquish, by and by, illness and senescence. In time, you will discover ways to move your mind to more durable media.” He tends to see the mind as immaculate code, the body as inefficient hardware—able to accommodate limited hacks but probably destined for replacement.

[...] The view of the future from Bostrom’s office can be divided into three grand panoramas. In one, humanity experiences an evolutionary leap—either assisted by technology or by merging into it and becoming software—to achieve a sublime condition that Bostrom calls “posthumanity.” Death is overcome, mental experience expands beyond recognition, and our descendants colonize the universe.

The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?


“I think political systems will use it to terrorize people,” Hinton said. Already, he believed, agencies like the N.S.A. were attempting to abuse similar technology. 

“Then why are you doing the research?” Bostrom asked.

“I could give you the usual arguments,” Hinton said. “But the truth is that the prospect of discovery is too sweet.” He smiled awkwardly, the word hanging in the air—an echo of Oppenheimer, who famously said of the bomb,

“When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.” 

The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?

Perhaps the most radical of his visions is that superintelligent A.I. will hasten the uploading of minds—what he calls “whole-brain emulations”—technology that might not be possible for centuries, if at all.

Bostrom, in his most hopeful mode, imagines emulations not only as reproductions of the original intellect “with memory and personality intact”—a soul in the machine—but as minds expandable in countless ways.

“We live for seven decades, and we have three-pound lumps of cheesy matter to think with, but to me it is plausible that there could be extremely valuable mental states outside this little particular set of possibilities that might be much better,” he told me.

The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?

The fascination with talking to a computer that could answer any question was always there for me.

[Amit Singhal]
Head of Google Search

In his view, public alarmism over AGI obscures the great potential near-term benefits and is fundamentally misplaced, not least because of the timescale. 

“We’re still decades away from anything like human-level general intelligence,” he reminds me. “We’re on the first rung of the ladder. We’re playing games.”

[Demis Hassabis]
Deep Mind Technologies (owned by Google)
The superhero of artificial intelligence: can this genius keep it in check?

Survival is not the principal motive for extending control […] technicians and scientists carry on their work largely as a surrogate activity; that is, they satisfy their need for power by solving technical problems. 

They will continue to do this with unabated enthusiasm, and among the most interesting and challenging problems for them to solve will be those of understanding the human body and mind and intervening in their development. 

For the “good of humanity,” of course.

[Ted Kaczynski]
Industrial Society and its Future, 164

Related posts:-
Guiding Fiction
Lines and Circles
Ideas With Weight 
Re-write It
Fear Visions 
Imagine Something Better
Twisted Out of Shape 
Masters of the Universe 
The Earth's the Limit 
Live Forever? 
The Tyranny of Novelty 
Fuck It
Lost Tribe