Facing the Intelligence Explosion 1: Personal Motivations

by Luke Muehlhauser on November 24, 2011 in Singularity

Update: Facing the Intelligence Explosion has a new home at FacingTheSingularity.com.

I’ve learned more in the past year than in the previous 5 years of my life combined. (Largely, because I got really good at independent learning.)

Also, I joined Machine Intelligence Research Institute and discovered an enormous treasure trove of knowledge and analysis that hadn’t been written down or well-organized anywhere.

I’ve been trying to share what I’ve learned in a long series of brief, carefully written, and well-referenced articles, but such articles take a long time to write. It’s much easier to write long, rambling, unreferenced articles.

This new series, ‘Facing the Intelligence Explosion,’ is my new attempt to rush through explaining as much material as possible. I won’t edit, I won’t hunt down references, and I won’t try to be brief. I’ll just write, quickly.

The subject matter won’t appeal to everyone. The unedited writing certainly won’t. Nevertheless I suspect what I cover will be interesting and informative to many. Besides, this is my personal blog; I’ll write what I want, here.

Personal background

I’ll begin with my personal background. It will help to know who I am and where I’m coming from. That information is some evidence about how you should respond to the other things I say.

I’ve told my personal story before, so here’s the quick version: I was raised a devoted evangelical Christian in Minnesota. I attended a Christian school, enthusiastically read pop-theology in my spare time, played in the worship band, went on short-term missionary trips around the world, experienced what I believed to be the Holy Spirit many times — I was a true believer.

Around age 22 I wanted nothing more than to be like Jesus to a lost and hurting world. Thus, I had to study the Historical Jesus, to figure out what “being like Jesus” meant. That’s when I learned all kinds of disturbing things about the Bible and the origins of Christianity. I wanted to protect my faith, so I read more and more “sophisticated” works of Christian apologetics. I also read a few books by atheists, so I could be “fair” in my investigation. The short story is that the atheists won, and I lost my faith in God completely.

This was horrifying at first, because my relationships, my morality, my sense of purpose, my life plans, and my picture of the world were all grounded in Christianity. Gradually, I built up a new worldview based on the mainstream scientific understanding of the world, and approach called “naturalism.” I launched Common Sense Atheism in November 2008 (at age 23) to explain atheism and naturalism to others, because I was unhappy with the lack of philosophical and rational seriousness displayed by the atheist blogosphere at that time.

My blog became one of the most popular atheism blogs rather quickly. I enjoyed translating the results of professional philosophy of religion for the mass public, and I enjoyed speaking with experts in the field for my podcast Conversations from the Pale Blue Dot.

Morality was always an important topic for me, and eventually I was writing more about morality than I was about philosophy of religion. When I launched my blog I was an error theorist, which is one way of being a moral anti-realist. Shortly after launching Common Sense Atheism, I encountered the work of Alonzo Fyfe, who presented the first theory of naturalistic moral realism that didn’t smell like bullshit to me. I came to accept and then promote his theory of “desire utilitarianism” (aka “desirism”) on my blog.

My moral views have continued to evolve, and my latest statement of moral theory is summarized here. I’ve come to think that moral language is so confused and contentious that we may want to abandon moral terms altogether, and I try to avoid classifying myself as either “moral realist” or “moral anti-realist” because, as always, whether I’m one or the other depends on what is meant by those terms, and people use the terms in a variety of ways. Whether anti-realism or naturalistic moral realism is “true” depends to some degree on one’s attitude toward the use of moral language, as Richard Joyce argues in his paper Metaethical Pluralism, and as I argue in Pluralistic Moral Reductionism. (I still think desirism is one particularly useful way to talk about morality, much better than many others, and it fits within the framework of pluralistic moral reductionism.)

I was also always interested in rationality, at least since my deconversion, during which I discovered that I could easily be strongly confident of things that were total nonsense. How could the human brain be so incredibly misled? Obviously, I wasn’t Aristotle’s “rational animal.” Instead, I was Gazzaniga’s rationalizing animal. Critical thinking was a major focus on Common Sense Atheism, and I spent as much time criticizing poor thinking in atheists as I did criticizing poor thinking in theists.

 

Intelligence explosion

My interest in rationality inevitably lead me (in mid 2010, I think) to the largest and best treasure trove of articles on the mainstream cognitive science of rationality: Less Wrong. It was here that I first encountered the idea of intelligence explosion and the need for Friendly AI, though I had encountered the mainstream machine ethics literature back in June 2009.

I tell the story of my first encounter with the famous paragraph from I.J. Good on intelligence explosion here. In short:

Good’s paragraph ran me over like a train. Not because it was absurd, but because it was clearly true. Intelligence explosion was a direct consequence of things I already believed, I just hadn’t noticed! Humans do not automatically propagate their beliefs, so I hadn’t noticed that my worldview already implied intelligence explosion.

I spent a week looking for counterarguments, to check whether I was missing something, and then accepted intelligence explosion to be likely (so long as scientific progress continued). And though I hadn’t read Eliezer on the complexity of value, I had read David Hume and Joshua Greene. So I already understood that an arbitrary artificial intelligence would almost certainly not share our values.

My response to this discovery was immediate and transforming:

I put my other projects on hold and spent the next month reading almost everything Eliezer had written. I also found articles by Bostrom and Omohundro. I began writing articles for Less Wrong and learning from the community. I applied to Machine Intelligence Research Institute’s Visiting Fellows program and was accepted. I quit my job in L.A., moved to Berkeley, worked my ass off, got hired, and started collecting research related to rationality and intelligence explosion.

As my friend Will Newsome once said, “Luke seems to have two copies of the ‘Take Ideas Seriously’ gene.”

 

Fanaticism?

Of course, what some people laud as “taking serious ideas seriously,” others see as an innate tendency toward fanaticism. Here’s a comment I could imagine someone making:

I’m not surprised. Luke grew up believing that he was on a cosmic mission to save humanity before the world ended with the arrival of a super-powerful being (the return of Christ). He lost his faith and with it, his sense of epic purpose. His fear of nihilism made him susceptible to seduction by something that felt like moral realism (desirism), and his need for an epic purpose made him susceptible to seduction by Singularitarianism.

One response I could make to this would be to say that this is just “psychologizing,” and doesn’t address the state of the evidence for the claims I now defend concerning intelligence explosion. That’s true, but again: Plausible facts about my psychology do provide some Bayesian evidence about how you should respond to the words I’m writing in this series.

Another response I could make would be to explain why I don’t think this is quite what happened, though elements of it are certainly true. (For example, I don’t recall feeling that the return of Christ was immanent or that I was on a cosmic mission to save every last soul, though as an evangelical Christian I was theologically committed to those positions. But it’s certainly the case that I am drawn to “epic” things, like the rock band Muse and the movie Avatar.) But I don’t want to make this post even moreso about my personal psychology.

A third response would be to appeal to social proof. There seems to be a class of Common Sense Atheism readers that has read my writing so closely that they have developed a strong respect for my serious commitment to rationality and changing my mind when I’m wrong, and so when I started writing about Singularity issues they thought, “Well, I used to think the Singularity stuff was pretty cooky, but if Luke is taking it seriously then maybe there’s more to it than I’m realizing,” and they followed me to Less Wrong (where I was now posting regularly). I’ll also mention that a significant causal factor in my being made Executive Director of Machine Intelligence Research Institute after so little time with the organization was that Machine Intelligence Research Institute staff could see that I was seriously devoted to rationality and debiasing, seriously devoted to saying “oops” and changing my mind and responding to argument, and seriously devoted to acting on decision theory rather than habit and emotion as often as I could.

In surveying my possible responses to the “fanaticism” criticism above, I’ve already put up something of a defense. But that’s about as far as I’ll take it. I want people to take what I say with a solid serving of salt. I am, after all, only human. Hopefully my readers will take into account not only my humanity but also the force of the arguments and evidence I will later supply concerning the arrival of machine superintelligence.

Previous post:

Next post:

{ 20 comments… read them below or add one }

Beelzebub November 24, 2011 at 3:30 am

You may recall that I’m a holdover from the days when CSA was specifically about atheism, but I’m also a person able to appreciate your interest in Singularity. I think I’ve always been pretty much in line with your prior skepticism about it, though I have also never been able to find anything intellectually wrong with it, so in a way I admire the fact that you have committed yourself to its study despite the somewhat “crank” opinion directed at it by the general public. In a very real way, I think that demonstrates a courage that I’m somewhat lacking. I watched most of Thiel’s recent talk and I agree with his assessment. Most people just don’t believe in this stuff.

  (Quote)

Beelzebub November 24, 2011 at 3:38 am

BTW — any intention to change your domain name to more accord with the new theme?

  (Quote)

Luke Muehlhauser November 24, 2011 at 3:58 am

Beelzebub,

Re: domain name. What would you suggest?

  (Quote)

Joshua Fox November 24, 2011 at 6:43 am

Thanks for the great post. Indeed, I’d love to see posts from lots of people describing their motivations for what they do.

So… what are your motivations for putting all this effort into SI? Your post describes the factual content you learned, not your internal drivers.

Mine are:
1. Recognizing the FAI problem, I want to help save the world and make a better future.
2. It is an incredible intellectual adventure with some fascinating people.

  (Quote)

Bruno Coelho November 24, 2011 at 7:41 am

Luke,

I want to ask some questions about your dedication to reducing the risks of AI. Strategically, do you think that the study and the research in philosophy of religion and normative ethics are completely useless, and have to end?

Other point, in terms of time, you align on the group that see the singularity gradually coming or on the abrupt side?

  (Quote)

Luke Muehlhauser November 24, 2011 at 10:43 am

Joshua,

About the same for me, yes.

  (Quote)

Luke Muehlhauser November 24, 2011 at 10:48 am

Bruno,

I’m still studying normative ethics, because that one matters for dealing with AI risks.

As for philosophy of religion… I think a few people should still be working to explain to people how we know the universe is running on physics, not magic. But thinking about the situation in terms of marginal investment, we obviously have way too many people in philosophy of religion already, and not nearly enough people working on existential risk reduction.

As for your other question: Are you talking about soft takeoff / hard takeoff, or something else?

  (Quote)

Peter Hurford November 24, 2011 at 11:13 am

There seems to be a class of Common Sense Atheism readers that has read my writing so closely that they have developed a strong respect for my serious commitment to rationality and changing my mind when I’m wrong, and so when I started writing about Singularity issues they thought, “Well, I used to think the Singularity stuff was pretty cooky, but if Luke is taking it seriously then maybe there’s more to it than I’m realizing,” and they followed me to Less Wrong (where I was now posting regularly).

This.

  (Quote)

Bruno Coelho November 24, 2011 at 6:22 pm

I’m talking about when do you think that the superintelligence will come. In this century?

  (Quote)

Bob Carlson November 24, 2011 at 7:42 pm

Thus, I had to study the Historical Jesus, to figure out what “being like Jesus” meant. That’s when I learned all kinds of disturbing things about the Bible and the origins of Christianity.

After studying the historical Jesus, what did you conclude about the historicity of Jesus? One reason I ask is that your background seems to have some similarities to that of Neil Godfrey, who, recently blogged about why he came to question the historicity of Jesus.

  (Quote)

Robert November 24, 2011 at 10:40 pm

“I also read a few books by atheists, so I could be “fair” in my investigation.”

This clearly shows You were never a Christian. You did not give your life to Christ.
The second you truly ask God for the truth about Christ, ask Him to be your Father, to forgive you of your sins, that you will accept his gracious offer, you will receive the Holy Spirit.
At that point you KNOW. God puts the truth in your mind as a fact. There is no investigating after that. That would be like investigating your name or confirming that was really your face in the mirror.
You dont investigate if you have 5 fingers when you know it for a fact.

I’ve heard this atheist story before. You dont understand what Christian faith even is. Paul saw the resurrected Christ and he said he had Faith in Him. There is no uncertainty. The faith your talking about is the kind it takes to believe human thought, in a spacetime universe, came together within the odds of 1 out of every atom in the universe. Thats faith. Anyone with the HS is positive.

What you did is collect data–you piled just enough to have human faith. Thats useless because all it takes is to remove one of your lines of data to collapse your belief. If God wanted belief to based on data he would be on TV right now. You have to go to Him. He has given us just enough to go to him and just enough to say its all BS so we wont be coerced. This weeds out the pretenders. You went with your head when this is a matter of the heart.
Atheists think they have modern science as their excuse when they are no different than any other unbeliever in any other era. Everyone has their reasons for not wanting authority over themselves. Science has clearly show the insurmountable odds of even the atom holding itself together randomly so they’re now grabbing for fantasies that make a mockery out of ockums razor like multiverses etc.
You know intelligence created the universe you just dont want to worship the Creator. Its the same in every century only this group claims to be geniuses. Im in the top 2% and this is not about intelligence its about wisdom and the heart.

You can pile up data all the live long day so you can sleep at night. Roam forums to hear what your itching ears want to hear but God created the world with a message that was designed to specifically draw those who are for God to Him. Its actually meaningless that something cannot come from nothing in a cause and effect spacetime universe. Even if you became unbiased and excepted the obvious, that fact wouldn’t draw you to Christ. Data doesn’t draw anyone…the heart does, love does…what Christ did does.

  (Quote)

Beelzebub November 25, 2011 at 12:59 am

Beelzebub,

Re: domain name. What would you suggest?

FacingSingularity.com, or something more appropriate? Of course, it’s up to you. Just seems like commonsenseatheism doesn’t jibe too much with your current focus.

  (Quote)

Beelzebub November 25, 2011 at 1:11 am

Robert,
You can’t will belief. In order to “give your life to Christ” you must first believe in Christ. It’s a dog biting its own tail; it begs the question. If you carefully analyze what you wrote, you’re saying precisely this. According to you, Luke was not a Christian because Luke was not a Christian. It’s a vacuous statement. I would say that Luke sounds like he had a much more typical Christian experience than what you describe. Even the likes of WL Craig say that Christianity should be rigorously scrutinized and that doubt is a part of faith. You sound like an extreme example of an unquestioning Christian.

  (Quote)

epistememe November 25, 2011 at 5:15 am

Robert,

Why is there different religions that is supposed to be based on revealed wisdom from a single source?

Religion makes people satisfied with “knowing” an incorrect answer.

  (Quote)

epistememe November 25, 2011 at 5:29 am

Bertrand Russel’s definition of faith: A belief which cannot be shaken by evidence to the contrary.

  (Quote)

T-Woo November 25, 2011 at 8:01 am

This clearly shows You were never a Christian.

Robert must think Luke is God. He just capitalized the “y” in “You” (referring to Luke).

Robert, please pray to God that he, I mean, He will forgive you for making a false god of Luke.

Amen.

  (Quote)

T-Woo November 25, 2011 at 8:05 am

Data doesn’t draw anyone…the heart does, love does…what Christ did does.

You mean like when Christ cast those demons out of that guy and into a herd of swine that then drowned themselves? Yeah, that story drew my <3 to Gentle Jesus too.

  (Quote)

drj November 25, 2011 at 9:06 am

Hey Luke,

I see you referenced Carrier’s latest blog post about his debate with a desirist, in one of the comments on lesswrong.

I think he makes a fairly convincing case that desirism is a subset of his own theory (I’ve often noted much consilience between the two). Do you have any quick or specific points where you might disagree with that conclusion? Just curious!

  (Quote)

Zeb November 25, 2011 at 10:17 am

Luke, glad to see you back to a more conversational style of blogging, even if your focus is less interesting to me than it once was. Still, if anyone can persuade me of the likelihood and immanence of Singularity, it is probably you so I look forward to this series. Curious, would you change the focus of your activities if you changed your estimation about the likelihood or immanence of singularity?

I’m glad you are watchful of your own possible tendency to leap from grand theory+mission to another. Have you also noted the irony of the fact that you left behind belief in a omniscient+omnipotent+omnibenevolent intelligent master of reality who grants eternal life in paradise, only to arrive at a belief that you can help to create the what is effectively that same? That’s no argument against the truth of your beliefs or the validity of your work, but it probably does count as evidence against the reliability of your belief forming process. (No offense; my own belief set is riddled with such ‘convenient coincidences’ that, to be fair, should make me and everyone else a little suspicious of my own reliability.)

  (Quote)

Peter Hurford November 25, 2011 at 12:23 pm

I see you referenced Carrier’s latest blog post about his debate with a desirist, in one of the comments on lesswrong.

I think he makes a fairly convincing case that desirism is a subset of his own theory (I’ve often noted much consilience between the two). Do you have any quick or specific points where you might disagree with that conclusion? Just curious!

Carrier’s post is at http://richardcarrier.blogspot.com/2011/10/goal-theory-update.html for those who are interested. I also found it compelling.

  (Quote)

Leave a Comment