Reading Yudkowsky, part 56

by Luke Muehlhauser on July 19, 2011 in Eliezer Yudkowsky,Resources,Reviews

AI researcher Eliezer Yudkowsky is something of an expert at human rationality, and at teaching it to others. His hundreds of posts at Less Wrong are a treasure trove for those who want to improve their own rationality. As such, I’m reading all of them, chronologically.

I suspect some of my readers want to “level up” their rationality, too. So I’m keeping a diary of my Yudkowsky reading. Feel free to follow along.

In his 504th post is a meetup post, followed by a few inspirational posts: On Doing the Impossible, Make an Extraordinary Effort, Shut up and do the impossible! Then, an aside on Eliezer’s Al-Box experiment, and then a post I surely identify with, Crisis of Faith:

As Premise Checker put it, “Had the idea of god not come along until the scientific age, only an exceptionally weird person would invent such an idea and pretend that it explained anything.”

And yet skillful scientific specialists, even the major innovators of a field, even in this very day and age, do not apply that skepticism successfully.  Nobel laureate Robert Aumann, of Aumann’s Agreement Theorem, is an Orthodox Jew:  I feel reasonably confident in venturing that Aumann must, at one point or another, have questioned his faith.  And yet he did not doubt successfully.  We change our minds less often than we think.

This should scare you down to the marrow of your bones.  It means you can be a world-class scientist andconversant with Bayesian mathematics and still fail to reject a belief whose absurdity a fresh-eyed ten-year-old could see.  It shows the invincible defensive position which a belief can create for itself, if it has long festered in your mind.

What general strategy would a religious person have to follow in order to escape their religion?


I’m sure that some, looking at this challenge, are already rattling off a list of standard atheist talking points – “They would have to admit that there wasn’t any Bayesian evidence for God’s existence”, “They would have to see the moral evasions they were carrying out to excuse God’s behavior in the Bible”, “They need to learn how to use Occam’s Razor -”

WRONG!  WRONG WRONG WRONG!  This kind of rehearsal, where you just cough up points you already thought of long before, is exactly the style of thinking that keeps people within their current religions.  If you stay with your cached thoughts, if your brain fills in the obvious answer so fast that you can’t see originally, you surely will not be able to conduct a crisis of faith.

…Without a convulsive, wrenching effort to be rational, the kind of effort it would take to throw off a religion – then how dare you believe anything, when Robert Aumann believes in God?

Eliezer proposes some warning signs for when a crisis of faith may be needed:

Not every doubt calls for staging an all-out Crisis of Faith.  But you should consider it when:

  1. A belief has long remained in your mind;
  2. It is surrounded by a cloud of known arguments and refutations;
  3. You have sunk costs in it (time, money, public declarations);
  4. The belief has emotional consequences (note this does not make it wrong);
  5. It has gotten mixed up in your personality generally.

None of these warning signs are immediate disproofs.  These attributes place a belief at-risk for all sorts of dangers, and make it very hard to reject when it is wrong.

Some positions of my own that suspiciously match some of these criteria include: atheism (2 & 3), desirism (2 & 3), and scientific realism (1 & 5).

The post goes on to recommend some techniques for staging a crisis of faith.

The Ritual is a followup ceremony to The Failures of Eld Science, and then there are more rationality quotes, and then Eliezer examines the evolutionary psychology story behind Why Does Power Corrupt?, which leads to Ends Don’t Justify Means (Among Humans).

Entangled Truths, Contagious Lies examines the issue of “not technically a lie.” Eliezer then lists some Traditional Capitalist Values to show how far our current capitalisms are from them.

Previous post:

Next post:

{ 2 comments… read them below or add one }

CharlesR July 19, 2011 at 6:16 pm

I stopped referring to myself as an ‘atheist’ largely to avoid #5. I don’t even use ‘skeptic’ because I thought it could lead to my needing too much evidence to accept a claim. I’ve settled on ‘rationalist’. Can’t think of a downside to that one.

  (Quote)

hf July 23, 2011 at 2:29 pm

Every name has a downside. *evil laughter*

  (Quote)

Leave a Comment