Reading Yudkowsky, part 37

by Luke Muehlhauser on May 15, 2011 in Design Argument,Resources,Reviews

AI researcher Eliezer Yudkowsky is something of an expert at human rationality, and at teaching it to others. His hundreds of posts at Less Wrong are a treasure trove for those who want to improve their own rationality. As such, I’m reading all of them, chronologically.

I suspect some of my readers want to “level up” their rationality, too. So I’m keeping a diary of my Yudkowsky reading. Feel free to follow along.

His 290th post is Explaining vs. Explaining Away. When Newton succeeded in unweaving the rainbow, he explained the rainbow, but he did not explain it away. Other things have indeed been explained away, though, like ghosts and gnomes.

What’s the difference?

I think that when physicists say “There are no fundamental rainbows,” the anti-reductionists hear, “There are no rainbows.”

If you don’t distinguish between the multi-level map and the mono-level territory, then when someone tries to explain to you that the rainbow is not a fundamental thing in physics, acceptance of this will feel like erasing rainbows from your multi-level map, which feels like erasing rainbows from the world.

When Science says “tigers are not elementary particles, they are made of quarks” the anti-reductionist hears this as the same sort of dismissal as “we looked in your garage for a dragon, but there was just empty air”.

Concerning gnomes…

Scientists didn’t do anything to gnomes, only to “gnomes”.  The quotation is not the referent.

But if you commit the Mind Projection Fallacy – and by default, our beliefs just feel like the way the world is – then at time T=0, the mines (apparently) contain gnomes; at time T=1 a scientist dances across the scene, and at time T=2 the mines (apparently) are empty.  Clearly, there used to be gnomes there, but the scientist killed them.

Bad scientist!  No poems for you, gnomekiller!

Well, that’s how it feels, if you get emotionally attached to the gnomes, and then a scientist says there aren’t any gnomes.  It takes a strong mind, a deep honesty, and a deliberate effort to say, at this point, “That which can be destroyed by the truth should be,” and “The scientist hasn’t taken the gnomes away, only taken my delusion away,” and “I never held just title to my belief in gnomes in the first place; I have not been deprived of anything Irightfully owned,” and “If there are gnomes, I desire to believe there are gnomes; if there are no gnomes, I desire to believe there are no gnomes; let me not become attached to beliefs I may not want,” and all the other things that rationalists are supposed to say on such occasions.

But with the rainbow it is not even necessary to go that far.  The rainbow is still there!

Which leads to Eliezer’s warning about Fake Reductionism:

I am guessing, though it is only a guess, that Keats could not have sketched out on paper why rainbows only appear when the Sun is behind your head, or why the rainbow is an arc of a circle.

If so, Keats had a Fake Explanation.  In this case, a fake reduction.  He’d been told that the rainbow had been reduced, but it had not actually been reduced in his model of the world.

This is another of those distinctions that anti-reductionists fail to get – the difference between professing the flat fact that something is reducible, and seeing it.

In this, the anti-reductionists are not too greatly to be blamed, for it is part of a general problem.

I’ve written before on seeming knowledge that is not knowledge, and beliefs that are not about their supposed objects but only recordings to recite back in the classroom, and words that operate as stop signs for curiosityrather than answers, and technobabble which only conveys membership in the literary genre of “science”

There is a very great distinction between being able to see where the rainbow comes from, and playing around with prisms to confirm it, and maybe making a rainbow yourself by spraying water droplets -

- versus some dour-faced philosopher just telling you, “No, there’s nothing special about the rainbow.  Didn’t you hear? Scientists have explained it away.  Just something to do with raindrops or whatever.  Nothing to be excited about.”

I think this distinction probably accounts for a hell of a lot of the deadly existential emptiness that supposedly accompanies scientific reductionism.

…For [anti-reductionists], the effect of hearing “Science has explained rainbows!” is to hang up a sign over rainbows saying, “This phenomenon has been labeled BORING by order of the Council of Sophisticated Literary Critics.  Move along.”

Savanna Poets has an interesting take on storytelling. Joy in the Merely Real responds to a poem by Keats:

…Do not all charms fly
At the mere touch of cold philosophy?
There was an awful rainbow once in heaven:
We know her woof, her texture; she is given
In the dull catalogue of common things.

Eliezer responds with wit:

You’ve got to admire that phrase, “dull catalogue of common things”.  What is it, exactly, that goes in this catalogue?  Besides rainbows, that is?

Why, things that are mundane, of course.  Things that are normal; things that are unmagical; things that are known, or knowable; things that play by the rules (or that play by any rules, which makes them boring); things that are part of the ordinary universe; things that are, in a word, real.

Now that’s what I call setting yourself up for a fall.

At that rate, sooner or later you’re going to be disappointed in everything – either it will turn out not to exist, or even worse, it will turn out to be real.

I have already remarked that nothing is inherently mysterious – nothing that actually exists, that is.  If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon; to worship a phenomenon because it seems so wonderfully mysterious, is to worship your own ignorance; a blank map does not correspond to a blank territory, it is just somewhere we haven’t visited yet, etc. etc…

Which is to say that everything – everything that actually exists – is liable to end up in “the dull catalogue of common things”, sooner or later.

The next post analyzes Joy in Discovery:

There is a story that one of the first men to realize that stars were burning by fusion – plausible attributions I’ve seen are to Fritz Houtermans and Hans Bethe – was walking out with his girlfriend of a night, and she made a comment on how beautiful the stars were, and he replied:  “Yes, and right now, I’m the only man in the world who knows why they shine.”

It is attested by numerous sources that this experience, being the first person to solve a major mystery, is atremendous high.  It’s probably the closest experience you can get to taking drugs, without taking drugs – though I wouldn’t know.

That can’t be healthy.

Not that I’m objecting to the euphoria.  It’s the exclusivity clause that bothers me.  Why should a discovery be worth less, just because someone else already knows the answer?

This leads to Bind Yourself to Reality:

So perhaps you’re reading all this, and asking:  “Yes, but what does this have to do with reductionism?”

Partially, it’s a matter of leaving a line of retreat.  It’s not easy to take something important apart into components, when you’re convinced that this removes magic from the world, unweaves the rainbow.  I do plan to take certain things apart, on this blog; and I prefer not to create pointless existential anguish.

Partially, it’s the crusade against Hollywood Rationality, the concept that understanding the rainbow subtracts its beauty.  The rainbow is still beautiful plus you get the beauty of physics.

But even more deeply, it’s one of these subtle hidden-core-of-rationality things.  You know, the sort of thing where I start talking about ‘the Way‘.  It’s about binding yourself to reality.

Science puts the fun back into life.

Rationality directs your emotional energies into the universe, rather than somewhere else.

If You Demand Magic, Magic Won’t Help ends with the Litany Against Being Transported Into an Alternate Universe:

If I’m going to be happy anywhere,
Or achieve greatness anywhere,
Or learn true secrets anywhere,
Or save the world anywhere,
Or feel strongly anywhere,
Or help people anywhere,
I may as well do it in reality.

Previous post:

Next post:

{ 11 comments… read them below or add one }

Haukur May 15, 2011 at 3:47 pm

“If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon; to worship a phenomenon because it seems so wonderfully mysterious, is to worship your own ignorance”

This seems like a total non-sequitur to me. Try using the same template to create arguments about other things. How about this:

“If I get turned on by a woman, that is a fact about my state of mind, not a fact about the woman; to hit on a woman because she seems so wonderfully sexy, is to hit on your own feelings.”

  (Quote)

Rufus May 15, 2011 at 11:06 pm

I thought a little Gabriel Marcel might be an appropriate response. This is from a very nice Stanford Encyclopedia article on Marcel:

“[Marcel] states that the broken world is one that is ‘on the one hand, riddled with problems and, on the other, determined to allow no room for mystery’ (Marcel 1995, p. 12). The denial of the mysterious is symptomatic of the modern broken world and is tied to its technical character, which only acknowledges that which technique can address: the problematic… ‘A problem is something which I meet, which I find completely before me, but which I can therefore lay siege to and reduce. But a mystery is something in which I am myself involved, and it can therefore only be thought of as a sphere where the distinction between what is in me and what is before me loses its meaning and initial validity’ (Marcel 1949, p. 117). A problem is a question in which I am not involved, in which the identity of the person asking the question is not an issue. In the realm of the problematic, it makes no difference who is asking the question because all of the relevant information is ‘before’ the questioner. As such, a problem is something that bars my way, placing an obstacle in front of me that must be overcome. In turn, the overcoming of a problem inevitably involves some technique, a technique that could be, and often is, employed by any other person confronting the same problem. Thus the identity of the questioner can be changed without altering the problem itself. This is why the modern broken world only sees the problematic: the ‘problematic’ is that which can be addressed and solved with a technique, e.g., changing a flat tire on an automobile or downloading security software to fix a virus on one’s computer. ” (B. Treanor 2010, http://plato.stanford.edu/entries/marcel/#6).

I think we can come up with at least one counter-example to Yudkowsky’s bald assertion, “…nothing is inherently mysterious – nothing that actually exists, that is. If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon; to worship a phenomenon because it seems so wonderfully mysterious, is to worship your own ignorance; a blank map does not correspond to a blank territory, it is just somewhere we haven’t visited yet, etc. etc…”

The counter is: γνῶθι σεαυτόν.

Complete self-knowledge is not possible. Here is my argument for why this is so: subject S at time t=o may take a moment to comprehend S@t=0, but in the process becomes S1@t=1 knowing S@t=0, we can continue this process, but Sn@t=n cannot come to know Sn@t=n and remain Sn@t=n. So, in this case, the object of ignorance is the ignorant object, i.e. the self-as-knower. The map is sometimes a territory.

There are plenty of other inscrutable paradoxes and mysteries, but this one is right in front of our noses. Really, I would have agreed with Yudkowsky that many phenomena are not inherently mysterious. It is when he makes the move to universalize some of his observations that he makes a hasty generalization, the most common fallacy of all. Then again, I am open to correction on this point.

Best,

Rufus

  (Quote)

Leon May 16, 2011 at 2:11 am

I think it’s actually behind our noses … :P

  (Quote)

Adito May 16, 2011 at 10:20 am

“If I get turned on by a woman, that is a fact about my state of mind, not a fact about the woman; to hit on a woman because she seems so wonderfully sexy, is to hit on your own feelings.”

This actually makes sense to me except the confusion at the end between the women and your feelings. Your phrase “to hit on” should be replaced with “to act on” and the problem goes away.

  (Quote)

Rufus May 16, 2011 at 11:32 am

I think it’s actually behind our noses … :P

Oh, that’s right. ;-). Sometimes I forget where my head is.

  (Quote)

Haukur May 16, 2011 at 1:07 pm

Your phrase “to hit on” should be replaced with “to act on” and the problem goes away.

That would make the example less incoherent – but also less analogous to the original paragraph I’m criticizing. Going back to the original, if we apply your fix to Eliezer’s text, we should replace “is to worship your own ignorance” with “is to act on your own ignorance”. And, indeed, that makes the paragraph a lot less incoherent. But that’s not what he wrote.

  (Quote)

Ryan May 16, 2011 at 1:45 pm

You know, that part about “fundamental rainbows” reminds me very much of what some people think about Daniel Dennett’s explanation of Consciousness, or the compatibilist view of Free Will, or physicalist explanations of emotions.

  (Quote)

Adito May 19, 2011 at 8:28 pm

Going back to the original, if we apply your fix to Eliezer’s text, we should replace “is to worship your own ignorance” with “is to act on your own ignorance”. And, indeed, that makes the paragraph a lot less incoherent. But that’s not what he wrote.

To worship is to act in a certain way so there’s no need to modify Eliezer’s statement.

What arouses you is clearly a function of your mind and your actions are also a function of your mind. But it does not follow from this that the object that arouses you or the objects you act towards are themselves a function of your mind.

So, in this case, the object of ignorance is the ignorant object, i.e. the self-as-knower. The map is sometimes a territory.

The self-as-knower can be broken down into component parts each of which can be understood separately and their relationships inferred when enough parts have been understood.

  (Quote)

Rufus May 19, 2011 at 9:25 pm

Adito,

The self-as-knower can be broken down into component parts each of which can be understood separately and their relationships inferred when enough parts have been understood.

How can I possibly know that any of these claims are true? Can the self-as-knower be broken down into component parts? Perhaps it cannot or should not be broken down into component parts.

You also assert that their relationships can be inferred when enough parts have been understood. However, it might be that the requisite part to understand the relationships between the parts is the self-at-present, a part that is never comprehended in the present. Perhaps some other unknowable component is required.

Also, you seem fairly certain that there is such a thing as “enough parts” to infer the relationships, but I see no reason why this is necessarily the case. Nor do I think we could ever really know how many parts must be accumulated before the inference of relationships between parts is reliable.

Further, even if you have enough parts to infer the relationships between component parts known, this information does not necessarily entail information about component parts that are not known, and so also the whole.

So, I see no reason to take your methodology as an avenue for overcoming the mystery of self-knowledge.

  (Quote)

Haukur May 22, 2011 at 9:59 am

To worship is to act in a certain way so there’s no need to modify Eliezer’s statement.

And “to hit on” is to act in a certain way – so if that’s all that’s required then my “If I get turned on by a woman” version should work as well. Yet, you seem to agree that it doesn’t.

But there’s no need to debate the analogy if you don’t think it makes the matter clearer. If you think Eliezer’s original makes sense then you can argue that directly. My claim is that his last sentence in no way follows from what came before it. The smallest fix I can think of to make Eliezer’s argument valid while retaining the word ‘worship’ at the end would be to add three words to it, like this:

“If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon; to worship a phenomenon because it seems so wonderfully mysterious, is to worship it because of your own ignorance.”

It’s easy to see how this is a valid argument if you use ‘mysterious’ in the sense ‘unknown’. But “to worship X” and “to worship Y because of X” are not at all the same thing. I don’t see any way in which the original version makes sense.

  (Quote)

Multiheaded July 6, 2011 at 4:59 am

“If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon; to worship a phenomenon because it seems so wonderfully mysterious, is to worship your own ignorance”

This seems like a total non-sequitur to me.Try using the same template to create arguments about other things. How about this:

“If I get turned on by a woman, that is a fact about my state of mind, not a fact about the woman; to hit on a woman because she seems so wonderfully sexy, is to hit on your own feelings.”

Um… duh. Is there an “essense of hotness” somewhere, an objective standard that any straight white man can detect and feel? Yes, everyone hits on his own feelings. I find that your comment was exceptionally thoughtless. That’s not a (g00d) reason to feel humiliated, it’s a reason to start thinking.

  (Quote)

Leave a Comment