Reading Yudkowsky, part 44

by Luke Muehlhauser on June 9, 2011 in Eliezer Yudkowsky,Resources,Reviews

AI researcher Eliezer Yudkowsky is something of an expert at human rationality, and at teaching it to others. His hundreds of posts at Less Wrong are a treasure trove for those who want to improve their own rationality. As such, I’m reading all of them, chronologically.

I suspect some of my readers want to “level up” their rationality, too. So I’m keeping a diary of my Yudkowsky reading. Feel free to follow along.

His 369th post is The Rhythm of Disagreement. When should we disagree with experts? Eliezer recounts some of his disagreements that turned into agreements, but finds no pattern yet.  Principles of Disagreement, though, has some advice:

There’s a rhythm to disagreement.  And oversimplified rules about when to disagree, can distract from that rhythm.  Even “Follow arguments, not people” can distract from the rhythm, because no one, including my past self, really uses that rule in practice.

The way it works in real life is that I just do the standard first-order disagreement analysis:  Okay, in real life, how likely is it that this person knows stuff that I don’t?

Not, Okay, how much of the stuff that I know that they don’t, have they already taken into account in a revised estimate, given that they know I disagree with them, and have formed guesses about what I might know that they don’t, based on their assessment of my and their relative rationality…

Why don’t I try the higher-order analyses?  Because I’ve never seen a case where, even in retrospect, it seems like I could have gotten real-life mileage out of it.  Too complicated, too much of a tendency to collapse to tribal status, too distracting from the object-level arguments.

Timeless Identity begins with a question:

People have asked me, “What practical good does it do to discuss quantum physics or consciousness or zombies or personal identity?  I mean, what’s the application for me in real life?”

Before the end of today’s post, we shall see a real-world application with practical consequences, for you, yes, you in today’s world.  It is built upon many prerequisites and deep foundations; you will not be able to tell others what you have seen, though you may (or may not) want desperately to tell them.  (Short of having them read the last several months of [the blog].)

The rest is a discussion of personal identity, following some arguments in Parfit’s Reasons and Persons, and responding with discoveries from the series on quantum mechanics. The moral is:

This whole episode is one of the main reasons why I hope that when I really understand matters such as these, and they have ceased to be mysteries unto me, that I will be able to give definite answers to questions that seem like they ought to have definite answers.

And it is a reason why I am suspicious, of philosophies that too early – before the dispelling of mystery – say, “There is no answer to the question.”  Sometimes there is no answer, but then the absence of the answer comes with a shock of understanding, a click like thunder, that makes the question vanish in a puff of smoke.  As opposed to a dull empty sort of feeling, as of being told to shut up and stop asking questions.

And another lesson:  Though the thought experiment of having atoms “replaced” seems easy to imagine in the abstract, anyone knowing a fully detailed physical visualization would have immediately seen that the thought experiment was physical nonsense.  Let zombie theorists take note!

Okay, but what’s the real-world application for all this?

Many throughout time, tempted by the promise of immortality, have consumed strange and often fatal elixirs; they have tried to bargain with devils that failed to appear; and done many other silly things.

But like all superpowers, long-range life extension can only be acquired by seeing, with a shock, that some way of getting it is perfectly normal.

If you can see the moments of now braided into time, the causal dependencies of future states on past states, the high-level pattern of synapses and the internal narrative as a computation within it – if you can viscerally dispel the classical hallucination of a little billiard ball that is you, and see your nows strung out in the river that never flows – then you can see that signing up for cryonics, being vitrified in liquid nitrogen when you die, and having your brain nanotechnologically reconstructed fifty years later, is actually less of a change than going to sleep, dreaming, and forgetting your dreams when you wake up.

You should be able to see that, now, if you’ve followed through this whole series.  You should be able to get it on agut level – that being vitrified in liquid nitrogen for fifty years (around 3e52 Planck intervals) is not very different from waiting an average of 2e26 Planck intervals between neurons firing, on the generous assumption that there are a hundred trillion synapses firing a thousand times per second.  You should be able to see that there is nothing preserved from one night’s sleep to the morning’s waking, which cryonic suspension does not preserve also.  Assuming the vitrification technology is good enough for a sufficiently powerful Bayesian superintelligence to look at your frozen brain, and figure out “who you were” to the same resolution that your morning’s waking self resembles the person who went to sleep that night.

The payoff is, according to Eliezer, a realistic chance at fucking immortality (or at least radical life extension). If true, that’s one hell of a payoff!

Someone hears about cryonics and thinks for 10 seconds and says, “But if you’re frozen and then revived, are you really the same person?

And if they happened to know all about quantum physics and could apply the abstract knowledge to real life, and they had followed the whole debate about zombies and resolved it against epiphenomenalism in general, then they would be able to visualize the braids in the river that never flows, and say, “Yes.”

But this knowledge is not common.

So they die.

Why Quantum? explains again why Eliezer has written so many posts about quantum mechanics.

Living in Many Worlds opens:

Some commenters have recently expressed disturbance at the thought of constantly splitting into zillions of other people, as is the straightforward and unavoidable prediction of quantum mechanics.

Others have confessed themselves unclear as to the implications of many-worlds for planning:  If you decide to buckle your seat belt in this world, does that increase the chance of another self unbuckling their seat belt?  Are you being selfish at their expense?

Just remember Egan’s Law:  It all adds up to normality.

When Einstein overthrew the Newtonian version of gravity, apples didn’t stop falling, planets didn’t swerve into the Sun.  Every new theory of physics must capture the successful predictions of the old theory it displaced; it should predict that the sky will be blue, rather than green.

So don’t think that many-worlds is there to make strange, radical, exciting predictions.  It all adds up to normality.

…Bear this in mind, when you are wondering how to live in the strange new universe of many worlds:  You have always been there.

The act of making decisions has no special interaction with the process that branches worlds.  In your mind, in your imagination, a decision seems like a branching point where the world could go two different ways.  But you would feel just the same uncertainty, visualize just the same alternatives, if there were only one world.  That’s what people thought for centuries before quantum mechanics, and they still visualized alternative outcomes that could result from their decisions.

Decision and decoherence are entirely orthogonal concepts. If your brain never became decoherent, then that single cognitive process would still have to imagine different choices and their different outcomes.  And a rock, which makes no decisions, obeys the same laws of quantum mechanics as anything else, and splits frantically as it lies in one place.

You don’t split when you come to a decision in particular, any more than you particularly split when you take a breath.  You’re just splitting all the time as the result of decoherence, which has nothing to do with choices.

There is a population of worlds, and in each world, it all adds up to normality: apples don’t stop falling.  In each world, people choose the course that seems best to them.

Previous post:

Next post:

{ 3 comments… read them below or add one }

Katie June 11, 2011 at 10:25 am

I’m curious, Luke, what do you think of Eliezer’s Quantum Physics sequence? It’s his most criticized, and least upvoted, and I get the sense that a lot of people on Less Wrong embraced his conclusions without fully understanding the arguments (since few of them are really physicists). That doesn’t mean he’s wrong, but this is an arena where it’s harder to trust him instinctively.


Luke Muehlhauser June 12, 2011 at 12:48 am


Haven’t read it in detail. :)


CharlesR June 12, 2011 at 9:49 pm

The argument he makes isn’t terrible difficult to understand. Many Worlds is just like Wave Functions Collapse with one less assumption: it removes the idea that wave functions collapse.


Leave a Comment