Reading Yudkowsky, part 27

by Luke Muehlhauser on April 10, 2011 in Eliezer Yudkowsky,Resources,Reviews

AI researcher Eliezer Yudkowsky is something of an expert at human rationality, and at teaching it to others. His hundreds of posts at Less Wrong are a treasure trove for those who want to improve their own rationality. As such, I’m reading all of them, chronologically.

I suspect some of my readers want to “level up” their rationality, too. So I’m keeping a diary of my Yudkowsky reading. Feel free to follow along.

His 209th post, Posting on Politics, warns that the next three posts will be about politics. The Two-Party Swindle ends:

Getting emotional over politics as though it were a sports game – identifying with one color and screaming cheers for them, while heaping abuse on the other color’s fans – is a very good thing for the Professional Players’ Team; not so much for Team Voters.

The American System and Misleading Labels shares some thoughts about how labels affect the American political process. The third politics post offers the rather obvious advice, Stop Voting for Nincompoops:

…by the time you get to choose one of four “serious” “front-running” candidates, that is, the ones approved by both the party structure and the media, you’re choosing between 90.8% nincompoop and 90.6% nincompoop.

I seriously think the best thing you can do about the situation, as a voter, is stop trying to be clever.  Don’t try to vote for someone you don’t really like, because you think your vote is more likely to make a difference that way.  Don’t fret about “electability”.  Don’t try to predict and outwit other voters.  Don’t treat it as a horse race.  Don’t worry about “wasting your vote” – it always sends a message, you may as well make it a true message.

Remember that this is not the ancestral environment, and that you won’t die if you aren’t on the winning side.  Remember that the threat that voters as a class hold against politicians as a class is more important to democracy than your fights with other voters.  Forget all the “game theory” that doesn’t take future incentives into account; real game theory is further-sighted, and besides, if you’re going to look at it that way, you might as well stay home.  When you try to be clever, you usually end up playing the Politicians’ game.

Clear your mind of distractions…

And stop voting for nincompoops.

If you vote for nincompoops, for whatever clever-sounding reason, don’t be surprised that out of 300 million people you get nincompoops in office.

The arguments are long, but the voting strategy they imply is simple:  Stop trying to be clever, just don’t vote for nincompoops.

Okay, enough politics for now! The next post is Rational vs. Scientific Ev-Psych:

Years ago, before I left my parents’ nest, I was standing in front of a refrigerator, looking inside.  My mother approached and said, “What are you doing?”  I said, “Looking for the ketchup.  I don’t see it.”

My mother reached behind a couple of bottles and took out the ketchup.

She said, “If you don’t see the ketchup, why don’t you move things around and look behind them, instead of just standing and staring into the refrigerator?  Do you think the ketchup is magically going to appear if you stare into the refrigerator long enough?”

And lo, the light went on over my head, and I said:  “Men are hunters, so if we can’t find our prey, we instinctively freeze motionless and wait for it to wander into our field of vision.  Women are gatherers, so they move things around and look behind them.”

Now this sort of thing is not scientifically respectable; it is called a “just-so story”, after Kipling’s “Just-So Stories” like “How the Camel Got His Hump”.  The implication being that you can make up anything you like for an evolutionary story, but the difficult thing is finding a way to prove it.

Well, fine, but I bet it’s still true.

Here is an odd thing:  There are some people who will, if you just tell them the Refrigerator Hypothesis, snort and say “That’s an untestable just-so story” and dismiss it out of hand; but if you start by telling them about the gaze-tracking experiment and then explain the evolutionary motivation, they will say, “Huh, that might be right.”  Because then, you see, you are doing Science; but in the earlier case, you are just making up a story without testing it, which is very Unscientific.  We all know that Scientific hypotheses are more likely to be true than Unscientific ones.

This pattern of belief is very hard to justify from a Bayesian perspective.  It is just the same hypothesis in both cases.  Even if, in the second case, I announce an experimental method and my intent to actually test it, I have notyet experimented and I have not yet received any observational evidence in favor of the hypothesis.  So in either case, my current estimate should equal my prior probability, from the degree to which the “just-so story” seems “just” and “so”.  You can’t revise your beliefs based on an expected experimental success, by Conservation of Expected Evidence; if you expect the experiment to succeed, that just says that it was a rationally convincing just-so story.

People confuse the distinction between rationality and science.  Science is a special kind of strong evidence, a subset of rational evidence; if I say that my socks are currently white, you have a rational reason to believe, but it is not Science, because there is no experiment people can perform to independently verify the belief.

If you have a hypothesis you have not figured out how to test with an organized, rigorous experiment, then your hypothesis is not very scientific.  When you figure out how to do an experiment, and more importantly, set out to do the experiment, then your hypothesis becomes very scientific indeed.  If you are judging probabilities using the affect heuristic, and you know that science is a Good Thing, then making the jump from “merely rational” to “scientific” might seem to raise the probability.

But this itself is not rational.  Figuring out a way to test a belief with an organized, rigorous, repeatable experiment is certainly a Good Thing – but it should not raise the belief’s rational probability in advance of the experiment succeeding!  A hypothesis may become “more scientific” because you are going to test it, but it doesn’t get any of the power of scientific confirmation until the experiment succeeds.  To whatever degree you guess that the experiment might work – that it’s likely enough to be worth performing – you must have a rational probabilistic belief in the hypothesis being true, in advance of any scientific confirmation of it.

I conclude that an evolutionary just-so story, whose predictions you cannot figure out how to test in any organized, rigorous, repeatable way, may nonetheless have a substantial rational credibility – equalling the degree to which you would expect a rigorous experiment to succeed, if you could only figure one out.

A Failed Just-So Story argues that a certain theory of the origin of religion is wrong.

But There’s Still a Chance, Right? opens:

Years ago, I was speaking to someone when he casually remarked that he didn’t believe in evolution.  And I said, “This is not the nineteenth century.  When Darwin first proposed evolution, it might have been reasonable to doubt it.  But this is the twenty-first century.  We can read the genes. Humans and chimpanzees have 98% shared DNA.  We know humans and chimps are related.  It’s over.

He said, “Maybe the DNA is just similar by coincidence.”

I said, “The odds of that are something like two to the power of seven hundred and fifty million to one.”

He said, “But there’s still a chance, right?”

The Fallacy of Gray opens with a quote from David’s Sling:

The Sophisticate:  “The world isn’t black and white.  No one does pure good or pure bad. It’s all gray.  Therefore, no one is better than anyone else.”

The Zetet:  “Knowing only gray, you conclude that all grays are the same shade.  You mock the simplicity of the two-color view, yet you replace it with a one-color view…”

Eliezer adds:

I don’t know if the Sophisticate’s mistake has an official name, but I call it the Fallacy of Gray.  We saw it manifested in yesterday’s post – the one who believed that odds of two to the power of seven hundred and fifty millon to one, against, meant “there was still a chance”.  All probabilities, to him, were simply “uncertain” and that meant he was licensed to ignore them if he pleased.

“The Moon is made of green cheese” and “the Sun is made of mostly hydrogen and helium” are both uncertainties, but they are not the same uncertainty.

Everything is shades of gray, but there are shades of gray so light as to be very nearly white, and shades of gray so dark as to be very nearly black.  Or even if not, we can still compare shades, and say “it is darker” or “it is lighter”.

One of the commenters added a quote from Asimov:

When people thought the earth was flat, they were wrong.  When people thought the earth was spherical, they were wrong.  But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.

Previous post:

Next post:

{ 3 comments… read them below or add one }

Kyle Key April 10, 2011 at 9:03 am

Or, you know, stop voting, ’cause we don’t need federal babysitters, and voting is the only legitimacy that the State has. We don’t need to pay people higher salaries than most people make so that they can decide what everyone else can and can’t do. Authority corrupts and non-consensual hierarchies are exploitative. Stop giving other people authority over you.


Kevin April 10, 2011 at 9:55 am

Kyle, not voting won’t change that. Its not like voting gives other people authority over you, it only decides who it will be. There will be authority over you regardless of whether you vote or not, not voting won’t change that.


Cyril April 11, 2011 at 8:53 pm

Other people will have authority over you whether you like it or not. Even if you we disbanded the government and had an anarchistic plot of land, people would still impose their wills upon others just like happens every day:


The list could go on and on, none of these things are in any way consensual or allowed. They’re all people imposing their wills upon others. The idea behind democracy is that we can vote and choose the people to have power over us and keep that power from being used for the above things or things like it.

Does that always work? No. But spouting nonsense about non-consensual hierarchies doesn’t help anyone.


Leave a Comment