Reading Yudkowsky, part 3

by Luke Muehlhauser on November 27, 2010 in Eliezer Yudkowsky,Resources,Reviews

AI researcher Eliezer Yudkowsky is something of an expert at human rationality, and at teaching it to others. His hundreds of posts at Overcoming Bias (now moved to Less Wrong) are a treasure trove for those who want to improve their own rationality. As such, I’m reading all of them, chronologically.

I suspect some of my readers want to improve their rationality, too. So I’m keeping a diary of my Yudkowsky reading. Feel free to follow along.

Yudkowsky’s 18th post is Useless Medical Disclaimers, in which he decries the uselessness of medical disclaimers that talk about “possible” complications, without attaching any rates of occurrence to them.

Archimedes’ Chronophone is an odd thought experiment with an unclear purpose. The point, as explained in Chronophone Motivations, is this:

…the most important things the Future would want to say to us are, amazingly enough, not things that everyone already knows. If you want to really benefit humanity, you’ve got to do some original thinking…

Next up is Self-deception: hypocrisy or akrasia?

In the traditional sense of the term, a hypocrite is a moral liar: someone who says a morality which they do not, themselves, believe. On the other hand, we don’t always live up to the goals we set for ourselves. If I really believe that I ought to exercise at least 3 times per week, but I don’t always do so, am I properly termed a “hypocrite”? The term akrasia, meaning “weakness of will” or “failure of self-control”, seems more appropriate. Even if I tell all my friends that they ought to exercise 3 times per week, that doesn’t necessarily make me a hypocrite. It’s good advice. (Now, if I claimed to always exercise 3 times per week, knowing that this claim was false, that would be dishonest.)

Accusations of hypocrisy garner a lot more attention than accusations of akrasia – because hypocrisy is a deliberate transgression. It is tempting to say “hypocrisy” when you really mean “akrasia”, because you’ll get more attention, but that can cause damage to innocent bystanders.

So what if someone who claims to want to know the truth self-deceives? Is he a hypocrite, or does he merely fail to live up to his own standards due to akrasia? Hard to say.

In Tsuyoku Naritai! (I Want To Become Stronger), Yudkowsky writes that:

[Some people think] the way of rationality is to beat your fist against your heart and say, “We are all biased, we are all irrational, we are not fully informed, we are overconfident, we are poorly calibrated…”

Fine. Now tell me how you plan to become less biased, less irrational, more informed, less overconfident, better calibrated.

In the followup, Tsuyoku vs. the Egalitarian Instinct, Yudkowsky writes:

If you can’t admit to yourself that you’ve done better than others [in being rational] – or if you’re ashamed of wanting to do better than others – then the median will forever be your concrete wall, the place where you stop moving forward. And what about people who are below average? Do you dare say you intend to do better than them? How prideful of you!

Maybe it’s not healthy to pride yourself on doing better than someone else. Personally I’ve found it to be a useful motivator, despite my principles, and I’ll take all the useful motivation I can get.

Now is a good time to jump through one of Yudkowsky’s central – and older – essays: Twelve Virtues of Rationality. Here are the bits that were most impactful on me:

The first virtue is curiosity. A burning itch to know is higher than a solemn vow to pursue truth… Be wary of those who speak of being open-minded and modestly confess their ignorance. There is a time to confess your ignorance and a time to relinquish your ignorance.

The second virtue is relinquishment. P. C. Hodgell said: “That which can be destroyed by the truth should be.” Do not flinch from experiences that might destroy your beliefs…

The third virtue is lightness. Let the winds of evidence blow you about as though you are a leaf, with no direction of your own… Surrender to the truth as quickly as you can.

The fourth virtue is evenness. One who wishes to believe says, “Does the evidence permit me to believe?” One who wishes to disbelieve asks, “Does the evidence force me to believe?” Beware lest you place huge burdens of proof only on propositions you dislike, and then defend yourself by saying: “But it is good to be skeptical.” …Therefore do not seek to argue for one side or another, for if you knew your destination, you would already be there.

The fifth virtue is argument. Those who wish to fail must first prevent their friends from helping them. Those who smile wisely and say: “I will not argue” remove themselves from help, and withdraw from the communal effort… Seek a test that lets reality judge between you.

The sixth virtue is empiricism. The roots of knowledge are in observation and its fruit is prediction… Do not ask which beliefs to profess, but which experiences to anticipate. Always know which difference of experience you argue about. Do not let the argument wander and become about something else, such as someone’s virtue as a rationalist. Jerry Cleaver said: “What does you in is not failure to apply some high-level, intricate, complicated technique. It’s overlooking the basics. Not keeping your eye on the ball.” Do not be blinded by words. When words are subtracted, anticipation remains.

The seventh virtue is simplicity… When you profess a huge belief with many details, each additional detail is another chance for the belief to be wrong. Each specification adds to your burden; if you can lighten your burden you must do so…

The eighth virtue is humility. To be humble is to take specific actions in anticipation of your own errors. To confess your fallibility and then do nothing about it is not humble; it is boasting of your modesty. Who are most humble? Those who most skillfully prepare for the deepest and most catastrophic errors in their own beliefs and plans…

The ninth virtue is perfectionism. The more errors you correct in yourself, the more you notice… If you tolerate the error rather than correcting it, you will not advance to the next level and you will not gain the skill to notice new errors… Hold yourself to the highest standard you can imagine, and look for one still higher…

The tenth virtue is precision. One comes and says: The quantity is between 1 and 100. Another says: the quantity is between 40 and 50. If the quantity is 42 they are both correct, but the second prediction was more useful and exposed itself to a stricter test… Each piece of evidence shifts your beliefs by exactly the right amount, neither more nor less. What is exactly the right amount? To calculate this you must study probability theory…

The eleventh virtue is scholarship. Study many sciences and absorb their power as your own. Each field that you consume makes you larger. If you swallow enough sciences the gaps between them will diminish and your knowledge will become a unified whole…

Before these eleven virtues is a virtue which is nameless.

…Every step of your reasoning must cut through to the correct answer in the same movement. More than anything, you must think of carrying your map through to reflecting the territory.

…Do not ask whether it is “the Way” to do this or that. Ask whether the sky is blue or green. If you speak overmuch of the Way you will not attain it.

Previous post:

Next post:

{ 1 comment… read it below or add one }

stuart November 28, 2010 at 11:18 pm

I’m certain that you’ve already though of this, but I think he’d be a good guest on your show.

In previous interviews and discussions I have listened to, he gives the impression of processing and considering what is being said in real time rather than having a ready response to things that happen to fall into a predetermined category (which is what most of us do and is not meant as a criticism of anyone).


Leave a Comment