AI researcher Eliezer Yudkowsky is something of an expert at human rationality, and at teaching it to others. His hundreds of posts at Less Wrong are a treasure trove for those who want to improve their own rationality. As such, I’m reading all of them, chronologically.
I suspect some of my readers want to “level up” their rationality, too. So I’m keeping a diary of my Yudkowsky reading. Feel free to follow along.
Honesty: Beyond Internal Truth explores the relation between honesty and rationality.
Nonparametric Ethics responds to a suggestion by Robin Hanson that because we are so prone to error, we should seek our simple moral rules. (Eliezer disagrees.)
Of greater interest to the readers of this site is probably Atheism = Untheism + Antitheism:
One occasionally sees such remarks as, “What good does it do to go around being angry about the nonexistence of God?” (on the one hand) or “Babies are natural atheists” (on the other). It seems to me that such remarks, and the rather silly discussions that get started around them, show that the concept “Atheism” is really made up of two distinct components, which one might call “untheism” and “antitheism”.
Next is a discussion of Timeless Decision Theory: Problems I Can’t Solve, followed by Ingredients of Timeless Decision Theory and Timeless Decision Theory and Meta-Circular Decision Theory.
Unspeakable Morality proposes some guidelines for arguing about morality with words:
- “Intuition” is not a trump card. If you had to spell out what your intuition was, and where it came from (evolution? culture?), and whether it has consequences beyond itself, it’s possible that we would find it unconvincing in the stark light of reflection; that we would wish to intuit some other intuition than this. We can’t hold up the intuition for reflective judgment unless we know what it is. So spelling it out, is important; and if you can win arguments by saying “Intuition!” then no one will bother to spell things out any more. Please try to say what sort of intuition it is.
- “I can’t put it into words” is believable to some extent, but constitutes weak evidence against the existence of valid justification. If this is a popular debate and no one on your side, politician or philosopher or interested scientist or eloquent blogger, is able to give a convincing justification in words, then that is stronger evidence that no good justification exists. The longer the failure continues, the stronger the evidence.
- Still, at the end of the day, we don’t really expect people to be very good at verbalizing moral intuitions, especially since most of them have incoherent explicit metaethics. So if you can give a justification for your political policy that stutters off into incoherence only at the point of explaining why pain is a bad thing – if you can give reasonable arguments for everything else up until that point – that’s probably about as much as we can demand of anyone short of a full-fledged master reductionist.
- But we also expect that people may pass judgments that they would revoke in the light of better information or new arguments; and, especially before passing to that limit, it may be that sociopaths do not overlap with the values shared by most in a society. So if A says that event B is inherently wrong and awful, and C disagrees on the grounds that it just doesn’t seem all that awful to them, then the burden of argument needs to lie on A before any social, legal, public action is brought into play. We should bear in mind that people of the past would have a lot of icky feelings about things that we, today, think are not only permitted but virtuous or even mandatory – the challenging of these icky feelings for good and sufficient public justification, was a key element of their relinquishment, which we regard as moral progress.
- “If anyone could actually update on the evidence, they would have a power far beyond that of Nobel Prize winners.”
(When encountering a need to discard some idea to which I was attached, or admit loss of a sunk cost on an avenue that doesn’t seem to be working out.)
- “The universe is already like [X], or not; if it is then I can only minimize the embarrassment by admitting itand adapting as fast as possible.”
(If the first mantra doesn’t work; then I actually visualize the universe already being a certain way, so that I can see the penalty for being a universe that works a certain way and yet believing otherwise.)
- “First understand the problem, then solve it.”
(If getting too caught up in proposing solutions, or discouraged when solutions don’t work out – the immediate task at hand is just to understand the problem, and one may ask whether progress has been made on this. From full understanding a solution usually follows quickly.)
- “Load the problem.”
(Try to get your mind involved and processing the various aspects of it.)
- “Five minutes is enough time to have an insight.”
(If my mind seems to be going empty.)
- “Ask only one thing of your mind and it may give it to you.”
(Focusing during work, or trying to load the problem into memory before going to sleep each night, in hopes of putting the subconscious to work on it.)
- “Run right up the mountain!”
(My general visualization of the FAI problem; a huge, blank, impossibly high wall, which I have to run up as quickly as possible. Used to accomodate the sense of the problem being much larger than whatever it is I’m working on right now.)
- “When the problem is solved, that thought will be a wasted motion in retrospect.”
(I first enunciated this as an explicit general principle when explaining to Marcello why e.g. one doesn’t worry about people who have failed to solve a problem previously. When you actually solve the problem, those thoughts will predictably not have contributed anything in retrospect. So if your goal is to solve the problem, you should focus on the object-level problem, instead of worrying about whether you havesufficient status to solve it. The same rule applies to many other habitual worries, or reasoning effort expended to reassure against them, that would predictably appear as wasted motion in retrospect, after actually solving the problem.)
- “There’s always just enough time when you do something right, no more, no less.”
(A quote from C. J. Cherryh’s Paladin, used when feeling rushed. I don’t think it’s true literally or otherwise, but it seems to convey an important wordless sentiment.
- “See the truth, not what you expect or hope.”
(When expecting the answer to go a particular way, or hoping for the answer to go a particular way, is exerting detectable pressure on an ongoing inquiry.)
The Sword of Good is a fragment of a novel that will never be written.