For me, desirism is not just a useful and prescriptive moral theory. Desirism could also be part of an engineering problem that will determine the fate of humanity.
Let me explain.
Many people, myself included, think that the Singularity is fewer than 300 years away. By Singularity, I mean the point at which we develop an artificial intelligence that surpasses humans at every intelligent task we can perform. At that point, the AI will be smart enough to improve itself, which will make it capable of improving itself even faster, which will make it capable of improving itself even faster, and so on.
Once we reach the Singularity, this AI will rapidly develop into a superintelligence that is capable of killing all humans or of ushering in a kind of utopia in which most of our problems of energy, health, and governance are solved.1
So besides the usual hardware and software challenges we must overcome before we reach the Singularity, we must be very careful to create a good superintelligence rather than an evil one.
So how do we program this superintelligence to be morally good? That’s a tough one.2 Eliezer Yudkowsky asks us to think of the Greeks. If the Greeks had designed an AI to be moral, this AI would have been instilled with the idea that slavery was useful in many contexts, that women were little more than property, that the most glorious death was one in battle, and so on.
And if we program a superintelligence with our most progressive values of today, would we be much better off in the long run? If we program it to value liberal democracy and equality and veganism, would we regret this in the long run? Almost certainly. I doubt we have come to the end of moral progress.
So really, we don’t want to program the superintelligence with our particular moral values. Rather, we want to program it with a kind of moral trajectory, or some kind of system for progressively figuring out what is moral.
Well, that’s what desirism is.
Desirism is a system for figuring out what is moral that respects no particular assumptions about moral values and political systems. It does not assume the goodness of democracy or veganism or equality or capitalism or, really, anything. It is not tied to the intuitions of philosophers of any age. Rather, it respects only a consideration of all the reasons for action that exist, whatever those happen to be according to scientific inquiry.
And in fact, according to desirism, only a superintelligence is even capable of answering a great many moral questions. We lowly humans can use heuristics and make some pretty good guesses about the moral facts relevant to some situations. But when it comes to enormously complex issues like how to treat animals or how to organize political systems, desirism says there is an enormous amount of scientific research that must be done first – perhaps more research than any merely human research team could ever conduct.
So there you go. Desirism is not just a plausible descriptive and prescriptive theory of morality for human practice. Desirism may well be the key to creating utopia and avoiding extinction.3
And if desirism is false, then we’d better figure out something better, with which to program the AI. Quickly.
- You can’t avoid the Singularity. You could outlaw it, which could delay it for centuries. You could set human civilization back to the stone age, which would delay it for millennia. But you can’t avoid it altogether without extinguishing the most intelligent species altogether. Moreover, there is no “learning curve,” and no second chances. Once AI reaches the Singularity, then if you fucked up, all of humanity will be dead within weeks. You can’t set a hundred scientists trying lots of different methods in a disorganized fashion, because chances are not good that the first one to hit the Singularity will have created a ‘Friendly’ AI. You have to hit the Singularity, you have to hit it once, and you have to be damn sure you hit it right. [↩]
- Another tough question is: “How do we know the superintelligence wouldn’t decide to reprogram its own motives and become evil? Yudkowsky uses an argument by analogy involving Gandhi. If you offered Gandhi a pill that would make him desire violence, would he take it? No, he would not, for he does not desire violence, and he does not desire to desire violence. But this is only an intuitive argument. We would want to prove this mathematically when the stakes are so high, and we have not done so yet. [↩]
- Technically, it would not be desirism proper used to program the AI. Rather, it would be a modified desirism. Why? Desirism is only concerned with desires that happen to be malleable in moral agents. In contrast, the moral programming used in Friendly AI will be written “from scratch,” meaning that all possible desires are “malleable.” That’s a bit different. [↩]