peterhartree, Tyler Cowen on effective altruism (December 2022), Effective Altruism Forum.
From the transcript of a talk given in December of 2022 and hosted by Luca Stroppa and Theron Pummer at the University of St Andrews. There's this brief comment:
The second part of the EA movement is concerned with a whole series of investments in fighting existential risk. So very often this takes the form of worry that artificial intelligence will turn into some form of AGI that may pulp us into paperclips or destroy us or have its way with the world and then human civilization as we know it. It's an interesting question, like historically, sociologically, why that's become intertwined with the anti-malaria bed nets, but it has. And I think actually for some decent reasons. But I would just say I'm in general more skeptical about the extreme emphasis on existential risk than a lot of effective altruism would be.
And there's this somewhat longer comment:
Another way that I think how I view existential risk is different from how many of the E.A. individuals view it. Actually I look at many, in particular, artificial intelligence issues from a national perspective, not a global perspective. So I think if you could wave a magic wand and stop all the progress of artificial intelligence, I'm not going to debate that debate now, but I'll just say I can see the case for just stopping it if you can wave the magic wand. Well, it's too dangerous, it's too risky and so on. We all know those arguments. But my view is you never have the magic wand at the global level. The UN is powerless. We're not about to set up some transnational thing that has any real influence over those decisions. Different nations are doing it. China is doing it. And the real question to ask is, do you want the U.S. to come in first and China to come in second, or rather China to come in first and the U.S. to come in second? It's just a very, very different question than should we wave the magic wand to stop the whole thing. Now, I don't want to debate that question either, but I'll just tell you my perspective, which is in part partial, as I discussed earlier, but in part I think just broadly utilitarian is I want the U.S. to win. So I think we should go full steam ahead on AI. And I think that for reasons I don't hear sufficiently frequently from the E.A. community, namely that the real question is U.S. gets it or China gets it. Then you have to pick a side and you can pick the side for utilitarian reasons, or you might in part pick the side for reasons that are, you know, partialistic. But nonetheless, freezing it all is not on the table.
I even worry that the people who obsess over AI alignment, I suspect their net impact to date has been to make AI riskier by getting everyone thinking about these scenarios through which AI could do these terrible things. So the notion that you just talk about or even try to do AI alignment, I don't think it's going to work. I think the problem is if AI is powerful enough to destroy everything, it's the least safe system you have to worry about. So you might succeed with AI alignment on say 97 percent of cases, which would be a phenomenal success ratio, right? No one like seriously expects to succeed on 97 percent of cases. But if you failed on 3 percent of cases, that 3 percent of very evil, effective enough AIs can still reproduce and take things over and turn you into paper clips. So I'm just not that optimistic about the alignment agenda. And I think you need to view it from a national perspective, not like where's the magic wand that's going to set all of this right. And in that sense, I think I'm different from EAs as I meet them.
Let me add a note about AI alignment. While I don't see much risk of future AI going rogue and wiping out humankind, that's not the only thing on the alignment agenda. Or rather, the alignment agenda has activities which are important and justifiable independently of any concern about existential risk. If you are concerned about existential risk, one thing you will do is try to figure out how these various neural network models work.
That's called interpretability and it is important independently of any concern for existential risk. In general principle we want to know how these devices function. Much of my work on ChatGPT is of that kind, though my methods are quite different from those currently working on the problem.
No comments:
Post a Comment