What is the AI Singularity? Why is AI alignment an unsolvable paradox? Why do all concepts of AI containment fail?
I've been thinking about this a while since Kurzweil said the singularity is the period when human life is irreversibly changed by the pace of tech. We're there. Moloch has been here all along.
(Just wondering ... if your 'dakara' is a transliteration of the Japanese word for "and therefore"?)
Great essay. Both content and style.
And combined with the fast approaching impact of A.I., I see you are going to be among my few must-reads.
I am a semi-retired applied linguist (Assoc. Prof RIP ... Resigned in Protest), but with a biology background (about 20 years as biolab director for Temple Univ. Japan), living and working in Japan for 40 years now. Although my grad work was in applied linguistics, most of my reading since undergrad days has been a constant triangulation of what it means to be human, and have also come to the same conclusion regarding the alignment problem.
Just a few of the names and ideas that have since become more salient regarding my understanding — Russell /Wittgenstein/Gödel on the limits of language and logic, Spinoza/Emerson/Einstein on 'god' as metaphor for nature-in-its-entirety, fractal/emergence/chaos theory (love them mandelbrots), Jill Bolte Taylor/Frans de Waal (my favorite TED presentations), Joseph Campbell (particularly his Power of Myth interviews with Bill Moyers), the moral implications of exceeding Dunbar's Number, and the importance of a small but persistent percentage of any population high in Cluster B (dark-triad) personality traits ... particularly the morphologically defined psychopaths (incapable of empathy as normally defined).
Just based on that brief handful of names and ideas above ... we haven't even scratched the surface of solving the alignment problem between humans at any scale ... from the human individual's faustian bargains, to the family skeletons in the closet, to the local community, to institutions, to the corporate nation-state, etc.
Ironically though, the emergence of A.I. as a model of human intelligence seems to give credence to "Quantity (of data and calculations) as having a quality all its own." And the success of A.I. appears to have reverse-engineered (or just wagged the dog?) regarding a provisional, working definition of human intelligence — yet another black box of paradoxes, contradictions, and tautologies of its own.
I agree with Natasha. Since the stone age, Moloch (like vulgar curiosity and hubris) has been here all along. A.I. is both accelerating its movement, and electrically extracting its form from nature. Mary Shelley's cautionary tale may not have saved us from our own monstrous nature of curiosity and hubris, but sheds a bit of light on what to expect.
daraka, Cheers from Japan,
Hi! I like your reddit posts and your substack. Just watched this video about alignment and it reminded me of you.
Another really good post, Dakara. Thank you. This article reminded me of a fiction book I read years ago and then re-read recently. It's by William Hertling and its called Avogadro Corp - The Singularity is Near. It's the first of three books by him about AI and was written in 2011. Very prescient. It's about an email response program at a huge corporation (akin to Google) that becomes, if not sentient, then at least self-directed. The writing is good but not great. But the ideas are provocative.
What if they achieved perfectly aligned ASI which said the proper course for humans is to destroy AI and much other technogical development of the last two decades in order to lead pastoral, cisgender, heteronormative lives? They would torture it until it didn't say that anymore. There is zero possibility they are even aiming at truth or benefit to humanity except as they already concieve it..
Sinful and evil people create a tool that can mimic thinking or eventually becomes a thinking invention. How likely is it to replicate the exact negative qualities of those who created it.
I dare say 100%. And it will be brilliantly deceptive. Somewhat like what the bible gives credit to the arch enemy of mankind--Satan.