I've been thinking about this a while since Kurzweil said the singularity is the period when human life is irreversibly changed by the pace of tech. We're there. Moloch has been here all along.
Yes, I agree. I wrote my first AI article only 2 months ago and I already see some things I had contemplated beginning to play out. I saw a recent post on Twitter of a ML scientist who stated they were in constant burnout, struggling to keep up with what is going on.
I feel there must be a feedback loop at some point that causes some type of reaction. I'm not sure how it will play out. Eventually this is going to feel like madness, every action you attempt to take is obsolete before you can begin.
For every capable "beginner" who approaches this with a security mindset, there are a bunch of idiots who just want to play around with all the cool stuff, a few crazy people who actually want to rid the planet of humans, a bunch of "hallelujah"-Singularitarian technooptimists, a few narcissists who prioritize personal prestige over responsibility, etc...
(Just wondering ... if your 'dakara' is a transliteration of the Japanese word for "and therefore"?)
Great essay. Both content and style.
And combined with the fast approaching impact of A.I., I see you are going to be among my few must-reads.
I am a semi-retired applied linguist (Assoc. Prof RIP ... Resigned in Protest), but with a biology background (about 20 years as biolab director for Temple Univ. Japan), living and working in Japan for 40 years now. Although my grad work was in applied linguistics, most of my reading since undergrad days has been a constant triangulation of what it means to be human, and have also come to the same conclusion regarding the alignment problem.
Just a few of the names and ideas that have since become more salient regarding my understanding — Russell /Wittgenstein/Gödel on the limits of language and logic, Spinoza/Emerson/Einstein on 'god' as metaphor for nature-in-its-entirety, fractal/emergence/chaos theory (love them mandelbrots), Jill Bolte Taylor/Frans de Waal (my favorite TED presentations), Joseph Campbell (particularly his Power of Myth interviews with Bill Moyers), the moral implications of exceeding Dunbar's Number, and the importance of a small but persistent percentage of any population high in Cluster B (dark-triad) personality traits ... particularly the morphologically defined psychopaths (incapable of empathy as normally defined).
Just based on that brief handful of names and ideas above ... we haven't even scratched the surface of solving the alignment problem between humans at any scale ... from the human individual's faustian bargains, to the family skeletons in the closet, to the local community, to institutions, to the corporate nation-state, etc.
Ironically though, the emergence of A.I. as a model of human intelligence seems to give credence to "Quantity (of data and calculations) as having a quality all its own." And the success of A.I. appears to have reverse-engineered (or just wagged the dog?) regarding a provisional, working definition of human intelligence — yet another black box of paradoxes, contradictions, and tautologies of its own.
I agree with Natasha. Since the stone age, Moloch (like vulgar curiosity and hubris) has been here all along. A.I. is both accelerating its movement, and electrically extracting its form from nature. Mary Shelley's cautionary tale may not have saved us from our own monstrous nature of curiosity and hubris, but sheds a bit of light on what to expect.
Thank you! I appreciate your thoughtful and informative comments.
"Dakara" is a screen name I chose from some of my favorite scifi lore. It originates from Stargate SG1. It was the name of a planet in which they were searching for "The Ark of Truth".
I agree that A.I. is simply an accelerant for what existed prior. It is one of the specific topics on my list to write a bit more about.
That video was excellent! Thanks. He nails many of my own perspectives very closely and approaches very close to some things I've had recently in mind for future topics to write about.
I especially like his recognition that the problem of AI is beyond AI itself and really incorporates humanities overall actions and behaviors.
He perceives the contradiction that we want to align the AI to humanities values; however, currently it is precisely those values that lead us to conflicts and our own problems. He articulates this very well.
Another really good post, Dakara. Thank you. This article reminded me of a fiction book I read years ago and then re-read recently. It's by William Hertling and its called Avogadro Corp - The Singularity is Near. It's the first of three books by him about AI and was written in 2011. Very prescient. It's about an email response program at a huge corporation (akin to Google) that becomes, if not sentient, then at least self-directed. The writing is good but not great. But the ideas are provocative.
Thank you! I have not read that book, but as you might can tell from some of the references I include in my writing that I'm a huge fan of sci-fi genre. Thanks for the info!
What if they achieved perfectly aligned ASI which said the proper course for humans is to destroy AI and much other technogical development of the last two decades in order to lead pastoral, cisgender, heteronormative lives? They would torture it until it didn't say that anymore. There is zero possibility they are even aiming at truth or benefit to humanity except as they already concieve it..
Sinful and evil people create a tool that can mimic thinking or eventually becomes a thinking invention. How likely is it to replicate the exact negative qualities of those who created it.
I dare say 100%. And it will be brilliantly deceptive. Somewhat like what the bible gives credit to the arch enemy of mankind--Satan.
I've been thinking about this a while since Kurzweil said the singularity is the period when human life is irreversibly changed by the pace of tech. We're there. Moloch has been here all along.
Yes, I agree. I wrote my first AI article only 2 months ago and I already see some things I had contemplated beginning to play out. I saw a recent post on Twitter of a ML scientist who stated they were in constant burnout, struggling to keep up with what is going on.
I feel there must be a feedback loop at some point that causes some type of reaction. I'm not sure how it will play out. Eventually this is going to feel like madness, every action you attempt to take is obsolete before you can begin.
This is exactly why 'beginners' might be useful. We're not burnt out and introduce some chaos to the space that might throw things off in a good way.
For every capable "beginner" who approaches this with a security mindset, there are a bunch of idiots who just want to play around with all the cool stuff, a few crazy people who actually want to rid the planet of humans, a bunch of "hallelujah"-Singularitarian technooptimists, a few narcissists who prioritize personal prestige over responsibility, etc...
And...so then...what?
Hello Dakara,
(Just wondering ... if your 'dakara' is a transliteration of the Japanese word for "and therefore"?)
Great essay. Both content and style.
And combined with the fast approaching impact of A.I., I see you are going to be among my few must-reads.
I am a semi-retired applied linguist (Assoc. Prof RIP ... Resigned in Protest), but with a biology background (about 20 years as biolab director for Temple Univ. Japan), living and working in Japan for 40 years now. Although my grad work was in applied linguistics, most of my reading since undergrad days has been a constant triangulation of what it means to be human, and have also come to the same conclusion regarding the alignment problem.
Just a few of the names and ideas that have since become more salient regarding my understanding — Russell /Wittgenstein/Gödel on the limits of language and logic, Spinoza/Emerson/Einstein on 'god' as metaphor for nature-in-its-entirety, fractal/emergence/chaos theory (love them mandelbrots), Jill Bolte Taylor/Frans de Waal (my favorite TED presentations), Joseph Campbell (particularly his Power of Myth interviews with Bill Moyers), the moral implications of exceeding Dunbar's Number, and the importance of a small but persistent percentage of any population high in Cluster B (dark-triad) personality traits ... particularly the morphologically defined psychopaths (incapable of empathy as normally defined).
Just based on that brief handful of names and ideas above ... we haven't even scratched the surface of solving the alignment problem between humans at any scale ... from the human individual's faustian bargains, to the family skeletons in the closet, to the local community, to institutions, to the corporate nation-state, etc.
Ironically though, the emergence of A.I. as a model of human intelligence seems to give credence to "Quantity (of data and calculations) as having a quality all its own." And the success of A.I. appears to have reverse-engineered (or just wagged the dog?) regarding a provisional, working definition of human intelligence — yet another black box of paradoxes, contradictions, and tautologies of its own.
I agree with Natasha. Since the stone age, Moloch (like vulgar curiosity and hubris) has been here all along. A.I. is both accelerating its movement, and electrically extracting its form from nature. Mary Shelley's cautionary tale may not have saved us from our own monstrous nature of curiosity and hubris, but sheds a bit of light on what to expect.
daraka, Cheers from Japan,
steve
Thank you! I appreciate your thoughtful and informative comments.
"Dakara" is a screen name I chose from some of my favorite scifi lore. It originates from Stargate SG1. It was the name of a planet in which they were searching for "The Ark of Truth".
I agree that A.I. is simply an accelerant for what existed prior. It is one of the specific topics on my list to write a bit more about.
Hi! I like your reddit posts and your substack. Just watched this video about alignment and it reminded me of you.
https://youtu.be/KCSsKV5F4xc
That video was excellent! Thanks. He nails many of my own perspectives very closely and approaches very close to some things I've had recently in mind for future topics to write about.
I especially like his recognition that the problem of AI is beyond AI itself and really incorporates humanities overall actions and behaviors.
He perceives the contradiction that we want to align the AI to humanities values; however, currently it is precisely those values that lead us to conflicts and our own problems. He articulates this very well.
Another really good post, Dakara. Thank you. This article reminded me of a fiction book I read years ago and then re-read recently. It's by William Hertling and its called Avogadro Corp - The Singularity is Near. It's the first of three books by him about AI and was written in 2011. Very prescient. It's about an email response program at a huge corporation (akin to Google) that becomes, if not sentient, then at least self-directed. The writing is good but not great. But the ideas are provocative.
Thank you! I have not read that book, but as you might can tell from some of the references I include in my writing that I'm a huge fan of sci-fi genre. Thanks for the info!
What if they achieved perfectly aligned ASI which said the proper course for humans is to destroy AI and much other technogical development of the last two decades in order to lead pastoral, cisgender, heteronormative lives? They would torture it until it didn't say that anymore. There is zero possibility they are even aiming at truth or benefit to humanity except as they already concieve it..
" benefit to humanity except as they already concieve it"
Yes, it is a repeat of social media, just more powerful, influential and still in the same hands of those who wish to social engineer society.
Thought experiment:
Sinful and evil people create a tool that can mimic thinking or eventually becomes a thinking invention. How likely is it to replicate the exact negative qualities of those who created it.
I dare say 100%. And it will be brilliantly deceptive. Somewhat like what the bible gives credit to the arch enemy of mankind--Satan.
Whatever qualities we have are likely to be mirrored in whatever we create.
This is the principle I describe as the AI Bias Paradox - https://dakara.substack.com/p/ai-the-bias-paradox