The Hubris Trap - quest to build super intelligence
Notes From the Desk: No. 22 - 2024.02.04
Notes From the Desk are periodic posts that summarize recent topics of interest or other brief notable commentary that might otherwise be a tweet or note.
The Hubris Trap
No greater prize could ever be held in the history of humankind beyond the creation of what we conceive as artificial super intelligence (ASI). The significance of the last ever human-created invention, the Utopia machine, is impossible to overstate.
If the reward is grand enough, humans are well known to sacrifice their own lives as well as risk others in attempts to achieve or obtain it. Do not think this endeavor is any different. The possibility to create the one ring to rule them all is an irresistible temptation. Is this analogy is an overstatement of the concern?
Within whose hands is the future of these outcomes currently held? Is it within the institutions that have been noble pillars of enlightenment, liberty and free thought? Or is it held within the very institutions that despise free speech, liberty, free thought, individuality and the ability for self-determination of the individual?
Indeed it is our Orwellian overlords who preside over the creation of an all-powerful entity they claim for our benefit. If they retain control over such a creation, what has precedent determined how it will be used? If they lose control of such a creation, then what is the outcome?
It would seem that successful alignment or alignment failure does not necessarily lead to significantly different ends. Humanity falls either under its own agency in control of power it is not prepared to manage or humanity will be managed by power it can not control.
Impossibility to perceive the paradox
Such ambitions to achieve the impossible and prove everyone wrong create strong emotional convictions to pursue goals even in the face of the near certainty of failure. In most contexts, this is one of humanity’s most admirable traits as it has led us to unthinkable accomplishments.
However, in order to pursue such endeavors those inclined must disregard all concerns of failure as nobody can succeed against all odds if they believe it is not possible to do so. The consequence is that it presents the real possibility of blind spots that are impossible to observe from those consumed by this quest.
Two striking paradoxes of logic underline the basis of theory. The first is that we can comprehend how a vastly superior intelligence will reason about the world and the second is that we can align it to unaligned humanity.
The only reasonable conclusion thus far is that everyone is wrong and will continue to be so. If it is possible to create an ASI, we are by definition Dunning-Krugers in relation to our understanding of that outcome. Can mere mortals hold the Promethean fire and not be consumed by the same? The future will be a series of unexpected plot twists.
Continue a deeper exploration into alignment failure and the meaning of The Singularity in the essay AI Singularity: The Hubris Trap
No compass through the dark exists without hope of reaching the other side and the belief that it matters …