Great work here. Seems like the more we tinker with AI, the more we learn the divide between our understanding of intelligence and intelligence itself. At some point, we'll have to come to the conclusion that statistical models can't explain very much. It's odd how obsessed our culture is with stats and probabilities, that there's even a notion of "laws of probability" when probability is a clever workaround rather than an explanation of any phenomenon.
Yes, the statistical models will have their uses, it is just they can't accomplish many of the types of tasks for which they are being hyped.
In the end, maybe many will have a greater appreciation and understanding for just how special true intelligence really is. In the future, I hope to write a bit more specifically on the distinction to bring more clarity to the topic.
You posted this on r/agi? Wow, that is the last place in the world to find people who are knowledgeable and rational about the subject ... it's fanboiism all the way.
Thank you! Ha! no I didn't post on r/agi. Someone else posted it there, but the comments were typical Reddit :-) I would estimate less than 1% read the article. Nonetheless, it was useful to read through to summarize common criticisms which I responded to in my latest post today.
Right, I lost track of my links. I was starting to read your latest post and got waylaid when I followed your link to Reddit, and baselessly assumed you were the one who posted it there. I loved the comment where someone with the IQ of a rutabaga called you a "dumbass" and likened you to someone in 1775 (!) "shitting on trains". These geniuses can't tell the difference between "LLMs aren't the route to AGI and here's why" and "LLM bad! ooga booga!"
What I want to know is if this is known or understood, then why have people who've won nobel prizes for their work in AI seem to think agi is coming soonish. Is there something they're missing or something you're missing 🤔.
I think many have got caught up in the exuberance of billions of investment dollars flowing into the AI industry. However, I believe we are going to see researchers starting to distance themselves from LLMs just as Lecun is now doing.
I believe many are still hopeful alternative methods will yield AGI soonish, but that is completely undiscovered territory. I think the hopeful look comes from mainly thinking we now have everyone working on this problem and someone is going to figure it out, versus we know how to do this and it is just some hard work to implement it.
Appreciate the reply, but I've got to say your comment just leaves me with more questions than it answers! The idea that demis hassabis or geoffery hinton are simply hoping that with the eyes of the world (and billions of funding) that completely paradigm shifting discoveries will be made if this isn't built on prior revelations(as seemed to have been happening in the llm field ) seems fanciful to me. They'd be stoking their professional reputation on hope. This isn't to denounce your articles (read both) I honestly think you bring up some very interesting points and may be right. But if you are then the current paradigm doesn't make sense. More and more of code is ai generated and we have some of the greatest ai researchers telling us this is happening and fast. None as far as I'm aware bring up alternatives routes to agi that exclude llms though yes lecun does disagree with much of the field, one lone dissenter (of renown) seems more of a statistical outlier if the rest say the opposite. Consequently I don't know what to make of your assertions either you're right and the world is wrong or you're wrong and the world is right. Either way we will just have to see. Nevertheless keep writing dude those were some excellently written articles , thoroughly enjoyed them.
HN isn't as bad as Reddit (low bar), but both sites featured a lot of comments based on a fallacy of affirmation of the consequent ... humans also hallucinate, humans also use statistics, etc. It reminds me of Carl Sagan's observation that "They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."
LLMs are remarkable and amazing, but the entire appearance of intelligence is obtained by being trained on trillions of fragments of text generated by intelligent humans. If they were trained on the output of a billion monkeys banging on typewriters, the appearance would disappear. They are great for querying, but over and over again when I use them as programming aids I see cracks in the facade that show a total lack of understanding. I'm constantly suggesting improvements to their responses and they go oh yeah, good idea, here's how that works ... which I already know, since I just told them about it--no human would respond like that. Ask them why they didn't provide the good idea in the first place and they offer vague generalities. The good idea is just another prompt for them to pattern match against, but lacking cognitive states they produce mediocre first-try algorithms. And when you aren't prompting them they are completely inert ... AGI would show initiative (but hopefully not to turn the world into paperclips).
You always do such a great job of distilling the primary details in common terms. That clarity is missing from many of the AI analysis which at times seems to be hall of mirrors that the author uses to try to bring fourth answers that are not there. I really enjoy reading your work on AGI.
It's important to put this in perspective. LLMs are a great time saver: hey ChatGPT, here is a a study I need you to create a table that compares X to Y. It's brilliant at that, you can go back and check it, boom, you're done. But you need to be conversant in the material first.
I have a 20 year old who wants to go into finance, and I tell him he still needs to be good at prompting because those with lesser skill will be using it to make decisions that will affect him, so he needs to develop even superior skills to spot errors while also being able to provide work with the tools that the industry is using. Being able to pick apart nonsense goes a long way to build a reputation for excellence. LLMs and this misplaced belief in their inerrancy will prohibit people from developing judgement if management is always asking "Well, what did Claude say?"
Yes, to your point, LLMs are probably one of the most awkward technologies ever invented, being both extremely useful and treacherous, with no distinction between the two being obvious to the user.
It is one of the topics I hope to cover more in depth soon, that being how to properly use LLMs.
Thank you for this relevant analysis, even if I am not a specialist or training, I am able to understand according to the subject covered.
As I mentioned earlier, for me it is the word intelligence that leads to confusion between data processing capacity and human intelligence-specific reasoning. I suppose it takes observation (of human brain function) and valid reasoning to get out of this confusion.
Yes, it is definitely a topic of some obscurity, and not always well agreed-upon-definitions. I hope to continue to write more on the subject and hopefully bring more clarity.
From comment: "Some folks are just determined to believe intelligence is a uniquely human attribute."
Indeed, if it is the human who defines what is intelligence (which is the case), he uses this definition to judge other species, according to his own criteria. What is already not totally objective, but allows to classify, categorize... according to its needs whether they are justified or not (subject too complex to dwell on here).
Should we not be more cautious in our assertions about the living? Of course referring solely or fully to the publications of researchers...is not a guarantee of veracity, objectivity... We are well aware that most of them pursue objectives other than the discovery and transmission of the knowledge to ordinary people (not being part of the elitist club).
And to conclude, let us not forget that we can only approach the world of living (and not living), other than our species, from our consciousness, without this we do not know precisely how it works, what is happening, how it acts and reacts actually and fully...in a word their intrinsic "capabilities".
Our definition of intelligence therefore allows us to judge whether a program, a machine...by the way produced by humans, is intelligent or not. Valid reasoning is sufficient to understand it.
Did you read the paper? Far from revealing that these models are purely statistical, it shows that AI models are coming up with a variety of sophisticated strategies to solve complex problems and requests. Here’s a quote: “Our results uncover a variety of sophisticated strategies employed by models. For instance, Claude 3.5 Haiku routinely uses multiple intermediate reasoning steps “in its head” 2 to decide its outputs. It displays signs of forward planning, considering multiple possibilities for what it will say well in advance of saying it. It performs backward planning, working backwards from goal states to formulate earlier parts of its response. We see signs of primitive “metacognitive” circuits that allow the model to know the extent of its own knowledge. More broadly, the model’s internal computations are highly abstract and generalize across disparate contexts.”
Yes, I read it. However, none of that says it is not a statistical model. Statistical derived pattern matching can self organize into some form algorithms. But those "algorithms" are still within the bounds of the architecture in which they execute.
It is still a function of probabilities that are operating against the training data. I don't deny that this can become extremely sophisticated. I mention this in the post. However, there are specific limits to that capability that emerge in certain forms such as hallucinations.
There is no mechanism present for the creation of semantic information. Yes, it can do some sophisticated "analysis" of patterns, that is no doubt useful, but it diverges from the capabilities we would expect from true understanding of fundamental concepts. Edge cases don't appear when there is fundamental understanding.
The models aren't coming up with a variety of sophisticated strategies. The developers of the models are coming up with these strategies for the model. The computer program is programmed to complete these "reasoning" steps.
"Some folks are just determined to believe intelligence is a uniquely human attribute."
Indeed, if it is the human who defines what is intelligence (which is the case), he uses this definition to judge other species, according to his own criteria. What is already not totally objective, but allows to classify, categorize... according to its needs whether they are justified or not (subject too complex to dwell on here).
Should we not be more cautious in our assertions about the living? Of course referring solely or fully to the publications of researchers...is not a guarantee of veracity, objectivity... We are well aware that most of them pursue objectives other than the discovery and transmission of the knowledge to ordinary people (not being part of the elitist club).
And to conclude, let us not forget that we can only approach the world of living (and not living), other than our species, from our consciousness, without this we do not know precisely how it works, what is happening, how it acts and reacts actually and fully...in a word their intrinsic "capabilities".
Our definition of intelligence therefore allows us to judge whether a program, a machine...by the way produced by humans, is intelligent or not. Valid reasoning is sufficient to understand it.
Great work here. Seems like the more we tinker with AI, the more we learn the divide between our understanding of intelligence and intelligence itself. At some point, we'll have to come to the conclusion that statistical models can't explain very much. It's odd how obsessed our culture is with stats and probabilities, that there's even a notion of "laws of probability" when probability is a clever workaround rather than an explanation of any phenomenon.
Yes, the statistical models will have their uses, it is just they can't accomplish many of the types of tasks for which they are being hyped.
In the end, maybe many will have a greater appreciation and understanding for just how special true intelligence really is. In the future, I hope to write a bit more specifically on the distinction to bring more clarity to the topic.
Very good.
You posted this on r/agi? Wow, that is the last place in the world to find people who are knowledgeable and rational about the subject ... it's fanboiism all the way.
Thank you! Ha! no I didn't post on r/agi. Someone else posted it there, but the comments were typical Reddit :-) I would estimate less than 1% read the article. Nonetheless, it was useful to read through to summarize common criticisms which I responded to in my latest post today.
Right, I lost track of my links. I was starting to read your latest post and got waylaid when I followed your link to Reddit, and baselessly assumed you were the one who posted it there. I loved the comment where someone with the IQ of a rutabaga called you a "dumbass" and likened you to someone in 1775 (!) "shitting on trains". These geniuses can't tell the difference between "LLMs aren't the route to AGI and here's why" and "LLM bad! ooga booga!"
lol, yes. I don't post much on the Reddit AI forums anymore. I was perma banned from r/Singularity a long time ago, lol.
I did post it on Hacker News, a bit more balanced commentary there, but still the top voted comment was dismissing it all.
What I want to know is if this is known or understood, then why have people who've won nobel prizes for their work in AI seem to think agi is coming soonish. Is there something they're missing or something you're missing 🤔.
So, let me be precise about the argument. What I have argued here is a criticism precisely against LLMs being or becoming AGI anytime soon.
Yann Lecun has stated LLMs are not the future. They were an off-ramp towards AGI. They won't get us there.
https://www.reddit.com/r/singularity/comments/1d5b57b/lecun_tells_phd_students_there_is_no_point/
And here is Francois Chollet saying the same thing - https://x.com/tsarnick/status/1800644136942367131
I think many have got caught up in the exuberance of billions of investment dollars flowing into the AI industry. However, I believe we are going to see researchers starting to distance themselves from LLMs just as Lecun is now doing.
I believe many are still hopeful alternative methods will yield AGI soonish, but that is completely undiscovered territory. I think the hopeful look comes from mainly thinking we now have everyone working on this problem and someone is going to figure it out, versus we know how to do this and it is just some hard work to implement it.
Appreciate the reply, but I've got to say your comment just leaves me with more questions than it answers! The idea that demis hassabis or geoffery hinton are simply hoping that with the eyes of the world (and billions of funding) that completely paradigm shifting discoveries will be made if this isn't built on prior revelations(as seemed to have been happening in the llm field ) seems fanciful to me. They'd be stoking their professional reputation on hope. This isn't to denounce your articles (read both) I honestly think you bring up some very interesting points and may be right. But if you are then the current paradigm doesn't make sense. More and more of code is ai generated and we have some of the greatest ai researchers telling us this is happening and fast. None as far as I'm aware bring up alternatives routes to agi that exclude llms though yes lecun does disagree with much of the field, one lone dissenter (of renown) seems more of a statistical outlier if the rest say the opposite. Consequently I don't know what to make of your assertions either you're right and the world is wrong or you're wrong and the world is right. Either way we will just have to see. Nevertheless keep writing dude those were some excellently written articles , thoroughly enjoyed them.
HN isn't as bad as Reddit (low bar), but both sites featured a lot of comments based on a fallacy of affirmation of the consequent ... humans also hallucinate, humans also use statistics, etc. It reminds me of Carl Sagan's observation that "They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."
LLMs are remarkable and amazing, but the entire appearance of intelligence is obtained by being trained on trillions of fragments of text generated by intelligent humans. If they were trained on the output of a billion monkeys banging on typewriters, the appearance would disappear. They are great for querying, but over and over again when I use them as programming aids I see cracks in the facade that show a total lack of understanding. I'm constantly suggesting improvements to their responses and they go oh yeah, good idea, here's how that works ... which I already know, since I just told them about it--no human would respond like that. Ask them why they didn't provide the good idea in the first place and they offer vague generalities. The good idea is just another prompt for them to pattern match against, but lacking cognitive states they produce mediocre first-try algorithms. And when you aren't prompting them they are completely inert ... AGI would show initiative (but hopefully not to turn the world into paperclips).
You always do such a great job of distilling the primary details in common terms. That clarity is missing from many of the AI analysis which at times seems to be hall of mirrors that the author uses to try to bring fourth answers that are not there. I really enjoy reading your work on AGI.
It's important to put this in perspective. LLMs are a great time saver: hey ChatGPT, here is a a study I need you to create a table that compares X to Y. It's brilliant at that, you can go back and check it, boom, you're done. But you need to be conversant in the material first.
I have a 20 year old who wants to go into finance, and I tell him he still needs to be good at prompting because those with lesser skill will be using it to make decisions that will affect him, so he needs to develop even superior skills to spot errors while also being able to provide work with the tools that the industry is using. Being able to pick apart nonsense goes a long way to build a reputation for excellence. LLMs and this misplaced belief in their inerrancy will prohibit people from developing judgement if management is always asking "Well, what did Claude say?"
Thank you very much! Much appreciated!
Yes, to your point, LLMs are probably one of the most awkward technologies ever invented, being both extremely useful and treacherous, with no distinction between the two being obvious to the user.
It is one of the topics I hope to cover more in depth soon, that being how to properly use LLMs.
Thank you for this relevant analysis, even if I am not a specialist or training, I am able to understand according to the subject covered.
As I mentioned earlier, for me it is the word intelligence that leads to confusion between data processing capacity and human intelligence-specific reasoning. I suppose it takes observation (of human brain function) and valid reasoning to get out of this confusion.
Yes, it is definitely a topic of some obscurity, and not always well agreed-upon-definitions. I hope to continue to write more on the subject and hopefully bring more clarity.
From comment: "Some folks are just determined to believe intelligence is a uniquely human attribute."
Indeed, if it is the human who defines what is intelligence (which is the case), he uses this definition to judge other species, according to his own criteria. What is already not totally objective, but allows to classify, categorize... according to its needs whether they are justified or not (subject too complex to dwell on here).
Should we not be more cautious in our assertions about the living? Of course referring solely or fully to the publications of researchers...is not a guarantee of veracity, objectivity... We are well aware that most of them pursue objectives other than the discovery and transmission of the knowledge to ordinary people (not being part of the elitist club).
And to conclude, let us not forget that we can only approach the world of living (and not living), other than our species, from our consciousness, without this we do not know precisely how it works, what is happening, how it acts and reacts actually and fully...in a word their intrinsic "capabilities".
Our definition of intelligence therefore allows us to judge whether a program, a machine...by the way produced by humans, is intelligent or not. Valid reasoning is sufficient to understand it.
Wildly inaccurate analysis here. Some folks are just determined to believe intelligence is a uniquely human attribute.
Maybe you could identify the error in analysis?
Did you read the paper? Far from revealing that these models are purely statistical, it shows that AI models are coming up with a variety of sophisticated strategies to solve complex problems and requests. Here’s a quote: “Our results uncover a variety of sophisticated strategies employed by models. For instance, Claude 3.5 Haiku routinely uses multiple intermediate reasoning steps “in its head” 2 to decide its outputs. It displays signs of forward planning, considering multiple possibilities for what it will say well in advance of saying it. It performs backward planning, working backwards from goal states to formulate earlier parts of its response. We see signs of primitive “metacognitive” circuits that allow the model to know the extent of its own knowledge. More broadly, the model’s internal computations are highly abstract and generalize across disparate contexts.”
Yes, I read it. However, none of that says it is not a statistical model. Statistical derived pattern matching can self organize into some form algorithms. But those "algorithms" are still within the bounds of the architecture in which they execute.
It is still a function of probabilities that are operating against the training data. I don't deny that this can become extremely sophisticated. I mention this in the post. However, there are specific limits to that capability that emerge in certain forms such as hallucinations.
There is no mechanism present for the creation of semantic information. Yes, it can do some sophisticated "analysis" of patterns, that is no doubt useful, but it diverges from the capabilities we would expect from true understanding of fundamental concepts. Edge cases don't appear when there is fundamental understanding.
The models aren't coming up with a variety of sophisticated strategies. The developers of the models are coming up with these strategies for the model. The computer program is programmed to complete these "reasoning" steps.
"Some folks are just determined to believe intelligence is a uniquely human attribute."
Indeed, if it is the human who defines what is intelligence (which is the case), he uses this definition to judge other species, according to his own criteria. What is already not totally objective, but allows to classify, categorize... according to its needs whether they are justified or not (subject too complex to dwell on here).
Should we not be more cautious in our assertions about the living? Of course referring solely or fully to the publications of researchers...is not a guarantee of veracity, objectivity... We are well aware that most of them pursue objectives other than the discovery and transmission of the knowledge to ordinary people (not being part of the elitist club).
And to conclude, let us not forget that we can only approach the world of living (and not living), other than our species, from our consciousness, without this we do not know precisely how it works, what is happening, how it acts and reacts actually and fully...in a word their intrinsic "capabilities".
Our definition of intelligence therefore allows us to judge whether a program, a machine...by the way produced by humans, is intelligent or not. Valid reasoning is sufficient to understand it.
DBAD
"Some folks are just determined to believe intelligence is a uniquely human attribute."
Non sequitur.