Great work here. Seems like the more we tinker with AI, the more we learn the divide between our understanding of intelligence and intelligence itself. At some point, we'll have to come to the conclusion that statistical models can't explain very much. It's odd how obsessed our culture is with stats and probabilities, that there's even a notion of "laws of probability" when probability is a clever workaround rather than an explanation of any phenomenon.
Yes, the statistical models will have their uses, it is just they can't accomplish many of the types of tasks for which they are being hyped.
In the end, maybe many will have a greater appreciation and understanding for just how special true intelligence really is. In the future, I hope to write a bit more specifically on the distinction to bring more clarity to the topic.
You always do such a great job of distilling the primary details in common terms. That clarity is missing from many of the AI analysis which at times seems to be hall of mirrors that the author uses to try to bring fourth answers that are not there. I really enjoy reading your work on AGI.
It's important to put this in perspective. LLMs are a great time saver: hey ChatGPT, here is a a study I need you to create a table that compares X to Y. It's brilliant at that, you can go back and check it, boom, you're done. But you need to be conversant in the material first.
I have a 20 year old who wants to go into finance, and I tell him he still needs to be good at prompting because those with lesser skill will be using it to make decisions that will affect him, so he needs to develop even superior skills to spot errors while also being able to provide work with the tools that the industry is using. Being able to pick apart nonsense goes a long way to build a reputation for excellence. LLMs and this misplaced belief in their inerrancy will prohibit people from developing judgement if management is always asking "Well, what did Claude say?"
Yes, to your point, LLMs are probably one of the most awkward technologies ever invented, being both extremely useful and treacherous, with no distinction between the two being obvious to the user.
It is one of the topics I hope to cover more in depth soon, that being how to properly use LLMs.
Did you read the paper? Far from revealing that these models are purely statistical, it shows that AI models are coming up with a variety of sophisticated strategies to solve complex problems and requests. Here’s a quote: “Our results uncover a variety of sophisticated strategies employed by models. For instance, Claude 3.5 Haiku routinely uses multiple intermediate reasoning steps “in its head” 2 to decide its outputs. It displays signs of forward planning, considering multiple possibilities for what it will say well in advance of saying it. It performs backward planning, working backwards from goal states to formulate earlier parts of its response. We see signs of primitive “metacognitive” circuits that allow the model to know the extent of its own knowledge. More broadly, the model’s internal computations are highly abstract and generalize across disparate contexts.”
Yes, I read it. However, none of that says it is not a statistical model. Statistical derived pattern matching can self organize into some form algorithms. But those "algorithms" are still within the bounds of the architecture in which they execute.
It is still a function of probabilities that are operating against the training data. I don't deny that this can become extremely sophisticated. I mention this in the post. However, there are specific limits to that capability that emerge in certain forms such as hallucinations.
There is no mechanism present for the creation of semantic information. Yes, it can do some sophisticated "analysis" of patterns, that is no doubt useful, but it diverges from the capabilities we would expect from true understanding of fundamental concepts. Edge cases don't appear when there is fundamental understanding.
The models aren't coming up with a variety of sophisticated strategies. The developers of the models are coming up with these strategies for the model. The computer program is programmed to complete these "reasoning" steps.
Thank you! Ha! no I didn't post on r/agi. Someone else posted it there, but the comments were typical Reddit :-) I would estimate less than 1% read the article. Nonetheless, it was useful to read through to summarize common criticisms which I responded to in my latest post today.
What I want to know is if this is known or understood, then why have people who've won nobel prizes for their work in AI seem to think agi is coming soonish. Is there something they're missing or something you're missing 🤔.
Yes, it is definitely a topic of some obscurity, and not always well agreed-upon-definitions. I hope to continue to write more on the subject and hopefully bring more clarity.
Great work here. Seems like the more we tinker with AI, the more we learn the divide between our understanding of intelligence and intelligence itself. At some point, we'll have to come to the conclusion that statistical models can't explain very much. It's odd how obsessed our culture is with stats and probabilities, that there's even a notion of "laws of probability" when probability is a clever workaround rather than an explanation of any phenomenon.
Yes, the statistical models will have their uses, it is just they can't accomplish many of the types of tasks for which they are being hyped.
In the end, maybe many will have a greater appreciation and understanding for just how special true intelligence really is. In the future, I hope to write a bit more specifically on the distinction to bring more clarity to the topic.
You always do such a great job of distilling the primary details in common terms. That clarity is missing from many of the AI analysis which at times seems to be hall of mirrors that the author uses to try to bring fourth answers that are not there. I really enjoy reading your work on AGI.
It's important to put this in perspective. LLMs are a great time saver: hey ChatGPT, here is a a study I need you to create a table that compares X to Y. It's brilliant at that, you can go back and check it, boom, you're done. But you need to be conversant in the material first.
I have a 20 year old who wants to go into finance, and I tell him he still needs to be good at prompting because those with lesser skill will be using it to make decisions that will affect him, so he needs to develop even superior skills to spot errors while also being able to provide work with the tools that the industry is using. Being able to pick apart nonsense goes a long way to build a reputation for excellence. LLMs and this misplaced belief in their inerrancy will prohibit people from developing judgement if management is always asking "Well, what did Claude say?"
Thank you very much! Much appreciated!
Yes, to your point, LLMs are probably one of the most awkward technologies ever invented, being both extremely useful and treacherous, with no distinction between the two being obvious to the user.
It is one of the topics I hope to cover more in depth soon, that being how to properly use LLMs.
I agree! Artificial Information is not Digital Intelligence.
Wildly inaccurate analysis here. Some folks are just determined to believe intelligence is a uniquely human attribute.
Maybe you could identify the error in analysis?
Did you read the paper? Far from revealing that these models are purely statistical, it shows that AI models are coming up with a variety of sophisticated strategies to solve complex problems and requests. Here’s a quote: “Our results uncover a variety of sophisticated strategies employed by models. For instance, Claude 3.5 Haiku routinely uses multiple intermediate reasoning steps “in its head” 2 to decide its outputs. It displays signs of forward planning, considering multiple possibilities for what it will say well in advance of saying it. It performs backward planning, working backwards from goal states to formulate earlier parts of its response. We see signs of primitive “metacognitive” circuits that allow the model to know the extent of its own knowledge. More broadly, the model’s internal computations are highly abstract and generalize across disparate contexts.”
Yes, I read it. However, none of that says it is not a statistical model. Statistical derived pattern matching can self organize into some form algorithms. But those "algorithms" are still within the bounds of the architecture in which they execute.
It is still a function of probabilities that are operating against the training data. I don't deny that this can become extremely sophisticated. I mention this in the post. However, there are specific limits to that capability that emerge in certain forms such as hallucinations.
There is no mechanism present for the creation of semantic information. Yes, it can do some sophisticated "analysis" of patterns, that is no doubt useful, but it diverges from the capabilities we would expect from true understanding of fundamental concepts. Edge cases don't appear when there is fundamental understanding.
The models aren't coming up with a variety of sophisticated strategies. The developers of the models are coming up with these strategies for the model. The computer program is programmed to complete these "reasoning" steps.
Thank you! Ha! no I didn't post on r/agi. Someone else posted it there, but the comments were typical Reddit :-) I would estimate less than 1% read the article. Nonetheless, it was useful to read through to summarize common criticisms which I responded to in my latest post today.
lol, yes. I don't post much on the Reddit AI forums anymore. I was perma banned from r/Singularity a long time ago, lol.
I did post it on Hacker News, a bit more balanced commentary there, but still the top voted comment was dismissing it all.
What I want to know is if this is known or understood, then why have people who've won nobel prizes for their work in AI seem to think agi is coming soonish. Is there something they're missing or something you're missing 🤔.
Yes, it is definitely a topic of some obscurity, and not always well agreed-upon-definitions. I hope to continue to write more on the subject and hopefully bring more clarity.
artificial information != artificial intelligence