Imagine asking a librarian “What was happening in Los Angeles in the Summer of 1989?” and that person fetching you … That’s modern LLMs in a nutshell.
I agree, but I think you’re still being too generous to LLMs. A librarian who fetched all those things would at least understand the question. An LLM is just trying to generate words that might logically follow the words you used.
IMO, one of the key ideas with the Chinese Room is that there’s an assumption that the computer / book in the Chinese Room experiment has infinite capacity in some way. So, no matter what symbols are passed to it, it can come up with an appropriate response. But, obviously, while LLMs are incredibly huge, they can never be infinite. As a result, they can often be “fooled” when they’re given input that semantically similar to a meme, joke or logic puzzle. The vast majority of the training data that matches the input is the meme, or joke, or logic puzzle. LLMs can’t reason so they can’t distinguish between “this is just a rephrasing of that meme” and “this is similar to that meme but distinct in an important way”.
Can you explain the difference between understanding the question and generating the words that might logically follow? I’m aware that it’s essentially a more powerful version of how auto-correct works, but why should we assume that shows some lack of understanding at a deep level somehow?
Can you explain the difference between understanding the question and generating the words that might logically follow?
I mean, it’s pretty obvious. Take someone like Rowan Atkinson whose death has been misreported multiple times. If you ask a computer system “Is Rowan Atkinson Dead?” you want it to understand the question and give you a yes/no response based on actual facts in its database. A well designed program would know to prioritize recent reports as being more authoritative than older ones. It would know which sources to trust, and which not to trust.
An LLM will just generate text that is statistically likely to follow the question. Because there have been many hoaxes about his death, it might use that as a basis and generate a response indicating he’s dead. But, because those hoaxes have also been debunked many times, it might use that as a basis instead and generate a response indicating that he’s alive.
So, if he really did just die and it was reported in reliable fact-checked news sources, the LLM might say “No, Rowan Atkinson is alive, his death was reported via a viral video, but that video was a hoax.”
but why should we assume that shows some lack of understanding
Because we know what “understanding” is, and that it isn’t simply finding words that are likely to appear following the chain of words up to that point.
Just if you were a hater that would be cool with me. I don’t like “ai” either. The explanations you give are misleading at best. It’s embarrassing. You fail to realise the fact that NOBODY KNOWS why or how they work. It’s just extreme folly to pretend you know these things. It’s been observed to reason novel ideas which is why it is confusing for scientists that work with them why it happens. It’s not just data lookup. You think entire Web and history of man fits in 8 gb? You are just educating people with just your basic rage filled opinion, not actual answers. You are angry at the discovery, we get that. You don’t believe in it. Ok. But don’t say you know what it does and how, or what openai does behind its closed doors. It’s just embarrassing. We are working on papers to try to explain the emergent phenomenon we discovered in neural nets that make it seem like it can reason and output mostly correct answers to difficult questions. It’s not in the “data” and it looks for it. You could just start learning if you want to be an educator in the field.
The Rowan Atkinson thing isn’t misunderstanding, it’s understanding but having been misled. I’ve literally done this exact thing myself, say something was a hoax (because in the past it was) but then it turned out there was newer info I didn’t know about. I’m not convinced LLMs as they exist today don’t prioritize sources – if trained naively, sure, but these days they can, for instance, integrate search results, and can update on new information. If the LLM can answer correctly only after checking a web search, and I can do the same only after checking a web search, that’s a score of 1-1.
because we know what “understanding” is
Really? Who claims to know what understanding is? Do you think it’s possible there can ever be an AI (even if different from an LLM) which is capable of “understanding?” How can you tell?
I’m not convinced LLMs as they exist today don’t prioritize sources – if trained naively, sure, but these days they can, for instance, integrate search results, and can update on new information.
Well, it includes the text from the search results in the prompt, it’s not actually updating any internal state (the network weights), a new “conversation” starts from scratch.
Yes that’s right, LLMs are context-free. They don’t have internal state. When I say “update on new information” I really mean “when new information is available in its context window, its response takes that into account.”
I agree, but I think you’re still being too generous to LLMs. A librarian who fetched all those things would at least understand the question. An LLM is just trying to generate words that might logically follow the words you used.
IMO, one of the key ideas with the Chinese Room is that there’s an assumption that the computer / book in the Chinese Room experiment has infinite capacity in some way. So, no matter what symbols are passed to it, it can come up with an appropriate response. But, obviously, while LLMs are incredibly huge, they can never be infinite. As a result, they can often be “fooled” when they’re given input that semantically similar to a meme, joke or logic puzzle. The vast majority of the training data that matches the input is the meme, or joke, or logic puzzle. LLMs can’t reason so they can’t distinguish between “this is just a rephrasing of that meme” and “this is similar to that meme but distinct in an important way”.
Can you explain the difference between understanding the question and generating the words that might logically follow? I’m aware that it’s essentially a more powerful version of how auto-correct works, but why should we assume that shows some lack of understanding at a deep level somehow?
I mean, it’s pretty obvious. Take someone like Rowan Atkinson whose death has been misreported multiple times. If you ask a computer system “Is Rowan Atkinson Dead?” you want it to understand the question and give you a yes/no response based on actual facts in its database. A well designed program would know to prioritize recent reports as being more authoritative than older ones. It would know which sources to trust, and which not to trust.
An LLM will just generate text that is statistically likely to follow the question. Because there have been many hoaxes about his death, it might use that as a basis and generate a response indicating he’s dead. But, because those hoaxes have also been debunked many times, it might use that as a basis instead and generate a response indicating that he’s alive.
So, if he really did just die and it was reported in reliable fact-checked news sources, the LLM might say “No, Rowan Atkinson is alive, his death was reported via a viral video, but that video was a hoax.”
Because we know what “understanding” is, and that it isn’t simply finding words that are likely to appear following the chain of words up to that point.
Just if you were a hater that would be cool with me. I don’t like “ai” either. The explanations you give are misleading at best. It’s embarrassing. You fail to realise the fact that NOBODY KNOWS why or how they work. It’s just extreme folly to pretend you know these things. It’s been observed to reason novel ideas which is why it is confusing for scientists that work with them why it happens. It’s not just data lookup. You think entire Web and history of man fits in 8 gb? You are just educating people with just your basic rage filled opinion, not actual answers. You are angry at the discovery, we get that. You don’t believe in it. Ok. But don’t say you know what it does and how, or what openai does behind its closed doors. It’s just embarrassing. We are working on papers to try to explain the emergent phenomenon we discovered in neural nets that make it seem like it can reason and output mostly correct answers to difficult questions. It’s not in the “data” and it looks for it. You could just start learning if you want to be an educator in the field.
The Rowan Atkinson thing isn’t misunderstanding, it’s understanding but having been misled. I’ve literally done this exact thing myself, say something was a hoax (because in the past it was) but then it turned out there was newer info I didn’t know about. I’m not convinced LLMs as they exist today don’t prioritize sources – if trained naively, sure, but these days they can, for instance, integrate search results, and can update on new information. If the LLM can answer correctly only after checking a web search, and I can do the same only after checking a web search, that’s a score of 1-1.
Really? Who claims to know what understanding is? Do you think it’s possible there can ever be an AI (even if different from an LLM) which is capable of “understanding?” How can you tell?
Well, it includes the text from the search results in the prompt, it’s not actually updating any internal state (the network weights), a new “conversation” starts from scratch.
That’s not true for the commercial ai’s. We don’t know what they are doing
Yes that’s right, LLMs are context-free. They don’t have internal state. When I say “update on new information” I really mean “when new information is available in its context window, its response takes that into account.”
So, what is ‘understanding’?
If you need help, you can look at marx for an answer that still mostly holds up, if your server is an indication of your reading habbits.
oh does he have a treatise on the subject?
He’s said some relevant stuff
nice
Im not sure it supports the argument he’s actually making, but its true and valid here.