Calling it a web crawler is just innacurate. You can give it access to a web search engine, which is how the “AI search engines” work, but LLMs can’t access the internet on their own. They’re completely self-contained unless you give them tools that let them do other things.
I would agree calling it a web crawler is inaccurate, but disagree with the reasoning; I think it’s more in the sense that calling an LLM a web crawler is akin to calling a search index a web crawler; in other words, an LLM could be considered a weird version of a search index.
Yeah, I can see that. It’s definitely more like a search index than a web crawler. It’s not great at being a search index though, since it can synthesize ideas but can’t reliably tell you where it got them from in the first place.
Calling it a web crawler is just innacurate. You can give it access to a web search engine, which is how the “AI search engines” work, but LLMs can’t access the internet on their own. They’re completely self-contained unless you give them tools that let them do other things.
I would agree calling it a web crawler is inaccurate, but disagree with the reasoning; I think it’s more in the sense that calling an LLM a web crawler is akin to calling a search index a web crawler; in other words, an LLM could be considered a weird version of a search index.
Yeah, I can see that. It’s definitely more like a search index than a web crawler. It’s not great at being a search index though, since it can synthesize ideas but can’t reliably tell you where it got them from in the first place.
Firmly agree with you on that.