Ok, done. I can report to you that it helped me cut down my personal search space. Imagine what such a tool could do in the hands of a subject matter expert with rudimentary critical thinking ability and the faintest hint of a grasp of using the scientific method to verify claims, wow..
You're making a fundamental error in your reasoning. An LLM's training corpus being fixed doesn't limit the system's total information processing capability when used as a tool by a human researcher. While the LLM itself can't generate truly novel information (per the data processing inequality), a human researcher using it as a dynamic search and analysis tool can absolutely generate new insights and discoveries through their interaction with it. The human-LLM system is open, not closed.
This is analogous to how a calculator cannot output any number that isn't computationally derivable from its programming, yet humans using calculators have discovered new mathematical proofs. The tool augments human capability without being AGI.
Your argument is essentially claiming that because a microscope can't generate new cellular structures, it can't help biologists make new discoveries.
You have a fundamental misunderstanding of how LLMs work, which is why you think they are magical.
Of course if you play the LLM Pachinko machine you can get all sorts of novel output from it, but it’s only useful for certain tasks. It’s great for translation, summarizing (also a kind of translation), and to some degree it can recall from its training corpus an interesting fact. And yes, it can synthesize novel content such as poetry, or adapt an oft-used coding pattern in a flavor specified by a prompt.
What it can’t do is come up with a new idea. At least not in a way better than rolling a dice. It may come up with an idea that you, dear reader, may not have encountered, which makes it great for education.
I don’t have anything more to say, but you’re welcome to continue this discussion with an agent of your choice.
