Humans use tools to extend their abilities. LLM can do the same. In this paper they didn’t allow tool use. When others gave the tower of hanoi task to llms with tool use, like a python env, they were able to complete the task.
But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes the LLM has "reasoned" that the solution to the problem is call out to a solution that it "knows" is out there, but that's not really "thinking" about how to solve a problem in the human sense.
What happens when some novel Tower of Hanoi-esque puzzle is presented and there's nothing available in its training set to reference as an executable solution? A human can reason about and present a solution, but an LLM? Ehh...
> But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out.
The paper doesn't give any evidence humans are able to do this. And I honestly find it very implausible. Even Gary Marcus admits in (1) that humans would probably make mistakes.
You are aware that humans created and solved the puzzle in the first place, right? Not sure I understand this line of reasoning that if there are humans in this world incapable of solving some problem then boom, checkmate, LLMs can reason about and understand problems like humans do.
LLMs are perfectly capable of writing code to solve problems that are not in their training set. I ask LLMs to write code for niche problems that you won't find answers to just by Googling all the time. The LLMs usually get it right.
Maybe they can, but what the human is able to do is examine the tower of Hanoi problem, and the derive the general rule for solving (odd or even number of disks).
Based upon that comprehension, we then need little working memory (tokens) to solve the problem, it just becomes tedious to execute the algorithm.. But the algorithm was derived after considering the first 3 or 4 cases.
Whereas for the moment, LLMS are just pattern matching; whereas we do the pattern match, then derive the generalised rule.
The Tower of Hanoi problem is terrible example for somehow suggesting humans are superior.
Firstly, there are plenty of humans who can’t solve this problem even for 3 disks, let alone 6 or 7. Secondly, LLMs can both give you general instructions to solve for any case and they can write out exhaustive move lists too.
Anyway, the fact that there are humans who cannot do Tower of Hanoi already rules it out as a good test of general intelligence anyway. We don’t say that a human doesn’t have “general intelligence” if they cannot solve Towers of Hanoi, so why then would it be a good test for LLM general intelligence?
> LLMs are perfectly capable of writing code to solve problems that are not in their training set.
Examples of these problems? You'll probably find that they're simply compositions of things already in the training set. For example, you might think that "here's a class containing an ID field and foobar field. Make a linked list class that stores inserted items in reverse foobar order with the ID field breaking ties" is something "not in" the training set, but it's really just a composition of the "make a linked list class" and "sort these things based on a field" problems.
>> Ie: a human with no experience of board games cannot reason about chess moves. A human with no math knowledge cannot reason about math problems.
Then how did the first humans solve math and chess problems, if there were none around solved to give them examples of how to solve them in the first place?
Incrementally, by tiny steps. Including a lot of doing first, then realizing later this is relevant to some chess/math thing.
Also the idea of "problems" like "chess problems" and "math problems" is itself constructed. Chess wasn't created by stacking together enough "chess problems" until they turned into a game - it was invented and tuned as a game for a long time before someone thought about distilling "problems" from it, in order to aid learning the game; from there, it also spilled out into space of logical puzzles in general.
This is true of every skill, too. You first have people who master something by experience, and then you have others who try to distill elements of that skill into "problems" or "exercise regimes" or such, in order to help others reach mastery quicker. "Problems" never come first.
Also: most "problems" are constructed around a known solution. So another answer to "how did the first humans solve" them is simply, one human back-constructed a problem around a solution, and then gave it to a friend to solve. The problem couldn't be too hard either, as it's no fun to not be able to solve it, or to require too much hints. Hence, tiny increments.
The problem with this is that anything presented can be claimed to be in the training set, which is likely a zetebyte in size if not larger. However the counter-factual, the LLM failing a problem that is provably in it's training set (there are many), seems to carry no weight.
This has not been my experience. They might do something in the right direction. They might write complete garbage. But the amount of time an LLM writes code that compiles and executes first time is vanishingly few for me. Perhaps I'd have better luck if I were doing things which weren't _actual_ niche problems.
Actually, my experience at least that when dealing with novel problems LLMs fail miserably. Try accessing uncommon API’s - or areas where you’re unsure an API actually exists (REST against Exchange for admin stuff!).
Both ChatGPT and Claude produce nice looking solutions dependent on non-existent libraries. Repeatedly.
