Fascinating work. I’m curious though, as these systems get more complex and have more latent knowledge/capabilities stored in their weights/dynamics, at what point(s) do the lines (higher dimensional boundaries?) between reasoning vs. retrieval get blurred. It’s hard for me to imagine that human reasoning doesn’t rely on or utilize some concrete or abstract elements of retrieval e.g., neurons activating and communicating with each other through dynamic feedback loops. Also, in some sense we are always retrieving information from our external environment as well.
Fascinating work. I’m curious though, as these systems get more complex and have more latent knowledge/capabilities stored in their weights/dynamics, at what point(s) do the lines (higher dimensional boundaries?) between reasoning vs. retrieval get blurred. It’s hard for me to imagine that human reasoning doesn’t rely on or utilize some concrete or abstract elements of retrieval e.g., neurons activating and communicating with each other through dynamic feedback loops. Also, in some sense we are always retrieving information from our external environment as well.