LLMs as General Pattern Machines: Use Arbitrary Tokens to Pattern Match?

(Part 1) This is an interesting paper as it shows that LLMs can represent, manipulate, and extrapolate more abstract, nonlinguistic patterns. This is an interesting finding as all along, we have been thinking that LLMs are just great text-based completers for text with some semantic meaning.
However, I show that the methods used in this paper may not be ideal. Firstly, using random tokens is not a good strategy, as the semantic priors of these tokens still get used and may influence the results!
(Part 2) Moreover, using a reward-based approach like in Decision Transformers still takes a long time to learn. I propose a Goal-Directed Decision Transformer instead and show that it outperforms the method used in this paper!
Using an LLM as a way to associate patterns will likely be the underpinning of intelligence. However, my view is that this approach of using abstract tokens is probably not the right one.
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Paper: arxiv.org/abs/2307.04721
Github: general-pattern-machines.gith...
Slides: github.com/tanchongmin/Tensor...
Part 2 here containing my ideas on Goal-Directed Decision Transformers as well as a 10-year plan on intelligence: • LLM as Pattern Machine...
Random Tokens for Labels does not affect much? - Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? arxiv.org/abs/2202.12837
Semantically Wrong Labels affect models - Larger Language Models do In-Context Learning Differently: arxiv.org/abs/2303.03846
Decision Transformer: arxiv.org/abs/2106.01345
My Related Videos:
LLMs to solve ARC: • LLMs as a system to so...
Learning, Fast and Slow: • Learning, Fast and Slo...
~~~~~~~~~~~~~~~~~~~~~~~~~~~
0:00 Introduction
1:09 Three approaches
8:49 Can we use random tokens?
13:43 Experiments to show mapping to random tokens may not work well!
25:38 Wrong Semantics Affect Performance
30:53 Sequence Transformation - ARC Challenge
42:08 Sequence Transformation - Grasp Detection and Forward Dynamics Prediction
45:55 Sequence Completion
49:42 Sequence Improvement - Decision Transformers
54:59 Sequence Improvement - Cart Pole
1:04:42 Markov Decision Process
1:14:45 Sequence Prediction in Cart Pole
1:18:00 Token semantic priors affect output in Cart Pole
1:22:00 How to improve Cart Pole tokenisation
1:23:55 Teaser: Is learning reward necessary?
1:32:00 Discussion
~~~~~~~~~~~~~~~~~~~~~~~~~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: / discord
LinkedIn: / chong-min-tan-94652288
Online AI blog: delvingintotech.wordpress.com/
Twitter: / johntanchongmin
Try out my games here: simmer.io/@chongmin

[@simonstrandgaard5503]

Great walk through.

[@johntanchongmin]

1:14:50 For Cart Pole, I just took a look at their Jupyter Notebook, and realise they did it by predicting action at each timestep, so this part is similar to Decision Transformer.
However, for embeddings, they just take the entire text of the sequence and embed it. This is suboptimal, as some numbers may be chunked together and some may not. It would be better if the embeddings are done per number instead of leaving it to the tiktoken tokeniser!

[@johntanchongmin]

Part 2 here: ruclips.net/video/rZ6hgFEe5nY/видео.html