I tried reproducing this research from scratch and the surprising thing was how expensive it would be from a compute perspective. I imagine that in the future, we’ll have pretrained sparse autoencoders shipped with pretrained models. Then the end user would fine tune both of them for their downstream application. That’s the thing that’s needed and maybe their approach
This is a terrific subject, and I hope things from mechanistic interpretability work back into neuroscience as better brain scanning comes on line. Extraction of information structure is the root of compression. What abstraction do we get when we train a model on the weights from another pre-trained model? Would it be able to predict the changes to those weights as that pre-trained model is then given more training? ("Model, model thyself.")
if you've got 100 million in a model and you can't figure out what's going awry, who you gonna call.. Goodfire! 😂 that's interesting though, I wasn't aware of sparse autoencoders
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
YOU HAVE THE BEST PODCAST ON RUclips. Edge of my seat over here with every episode. Thanks for making these!
I tried reproducing this research from scratch and the surprising thing was how expensive it would be from a compute perspective.
I imagine that in the future, we’ll have pretrained sparse autoencoders shipped with pretrained models. Then the end user would fine tune both of them for their downstream application. That’s the thing that’s needed and maybe their approach
What an absolutely fascinating and exciting subject. I hope the field gets all the money thrown at it!
This is a terrific subject, and I hope things from mechanistic interpretability work back into neuroscience as better brain scanning comes on line. Extraction of information structure is the root of compression. What abstraction do we get when we train a model on the weights from another pre-trained model? Would it be able to predict the changes to those weights as that pre-trained model is then given more training? ("Model, model thyself.")
👏
if you've got 100 million in a model and you can't figure out what's going awry, who you gonna call.. Goodfire! 😂 that's interesting though, I wasn't aware of sparse autoencoders