I was just coming to type a comment very much like yours. I agree entirely. LM Studio already does this flawlessly. Ollama's domination of the local model space and its terribly limiting configuration options are really holding back the local LM community. I have found two packages that naturally share system resources and, in the cases of those models that don't fit in VRAM, run immensely faster in. LM Studio is my top pick for that very reason.
@@fahdmirza Sir, I sent you an email earlier today regarding my initial testing of the new LM Studio release. Let me know if you found the information helpful and/or would like more in-depth analysis of such evaluations as I go. I have come to very much appreciate your coverage of new models and versions being released each day, and I often will go test them myself after you cover them here to get more insight on each one. I'm happy to share my results anytime.
I'm putting BTRFS on SSD and have to use zRAM instead of Swap to keep from thrashing my drives with writes. This is uncannily similar in the data process handling though the motivation is diff. I am wondering (now) if some of the same architectural benefits can be applied to post-training inference and post inference processes (like zRAM for vRAM) ? It's becoming a big box to think outside of...
![Felix "Unfair" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/Oswujxm2Ag0/mqdefault.jpg)
Very interesting, thanks! I find that LM Studio 3.5 chooses very well an optimized balance for CPU and GPU cooperation (to run free AI locally).
I was just coming to type a comment very much like yours. I agree entirely. LM Studio already does this flawlessly. Ollama's domination of the local model space and its terribly limiting configuration options are really holding back the local LM community. I have found two packages that naturally share system resources and, in the cases of those models that don't fit in VRAM, run immensely faster in. LM Studio is my top pick for that very reason.
Spot on.
@@fahdmirza Sir, I sent you an email earlier today regarding my initial testing of the new LM Studio release. Let me know if you found the information helpful and/or would like more in-depth analysis of such evaluations as I go.
I have come to very much appreciate your coverage of new models and versions being released each day, and I often will go test them myself after you cover them here to get more insight on each one. I'm happy to share my results anytime.
@@prague5419 Thanks a lot, yes I replied.
I'm putting BTRFS on SSD and have to use zRAM instead of Swap to keep from thrashing my drives with writes. This is uncannily similar in the data process handling though the motivation is diff. I am wondering (now) if some of the same architectural benefits can be applied to post-training inference and post inference processes (like zRAM for vRAM) ? It's becoming a big box to think outside of...
Very useful video sir
Thanks for the kind words, I am glad it was helpful.