Apple MLX: Build Your Own Private AI Server
HTML-код
- Опубликовано: 26 апр 2024
- 👋 Hi everyone! In today's tutorial, I'll show you how to set up and run a private AI server locally using MLX server, completely independent of the internet. 🖥️💻📱 We'll dive into how to create a user-friendly chat interface and test it directly from your phone. Perfect for enthusiasts and developers looking to harness the power of AI within their private network! Apple MLX: Build Your Own Private AI Server
🔍 What You'll Learn:
How to install and configure MLX server
Setting up ChainLit UI for a seamless chat experience
Running your private AI on both your computer and mobile device
🌐 mlx_server running on Mac
🤖 @chainlit_io Chat on my iPhone
🏠 100% Local AI
📍 Local Wifi Network
🔒 Data remains private
🔗 Resources:
Sponsor a Video: mer.vin/contact/
Do a Demo of Your Product: mer.vin/contact/
Patreon: / mervinpraison
Ko-fi: ko-fi.com/mervinpraison
Discord: / discord
Twitter / X : / mervinpraison
Code: mer.vin/2024/04/mlx-server-app/
📌 Timestamps:
0:02 - Introduction to MLX server setup
0:29 - Step-by-step MLX server configuration
1:07 - Installing MLX and starting the server
1:49 - Creating a UI with ChainLit
2:42 - Testing the chat interface
3:32 - Running the setup on your mobile device
💡 Make sure to subscribe and hit the bell icon to get notifications about more content like this. Smash the like button if you find this video helpful-it supports the channel and helps others discover our tutorials!
#MLX #Server #Local #MLX #AppleMLX #MLXAppleSilicon #MLXSharedMemory #MLXUsageExamples #MLXInstallationGuide #MLXVsPytorch #UnifiedMemoryMLX #AppleMLXInnovations #MLXMachineLearning #MLXQuickStart #PythonAPIMLX #MLXNeuralNetworks #MLXFineTune #MLXFineTuning #AppleMLXFineTuning #AppleAI #MLXTrain #AppleSiliconAI #AppleSilicon #AppleML #AI #MacOSLLM #MacOSAI #ML #Apple #MLTutorial #MLXTutorial #Tutorial #Guide - Хобби
one thing I would add for the novices like myself trying to follow along is the need for "pip3 install openai", probably obvious for most but I missed it initially
Another outstanding video. Short but informative, practical, link to code that works on the first go! Awesome!
Thank you
This is outstanding! Thank you Mervin!
great video Mervin, thank you!
very nice - great video , great topic... thank you Mervin
Thanks Mervin, great concise video. I have one suggestion for improvement. Can you put the terminal commands in the description?
Hi Mervin. A lot of companies have been connecting their AI chatbot on their social medias like whatsapp and also in multiple languages. Could you show us how can we do that? Basically a RAG connected on our socials in several different languages.
Love this 👍👍👍
Thank you
Great tutorial! Two requests: please provide sysreqs for the setups in these videos and for this one maybe show how to access this remotely.
Excellent video Mervin. Does this require GPU or it can perfectly run on CPU.
Thanks!
Thank you
brilliant work. can you create a video on how to train llama3 and then use groq with that model? maybe even how to deploy it on places like digital ocean or aws. Thanks keep it up
Thank
What kind of m achine on mac does this work well? Feel like it doesn't work at all on my m1 macbook air. Seems like the server and the ui are not commuicating, or my machine is too slow
I just purchased an m3 max macbook pro, so I will probably actually implement what you are suggesting in this video
Sure
How much ram did you get with the mac book. The M chips are great at running LLMs but the limiting factor is RAM. If you maxed out the ram to 128gb on the M3 then you can run almost any OS model out there - quantized.
@@vincentnestler1805 yeah, I got the maxed out version. Only one terabyte of storage though.128 gb of ram.
@@MervinPraison Do you think I am a marketing bot or something? I just didn't expect a comment from a RUclips content creator to be so curt and seemingly skeptical about my purchase. You could have just ignored my comment. What are you really adding to the convo by saying, "Sure"?
I really appreciate your tutorials by the way. Thanks for posting them.
Interesting I could do the same with ollama server also correct?
thanks your tutorial! is it only for apple machine?
The framework is made for apple
@@BangaloreRUclips is it possibblr to extend to window or linux?
I will do a tutorial soon to do it for windows and Linux but with a different package
@@lcwmg there’s other libraries that’ll give you way more flexibility for windows. Look for cuda optimising libraries. windows is better for ML anyday. All the best.
@@lcwmg there’s other libraries that’ll give you way more flexibility on windows it is better for ML anyday. All the best.
Hello mate, is this possible on Windows
Also, production ready so this can be used by a local business vía internet 😢
Sorry this is available only for Mac decide
But I will create another one video to include both windows and Linux
@@MervinPraison thanks mate
Genuine question, why use mlx server when If you use streamlit this is provided by out from the box? I can access my streamlit app trough my phone
Yes you can use steam lit instead of chain lit
MLX for serving LLM llama 3
When I run pip install mlx-lm I get an error.
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible
Make sure you install it on Mac Computers
@@MervinPraison I don’t have a mac. Is this solution for Mac only? Would have been nice to mention that in the video
I am working on a project, and the user talks to the model via text
👍
@@MervinPraison What projects are you working on?
This is misleading. You can access any port on the local network from any browser on a device connected to the network. It has nothing to do with your phone other than that your phone has a browser. If you left your house, you'd no longer even have access to the server; you'd need to set up tunneling for that, which you can do most easily with a service like ngrok. But this is not running on your phone or anything of the sort.
Why so many cuts in your video?
Particularly in which section ?
Can you and Matthew Berman please take one day off per week...ideally on the same day, so that I get one day off a week to play with my dogs, cut the grass, drink a cocktail, go for a swim, and read a fictional book - without having to learn anything new for the day.
Haha 😂 love this
Need to talk to him and make a deal
you are NOT running “this on your phone”. you simply open the web ui from it. which is a completely different thing.
At least explain first why Apple MLX, what are the features and benefits compare to others.