Apple M3 Max MLX beats RTX4090m
HTML-код
- Опубликовано: 16 май 2024
- Try Paperlike here: paperlike.com/alex
Apple MacBook Pro with the M3 Max chip is even more capable in Machine Learning workflows now that MLX Framework is out. Here I test it against the nVidia RTX 4090 laptop version in one of my typical workflows - speech to text.
Run Windows on a Mac: prf.hn/click/camref:1100libNI (affiliate)
Use COUPON: ZISKIND10
🛒 Gear Links 🛒
🍏💥 New MacBook Air M1 Deal: amzn.to/3S59ID8
💻🔄 Refurb MacBook Air M1 Deal: amzn.to/45K1Gmk
🎧⚡ Great 40Gbps T4 enclosure: amzn.to/3JNwBGW
🛠️🚀 My nvme ssd: amzn.to/3YLEySo
📦🎮 My gear: www.amazon.com/shop/alexziskind
🎥 Related Videos 🎥
* 🤖 REALITY vs Apple’s Memory Claims | vs RTX4090m - • REALITY vs Apple’s Mem...
* 👨💻 Cheap vs Expensive MacBook for ML | M3 Max - • Cheap vs Expensive Mac...
* 🤖 INSANE Machine Learning on Neural Engine - • INSANE Machine Learnin...
* 👨💻 M1 DESTROYS a RTX card for ML - • When M1 DESTROYS a RTX...
* 🌗 RAM torture test on Mac - • TRUTH about RAM vs SSD...
* 👨💻 M1 Max VS RTX3070 - • M1 Max VS RTX3070 (Ten...
🛠️Code🛠️
github.com/TristanBilot/mlx-b...
github.com/ggerganov/whisper.cpp
- - - - - - - - -
❤️ SUBSCRIBE TO MY RUclips CHANNEL 📺
Click here to subscribe: www.youtube.com/@azisk?sub_co...
- - - - - - - - -
📱LET'S CONNECT ON SOCIAL MEDIA
ALEX ON TWITTER: / digitalix
- - - - - - - - -
#m3max #m2max #machinelearning 
Наука
JOIN: youtube.com/@azisk/join
yeah but can it run crysis
yeah but can crysis feed family? idk
@@adrimi5 family goes on diet after each new pro device gets released
Indeed it can, if you know the right toolkits to download and terminal commands to run
Yep! Using Crossover.
I’m running GOG Cyberpunk 2077 on my M3 Max using crossover.
Awesome video! I would love to see more LLM or other DL architecures benchmarked between the M3 Max and the RTX 4090m laptop. A definitive video saying the M3 Max is X% better/worse than the 4090m for RNN, CNN, or transformer architecutres would be a gold mine for other AI/ML devs like me!
Watched tens of your videos before upgrading from my old i9 MacBook Pro to my M3 Max MacBook Pro.
Nowadays I still watch your videos (even if I already have an M3 MacBook) because I like the way you make your content - pragmatism, tone of voice, length and cuts.
👏
How much RAM did you get? I cannot decide between 36 GB, 48 GB or maybe even 64 GB (for future proofing).
@@tybaltmercutio same situation as OP, went with 14" and 64gb ram
i just keep my i9 macbook pro alongside with alienware rtx4090
Great to see more MLX content. Please do a comparison with Stable Diffusion MLX vs PC!
Found your channel from Fireship vid ~2ya. Awesome stuff!
How would a Mac Studio M2 32GB stack up vs the MBP M3?
to fine tune llama on m3 max, what size Llama work?how fast?can you release a video for this topic？
Hey Alex, I was wondering if you have a video planned for your EDC as a software engineer. I’ve been looking for a light case that I can carry around for my 16 MacBook with the 12.9 iPad. Trying to get ideas of what you utilize
Thank you! What is the correct way of comparing my current AMD Radeon Pro 5300M 4 GB (MacBook Pro 2019) to an Apple M silicons? In terms of a MacBook gaming experience. I am playing a game from time to time and would like to make sure that a M chip won't take it away from me :)
Hmm this difference mayo is from ram/vram sharing on arm Macs.
ARM GPU can use up to 75% of ram as vram. I don’t know that you’ve 64/96/128 RAM versions, but in all cases will be more vram than 20gb in 4090.
Hi can you suggestion which laptop best for LLM + Deep learning I did want to any pc can you please help me
Very nice Video, but can you try Faster Whisper for python on your the devices?
No it’s not faster. You’re not using fast whisper. Also python implementation absolutely uses the gpu. Set device to mps
Yes can we discuss for setting hardware for building llm
Try that unplugged...
@@stephanemignot100 of course man, if you plug it in it’s faster. If you leave it unplugged it slower I’m not debating the fact that the M3 Max is a wonderful chip. All I’m saying is that even the Nvidia 4090 at its peak capability is faster if you want to say that the battery is worse, absolutely not denying that but the M3 Max GPU is not faster than the 40,90
@@asjsjsienxjsks6734090 doesn’t have 19GB VRAM 😂
@@RunForPeace-hk1cu where did I say that?
Is there anyway to run mlx inside xcode ios project?
Can you make iPad and iPhone app versions of these tests so we can benchmark m4 on iPad in couple of days?
WSL & even Windows itself has a lot of overhead. If you wanted a more "Apples to Apples" comparison, you should've compared it with the 4090 laptop running something like Clear Linux or Ubuntu. It likely would've not closed the gap but the results would be a lot better.
It does close the gap, it actually easily outperforms M3 with a completely flatlined system, it’s just Apple has a nicer interior than most offbrand and microsoft computers. A maxed Lenovo for example outperforms a maxed M3 on UE5.
Hi Alex! ❤ love ya my guy 😊your videos are incredible! Can’t wait to fork 🍴
Can you make a video on how to install llama using ml?
Hi alex can i get your mentorship session i m ready to pay for hardware setup for building llm
Thanks.
Could you pls make a video on stable diffusion ComfyUI on Mac, I don’t know why nobody ever made any videos about it
In French bilot sounds like “be low”
Anybody have a roadmap for me to learn on what about a language or framework performs better on one arch or another. How clever can tensor operations get? Python I get. But what is it b/w mlx, cpp and ggml, jax and mojo?
Alex, I found your channel when researching for my M3 max laptop purchase. I love your benchmark methodology, but also wish I could copy some of your workflows. If you added a code repository to your membership, I would join!
As much as I'd like you to join, there is no need to join to see my repos. This is a "better late than never" repo of my tests which I recently started: github.com/alexziskind1/machine_tests
Hey, amaizing video very useful, 5:18 - i am interesting to see the video how to install whisper with support of GPU etc.
Coming soon!
@@AZisk - i already testing with Nvidia P40, but its was interesting to see your results
Hello guys! I might sound weird but how can I look at the subscriptions?:D
Wow, exciting results! I was always optimistic that Apple's unified memory architecture would pay dividends in certain workloads, and MLX appears to be effectively exploiting that paradigm shift.
Keep up the good work! Love the channel!
My apologies if I am being dumb, by why wouldn't you use an NPU for this machine learning process, as I thought this is the sought of task NPUs were designed for, and maybe even better at than a GPU? And if you could, how would the performance compare when running on an Apple Silicon NPU (on paper M3 NPU is 18 TOPS for FP16)? And as every processor manufacturer is now getting on the AI bandwagon, you could even extend it to compare the performance of AMD 7000 series with AI NPU (10 TOPS, 8000 series NPUs 16 TOPS) or Intel's Meteor Lake core Ultra with NPU (10 TOPS)? Of course, the processor I would really like to see would be Qualcomm's Snapdragon X Elite with its 45 TOPS NPU, but that's yet to be released.
Have not had good luck running ai workloads on wsl or wsl2 with a discrete gpu. Everything says my gpu is being used incl docs but performance is pathetic.
Great video, Alex! You have some really enjoyable content on your channel.
Are you able to send me one of your old M-series Macs; I’m a student and I’m trying to learn some ML/AI stuff.
Want to watch the stable diffusion one. Want to meet up? I'm in DMV
RTX 4090m is equivalent to the desktop RTX 3080 btw.
NOPE. RTX 4090 mobile = 3090Ti desktop = 4070TI . 40 tflops all.
Have you tried timing all the machines with the model already loaded in the GPU's ram to test the raw compute power? It would also be a fairer comparison with cloud-hosted solutions. Anyways, wild that Apple hasn't sent anything to the only ML/AI reviewer on RUclips. AI/ML is the core reason for me to update from M1/2 to M3 Max.
4090 should be faster as long as the model fits in the VRAM... if the model goes outside... it will be slower.
Serious question: why would anyone buy a windows pc when you can buy a Mac that not only can run windows on it but runs windows BETTER than a windows pc??? In buying a computer soon and would appreciate the feedback. Thanks.
If power usage isn't your concern, then a PC can and will be faster. 4th Gen Core i9 + RTX 4090 will likely dominate in all benchmarks. For truly mobile performance (as in on battery, not plugged into a wall), Apple undeniably has the best product on the market right now. So long as you don't want to play any games on it.
for mobile platforms apple makes sense, for in house usage, it still lags behind by a lot, unless you are already deep into the apple ecosystem or simply prefer it, for pretty much every benchmark the only metric apple is going to win is in power usage which matters a lot in laptops, in desktop, not so much when while using more power, will get the job done far quicker.
very interesting video ... but why do you have so many laptops lying around? :o
for testing
My takeway is some fancy tech words to explore next week 😢
I hope all of these were plugged in and not on battery. Also on the win laptop please go to power plan and make sure the gpu is maxed out
Its a fucking laptop and we dont usually using charger outside… with that huge charger waste of space in the bag
@@MrFhelix17 excuse me? Without the PSU the test is pretty much IRRELEVANT, I can't believe I'm reading such a silly comment, what a pointless video then! Those GTX laptops power right down when running on battery.
Can't believe this how pointless can people be!
@@motherofallemails It has a BATTERY 😮😮🔋🙀🤯😱 (this is ragebait. Please get mad)
@@ClearGalaxies so has my laptop, the rtx goes into super low power mode when running on battery, otherwise it would drain the battery in no time at 160W, you can't do anything practical off the batteries! the fact that this test was run off battery power makes this channel a joke, sorry.
In fact I'm a bit annoyed at having wasted my time. I'm OUT. 🤬
@@motherofallemails I was trolling. I know 💙
7:23 Vision Pro Light Seal Cushion spotted 👀
you got me. i still have mine
Wait what! Last week or two when I checked, Whisper still didn’t support Metal!
Been using whisper metal via python and whisper.cpp for months now
Bro litterlay had a dozen macs!
Hi, Alex How are you.. ?
yo!
We want more content about MLX
"PC Master Race" on suicide watch !! 😂
(and yes, it's quite probably the M-series chips' Unified Memory architecture that's making the difference here)
lolz....the limitation here is PCIe bottleneck...not Nvidia GPU.... if NVLink protocol running on PC it's will destroy day and night M3 max
?... Lol are you aware that Nvidia is in the making of ARM SOC themselves. You know what that means... Dont you ?... I hate Nvidia pricing. But I know one thing. Thease guys dont play when it comes to performance. Every one knows that when Nvidia releases ARM based SOC in upcoming years... Its gonna destroy everything on the market. Like it always does. Also... This laptop does NOT have RTX 4090. Not even close...
if nvidia starts making the entire SoC, they might beat apple, but they are doing too well in just discrete gpus to try that
@@gytispranskunas4984 why do you hate nvidia pricing? They cost same as AMD but providing RT cores, cuda and they are more stable. Quality and R&D costs money too
PC users huffing copium in the comments section 😂
hi lenovo loq i5 12450h 8gb 4060 80k vs ideapad ryzen 7 5800h 6gb 3060 71k purpose machine learning college purpose
You forget something, when you tried to make a benchmark you faced the same issue, you couldn't use the whole performance of the GPU/CPU when you used Windows or WSL, and you achieved that when moved to Linux. please do it and tell me the results.
I love your videos.
WSL uses hyperv, there is no way around it.
MSI laptops are always noisy. If you need a powerful and less noisy windows laptop then Lenovo Legion 9i is a better choice
Haven't tried that one yet. Thanks
Finally MLX 🔥
Part of Apple's long game here is to absolutely dominate the mobile market in every way, and part of that domination is going to require robust machine learning capabilities and speed even for small models that are better suited for mobile uses of machine learning applications. They make their machines able to run small models insanely fast and that's where they're going to have a huge edge in the future
Google has a better transcriber in their API Vertex called USM tbh
Then why is the RUclips one still trash?
Two MacBook Pros died after 14 months. If I could buy. a new one every year, that would be just GREAT.
8GB of RAM is not enough but Apple figures that profits are better than selling a computer with enough memory to do the job. "Job" - does that remind you of someone??? Too bad we are Cooked.
I want a MacBook that has Apple silicon soooooo badddd 😭😭😭😭😭
What’s your use case? The battery life on even the M1/M2 chips is phenomenal, the M3 chip mostly just adds performance. If you’re using it for light tasks, save some $$$ and get an M1 or M2 series chip
@@markclayton8977 I’m a photographer I use adobe PS adobe Lr and LRC plus Xcode for my camera app I’m working on and I need to connect to two displays
🥵
Red eyes! Check if this is normal
I have no idea why the hack I am watching this now, but everything you say sounds cool. :))
Ps: no idea how to code at all, wish I could.
Insanely fast model is actually way faster in 4090
Doesn't matter if there's like zero software to use on silicon, it just is thar devs always do windows, only billionaire devs support mac, or browser game devs
First again from X to youtube.
Eveyday i get more impressed with the apple chips and unified memory 😊
2nd
too fast
But … 8 GB on MacOS is like 16 GB on Windows 🤔
Soooo, the real title of this video should be MLX extremely poorly optimized for CUDA cores.
MLX does not run on PC’s and there are no CUDA cores on Apple Silicon 🤷♂
Use a simple RTX 4060 laptop without power plugged in.
too bad the proprietary silicon is anchored to the pos company which is apple, I don't want to spend 800 dollars on an extra 64gb of memory.
Whisper isn't AI.. no true AI yet exists lol
Nvidia seriously needs to up the game with VRAM capacity. But why would they, when their competitors are as useless as Intel and AMD.
or apple
h1874 Apple chips have a lot of memory
@@divyanshbhutra5071Nvidia is working on ARM.
They'll release something more powerful (even without tight optimization) than what Apple can ever hope to achieve.
And kill off the h100 market? 😂😂😂😂😂
You’re so naive
@@utkarsh1874m2ultra has 192gb memory 😂😂😂😂 what are u on about?
Python has contributed more to carbon emissions than any other programming language.
lol
So many tech-bros on the net bragging about their AI on 4090's using Python, AS IF using Python is something about which to brag (when it comes to performance or efficiency.)
@@TheDanEdwardsWhich programming language would you say is the best??
8GB is like 16GB
Anyone interested in LLM will have the knowledge or experience to buy the right machine for their use. Almost no base config Mac buyer is going to really care about playing with LLM code.
But that is reflected to the machine/OS itself, and the GPU VRAM can't even run the whole OS.. actually can't even reach any data from the System RAM.. you need to copy the data from System RAM to the GPU RAM to let the GPU to use it.. so this 2 different things what you mixing together.. can the 16GB RTX 4090 run a full benchmark? (as it run the operation system too, not just part of the benchmark...)
with all of that machine, you should make GA xD as i need your m3 max moahahaha
French "l" sounds like "l". If it were double "ll" it would've sounded like "y".
darn. should have asked my wife before vid.
Hide your kids, hide your wife
Actually, if this is true then you didn't pick the best machine for competition cuz there are bazillion non apple laptops, the mathematical consequence is that one of them has to beat the mac, so clickbaiting us with this title is awful
What kind of mathematics is that ?
@@Intel101-pe1et statistics bro, plus probability
And apple dares to say 8GB are enough
not for ml. nobody said 8gb is enough for ml.
Microsoft said 640kB was enough.
But with windows laptops, you will spend only a few dollars on upgrading ram, but for apple you'll spend much more.
and you are stuck to a wall outlet
Well, you need to carefully specify the use case of the ram. In AI world, the only ram matters is the one on graphic card and it is not relatively cheaper to upgrade compare to mac
What is the purpose of this computing power? Do you need it every moment of your day? And if you don't have it, is it a serious issue? I have a Mac Mini M2 at home. I also have 2 Windows PCs. I have no affection for these two machines that heat up, blow, scream, make a loud noise to obtain the power you're talking about. Not to mention the poor quality of plastics that crack and the miserable battery life of the laptop (whose power supply is larger and heavier than my Mac Mini M2). The production of PCs should be stopped.
Apple beats the competition. As usual 🥱 #PCMasterRace? More like #PCObselete 😂 /j
PCs are still better in multiple ways that Macs aren’t. Far from irrelevant
@@crestofhonor2349 you're right. I was just trolling 💚
4090m is FAR superior
Did you watch the video
Can I have your cheapest mac air m1 please? 😍