i blame these developments on fat old white men - they will be refurbs in 4-5 yrs - it is going to take 5-10 yrs for bigger more significant breakthroughs but they will happen and make this current paradigm look petty - more than likely #tco #roi #cots #refurb long tail #release eng
the old days: people use ten floppies to store their OS early 2000: people run their os on hard drives 21 centry: people put their os on RAM the year of 2023: people run their os on L3 cache
Может какие нибудь програмисты старой школы сделают оптимизации чего то безумного на asm под эту мощь. Вроде тех энтузиастов, кто писал собственные OS на asm. Не помню название OS
This is the market where I knew 3D cache was going to be beneficial, more so than a PC, but as is shown it depends on use case, which is why stacked cache isn't going to be something that ends up on every IC AMD makes as people I've debated with suggested. The problem of this tech is no just the stacking technology, it's also creating a flat surface on the top of the entire package, which means having to add material over the core die, and I assume that material is designed to conduct heat as quickly as possible since heat now has to travel further before a heat exchanger (the cooler) can wick the heat away. Exciting times though because stacking technology along with interconnect technology adds a lot of tools chipmakers can use, and I expect that the next 5 years is going to see that performance curve for compute power take another good jump. THANKS AMD! Intel wasn't getting us anywhere quickly. AMD has pushed the world of compute about 15 years, or when Zen 4 and Zen 5 come out, over 6 years, starting with Zen 2, and if you want to add "the core wars", where Intel couldn't ignore high core count computing anymore, then going back to the beginning of Zen. I would say Intel brought big-little to X86-64, but to me that STILL seems like a response to power consumption more than anything else. I KNOW that for server it's going to become more important, but for desktop, not so much. And the reason it will be more important for server is very high core count computing. With stacked cache it opens the door even more for CPUs pushing a couple hundred cores. I think the bigger thing Intel has brought in is accelerators. How this plays out is pretty obvious. It will be interesting to see how AMD responds to this with their recent purchase of Xilinx. Memory bandwidth is still one more issue for high core count server CPUs, but that's going to get there soon.
1:40 Normally, I get suspicious when I hear something like, "we finished dinner, went back to his hotel room, and filmed a quickie." ;) Awesome video! Crazy mico-tech is becoming more common these days, I can't wait to see what our future holds!
Imagine having to tell people you make videos that have demographics of like 97%+ adult males in places where people do not understand what servers and switches are.
If you want to see something that's relatively crazy from the HPC world/space -- there are actually applications that use a hybrid MPI/OpenMP software architecture where going in between nodes, it uses MPI, but within a node, it uses OpenMP for the parallelisation. The purpose of this is to reduce the MPI overhead, which really starts to kick in when you have 2^7 (128 cores) and above because of the number of pair-wise connections you have to make with said MPI. So imagine that you are running a vehicle crash simulation or a computational fluid dynamics simulation where it thinks that it has 1.5 GB of "RAM", when really, that's L3 cache; which would be just crazy. And then, to be able to cache that much data - that's just insanity!!!
Patrick gets more and more hyped about self-introduction with each video. I love that start. It's like one of those catchy tunes that gets and stays stuck in one's head forever.
Hey Patrick and crew, fun video, thanks. The kernel mitigations for the various microarchitectural flaws are pretty expensive; I think some of the fixes dump processor caches on syscall entry and exit. I wonder if turning off the mitigations would show more benefit to this crazy cache. Since RUclips eats many of my comments with links I'll reply with a link and hope it works.
IF workloads not doing that much context switch, it won't affect the performance that much. Most of mitigations do cache line flush only when context switch happens.
@@sfalpha yeah, perhaps the math-heavy benchmarks won't show much difference. But some of those benchmarks are very syscall heavy. Check vmstat 1 output when running your favorite benchmarks to see just how many context switches there are.
Ha, yeah, it looks like RUclips ate my comment with the new Linux kernel command line. Search for "make Linux fast again" and add the usual commercial tld.
Short Version : It depends on your Workload. I bet that some Software Companys will optimize their Software for it , you can archive relativly big Performance gains if your Software is Cache intensive and you adjust the Software Size to the aviable Cache Size so that the Software basically can live in the Cache .
I would really be interested in Java and .NET benchmarks. That giant amount of cache will help a lot with the pointer heavy nature of garbage collected languages (I think)
No performance improvement on NGINX is no surprise. They mostly do network things and moving data around. But for Application Servers running Java, PHP, Python, Node, etc. It will have some improvements on larger L3 cache if old JIT compiled code already saturated L3 before upgrade to Milan-X.
Fpga development is the most stressful programming I've done. I used vhdl in 2002 for my college final project to design usb hub controller. Each compilation took 10 minutes then simulation took 5 minutes, which meant in 1 hour i could only do 4 editings😅
@@Darkknight512 but my friends whose final projects were regular c based software compiled their codes in seconds. That comparison was stressful considering i had to graduate before my scholarship ended😂
I think xilinx actually has an answer record saying if you probably shouldn't use too many threads for implantation as that would just make cores fight for memory bandwidth.
there are a number of indie games that I can now store entirely in L3 Cache. who even needs local storage when you have all the ram and Cache in the world
@@NVMDSTEvil bad, frequency is too low for such low core count (64). GPUs are a lot more parallel so you have to compensate with GHz, which the thermals from stacking prevent you from achieving.
Great stuff guys! I would only disagree with low power optimized. If you can collapse for your scale-out workload from 10 servers to 5 with say a bit more hungry Milan-X CPUs then in a big picture, a lot of power is saved!
the only way to get 768MB of L3 cache is to have all 8 CCD chiplets populated, which means that the 7373X only has 2 cores active on each CCD. i wonder if they are able to bin the CCDs before they add the vcache on them. this may also explain why there is not an 8 core X part, it would only have 1 active core per CCD (this would likely cause too much of a thermal imbalance on the CCD to use with vcache)
ERRMAHGERRDD its Wendawg! Ive been wondering when my 2 fav tech youtubers would get together on something! Although if there was a video just shooting the shit that also had Dr. Cutress involved that would be pretty sweet too.
First of all, I like the content both of you usually present on the internet. Thanks for that. But in case of Milan-X, both of you totally missed the point :) The big thing with Milan-X is, that is finally implements one of the "holy grails" in silicon manufacturing: stacked dies with through silicon interconnects (TSV for short). TSV is just one of these breakthrough technologies, that changed manufacturing significantly, just like CMOS, non-silicon interconnects, EUV lithography, etc. With TSVs, you can finally "split"a chip onto multiple chips while maintaining full-speed interconnects at on-silicon electrical loads. In essence, they (AMD) just managed to implement a way of massively increasing silicon real-estate without making bigger chips. Milan-X is just a technology demonstrator. Nothing of the EPYC architecure has changed, they just used the tech to slap on more L3 cache. Which makes sense, if you try to figure out, if you got all kinks of the package assembly process right. What can be expected in the future, is a radically re-architected version of EPYC: remove all L3-cache from the processor chiplets and use the gained real estate to significantly increase L1/L2 cache sizes. L3 is then implemented in chip stacks on to of the processor chiplets, or - valid in EPYCs arch - stack the entire L3 cache on top of the IO die. L2 and especially L1 caches are very costly in terms of silicon real estate. Which is why we still stick to ancient L1 cache sizes since decades. Somwhat the same applies to L2 caches. With TSVs, this is going to change radically and will significantly affect performance of upcoming chips. I expect a few more years before this will be put in effect in full swing, but when done, we will get really fast CPUs. This is what makes Milan-X so significant. Milan-X will go down in history as one of these landmark chips, that really changed the industry. IMHO, that should have been pointed out in the video.
I'd love to se a comparison between the "Normal" Rome 7302/7302P, Milan 7313/7313P and the Milan-X 7373X. Single socket, 16core servers have been very popular with service providers and others that need to synchronize their VMware licensing with the licensing models for the virtual workloads (Microsoft licensing, 16 cores total in a physical server is cost-optimal for MS products). Also, the 7373X is the highest Cache pr. core configuration. As for upgrades, many of the servers running Rome 7302/7302P CPU's today can be easilly upgraded to f.eks. 7373X by a simple CPU swap.
70% cpu power usage is for data movement, less movement, efficient caches way and size are better than recall it from ext memory like on ram or storage latency hit very hard on performance
something i'm thinking about: as cache gets larger, performance improves, but only really once a process has been running long enough for its most frequently accessed data to have been accessed and cached. My assumption is that from a cold start, this CPU's performance looks exactly the same as an equivalent CPU with a more normal amount of cache, both start out really slow, and then the speed starts rapidly increasing as more and more memory accesses turn into cache hits, until it hits some steady state when the cache is "full". The difference just being that that ramping period continues for longer with Milan-X, leading to a higher steady-state IPC at the expense of taking longer to hit that steady state. this is just my intuition smashing together the facts that the memory bus is the same and the fact that the cache is larger, therefore time for the cache to fill is longer. my big question is: does this mean context switches between processes are gonna get even more expensive? or am i overestimating how disruptive context switching is to the cache?
to be clear, i'm not asking if a much larger cache would increase the absolute cost of context switches, it seems pretty clear that the absolute worst-case scenario would be exactly equivalent performance. i'm just wondering if the relative improvement from increasing the cache size will be more sensitive to context switches (say, a process performing amazingly with a fully primed cache, but the first few percent of that process' time slots plus the moments after running a syscall are spent running with a high cache miss rate until it gets back in the groove)
A byte of memory is loaded once the first read or first write is done to that byte. Funny thing is that since CPU memory is chunked in "cachelines", the 63 bytes right next to it are also loaded. Meaning a lot of stuff will probably be already in cache before its accessed first. But essentially you're right. Cache is loaded on-demand and only after the first load can speedups of cache be retrieved, unless the programmer does some preloading but that's a whole other can of worms.
So use case, engineering and some DB. For DB that's optimized to have data stored in RAM I could see this helping. How much? You still have to sync the data between RAM and storage. It will be interesting to see.
@@mzamroni thats plenty to run my website both the webserver and php excluding the database. I can just imagine the amount of php performance i could get from this without needing any RAM. Just debian, lsphp uses 80MB of ram.
Im so disappointed that AMD didn't give us 3D v-cache with the 5000 series threadripper pro after such a long delay. Gonna stick with my 3000 threadrippper pro until v-cache finally arrives.
@@ServeTheHomeVideo The refresh cycle at work is not great. 2x E5-4617 v1 in a 4 socket server (for some reason) and DDR 1333. I started last year and when I asked for new VMs, they had to be put on a different machine that was new enough to support ESXI 6.7. 😱 I managed to budget for a replacement for this year and want to ensure I have enough overhead for 5 year.
Do you have the configuration for the Linux kernel that you use for your benchmarks somewhere, maybe on the main site? I'd like to run a couple comparisons based on your data =)
I was under the impression that with the extended cache pool, the processor would run hotter at lower frequencies because each thread is... parked(?) less often waiting on code to come from RAM/ROM. meaning it cant hit the higher speeds without better cooling
@@ServeTheHomeVideo yeh we know, but what as a consumer we really needs is 128 PCI lanes min, aka 7 slots with 16 PCIe lanes, with GPU that actually install in 1 slot ( reason is VFIO and NVME Raid Cards as well as USB addons ( i wish they would give each USB slot a seperate pci id to pass USB slot 1 per 1 through to VM))
How soon till they adapt a newer standard for transfer from memory to the cpu like maybe instead of a trace have a link that can be upgraded over time.
@@ServeTheHomeVideo thank you I definitely will I keep hearing and seeing how these traces are becoming a bottleneck like signaling of PCIe 3-4 was alot of work I can only imagine PCIe Gen 6. Going to be a interesting few years. Also what do you think the availability of even the lowest end Epyc 12c will be thinking of building a proper Server for my needs and keeping my DS1621+ as a failover once I build out a nice strong 10GB Network.
@@ServeTheHomeVideo thank you I'll def look into what's available as it's time to make a real server as great as a Asus x570 i-gaming is I'd rather have a dedicated solution. Thank you for all this content serving it up in a way even I can understand 👍
@@ServeTheHomeVideo considering all the extra 7nm wafer used, NRE, assembly complexity, and yield loss. I imagine their gross margin per chip is probably wayy lower. Seems to be more of a get it out there in everyones racks price.
I remember a time (year is vague) 1.5GB was a common max amount of RAM a server would support. It's crazy to consider that's hardly useable on a phone these days
cpu vendors coming up with crazy concepts in order to compete with GPU. I wonder whether it's a lost cause from the get-go. Some of those benchmarks would be 10x faster on gpu, anyway
BIG BADA-BOOM!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! oh no... wait... MULTI-PASS!!!!! (the BEST part of the movie is the very beginning when Corban Dalls (Bruce Willis) gets a 'knock' on his door and looks out the peep hole to see no one there, but when he opens it there's a guy with a a gun and a picture of the empty stuck to his head band, so that when Corban looked out the peep hole he just saw the picture...... BUAHAAAA!!!! That whole scene is HILARIOUS!!!!! - "TAKE IT!!1 I DON'T NEED IT!!!" HAHAHAHAHA!!!!!
Isn't stacking another die on top of a die that already has trouble getting rid of it's heat problematic? What keeps the CPU dies from cooking themselves with SRAM dies on top of them? This is the same question I have with zen3 3d products coming any day now. Speaking of which, I've recently given thought to a Threadripper pro or epyc workstation because of the pcie lanes that both Intel and amd seem dead set against providing on Ryzen and amd seems keen to murder the consumer hedt parts which drives me crazy because there just isn't enough pcie for adding several nvme adapters, dual or quad 4k60 video capture cards, multiple GPUs for cuda acceleration, 10g networking (which SHOULD be onboard everything now and very much isn't), etc. Threadripper 3970x on zen2 looks stupid in terms of compute power next to a 5959x Ryzen part, even the recently announced Ryzen pro stuff is going to be zen3 based instead of waiting and releasing a zen5 part at the same time as the Ryzen parts drop in q4. Ugh. The CPU market is driving me batty. 16/0/4 plus an nvme or 3 (2 of which are shared) or 8/8/4 lanes just aren't enough for high end video production and nvme storage needs. And don't get me started on ryzen's lack of ecc support smh.
Holding out on us Patrick stringing out the serial; late market, SRAM bit charge not scaling v logic is not analog that are circuit bridges of various sorts. Leading, lagging, balanced, various amplified frequency emissions that sort of thing. Well, 8C 3D late market Win 10 system CPU is 5800X3D at $449 is a good price on AMD to channel cost : price / margin the component price from TSMC to AMD is $210 + x1.55 for AMD hi volume customers and x2 to lower volume resellers. Wendell its spatial. x:y:z multi planes, mathematical determination x:y:z to calculate and compare to deliver a result is one application space then also data bases, on line transactional processing spreadsheets, modeling, simulation etc. And if you want 256 MB over 96 MB L3 get Milan 73F3 for $1655 is also a good price on AMD margin sacrifice for Epyc 73F3. Price analysis, yea I'm waiting you know how to get in touch with me. Windell, deep knowledge, come on OLTP and Open MP is not deep knowledge. Subject knowledge to work on it Wendell and Patrick create the benchmark standards and quit talking around or other creative and less than creative engineers will just move in on you and take your ideas that you both did not implement on. I can do that because I state ARs for engineers to do that so get to work! mb
What is the actual power consumption of the IO die and why is it so much higher than the *checks notes* 6.6 watts AMD states it uses? Please let them know that they can get from 6.6 to 5 watts if they triple the cost of the IO die manufacturing process.
When a CPU looks large in comparison to Patrick's presence... you know it's serious (and crazy expensive) business.
Ha! Thanks Jeff.
I'm looking forward to you seeing Wendell in one of your videos.
Jeff, is that called a balanced load? Physical size versus presence... :D
Server cpu indeed gives most profit per transistor than desktop cpu and more over desktop gpu.
No wonder that rdna2 availability is very scarce
i blame these developments on fat old white men - they will be refurbs in 4-5 yrs - it is going to take 5-10 yrs for bigger more significant breakthroughs but they will happen and make this current paradigm look petty - more than likely #tco #roi #cots #refurb long tail #release eng
These CPUs have as much cores as Opterons had bits and as much cache as Opterons had RAM. How time flies.
Hell, Zen3 micro-op cache is larger than the RAM on my first micro 😆😆😁
Oh my god he is always so hyped making those videos. Love the energy!
How can you not be with crazy CPUs like this?!?!
@@ServeTheHomeVideo they are VERY omg these days ....
the old days: people use ten floppies to store their OS
early 2000: people run their os on hard drives
21 centry: people put their os on RAM
the year of 2023: people run their os on L3 cache
А что дальше? Half life alyx в кеше процессора мобильного шлема?))
Думаю авто перевод справиться.
Может какие нибудь програмисты старой школы сделают оптимизации чего то безумного на asm под эту мощь.
Вроде тех энтузиастов, кто писал собственные OS на asm. Не помню название OS
NICE!!!! I watch @Level1Techs Wendel as well and been watching him since he was part of another channel... Nice to see you have him on as a guest.
we don't talk about the other channel 🤫
Wendell is making the rounds today!
First he's a Potato, now he got Served.
@@d00dEEE Level1Home
@@JeffGeerling Potato1Home
He probably got a lot of cache.
This is the market where I knew 3D cache was going to be beneficial, more so than a PC, but as is shown it depends on use case, which is why stacked cache isn't going to be something that ends up on every IC AMD makes as people I've debated with suggested.
The problem of this tech is no just the stacking technology, it's also creating a flat surface on the top of the entire package, which means having to add material over the core die, and I assume that material is designed to conduct heat as quickly as possible since heat now has to travel further before a heat exchanger (the cooler) can wick the heat away.
Exciting times though because stacking technology along with interconnect technology adds a lot of tools chipmakers can use, and I expect that the next 5 years is going to see that performance curve for compute power take another good jump.
THANKS AMD! Intel wasn't getting us anywhere quickly. AMD has pushed the world of compute about 15 years, or when Zen 4 and Zen 5 come out, over 6 years, starting with Zen 2, and if you want to add "the core wars", where Intel couldn't ignore high core count computing anymore, then going back to the beginning of Zen.
I would say Intel brought big-little to X86-64, but to me that STILL seems like a response to power consumption more than anything else. I KNOW that for server it's going to become more important, but for desktop, not so much. And the reason it will be more important for server is very high core count computing. With stacked cache it opens the door even more for CPUs pushing a couple hundred cores. I think the bigger thing Intel has brought in is accelerators. How this plays out is pretty obvious. It will be interesting to see how AMD responds to this with their recent purchase of Xilinx. Memory bandwidth is still one more issue for high core count server CPUs, but that's going to get there soon.
1:40 Normally, I get suspicious when I hear something like, "we finished dinner, went back to his hotel room, and filmed a quickie." ;)
Awesome video! Crazy mico-tech is becoming more common these days, I can't wait to see what our future holds!
Imagine having to tell people you make videos that have demographics of like 97%+ adult males in places where people do not understand what servers and switches are.
If you want to see something that's relatively crazy from the HPC world/space -- there are actually applications that use a hybrid MPI/OpenMP software architecture where going in between nodes, it uses MPI, but within a node, it uses OpenMP for the parallelisation.
The purpose of this is to reduce the MPI overhead, which really starts to kick in when you have 2^7 (128 cores) and above because of the number of pair-wise connections you have to make with said MPI.
So imagine that you are running a vehicle crash simulation or a computational fluid dynamics simulation where it thinks that it has 1.5 GB of "RAM", when really, that's L3 cache; which would be just crazy.
And then, to be able to cache that much data - that's just insanity!!!
Patrick and wendell are like two peas in a nerdalicious pod. You are both are great.
It was fun just hanging out at AMD last week with him. The last time we had a meal together was in Taipei in like 2019.
Patrick gets more and more hyped about self-introduction with each video. I love that start. It's like one of those catchy tunes that gets and stays stuck in one's head forever.
I am one of the few people that gets genuinely excited about servers
Legend, big hug from Australia, great work :)
Nerds! love to see you guys having fun, thanks, stay safe and strong ;)
Are they the fire, water, earth and wind stones from '5th Element' ?
Leeloo being the 5th?
Multipass....
Do I win the 3080?
You got it!
@@ServeTheHomeVideo Got what? The 3080 or the Multipass?
@@d00dEEE yes
A good video. I'll never use these CPU's but it's nice to see the details. These are crazy powerful.
Hey Patrick and crew, fun video, thanks.
The kernel mitigations for the various microarchitectural flaws are pretty expensive; I think some of the fixes dump processor caches on syscall entry and exit. I wonder if turning off the mitigations would show more benefit to this crazy cache. Since RUclips eats many of my comments with links I'll reply with a link and hope it works.
IF workloads not doing that much context switch, it won't affect the performance that much. Most of mitigations do cache line flush only when context switch happens.
@@sfalpha yeah, perhaps the math-heavy benchmarks won't show much difference. But some of those benchmarks are very syscall heavy. Check vmstat 1 output when running your favorite benchmarks to see just how many context switches there are.
Ha, yeah, it looks like RUclips ate my comment with the new Linux kernel command line. Search for "make Linux fast again" and add the usual commercial tld.
One of my favorite graphics cards of all time the 8800gtx had 768 MB of vram lol.
Hey this is Patrick from STH, watch me wave around 17,000$ like it aint shit.
🤭🤣
Love the 5th element blocks in the background
Nailed it!
Short Version : It depends on your Workload. I bet that some Software Companys will optimize their Software for it , you can archive relativly big Performance gains if your Software is Cache intensive and you adjust the Software Size to the aviable Cache Size so that the Software basically can live in the Cache .
I would really be interested in Java and .NET benchmarks. That giant amount of cache will help a lot with the pointer heavy nature of garbage collected languages (I think)
Yay Wendell
WendOS ?
Play a game of telephone: WendOS => WindOS => Windows... NOOOOOO!
No performance improvement on NGINX is no surprise. They mostly do network things and moving data around.
But for Application Servers running Java, PHP, Python, Node, etc. It will have some improvements on larger L3 cache if old JIT compiled code already saturated L3 before upgrade to Milan-X.
I like big caches and I cannot lie.
This man talks with more enthusiasm about these things than I do about juicy burgers.
Patrick casually flashing over $16,000 of silicon bling in the intro XD
That's like $10 per MB of cache
Wow! L3 is soooo big! That means more power and it will get hot!
Still 280W TDP.
Still 280 tdp same as regular vanilla milan without 3d vcache.
I love big caches and I cannot lie
"Hey we should film something, so we went back to his hotel room" :D :D :D
Nice Fifth Element props in the background.
You got it!
As an FPGA developer, I must say, this is the first time I have seen RTL development tools in bunch of CPU benchmarks.
Fpga development is the most stressful programming I've done.
I used vhdl in 2002 for my college final project to design usb hub controller.
Each compilation took 10 minutes then simulation took 5 minutes, which meant in 1 hour i could only do 4 editings😅
@@mzamroni Thats not very long actually. A compile for me at work is 2-7 hours, and we don't even use the huge chips. These are medium sized FPGAs.
@@Darkknight512 but my friends whose final projects were regular c based software compiled their codes in seconds.
That comparison was stressful considering i had to graduate before my scholarship ended😂
@@mzamroni but fun fact: if you can do FPGA development, then you're basically a necromancer to older gamer's and nerds :)
I think xilinx actually has an answer record saying if you probably shouldn't use too many threads for implantation as that would just make cores fight for memory bandwidth.
there are a number of indie games that I can now store entirely in L3 Cache. who even needs local storage when you have all the ram and Cache in the world
Crazy to think about! I can certainly remember days when having 1.5GB of RAM was almost impossible.
@@ServeTheHomeVideo Hmm.. wonder how running some games in software rendering mode would perform with these.
@@NVMDSTEvil bad, frequency is too low for such low core count (64). GPUs are a lot more parallel so you have to compensate with GHz, which the thermals from stacking prevent you from achieving.
It's coming!
Great stuff guys! I would only disagree with low power optimized. If you can collapse for your scale-out workload from 10 servers to 5 with say a bit more hungry Milan-X CPUs then in a big picture, a lot of power is saved!
the only way to get 768MB of L3 cache is to have all 8 CCD chiplets populated, which means that the 7373X only has 2 cores active on each CCD. i wonder if they are able to bin the CCDs before they add the vcache on them. this may also explain why there is not an 8 core X part, it would only have 1 active core per CCD (this would likely cause too much of a thermal imbalance on the CCD to use with vcache)
L1Tech baby! Woooooooo!
ERRMAHGERRDD its Wendawg! Ive been wondering when my 2 fav tech youtubers would get together on something! Although if there was a video just shooting the shit that also had Dr. Cutress involved that would be pretty sweet too.
Ian stayed with me in December when he was in Austin. We just forgot to shoot something. It will happen
Wendell is making the rounds today!
🐻
First of all, I like the content both of you usually present on the internet. Thanks for that. But in case of Milan-X, both of you totally missed the point :)
The big thing with Milan-X is, that is finally implements one of the "holy grails" in silicon manufacturing: stacked dies with through silicon interconnects (TSV for short). TSV is just one of these breakthrough technologies, that changed manufacturing significantly, just like CMOS, non-silicon interconnects, EUV lithography, etc. With TSVs, you can finally "split"a chip onto multiple chips while maintaining full-speed interconnects at on-silicon electrical loads. In essence, they (AMD) just managed to implement a way of massively increasing silicon real-estate without making bigger chips.
Milan-X is just a technology demonstrator. Nothing of the EPYC architecure has changed, they just used the tech to slap on more L3 cache. Which makes sense, if you try to figure out, if you got all kinks of the package assembly process right.
What can be expected in the future, is a radically re-architected version of EPYC: remove all L3-cache from the processor chiplets and use the gained real estate to significantly increase L1/L2 cache sizes. L3 is then implemented in chip stacks on to of the processor chiplets, or - valid in EPYCs arch - stack the entire L3 cache on top of the IO die. L2 and especially L1 caches are very costly in terms of silicon real estate. Which is why we still stick to ancient L1 cache sizes since decades. Somwhat the same applies to L2 caches. With TSVs, this is going to change radically and will significantly affect performance of upcoming chips. I expect a few more years before this will be put in effect in full swing, but when done, we will get really fast CPUs.
This is what makes Milan-X so significant. Milan-X will go down in history as one of these landmark chips, that really changed the industry. IMHO, that should have been pointed out in the video.
Wendell !
I'd love to se a comparison between the "Normal" Rome 7302/7302P, Milan 7313/7313P and the Milan-X 7373X. Single socket, 16core servers have been very popular with service providers and others that need to synchronize their VMware licensing with the licensing models for the virtual workloads (Microsoft licensing, 16 cores total in a physical server is cost-optimal for MS products). Also, the 7373X is the highest Cache pr. core configuration. As for upgrades, many of the servers running Rome 7302/7302P CPU's today can be easilly upgraded to f.eks. 7373X by a simple CPU swap.
yup! just flash those in front of my R7 3700x eyes! where's Ryker! just work me over!! super presentation! big league!
Leeluu Dallas MultiPass!
Great reference.
70% cpu power usage is for data movement, less movement, efficient caches way and size are better than recall it from ext memory like on ram or storage latency hit very hard on performance
Crazy to think you could run a whole server on 500w psu's 13 years ago now it's perhaps 250w per cpu
A modern top-SKU server even without accelerators/ storage we now budget 1kW for.
@Ray Johnson What the hell do you have in your desktop? Do you have like 4 3090s?
something i'm thinking about: as cache gets larger, performance improves, but only really once a process has been running long enough for its most frequently accessed data to have been accessed and cached. My assumption is that from a cold start, this CPU's performance looks exactly the same as an equivalent CPU with a more normal amount of cache, both start out really slow, and then the speed starts rapidly increasing as more and more memory accesses turn into cache hits, until it hits some steady state when the cache is "full". The difference just being that that ramping period continues for longer with Milan-X, leading to a higher steady-state IPC at the expense of taking longer to hit that steady state.
this is just my intuition smashing together the facts that the memory bus is the same and the fact that the cache is larger, therefore time for the cache to fill is longer. my big question is: does this mean context switches between processes are gonna get even more expensive? or am i overestimating how disruptive context switching is to the cache?
to be clear, i'm not asking if a much larger cache would increase the absolute cost of context switches, it seems pretty clear that the absolute worst-case scenario would be exactly equivalent performance. i'm just wondering if the relative improvement from increasing the cache size will be more sensitive to context switches (say, a process performing amazingly with a fully primed cache, but the first few percent of that process' time slots plus the moments after running a syscall are spent running with a high cache miss rate until it gets back in the groove)
A byte of memory is loaded once the first read or first write is done to that byte. Funny thing is that since CPU memory is chunked in "cachelines", the 63 bytes right next to it are also loaded. Meaning a lot of stuff will probably be already in cache before its accessed first. But essentially you're right. Cache is loaded on-demand and only after the first load can speedups of cache be retrieved, unless the programmer does some preloading but that's a whole other can of worms.
So use case, engineering and some DB. For DB that's optimized to have data stored in RAM I could see this helping. How much? You still have to sync the data between RAM and storage. It will be interesting to see.
That's insane. When can we have enough cache to not need RAM on consumer PCs lol.
It's only 768 MB.
We still need ram.
It's very large for cpu caches but small for ram.
Low end pcs have 8 GB RAM
That’s basically where Apple is (kinda) headed with the M1… ;)
@@mzamroni thats plenty to run my website both the webserver and php excluding the database. I can just imagine the amount of php performance i could get from this without needing any RAM. Just debian, lsphp uses 80MB of ram.
Im so disappointed that AMD didn't give us 3D v-cache with the 5000 series threadripper pro after such a long delay. Gonna stick with my 3000 threadrippper pro until v-cache finally arrives.
Supermicro Epyc atx motherboard is only $700.
I will go with epyc rather than thread ripper pro 5000
Hi Wendell
With that much L3 it would be a neat if AMD allowed them to boot without RAM. It would be handy for troubleshooting.
You could do that on the old Xeon Phi's actually.
@@ServeTheHomeVideo That's neat. I didn't know that.
Cash per core vs. cache per core
I see Wendel,I click.
I hope we see GB cache in desktop CPUs soon.
I've been lusting all day. Might put some 7373X in my server purchase. Why not?
It may also come down to availability for these. We have to wait and see how that pans out.
Yeah. I don't need it. I'll put it in my request for bid as a nice to have but not required.
@@ServeTheHomeVideo The refresh cycle at work is not great. 2x E5-4617 v1 in a 4 socket server (for some reason) and DDR 1333. I started last year and when I asked for new VMs, they had to be put on a different machine that was new enough to support ESXI 6.7. 😱 I managed to budget for a replacement for this year and want to ensure I have enough overhead for 5 year.
I don't need it, I can't afford it... but I want it.
I guess this would have fantastic impact interpreted languages like PHP, Python.
IBM had gigabyte cache on their Z some years back.
Presenting: one Wendel (Patrick for scale, colorized 2022)
Do you have the configuration for the Linux kernel that you use for your benchmarks somewhere, maybe on the main site? I'd like to run a couple comparisons based on your data =)
The specintrate of the 7773X is almost identical to the 7763. Looks like the larger L3 compensates for the lower core frequency. Just an observation.
Larger L3 surely make the CPU work a lot harder.
Tokamak twisted plasma flows and realtime adjustment of the magnetic fields? Or is it more of a clock speed function?
I was under the impression that with the extended cache pool, the processor would run hotter at lower frequencies because each thread is... parked(?) less often waiting on code to come from RAM/ROM. meaning it cant hit the higher speeds without better cooling
Power as well.
So when are we going to get this 1.5GB of cache on consumer CPU aka Ryzen and ThreadRipper where we need this nor than companies!
Probably some time before Ryzen gets 1GB.
@@ServeTheHomeVideo yeh we know, but what as a consumer we really needs is 128 PCI lanes min, aka 7 slots with 16 PCIe lanes, with GPU that actually install in 1 slot ( reason is VFIO and NVME Raid Cards as well as USB addons ( i wish they would give each USB slot a seperate pci id to pass USB slot 1 per 1 through to VM))
Very interesting
Awesome video and very throrough. Multipass!
Multipass!!!
For something like emulation this cache can do wonders, imagine emulating ps3 and using the L3 cache as emulated SPU memory..
*"DOOM must on L3 cache be run"* - Cato The Elder"
Love the 4 elements, you're just missing Leeloo.
Excellent!
How soon till they adapt a newer standard for transfer from memory to the cpu like maybe instead of a trace have a link that can be upgraded over time.
Maybe thinking co-packaged optics? See my SC21 chip discussion with Raja Koduri for that.
@@ServeTheHomeVideo thank you I definitely will I keep hearing and seeing how these traces are becoming a bottleneck like signaling of PCIe 3-4 was alot of work I can only imagine PCIe Gen 6.
Going to be a interesting few years. Also what do you think the availability of even the lowest end Epyc 12c will be thinking of building a proper Server for my needs and keeping my DS1621+ as a failover once I build out a nice strong 10GB Network.
It is sometimes cheaper to just buy a full EPYC server than to build one. Getting Rome is also sometimes faster/ less expensive.
@@ServeTheHomeVideo thank you I'll def look into what's available as it's time to make a real server as great as a Asus x570 i-gaming is I'd rather have a dedicated solution.
Thank you for all this content serving it up in a way even I can understand 👍
Next week we will have an ASUS Alder Lake build that is going to be fun.
Wendel doesn't fly he travels via server rack. Silly mistakes
Seimens? We like Rockwell Automation.
8800 usd, only like a 1000$ premium over regular milan
Yes. It is very interesting from a product positioning perspective.
@@ServeTheHomeVideo considering all the extra 7nm wafer used, NRE, assembly complexity, and yield loss. I imagine their gross margin per chip is probably wayy lower. Seems to be more of a get it out there in everyones racks price.
Nice shirt.
This looks good in 2160p
I remember a time (year is vague) 1.5GB was a common max amount of RAM a server would support. It's crazy to consider that's hardly useable on a phone these days
It isn't "hardly usable", the software is just garbage.
@@Kobrar44 I wonder how much of it is simple DB engines needlessly locking data into RAM. That and excessive IO are why I dislike them.
@@Kobrar44 Manjaro runs well on my 2gb ram laptop
cpu vendors coming up with crazy concepts in order to compete with GPU. I wonder whether it's a lost cause from the get-go. Some of those benchmarks would be 10x faster on gpu, anyway
There are many software use cases that are not suitable for gpu.
Web server software (httpd, jsp, php, etc) can't run on gpu.
My first dual "core" pc (dual slot 1 pentium III) only had 2gb of ram lol
but will it store?
BIG BADA-BOOM!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
oh no... wait...
MULTI-PASS!!!!!
(the BEST part of the movie is the very beginning when Corban Dalls (Bruce Willis) gets a 'knock' on his door and looks out the peep hole to see no one there, but when he opens it there's a guy with a a gun and a picture of the empty stuck to his head band, so that when Corban looked out the peep hole he just saw the picture...... BUAHAAAA!!!! That whole scene is HILARIOUS!!!!! - "TAKE IT!!1 I DON'T NEED IT!!!" HAHAHAHAHA!!!!!
cash is important because it allows you to make payments for things even if the power is out.
AMD: "More cash good. Cache good too."
“Your room or my room”? As long as you two are happy. Wink wink nudge nudge.
I need to see your multipass, sir...
Excellent
Does that say $8800 for that cpu, holy sheetz
Isn't stacking another die on top of a die that already has trouble getting rid of it's heat problematic? What keeps the CPU dies from cooking themselves with SRAM dies on top of them? This is the same question I have with zen3 3d products coming any day now. Speaking of which, I've recently given thought to a Threadripper pro or epyc workstation because of the pcie lanes that both Intel and amd seem dead set against providing on Ryzen and amd seems keen to murder the consumer hedt parts which drives me crazy because there just isn't enough pcie for adding several nvme adapters, dual or quad 4k60 video capture cards, multiple GPUs for cuda acceleration, 10g networking (which SHOULD be onboard everything now and very much isn't), etc. Threadripper 3970x on zen2 looks stupid in terms of compute power next to a 5959x Ryzen part, even the recently announced Ryzen pro stuff is going to be zen3 based instead of waiting and releasing a zen5 part at the same time as the Ryzen parts drop in q4. Ugh. The CPU market is driving me batty. 16/0/4 plus an nvme or 3 (2 of which are shared) or 8/8/4 lanes just aren't enough for high end video production and nvme storage needs. And don't get me started on ryzen's lack of ecc support smh.
AMD is stacking the SRAM die above the cache portion of the CCD, not over the computer cores. Part of the reason is to manage heat.
Rome wasn't built in a day, but it was launched in one 😁
The easter egg is from Fifth element it's THE STONES. You should be protecting them.
Super!
Multipass!
Nice! 100%
@@ServeTheHomeVideo and here's me thinking it was a pun on multipathing logic for the Infinity Fabric :P
F****** Hell - I remember when 32 kilobyte was a big deal.
Чего он такой довольный всегда ?))
Wait why are the earth/fire/water/air stones from the 5th element in the thumbnail of this video? LOL
Better than a blue background :-)
The Elements! BZZZZZZZZZZ
Great job!
I am sure Intel will stack some Bitcoin on top of their CPUs.
but will it cash?
Holding out on us Patrick stringing out the serial; late market, SRAM bit charge not scaling v logic is not analog that are circuit bridges of various sorts. Leading, lagging, balanced, various amplified frequency emissions that sort of thing. Well, 8C 3D late market Win 10 system CPU is 5800X3D at $449 is a good price on AMD to channel cost : price / margin the component price from TSMC to AMD is $210 + x1.55 for AMD hi volume customers and x2 to lower volume resellers. Wendell its spatial. x:y:z multi planes, mathematical determination x:y:z to calculate and compare to deliver a result is one application space then also data bases, on line transactional processing spreadsheets, modeling, simulation etc. And if you want 256 MB over 96 MB L3 get Milan 73F3 for $1655 is also a good price on AMD margin sacrifice for Epyc 73F3. Price analysis, yea I'm waiting you know how to get in touch with me. Windell, deep knowledge, come on OLTP and Open MP is not deep knowledge. Subject knowledge to work on it Wendell and Patrick create the benchmark standards and quit talking around or other creative and less than creative engineers will just move in on you and take your ideas that you both did not implement on. I can do that because I state ARs for engineers to do that so get to work! mb
Right structured data v unstructured needs some processing first on the coprocessor setting up to feed the CPU. mb
Real like examples that's you Patrick and Wendell's job to take that AR what r u waiting for? mb
5th element
Great catch
Amd can reduce epyc power consumption if the io chiplet uses better manufacturing process.
Gf 12nm is ancient compared to tsmc 7nm.
Are they still using GF 12nm? I thought they moved to TSMC 10nm or 7nm for the IO dies
@@pieluver1234 wikipedia even says 12nm for desktop and 14 nm for server io chiplet
en.m.wikipedia.org/wiki/Zen_3
What is the actual power consumption of the IO die and why is it so much higher than the *checks notes* 6.6 watts AMD states it uses?
Please let them know that they can get from 6.6 to 5 watts if they triple the cost of the IO die manufacturing process.
if x86 fron end is by passed, hmmmm APLHA 2/3 of egenirs
Rome wasn't launched in a day.