Just when I thought my 4060 Ti was the crown jewel of my home server, Nvidia drops the A1000-a teensy AI beast making me wish AMD had more compact powerhouses with decent VRAM. Sure, I can squeeze in SDXL or a 13b llama, but no Flux? Painfully slow. AI... bleh. At least, after watching this video, my girlfriend now believes laughter is the best aphrodisiac!
My LLM experience with a 4070 showed that calculation speed is not an issue. Text generation is 10x faster than you can read. Only VRAM limits to what models I can run.
Everyone:- Wow, what a tiny Graphics card. Me:- Thats a Rtx 3050 6GB with 8 gb of memory along with reduced power limit. (Seriously. It has the same amount of cuda cores along with the same gen architecture)
Got the A2000 ADA and I haven't had the chance to run it's through it's paces but it's so good it's size. Good to see a vid on it Wendell! Have fun with it :D
I have seen people use these smaller cards with a gaming GPU and a Threadripper CPUs. For some reason they claimed it boosted performance in some applications like Da Vinci Resolve and similar, and maybe even Adobe products (Photoshop / Lightroom). Would love an explanation or links that explain this type of card and the benefits using them in combination with a normal gaming GPU and threadripper CPUs
Thanks for the upload, interesting to hear some options as I'm building a home AI/NAS server for Frigate and Ollama to support Home Assistant. While the power/heat/size is a bonus, my local retailer does the RTX 4060Ti for the same price - twice the CUDA and twice the tensor cores. At half the price it may be worth it for a home AI server but for now I'm planning to use a Coral TPU I already have to do the detection and hand off to a GPU to keep the GPU quiet and cool until it's needed.
@@filmsinviolet very early days for me so just some initial research and a plan to use some old hardware for a proof of concept. I haven't found any write up on this specific approach. The basic premise is that you can have only 1 device inferencing per camera stream so have the TPU on the low-res stream for detection and then trigger the GPU to do identification/recognition (maybe even face & license plate) by passing the object bounding box to the high-res feed. I'm still trying to get Frigate running on the old HW in the first place, if it works then I might spring for a RTX 4060, 33% more CUDA, 33% more (next gen) Tensor and 60% of the price!
if youre power or size restrained sure but otherwise even a new 4060 ti (8gb) has around 7 times the TOPS compared to this at the same price and you can upgrade to 16gb for around 100bucks extra
If I understood correctly: how did you fit the Nvidia A1000 into the Synology DS1821+ NAS? I'm asking because the GPU is x16 but the NAS only has a x8 slot if I saw/remember correctly. Thanks for the video!
@@danieloberhofer9035 Yes electrically, but the slot size might be the problem here. I mean that the GPU physically doesn't fit into the slot. The above mentioned riser cable could be the solution here but he didn't mentioned anything like that in the video (or I overheard that?) so that's why I'm asking :)
@@kenoalbatros You realize you can simply trim the plastic off the end of any PCIE slot with a razor in about 15 seconds, no? You can fit a 16x card in a 1x slot with in about as long as it took to type this message.
@@tim3172 Yes I did already realize that and it actually works! Did something like that a few years ago using a hot nail to burn away the plastic to fit a GTX 550 Ti and a GTX 970 into an old HP server! I don't really know how to do it with a razor as that seems really risky to me but well.. probably works to. But how does your answer relate to my question? This is nothing you should have to do when buying new hardware, at least in my opinion. As it probably voids the guarantee.
I bought one for my TrueNAS some months back but the Linux kernel they were using didn’t support it so I was unable to get it working and returned it. With the newest Electric Eel version I’m considering giving this a try again. I didn’t like the $400 price tag or the lack of AV1 encode. I ended up getting an Intel ARC Pro A40 which was half the price and included AV1 encode. I just want to use with Docker for handbrake, jellyfin, and maybe home assistant which I haven’t fiddled with yet.
@ No specific reason. The Sparkle A310 ECO is very similar at nearly half the price. A40 is longer, has different fan, 4 mini-DPs. A310 has two mini-DPs + HDMI, shorter, different fan and shroud which may or may not be quieter/noisier. I prefer reference cards and I’m not super price sensitive when making purchasing decisions. The $90 difference between A40 and A310 ECO was acceptable to me.
Looks like what I'd need power- and heat-wise to do a fully local voice based AI for very modest needs. Price and availability where I live are both problems, though.
Seems like new M4 mini could match your use case running ollama for HomeAssistant. A little more pricey but much easier to buy and integrate. Killer feature for HomeAssistant speech is that it consumes
@@SocialNetwooky LMGTFY. A RTX A1000 is $440 and a MacMini is $599 so less than 40% more but you get double the RAM and a whole computer. Plus you would easily save back the extra cost on power savings over its lifetime and it has enough VRAM to transcode HDR video while holding a full Ollama 7b model. A no brainer for the ascribed use case of home automation.
@@maxlyth true. although personally I'd reel back from entering the apple cult, but that's on another page :) Also, according to the nVidia Store, the A1000 is $365
I'm in the UK where the M4 Mini is £599 and cheapest A1000 is £439. Sadly i just spent £700+ on an MS01 and it is still a power pig drawing 35W+ at idle. Strapping an A1000 into it is going to take that over 60W (likely over 100W doing work) and cost almost double the M4 Mini. I might miss the 10Gb network but all the containers I use run great on Arm. I'd be interested to know how docker manages video ram and whether an Ollama model would actually run. If an M4 doesn't work it might help with my MS01 buyer's remorse.
I wish MDP wasn't such a terrible connector. I guess the ports on the card ports are OK but man the cables absolutely do not last. If you ever encounter monitor problem in the field and MDP is involved just swap the cable. Don't even think about just swap it out it will save you so many headaches.
C'mon you missed abit off - these are OpenGL cards and not DirectX. This card has also been superceded by the 'RTX A1000 ada' card though this card is still being sold direct from Nvidia and others. OpenGL is the primary graphics language for Autocad, Revit, Solidworks, etc for 3d rendering
How will a1000 work as an eGPU, interesting to use with an oculink. I would nice to have a setup like this when its needed. Also what is the tpu when its idea?
you can build ai rack for secutiry camera networking ai monitor your home for everything it sees..nvidia is leading the way for security industry market ....feel angry for amd to miss another opportunity in the market that i work for security IP cams for every business they use NVIDIA ai eye that watch their car lots and banking without guard presence
I have a Thinkstation M920Q Tiny, I'm interested in this, the A2000, an Ada equivalent or whatever including the single slot conversion. Just not in the price. IT STARTS AT 600€ HERE.
Why do they release another previous gen card, instead of making it current gen? This is basically an underclocked RTX 3050, while they could've made an underclocked RTX 4060.
For Cad application, even complex models will work just fine on this. I used to work with a QUADRO p4000, which has the same amount of Vram and less cuda cores, and I handles 4-5000 parts assemblies and very complex geometries with ease. I don't know how rendering will work with this, but when it's about modeling something, I think this is perfect
Form factor, low wattage and performance/watt are its only selling points. Used cards will be sought after at some point, but new cards make no sense for normal people with normal desktop PCs. Even the cheapest 3050 beats this thing in AI workloads.
The A2000 is pointless unless you need a low profile card. If you indeed do, then it's the only card of that size and form factor that'll offer you 12GB of ECC VRam. I have an A4000 (ampere) in my workstation, which I also game on occasionally. It is roughly equivalent in rasterization performance to a 3070. Sometimes a bit slower, sometimes a bit quicker, closer to the 3070Ti it's based on, depending on whether the game favors clockspeed or memory bandwidth/size. Mine is watercooled and shunt-modded to 180W, which makes it almost an equivalent to the 3070Ti in many games. If you don't need any Quadro-specific features, get the 4060. Better, much quieter cooling plus you'll get access to 4000series features like frame generation, which the 3000series cards are still excluded from.
You can get a 3060 for gaming cheaper with 12GB, and will run any game at 1080p at 100 FPS. You only need a 4060 if you want to "game in 4k" but that's just dumb to be honest.
I tried to ask a LEO AI. This is what I got in return. Based on the provided search results, it's challenging to provide an exact number of brain synapses fired during an average 8-hour sleep. However, we can extract relevant information to provide an estimate. One study mentioned in the search results used serial block-face scanning electron microscopy to reconstruct approximately 7,000 spines containing synapses in mouse motor and sensory cortices [10]. The study found that after 7 hours of sleep, the axon-spine interface (ASI), a structural measure of synaptic strength, decreased by approximately 18% compared to 7 hours of spontaneous wake at night or enforced wake during the day. Another study mentioned the importance of synaptic strength being regulated across the 24-hour sleep/wake cycle, with overall synaptic strength being the primary focus, rather than firing rate [8]. Firing rates may or may not follow changes in synaptic strength, depending on neuromodulatory tone and the balance between excitation and inhibition. Regarding human neuron firing rates, one study estimated figures in the 0.3-1.8 range, consistent with estimates from the neocortex [11]. However, this information does not directly address the number of synapses fired during sleep. The sleep cycle, as described in another study, involves periods of non-rapid eye movement (NREM) sleep and rapid eye movement (REM) sleep, with the brain going in and out of REM periods about 4-5 times during an 8-hour sleep [5]. Considering these findings, it's essential to note that the number of synapses fired during sleep is not a straightforward calculation. Synaptic strength and firing rates are complexly intertwined, and changes in synaptic strength may not directly translate to a specific number of synapses fired. However, we can make an educated estimate based on the 18% decrease in ASI observed in the mouse study. Assuming a similar decrease in synaptic strength during human sleep, and considering the estimated 0.3-1.8 firing rate range for human neurons, it's possible that a significant proportion of synapses are "fired" or activated during an 8-hour sleep period. A rough estimate could be that tens of thousands to hundreds of thousands of synapses are potentially "fired" or activated during an 8-hour sleep period, considering the complexity of brain networks and the dynamic changes in synaptic strength and firing rates. However, this estimate should be treated as a rough order-of-magnitude calculation, rather than a precise figure. **In summary:** While it's challenging to provide an exact number of brain synapses fired during an average 8-hour sleep, our analysis suggests tens of thousands to hundreds of thousands of synapses may be potentially "fired" or activated during this period, based on the complex interplay between synaptic strength and firing rates.
Finally a single-slot low-profile card with some actual oomph. Eyeing this to replace my RX6400 in my 1L mini PC... 👀 No AV1 encode is a shame and feels artificially imposed, though.
thats because its last gen hardware, which predates av1. honestly an rtx 2000e would probably blow it out of the water, if you could actually find one somehow, at a non scalp price.
Please, check out this real low end budget GPU for AI: RTX 4060ti 16GB. Uses same processor AD106 as RTX A3000 mobile; memory bandwidth: 288 GB/s; 22TFlops in fp16. Inference with model llama 3.2 3B GGUF (Ollama) is around 80 tokens/sec, thanks to good memory bandwidth. And that 16GB has room to run larger 12B GGUF models with 8k of context!
8Gb of Vram is the bare minimum to run small models like SD1.5 with very few extra tools and feel pretty slow to generate 512x512 images. It will be a pain while producing frustrating quality unless you're into making small cartoonish images. If you're on a budget go with a 4060Ti with 16Gb, it will be slower in calculation but at least you'll be able to run medium models like SDXL with a bunch of Loras and tools like Controlnet up to 1024x1024, and it will cost significantly less than this A1000. If you can wait it would be wiser to at least look forward the announcement of RTX5000 series to know what it will really be made of. Until then play with online providers to be sure it is your thing, cause if it is, as such activity can be pretty addictive, you'll very fast want at least 24Gb in your GPU and be frustrated by a mid-range 4070Ti... and it cost more to upgrade from there than to go right ahead to a X090. On the lowcost side the 5060Ti may very likely be worth waiting than go for a 4060Ti, and on the high end side, If the 5090 will indeed have 32Gb as rumors say, it will be a banger. An expensive one for sure, but way cheaper than a second hand 40Gb A100 with about the same capacity. Sounds bad for our wallets, but for real playing with generative AI pushes very fast to such needs. And if that's really your thing you'll find ways to sell your images. But for that to be effective you'll need to produce a lot of high quality ones in a very short amount of time so you can select best result within a large batch, and control your production as much as possible to get there with Loras you trained + addons like controlnet etc, and maybe even have a trained LLM dedicated to optimize prompts running in parallel.
@@SocialNetwooky But when you have the RTX 3090, then why would you need the RTX A1000? Its better to just have two rtx 3090 cards then. Or just go with a single card. The 8GB VRAM the A1000 adds is negligble and not the worth the hassle of having two cards. One card with 24GB vram or two giving a total of 48GB makes sense.
@@sorkmine you need to be able to power 2x3090's , let alone find the room in your case. The additional 8GB are crucial if you want to run >20b models, if you want to run multiple >10b model (mistral Nemo for example) side by side, or want to take advantage of bigger contexts ... Or all of that together. A new a1000 is also still much cheaper than a used 3090, so there is that too
@@SocialNetwooky Yeah? Thats what the power supply is for. I power a 3080 TI and a 4080 Super with a 800W PSU just fine. As for space, they did get too hot when up close to another so i just used a riser and zip ties for the secondary card. Its little DIY but works like a charm.
@@sorkmine good for you. I have a RTX 3090, slightly above 15TB Drivespace over 7 nvme/m2 SDs, 64GB RAM ... so yeah .. an A1000 is exactly what I need right now, not another card with a TDP of 350W ... Also, your 800W PSU? Nice if it works for you, but personally I'd be weary. The suggested PSU for the cards are 750W and 700W respectively ... but you do you :)
Overpriced GA107 nonsense. I mean seriously, I can have a much higher clocked RTX 3050 8GB with 2 more SM's giving me 8 more tensor cores, 256 more cuda cores, and considerably higher throughput on int8, FP16 & 32 thanks to the higher clocks... all for 175 bucks LESS! What do I get with the A1000? It fits in a smaller chassis, doesn't need aux12v power, and Nvidia's bogus "certified" drivers... for nearly double the price, who freaking cares.
This isn't the kind of thing a "normal" person buys. not new, anyway. New, it's bought exclusively by corporate expense accounts. Then, in five or so years' time when their hardware needs to be replaced, because they didn't get the same level of infamy, they're sold at fire sale prices on ebay and the like since even the lessors of such hardware have no room to store or hoard any for certified repairs.
@@marioprawirosudiro7301 yeah .. .Form factor and power consumption. I have a RTX3090 which is more than sufficient to run any model that fits in the 24GB VRAM at a more-than-acceptable speed, and neither my mainboard, nor my case nor my PSU could deal with another full size power hungry beast. This fills my need for extra VRAM nicely for an okay price.
The 3060 is 12GB and is cheaper. Same price as A400. But they do use A LOT MORE power than the A series, so in power alone, it will probably get more expensive over time.
@@lucasljs1545 Did you forget the notion that this is not bought by individuals, but rather almost exclusively by business accounts? Businesses needing such GPUs for completing a task are not of the mindset to fiddle with modifying drivers and usually want their computer hardware to run with no more than a few minutes of down time _per year._ The prices on these Quadro cards are not for the rest of us; not new, anyway. A home user tinkering with the kind of work task that such a Gpu can do is usually fine with having their pc down for a day or three if it has a glitch interrupting a rendering task. A business can usually face multi million dollar fines for being late, depending on industry. Source: with the NFL exclusivity deal EA signed, EA would literally be fined millions for every day late a new Madden is from the start of a new season. Would not be surprised if 2K deals with the same regarding the NBA.
@@Level1Techs IIRC A4000 is limited to about 16 GB. For 24GB or more its the A5000 & the A6000 supports up to 48GB. I would be nice if the lower power cards have more VRAM as even the low end consumer grade GPUs now have 8GB, are cheaper than the A4000
Also for single slot cards worth looking at the new RTX 2000E Ada generation, downclocked vs the double slot version but interesting in compact form factors. Wish NVIDIA would make the full stack the latest generation like on laptops
@@quantum5661 Scan computers in the UK have it, quick google only shows 1 site with backorders for USA that I can see so yes seems hard to find everywhere. Very odd silent launch around the same time as the A1000 based on older generation
Of course this comes out just after I put a 4060 Ti in my home server. I guess I can console myself with having enough VRAM to fit SDXL or a 13b ollama. No Flux though, I’ve tried and it’s painfully slow.
@@milhouse8166 No, it's just for my server. The machine mostly does transcoding for jellyfin, but I run AI on it sometimes too, when I can't be bothered remote booting my workstation just to use its 4090.
If you really want to run LLMs or SD, it's a toss up. For the same money you can get a P40 24GB, so it depends a LOT on the workflows you want to do. 8GB limits you to 7B LLMs and SD 1.5 images, but this thing will give decent performance at a low power cost, but if you want to do conversational AI or one of the mode modern image generation models (SDXL or Flux, for example) it's not going to cut it and you can go with the much slower P40. Or, if you want to do the basic CUDA stuff that the A1000 can do and save a couple hundred bucks, you can get P4 8GB card and do most of the same stuff at about the same power envelope, just a lot slower.
Just when I thought my 4060 Ti was the crown jewel of my home server, Nvidia drops the A1000-a teensy AI beast making me wish AMD had more compact powerhouses with decent VRAM. Sure, I can squeeze in SDXL or a 13b ollama, but no Flux? Painfully slow. AI... bleh. At least, after watching this video, my girlfriend now believes laughter is the best aphrodisiac! 💕
Yeah, “current” as in “Oh God Bob, we ordered way too many Ampere chips in that shitty Samsung 8 nm node and still have warehouses full with them even in 2024, what are we gonna do?”
With the power within integrated chips, it make you wonder why PC Cards are so big, yes they are better, but they are as big as some pc’s these days. It would be good to see low priced sff cards.
it cant do av1 encoding FAIL ! americans that have isus with stock re appering in the market after its aborted from production ..... is funny kinda of ironic !
More tiny graphics cards please! Wish AMD had a few models that were this small that weren't so low-VRAM.
Agreed! Perhaps they will be willing to entice an OEM to step into SFFPC GPUs.
Just when I thought my 4060 Ti was the crown jewel of my home server, Nvidia drops the A1000-a teensy AI beast making me wish AMD had more compact powerhouses with decent VRAM. Sure, I can squeeze in SDXL or a 13b llama, but no Flux? Painfully slow. AI... bleh. At least, after watching this video, my girlfriend now believes laughter is the best aphrodisiac!
Agreed!
Inexpensive little GPUs with 10 gigs of vram plz AMD.
AMD also needs to step up their transcoding game
Maybe the Navi4 mid to low end push will be promising
My LLM experience with a 4070 showed that calculation speed is not an issue. Text generation is 10x faster than you can read. Only VRAM limits to what models I can run.
thats what i have found with this card I'm bumping against the vram limits hard
My exact experience as well... Although I have found heavier models that chug to produce dramatically more satisfying results.
Same experience here.
Everyone:- Wow, what a tiny Graphics card.
Me:- Thats a Rtx 3050 6GB with 8 gb of memory along with reduced power limit.
(Seriously. It has the same amount of cuda cores along with the same gen architecture)
So it's a cut down 3050Ti mobile.
For the cheap, cheap price of just 450 bucks
So it's basically a Trillion $ "Recycling" business?
this thing is 450$? you can get a 4060lp, which is also 8gb, for less.
A2000 is more powerful and can be had for ~250 USD second hand. It's also 70W, dual slot and 6 gigs.
I’d love to see a home assistant with ollama automation set up with this. Show us how it would work how smooth it is and how effective it is.
Got the A2000 ADA and I haven't had the chance to run it's through it's paces but it's so good it's size. Good to see a vid on it Wendell! Have fun with it :D
You mean the "RTX 2000 Ada"? The Ada version doesn't have the "A" in front of the "2000". It's the Ampere version that is "A2000".
I have seen people use these smaller cards with a gaming GPU and a Threadripper CPUs. For some reason they claimed it boosted performance in some applications like Da Vinci Resolve and similar, and maybe even Adobe products (Photoshop / Lightroom).
Would love an explanation or links that explain this type of card and the benefits using them in combination with a normal gaming GPU and threadripper CPUs
Thanks for the upload, interesting to hear some options as I'm building a home AI/NAS server for Frigate and Ollama to support Home Assistant. While the power/heat/size is a bonus, my local retailer does the RTX 4060Ti for the same price - twice the CUDA and twice the tensor cores. At half the price it may be worth it for a home AI server but for now I'm planning to use a Coral TPU I already have to do the detection and hand off to a GPU to keep the GPU quiet and cool until it's needed.
do you have this setup written up somewhere or seen someone do a write up on? sort of exactly what I've been hoping to get started on.
@@filmsinviolet very early days for me so just some initial research and a plan to use some old hardware for a proof of concept. I haven't found any write up on this specific approach.
The basic premise is that you can have only 1 device inferencing per camera stream so have the TPU on the low-res stream for detection and then trigger the GPU to do identification/recognition (maybe even face & license plate) by passing the object bounding box to the high-res feed.
I'm still trying to get Frigate running on the old HW in the first place, if it works then I might spring for a RTX 4060, 33% more CUDA, 33% more (next gen) Tensor and 60% of the price!
thanks for making the price shoot the hell up before i got around to getting one
if youre power or size restrained sure but otherwise even a new 4060 ti (8gb) has around 7 times the TOPS compared to this at the same price and you can upgrade to 16gb for around 100bucks extra
the things i'd do for a 4060ti 16gb low profile card
Won't the 3060 12gb be good too? It is waaay cheaper.
If I understood correctly: how did you fit the Nvidia A1000 into the Synology DS1821+ NAS? I'm asking because the GPU is x16 but the NAS only has a x8 slot if I saw/remember correctly. Thanks for the video!
riser cable?
Runs at 8 lanes, that's nothing special.
@@danieloberhofer9035 Yes electrically, but the slot size might be the problem here. I mean that the GPU physically doesn't fit into the slot. The above mentioned riser cable could be the solution here but he didn't mentioned anything like that in the video (or I overheard that?) so that's why I'm asking :)
@@kenoalbatros You realize you can simply trim the plastic off the end of any PCIE slot with a razor in about 15 seconds, no? You can fit a 16x card in a 1x slot with in about as long as it took to type this message.
@@tim3172 Yes I did already realize that and it actually works! Did something like that a few years ago using a hot nail to burn away the plastic to fit a GTX 550 Ti and a GTX 970 into an old HP server!
I don't really know how to do it with a razor as that seems really risky to me but well.. probably works to.
But how does your answer relate to my question? This is nothing you should have to do when buying new hardware, at least in my opinion. As it probably voids the guarantee.
thanks for mentioning ollama. Never knew that existed.
I bought one for my TrueNAS some months back but the Linux kernel they were using didn’t support it so I was unable to get it working and returned it. With the newest Electric Eel version I’m considering giving this a try again. I didn’t like the $400 price tag or the lack of AV1 encode. I ended up getting an Intel ARC Pro A40 which was half the price and included AV1 encode. I just want to use with Docker for handbrake, jellyfin, and maybe home assistant which I haven’t fiddled with yet.
Why not a cheaper ARC Non Pro card?
@ No specific reason. The Sparkle A310 ECO is very similar at nearly half the price. A40 is longer, has different fan, 4 mini-DPs. A310 has two mini-DPs + HDMI, shorter, different fan and shroud which may or may not be quieter/noisier. I prefer reference cards and I’m not super price sensitive when making purchasing decisions. The $90 difference between A40 and A310 ECO was acceptable to me.
This is NASty hardware!
Looks like what I'd need power- and heat-wise to do a fully local voice based AI for very modest needs. Price and availability where I live are both problems, though.
AN A1000 JUST FLEW OVER MY HOUSE!
The ghoulish Golden Girls in the background are choice!
”are choice”.. wtf kind of language are you blabbering?
@@RocketLR English.
I run llms on a 3060 and it feels ok speed for my needs. Some 4060s have 24gb ram
Seems like new M4 mini could match your use case running ollama for HomeAssistant. A little more pricey but much easier to buy and integrate. Killer feature for HomeAssistant speech is that it consumes
Truuuuu
define 'a little' ? ;)
@@SocialNetwooky LMGTFY. A RTX A1000 is $440 and a MacMini is $599 so less than 40% more but you get double the RAM and a whole computer. Plus you would easily save back the extra cost on power savings over its lifetime and it has enough VRAM to transcode HDR video while holding a full Ollama 7b model. A no brainer for the ascribed use case of home automation.
@@maxlyth true. although personally I'd reel back from entering the apple cult, but that's on another page :) Also, according to the nVidia Store, the A1000 is $365
I'm in the UK where the M4 Mini is £599 and cheapest A1000 is £439. Sadly i just spent £700+ on an MS01 and it is still a power pig drawing 35W+ at idle. Strapping an A1000 into it is going to take that over 60W (likely over 100W doing work) and cost almost double the M4 Mini. I might miss the 10Gb network but all the containers I use run great on Arm. I'd be interested to know how docker manages video ram and whether an Ollama model would actually run. If an M4 doesn't work it might help with my MS01 buyer's remorse.
I wish MDP wasn't such a terrible connector. I guess the ports on the card ports are OK but man the cables absolutely do not last. If you ever encounter monitor problem in the field and MDP is involved just swap the cable. Don't even think about just swap it out it will save you so many headaches.
C'mon you missed abit off - these are OpenGL cards and not DirectX. This card has also been superceded by the 'RTX A1000 ada' card though this card is still being sold direct from Nvidia and others. OpenGL is the primary graphics language for Autocad, Revit, Solidworks, etc for 3d rendering
How will a1000 work as an eGPU, interesting to use with an oculink.
I would nice to have a setup like this when its needed.
Also what is the tpu when its idea?
you can build ai rack for secutiry camera networking ai monitor your home for everything it sees..nvidia is leading the way for security industry market ....feel angry for amd to miss another opportunity in the market that i work for security IP cams for every business they use NVIDIA ai eye that watch their car lots and banking without guard presence
Your enabling my experimental addiction
Hi
How about using it with the HPE Gen 10 Plus?
i'll just stick to a tuned vega 64 i can buy 8 for the same price or just
Was that Gekbench 6 score on Linux or Windows?
I've been running llama 3.1 8B off my 1660 TI 6GB VRAM laptop GPU for a while now. Can even finetune a Phi2 model with unsloth on it.
that’s actually amazing
I need this with 16GB of VRAM...
I have a Thinkstation M920Q Tiny, I'm interested in this, the A2000, an Ada equivalent or whatever including the single slot conversion. Just not in the price. IT STARTS AT 600€ HERE.
one can hope the rtx 2000E will become decently common once next generation workstation cards launch
are those halloween portrait in the back?
I've wanted it for a while. But it's way too expensive for my ability lol
Pretty neat card ♦️
Nice run-down
I just wish they'd sell similar with more vram for media server encoding 8gb is just not worth that kinda money
Gaming benchmarks?
Why do they release another previous gen card, instead of making it current gen? This is basically an underclocked RTX 3050, while they could've made an underclocked RTX 4060.
I'd love to see you test the IBM Spyre card...
Hey dude. i want to get into CAD and want to experiment with som language models/AI locally. How cheap GPU could i realisticly get?
For Cad application, even complex models will work just fine on this. I used to work with a QUADRO p4000, which has the same amount of Vram and less cuda cores, and I handles 4-5000 parts assemblies and very complex geometries with ease.
I don't know how rendering will work with this, but when it's about modeling something, I think this is perfect
@@12gark Thanks. I got a good offer on a a2000, but was too slow. I'm just not sure about the AI bit.
@@sjoroverpirat not sure either, I worked as a cad designer for years, but ai is not for me. 🤣
Form factor, low wattage and performance/watt are its only selling points. Used cards will be sought after at some point, but new cards make no sense for normal people with normal desktop PCs. Even the cheapest 3050 beats this thing in AI workloads.
Are the a2000 and a4000 good for gaming? In my country I can find used ones (used by bitcoin miners) at the same prize as a 4060
The A2000 is pointless unless you need a low profile card. If you indeed do, then it's the only card of that size and form factor that'll offer you 12GB of ECC VRam.
I have an A4000 (ampere) in my workstation, which I also game on occasionally.
It is roughly equivalent in rasterization performance to a 3070. Sometimes a bit slower, sometimes a bit quicker, closer to the 3070Ti it's based on, depending on whether the game favors clockspeed or memory bandwidth/size. Mine is watercooled and shunt-modded to 180W, which makes it almost an equivalent to the 3070Ti in many games.
If you don't need any Quadro-specific features, get the 4060. Better, much quieter cooling plus you'll get access to 4000series features like frame generation, which the 3000series cards are still excluded from.
You can get a 3060 for gaming cheaper with 12GB, and will run any game at 1080p at 100 FPS. You only need a 4060 if you want to "game in 4k" but that's just dumb to be honest.
You need to get ahold of the heilon m.2 ai acclerators
thats pretty cool for a card that size
Guess new macmini M4 is better at least for Ai
Does it support sriov?
pppssh. I do a hundred trillion calculations in my sleep.
and none of them are valuable 😂
Me too but all of them wrong 😂
I tried to ask a LEO AI. This is what I got in return.
Based on the provided search results, it's challenging to provide an exact number of brain synapses fired during an average 8-hour sleep. However, we can extract relevant information to provide an estimate.
One study mentioned in the search results used serial block-face scanning electron microscopy to reconstruct approximately 7,000 spines containing synapses in mouse motor and sensory cortices [10]. The study found that after 7 hours of sleep, the axon-spine interface (ASI), a structural measure of synaptic strength, decreased by approximately 18% compared to 7 hours of spontaneous wake at night or enforced wake during the day.
Another study mentioned the importance of synaptic strength being regulated across the 24-hour sleep/wake cycle, with overall synaptic strength being the primary focus, rather than firing rate [8]. Firing rates may or may not follow changes in synaptic strength, depending on neuromodulatory tone and the balance between excitation and inhibition.
Regarding human neuron firing rates, one study estimated figures in the 0.3-1.8 range, consistent with estimates from the neocortex [11]. However, this information does not directly address the number of synapses fired during sleep.
The sleep cycle, as described in another study, involves periods of non-rapid eye movement (NREM) sleep and rapid eye movement (REM) sleep, with the brain going in and out of REM periods about 4-5 times during an 8-hour sleep [5].
Considering these findings, it's essential to note that the number of synapses fired during sleep is not a straightforward calculation. Synaptic strength and firing rates are complexly intertwined, and changes in synaptic strength may not directly translate to a specific number of synapses fired.
However, we can make an educated estimate based on the 18% decrease in ASI observed in the mouse study. Assuming a similar decrease in synaptic strength during human sleep, and considering the estimated 0.3-1.8 firing rate range for human neurons, it's possible that a significant proportion of synapses are "fired" or activated during an 8-hour sleep period.
A rough estimate could be that tens of thousands to hundreds of thousands of synapses are potentially "fired" or activated during an 8-hour sleep period, considering the complexity of brain networks and the dynamic changes in synaptic strength and firing rates. However, this estimate should be treated as a rough order-of-magnitude calculation, rather than a precise figure.
**In summary:** While it's challenging to provide an exact number of brain synapses fired during an average 8-hour sleep, our analysis suggests tens of thousands to hundreds of thousands of synapses may be potentially "fired" or activated during this period, based on the complex interplay between synaptic strength and firing rates.
CUDA? I hardly know here!
Finally a single-slot low-profile card with some actual oomph.
Eyeing this to replace my RX6400 in my 1L mini PC... 👀
No AV1 encode is a shame and feels artificially imposed, though.
thats because its last gen hardware, which predates av1.
honestly an rtx 2000e would probably blow it out of the water, if you could actually find one somehow, at a non scalp price.
@@quantum5661 oooh, I did not know that. No wonder the price seemed almost reasonable. I'mma have to skip it then.
💀
@@quantum5661plus there's sff coolers available for purchase for it
@@quantum5661 the A2000 is like 3 times the price tho.
The A2000 is super affordable right now, and just better! minus Vram, there's just a 6 and 12 gig model and the 12 gig is more expensive.
Oh man, what's up with the creepy pictures in the background?
Please, check out this real low end budget GPU for AI: RTX 4060ti 16GB.
Uses same processor AD106 as RTX A3000 mobile; memory bandwidth: 288 GB/s; 22TFlops in fp16.
Inference with model llama 3.2 3B GGUF (Ollama) is around 80 tokens/sec, thanks to good memory bandwidth.
And that 16GB has room to run larger 12B GGUF models with 8k of context!
great job
Ai to do administrator stuff?
Could this be used with Stable Diffusion?
8Gb of Vram is the bare minimum to run small models like SD1.5 with very few extra tools and feel pretty slow to generate 512x512 images. It will be a pain while producing frustrating quality unless you're into making small cartoonish images.
If you're on a budget go with a 4060Ti with 16Gb, it will be slower in calculation but at least you'll be able to run medium models like SDXL with a bunch of Loras and tools like Controlnet up to 1024x1024, and it will cost significantly less than this A1000.
If you can wait it would be wiser to at least look forward the announcement of RTX5000 series to know what it will really be made of. Until then play with online providers to be sure it is your thing, cause if it is, as such activity can be pretty addictive, you'll very fast want at least 24Gb in your GPU and be frustrated by a mid-range 4070Ti... and it cost more to upgrade from there than to go right ahead to a X090.
On the lowcost side the 5060Ti may very likely be worth waiting than go for a 4060Ti, and on the high end side, If the 5090 will indeed have 32Gb as rumors say, it will be a banger. An expensive one for sure, but way cheaper than a second hand 40Gb A100 with about the same capacity.
Sounds bad for our wallets, but for real playing with generative AI pushes very fast to such needs. And if that's really your thing you'll find ways to sell your images. But for that to be effective you'll need to produce a lot of high quality ones in a very short amount of time so you can select best result within a large batch, and control your production as much as possible to get there with Loras you trained + addons like controlnet etc, and maybe even have a trained LLM dedicated to optimize prompts running in parallel.
@@Bloodysugar Thank you for this!
Yes i will watch a vid about a product I already own
H265?
Any Synology?
Doesnt matter how fast it is when the lack of ram makes it basically useless for LLM's.
but paired with a 3090 in a normal sized Rig you get 32GB VRAM with barely a power increase.
@@SocialNetwooky But when you have the RTX 3090, then why would you need the RTX A1000? Its better to just have two rtx 3090 cards then. Or just go with a single card. The 8GB VRAM the A1000 adds is negligble and not the worth the hassle of having two cards. One card with 24GB vram or two giving a total of 48GB makes sense.
@@sorkmine you need to be able to power 2x3090's , let alone find the room in your case. The additional 8GB are crucial if you want to run >20b models, if you want to run multiple >10b model (mistral Nemo for example) side by side, or want to take advantage of bigger contexts ... Or all of that together. A new a1000 is also still much cheaper than a used 3090, so there is that too
@@SocialNetwooky Yeah? Thats what the power supply is for. I power a 3080 TI and a 4080 Super with a 800W PSU just fine. As for space, they did get too hot when up close to another so i just used a riser and zip ties for the secondary card. Its little DIY but works like a charm.
@@sorkmine good for you. I have a RTX 3090, slightly above 15TB Drivespace over 7 nvme/m2 SDs, 64GB RAM ... so yeah .. an A1000 is exactly what I need right now, not another card with a TDP of 350W ... Also, your 800W PSU? Nice if it works for you, but personally I'd be weary. The suggested PSU for the cards are 750W and 700W respectively ... but you do you :)
Buy your tech now before the Trump tariffs kick in!
Try RTX A2000T ADA -> Single Slot 50W with 16GB VRAM.
no such thing.
theres the RTX A2000, and an RTX 2000, and a RTX 2000E, but no RTX A2000T.
@quantum5661 OK ... one Letter 👍 what ever ... THX
And for kamikaze drones too
Without an AV1 Encoder, this is useless to me TBH. Otherwise, it's a fine card. I'm eyeing on RTX 2000E at this point.
What apps need the AV1?
So it's like a mobile 3050?
Your intro music reminds me of playing Grand Turismo
Why so expensive
He said hmu see you in forum... All the time ives used the forum ive never gotten a response or a post reply from him. So fact check on that.
0:45 lost me at 8GB
Overpriced GA107 nonsense. I mean seriously, I can have a much higher clocked RTX 3050 8GB with 2 more SM's giving me 8 more tensor cores, 256 more cuda cores, and considerably higher throughput on int8, FP16 & 32 thanks to the higher clocks... all for 175 bucks LESS! What do I get with the A1000? It fits in a smaller chassis, doesn't need aux12v power, and Nvidia's bogus "certified" drivers... for nearly double the price, who freaking cares.
This isn't the kind of thing a "normal" person buys. not new, anyway. New, it's bought exclusively by corporate expense accounts. Then, in five or so years' time when their hardware needs to be replaced, because they didn't get the same level of infamy, they're sold at fire sale prices on ebay and the like since even the lessors of such hardware have no room to store or hoard any for certified repairs.
I care. I like the form factor. And judging from the comments of SFF forums, a lot of other people do too.
@@marioprawirosudiro7301 yeah .. .Form factor and power consumption. I have a RTX3090 which is more than sufficient to run any model that fits in the 24GB VRAM at a more-than-acceptable speed, and neither my mainboard, nor my case nor my PSU could deal with another full size power hungry beast. This fills my need for extra VRAM nicely for an okay price.
The 3060 is 12GB and is cheaper. Same price as A400. But they do use A LOT MORE power than the A series, so in power alone, it will probably get more expensive over time.
@@lucasljs1545 Did you forget the notion that this is not bought by individuals, but rather almost exclusively by business accounts? Businesses needing such GPUs for completing a task are not of the mindset to fiddle with modifying drivers and usually want their computer hardware to run with no more than a few minutes of down time _per year._
The prices on these Quadro cards are not for the rest of us; not new, anyway. A home user tinkering with the kind of work task that such a Gpu can do is usually fine with having their pc down for a day or three if it has a glitch interrupting a rendering task. A business can usually face multi million dollar fines for being late, depending on industry. Source: with the NFL exclusivity deal EA signed, EA would literally be fined millions for every day late a new Madden is from the start of a new season. Would not be surprised if 2K deals with the same regarding the NBA.
A lot cheaper and even better option for ai etc is just a used rtx 3060 12gb
great
Tin snips are life.
I'd rather use OpenAIs realtime API for advanced voice mode in my HA.
(not so-)open AI thanks you for your service ;)
Is this a rejected a2000?
I wish Synology would enable gpu passthrough in their virtual machine manager. Like QNAP.
If you got access to the terminal, I’m sure there’s a way
Too bad it doesn't have at least 12gb
is it better than a $400 commercial gpu from intel amd or nvidia? if the answer is no then I dont see the point
But can it game? :D
Meh, only 8GB of VRAM. Would be better if it had at least 24 GB or 32 GB for AI applications.
the a4000 has 20gb from the other review :)
@@Level1Techs IIRC A4000 is limited to about 16 GB. For 24GB or more its the A5000 & the A6000 supports up to 48GB.
I would be nice if the lower power cards have more VRAM as even the low end consumer grade GPUs now have 8GB, are cheaper than the A4000
@@Level1Techsiirc a4000 is only 16gb, and stuff like 4000sff 20gb costs 1100+ usd
@@guytech7310 he's thinking the 4000 Ada, which is 20 (been eyeing it for a little bit, which is how I remember)
Also for single slot cards worth looking at the new RTX 2000E Ada generation, downclocked vs the double slot version but interesting in compact form factors. Wish NVIDIA would make the full stack the latest generation like on laptops
I've not seen a single listing for that gpu
@@quantum5661 Scan computers in the UK have it, quick google only shows 1 site with backorders for USA that I can see so yes seems hard to find everywhere. Very odd silent launch around the same time as the A1000 based on older generation
But I want int32 and int64…
Entry level workstation cards are for suckers.
*_"This Teensy Tiny GPU Packs A Punch for AI"_*
Uh....no, 8GB of VRAM doesn't get you very far with local language models.
Of course this comes out just after I put a 4060 Ti in my home server.
I guess I can console myself with having enough VRAM to fit SDXL or a 13b ollama. No Flux though, I’ve tried and it’s painfully slow.
how 1 day ago? Who are you?
Youre using it for more than a server right?
Probably either a patreon, or channel member, and got early access. @@TuxikCE
@@TuxikCE I'm Susanna, who are you?
@@milhouse8166 No, it's just for my server. The machine mostly does transcoding for jellyfin, but I run AI on it sometimes too, when I can't be bothered remote booting my workstation just to use its 4090.
Very minor correction, but these are technically 4th generation Tensor cores. The 1st generation was on the Titan V. Turing was second gen.
50w total board power is great, could be really helpful for a home server running your own Ollama database.
Any good for plex? 😂
Intel A310 still winning 😂
That is crazy, its no bigger than those old gt cards.
No AV1 Encode ? Im out !
If you really want to run LLMs or SD, it's a toss up. For the same money you can get a P40 24GB, so it depends a LOT on the workflows you want to do. 8GB limits you to 7B LLMs and SD 1.5 images, but this thing will give decent performance at a low power cost, but if you want to do conversational AI or one of the mode modern image generation models (SDXL or Flux, for example) it's not going to cut it and you can go with the much slower P40. Or, if you want to do the basic CUDA stuff that the A1000 can do and save a couple hundred bucks, you can get P4 8GB card and do most of the same stuff at about the same power envelope, just a lot slower.
thank you for comparing it to the P2200 that's exactly what i have
Just when I thought my 4060 Ti was the crown jewel of my home server, Nvidia drops the A1000-a teensy AI beast making me wish AMD had more compact powerhouses with decent VRAM. Sure, I can squeeze in SDXL or a 13b ollama, but no Flux? Painfully slow. AI... bleh. At least, after watching this video, my girlfriend now believes laughter is the best aphrodisiac! 💕
Bot
Can you game on it?
Yes.
SHOULD you game on it?
NVIDIA sure are bean counters, not sending you a current Ada Lovelace RTX 2000E (also 50 W, SFF but 16 GB memory)…
this one is current, for the A1000 series. I asked about that.
Yeah, “current” as in “Oh God Bob, we ordered way too many Ampere chips in that shitty Samsung 8 nm node and still have warehouses full with them even in 2024, what are we gonna do?”
How many cores does it actually have? It really irks me that NVIDIA calls SIMD lanes "cores".
With the power within integrated chips, it make you wonder why PC Cards are so big, yes they are better, but they are as big as some pc’s these days. It would be good to see low priced sff cards.
Almost thought the title read "Tiny GPU packs a punch for Ai Bussin'"
Nvidia hires aliens to design this stuff. They live on the moon.
AI alien
AI.. bleh
it cant do av1 encoding FAIL !
americans that have isus with stock re appering in the market after its aborted from production ..... is funny kinda of ironic !