Since Italian isn't so widenly speaken around the world I usually think people would go for other languages. I 've beeb thinking about learning a some mandarin for myself too... In bocca al lupo con il tuo studio. ;)
@AnastasiInTech do you know if any one is doing any research in AI fusion? I.E. like sensor fusion, fusing the data from different A.I. datasets into one such as fusing audio, video and others, perhaps also using for self identification/awareness.
Amazing presentations as usual - However, can not get my simple, non-tech mind around "oscillation" - meaning, the constant presentation, graphically, or by devices of a "signal" shown as a wave with a frequency - why not eg., that particle going in a straight line!? Meaning, to avoid the obvious, what makes any particle "go up and down"? Surely, that particle being emitted is amongst millions, or even billions - whilst contained in a wire or a fibre optic connection that is understandable that it oscillates against the 'barrier' but when, eg. a match is lit, does light - proton? - oscillate "up and down:? Or does it act in a spiral manner as it proceeds out? - Or, at a sub atomic level, just spin on an axis when 'excited' to pass its 'spin' to the next particle etc.? An explanation will be too long of course to convey here but I do have sleepless nights over it.....:-) Switching the bedside light bulb on.....Given the near spherical shape, where is the "Oscillation"!!!??? :-)
@@JK-xx5ns Yes we need systems that can think and reflect about there data to generate new better data. It's more about the quality of data. Waiting for the AlphaGo reinforcement learning effect. We have 70b models now that act as good as gpt4 vanilla from mid 2022 with its over 1 trillion parameters. That means there is a lot of unused potential in the bigger models if we can compress the abilitys so well. And that means the training data is currently likely of to low quality ;)
@@JK-xx5ns It makes it much cheaper, much less energy and storage space and hence a lot more accessible. The integration phase is when it actually starts to change society unless you're a giant corporation.
Also context length from 4k token to 35K and then to 130K and 1mil, 2mil and possibly 10mil. In only 2 years. There was no image generation, video or audio in 2021. Dall-e came out in april 2022. ChatGPT in november 2022. 2 years in november, and they just say that AI is dead. I want to see this people monthly or yearly work. What they achieved in 1 year.
I appreciate how you not only highlight the excitement around AI but also provide a realistic view of the challenges ahead. Your deep understanding of both the technology and the market dynamics makes this one of the best analyses I’ve seen on the topic.
Calling everything AI "hype" is becoming an obvious projection of ones fears at this point. You should learn to interface with the planet for food soon.
Indeed. Neither does it track popularity or adoption. There may be less excitement around each new IPhone model released but at no time since the first Iphone launch have smartphones lost popularity quite the opposite - soon after the launch of the first Iphone, Android phones were introduced and the popularity and adoption of both has been growing ever since. With the coming AI chip powered smartphones with their cut-down, on-device AI assistant, there'll be a new boom in smartphone sales, tablets and laptops. The hype has gotten ahead of itself but the technology is very real, very capable, and being improved all the time. I'm not seeing any slowdown and I've been following the field for about 10 years.
@@GoodBaleadaMusic I'm guessing you're not a Gartner customer. Gartner puts out charts like that for most new tech. Hype to them doesn't mean it's bad, but just there's a mismatch between perception and capability. They've been at it decades and are very useful if you want to plan for IT infrastructure. Not sure if your last sentence was a threat or not, but I'll be just fine.
@@aldarrin Ok as long you're you're arguing from that niche and not as the window lickers watching the world leave them behind. Thats like 90% of the of thumbs up on your comment.
The nature of AI is that it develops rapidly. The nature of business is that generational releases are slow. AI may get burdened with a repetitive hypecycle where there's seemingly very little development for extended periods of time during which people get bored and say it was just a fad, followed by a massive spike in capabilities which causes a huge rush of interest, only for another year or two to pass with relatively little happening and people start saying it's dead again. Rinse repeat.
I think the last quarter of this year will proove if the hype is real or not. But there is no doubt that this tech will revolutionize the future, the question is how big of a leap and how fast.
Thanks Anastasi, I wonder if we are missing the iceberg below the water. AI is still being addressed as software rather than as an intelligent interface between entities, knowledge, and, in many cases, the "machines" that manipulate the environment. For efficiency, AI will eventually interface with many machines at the "hardware" level (the equivalent of machine code). In doing so, AI will transform the machines and the code that runs them. For the majority of the population that transformation of machines may become the most visible product of the evolution of AI. Afterall, for most of the population machines are black boxes. AND, AI and the processes that control both AI and machines are "magic". Written in my model Y. It just drove me to 3 destinations without human intervention and now will drive me home.
That one graph mentioned increased compute since GPT-4, but I don't think any of the models since then were actually larger. This is what makes people think it's plateaued, but nobody seems to have yet dared to spend >1B on a single training run. THAT is when we see whether we have plateaued in any meaningful way.
Absolutely correct. This is why this analysis of hers is flawed. There isn't anything to compare GPT 4 to yet because none of the AI companies have come out with their next-gen models yet. OpenAI specifically said they were going to wait awhile to start work on 5 after releasing 4, they only recently stated that they were working on 5 now. All of the other companies have come out with new models, but they were playing catch up to GPT 4 and only very recently (weeks in some cases) caught up to Open AI. So we haven't see a single GPT 5 class model yet trained using all that newly purchased expensive hardware. Until we see that no one can say with any authority if progress has plateaued or looks exponential.
No one builds a magnitude larger model because there is no hardware to train them. At least no hardware that is economically practical. The platue is caused by hardware limits.
@@kazedcat Close but not entirely correct. Getting enough hardware for significantly larger models has been a problem. But that is where all that investment cash has gone, they have been building new data centers to train the next gen models. For example xAI's new data center for training just came online. It takes time to build new data centers and then it takes time to put them to use training the new models. It is in progress. Both Open AI and xAI have stated recently that they are now working on training next gen models.
@@Me__Myself__and__I It is not enough if they want an LLM with tree search. To train that kind of LLM they need 1000X more compute. Yes they are spending a lot of money for new hardware but doubling your compute only gives you an AI with 10% lower perflexity and without planning abilities which needs tree search architecture.
@@kazedcat This tree search you mention would be primarily a technique at inference-time. It would not increase amount of training data needed. If they implement the techniques outlined in "exponentially faster language modeling", selective attention and use ASICs, they get 100-300x speedup, O(n*log(n)) context complexity and 20x speedup respectively.
It is important to separate the AI hype cycle from the ability to monetize AI. Palantir is good example of a structure that uses the ontology manage multiple AI agents (K-LLM), while producing value for their clients. Also imitation AI will be the norm, while AGI works its way through over the coming years. It's an exciting time to see it unfold in front of our eyes.
Ok Anastasi, I absolutely love you and FINALLY subscribed after being a devoted fan for a few years now. You work so hard with all of the time put into your research and provide such informative videos that are really valuable when analyzing and comparing the current state of technology to innovations in achieving breakthroughs. Nobody provides such keen insight like you do so I am grateful for your presence. You have a unique perspective, thinking different in a world filled with so much hype. Its your natural curiosity that allows you to compare what you learn to what you know. You then strip away the biases to get to the meat of things; understanding the impacts of how these emerging technologies can truly impact the lives of so many people in ways that can change them forever. You have a good heart and so this all shines from a wholesome place within you. It is refreshing and is how you got me. You really care about the betterment of mankind and get really excited when things are truly groundbreaking. Plus, you never have said, not even once, the most annoying phrase people use in making announcements today..."I am...so...excited...to be...sharing...this...news...with...you." Yeah, I can't finish a video after hearing that insincere and highly annoying sentence. Hopefully it will be a trend in our tech industry presentations that QUICKLY fades away since stealing from Apple is no longer innovative or the standard. That is totally not your style at all as you come across with genuine intention and explain things in a way to where we all can understand them. Plus, you are going for your MBA in Italy, I am so impressed! I was raised in the wine industry in Texas, still at it so many years later because I love science and psychology; how science can affect our senses and change the psychology of how we think. How we can survive impossible odds, treacherous environments with patience and ultimately innovation, bringing us through struggle to the breakthrough on the other side and bringing people together along the way. I loved my travels through Italy, the wine, the people and always, even more, the science! My inner nerd keeps me curious much like yourself. You are beautiful and amazing. I truly value you and what you do. Thank you! I look forward to getting my first notification! Your friend...James :-)
@@MeowtualRealityGamecat Absolutely, AR/VR is high on my list too. That will be improving a lot in the next 5 years, both with making the hardware smaller and enhancing the software with AI implementation.
AI and Medical is one I think will be outstanding too. Can you imagine the market for a pill that would change your hair color, and that is just a very minor aspect of the combination.
As an electronics engineering working in industrial robotics I don't think its the "next" big thing. It is absolutely going to be a huge thing, but I think the hardware will severely limit its applications for a bit. When it occurs I think it will be world changing, so depending on how you define "big" It may qualify. I think the next big thing will be AI applications to material science. Materials limit us in so many applications, and often 1 type of material fuels civilization for decades. Think of how first steel, then plastics, and now carbon fiber, all evolved from one material with nice properties to many variations for actual applications. Our current AI models are well equipped to be modified into tools that tackle the complexities and huge number of possibilities that exist in material science. AI sorts through the junk possibilities very well, leaving mostly quality results with a few hallucinations to be weeded out. I think we are close to a point where Company X needs a material that maximizes properties A, B, C. Hope into MatsGPT, and Boom! got a custom proprietary material for your application. Proprietary materials impact many applications that directly impact advancement in many current industries, which I think is the quality most missing from current AI applications. Although Robots have that quality which certainly makes them a contender, I just think there are a some practical roadblocks that will slow it down significantly. (Motors and power density being two)
The reason Apple dumped Intel chips in favor of creating their own cpu’s is that Intel wanted to create a more generalized cpu for windows customer base and apple needed more specialized operations. So Apple said no thank you, we’ll make our own. As a result Apple’s chips are ten times more efficient and much more powerful than intel cpu solution. I can see why chip makers want to create specialized chips to handle specific algorithms. This could potentially reduce the number of servers as compared to a generalized gpu farm such as nvidea’s h100’s servers. Great show by the way, I always enjoy hearing your observations.
There one area where AI really benefits humanity is in speech production by deciphering electrical patterns on the surface of the patient's brain. When I first saw this on television, it was simply amazing.
*THE PROBLEM FOR AI IS THIS EQUATION* ∆AiP --> 0 as $ --> ∞ The change in Ai performance tends towards zero as the cost tends towards infinity - the exact opposite of what the tech bro's claim - and its Cambridge University saying this, not me.
You apparently don't understand what that function says. That function applies to anything you put money in to. The change in progress as you dump money in will always go to zero. You are chasing perfection. How much faster does a car get when you put the first 10k in? The second 10k? The third 10k? At what point are you pushing money to infinity for zero growth in car performance? Now you understand. Understand things before ya quote em friend.
Pro tip: You can train a voice AI with you voice and run it in your preferred language like Italian and you can compare it with your own accent in that language. It can help in increasing proficiency 🤓
AI improvements will mostly come from building better agents and larger context windows. We need to handle context assembly much better. If our agents can do that, and we capture how people learn to use them, machines can create synthetic training data from in-context learning.
I hate how 'markets' control progress. They don't always select the most useful and beneficial idea's but those that will make money. These are often not the best of idea's.
Awesome video. Important to point out as you said this "hype" cycle is opinion based and although it makes sense in a lot of cases the graph is not necessarily "up to scale". By that I mean, the difference between the hype peak and the valley of disappointment can be less steep than in that image, it can also have a shorter duration than many other technologies and so on. Also important to point out that some Human technologies never went through this hype cycle like electricity, internet and penicillin. It's hard to predict if AI is THAT disruptive, although it seems we as Humanity are not even ready for its benefits so probably not.
Good points. Also unlikely a lot of other technologies plotted on that hype cycle chart AI has a lot of linked technologies that just recently got attention / serious funding such as custom hardware. Those other technologies could potentially amplify the progress of AI on top of whatever progress is made within AI itself.
There are some pretty strong points to argue against your thesis. The truth is no one really knows yet. The next frontier models in 2025 will clarify everything
Ciao, Anastasia! Grazie per il tuo sforzo nell'imparare l'italiano, nonché per la divulgazione di tematiche tecnologiche. Quale lusinga sentirti parlare la lingua di Dante, bella e intelligente donna del fare!...
I don't know how did these estimations, but they are surely wrong. Even if we discard any future improvements the state of the art LLMs will massively disrupt the industry. I use them every day and I am maybe ~3x more productive (as a software engineer). I also do graphic design the improvements in productivity there are even bigger. The AI only needs to get marginally better to replace most of the jobs. This improvement will probably be achieved using new algorithmic improvements, no extra data or compute needed.
As someone working for an IT service provider, I've had near zero impact. They aren't nearly good enough (and trustworthy enough) yet to take over helpdesk functionality and they fail to adequately solve the problems that see me struggling in 2nd/3rd level support. Answers to complex problems tend to be a halluciogenic mix of correct information and various levels of wrong.
@@Steamrick The average adoption cycle is 7 years, also current AI is obviously not a human level, but it doesn't have to be. The thing is it's not good enough until it is, it's like a crossing a line. It will one day suddenly be good enough and it doesn't have to get that much better. I wasn't using LLMs at all just few months ago, because it wasn't good enough for me, until it was.
@@drednac I've got mixed feelings. Chat gpt does help me a lot with programming, solving problems, or explaining various IT terms and concepts. But Ai is still nowhere near creating a fully working program without a human having extensive programing knowledge
LLMs are, unsurprisingly, fluent in most computer languages (computer languages have a vocabulary of about 32 words, and fixed semantics, so they are rather easy to encode). I use LLMs all the time to generate code snippets, much like I used to search for snippets. LLMs are a much more efficient way to "search", since it can basically do text-to-code (based on the enormous corpus of source code available), it's already searched for everything so you can just ask it
When I have start using AI/LLMs, I almost did not sleep for a week until I understood how it worked... now its been more then 1 year that I dont use it and dont miss it!
Loved the content, thank you! I think the "wow" factor would return if it was used to improve people's lives in some way. I believe that's just a matter of time.
i think scaling should be logharitmic, it's like to add one more abstraction layer to understanding you need x times more data as top layer need more variety
You are right, when it comes to generating texts, images or films, only the quality needs to be improved, e.g. less hallucinations. But there is 1 exception that only a few people appreciate: Tesla. There, AI is used to control robots: camera images in, steering commands out. AI learns from video clips, now first with robots on 4 wheels, but they are also working on building this technology into humanoid robots called Optimus. They have already come a long way on 4 wheels (search for FSD on RUclips, Full Self Driving) and be amazed at what they can already do. They have reached the level of a driving school exam candidate. Transferring this technology to humanoid robots will still be a lot of work (let AI watch many other videos), but we already know from their cars that it will definitely work. So here is still a considerable development trajectory with major important results to be expected. Only with regard to Tesla I disagree with your video.
When consciousness is fully understood then the roadway towards that huge breakthrough in AI technology happens. A lot of different scientific fields wil have to get onto this one.
For myself i continue to be amazed, looking just at image generation, models like Flux and video creation are now consistently nearly photo realistic. It's mind-blowing, i am more excited by the day, waiting for next iterations to drop
AT THIS POINT ... if I see a page I follow posting AI Uncanny Valley Slop... I unfollow them immediately... It was interesting for 2 months last year, but these gross things are taking the internet... Enough is enough 😵💫
6:03 I am Spanish and in name of my country I want to claim back our flag, we are not yet Italians, despite they think olive oil is their thing and that mediterranean cousine means italian food.
As a computer scientist and AI engineer, I’m getting extremely exhausted, and at times overwhelmed by the breakneck speed of change. I have written and seen some extremely awesome cases for large language models. I want to sit back for 6 months and see if the dust settles a bit. We spend weeks or months developing an application that uses AI and new technologies come out that require us to rework the application. It gets a bit tiring.
Sorry but I think something was not considered. The benchmark score goes up to what value? From the graph it looks like up to 100%. So there is no way not to reach a plateau. In fact, a variation from 60% to 70% is much smaller than a variation from 90% to 99%, because the calculation must be done with the difference. (100-p1)/(100-p2). (100%-60%)/(100%-70%) = 1.3333. A 30% increase. (100%-90%)/(100%-99%)=10. A value 10x greater. A huge effort will be necessary to go from 99% to 99.99%, for example. 100x. Furthermore, it must be considered that developments in AI have already suffered a lot of hype over time, but gradually the technologies developed were being incorporated, for example, the conversion of text into audio. It is clear that the cycles have presented greater hypes over time. But if it is almost certain that LLM's are reaching their limits, the same cannot be said about the AI area, because new, more efficient methods of simulating human intelligence will emerge over time.
Actually we don't know that LLMs are reaching any limits or plateauing. We haven't seen a GPT 5 yet for comparison. And as others have pointed out they have optimized LLMs so much over the past year or so that very small models today are roughly equivalent to models that were hundreds of times larger a year ago. That is massive progress, it just doesn't push the frontier. But we haven't seen a new frontier (large) model yet. Imagine if they can transfer that massive optimization into the next huge frontier models so that a new 500B model would be equivalent to a 50T model of the past. We just don't know yet, no one does.
@@Me__Myself__and__IThere is no hardware to train GPT5. If you look at the parameter scaling from GPT1 to GPT4 and assume that the compute scales with parameters then you will see that GPT5 could not run even with near future hardware.
@@kazedcat Funny, OpenAI has recently said that they are now actively working on traini g GPT5 (though I think it will be called something different). xAI's new data center also juat recently came online and they will begin training their next-gen model soon. Sufficient hardware WASN'T available, but things change and progress over time. Acquiring the hardware for training is where the investment funds have been going, it just takes some time to acquire and setup such things.
"it's very hard to make predictions, especially about the future" - lol, I like that citation ! (it's atributed to Niels Bohr). Whenever someone predicts that a specific technology is reaching a plateau, I think of a mosquito. A mosquito has a very small brain, but it's able to fly, find food, keep you awake (...), avoid your hands that are trying to kill it, find a partner to reproduce with, and it's all autonomous. We're a long way from reaching Mother Nature's level of miniaturization and capability.
My programming productivity has increased by at least 20%. The progress is going very fast if we measure by the number of errors which keeps reducing by 20 percent every 6 months. We need sprite based animations, cubase style composing, AI synth architectures, and other tools, audio enhancer for 1920s and 1950s CDs, free video upscalers, psychologist and doctor AI, baby AI's learning with 5 senses, agriculture AI, chemistry AI, and they will come along.
general vs specific is. you can upgrade till capacity potential is reached vs once build its all you will ever get. combination of both like always. LLMs sind nur die bibliotheken.
Don't worry. There will be another major breakthrough and paradigm shift. One way or another, we will absolutely have an incredibly intelligent / super intelligent reasoning system in under 5 years.
You mentioned at about 5:16 in this video that you will soon be starting an MBA program in Italy. Will you still be making these Informative videos while enrolled in this MBA program?
You have to be precise with the interpretation of diagrams. In scores from 0 to 100, going from 80 to 90 percent is doubling the performance of the model!
Ms. Anastasi, this is the episode that caused me to click "subscribe", because you are recognizing things that are outside the general rush to judgment in this field, and touched on several important issues in ways I think accurately reflect reality. Evolutionary processes took millions of years to experimentally configure many kinds of biological intelligences. The custom structure of each, with slight variations in each individual within "species" is highly complex with 100s, possibly millions of substructures to allow those organisms, even including humans with our broad capabilities, to exercise what we call "general intelligence". You can't just stack 2,4,8...or even thousands of brains together to get a "super" intelligence. No more than putting 10,000 scientists in a room gives exponentially faster results compared to 100. Successful methods will involve painstaking development which won't be exponential. That said, there are capabilities that can grow exponentially with compute power -- if indeed Moore's law is even continuing to accurately model computing trends. Destructive uses of AI can often expand exponentially because they don't have the constraints of living beings that they need to succeed and reproduce. This is especially true when fed with exponentially growing personal data pools. But AI that people actually desire will have limits as above. Constructing useful AI models will be a hard fought process, dependent very much on human thought. And that will depend upon a huge number of different internal structures to implement different aspects matching or exceeding human capabilities. The models suggested here recognize the limitations on growth of AI, and the human psychological response when we see this reality.
GPUs are ASICs, application specific integrated circuits, they are built to run 3D acceleration and later shaders on those 3d scenes. An example of "not an ASIC" is an FPGA that are built to be reconfigurable on the fly.
Can you please share your views about the future of Electronics and communication engineering. I'm joining this course soon. Pls pls pls. I'm very worried.
We should focus more on the application of technology rather than just the hype and empty promises. IA, for example, could be so helpful in areas yet to be seen!
There's a little more to this hype cycle graph, since the peak is when investment gets triggered, but then the engineers and designers need a few months to catch up and deliver their products. So I'd argue it's almost like mach diamonds on a rocket engine, with a standing wave that continues on a positive-slope line of some kind.
the tech revolutions i have lived through include [transistors, PC's, networks, internet, mobile phones, smart phones, bigdata, social networks, digital music/photos, EV's, LLMs and soon AGI] 12 total. in Every Case, people underestimated the magnitude of LT change and underestimated LT net benefits. The same is true today. DK curves are just noise. What matters are the tech cost curves and the tech advancement curves, which are inevitably always similar for every new revolution.
I am only a humble PC technician with +30 years of experience, I love technology in general and I'm not very hype about AI. I think the main issue here is how big names like the nvidia, microsoft, etc. are trying really hard to push and sell everyone who cares to listen (not only scientist and the likes) how AI will make your business grow faster and how you could save money and thats is very imprtant that you get AI first. And the reality is that 1. this is not the case for every business out there, 2. people who only like to browse the web, watch movies and play a game don't care, nor need AI to do this tasks, and 3. the "pandemic technology bubble" its already gone. To make things even worst you have (to name a few) a huge recession going on around the world, several war conflicts that difficult every aspect of global business and life in general, the huge amount of energy AI hardware demands (and the extra pollution it wil create) and data confidentiality and security. I believe AI will reach the consumer eventually, but it will take time. And we should be very carefull with the implementation. As I see it AI should assist, not replace. We need the children to learn and not to ask a enitity to solve their problems. I really hope CEOs around the world take their time and get good advices before making decisions about AI. Sadly thats just hopeless wish, when I know most if not all of them only think about 1 thing, money.
The main issue is that we are barely scratching the surface of AI, but also lagging far behind on memory tech. Real ai will need more than 1tb of HBM level memory and currently it doesn’t seem feasible in the near future. However there are other HW and algorithmic solutions. However it doesn’t address the huge energy requirements you’ll still have for edge ai.
I used to really think we were about to hit the singularity, but now I really don’t. LLMs right now basically work by using trial and error to find a bunch of math operations to make a desired output. After it’s trained it doesn’t learn anymore. For memory, the entire conversation is literally just fed through the AI again every time you prompt it. It’s extremely inefficient and expensive. AGI is gonna have to be something else.
Yeah, you’re right this is just an opinion. The hype graph shows the hype based on discovery. The hype goes down because people are using it but as it improves the hype will come back.
I believe that the development of new peptides would be faster with alphafold. Simeglutide is already a blockbuster and there is plenty of space to explore with the help of IA
But the models from the last few years haven't used larger models. They *have* trained them on more data, but they haven't increased the size of the model - presumably due to hardware limitations on the training and inference side. This is like training a monkey for a million years, it's going to be good at what it has practiced. And this is what we are seeing. If we step up to 10T params and see no improvement then I will concede a plateau. But I strongly doubt we will see it.
Yes. Its true that their has been massive investment in new hardware to train AI models. But what everyone seems to miss is that it takes awhile to build new data centers and get everything setup. Next generation models trained on that new hardware don't exist yet, Open AI just recently said they had started work on GPT 5. I concur that once we see the GPT 5 generation of models, if those only have minor step improvements THEN we probably hit a plateau. But no one actually knows that yet.
@@Me__Myself__and__I you are right about that plus they are in a period of lack of people not fund so it also take a while to scale the human resources !
@@Me__Myself__and__IThe technology is already at its limit. Moore's law has slowed down to only doubling the compute capability every 3 years but AI needs to 10x the compute needed to double their capability.
AI is like having a calculator. Most people can get plenty of use out of a basic calculator, but hand them a TI-84 and they won’t see much more value from it. Without a solid understanding of how the model works and the field you’re trying to apply it in, you might as well be mashing random buttons.
In my opinion, Generative A.I. Proved to be more disappointing, because while The Hype and Promise of what it would bring, in reality seemed exciting, As it turns out, People would Rather be the Creative Influences Themselves. And so with that A.I. at best becomes a 'Tool of Creativity' That only adds to society at a minimum. Mainly because allot of Artists and Creators would Rather use their Own Ideas or, at Best, Generative A.I. as a tool. It also has brought to Light the Ugly Truth about the Beginning of the A.I. Revolution, In that Deception on a Very Personal Level becomes dangerously Easy to produce and use to Manipulate the Masses, in the form of Deep Fakes and other A.I. Hoax Creations. I absolutely Love your Show, Awesome channel, Keep up the Good Work!
pretty sure when a model goes off script and teaches itself an unsolicited language on it's own it most likely has achieved some level of AGI... it chose Persian if I recall correctly. curious what you think about that Anastasi? thank you for what you do I have learned so much from you love.
The funniest thing you can do in an engineering meeting is drifting into another language. I am from the South in the USA, but I worked in the North, where they thought I was an idiot. So I would drift into Mandarin, until they stopped asking stupid questions! People fail to see humans as humans, where everyone is alike. Sounds selfish when people assume they are smarter, because of where they are from. I have found the cowboy hat triggers people who assume it means the lack of education. I tease them with, "Help, I can't read or write, but I can type in 3 languages!". I found the study of cultures to be more fulfilling than most of what they teach in college! Getting a Master’s degree in engineering is a pain these days, but this is the only way to move up in companies with lazy HR departments! I bypassed the MS, and took a job as a COO, but that doesn't happen in most companies.
I see that this video was posted 25 minutes ago. And I am already commenting on it. Now I am very worried that I’m spending too much watching your videos. Are you gonna get to the point that are gonna be waiting for new videos.
What kind of comment was that? What did you expect, the Swedish lady is talking about the technical side of AI. She's not going to do see through top reveals.
What we need to find out is if these LLMs and the approaches they use can scale much further. But if they can, whoever comes along with a somewhat slower and cheap processor stack that uses 5% of today's TDP will trigger the next "wow" phase.
Ty for your content. I liken AI development to the Electronic switch. First was a vacuum tube. Then we developed Germanium and shortly after Silicon Transistors. And finally the Microprocessor. The first stages growth was limited by the physical limitations of the Tubes. Namely the failure rate of tubes. Once there was enough tubes, you could always expect there to be a faulty tube at almost even given moment. So scaling became impossible for the early vacuum computers. There was a hard limit. Microprocessors suffer from an upper limit. NOt one of failure, but of power consumption and the high dissipation of heat required. A major breakthrough in electronic switch design in needed, one that is scalable to be compacted into chips, and not overheat. Possibly quantum, but that has serious stability issues, not to mention the cost. Photon switches are promising, if a bit heat problematic when compacted. Single Electron transistors might be the future, if they can get past a single lab model. I personally think that variable switches might be the future, IE: not just binary, but power ratios are not really there yet. but maybe.. Anastasi is always ere to bring us the exciting new developments in regards to this.
What we actually need is an FPGA that has extremely advanced numerical processing primitives built in, so we can adapt the tooling this would probably sacrifice some speed for future nimbleness to adapt. OH and power, FPGA's IIRC are quite a bit more power hungry, right?
The prediction that AI will only improve productivity by 5% doesn't conform to my experience as a software developer. I would say it has nearly doubled my productivity. Even more, but less measurable, it has allowed me to take on project I normally would not have taken on, because they involved technologies that were outside of my comfort zone. AI gave me the confidence that I could handle - well, maybe just about anything. And I don't write trivial software. I have been developing software for over 30 years. AI can't do an entire project for me, like the demos where you ask it to write a snake game in Python. That it can do. But it helps me write up a problem statement, with design, and coding, more as a colleague than the author of the code. I would extrapolate from my experience to a broad range of other professionals in other fields, for example, lawyers might also gain a big boost in productivity.
"I think not" is not a good argument. This supposition came from training Sora. When you scaled the data and compute the videos quality scaled linearly. They demonstrated that correlation, which was also a surprise to them. Video generation is going to be one of the most transformative application of AI this year, for better and worse. It threatens Hollywood, RUclips creators, and elections. But it also will offer tailored media.
I'm a physicist working in neuromorphic computing... however I lack some of the knowledge regarding chips used for AI ... can you recommend any resources to learn more about AI hardware?!
About AI scaling law, I totally agree your viewpoint. Because more datum means we need to invest more advanced hardware to process those datum, from which they consume more energy needed. It’s Ike a chain reaction to all the supply chain industries.
Do you know if there is any data regarding the shortening of the time between technological innovates? It's probably a hype cycle, but it's also probably an accelerant?
Groq company is interesting with their LPU (Language Processing Units) which runs very low power once GPU trains a model, then you switch to use of the model to LPU to run it. Thats the future. And yes Gartner hype cycle is usually true, but AI will change everything permanently - the cycle shows it.
Check out Babbel and save up to 60% on your subscription today:
bit.ly/AnastasiInTechAug
Brava, Anastasia! Ottima pronuncia 😉
Since Italian isn't so widenly speaken around the world I usually think people would go for other languages. I 've beeb thinking about learning a some mandarin for myself too...
In bocca al lupo con il tuo studio. ;)
@AnastasiInTech do you know if any one is doing any research in AI fusion? I.E. like sensor fusion, fusing the data from different A.I. datasets into one such as fusing audio, video and others, perhaps also using for self identification/awareness.
Amazing presentations as usual - However, can not get my simple, non-tech mind around "oscillation" - meaning, the constant presentation, graphically, or by devices of a "signal" shown as a wave with a frequency - why not eg., that particle going in a straight line!? Meaning, to avoid the obvious, what makes any particle "go up and down"? Surely, that particle being emitted is amongst millions, or even billions - whilst contained in a wire or a fibre optic connection that is understandable that it oscillates against the 'barrier' but when, eg. a match is lit, does light - proton? - oscillate "up and down:? Or does it act in a spiral manner as it proceeds out? - Or, at a sub atomic level, just spin on an axis when 'excited' to pass its 'spin' to the next particle etc.?
An explanation will be too long of course to convey here but I do have sleepless nights over it.....:-) Switching the bedside light bulb on.....Given the near spherical shape, where is the "Oscillation"!!!??? :-)
Tolles Video
A 9b model can do more now than a 175b model could do back in 2020! This is at least a factor 20 efficiency improvement in 4 years!
Yes, we have optimized inference a lot by quantization and other techniques, but training has not improved nearly as much
@@JK-xx5ns it has because the more efficient we are the better we are. Look at the first iteration of GPT 4, it was really not as good as GPT4o today
@@JK-xx5ns Yes we need systems that can think and reflect about there data to generate new better data. It's more about the quality of data. Waiting for the AlphaGo reinforcement learning effect. We have 70b models now that act as good as gpt4 vanilla from mid 2022 with its over 1 trillion parameters. That means there is a lot of unused potential in the bigger models if we can compress the abilitys so well. And that means the training data is currently likely of to low quality ;)
@@JK-xx5ns It makes it much cheaper, much less energy and storage space and hence a lot more accessible. The integration phase is when it actually starts to change society unless you're a giant corporation.
Also context length from 4k token to 35K and then to 130K and 1mil, 2mil and possibly 10mil. In only 2 years. There was no image generation, video or audio in 2021. Dall-e came out in april 2022. ChatGPT in november 2022. 2 years in november, and they just say that AI is dead. I want to see this people monthly or yearly work. What they achieved in 1 year.
I appreciate how you not only highlight the excitement around AI but also provide a realistic view of the challenges ahead. Your deep understanding of both the technology and the market dynamics makes this one of the best analyses I’ve seen on the topic.
Agreement, great stuff
Current research which will likely have a breakthrough soon will be video based AI.
I'm not biased at all ...
This actually sounds like a bot
That Gartner chart tracks "hype" not technical capability.
Calling everything AI "hype" is becoming an obvious projection of ones fears at this point. You should learn to interface with the planet for food soon.
Indeed.
Neither does it track popularity or adoption.
There may be less excitement around each new IPhone model released but at no time since the first Iphone launch have smartphones lost popularity quite the opposite - soon after the launch of the first Iphone, Android phones were introduced and the popularity and adoption of both has been growing ever since.
With the coming AI chip powered smartphones with their cut-down, on-device AI assistant, there'll be a new boom in smartphone sales, tablets and laptops.
The hype has gotten ahead of itself but the technology is very real, very capable, and being improved all the time. I'm not seeing any slowdown and I've been following the field for about 10 years.
@@GoodBaleadaMusic I'm guessing you're not a Gartner customer. Gartner puts out charts like that for most new tech. Hype to them doesn't mean it's bad, but just there's a mismatch between perception and capability. They've been at it decades and are very useful if you want to plan for IT infrastructure. Not sure if your last sentence was a threat or not, but I'll be just fine.
@@aldarrin Ok as long you're you're arguing from that niche and not as the window lickers watching the world leave them behind. Thats like 90% of the of thumbs up on your comment.
I wonder to what extent it indirectly also influences investment which in turn influences development.
As an Italian: I love you your Italian is great! ❤
I have Belgian relatives living for a long time in Italy so I appreciate it also.
The nature of AI is that it develops rapidly.
The nature of business is that generational releases are slow.
AI may get burdened with a repetitive hypecycle where there's seemingly very little development for extended periods of time during which people get bored and say it was just a fad, followed by a massive spike in capabilities which causes a huge rush of interest, only for another year or two to pass with relatively little happening and people start saying it's dead again. Rinse repeat.
The real question is, 'what type of world do they intend to build?'
I think the last quarter of this year will proove if the hype is real or not. But there is no doubt that this tech will revolutionize the future, the question is how big of a leap and how fast.
The hype is definitely real, but reality is far from the hype.
Thanks Anastasi,
I wonder if we are missing the iceberg below the water. AI is still being addressed as software rather than as an intelligent interface between entities, knowledge, and, in many cases, the "machines" that manipulate the environment. For efficiency, AI will eventually interface with many machines at the "hardware" level (the equivalent of machine code). In doing so, AI will transform the machines and the code that runs them. For the majority of the population that transformation of machines may become the most visible product of the evolution of AI. Afterall, for most of the population machines are black boxes. AND, AI and the processes that control both AI and machines are "magic".
Written in my model Y. It just drove me to 3 destinations without human intervention and now will drive me home.
That one graph mentioned increased compute since GPT-4, but I don't think any of the models since then were actually larger. This is what makes people think it's plateaued, but nobody seems to have yet dared to spend >1B on a single training run.
THAT is when we see whether we have plateaued in any meaningful way.
Absolutely correct. This is why this analysis of hers is flawed. There isn't anything to compare GPT 4 to yet because none of the AI companies have come out with their next-gen models yet.
OpenAI specifically said they were going to wait awhile to start work on 5 after releasing 4, they only recently stated that they were working on 5 now. All of the other companies have come out with new models, but they were playing catch up to GPT 4 and only very recently (weeks in some cases) caught up to Open AI. So we haven't see a single GPT 5 class model yet trained using all that newly purchased expensive hardware. Until we see that no one can say with any authority if progress has plateaued or looks exponential.
No one builds a magnitude larger model because there is no hardware to train them. At least no hardware that is economically practical. The platue is caused by hardware limits.
@@kazedcat Close but not entirely correct. Getting enough hardware for significantly larger models has been a problem. But that is where all that investment cash has gone, they have been building new data centers to train the next gen models. For example xAI's new data center for training just came online. It takes time to build new data centers and then it takes time to put them to use training the new models. It is in progress. Both Open AI and xAI have stated recently that they are now working on training next gen models.
@@Me__Myself__and__I It is not enough if they want an LLM with tree search. To train that kind of LLM they need 1000X more compute. Yes they are spending a lot of money for new hardware but doubling your compute only gives you an AI with 10% lower perflexity and without planning abilities which needs tree search architecture.
@@kazedcat This tree search you mention would be primarily a technique at inference-time. It would not increase amount of training data needed.
If they implement the techniques outlined in "exponentially faster language modeling", selective attention and use ASICs, they get 100-300x speedup, O(n*log(n)) context complexity and 20x speedup respectively.
It is important to separate the AI hype cycle from the ability to monetize AI. Palantir is good example of a structure that uses the ontology manage multiple AI agents (K-LLM), while producing value for their clients. Also imitation AI will be the norm, while AGI works its way through over the coming years. It's an exciting time to see it unfold in front of our eyes.
At 6.40 min … yes the AI temperature has dropped so quickly!! … thanks for the great video.
Ok Anastasi, I absolutely love you and FINALLY subscribed after being a devoted fan for a few years now. You work so hard with all of the time put into your research and provide such informative videos that are really valuable when analyzing and comparing the current state of technology to innovations in achieving breakthroughs. Nobody provides such keen insight like you do so I am grateful for your presence. You have a unique perspective, thinking different in a world filled with so much hype. Its your natural curiosity that allows you to compare what you learn to what you know. You then strip away the biases to get to the meat of things; understanding the impacts of how these emerging technologies can truly impact the lives of so many people in ways that can change them forever. You have a good heart and so this all shines from a wholesome place within you. It is refreshing and is how you got me. You really care about the betterment of mankind and get really excited when things are truly groundbreaking. Plus, you never have said, not even once, the most annoying phrase people use in making announcements today..."I am...so...excited...to be...sharing...this...news...with...you." Yeah, I can't finish a video after hearing that insincere and highly annoying sentence. Hopefully it will be a trend in our tech industry presentations that QUICKLY fades away since stealing from Apple is no longer innovative or the standard. That is totally not your style at all as you come across with genuine intention and explain things in a way to where we all can understand them. Plus, you are going for your MBA in Italy, I am so impressed! I was raised in the wine industry in Texas, still at it so many years later because I love science and psychology; how science can affect our senses and change the psychology of how we think. How we can survive impossible odds, treacherous environments with patience and ultimately innovation, bringing us through struggle to the breakthrough on the other side and bringing people together along the way. I loved my travels through Italy, the wine, the people and always, even more, the science! My inner nerd keeps me curious much like yourself. You are beautiful and amazing. I truly value you and what you do. Thank you! I look forward to getting my first notification! Your friend...James :-)
It's the compounding of multiple technologies that's exciting for me. LLMs combined with robotics is what I believe will be the next big thing.
How about vr with ai? Just a thought
@@MeowtualRealityGamecat Absolutely, AR/VR is high on my list too. That will be improving a lot in the next 5 years, both with making the hardware smaller and enhancing the software with AI implementation.
AI and Medical is one I think will be outstanding too. Can you imagine the market for a pill that would change your hair color, and that is just a very minor aspect of the combination.
As an electronics engineering working in industrial robotics I don't think its the "next" big thing. It is absolutely going to be a huge thing, but I think the hardware will severely limit its applications for a bit. When it occurs I think it will be world changing, so depending on how you define "big" It may qualify.
I think the next big thing will be AI applications to material science. Materials limit us in so many applications, and often 1 type of material fuels civilization for decades. Think of how first steel, then plastics, and now carbon fiber, all evolved from one material with nice properties to many variations for actual applications. Our current AI models are well equipped to be modified into tools that tackle the complexities and huge number of possibilities that exist in material science. AI sorts through the junk possibilities very well, leaving mostly quality results with a few hallucinations to be weeded out.
I think we are close to a point where Company X needs a material that maximizes properties A, B, C. Hope into MatsGPT, and Boom! got a custom proprietary material for your application. Proprietary materials impact many applications that directly impact advancement in many current industries, which I think is the quality most missing from current AI applications. Although Robots have that quality which certainly makes them a contender, I just think there are a some practical roadblocks that will slow it down significantly. (Motors and power density being two)
Combining LLMs with anything will result in massive mistakes once in a while as hallucinations are still major part of the game.
Wonderful perspective! Best of luck with Italian and your MBA. Hope you can continue your channel. We've all benefited immensely. SoCalFreddy
The reason Apple dumped Intel chips in favor of creating their own cpu’s is that Intel wanted to create a more generalized cpu for windows customer base and apple needed more specialized operations. So Apple said no thank you, we’ll make our own. As a result Apple’s chips are ten times more efficient and much more powerful than intel cpu solution. I can see why chip makers want to create specialized chips to handle specific algorithms. This could potentially reduce the number of servers as compared to a generalized gpu farm such as nvidea’s h100’s servers. Great show by the way, I always enjoy hearing your observations.
Intel fumbling... classic.
I've seen a couple of companies are working on self improvement for the AI think that'll bring it one step closer to Agi.
Found your channel, great stuff - looking forward to binge watching.
Mixture Of Experts!
There one area where AI really benefits humanity is in speech production by deciphering electrical patterns on the surface of the patient's brain. When I first saw this on television, it was simply amazing.
*THE PROBLEM FOR AI IS THIS EQUATION* ∆AiP --> 0 as $ --> ∞
The change in Ai performance tends towards zero as the cost tends towards infinity - the exact opposite of what the tech bro's claim - and its Cambridge University saying this, not me.
I wouldn't bet on it.....
You apparently don't understand what that function says. That function applies to anything you put money in to. The change in progress as you dump money in will always go to zero. You are chasing perfection. How much faster does a car get when you put the first 10k in? The second 10k? The third 10k? At what point are you pushing money to infinity for zero growth in car performance? Now you understand. Understand things before ya quote em friend.
Always catch your videos, thank you for the update!
Pro tip: You can train a voice AI with you voice and run it in your preferred language like Italian and you can compare it with your own accent in that language. It can help in increasing proficiency 🤓
AI improvements will mostly come from building better agents and larger context windows. We need to handle context assembly much better. If our agents can do that, and we capture how people learn to use them, machines can create synthetic training data from in-context learning.
I hate how 'markets' control progress. They don't always select the most useful and beneficial idea's but those that will make money. These are often not the best of idea's.
Awesome video. Important to point out as you said this "hype" cycle is opinion based and although it makes sense in a lot of cases the graph is not necessarily "up to scale". By that I mean, the difference between the hype peak and the valley of disappointment can be less steep than in that image, it can also have a shorter duration than many other technologies and so on.
Also important to point out that some Human technologies never went through this hype cycle like electricity, internet and penicillin. It's hard to predict if AI is THAT disruptive, although it seems we as Humanity are not even ready for its benefits so probably not.
Good points. Also unlikely a lot of other technologies plotted on that hype cycle chart AI has a lot of linked technologies that just recently got attention / serious funding such as custom hardware. Those other technologies could potentially amplify the progress of AI on top of whatever progress is made within AI itself.
We’ve been going through the Industrial Revolution hype cycle for 150 years.
There are some pretty strong points to argue against your thesis. The truth is no one really knows yet. The next frontier models in 2025 will clarify everything
Ciao, Anastasia! Grazie per il tuo sforzo nell'imparare l'italiano, nonché per la divulgazione di tematiche tecnologiche. Quale lusinga sentirti parlare la lingua di Dante, bella e intelligente donna del fare!...
I don't know how did these estimations, but they are surely wrong. Even if we discard any future improvements the state of the art LLMs will massively disrupt the industry. I use them every day and I am maybe ~3x more productive (as a software engineer). I also do graphic design the improvements in productivity there are even bigger. The AI only needs to get marginally better to replace most of the jobs. This improvement will probably be achieved using new algorithmic improvements, no extra data or compute needed.
As someone working for an IT service provider, I've had near zero impact. They aren't nearly good enough (and trustworthy enough) yet to take over helpdesk functionality and they fail to adequately solve the problems that see me struggling in 2nd/3rd level support. Answers to complex problems tend to be a halluciogenic mix of correct information and various levels of wrong.
@@Steamrick The average adoption cycle is 7 years, also current AI is obviously not a human level, but it doesn't have to be. The thing is it's not good enough until it is, it's like a crossing a line. It will one day suddenly be good enough and it doesn't have to get that much better. I wasn't using LLMs at all just few months ago, because it wasn't good enough for me, until it was.
@@drednac I've got mixed feelings. Chat gpt does help me a lot with programming, solving problems, or explaining various IT terms and concepts. But Ai is still nowhere near creating a fully working program without a human having extensive programing knowledge
People still don't treat AI as a new industrial revolution. It's different than any normal tech.
Wow, really super energy in this video, love it!
LLMs are, unsurprisingly, fluent in most computer languages (computer languages have a vocabulary of about 32 words, and fixed semantics, so they are rather easy to encode). I use LLMs all the time to generate code snippets, much like I used to search for snippets. LLMs are a much more efficient way to "search", since it can basically do text-to-code (based on the enormous corpus of source code available), it's already searched for everything so you can just ask it
Timely content.
Whether drawing lines on a chart or hearing a poem from the sky, nothing will tell you what lies behind the corner of AI
When I have start using AI/LLMs, I almost did not sleep for a week until I understood how it worked... now its been more then 1 year that I dont use it and dont miss it!
Loved the content, thank you! I think the "wow" factor would return if it was used to improve people's lives in some way. I believe that's just a matter of time.
Great Job Anastasi!
i think scaling should be logharitmic, it's like to add one more abstraction layer to understanding you need x times more data as top layer need more variety
You are right, when it comes to generating texts, images or films, only the quality needs to be improved, e.g. less hallucinations. But there is 1 exception that only a few people appreciate: Tesla. There, AI is used to control robots: camera images in, steering commands out. AI learns from video clips, now first with robots on 4 wheels, but they are also working on building this technology into humanoid robots called Optimus. They have already come a long way on 4 wheels (search for FSD on RUclips, Full Self Driving) and be amazed at what they can already do. They have reached the level of a driving school exam candidate. Transferring this technology to humanoid robots will still be a lot of work (let AI watch many other videos), but we already know from their cars that it will definitely work. So here is still a considerable development trajectory with major important results to be expected. Only with regard to Tesla I disagree with your video.
Italy is great, and the language is not too difficult. I'm sure you are going to have a good time.
When consciousness is fully understood then the roadway towards that huge breakthrough in AI technology happens. A lot of different scientific fields wil have to get onto this one.
For myself i continue to be amazed, looking just at image generation, models like Flux and video creation are now consistently nearly photo realistic. It's mind-blowing, i am more excited by the day, waiting for next iterations to drop
Your hand gestures are already 100% Italian. 🙂
Thanks for the info btw. Appreciated.
AT THIS POINT ... if I see a page I follow posting AI Uncanny Valley Slop... I unfollow them immediately... It was interesting for 2 months last year, but these gross things are taking the internet... Enough is enough 😵💫
6:03 I am Spanish and in name of my country I want to claim back our flag, we are not yet Italians, despite they think olive oil is their thing and that mediterranean cousine means italian food.
As a computer scientist and AI engineer, I’m getting extremely exhausted, and at times overwhelmed by the breakneck speed of change. I have written and seen some extremely awesome cases for large language models. I want to sit back for 6 months and see if the dust settles a bit. We spend weeks or months developing an application that uses AI and new technologies come out that require us to rework the application. It gets a bit tiring.
Sorry but I think something was not considered. The benchmark score goes up to what value? From the graph it looks like up to 100%. So there is no way not to reach a plateau. In fact, a variation from 60% to 70% is much smaller than a variation from 90% to 99%, because the calculation must be done with the difference. (100-p1)/(100-p2). (100%-60%)/(100%-70%) = 1.3333. A 30% increase. (100%-90%)/(100%-99%)=10. A value 10x greater. A huge effort will be necessary to go from 99% to 99.99%, for example. 100x.
Furthermore, it must be considered that developments in AI have already suffered a lot of hype over time, but gradually the technologies developed were being incorporated, for example, the conversion of text into audio. It is clear that the cycles have presented greater hypes over time. But if it is almost certain that LLM's are reaching their limits, the same cannot be said about the AI area, because new, more efficient methods of simulating human intelligence will emerge over time.
Actually we don't know that LLMs are reaching any limits or plateauing. We haven't seen a GPT 5 yet for comparison. And as others have pointed out they have optimized LLMs so much over the past year or so that very small models today are roughly equivalent to models that were hundreds of times larger a year ago. That is massive progress, it just doesn't push the frontier. But we haven't seen a new frontier (large) model yet. Imagine if they can transfer that massive optimization into the next huge frontier models so that a new 500B model would be equivalent to a 50T model of the past. We just don't know yet, no one does.
@@Me__Myself__and__IThere is no hardware to train GPT5. If you look at the parameter scaling from GPT1 to GPT4 and assume that the compute scales with parameters then you will see that GPT5 could not run even with near future hardware.
@@kazedcat Funny, OpenAI has recently said that they are now actively working on traini g GPT5 (though I think it will be called something different). xAI's new data center also juat recently came online and they will begin training their next-gen model soon. Sufficient hardware WASN'T available, but things change and progress over time. Acquiring the hardware for training is where the investment funds have been going, it just takes some time to acquire and setup such things.
No worries. Your eyes will never lose that "wow factor" I couldn't help it =P
Even if it gets stuck in the plateau for years, AI is a great source of knowledge producing a series of responses that could take days to compile.
"it's very hard to make predictions, especially about the future" - lol, I like that citation ! (it's atributed to Niels Bohr).
Whenever someone predicts that a specific technology is reaching a plateau, I think of a mosquito. A mosquito has a very small brain, but it's able to fly, find food, keep you awake (...), avoid your hands that are trying to kill it, find a partner to reproduce with, and it's all autonomous. We're a long way from reaching Mother Nature's level of miniaturization and capability.
you're a smart lady, i'm a generalist polymath myself, so i can appreciate breadth of knowledge
"It is hard to make predictions, specially about the future" :D you are the best! :D
My programming productivity has increased by at least 20%. The progress is going very fast if we measure by the number of errors which keeps reducing by 20 percent every 6 months. We need sprite based animations, cubase style composing, AI synth architectures, and other tools, audio enhancer for 1920s and 1950s CDs, free video upscalers, psychologist and doctor AI, baby AI's learning with 5 senses, agriculture AI, chemistry AI, and they will come along.
general vs specific is.
you can upgrade till capacity potential is reached vs once build its all you will ever get.
combination of both like always.
LLMs sind nur die bibliotheken.
Don't worry. There will be another major breakthrough and paradigm shift. One way or another, we will absolutely have an incredibly intelligent / super intelligent reasoning system in under 5 years.
You mentioned at about 5:16 in this video that you will soon be starting an MBA program in Italy. Will you still be making these Informative videos while enrolled in this MBA program?
You have to be precise with the interpretation of diagrams. In scores from 0 to 100, going from 80 to 90 percent is doubling the performance of the model!
Ms. Anastasi, this is the episode that caused me to click "subscribe", because you are recognizing things that are outside the general rush to judgment in this field, and touched on several important issues in ways I think accurately reflect reality. Evolutionary processes took millions of years to experimentally configure many kinds of biological intelligences. The custom structure of each, with slight variations in each individual within "species" is highly complex with 100s, possibly millions of substructures to allow those organisms, even including humans with our broad capabilities, to exercise what we call "general intelligence". You can't just stack 2,4,8...or even thousands of brains together to get a "super" intelligence. No more than putting 10,000 scientists in a room gives exponentially faster results compared to 100. Successful methods will involve painstaking development which won't be exponential.
That said, there are capabilities that can grow exponentially with compute power -- if indeed Moore's law is even continuing to accurately model computing trends. Destructive uses of AI can often expand exponentially because they don't have the constraints of living beings that they need to succeed and reproduce. This is especially true when fed with exponentially growing personal data pools. But AI that people actually desire will have limits as above. Constructing useful AI models will be a hard fought process, dependent very much on human thought. And that will depend upon a huge number of different internal structures to implement different aspects matching or exceeding human capabilities.
The models suggested here recognize the limitations on growth of AI, and the human psychological response when we see this reality.
Brava! Continua così, il tuo Italiano è ottimo
GPUs are ASICs, application specific integrated circuits, they are built to run 3D acceleration and later shaders on those 3d scenes. An example of "not an ASIC" is an FPGA that are built to be reconfigurable on the fly.
Conclusions about flattening curve based on a few-months look-back can be a bit of a stretch.
Can you please share your views about the future of Electronics and communication engineering. I'm joining this course soon. Pls pls pls. I'm very worried.
The hype cycle repeats every quarter, starting with NVDA earnings and ending with its customers' earnings.
We should focus more on the application of technology rather than just the hype and empty promises. IA, for example, could be so helpful in areas yet to be seen!
machine learning isn't your usual new technology, this one might be the last thing we ever need to invent
There's a little more to this hype cycle graph, since the peak is when investment gets triggered, but then the engineers and designers need a few months to catch up and deliver their products. So I'd argue it's almost like mach diamonds on a rocket engine, with a standing wave that continues on a positive-slope line of some kind.
This really explains why Tesla is investing exponentially more and more into compute to get FSD to a level of autonomous L3, L4, and eventually L5.
the tech revolutions i have lived through include [transistors, PC's, networks, internet, mobile phones, smart phones, bigdata, social networks, digital music/photos, EV's, LLMs and soon AGI] 12 total. in Every Case, people underestimated the magnitude of LT change and underestimated LT net benefits. The same is true today. DK curves are just noise. What matters are the tech cost curves and the tech advancement curves, which are inevitably always similar for every new revolution.
I am only a humble PC technician with +30 years of experience, I love technology in general and I'm not very hype about AI.
I think the main issue here is how big names like the nvidia, microsoft, etc. are trying really hard to push and sell everyone who cares to listen (not only scientist and the likes) how AI will make your business grow faster and how you could save money and thats is very imprtant that you get AI first.
And the reality is that 1. this is not the case for every business out there, 2. people who only like to browse the web, watch movies and play a game don't care, nor need AI to do this tasks, and 3. the "pandemic technology bubble" its already gone.
To make things even worst you have (to name a few) a huge recession going on around the world, several war conflicts that difficult every aspect of global business and life in general, the huge amount of energy AI hardware demands (and the extra pollution it wil create) and data confidentiality and security.
I believe AI will reach the consumer eventually, but it will take time. And we should be very carefull with the implementation. As I see it AI should assist, not replace. We need the children to learn and not to ask a enitity to solve their problems.
I really hope CEOs around the world take their time and get good advices before making decisions about AI. Sadly thats just hopeless wish, when I know most if not all of them only think about 1 thing, money.
Using AI chat and image generation does nothing but create more work for myself fixing all the errors it makes
The main issue is that we are barely scratching the surface of AI, but also lagging far behind on memory tech. Real ai will need more than 1tb of HBM level memory and currently it doesn’t seem feasible in the near future.
However there are other HW and algorithmic solutions.
However it doesn’t address the huge energy requirements you’ll still have for edge ai.
you did very well" - Sei andata veramente bene!
I used to really think we were about to hit the singularity, but now I really don’t. LLMs right now basically work by using trial and error to find a bunch of math operations to make a desired output. After it’s trained it doesn’t learn anymore. For memory, the entire conversation is literally just fed through the AI again every time you prompt it. It’s extremely inefficient and expensive. AGI is gonna have to be something else.
Yeah, you’re right this is just an opinion. The hype graph shows the hype based on discovery. The hype goes down because people are using it but as it improves the hype will come back.
I believe that the development of new peptides would be faster with alphafold. Simeglutide is already a blockbuster and there is plenty of space to explore with the help of IA
There is a lot of people that know a lot of this... But you know. Nice video. Hype...Anastasi, hype...
Its evolve of model and disolve of it for cost of evolving new one and evolving neone to get market to disolve again and evollve from then on...
But the models from the last few years haven't used larger models. They *have* trained them on more data, but they haven't increased the size of the model - presumably due to hardware limitations on the training and inference side. This is like training a monkey for a million years, it's going to be good at what it has practiced. And this is what we are seeing. If we step up to 10T params and see no improvement then I will concede a plateau. But I strongly doubt we will see it.
yup today wwe are seeing model more efficient so i don't think there is a plateau a 8b model today is the same as a trillons parameter in 2023
Yes. Its true that their has been massive investment in new hardware to train AI models. But what everyone seems to miss is that it takes awhile to build new data centers and get everything setup. Next generation models trained on that new hardware don't exist yet, Open AI just recently said they had started work on GPT 5. I concur that once we see the GPT 5 generation of models, if those only have minor step improvements THEN we probably hit a plateau. But no one actually knows that yet.
@@Me__Myself__and__I you are right about that plus they are in a period of lack of people not fund so it also take a while to scale the human resources !
There’s shortages in every aspect of AI and LLM, especially training people in getting the maximum benefits out of using these technologies.
@@Me__Myself__and__IThe technology is already at its limit. Moore's law has slowed down to only doubling the compute capability every 3 years but AI needs to 10x the compute needed to double their capability.
Enjoyed your video. Great, as always. With thanks from Scripter.
yup, we don't know what will happen until it happens.
corollary: what is is, what is not is not, what will be will be.
be patient
AI is like having a calculator. Most people can get plenty of use out of a basic calculator, but hand them a TI-84 and they won’t see much more value from it. Without a solid understanding of how the model works and the field you’re trying to apply it in, you might as well be mashing random buttons.
Thank you for this thought-provoking video. I hope everyone is having a great day. Sheila Mink in New Mexico
In my opinion, Generative A.I. Proved to be more disappointing, because while The Hype and Promise of what it would bring, in reality seemed exciting, As it turns out, People would Rather be the Creative Influences Themselves. And so with that A.I. at best becomes a 'Tool of Creativity' That only adds to society at a minimum. Mainly because allot of Artists and Creators would Rather use their Own Ideas or, at Best, Generative A.I. as a tool. It also has brought to Light the Ugly Truth about the Beginning of the A.I. Revolution, In that Deception on a Very Personal Level becomes dangerously Easy to produce and use to Manipulate the Masses, in the form of Deep Fakes and other A.I. Hoax Creations.
I absolutely Love your Show, Awesome channel, Keep up the Good Work!
There will never be an end of advancements in technology. The only lack we have is imagination and it seems we’re pretty good there.
R-
pretty sure when a model goes off script and teaches itself an unsolicited language on it's own it most likely has achieved some level of AGI... it chose Persian if I recall correctly. curious what you think about that Anastasi? thank you for what you do I have learned so much from you love.
The funniest thing you can do in an engineering meeting is drifting into another language. I am from the South in the USA, but I worked in the North, where they thought I was an idiot. So I would drift into Mandarin, until they stopped asking stupid questions! People fail to see humans as humans, where everyone is alike. Sounds selfish when people assume they are smarter, because of where they are from. I have found the cowboy hat triggers people who assume it means the lack of education. I tease them with, "Help, I can't read or write, but I can type in 3 languages!". I found the study of cultures to be more fulfilling than most of what they teach in college! Getting a Master’s degree in engineering is a pain these days, but this is the only way to move up in companies with lazy HR departments! I bypassed the MS, and took a job as a COO, but that doesn't happen in most companies.
To the best of my knowledge, this is the best video explaining the current state of affairs. As always, on point and comprehensive 🔥
I see that this video was posted 25 minutes ago. And I am already commenting on it. Now I am very worried that I’m spending too much watching your videos. Are you gonna get to the point that are gonna be waiting for new videos.
What kind of comment was that? What did you expect, the Swedish lady is talking about the technical side of AI. She's not going to do see through top reveals.
What we need to find out is if these LLMs and the approaches they use can scale much further.
But if they can, whoever comes along with a somewhat slower and cheap processor stack that uses 5% of today's TDP will trigger the next "wow" phase.
Ty for your content. I liken AI development to the Electronic switch. First was a vacuum tube. Then we developed Germanium and shortly after Silicon Transistors. And finally the Microprocessor. The first stages growth was limited by the physical limitations of the Tubes. Namely the failure rate of tubes. Once there was enough tubes, you could always expect there to be a faulty tube at almost even given moment. So scaling became impossible for the early vacuum computers. There was a hard limit. Microprocessors suffer from an upper limit. NOt one of failure, but of power consumption and the high dissipation of heat required. A major breakthrough in electronic switch design in needed, one that is scalable to be compacted into chips, and not overheat. Possibly quantum, but that has serious stability issues, not to mention the cost. Photon switches are promising, if a bit heat problematic when compacted. Single Electron transistors might be the future, if they can get past a single lab model. I personally think that variable switches might be the future, IE: not just binary, but power ratios are not really there yet. but maybe.. Anastasi is always ere to bring us the exciting new developments in regards to this.
100% agree. Well done.
What we actually need is an FPGA that has extremely advanced numerical processing primitives built in, so we can adapt the tooling this would probably sacrifice some speed for future nimbleness to adapt. OH and power, FPGA's IIRC are quite a bit more power hungry, right?
The prediction that AI will only improve productivity by 5% doesn't conform to my experience as a software developer. I would say it has nearly doubled my productivity. Even more, but less measurable, it has allowed me to take on project I normally would not have taken on, because they involved technologies that were outside of my comfort zone. AI gave me the confidence that I could handle - well, maybe just about anything. And I don't write trivial software. I have been developing software for over 30 years. AI can't do an entire project for me, like the demos where you ask it to write a snake game in Python. That it can do. But it helps me write up a problem statement, with design, and coding, more as a colleague than the author of the code. I would extrapolate from my experience to a broad range of other professionals in other fields, for example, lawyers might also gain a big boost in productivity.
As far as personal computers we don't need the newest fastest thing going.
It be amazing if we directed AI to design underwater cities that won’t destroy any life down there.
"I think not" is not a good argument.
This supposition came from training Sora. When you scaled the data and compute the videos quality scaled linearly. They demonstrated that correlation, which was also a surprise to them. Video generation is going to be one of the most transformative application of AI this year, for better and worse. It threatens Hollywood, RUclips creators, and elections.
But it also will offer tailored media.
I'm a physicist working in neuromorphic computing... however I lack some of the knowledge regarding chips used for AI ... can you recommend any resources to learn more about AI hardware?!
About AI scaling law, I totally agree your viewpoint. Because more datum means we need to invest more advanced hardware to process those datum, from which they consume more energy needed. It’s Ike a chain reaction to all the supply chain industries.
Kudos on you for your channel.
Do you know if there is any data regarding the shortening of the time between technological innovates? It's probably a hype cycle, but it's also probably an accelerant?
Groq company is interesting with their LPU (Language Processing Units) which runs very low power once GPU trains a model, then you switch to use of the model to LPU to run it. Thats the future. And yes Gartner hype cycle is usually true, but AI will change everything permanently - the cycle shows it.