Transformers, explained: Understand the model behind GPT, BERT, and T5
HTML-код
- Опубликовано: 8 июн 2024
- Dale’s Blog → goo.gle/3xOeWoK
Classify text with BERT → goo.gle/3AUB431
Over the past five years, Transformers, a neural network architecture, have completely transformed state-of-the-art natural language processing. Want to translate text with machine learning? Curious how an ML model could write a poem or an op ed? Transformers can do it all. In this episode of Making with ML, Dale Markowitz explains what transformers are, how they work, and why they’re so impactful. Watch to learn how you can start using transformers in your app!
Chapters:
0:00 - Intro
0:51 - What are transformers?
3:18 - How do transformers work?
7:41 - How are transformers used?
8:35 - Getting started with transformers
Watch more episodes of Making with Machine Learning → goo.gle/2YysJRY
Subscribe to Google Cloud Tech → goo.gle/GoogleCloudTech
#MakingwithMachineLearning #MakingwithML
product: Cloud - General; fullname: Dale Markowitz; re_ty: Publish; 
Наука
Ability to break down complex topic is such an underrated super power. Amazing job.
Transformers! More than meets the eye.
😂
Transformers! Robots in disguise!
Autobots wage their battle to fight the evil forces of the Decepticons!!!!!
Transformers! No money to buy…
Oczywiście
How did you condense so many pieces of information in such a short time? This video is on a next level, I loved it!
Great explanation of the key concept of position encoding and self attention. Amazing you get the gist covered in less than 10 minutes.
@Dino Sauro tell me more...
@Dino Sauro thanks for the heads up
She has one of the wealthiest company on earth providing her resources. First hand access to engineers, researchers, top notch communicators and marketing employees.
@@an-dr6eu True, but this young lady talks a mile-a-minute from memory. She's knows it cold regardless of the resources at Google.
@@michaellavelle7354 Her explanation is absolutely useless. Have you ever programmed a Transformer model from scratch to verify what she has explained?
This is awesome. This has been one of the best overall breakdowns I've found. Thank you!!
You have the gift of making things simple to understand. Keep up the good work 🙏
Amazing video! 🎉 You explained that difficult concepts of Transformers so clearly and made it easy to understand. Thanks for all your hard work!🙌👍
Are you serious? The concepts were not really explained. Just a summary of what they do but not how they work behind the scenes.
No.
This is a GREAT explanation! please lower the background music next time it could really help. thanks again! awesome video
This is a really awesome video! Thank you so much for simplyifying the concepts.
This is such an informative video about transformers in machine learning! It's amazing how a type of neural network architecture can do so much, from translating text to generating computer code. I appreciate the clear explanations of the challenges with using recurrent neural networks for language analysis, and how transformers have overcome these limitations through innovations like positional encodings and self-attention. It's also fascinating to hear about BERT, a popular transformer-based model that has become a versatile tool for natural language processing in many different applications. The tips on where to find pertrained transformer models and the popular transformers Python library are super helpful for anyone looking to start using transformers in their own app. Thanks for sharing this video!
i really enjoyed the concepts you explained. simple to understand
Thanks you did a great job. I spent some time already looking at different videos to capture the high level idea of what transformers are about and yours is the clearest explanation. I actually do have an educational background in neutral networks but don't go around remembering every details or the state of the art today so somebody removing all the unessesary technical details like you did here is very useful.
So easy and clear to understand. Thanks
I loved it and very simple ,clear explanation.
Amazing video! Nice explanation and examples 😄👍
I would like to see more videos like this and practices ones
Where is optimus prime?
He's on the thumbnail...
He will be in theaters in June 9... Transformers : Rise of breasts..
😂😂😂😂
Where are robotaxis?
We got lied to
This has to be the best explanation so far, and by a very large margin.
Thank you for watching! We appreciate the kind words. 🤗
Dale you are so good at explaining this tech, thank you!
Love the content and thanks for the great video! (one thing that might help is lower the background music a bit, I found myself stopping the video because I thought another app was playing music)
Charm, intelligence and clarity! Thanks!
Love how you simplified it. Thank you
It s so simplified that you can t understand anything
Nice amount of info parted in this video. Very clear info on what Transformers are and what made them so great.
Wow, this is so well explained.
Excellent presentation and explanation of concepts
I love how to simplify something so complex, thank you so much Dale, the explanation was perfect
how did you do that
@@asstimus-prime This one? Just type ":" (colon) followed by "thanksdoc" and end it with another colon. I can add other emojis like 🤟too!
@@nahiyanalamgir7056 it needs desktop RUclips i think
@@asstimus-prime Apparently, it does. When will these apps be consistent across devices and platforms?
@@nahiyanalamgir7056 thanks though
Thank you so much. I really needed this video, other videos were just confusing
Fantastic!. Thanks for simplifying the concept
Such a simple yet revolutionary 💡idea
This is an excellent video introduction for transformers.
This was a really, really awesome breakdown 👏🏾
That's a really good high-level explanation!
This is one of the best vids I've watched on this topic!
The visuals are very helpful. Thanks.
You're very welcome!
Positional Encoding, Attention and Self Attention. That's it! Really well summarized.
This is a very well produced video. Credits to the presenter and those involved in production with the graphics
Thanks for your hard work.This video is very helpful!!!
Hi Google! First of all, thank you for this wonderful video. I'm working on a multiclass (single label) supervised learning that uses Bert for transfer learning. I've got about 10 classes and a couple hundred thousand examples. Any tips on best practices (which Bert variants to use, what order of magnitude of dropout to use if any)? I know I could do hyperparameter search but that'd probably cost more time and money than I'm comfortable with (for a prototype), so I'm looking to make the most out of my local Nvidia 3080.
You have no idea how much time I potentially have saved just by reading your blog and watching this video to get me up to speed quickly on this. "Liked" this video. Thanks
Very well explained.. This really is a high level view of what Transformers are, but it's probably enough to just get your toes wet in the field!
so super helpful for my thesis, thank u
super well done. Thanks for this!
crazy how things have changed so much
thank you! I'm just starting to learn about gpt and this was quite helpful, though I will have to watch it again :)
Very interesting, informative, this added perspective to a hyped-up landscape. I'll admit, I'm new to this, but when I hear "pretrained transformer" I didn't even think about BERT. I appreciate getting the view from 10,000 feet.
Amazing explanation!
Thanks Ma'am. You broke it down well.
wow, what a great summary! thanks!!!
I knew little on transformers before this video. I know little on transformers after this video. But I guess in order to know some, we'll need a 2-3 hours video.
Very well explained. Thank you.
Informative! Thank you
Very good lecture, thanks!
Easiest to understand explaination ive heard so far
Takeaways:
A transformer is a type of neural network architecture that is used in natural language processing. Unlike recurrent neural networks (RNNs), which analyze language by processing words one at a time in sequential order, transformers use a combination of positional encodings, attention, and self-attention to efficiently process and analyze large sequences of text.
Neural networks, Convolutional neural networks (for image analysis), Recurrent neural networks (RNNs), Positional encodings, Attention, Self-attention
Neural networks: A type of model used for analyzing complicated data, such as images, videos, audio, and text.
Convolutional neural networks: A type of neural network designed for image analysis.
Recurrent neural networks (RNNs): A type of neural network used for text analysis that processes words one at a time in sequential order.
Positional encodings: A method of storing information about word order in the data itself, rather than in the structure of the network.
Attention: A mechanism used in neural networks to selectively focus on parts of the input.
Self-attention: A type of attention mechanism that allows the network to focus on different parts of the input simultaneously.
Neural networks are like a computerized version of a human brain, that uses algorithms to analyze complex data.
Convolutional neural networks are used for tasks like identifying objects in photos, similar to how a human brain processes vision.
Recurrent neural networks are used for text analysis, and are like a machine trying to understand the meaning of a sentence in the same order as a human would.
Positional encodings are like adding a number to each word in a sentence to remember its order, like indexing a book.
Attention is like a spotlight that focuses on specific parts of the input, like a person paying attention to certain details in a conversation.
Self-attention is like being able to pay attention to multiple parts of the input at the same time, like listening to multiple conversations at once.
Great, you learned how to copy paste
@@an-dr6eu first step on becoming a programmer
@@an-dr6eu your comment comes over somewhat 'catty' 😢
OMG the BEST transformers video EVER!
Simply loved it!
Excellent explanation i ever seen, recommending everyone's this link
Simplest Explanation ever
Soo cool! Great work
I have more respect for Google after watching this Video. Not only did they provided their engineers with the funding to research, but they also let other companies like OpenAI to use said research. And they are opening up the knowledge for the general public with these video series.
Nicely done. Very helpful. Thanks!
Great video. Thank you!
Thanks! This is a great intro video!
very well explained.👍
Great video for people who are curious but don’t really want to (or can’t) understand how transformers actually work.
Very informative video. Thank you!
Amazing video, thank you so much!
NICE SUPERB PRESENTATION
Thanks! Great video.
great video, thanks!
a very nice video. thanks
Super Explanation!!
Thanks, that was very interesting
It was funny and instructive. Thanks 🙂
phenomenal video
Please remove background music, it's really disturbing when you only listen to this otherwise great video
Thank you
do transformers learn the internal representation one language at a time or all of them at the same time? I remember that Chomsky said that there's no underlying structure to language and that for every rule you try to make you'll always find an edge case that contradicts the rule.
Very well explained. This video is must watch for anyone who wants to demystify the latest LLM technology. Wondering if this could be made into a more generic video with a quick high-level intro on neural networks for those who aren't in the field. I bet there are millions out there who want to get a basic understanding of how ChatGPT/Bard/Claude work without an in-depth technical deep dive.
Great video.
woww, she's good at explaining things
Thank you for sharing
Good(Pro) Explanation.
Very impressive video. Thanks for the way you shared information via this video.
Reference your video timeline 05:05, how you created such a video, please.
Great video. Thx.
Wowww….thanks for clarifying my confusion.
Great content 👍
this is brilliant
Fantastic video
Well written script. Appreciated.
Thanks a lot.
Amazing!
From 5:28, shouldn't it be the following:
"when the model outputs the word “économique,” it’s attending heavily to both the input words “European” and “Economic.” "?
For européenne, I see that it is attending only to European. Please let me know if I am missing something here. Thanks for the great video.
When I saw this title, I was hoping to better understand the mathematical workings of transformers such as matrices and the like. Maybe you could do a follow-up video explaining mathematically how transformers work.
thank you for your time
I wish they don't embed music on the background, it makes harder to follow the conversations.
When I was a kid, I knew the trouble of translation were due to literally translation words, without contextual/ sequential awareness. I knew it's important to distinguish between synonyms. I've imagined there's a button that generate the translation output then you can highlights the you words that doesn't make sense or want improvement on it . then regenerate text translation. this type of nlp probably exist before I program my first hello world (+15y ago)!
Positional encoding = time, attention = context, self attention = thumbprint (knowledge)... looks like a good start for AGI 😀
Thank you!
10/10. Very helpful
Well done explanation.
You have actually given the BEST explanation on Neural Machine Translation that I read so far but you are missing a few elements
But your explanations, your analyses and your delivery are excellent. You're definitely a great communicator and teacher.
Actually Google and others have an algo they're not interested in sharing and I pretty much know what it is. I am working with my programmer on the coding of my new app, the revolutionary Universal Sentence builder and the Universal Dictionary and I keep adding and changing stuff to simplify the concept and I push at a later date the programming of my Sentence Analyser app. It is like most of my apps a simple (and brilliant concept) coded with very few lines of code.
You know Alfred Hitchcock was always adapting into the screen his scenario never changing anything not even a comma while Francis Ford Copolla (The Godfather) was doing the opposite: They say that his script was like a newspaper that had new contents every day. Well I am more like Copolla with my apps. I change stuff all the time and I usually make my programmers go crazy. It's a good sign. :-) Mind you I don't know if one can do like Hitchcock with an app. Come up with a definite version once and for all. This would be quite an achievement!
In the case of my Universal Sentence builder, the main task was to process the data entered by the user and we've been at it since July 2022. :-) It's either I am dumb or it is a complex task. Actually it is the latter for I have started with French, this langage being the most complex in the world. The good news is I am sure I will be imitated but you can rest assured that my imitators will also have a jolly hard time with French :-)