I also really liked the whole silver snow and blue water with crystals thing which *clearly* was a result of these being *unicorns* we're talking about. "Magical stuff", basically.
Using a billion data point based AI with an extremely complex understanding of the human language to finish my Voldemort, Harry, and Snape love triangle fan fiction...
Generating text is easy: In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. I am in the morning but I cant't find it on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen
I'm not sure how I feel about Rob Miles "in-jokes" getting so many likes. Maybe he'll give up his academic endeavors and start seriously monetizing RUclips videos. Instead of a stamp-making machine he might create a RUclips video-making machine!
This video is probably just the result of AI been given the entirety of RUclips videos as a dataset and being ask to make one in a Computerphile 'style'. Thus, freeing up time for Rob to work on the stamp collecting AI in the morning. Unfortunately, he couldn't find it on the phone screen on the phone screen. Tetracorns found to have alien DNA according to Perez, the South American professor. Right, we mentioned this several paragraphs ago.
Gostaria de ter que fazer uma daily sobre o Lenovo e percebi que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato
It's surreal to be re-watching all these videos only 3-5 years later, shortly after the public release of chatGPT. So much has changed in such a short span of time that it feels as if AI development is accelerating, or at the least AI development is finally nearing a tipping point where it has the potential to be the driving force behind the next technological revolution/paradigm-shift. I can imagine the level of technological and societal change brought about by near-future AI easily exceeding the change brought about by the discovery of electricity and the industrial revolution I can't properly describe the feeling of being able to witness and interact with the sorts of systems I've been dreaming would be possible ever since the mid-90's, but if I'm being totally honest, now that the sort of tech I was wishing for is here and becoming increasingly advanced, well, I'm a bit more worried about the existential risks than I ever thought I would be... not that I wasn't concerned before, but the risk of serious accident and/or misuse somehow seems more likely to me now than it did 5 years ago, and I suspect a lot of people are likely finding themselves in that same boat. I suppose that all we can really do is stay in the loop, hope for the best, and try super hard to NOT be the person who accidentally ends the world with AI.
IDK man, I don't think we have been watching the same videos. This particular vid is a great example of how language models work and how they can be immensely impressive but not AGI by any means. Even today in 2023 I think there is a lot of hype about.
I wish I had that thing to write my master diploma for me. Give it an abstract and it reasonably fills in the blanks with rambling no one really cares about.
In Standard English we would say writing a "thesis for my Master's degree". Considering you're having trouble even naming it in English; then yeah, you'd better get an AI. Or, at least, don't try to write it in English!
Dexxus researchers were afraid of the scope. Imagine spam with this kind of quality, uniquely addressed to you based on your social media profile. Or tons of plausible-looking fake social media accounts reposting auto generated news articles...
Computer scientists made a fan fiction AI. They then fed it a paragraph about English speaking unicorns and out came the article Rob reads in the video.
@@sth128 ah, anything with computer science I will click on and watch, but I have no idea what's going on here. I think that computer scientists are cute though
Summary: make an AI --> shove a bunch of articles in it --> AI analyzes how the articles are formatted (sentence structure, flow, syntax, etc.) --> AI then tries to make its own based on those observations. Results = unicorns.
A unicorn or a different one of the disabled people who to attack properly and I were not sure what that is the next one is the next one I bought a new portal frame and I were able to do better than I expected you to be in the last week and then I would love to see actually you have any of you guys are you still making it was a car my uncle had once upon a time I wanna go rallying and I were able to do better than I expected you to be in the last week and then I would love to see actually you have any problems with the sale of the disabled people you have any advice with regards to the contrary to the contrary to the contrary to the contrary to the contrary
*_...maybe it misspelled "forehorns"... We 'fortran'ed computer-generated-poetry back in the '70's e.g. "I sing a blue guitar large-eyed and bearded bronze but not a star..." pondering whether that was input because we certainly didn't read all the input... (We also implemented recursive-subroutine-calling, in Fortran)..._*
Instead of a stamp-collecting AI, Miles and Riley are working on a RUclips video-making AI. Complete with Cartoon Unicorn clickbait. This is what the AI apocalypse looks like!
It consistently stuck to a location in South America and gave the professor a Spanish name... And was able to point out that the unicorns speaking English was "surprising" to the researchers 😄
40 GB of text is indeed a extraordinarily large amount. My E-book collection is about 1GB, and that's with cover art included in the file sizes. Though, not every story has cover art, so the text of my E-book collection is probably only around 500 to 800 MB. And even though I really like science, I wouldn't be able to bring myself to read anywhere near 40 GB of scientific press releases. (I understand that those are only a small portion of the total list of website categories that it was trained on.) I would go nuttier than I already am.
And I'm sitting here at 1:37 and reading "turn that 211" in the captions and wondering... were the captions generated via this language model? If so... the model needs more training. But if they were done by a human, that human's model needs more training. Then imagine when the network does get BETTER results than the human... that's when things will really take off. Just look at what google is doing with that robocall thing... The future will be both awesome and scary. Robots, calling other robots, plotting the robot uprising. I welcome our eloquent overlords.
Doing a bit of research, the size data set is roughly equivalent 133,000 books if we assume the average book is 300 pages. Large for sure but we could absolutely go bigger. For reference the library of congress is holding 6 million books, not that we can digitize them really.
Build a useful AI: one that scans printed pages and CORRECTLY transcribes them into digital format, that would be a very useful AI. Alternatively let the state hire unemployed people to do the job manually...
I can't help wondering how many of those source articles were written by previous-not so successful-AI attempts, which someone found and posted on Reddit for a laugh ;-)
Makes you wonder exactly how much information a human needs to do something like this, but also how do humans learn so quickly? Surely we haven't managed to absorb 40GB of information to come up with something like that. I'm pretty sure we don't have to repeat train on the same thing hundreds of thousands of times either. It would be great to be able to find the answers to these questions.
@@wktodd Well I could make up fairy stories by age 6/7, maybe not with detail of south America, but still, I definitely hadn't read 40GB of text by then. My guess is we do stories from memory, rather than directly from short/medium term past, although that _might well_ give current context. Pick out relevant memories and convert to text/speech whatever - memory seems to be discouraged in Neural nets (overfitting). I can see that GPT-2 is hopeful of storing everything in one model, but I have doubts - the capsule network idea seems more realistic. Is it possible to pick out single words in order, ignoring language rules at first and then use simpler rules to string them together into an intelligible sentence? I'm Sure I've seen a couple babies do this - but they had to learn the rules over quite some time. The attention seems to do the language rules very well, but randomness and statistical word choosing??? Even when writing this, I had to choose non-statistically, or at least as far as I was aware...I had to consider if it was understandable...and words needed changing so that things were made 'softer', can you work out where I originally placed the word 'sensible'... grammar not so much. :-/ ).
@@threeMetreJim Ah but you can assimilate data by sight, sound, smell , and touch, your parents , siblings , friends all helped to feed you data, that is far richer than plain text . So, by 6/7 you would have absorbed and analyzed (although not stored ) far more information than in 40gb of text.
@@NortheastGamer but if I were to be accurate I would add a second (maternal) surname. To be fair it's usual for it to be omitted in non formal situations.
4:22 to add perspective Each letter is a byte So the amount of letters in a word is that amount of bytes Apple = 7 bytes 1,000,000 bytes = 1 megabyte 1 gigabyte = 1000 mega bytes 45GB of text data 4500 megabytes 4,500,000 bytes And thats counting all punctuation
I must eat my hat now, because I claimed, that you can't understand the language without some knowledge about the real world, and now this thing seems to get some understanding about the real world just from correllations between words.
Why would you ever say that. Your brain understands the language just by correlations between words. This was 30GB. You brain is Petabytes. You're just a bigger model than this.
How is the ability of an AI to create somewhat plausible bogus text useful? (except for nefarious purposes) I would want my AI to meaningfully respond to things I ask it about real life situations and these scenarios don't have me convinced that's possible. The ability to create legible text does not, to my understanding reflect an ability to actually understand what I might say and respond as reasonably. This setup doesn't seem aimed at making an AI understand anything, just produce plausbile stuff without any meaning behind it. Am I wrong?
An AI such as this can be repurposed for a variety of uses, in its current state though it is limited but the most useful is perhaps using it translate text into other languages in a manner more realistic to how we speak. One could also theoretically make a video game in which the dialog and story is different for everyone or even just realistic character dialog. Each character could theoretically have a true individuality. One big use for this is that it can allow robotic reporting in conflict areas around the globe. The accuracy that it currently has Is fairly remarkable and if it was made to generate text about on the topic of war then we could have drones with converting the AI's text to speech and allowing a more human and realistic way of reporting in war zones or dangerous areas, live and without risk to a reporter.
@@mrosskne in this case maybe the ability to correctly conextualize arbitrarily? I'm not an expert on epistemology or neuroscience but surely there is a difference between being able to surround words with other words in a plausable way based on the entire corpus of text available on the internet as a reference and being able to take a concept and apply it in a different contexts, which to me is one (of possibly many) indications that understanding has taken place. seeing how e.g. chatgpt has done on physics tests (see recent computerphile video) this kind of understanding does not seem to take place.
does that thing would end up giving good aproximations of the future? give it history books, and increase the amount of data as you get closer to the present, gdp, investments,migrations, etc.... pretty sure it will destroy the stock market already.
It has seen some code, and if you feed the small and medium models stuff that looks like code, it will produce stuff that looks like code (but which usually doesn’t compile)
@@drdca8263 Might have to do with the fact that most of stackoverflow code doesn't compile. Either because that's what brought author to stackoverflow in the first place or because those code snippets are not complete compilation units.
I mean it does kind of make sense, it could simply be referring to specific group of four unicorns that were "horned", meaning of possession of a horn. Maybe all the other unicorns had theirs taken by poachers or something.
@Robin Powell You have a point - ...but not out of the blue. If some or almost all unicorns had their horns taken, that is such a detail that no writer would forget to mention.
the average person, if they saw a horse with four horns on its head, wouldn't call it a quadricorn. they'd call it a four horned unicorn. because human speech isn't rigorous.
"While their origins are still unclear, some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, 'In South America, such incidents seem to be quite common.'"
I think in future, an advanced version of this could be used by detectives to find clues for their investigations. I think it is really possible because of several reasons.
In a generation there will be a website somewhere with something like this writing research papers for students in high school, and teachers will never know the difference. This might even just render the high school research paper unnecessary.
Sooo sick.. Makes me question if intellegence and creativity are just math..? Sure, in some way they are signals in our biological 'neural net' but its impressive to see a computer model get produce so realistic results..
I used the app and started off with the phrase "The greatest discovery of the twenty-first century" and got this. It got stuck in a loop with the most likely prediction so I varies the choices. I feel like it could provide much more variety of detail even if the content is not helpful. Maybe it could help someone get an easy C grade for an essay. "The greatest discovery of the twenty-first century is that the human brain is not a single entity, but a collection of interconnected brain cells. The brain is a complex network of interconnected brain cells. The connections between these cells are not just physical, but also chemical, electrical, and chemical-like. The chemical and electrical properties of these cells are determined by the chemical and electrical properties of the surrounding tissue. This means that the brain is not a single entity, but a collection of interconnected brain cells."
"The greatest discovery of the twenty-second century is that humans are not alone in the universe. We share a common ancestry with other life forms, and we are all descended from a common ancestor who lived millions of years ago in what we now call the primordial soup. Our origins lie in the primordial soup. If we are to understand the origins of life, we must understand the primordial soup itself and what makes it unique from other life forms."
![The moment we stopped understanding AI [AlexNet]](/img/1.gif)
I also really liked the whole silver snow and blue water with crystals thing which *clearly* was a result of these being *unicorns* we're talking about. "Magical stuff", basically.
+
Using a billion data point based AI with an extremely complex understanding of the human language to finish my Voldemort, Harry, and Snape love triangle fan fiction...
Harry Potter and a Stone.
Imagine. You could leave it running overnight and wake up to thousands and thousands of Harry Potter slash fiction novels. Humans have been replaced.
Lol
Hucrux, the lost nose
Game of Thrones could have ended so much better...
Generating text is easy:
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
I am in the morning but I cant't find it on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen on the phone screen
I'm not sure how I feel about Rob Miles "in-jokes" getting so many likes. Maybe he'll give up his academic endeavors and start seriously monetizing RUclips videos. Instead of a stamp-making machine he might create a RUclips video-making machine!
This video is probably just the result of AI been given the entirety of RUclips videos as a dataset and being ask to make one in a Computerphile 'style'. Thus, freeing up time for Rob to work on the stamp collecting AI in the morning. Unfortunately, he couldn't find it on the phone screen on the phone screen. Tetracorns found to have alien DNA according to Perez, the South American professor. Right, we mentioned this several paragraphs ago.
Gostaria de ter que fazer uma daily sobre o Lenovo e percebi que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato de que o estresse é o fato
Maybe ramp up the softmax temperature? Might help with the looping
They can't find find the actual letters on the phone screen keyboard so they are trying to say something using just auto-suggestions.
Where it all began
Seriously, super interesting seeing all the comments, feels like reading 1950s predictions of the future. So much has changed
Just a few years later we have GPT-4
I want a Spotify to offer a service where I train it with my EEG output and it generates songs that always give me chills.
RyanFromUltrasound that’s actually a really intriguing idea...
What if it becomes too effective and you get chilled to death?!?
From the title, I thought this might be about a type of AI that's so rare, that it's a mythical creature.
Seeing how it wasn't released it may as well be!
triton62674,
A point to you.
It's surreal to be re-watching all these videos only 3-5 years later, shortly after the public release of chatGPT. So much has changed in such a short span of time that it feels as if AI development is accelerating, or at the least AI development is finally nearing a tipping point where it has the potential to be the driving force behind the next technological revolution/paradigm-shift. I can imagine the level of technological and societal change brought about by near-future AI easily exceeding the change brought about by the discovery of electricity and the industrial revolution
I can't properly describe the feeling of being able to witness and interact with the sorts of systems I've been dreaming would be possible ever since the mid-90's, but if I'm being totally honest, now that the sort of tech I was wishing for is here and becoming increasingly advanced, well, I'm a bit more worried about the existential risks than I ever thought I would be... not that I wasn't concerned before, but the risk of serious accident and/or misuse somehow seems more likely to me now than it did 5 years ago, and I suspect a lot of people are likely finding themselves in that same boat.
I suppose that all we can really do is stay in the loop, hope for the best, and try super hard to NOT be the person who accidentally ends the world with AI.
The world ending because of AI has a large probability based on the vastness of the search space and the small number of cases AI could be aligned.
IDK man, I don't think we have been watching the same videos.
This particular vid is a great example of how language models work and how they can be immensely impressive but not AGI by any means.
Even today in 2023 I think there is a lot of hype about.
Now waiting for news article in guardian
"Computer scientist in UK confirms existence of unicorns in South America"
A day later: Dr. Perez missing!
And with all of this incredible technology, humans will put it to use generating clickbait articles
I believe that's the exact reason why they didn't release the fully trained model.
@@jasonschuler2256 rrtehr
Clickbait requires sofistication dont you think?
I wish I had that thing to write my master diploma for me.
Give it an abstract and it reasonably fills in the blanks with rambling no one really cares about.
In Standard English we would say writing a "thesis for my Master's degree". Considering you're having trouble even naming it in English; then yeah, you'd better get an AI. Or, at least, don't try to write it in English!
Tbh let it write a rough draft for you, then go back and polish it
RUclips: computerphile uploaded
Me: go away Im sleeping
RUclips: it's Rob Miles
Me: *instant click*
Then what?
That shitpost was clearly generated... Hello GPT2 we know it's you.
Indeed.
, I'm*
RUclips: Well I know whom *your* crush is then.
This bot should be unleashed upon r/WritingPrompts
You can find a fine tuned model bot with Writing Prompts on the r/SubredditSimulatorGPT2
Awesome. This system can generate everything I *hate* about popular media coverage of scientific discoveries!
It's almost like this can be used to create news articles or tweets...
almost?
That’s why they didn’t release the full trained model
And? It's already rather easy for people to do that.
Dexxus researchers were afraid of the scope. Imagine spam with this kind of quality, uniquely addressed to you based on your social media profile. Or tons of plausible-looking fake social media accounts reposting auto generated news articles...
They invented the ultimate fake news writer.
Anything with unicorns I will click on and watch but I have no idea what's going on here. I think that unicorn is cute though.
Anything with Rob Miles I will click on and watch but I have no idea what's going on here. I think that Rob Miles is cute though.
Computer scientists made a fan fiction AI. They then fed it a paragraph about English speaking unicorns and out came the article Rob reads in the video.
@@sth128 ah, anything with computer science I will click on and watch, but I have no idea what's going on here. I think that computer scientists are cute though
Summary: make an AI --> shove a bunch of articles in it --> AI analyzes how the articles are formatted (sentence structure, flow, syntax, etc.) --> AI then tries to make its own based on those observations. Results = unicorns.
@@JamieAtSLC uh oh, Rob's personal AI has developed feelings for him. we had better contain it
>unicorns are aliens
IT KNOWS TOO MUCH
woke!
isn't this the one that they said it was too dangerous to publish the full program publicly?
Yep. We talked about that aspect of things as well so it will probably be in a future video
@@RobertMiles2 thanks for making videos, machine learning is really a fascinating subject.
And all that does is to encourage someone to try and equal, or better it.
@@threeMetreJim the human race is unstoppable
Damn them, my conference article is due tomorrow. I though I just found a solution on how not to make this an all-nighter...
"A unicorn the same way as to the same way as to the same way as to the same way as to the same way" says predictive text on my phone.
"A unicorn can get us all together after the fight is done and they are doing it for the best"
"A unicorn is a service from Riot Games Player Support Specialist Riot Games Player Support Specialist Riot Games Player Support Specialist..."
A unicorn or a different one of the disabled people who to attack properly and I were not sure what that is the next one is the next one I bought a new portal frame and I were able to do better than I expected you to be in the last week and then I would love to see actually you have any of you guys are you still making it was a car my uncle had once upon a time I wanna go rallying and I were able to do better than I expected you to be in the last week and then I would love to see actually you have any problems with the sale of the disabled people you have any advice with regards to the contrary to the contrary to the contrary to the contrary to the contrary
now imagine if your phone was one hundred billion times more powerful
If Skynet becomes self aware reading /r/Brony then Judgement Day is virtually assured.
Your values will be satisfied through friendship and ponies.
It will only watch all seasons MLP on loop, no worry.
Something I never expected to see.. an Unicorn talking in a Computerphile video. My life is complete now!!!!
If nothing else AI will bring us lots of laughter. At least we will go extinct with a smile...
People: programming must be so dangerous, you might be arrested for hackering Nasa...
Computerphile: _Unicorn AI_
2:50 This is also how it works with people. Go figure.
I have the entirety of Wikipedia downloaded and that's only 70.5 gigabytes on my laptop so 40 is pretty gigantic!
that’s quite cool. why? do you update it from the live website?
Well this is all very interesting, but why is Rob in prison today?
Our AI overlord found out he was designing security systems for AIs
He threw three doubles
His stamp collecting AI tried to conquer the world
@@abcdefghier Ah, yes. He built that rogue AI that caused the world-wide stamp shortage of 2019.
roko's basilisk is retroactively punishing him in the past from the future
*_...maybe it misspelled "forehorns"... We 'fortran'ed computer-generated-poetry back in the '70's e.g. "I sing a blue guitar large-eyed and bearded bronze but not a star..." pondering whether that was input because we certainly didn't read all the input... (We also implemented recursive-subroutine-calling, in Fortran)..._*
Instead of a stamp-collecting AI, Miles and Riley are working on a RUclips video-making AI. Complete with Cartoon Unicorn clickbait. This is what the AI apocalypse looks like!
omg hes so cute, I love nerdy guys. I bet he talks like this all the time and no one knows what he's talking about but I do.
I like the fact now (year 2023) that we're at GPT-4 , all what these commenters were asking for have come to pass.
It consistently stuck to a location in South America and gave the professor a Spanish name... And was able to point out that the unicorns speaking English was "surprising" to the researchers 😄
If this had been implemented in games to create random quests, ohh my lord I would never get bored :)
All of the quests are protect the client like RE 4's Ashley
All your base are belong to us
Your values will be satisfied through friendship and ponies, and it will be completely consensual.
Underrated1 Fabulous idea! I'd be down for that!
daggerfall mod now
Well now GPT 3 is released and I'm watching this video again
Another video so soon? Mr. Miles you spoil us
the world's smartest philosophical zombie.
it's not a zombie.
I can see why the full-parameter model of GPT2 hasn't been released
and now 3 years later there is GPT-3...
*3.5
*4
And now we have GPT-3, which makes GPT-2 look tiny by comparison.
The SEO ramifications alone are frightening.
40 GB of text is indeed a extraordinarily large amount. My E-book collection is about 1GB, and that's with cover art included in the file sizes. Though, not every story has cover art, so the text of my E-book collection is probably only around 500 to 800 MB.
And even though I really like science, I wouldn't be able to bring myself to read anywhere near 40 GB of scientific press releases. (I understand that those are only a small portion of the total list of website categories that it was trained on.) I would go nuttier than I already am.
That's pretty much the point of machine learning. Our brain cannot process as much information as quickly as a computer or remember as much of it.
And I'm sitting here at 1:37 and reading "turn that 211" in the captions and wondering... were the captions generated via this language model?
If so... the model needs more training. But if they were done by a human, that human's model needs more training.
Then imagine when the network does get BETTER results than the human... that's when things will really take off.
Just look at what google is doing with that robocall thing... The future will be both awesome and scary.
Robots, calling other robots, plotting the robot uprising. I welcome our eloquent overlords.
This is the earliest I've been to a Computerphile video
Me too!
May I borrow this AI for writing my English papers?
GPT-2 now looks like a joke compared to GPT-4. Now imagine how GPT-9 or something will be.
Talking about Unicorns openly. Yet it becomes unhinged once aliens are thrown into the mix.
Well the prompt was unicrons so it had to build from there
my my, chatgpt-3 sure is a step up from this.
This is getting surreal. They need to give back to reddit and create a subreddit where it continues the most upvoted user submitted story once a day.
Not exactly the same, but check out /r/SubSimulatorGPT2
I think I read about that on Reddit.
Will it recognise itself?
the subject was unicorns but i was totally convinced by the generated text it's so scary
Funny watching a warewolf reading about unicorns.
Welcome to the Marvel - My Little Pony Extended Universe
The bot writes better than I do!
Wow first time I learned what gpt stands for.
Global Partition Table? :P
generative pre-trained
Doing a bit of research, the size data set is roughly equivalent 133,000 books if we assume the average book is 300 pages. Large for sure but we could absolutely go bigger. For reference the library of congress is holding 6 million books, not that we can digitize them really.
Build a useful AI: one that scans printed pages and CORRECTLY transcribes them into digital format, that would be a very useful AI.
Alternatively let the state hire unemployed people to do the job manually...
I can't help wondering how many of those source articles were written by previous-not so successful-AI attempts, which someone found and posted on Reddit for a laugh ;-)
Makes you wonder exactly how much information a human needs to do something like this, but also how do humans learn so quickly? Surely we haven't managed to absorb 40GB of information to come up with something like that. I'm pretty sure we don't have to repeat train on the same thing hundreds of thousands of times either. It would be great to be able to find the answers to these questions.
Oh I suspect that while growing up , humans absorb much much more than 40GB , and over a lifetime ???
@@wktodd Well I could make up fairy stories by age 6/7, maybe not with detail of south America, but still, I definitely hadn't read 40GB of text by then. My guess is we do stories from memory, rather than directly from short/medium term past, although that _might well_ give current context. Pick out relevant memories and convert to text/speech whatever - memory seems to be discouraged in Neural nets (overfitting). I can see that GPT-2 is hopeful of storing everything in one model, but I have doubts - the capsule network idea seems more realistic. Is it possible to pick out single words in order, ignoring language rules at first and then use simpler rules to string them together into an intelligible sentence? I'm Sure I've seen a couple babies do this - but they had to learn the rules over quite some time. The attention seems to do the language rules very well, but randomness and statistical word choosing??? Even when writing this, I had to choose non-statistically, or at least as far as I was aware...I had to consider if it was understandable...and words needed changing so that things were made 'softer', can you work out where I originally placed the word 'sensible'... grammar not so much. :-/ ).
@@threeMetreJim Ah but you can assimilate data by sight, sound, smell , and touch, your parents , siblings , friends all helped to feed you data, that is far richer than plain text . So, by 6/7 you would have absorbed and analyzed (although not stored ) far more information than in 40gb of text.
Seems very close to passing the turing test?
It does better than some humans...
Great channel with great question and answers.
I am so impressed.
This could solve writers block for thousands of people.
This could make writers obsolete
They primed for the unicorn story.
This is frightening, Jorge Pérez is the name of one of my computer science professor at my university. He specializes in deep learning and AI.
Anderson Cooper is interviewing the unicorn leader tonight
you're correct: Jorge Pérez = Horr-Heh Peh-reth (like George Peterson)
actually quite popular Spanish name and surname.
Did it basically pick the Spanish equivalent of John Smith?
@@NortheastGamer close. That would be Juan Pérez.
@@NortheastGamer but if I were to be accurate I would add a second (maternal) surname. To be fair it's usual for it to be omitted in non formal situations.
@@aliedperez Juan Herrera?
@@Turalcar that can work too :)
Yay more robert miles
4:22 to add perspective
Each letter is a byte
So the amount of letters in a word is that amount of bytes
Apple = 7 bytes
1,000,000 bytes = 1 megabyte
1 gigabyte = 1000 mega bytes
45GB of text data
4500 megabytes
4,500,000 bytes
And thats counting all punctuation
Don't you mean 4500*1,000,000(Bytes). 45GB = 48,318,382,080 bytes exactly.
Got some festival wristbands on, Rob?
I must eat my hat now, because I claimed, that you can't understand the language without some knowledge about the real world, and now this thing seems to get some understanding about the real world just from correllations between words.
Why would you ever say that. Your brain understands the language just by correlations between words. This was 30GB. You brain is Petabytes. You're just a bigger model than this.
@@marsovac how did you go about measuring the memory capacity of the brain?
I think this demonstrates more than anything else the paucity of actual intelligence in academic press releases.
most things humans write are largely generic and interchangable. there's nothing unique about academic press releases.
How is the ability of an AI to create somewhat plausible bogus text useful? (except for nefarious purposes) I would want my AI to meaningfully respond to things I ask it about real life situations and these scenarios don't have me convinced that's possible. The ability to create legible text does not, to my understanding reflect an ability to actually understand what I might say and respond as reasonably.
This setup doesn't seem aimed at making an AI understand anything, just produce plausbile stuff without any meaning behind it. Am I wrong?
It can be used as an inspiration for story tellers.
An AI such as this can be repurposed for a variety of uses, in its current state though it is limited but the most useful is perhaps using it translate text into other languages in a manner more realistic to how we speak. One could also theoretically make a video game in which the dialog and story is different for everyone or even just realistic character dialog. Each character could theoretically have a true individuality. One big use for this is that it can allow robotic reporting in conflict areas around the globe. The accuracy that it currently has Is fairly remarkable and if it was made to generate text about on the topic of war then we could have drones with converting the AI's text to speech and allowing a more human and realistic way of reporting in war zones or dangerous areas, live and without risk to a reporter.
what is understanding?
@@mrosskne in this case maybe the ability to correctly conextualize arbitrarily? I'm not an expert on epistemology or neuroscience but surely there is a difference between being able to surround words with other words in a plausable way based on the entire corpus of text available on the internet as a reference and being able to take a concept and apply it in a different contexts, which to me is one (of possibly many) indications that understanding has taken place. seeing how e.g. chatgpt has done on physics tests (see recent computerphile video) this kind of understanding does not seem to take place.
That is eerily realistic... very neat!
does that thing would end up giving good aproximations of the future? give it history books, and increase the amount of data as you get closer to the present, gdp, investments,migrations, etc.... pretty sure it will destroy the stock market already.
That's basically Psychohistory. We'll see, oh yes, we'll see....
Scary and fascinating at the same time. I wonder how this model would perform in generating multiple coherent pages
"attention" only is not enough for rich modelling. We also need memory units, for example LSTM-like gated units
Thinking that the end of humanity will be caused by conscious, rougue AI seems too self confident now.
They might as well have released it, anyone that knows tensorflow and has the GPU resources could re-implement it.
Feed it code.
It has seen some code, and if you feed the small and medium models stuff that looks like code, it will produce stuff that looks like code (but which usually doesn’t compile)
A lot of reddit links are to stackoverflow
@@drdca8263 Might have to do with the fact that most of stackoverflow code doesn't compile. Either because that's what brought author to stackoverflow in the first place or because those code snippets are not complete compilation units.
That "four horned unicorns" made me truly laugh - out loud - a while! 😂
A four-horned unicorn will beat a single-horned quadricorn any day of the week.
I mean it does kind of make sense, it could simply be referring to specific group of four unicorns that were "horned", meaning of possession of a horn. Maybe all the other unicorns had theirs taken by poachers or something.
@Robin Powell
You have a point - ...but not out of the blue. If some or almost all unicorns had their horns taken, that is such a detail that no writer would forget to mention.
the average person, if they saw a horse with four horns on its head, wouldn't call it a quadricorn. they'd call it a four horned unicorn. because human speech isn't rigorous.
This is scarily impressive
This guys voice is addictive
What is crazy is that this thing can generate RUclips comments or even 4chan posts
I had a go at generating RUclips comments on my second channel: watch?v=M6EXmoP5jX8
some machine un-learning may be needed before it can generate convincing 4chan posts
so can my toilet
@@mrosskne in hindsight that wasn't really a big achievement, for such a task braincells are optional
Please make a video about Bert and XLnet as well
"While their origins are still unclear, some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, 'In South America, such incidents seem to be quite common.'"
It was trying to tell us that AI has mastered communication through human language... It was a warning.
Intelligence seems nothing more than an emergent property of extreme data processing.
I think this means some of my code comments have ended up in the GPT-2 training data.
Time to rename GPT2 to Optimus Prime.
Gee! What is the probability of news agencies already using this text generator? I'd guess it's pretty high.
I think in future, an advanced version of this could be used by detectives to find clues for their investigations. I think it is really possible because of several reasons.
What do you think of it now?
In a generation there will be a website somewhere with something like this writing research papers for students in high school, and teachers will never know the difference. This might even just render the high school research paper unnecessary.
we already have systems that can reliably predict if text was ai generated
to whomever has drawn the unicorns: I see you had a different type of horn in mind.
That's scary and fascinating at the same time.
Maybe it could produce some interesting Kōan ...
Checkmate, bloggers.
Omg... Dont fade out... What else can it do
Sooo sick.. Makes me question if intellegence and creativity are just math..? Sure, in some way they are signals in our biological 'neural net' but its impressive to see a computer model get produce so realistic results..
intelligence and creativity are patterns of atoms.
It's like written by a 12 y.o. kid with a good brain and lots of imagination. Really impressive!
10:08 best part
You can tell that it's a fake news article by the fact that it didn't use clickbait.
This is basically what a brain is. More data, more accurate.
I used the app and started off with the phrase "The greatest discovery of the twenty-first century" and got this. It got stuck in a loop with the most likely prediction so I varies the choices. I feel like it could provide much more variety of detail even if the content is not helpful. Maybe it could help someone get an easy C grade for an essay.
"The greatest discovery of the twenty-first century is that the human brain is not a single entity, but a collection of interconnected brain cells. The brain is a complex network of interconnected brain cells. The connections between these cells are not just physical, but also chemical, electrical, and chemical-like. The chemical and electrical properties of these cells are determined by the chemical and electrical properties of the surrounding tissue. This means that the brain is not a single entity, but a collection of interconnected brain cells."
"The greatest discovery of the twenty-second century is that humans are not alone in the universe. We share a common ancestry with other life forms, and we are all descended from a common ancestor who lived millions of years ago in what we now call the primordial soup. Our origins lie in the primordial soup. If we are to understand the origins of life, we must understand the primordial soup itself and what makes it unique from other life forms."
That is using the smaller publicly released model, right?
I wonder if it could continue "Can it run crisis?"
Do we know how many similar texts are in the dataset? Maybe it just quoted existing paragraphs with small substitutions, and mixed them together.
add to this the "1% rule (Internet culture)" and we're actually the neural net being trained