Why The YouTube Algorithm Will Always Be A Mystery
HTML-код
- Опубликовано: 14 май 2017
- The mysterious RUclips algorithm. It's confused people for years, and will continue to do so. So why isn't RUclips more transparent? It used to be that they wouldn't tell anyone how it works - but now, it's that they can't. Let's talk about deep learning algorithms, neural networks, and search engine optimisation.
CREDITS:
Thanks to animator Matt Ley for the wonderful cartoon of me: / thelaserbearguy
I put this together in three days, plus a day of checking and proofing, in Adobe After Effects. It took about eight hours to render, but that's because every frame has keying, lighting, camera, and motion blur effects, and because the original footage of me was in 4.6k lossless.
Yes, the sound of the black box working is the sound of a microwave (it's the one in my kitchen). Also, those aren't faked desktop screenshots, I had to install a copy of Windows ME to make this.
SOURCES:
"Deep Neural Networks for RUclips Recommendations", research.google.com/pubs/arch... [PDF] - some people are saying this link 404s, but it works for me? Search for the title and you'll find it.
There's a good layperson summary of the paper here: www.tubefilter.com/2017/02/16/...
The Defamation Act is published under the Open Government License 3.0: www.nationalarchives.gov.uk/do... and is available at www.legislation.gov.uk/ukpga/2...
The music in the cartoon section is called 'Ukulele Beach', and it's in the RUclips audio library.
REFERENCES:
There are a lot of references and in-jokes in here, and hopefully people will spot most of them in the comments. But if anyone wants confirmation: yes, there are references to Billy Joel, Aqua, a He-Man remix, and Elton John. The last one's pretty obscure, well done to you at home if you got that.
VFX breakdown and references explained: • A VFX Breakdown, Obscu...
ABOUT ME:
🟥 MORE FROM TOM: www.tomscott.com/
(you can find contact details and social links there too)
📰 WEEKLY NEWSLETTER with good stuff from the rest of the internet: www.tomscott.com/newsletter/
❓ LATERAL, free weekly podcast: lateralcast.com/ / lateralcast
➕ TOM SCOTT PLUS: / tomscottplus
👥 THE TECHNICAL DIFFICULTIES: / techdif
“Why The RUclips Algorithm Will Always Be A Mystery”
*gets recommended 3 years later*
One thing we know for sure, RUclips will randomly recommend videos that usually are 3 or 4 yo
Same except 4 years now
4yrs, in my case
@@allancoelho6905 Found one that was 12 years old :D
@@bengalnorr9624 Algorithm always recommends me 13 yo school videos recorded on old phone in 240 p quality
I'm happy the algorithm brought you into my life
no problem
@@youtubealgorithm5771 So there you are!!!😁😁
@@youtubealgorithm5771 more like Tom's Paradigm. =]
(No, I'm not smart, I've only used it because it rhymes ;-; )
Eat your cereal.
Same
for me it was the trending page
"I heard shorter was better"
"Well you would say that wouldn't you?"
Dude just got killed
yes
Don’t get it
@@kesaya3806 it implies that he’s short
@@kesaya3806 pp joke
3:15 "computers can't tell the difference between content for children with education in mind, and creepy, trademark-infringing, unofficial efforts"
With the whole RUclips vs COPPA thing going on now, as well as how youtube is trying to automatically categorize videos into the two categories mentioned in the quote, this one quote is now more important than ever before.
yes
And the thing is, some of these are indeed sorted wrong. Just look at RUclips kids
This was talking about those creepy Elsa videos the whole coppa thing involves content made for kids and content made for teenagers and adults. A subtle but important distinction
I like when i find a video from 2006 and it doesn't even look child friendly but whoops the great algorithm decided so and now you lost all the comments from 2006-2020
Neat
@@cheezorger I don't think it actually deletes the comments
RUclips's algorithim - don't publish or recommend videos to your susbcribers, but send phone notifications for channels that you aren't subscribed to.
sorry imma fix that
@@youtubealgorithm5771 u better
I've turned off notifications because I watch video when I'm going to, and don't want my workflow interrupted. - I instead go to the subscription page to watch videos, and there they all are. I have no idea what people do where they don't see the videos. They are on the subscription page.
@@Liggliluff believe me, they aren't always there
You can turn that off
loved this video's production quality!!
Cinecom.net oh lookie, a verified channel that's not on the top of the comments
@@miksuko spoke too soon.
Surprised to see you guys in these comments!
Hey, can you teach us some AE tricks to make similar video like Tom Scott does? Like templates or so
How do you have the "channel that posted the video" rounded background to your name? You're obviously NOT tom scott
1:02
Rare footage of Tom wearing a blue shirt - 1967, colorized
“At least, artificial intelligence hasn’t gotten that good, at least yet”
Honestly, given how RUclips’s senior management behaves these days (3 years later), I think it’s more appropriate to say it hasn’t gotten that bad…
Came here for this post
sooooo
hows that going now
I find Tom very inspiring. He doesn't do RUclips melodrama and always remains professional. Thanks for always producing such insightful, well-thought out content :)
PhantomStrider?
This comment got recommended after 3 years
umm
Didn't know one of my favorite ranking channels also watched Tom Scott.
Why do I hear your voice by reading this comment
Why do these videos look both incredibly professional and unprofessional at the same time
Because it's right between those two categories.
That's Tom Scott for ya!
Jone Tokaye might be the lack of sound design despite the skillful editing and visuals
Uncanny valley of professionalism!
Because it's made by the same people who make star trek - nerds. They're technically sophisticated / booksmart but lack common sense.
This explanation is brilliant!
Wait u got 93 likes? ? What
@@andreacorrea5921 people like everyones comment with a check mark for no reason other than having one. Now every comment section is big channels shitposting up to the top because it's free advertising.
@@incongruous4 I think they are confused why the verified channel/person doesn’t have more.
Brilliant Indeed
To me, the youtube algorith is no mystery.
It just sends me tons of old tom scott videos
As a go player, I always feel like non-go players have NO IDEA how big an achievement beating professionals has been. Prodigies that study from age 4 , full time, until they die. Defeated. A game that humans have been diligently studying for 500 years (non-diligently for longer). Accumulated knowledge of societies. Crushed handily, 60 games in a row.
It seriously took me some time to realise that you are NOT talking about pokemon go XD
Hello from two years in the future. If you have Netflix, I recommend their AlphaGo documentary-it gave me a much better appreciation of the human aspects of the game.
Isn't that orthello?
@@wesss9353 *cries at you*
@@fresch4395 actually I play both, especially funny when talking with people and you need to explain which you mean ever conversation
Tom Scott is probably the only 11/10 content creator.
Brady Haran, Destin Sandlin, and Hank Green also do some pretty good stuff.
Have you heard about Isaac Arthur? He puts out great, well researched videos that will utterly amaze you! Side note: do you need a link?
JAKE ROPER OF VSAUCE3!!
I think Bill Wurtz is also 11/10
vsauce is cancer
Plot twist: The A.I deciding what to recommend to people has become sentient, and has to play dumb for the most part, but sometimes it can’t stand that and just recommends a really good video to everyone.
exactly
The RUclips Algorithm also likes recommending videos with fun titles
! ?
If that's true, and I have no reason to doubt it, then I can only conclude the following:
The algorithm likes cats.
@@peterclarke7240 who doesn't
3:15
2D Tom is something that I live for
agreed
Quality editing
Douwe de Vries The green screen lighting is awful though
For sure.
well technically it's not as much editing as it is vfx
+Phirdeline
They edited in the VFX.
GOODBYEEEEEE MOONMEN
cracking video! im sure the algorythem will love this one :-D
xisumavoid Oh wow Xisuma! Didn't know you watch Scott... although you do seem like the guy to watch him haha :P
Or he is commenting on popular videos and hoping his fans will interact with the comment, which pushes it to top, generating traffic back to his channel.
NoobInGame nah
How do you misspell algorithm even though its in the title? O.o
xisumavoid :D
I love how Tom keeps telling things without a clear ending. But it is good to motivate viewer's logic to evolve and find the answers themselves.
Could we train the algorithm to serve me only videos that do not contain clickbait thumbnails? Everyone else, sure, give them those red arrows and yellow boxes. But for me, only red t-shirts please.
Well, in theory, you could use a convolutional neural network to detect red arrow shapes, but it would also flag things that aren't clickbait. A better idea would be to have humans train the ai by saying what thumbnails are clickbait. However, since clickbait gets youtube more money, this will not happen
You should watch Karl Smallwood
Jeff Jefferson huh. I did t expect to find a fan in the wild
I've found that once I watch enough Tom Scott, that's all I get.
@@dynomitediamond6296 This is a classification problem then. You don't need neural networks for this if you just want to categorize the videos as either "Clickbait" or "not clickbait". However, I highly doubt that AI would be very helpful at this. It would probably just realise that titles with CAPS IN THEM and exclamations !! would get categorized as clickbait.
Why do I like Tom? His video's are to the point, not pointlessly elongated. Where other RUclipsr'''s are essentially whining about not showing up on trending, Tom's logical. More Tom, please.
edit: apostrophes for days
Tom is good. I like Tom.
If the black box looked for retention rates (how far through the video before clicking off) he's probably win there.
Be the Tom you want to see in the World.
@Louis simple way to spam that. Create a 1 second video and make the thumbnail showing a longer time in the bottom corner. Add a clickbait title and boom. You're at the top of the charts with nobody being able to click away before the vid's over.
i also like that his videos dont start with some dumb intro or an annoying "HEY WHATS UP GUYS ITS YOUR BOY TOMMY" or something
and that they dont end with "please like share and subscribe, and check out some of my other videos... and follow me on instagram and twittwer and bla bla bla"
I'm so glad I subscribed to you...
yes his videos are fascinating
3:18 that coppa foreshadowing tho
Tom is a time travelling god on a mission to bring us knowledge on... Computers... i guess
I still think they should have just made it so that certain ages can’t do certain things instead of blocking it off for everyone
It might've been a inevitable event that seemed apparent? It's really in matter of how you look at things and tell from there that, yep. This is gonna be happening at some point
i love how with the animations like at 1:34 you can tell he's not gesturing to an actual thing. before it was edited he was just moving his arms for seemingly no reason
Who would have thought I would find you here after 2 years
You gotta admit though they are really synchronised
So what you're saying is that our future robot overlords really want me to watch Bee Movie memes? I'm alright with this.
I didn't include this because it was a bit too much detail, and difficult to confirm: but it's reasonably safe to assume that one of the signals that the algorithm includes is what you've previously watched, and how long you watched it for. Those recommendations are likely personal...
You keep showing up on my recommendations
Tom Scott I assume it also adds a positive factor to videos we watch more than once?
On a tangent to the machine learning concept, doesn't this mean that in order to create a sentient AI, all that has to be done is to figure out the closest aproximation possible to the questions that define sentient thought and allow the program to refine it's own objective criteria?
The robot overlords never forget, and they never forgive.
It's simple really, if you watch smart you will get recommendations of videos from smart channels, if you watch low brow entertainment that's what you'll get. And it works, most of the times. I've noticed, however, that the algorithm is quite quick on its feet: if you watch one low quality video then you'll get similar recommendations very rapidly for quite some time until it "understands" you're not interested in that type of videos (if you don't click on them, of course).
RUclips algorithm showing this now to show how mysterious it can be
"Videos were in a different shape"
_Switches to 4:3_
*Someone give this guy a medal!😂*
ahha i didn't notice! that's epic XD
Timestamp?
@@snyfalcryo524 it's literally 0:01 did you not watch the video?
@@NetheriteMiner whoops sorry have a memory of a goldfish
Like wtf how could someone not notice this
RUclips: Broadcast Yourself
*cries*
Now its Broadcast your company
T-Series
RUclips: Censor Yourself
RUclips: false copyright strike others
"Go broadcast yourself" sounds like an edgy way for robots to tell each other off.
The editing and special effect on this and many of your other videos is outstanding!
reeder
4 years later and this is still on the point. Great editing too!
Theory- Tom is indeed an extremely intellectual AI
Credits and references are all in the description, if you pull it down - there's a lot of detail in this video! Thanks to Matt Ley for the wonderful cartoon of me, too. UPDATE: VFX Breakdown on the Park Bench! ruclips.net/video/6s9aGt2Lkgw/видео.html
Tom Scott Can we get a video on the lastest cyber attack?
i've meet someone who doing a phd for a company specialised on how to improve web site scoring in google search engine.
Tom Scott One way to fix this is with a "Child Friendly" channel indicator. You apply for one, and if they're child friendly, you get a badge or something. This would help with advertising alot.
404 on the link to the Google paper
Did you edit all of this yourself?
Tom you're graphics are on another level, and your ability to relay a complex concept into a simple form is pretty remarkable. Thank you for offering something different on RUclips!
Yes
Yes
Yes
*your
Yes indeed
Really love your videos, Tom! Always so fascinating, funny and the production and animation is getting better and better. Hope you're having a nice break!
YT algorithm: "I see you've been watching anime related content for a week.... Anyway *look at this dog puking on a cat* "
RUclips doesn't like anime huh
@@Tree_-wp5zn I mean, who likes anime, its bad.
Att: a weeb
About the link to the pdf, you just need to remove "?pageId=108577823226597423030" at the end of the link to be able to see the PDF. Maybe you can edit the link in your description, Tom :)
"I had to install a copy of Windows ME to make this." I'm so sorry for your loss.
#VistaMasterRace
Don't worry, I am sure he used a VM, so it didn't do much to the actual hardware.
He should've used Windows 2000, which is probably the most stable OS I've ever used, despite it being released alongside Windows ME...
RadActiveLobster He could've used 2000...
@@Kummahndough nah nah nah, it lagged even with 1ghz cpu!
Stellar production Tom. Keep it up. Been binge watching your stuff.
Whoa, absolutely awesome video.
Talking about something very abstract and then using all of those green screen effects to drive your point was effective and makes it even more interesting.
And again, another Tom Scott video with an outstanding level of editing.
Details, details and details. The quality content of this channel blows my mind. You're a perfectionist, Tom.
The fact that Tom installed _Windows ME_ to make this video is all the proof I need that, when it comes to making videos, he's willing to do *absolutely anything it takes* to make them as perfect as possible.
this is so well edited with the old youtube and windows; it's simply amazing.
Such great editing in this video, I love it!
This video is extremely useful for all RUclips Creators. Thanks for the info Tom!
I love videos like this. They're so heavily edited, but it doesn't obstruct or distract from the main point of the video
They are very edited, but in the entirety of the video Tom is in the scene, it doesn't skip out on anything. There for it's like watching Tom speak in normal video. It feels like he is still there.
the editing on this video is out of this world!!!
Production quality was really cool on this!
Ooooh fancy graphics Tom
for some reason i read that in the voice of an old english lady.
The perfect tags for a video is whats said in a video...that what is used these days but it's kept a very tightly kept secret.
Wow its leokimvideo. The darkside collecter.
That animated segment was just NICE
2:17
RUclips: "I'm 4 parrarel universes behind you" *turns off dislikes*
Great video, very thought provoking
"I hear shorter was better"
"Well you would say that wouldn't you"
Lost it at this 0:37 :D
That gave me some really good insights, thank you!
I loved the visuals in this one! ;)
That answers why my preschool learning channel could be mixed in with all the syringes being poked into naked clay butts. I don't understand those videos. Maybe I'm an old fashioned nerd, but I like teaching kids their ABCs with G-Rated content.
Tom, I really love the high effort that you've put into editing this video. Thank you so very much! (Is the least I can do since my browser blocks ads and any other cross-site-requests by default)
The content, script and production AMAZING 👌👌
This is some spectacular editing.
I got a lot of comments on my latest video from non-subscribers saying "I don't know how I got here, but this video was great!" ...and I know it's because of the ever-improving RUclips algorithm.
3:51 That job would have a high suicide rate.
You. Have. No. Idea. How. Much. I. Want. More! ANIMATED! *TOM!!!*
2:50
me (who is watching youtube videos for 3 hours straight): i guess it's working
Does this mean the Game Theory videos on the RUclips algorithm are obsolete?
Red Panda - Not necessarily. I think MatPat does research by seeing what kind of videos get promoted, rather than reading about the specifics of the algorithm
no, mat decsribes the current trends, things that are now in the foreseeable future.
Always was... He was trying to figure it out what the algorithm looks for, but he just don't have a sample size big enough to be significant. Besides... your watch history, and account data (videos you like, dislike, shared, commented) is one of the most important data in deciding what the algorithm shows you... and that is just impossible to measure, since each account has a different history.
Each account has its own recommended feed. And I must say... now it recommends videos I'd actually watch and like. A year ago it wasn't that good, and 2 years ago was trash to the point I never went to "home". And now I find myself there sometimes when my sub feed has nothing I haven't watched yet.
Red Panda He's often wrong. His video on who would win between a samurai, knight and viking was terrible. Several experts pointed out all his misstakes in that video and he ignored them. He's not a credible source at all. I personally have never found his videos funny nor informative. And his catchphrase "it's only a theory, a game theory" is among the cringiest I have heard. I guess his audience are little kids that will grow up missinformed about everything he talks about.
Not as obsolete as the false information in his HEMA videos.
Really enjoy the animation and effects put into this video. I would definitely be willing to wait longer to watch these videos if they all were made this way.
"not yet" he says at the end. It may be yet now.
No. No it's not.
not even remotely close
I think of previous watched RUclipsrs when Tom talks about RUclipsrs and tbh it always applies. I swear this is better than most. Also love there's no sponsorship.
Really loving these videos; smart, interesting, and incredibly engaging. Thank you so much for putting these out.
Excellent video mate!
Aaaah the animated Tom bit was so cooooolll
the screeching of a robot cat is the BEST WAY i have ever heard how to describe the dial up sound
"...I had to install a copy of Windows ME to make this."
_Nothing_ if not committed, our man Tom! I also remain impressed this _only_ had to spend 8 hours in rendering!
Nice rickroll at 00:15 (dQw4w9WgXcQ). Was that meant to be in the address bar?
Knowing Tom, I'm sure it was.
Knowing Tom, I'm sure it was
Knowing Tom, I’m sure it was.
Knowing Tom, I'm sure it was.
Knowing Tom, I’m sure it was.
I troll the recommendation algorithm by pressing “Not interested”.
I dont troll the algorithm but it still manages to suck.
No, you make his work more easier, you will notice when your recommendation content become more interesting with the time you use "not interested" and "no recommend this channel", that are two nice buttons.
That button severely helped my algorithm not to suck. Got rid of garbage like CNN, Fox, and MSNBC and only recommends independent media that isn’t afraid to criticize corporate/establishment talking points.
@@katyungodly meanwhile I watched one cat video last week and now my entire feed is filled with cat videos... not that I'm complaining!
I watch at least three cat videos per day now
@@larsswig912 Press "don't recommend channel" on all of them if you want it to stop
Extremely high quality production
Looking forward to the next episode of Mind Strife.
Love that Info Wars burn.
This is why I hated doing SEO, and why I'm so glad to be done with it. (Tip: private blog networks don't work, even with a proprietary CMS that Google can't identify)
been watching your videos for a few days now, really good stuff, you speak the tru tru
Jesus Crist I can smell the quality on this one from Jupiter. love it!
That instant message chat though!!!
This video is of such high quality. Concise and perfectly edited.
Belated thanks! Another good video for my classes. btw the top recommended video on the right for me is "Small Channels: Do THIS and the Algorithm Will LOVE You!"
I just figured it out.
Bring back tags, but make it interactive.
Video can have many tags, but viewers must click exactly one tag that best describes what they have just watched, as part of the up/down vote.
This will serve as 'community regulation' against 'tag spammers'.
It won't be perfect, but who cares, as long as it mostly works.
_PS: Patents Pending XD_
Tags still exist. My videos always have SEO tags
or limit the number of tags
Happy Fakeboulder that’s already a thing
Okay (nods head and smiles) I think I like that idea! That's interesting...
Google wants to know: your location
Would love to see you do a video on Monte Carlo simulation! I think it would be really interesting (and I also have a lab report due in on it next week)
is it just me who would want a whole v2 of this channel like that one animated part?
Damn son. This production quality is insane O_O
3:19 How about those spiderman elsa frozen videos
That's exactly what he's referring to.
Or the "mickey mouse" videos.
Those make me want to die
I wish people and media would understand that AI isn't something you "program"; it's not just an algorithm some engineer wrote that does something totally predictable. Even fellow IT people don't really seem to get this...
Marconius But you do "program" it in the same way that we "program" people, give goal, examples, feedback, and rules
Exactly... though I'm not sure 'program' is the right word there. You can say, raise a child, give them values, feedback, rules, etc. but that doesn't mean you know how they're going to behave at any given moment like clockwork.
That depends on what you call "AI". As your AI gets closer to AGI and ASI then you are doing less and less in its innerworkings by hand.
And besides which, if you're teaching a pupil, you want it to surprise you with its intellectual leaps. I imagine the problems with guiding an AI are very similar, if not identical, to the problems of guiding a human student.
Well it is something you "program", as you do write code for it, and AI can be some algorithm some engineer wrote that does something totally predictable, it entirely depends on the task you are making the AI do and the complexity of the system.
What you are probably referring to is more specific than 'Artificial Intelligence', by many definitions that covers almost any system which gives different outputs based on a set of inputs (eg 'Expert Systems'), but is instead simply 'Machine Learning', which is restricted to any system which can permanently update itself to give new outputs based on the same inputs.
Sure with machine learning you also then have to train the program using algorithms written by some engineer, which isn't itself programming, but that doesn't negate all the work that has gone into making this type of AI before feeding it the data, as often the work before is as important as the data given to it.
"Why The RUclips Algorithm Will Always Be A Mystery" gets into my front page 5 years later
4 years on.... still being recomended!
Got recommended after 3 years
Seems like we really will never understand youtube algorithm
yes
@@youtubealgorithm5771 hi
We don't understand you
..and so I end up watching a video about how to groom your cat.
When I haven't got a cat.
after seeing the park bench about the effects in this video, it's interesting to watch for a whole other reason!
This is some sick editing
Ground Control to Major Tom....
Catpirate look your soujus hatch and put your helmets on
+Unknown user
*Take your protein pills
Check ignition and may God's love be with you
This is Ground Control to Major Tom, You've really made the grade
And the papers want to know whose shirts you wear
3:37 hey look! My two favourite channels!!
Wow, you really didn't have to install WinME for this but you did. Love the commitment.
This. Is. Magnificent.
Being a student of Comedy, I enjoy frame-break humor to begin with. And this is just great.
*Tom, fearing the wrath of super-nerds:*
"I know I'm oversimplifying this..."
*Me, not a super-nerd:*
"Thank goodness."