@@Socratica It's fun when you hear of similar analysis done to uncover ghost writers or shared authorship. Shakespeare, Rowling, and The Federalist Papers all come to mind.
That's a very intuitive and helpful explanation, thank you. But pray tell, prithee even, is not some relationship between words in individual sentences what we would prefer (smaller angels)? It seems odd to me that when creating embeddings we're focused on these huge arcs rather than the smaller arcs that build understanding on a more basic level. The thresh-hold for AI in GPT 3 seems to have been on a huge amount of text, but isn't there some way to make that smaller? For most of us, that's the only way we can even contribute, as we just don't have the computer-hardware.
I don't like removing "stop words" from the statistics, because their frequency is still meaningful. Even though everybody uses the word "the" frequently, some use it much more than others; and that is some characteristic that should not be ignored. So rather, I would suggest performing some kind of "normalization"; like dividing each word count by the average occurrence rate of that particular word in natural language. Instead of just word counts, the vector coordinates will consist of relative use rate of the particular word in the book compared the average use rate in general language. That would make a much more precise comparison. Because not just "stop words" are very common, some words are inherently much more common than others. Although I did not make the experiment, I suspect that in this way, everything will have a much lower cosine similarity.
pretty good, but the visualization of the results could have been made in something other than a table. That way, you wouldn't have to explain why the diagonal is 1, and that every number appears twice (mirrored along the diagonal). You'd end up with just 45 rather than 100 datapoints, and then compare the "top 10" across the different measurements. This would be much easier to follow.
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
𝙄𝙣𝙩𝙧𝙤𝙙𝙪𝙘𝙞𝙣𝙜 𝙎𝙤𝙘𝙧𝙖𝙩𝙞𝙘𝙖 𝘾𝙊𝙐𝙍𝙎𝙀𝙎
www.socratica.com/collections
Fantastic video, great to see another Socratica video in my feed
Keep em coming, the courses are looking good too.
This is phenomenal! Here I was, thinking we were just going to talk about the cos a = a approximation in trig. Bonus!
It was a fun surprise to learn about this technique 💜🦉
@@Socratica It's fun when you hear of similar analysis done to uncover ghost writers or shared authorship. Shakespeare, Rowling, and The Federalist Papers all come to mind.
So cool, being able to find similarities in books from neighboring time periods was fascinating.
It really makes us curious about a lot of the more recent writers-can you use this to find out which older writers influenced them!
That's a very intuitive and helpful explanation, thank you. But pray tell, prithee even, is not some relationship between words in individual sentences what we would prefer (smaller angels)? It seems odd to me that when creating embeddings we're focused on these huge arcs rather than the smaller arcs that build understanding on a more basic level. The thresh-hold for AI in GPT 3 seems to have been on a huge amount of text, but isn't there some way to make that smaller? For most of us, that's the only way we can even contribute, as we just don't have the computer-hardware.
I don't like removing "stop words" from the statistics, because their frequency is still meaningful. Even though everybody uses the word "the" frequently, some use it much more than others; and that is some characteristic that should not be ignored.
So rather, I would suggest performing some kind of "normalization"; like dividing each word count by the average occurrence rate of that particular word in natural language.
Instead of just word counts, the vector coordinates will consist of relative use rate of the particular word in the book compared the average use rate in general language.
That would make a much more precise comparison. Because not just "stop words" are very common, some words are inherently much more common than others.
Although I did not make the experiment, I suspect that in this way, everything will have a much lower cosine similarity.
Very useful info, and the approach was excellent, very fun too
pretty good, but the visualization of the results could have been made in something other than a table. That way, you wouldn't have to explain why the diagonal is 1, and that every number appears twice (mirrored along the diagonal). You'd end up with just 45 rather than 100 datapoints, and then compare the "top 10" across the different measurements. This would be much easier to follow.
Interesting!! We'd love to see a sketch of what you have in mind!
@@Socraticapls upload more videos on AI and machine learning
Thank you