Vision ends when the viewer (agent 1) sees the words. Language begins when the viewer (agent 1) combines the words it has seen with "prior knowledge" and then communicate "value added" information to a listener (agent 2). For example, when agent 1 "sees" (vision) the name Lewis Hamilton, it must be able to use its knowledge about Hamilton to effectively engage in a coherent conversation with an expert about this great F1 driver. At the moment state of the art like GPT3 can fake a coherent only when communicating with non-experts.
This is about semantic interpretation. Whether image recognition and natural language processing could share the same "back end" for semantic interpretation and abstraction. I wonder if one could train an convolutional NN and a transformer to spit out the same semantic vector. So a natural language description of a picture and the picture would be compressed into the same (or similar) vector space coordinates ? :/
The question "Where does vision end and language start?" was intriguing. It shows a potential final destination that needs to achieved for DL based AI.
Windows start, should be, On/off, it is here Your illiteracy begin, better open some windows and get some fresh air, and understand the nature of dictator-principle. AI, is illiteracy and superstition, - intelligence can never be artificial. Repeating dead mantras, is Not individual thinking. The development of Consciousness and Language is two sides of the very same development, based on Eternal Principles.
I think the wife example is quiet bad because there is a sexual component in the perception of the other, probably with a friend there will be more objectivity. Also yes if you have human level speech recognition and understanding you'll have the vision for free, understanding text is just a primitive form of acquiring information, replace objects on a picture by words and voila.
This is just my ballpark guess, but i think it should be empirical risk minimization and something around no free lunch theorem. i dont know the third one
I literarily suffer from the same cosmetic matter this respectable person suffers from. I use a solution daily, I understand how you can get used to it but please for the sake of other people research a solution too. I felt embarrassed to mention it and not many will, but I care about AI and those pushing it forward. Beyond being highly intelligent, you are an attractive person👍
Language has much higher dimensionality than vision. Vision has three basic dimensions and that could probably be abstracted up to thousands or millions. Language has over 6,500 basic dimensions. The abstraction of these basic dimensions may go into the trillions
This conversation really seemed to enlighten me on how language would have been impossible with sight and hearing. I can see that a word can have many definitions without the presence of a visual or tone of voice. So for the computer to learn. If we relate these few in the algorithm things so that the computer can as we did. If the computer is a rigid piece of electronics, isn't that how life began billions of years ago? Maybe with a better architect.
This was an interesting conversation! Lex - I wonder if the title should be "Language vs. Vision" instead. 6:56 - In terms of Generative AI, can Language and Vision both work to improve each other, like an arms race? How will the AI model and algorithm decide when to determine a pass or fail result for either/or?
Hey lex, one cool thing would be to add some more media to the conversations. Show the guests some clips, read them news, and then we would like to hear their opinion. Great job ✋🏻👏🏻
i believe cnn and nlp should stand as inputs for decision making systems and reinforcement learning should explore space for actions, state and targets states. so the 2 first are more like perception constructor and the last as decision space explorer
Which field have more jobs(NLP or CV)? It seems to me that so far there are a lot more applications for CV, and therefore CV has more jobs opportunities than NLP. Simply search “computer vision job USA” in google and “NlP jobs USA”, the comparison result of both will show that CV has more jobs. Wonder what is your 2 cent on it? Maybe I am wrong?
Throwing out a question here, as there are some clever people in the thread. Anyone care to help me understand why "natural language" (and does that exclude body language and tone of voice?) would be important for AI? As an example; IKEA furniture assembly instructions don't need words to explain stuff to humans. And being a poet is not a requirement for human level intelligence, right?
Your examples are more about language generation, even if important, the hot topic nowadays is language understanding. Understanding language hides a lot of very difficult challenges. Among them reasoning about entities is one of the most difficult one. Each time we speak we refer to events happened in the past and in the present, make implicit relations between entities and talk about abstract things. The language is the description of the world in which we live and the abstract world we have created (the concept of nations, politics, jokes etc.). To understand language a machine needs at first to understand the world we have built. We are far from achieving something like that with AI. How can we pretend to have an "intelligent" machine if it can not understand us?
I don't understand the difficulty in the "Where vision ends and language starts" question. I imagine an advanced enough vision system can just recognize that a particular region of pixels assortment represents text, from that point it can be converted to raw text (which is a decades-old solved problem) and then fed to an NLP pipeline for interpretation. Imu, it's not a vision system's role to accomplish language understanding, but it would be ideal if it could at least identify what is text and relay it to the NLP component.
The problem is that we are trying to make a robotic brain from scratch. Maybe the solution is to give initials steps so that it doesn't start from 0. It's like when you learn other language. You already know what is a dog, but need to learn how to say it in other "way" and when you should say it.
How does that apply? When a baby is born he does not know what a dog is. The only thing he starts with are unconscious behaviors, like "cry if hungry". In that sense starting from scratch seams very similar.
"where the vision ends, language begins" this line touches my heart!
Not only is this guy brilliant, he’s just such a nice guy
Vision ends when the viewer (agent 1) sees the words. Language begins when the viewer (agent 1) combines the words it has seen with "prior knowledge" and then communicate "value added" information to a listener (agent 2). For example, when agent 1 "sees" (vision) the name Lewis Hamilton, it must be able to use its knowledge about Hamilton to effectively engage in a coherent conversation with an expert about this great F1 driver. At the moment state of the art like GPT3 can fake a coherent only when communicating with non-experts.
Vision take the visual input, Brain caches the sentences, NLP begins? If cache is out of memory, Vision goes back and queries the same input again?
This is about semantic interpretation. Whether image recognition and natural language processing could share the same "back end" for semantic interpretation and abstraction. I wonder if one could train an convolutional NN and a transformer to spit out the same semantic vector. So a natural language description of a picture and the picture would be compressed into the same (or similar) vector space coordinates ? :/
There is already shine datasets for this task: you build net for NLP, net for CV and minimize KL-div between two hidden spaces
The question "Where does vision end and language start?" was intriguing. It shows a potential final destination that needs to achieved for DL based AI.
✔️John Venn liked this.
Windows start, should be, On/off, it is here Your illiteracy begin,
better open some windows and get some fresh air, and understand the nature of dictator-principle.
AI, is illiteracy and superstition, - intelligence can never be artificial.
Repeating dead mantras, is Not individual thinking.
The development of Consciousness and Language is two sides of the very same development, based on Eternal Principles.
@@holgerjrgensen2166 more like returnal than eternal 🤔😘
What do You mean, if You know what You're saying.
Ru-Mu,
okai, can means allmost any thing, in danish, it is åvkæj, just a sound-combination.
"i am going to explain why"...opens by asking a question, nice!
0:49 The Word is "Interdisciplinary"
Man. I hate that word.... It stems from artificial boundaries that we've created due to historical happenstance.
once the vision can read the language, the loop is complete
Man this is going to finally get watched by people
I think the wife example is quiet bad because there is a sexual component in the perception of the other, probably with a friend there will be more objectivity.
Also yes if you have human level speech recognition and understanding you'll have the vision for free, understanding text is just a primitive form of acquiring information, replace objects on a picture by words and voila.
You should get a linguist on lex might be interesting to talk about the hermeneutic aspect of language learning and interpretation for AGI
Lex, how about a podcast with Shai Ben-David on advances on the theoretical side of ML?
YES, THIS
0:54 - Does anyone know what those principles are??
This is just my ballpark guess, but i think it should be empirical risk minimization and something around no free lunch theorem. i dont know the third one
I literarily suffer from the same cosmetic matter this respectable person suffers from.
I use a solution daily, I understand how you can get used to it but please for the sake of other people research a solution too. I felt embarrassed to mention it and not many will, but I care about AI and those pushing it forward. Beyond being highly intelligent, you are an attractive person👍
I thought the interviewer was smart, but Ilya is on a different level.
Language has much higher dimensionality than vision. Vision has three basic dimensions and that could probably be abstracted up to thousands or millions. Language has over 6,500 basic dimensions. The abstraction of these basic dimensions may go into the trillions
This conversation really seemed to enlighten me on how language would have been impossible with sight and hearing. I can see that a word can have many definitions without the presence of a visual or tone of voice. So for the computer to learn. If we relate these few in the algorithm things so that the computer can as we did. If the computer is a rigid piece of electronics, isn't that how life began billions of years ago? Maybe with a better architect.
This was an interesting conversation! Lex - I wonder if the title should be "Language vs. Vision" instead. 6:56 - In terms of Generative AI, can Language and Vision both work to improve each other, like an arms race? How will the AI model and algorithm decide when to determine a pass or fail result for either/or?
Hey lex, one cool thing would be to add some more media to the conversations. Show the guests some clips, read them news, and then we would like to hear their opinion. Great job ✋🏻👏🏻
i believe cnn and nlp should stand as inputs for decision making systems and reinforcement learning should explore space for actions, state and targets states. so the 2 first are more like perception constructor and the last as decision space explorer
So interesting to look back at this interview now, in the wake of GPT4.
Which field have more jobs(NLP or CV)? It seems to me that so far there are a lot more applications for CV, and therefore CV has more jobs opportunities than NLP. Simply search “computer vision job USA” in google and “NlP jobs USA”, the comparison result of both will show that CV has more jobs. Wonder what is your 2 cent on it? Maybe I am wrong?
It will change this year or maybe in 2022.
It's 2024 and your opinions are still correct so far
Lex found an opportunity to sneak in his interest in monogamous marriage then smiled
Throwing out a question here, as there are some clever people in the thread. Anyone care to help me understand why "natural language" (and does that exclude body language and tone of voice?) would be important for AI? As an example; IKEA furniture assembly instructions don't need words to explain stuff to humans. And being a poet is not a requirement for human level intelligence, right?
Your examples are more about language generation, even if important, the hot topic nowadays is language understanding. Understanding language hides a lot of very difficult challenges. Among them reasoning about entities is one of the most difficult one. Each time we speak we refer to events happened in the past and in the present, make implicit relations between entities and talk about abstract things. The language is the description of the world in which we live and the abstract world we have created (the concept of nations, politics, jokes etc.). To understand language a machine needs at first to understand the world we have built. We are far from achieving something like that with AI.
How can we pretend to have an "intelligent" machine if it can not understand us?
Beautiful hair
Yes, the manmade world (physicality) our thought and action is primarily governed by language. So, language is fundamental.
I don't understand the difficulty in the "Where vision ends and language starts" question. I imagine an advanced enough vision system can just recognize that a particular region of pixels assortment represents text, from that point it can be converted to raw text (which is a decades-old solved problem) and then fed to an NLP pipeline for interpretation. Imu, it's not a vision system's role to accomplish language understanding, but it would be ideal if it could at least identify what is text and relay it to the NLP component.
The problem is that we are trying to make a robotic brain from scratch. Maybe the solution is to give initials steps so that it doesn't start from 0. It's like when you learn other language. You already know what is a dog, but need to learn how to say it in other "way" and when you should say it.
How does that apply? When a baby is born he does not know what a dog is.
The only thing he starts with are unconscious behaviors, like "cry if hungry".
In that sense starting from scratch seams very similar.
I think vision lags language because it doesn't have a lot of labeled data
Great Illya
Computer vision fascinating more
8:15 such a blue pilled Lex
man.. I find it interesting how I really respect Ilya for what he achieved but I just don't agree with his views on things most of the time.
Hey Lex
Rezpect ze russians🇷🇺
hmmm
JUST
Vision ends when language starts.
hav 2 say that the dumbest animals hav vision but not langwage
Language
Why?
@@leecharlie2513 because language is a representation.
@@shreeyatyagi But isn’t the recent GPT-3 demonstrating very promising result to generating meaningful text and dialog?
You are the most nicest cutest thing!
Read what Lacan says about language. Not chomsky.
🐸🐸🐸🐸🐸🐸