Thanks for teaching and explaining this to this 78 years old lady. I have been learning so much about subject matters I never paid attention to before I started to invest in Tesla since 2018. 😊
Thank you Hans, James and Scott. This is exactly the type of discussion I've been craving. It's good to know that locomotion NNs are small. That means manipulation NNs will also be relatively small too, which leaves compute capacity for task planning and execution. I think the architecture for task learning will need many iterations because this is a layered problem space. Tesla hit local performance maxima (dead ends) several times with FSD. I expect the same cycle of hitting a ceiling and doing an architectural reset will also be required for Optimus. I hope you will cover how you see the balance between edge compute and supercomputer clusters for task learning and optimization. Elon's statements about using future FSD hardware as a distributed NN compute network may be one way they can access the compute cycles they will need.
Thank you Hans, Scott and James - that was a great discussion. I was so pleased when I realised that it was only 'part 1'. Hoping you make a series out of this and not just 1 more part. There's so many exciting developments happening all at once, I love it!
Thank you, Hans. Your expertise is broad, and your presentation skills are excellent. I like how you think before answering questions where I’ve seen you in other venues. Your channel will grow.
One word comes to mind: convergence. Combining AI LMM software, actuator hardware coupled with engineering and manufacturing know-how is a paradigm shift combination. How fortunate at this moment we are to have commercial companies such as Tesla with the resources and commitment to see this thing through. When we get there is only a matter of time.
@77chickox @HansCNelson We’re already there! The Eureka paper (October 2023) showed that GPT-4 can be used to design reward functions for RL environments that use “distinct robot morphologies”. It even outperforms expert human-engineered reward functions on 83% of tasks! The paper is titled “Eureka: Human-Level Reward Design via Coding Large Language Models”. I’m surprised it wasn’t discussed in this podcast!
@@noahlane7426 Thanks for sharing. While not well versed in the details, this paper appears to be a significant step forward and illustrative of the rapid advancement of machine learning. Perhaps in the near future we will have a functional prototype performing generalized tasks.
Amazing duo of James Douma and Scott talong with videos of simulations from ETH Zurich. I suspect there will be a lot of demand for Scott Walter's factory simulation. A hybrid factory/humanoid robot simulation would be amazing!
When lifting asymmetrically (ie. a weight in one hand) we compensate by adjusting our center of gravity. Rotating and tilting the head can be a very fine adjustment of center of gravity. Such adjustments are necessary for static positions, and dynamic movements to look natural, graceful, balanced. Our neck has three DOF, so two is a compromise. but maybe good enough.
Actually I think the addition of actuators to move the head may have much greater psychological acceptance impact than psycho-physical behavioral or functional impact. Humans place great importance and empathy on eye-to-eye (or, in this case, facial- or intentional-) contact. If you are child (or adult), a student (or teacher), a worker (or manager), etc., you want to know that the robot you are interacting with is paying attention to you. This is a step away from the "uncanny valley". Interesting that so much of the following discussion in the video focused on the head movement and on the importance of human attentional and focal awareness. Another Optimus head feature even further enhances this - the optical lighting strip that is shaped almost like a person's "jawbone". This clever touch gives the face a 3-D look (as does the nose and jaw of a person). And it can change color - rather like a blush or facepalm. Optimus does not need silly facial screen images, etc., to convey non-verbal signals when it can use body language. On the other hand, it does need sophisticated AI algorithms to proper incorporate and implement such abilities.
When humans move about they lead with their head. Animators learn this early on. (also that walking is just controlled falling forward) American Football players use that when the face off to make the opponent move where they want. The new Teslabot head movement will help humans around the bot know it's motion direction intent, Head motion also humanizes the bot to feel less alien.
I just thought of a great t- shirt, on the front of the shirt, ' The People are Leaving. Because.....then on the back of the shirt is written, Because the Robots are coming!'.
I have a question that needs to be answered. It's important for the future of Tesla and TSLA. Does Optimus have the hip and back joints necessary for it to get into and exit a vehicle? They're going to need transport and if it has to be loaded into the vehicle by a human then that's feeble. Dr. Walter, what could be done to the structure of the robot that allows the Optimus design to enter into and exit vehicles?
"The torso needs to bend forward, So a new servo, or design a waist like on Figure 01. First a butt that won't rip up the seats. Second a flexible torso. The recently add neck motion helps. But too stiff in the spine." Scott Walter, PhD
Excellent podcast- thanks! Non-verbal communication functionality- reminds me of a key scene from I-Robot, where the robot winks at Will Smith- it had learned a non-verbal communication skill.
This way likely recorded before the release of the "naked" bot walking video. It is on a MoCap stage with reflective makers all over it. Perfect for a Shim model feedback loop. Perhaps in realtime.
Videos Simulations Training Robot bodies 33:34 Aloha robot Stanford Lab - cooking 3-course Cantonese meal 35:40 deep mind video - reinforcement learning - Tech Insider 38:20 deep mimic - example guided deep reinforcement learning of physical based character skills- Reference Motion capture - simulation - university of British Columbia 41:16 ETH Zürich - Simulation to real robots - with additional NN-layer 42:04 - what did we learn in the last 5 years and why Optimus can’t do it: cost of failing/braking 45:25 - parallel deep reinforcement learning - 47:18 - where we are 48:30 - Optimization- Covariance matrix adaptation - Hansen 2006 - efficiency demo - skeletons with muscles - Flexible Muscle-Based Locomotion for Bipedal Creatures 51:22 - NVIDIA-simulation tool 52:21 - it would work with robots if they were cheap enough
@33:20 The light could be used to distinguish between bots, but having multiple bots in a room where this confusion might occur (meaning there are no other visual cues between bot and/or they're current activity) creates an agent Smith scenario where you have multiple bots with the same and independent brain 🧠
I do not but searching RUclips on my name seems to work pretty well. I think there are maybe 50 total but I haven't done many in the last year or so if you're not finding recent stuff that's probably why.
I almost forgot the most important points in a stiff Formula one Chassis race car. The keyword is less variable. Imagine trying to adjust a Race Car with a flexible frame, you would never be able to really get the same desired adjustment to tune the race car. I think the Rigid chassis of a bot is important also so that the actuators which are precise can be precise as well. The keyword is less variable makes for easier consistency makes for better performance like "Thread a Needle". over and over and over cause of less variable of motion
I respect all podcasters who cover Tesla company. End of the day, for a stock investor of tesla, it is not going up. Whatever great inventions that Tesla is doing, stock is grounded for unknown reasons to me. Great work about covering optimus. An individual investor like me in tesla stock, all the great things Tesla matters a lot, but if the stock doesn't go up, my investment is a dead money. Probably CD has better yields with capital protection. I hope wall street understand what Tesla is doing.
Wow, so much great info! This begins to fill a void for want of understanding. I am beginning to have an intuition rising out of the fog about what training neural nets is about, and how different from heuristics programming. I so look forward to Part 2!! I wish I could characterize the process better in my own words, but I have a ways to go. On 26Aug23 Elon and Ashok Elluswamy said: FSD v12 is AI driving by cameras and neural-nets. If you could take a Functional MRI (FMRI) radio graph of FSD AI neural net performing a complex task, and compared it to an organic brain FMRI like a dragon fly brain FMRI while in flight chasing a prey, would you be able to see - or expect to see - similar kinds of activity in the energized patterns in the respective neural nets? How would the complexity compare? We’re on the cusp of incredibly amazing things - WOW!! I so enjoy driving my Model Y, name Independence, and all the interest and excitement it brings!
Enjoyed the propriocection comparison of human and robot. If it fits in can you discuss human vestibular function and what robots do instead to maintain balance in a dynamic situation.
Michelin has a Spot robot from Boston Dynamics in Lexington SC. It has a person who is it's handler, and BD is helping figure out how to make it useful. It does some pretty cool stuff, but how it's used still needs some refinement. Spot is expensive, so making it earn it's keep is going to be a challenge. It's basically a sensor platform. LOL, try drawing a picture of something you are looking at, like a glass, with a cover over your drawing hand. Our propria cession is terrible.
Yes James like new Formula one race cars, the chassis is super stiff and the suspension takes over versus the historical race car where the frame would twist and bend. So the Robots are stiff but the actuators is able to make the bot less Stiff looking and more human like. I use to Road Race with SCCA. It's amazing to feel the difference between a full road cage car and one that is not. What scary going into a 1.5 G turn and hear the metal making tearing noise. This is wearing a Helmet that keeps noise out and here is your open exhaust car making loud noises yet I can hear the twisting of metal. I just wanted you to have a different perspective insight. Conclusion: the stiff chassis can be soften with a great suspension like a Bot can be soften using Neural Net to control actuators to soften the Bot
James spoke about a paper from the ETH Zurich that uses a shim layer to connect RL simulation results to real world robots. A reference for that paper would be great. Thanks
Hah. Watching the video and James talking about how we can figure out where others are looking at. I was just writing about how the lightband around Optimus's head is probably functional for that specific reason to make it easier to see where the bot is looking at. And immediately James mentions the exact same thing. Duh. 😁
I did a study five years ago and concluded that 8 hours should be easy and 16 doable. With the charge rate that is possible on current generation batteries you could make a highly functional robot with as little as maybe 3 hours of runtime - it would just take a break when it was running low and go plug in for 15 minutes to get back up to 90%.
I won't claim to be an "expert" but looking at those fingers, could it even pick up a needle to thread it, Start a roll of tape, put a lid on a cup, make change out of a register? How long would it take to peel a label off the backing? tie a shoe, pick up a pen and write etc. etc. A person, without looking, can pick up just one small screw out of a pile and orient it in their hand. A person can reach into their pocket and count change without looking. This thing doesn't even have pockets.
It won't be able to do all of those given the current DOFs and sensors on the hands - which is why the hands will have to improve if the robot is going to do detailed work. And it will, though there will be a lot of useful things it can do in the current form it's still got quite a way to go before it can match the incredible dexterity of human beings.
@@JamesDouma "a lot of useful things" Everyone always says that. When I look around a factory I don't see anything it could do. Please tell me the "easy" jobs a humanoid will be able to do! There are no "easy" jobs that aren't already automated. Enthusiasts will say "It's not like FSD because you don't have to be 100% accurate". Then it will fail in its job. If it drops a box of screws, breaks something expensive, falls and breaks itself, stops production etc. it's costs could be more than the savings. Enthusiasts will say "It doesn't need to be fast because it can work long hours". Then it will fail in its job. Factories are built for people to operate, so it will have to operate like a person. Asset utilization is important. Otherwise you'll have to create dumb jobs for it like overnight stock-boy. Call me when they demonstrate a stock-boy, if I'm not dead. Enthusiasts will say "we'll need them on mars" On mars you will want a smart crane/robot that can manipulate and join large habitat panels and move machinery. And you would want a smaller smart robot with tools instead of hands so it can manipulate/repair small cables/connectors quickly without having to pick up bulky hand tools. And it should have extending legs rather than get on a ladder. It would be ridiculous to have humanoids operate cranes, use wire cutters and climb ladders. You may only need two humanoids to set up a base because humans didn't evolve to use technology.
A bit off the wall but something that’s been nagging at the back of my mind - there’s one sense that seems to be missing from both Tesla cars and bots and that is audio. Human drivers do use audio for instance the sound of a siren will alert a driver before an emergency vehicle comes into sight, the screech of brakes or a horn causes a high-level interrupt in the human drivers attention, a robot obviously needs to understand spoken commands and yet I don’t recall seeing this in any video - am I missing something? We’ve seen that you need a good lead time in which to gather training data. Do Tesla cars ship with external microphones. Do they send back audio data to Tesla?
There are likely body forms that are more functional than a humanoid but a humanoid is a very good general form to have for the era where we need general purpose robots to start helping out with jobs that humans are already doing. Humanoid robots can drop into human jobs temporarily or permanently by working in the same spaces, doing the same tasks, and using the same tools as a human. Humanoids can be trained by mimicking how a human does a task either from video or from observing an "in person" demo. And humanoids are easy for anybody to understand, interact with, and train.
As robots proliferate there will be specialization but there's a big advantage in being able to scale out a single advanced robot that can be dropped into a huge variety of jobs that need doing with minimal need to change the task or the environment to suit the robot.
how much of the teslabot compute can be wirelessly offloaded to independent computer systems via just sending sensor data and receiving control commands so doesn't have to lug around a brain or consume energy churning cpu cycles and cooling the chassis?
I don’t understand why everyone (figure, Tesla, google) are hell bent on agi robots. Or even humanoid robots. What we really need is a house maid robot, it doesn’t matter if it doesn’t look like humans or have wide capabilities. All we need is for it to do less than 5 tasks at a quarter of human level and a 10th of efficiency but in an arbitrary home set up. 1. Load dishwasher 2. Empty dishwasher and put it in drawer. 3. Load washer, dryer. 4. Fold the dried clothes, iron and put in in a drawer. 5. Arrange the house by placing things in its place. Essentially a housemaid service that’s accessible to countries with cheap labour. If these companies can focus on just these 5 tasks, a lot of people will be willing to pay 20-30k usd without a question. Instead they are trying to solve the hard problem which has diminishing returns.
I've worked in the mobile world in the past and retooling a line to produce a new model phone takes 6-10months with an existing factory. Building a factory takes 12-18months and if we look at how long it took Tesla to build gigafactory texas. Building a new a factory line to build optimus doesn't happen over night or in 12 months. That's assuming you have finalized all of the materials and designs. Given the design isn't done and the material science is still in progress, 5 years sounds super aggressive. Scaling up manufacturing for a new kind of material takes time.
Tesla could have a top half humanoid bot running on wheels and also hanging from tracks on the ceiling to get into tight spots efficiently in factories.
Key is focus on getting the robot to do 1 or 2 tasks reasonably ok in a random environment. Ex: load the laundry, load the dishwasher, put them in a drawer. The problem is - A. The billionaires who have the money to achieve this aren’t aware of how much of a pain is it for normal people because they never have to do it. b. Dumb people, as you can see in this comment section, are more excited by gimmicks such as juggling 3 balls or giving funny philosophical answers.
You three are my people, apparently. Speaking of lightbeard comms, there is solid, flashing and probably four colors-eight states, which is probably a limit. Might be fun to spec those with this group.
Great to see James again.
The GOAT
Alright ...JAMES IS BACK...!!! Always wanted James on the Board for Tesla...!
Anything Optimus gets my interest 😊. Having both these guests on together is fantastic - well done Hans!
I see Douma and instantly know it's a must watch.
Never tire of listening to Douma, thanks to you and your guests😊
Very happy to see James Douma back! Appreciate you James! (Hans & Scott - you’re awesome too!)
Yea but James is aaaawwwweeeeessssooommmeeeee 😄
Thanks for teaching and explaining this to this 78 years old lady. I have been learning so much about subject matters I never paid attention to before I started to invest in Tesla since 2018. 😊
you are amazing
Chapeau Madame.. 👋🏻😁
I think it's wonderful, a 78 year old lady is so interested in this. You must be an amazing person to speak to maam. Respect.
When I see anything new from James (or Dave Lee), I watch and listen hard.
Love James. So smart and well spoken.
Amazing. The fact that we can listen to James speak for free is truly a blessing.
this was wildly informative...thanks Hans. douma is amazing for the community.
EXCELLENT job by James explaining the evolution of robotic reinforcement learning.
Great. I've been eagerly awaiting a new Optimus update from James. Scott seems insightful as well.
James is back finally!
Yes, James is back!!!!! 🎉
An absolute great discussion by two of the smartest people in robotics and AI
Wonderful discussion and super great to have James back. I'm ready for part 2!!!
Thank you Hans, James and Scott. This is exactly the type of discussion I've been craving. It's good to know that locomotion NNs are small. That means manipulation NNs will also be relatively small too, which leaves compute capacity for task planning and execution. I think the architecture for task learning will need many iterations because this is a layered problem space. Tesla hit local performance maxima (dead ends) several times with FSD. I expect the same cycle of hitting a ceiling and doing an architectural reset will also be required for Optimus.
I hope you will cover how you see the balance between edge compute and supercomputer clusters for task learning and optimization. Elon's statements about using future FSD hardware as a distributed NN compute network may be one way they can access the compute cycles they will need.
We didn't get into that at all in this round of the conversation (part 1 or 2), but that's a great suggestion for a future episode!
@@HansCNelson Thanks I look forward to it.
Thank you all for keeping us up to speed on the swiftly changing field of advanced robotics.
Great panel! I love hearing from both of your guests. 👍👍👍
Glad you enjoyed it!
So happy to see James Douma, he’s awesome!
James is a must watch
Love this format where each person talks and they do not interrupt each other. Informative.
Excellent. Looking forward to part 2!
Thanks Hans. Such an interesting piece and great that you let James "off the leash" so to speak with no interruptions. Good stuff. Thank you.
Great job Hans, love hearing James, Scott and you collaborating
Let's go HN. The time has come for you and your engineering/technical expertise to grow this style content. This content is frontier. More please.
Thanks Hans for organising this fascinating discussion
James's concept, ideology, understanding and attitude towards the technology is diamond solid, not just rock solid. He's not swayed by anything.
Thank you Hans, Scott and James - that was a great discussion. I was so pleased when I realised that it was only 'part 1'.
Hoping you make a series out of this and not just 1 more part. There's so many exciting developments happening all at once, I love it!
Thanks for bringing back James: He is next level!
Like the T-shirt, time to talk about replacing judges with robots😂
May all the world see this video. It is a heads-up declaration of what the future holds.❤
Thank you, Hans. Your expertise is broad, and your presentation skills are excellent. I like how you think before answering questions where I’ve seen you in other venues. Your channel will grow.
One word comes to mind: convergence. Combining AI LMM software, actuator hardware coupled with engineering and manufacturing know-how is a paradigm shift combination. How fortunate at this moment we are to have commercial companies such as Tesla with the resources and commitment to see this thing through. When we get there is only a matter of time.
@77chickox @HansCNelson We’re already there! The Eureka paper (October 2023) showed that GPT-4 can be used to design reward functions for RL environments that use “distinct robot morphologies”. It even outperforms expert human-engineered reward functions on 83% of tasks! The paper is titled “Eureka: Human-Level Reward Design via Coding Large Language Models”. I’m surprised it wasn’t discussed in this podcast!
@@noahlane7426 Thanks for sharing. While not well versed in the details, this paper appears to be a significant step forward and illustrative of the rapid advancement of machine learning. Perhaps in the near future we will have a functional prototype performing generalized tasks.
Great content guys
THANKS HANS 🤗FOR HOSTING JAMES AND SCOTT,VERY INTERESTING 🧐 AND INSIGHTFUL 💚💚💚
Amazing duo of James Douma and Scott talong with videos of simulations from ETH Zurich. I suspect there will be a lot of demand for Scott Walter's factory simulation. A hybrid factory/humanoid robot simulation would be amazing!
Dr. Scott and James. Two thumbs up!
Excellent! Love having all three of you guys talking together! More please!
Excellent video thanks to all !
Like to see Optimus juggle 3 balls🤹♂️.
Cheers Hans and the guys
hell yeah...the doumer in da house
Hans thank you so much for this discussion! James is always a must watch.
When lifting asymmetrically (ie. a weight in one hand) we compensate by adjusting our center of gravity. Rotating and tilting the head can be a very fine adjustment of center of gravity. Such adjustments are necessary for static positions, and dynamic movements to look natural, graceful, balanced. Our neck has three DOF, so two is a compromise. but maybe good enough.
♥ *Douma & Walter together ?* _grab your popcorn!_
James knowledge always blow my mind 🤯
Phew ! My brain is steaming listening to James ! Brilliant...
Thanks to all three of you.
Our pleasure!
James Douma makes all this very accessible. Great video
James is such a genius!
I always expect greatness from James, but somehow he still blows me away with his insight.
I wonder if they will use 48volt?
50v 2.3kWh
Actually I think the addition of actuators to move the head may have much greater psychological acceptance impact than psycho-physical behavioral or functional impact. Humans place great importance and empathy on eye-to-eye (or, in this case, facial- or intentional-) contact. If you are child (or adult), a student (or teacher), a worker (or manager), etc., you want to know that the robot you are interacting with is paying attention to you. This is a step away from the "uncanny valley".
Interesting that so much of the following discussion in the video focused on the head movement and on the importance of human attentional and focal awareness. Another Optimus head feature even further enhances this - the optical lighting strip that is shaped almost like a person's "jawbone". This clever touch gives the face a 3-D look (as does the nose and jaw of a person). And it can change color - rather like a blush or facepalm. Optimus does not need silly facial screen images, etc., to convey non-verbal signals when it can use body language. On the other hand, it does need sophisticated AI algorithms to proper incorporate and implement such abilities.
When humans move about they lead with their head. Animators learn this early on. (also that walking is just controlled falling forward) American Football players use that when the face off to make the opponent move where they want. The new Teslabot head movement will help humans around the bot know it's motion direction intent, Head motion also humanizes the bot to feel less alien.
29:00 Non verbal clues. . . Waiting for Optimus doing an “eye roll”
Looking forward to part II. 👍
I just thought of a great t- shirt, on the front of the shirt, ' The People are Leaving. Because.....then on the back of the shirt is written, Because the Robots are coming!'.
A more appropriate one would be "The robots are here".
Yes James!!!!!!
14:03 I think it is interesting what the purpose is with the cans at the other desks. They reminds of 4680 batteries.
I have a question that needs to be answered. It's important for the future of Tesla and TSLA.
Does Optimus have the hip and back joints necessary for it to get into and exit a vehicle? They're going to need transport and if it has to be loaded into the vehicle by a human then that's feeble.
Dr. Walter, what could be done to the structure of the robot that allows the Optimus design to enter into and exit vehicles?
"The torso needs to bend forward, So a new servo, or design a waist like on Figure 01.
First a butt that won't rip up the seats. Second a flexible torso. The recently add neck motion helps. But too stiff in the spine."
Scott Walter, PhD
Excellent podcast- thanks! Non-verbal communication functionality- reminds me of a key scene from I-Robot, where the robot winks at Will Smith- it had learned a non-verbal communication skill.
Great ❤
Can u post the sources of the videos u demonstrate ?
Just added them to the description a few minutes ago.
Great call out.
The 2DOF neck movers are low cost weedy actuators that add immeasurable humanness.
Amazing stuff, love it
This way likely recorded before the release of the "naked" bot walking video. It is on a MoCap stage with reflective makers all over it. Perfect for a Shim model feedback loop. Perhaps in realtime.
Yep. Recorded on 1/17/24
Videos Simulations Training Robot bodies
33:34 Aloha robot Stanford Lab - cooking 3-course Cantonese meal
35:40 deep mind video - reinforcement learning - Tech Insider
38:20 deep mimic - example guided deep reinforcement learning of physical based character skills- Reference Motion capture - simulation - university of British Columbia
41:16 ETH Zürich - Simulation to real robots - with additional NN-layer
42:04 - what did we learn in the last 5 years and why Optimus can’t do it: cost of failing/braking
45:25 - parallel deep reinforcement learning -
47:18 - where we are
48:30 - Optimization- Covariance matrix adaptation - Hansen 2006 - efficiency demo - skeletons with muscles - Flexible Muscle-Based Locomotion for Bipedal Creatures
51:22 - NVIDIA-simulation tool
52:21 - it would work with robots if they were cheap enough
James was awesome as usual. Scott is a good guy too. But James is awwwweeeeesommmmeee
James, please join Tesla Board 🙏
@33:20 The light could be used to distinguish between bots, but having multiple bots in a room where this confusion might occur (meaning there are no other visual cues between bot and/or they're current activity) creates an agent Smith scenario where you have multiple bots with the same and independent brain 🧠
Nice choice of guests
Fun stuff . Thanks !
Tesla stock tanking at the moment is the best that could have happened. Im starting to build a position from now and out the year, and will HOLD
This was great
James, do you have a youtube page that points to all of videos that you are in?
I do not but searching RUclips on my name seems to work pretty well. I think there are maybe 50 total but I haven't done many in the last year or so if you're not finding recent stuff that's probably why.
I almost forgot the most important points in a stiff Formula one Chassis race car. The keyword is less variable. Imagine trying to adjust a Race Car with a flexible frame, you would never be able to really get the same desired adjustment to tune the race car. I think the Rigid chassis of a bot is important also so that the actuators which are precise can be precise as well. The keyword is less variable makes for easier consistency makes for better performance like "Thread a Needle". over and over and over cause of less variable of motion
I respect all podcasters who cover Tesla company. End of the day, for a stock investor of tesla, it is not going up. Whatever great inventions that Tesla is doing, stock is grounded for unknown reasons to me. Great work about covering optimus. An individual investor like me in tesla stock, all the great things Tesla matters a lot, but if the stock doesn't go up, my investment is a dead money. Probably CD has better yields with capital protection. I hope wall street understand what Tesla is doing.
Wow, so much great info! This begins to fill a void for want of understanding. I am beginning to have an intuition rising out of the fog about what training neural nets is about, and how different from heuristics programming.
I so look forward to Part 2!!
I wish I could characterize the process better in my own words, but I have a ways to go.
On 26Aug23 Elon and Ashok Elluswamy said: FSD v12 is AI driving by cameras and neural-nets. If you could take a Functional MRI (FMRI) radio graph of FSD AI neural net performing a complex task, and compared it to an organic brain FMRI like a dragon fly brain FMRI while in flight chasing a prey, would you be able to see - or expect to see - similar kinds of activity in the energized patterns in the respective neural nets? How would the complexity compare?
We’re on the cusp of incredibly amazing things - WOW!! I so enjoy driving my Model Y, name Independence, and all the interest and excitement it brings!
Enjoyed the propriocection comparison of human and robot. If it fits in can you discuss human vestibular function and what robots do instead to maintain balance in a dynamic situation.
Michelin has a Spot robot from Boston Dynamics in Lexington SC. It has a person who is it's handler, and BD is helping figure out how to make it useful. It does some pretty cool stuff, but how it's used still needs some refinement. Spot is expensive, so making it earn it's keep is going to be a challenge. It's basically a sensor platform. LOL, try drawing a picture of something you are looking at, like a glass, with a cover over your drawing hand. Our propria cession is terrible.
Yes James like new Formula one race cars, the chassis is super stiff and the suspension takes over versus the historical race car where the frame would twist and bend. So the Robots are stiff but the actuators is able to make the bot less Stiff looking and more human like. I use to Road Race with SCCA. It's amazing to feel the difference between a full road cage car and one that is not. What scary going into a 1.5 G turn and hear the metal making tearing noise. This is wearing a Helmet that keeps noise out and here is your open exhaust car making loud noises yet I can hear the twisting of metal. I just wanted you to have a different perspective insight.
Conclusion: the stiff chassis can be soften with a great suspension like a Bot can be soften using Neural Net to control actuators to soften the Bot
Great insights 👍👍
James spoke about a paper from the ETH Zurich that uses a shim layer to connect RL simulation results to real world robots. A reference for that paper would be great. Thanks
Hah. Watching the video and James talking about how we can figure out where others are looking at. I was just writing about how the lightband around Optimus's head is probably functional for that specific reason to make it easier to see where the bot is looking at. And immediately James mentions the exact same thing. Duh. 😁
I wonder how long the bot can run on a charge
I did a study five years ago and concluded that 8 hours should be easy and 16 doable. With the charge rate that is possible on current generation batteries you could make a highly functional robot with as little as maybe 3 hours of runtime - it would just take a break when it was running low and go plug in for 15 minutes to get back up to 90%.
James forgot there was another guest. James… social skills… sharing air time… social awareness…
1:01:10 So what he is saying is the test of really great advancement in robotic control is if we can make a tentacle robot work...
I won't claim to be an "expert" but looking at those fingers, could it even pick up a needle to thread it, Start a roll of tape, put a lid on a cup, make change out of a register? How long would it take to peel a label off the backing? tie a shoe, pick up a pen and write etc. etc. A person, without looking, can pick up just one small screw out of a pile and orient it in their hand. A person can reach into their pocket and count change without looking. This thing doesn't even have pockets.
It won't be able to do all of those given the current DOFs and sensors on the hands - which is why the hands will have to improve if the robot is going to do detailed work. And it will, though there will be a lot of useful things it can do in the current form it's still got quite a way to go before it can match the incredible dexterity of human beings.
@@JamesDouma "a lot of useful things" Everyone always says that. When I look around a factory I don't see anything it could do. Please tell me the "easy" jobs a humanoid will be able to do! There are no "easy" jobs that aren't already automated. Enthusiasts will say "It's not like FSD because you don't have to be 100% accurate". Then it will fail in its job. If it drops a box of screws, breaks something expensive, falls and breaks itself, stops production etc. it's costs could be more than the savings. Enthusiasts will say "It doesn't need to be fast because it can work long hours". Then it will fail in its job. Factories are built for people to operate, so it will have to operate like a person. Asset utilization is important. Otherwise you'll have to create dumb jobs for it like overnight stock-boy. Call me when they demonstrate a stock-boy, if I'm not dead. Enthusiasts will say "we'll need them on mars" On mars you will want a smart crane/robot that can manipulate and join large habitat panels and move machinery. And you would want a smaller smart robot with tools instead of hands so it can manipulate/repair small cables/connectors quickly without having to pick up bulky hand tools. And it should have extending legs rather than get on a ladder. It would be ridiculous to have humanoids operate cranes, use wire cutters and climb ladders. You may only need two humanoids to set up a base because humans didn't evolve to use technology.
oh god.. did douma grace all us plebians with a 1/2 hour of his time once again? woopy.
A bit off the wall but something that’s been nagging at the back of my mind - there’s one sense that seems to be missing from both Tesla cars and bots and that is audio. Human drivers do use audio for instance the sound of a siren will alert a driver before an emergency vehicle comes into sight, the screech of brakes or a horn causes a high-level interrupt in the human drivers attention, a robot obviously needs to understand spoken commands and yet I don’t recall seeing this in any video - am I missing something? We’ve seen that you need a good lead time in which to gather training data. Do Tesla cars ship with external microphones. Do they send back audio data to Tesla?
Honestly a work robot should look more like an octopus with hands and feet. Optimus is really more for inspiration.
There are likely body forms that are more functional than a humanoid but a humanoid is a very good general form to have for the era where we need general purpose robots to start helping out with jobs that humans are already doing. Humanoid robots can drop into human jobs temporarily or permanently by working in the same spaces, doing the same tasks, and using the same tools as a human. Humanoids can be trained by mimicking how a human does a task either from video or from observing an "in person" demo. And humanoids are easy for anybody to understand, interact with, and train.
As robots proliferate there will be specialization but there's a big advantage in being able to scale out a single advanced robot that can be dropped into a huge variety of jobs that need doing with minimal need to change the task or the environment to suit the robot.
@@JamesDouma thanks James
Spectrum Boy is a supervillain in disguise, and going to destroy the world...
how much of the teslabot compute can be wirelessly offloaded to independent computer systems via just sending sensor data and receiving control commands so doesn't have to lug around a brain or consume energy churning cpu cycles and cooling the chassis?
Let's gooo
I don’t understand why everyone (figure, Tesla, google) are hell bent on agi robots. Or even humanoid robots. What we really need is a house maid robot, it doesn’t matter if it doesn’t look like humans or have wide capabilities.
All we need is for it to do less than 5 tasks at a quarter of human level and a 10th of efficiency but in an arbitrary home set up.
1. Load dishwasher
2. Empty dishwasher and put it in drawer.
3. Load washer, dryer.
4. Fold the dried clothes, iron and put in in a drawer.
5. Arrange the house by placing things in its place.
Essentially a housemaid service that’s accessible to countries with cheap labour. If these companies can focus on just these 5 tasks, a lot of people will be willing to pay 20-30k usd without a question. Instead they are trying to solve the hard problem which has diminishing returns.
I've worked in the mobile world in the past and retooling a line to produce a new model phone takes 6-10months with an existing factory. Building a factory takes 12-18months and if we look at how long it took Tesla to build gigafactory texas. Building a new a factory line to build optimus doesn't happen over night or in 12 months. That's assuming you have finalized all of the materials and designs. Given the design isn't done and the material science is still in progress, 5 years sounds super aggressive. Scaling up manufacturing for a new kind of material takes time.
The chances of AGI that can drive a robot better than FSD being achieved by a 3rd party are not zero. Is TSLA still a good investment then?
The creation of the thumb is humans down fall.
Tesla could have a top half humanoid bot running on wheels and also hanging from tracks on the ceiling to get into tight spots efficiently in factories.
Key is focus on getting the robot to do 1 or 2 tasks reasonably ok in a random environment. Ex: load the laundry, load the dishwasher, put them in a drawer.
The problem is -
A. The billionaires who have the money to achieve this aren’t aware of how much of a pain is it for normal people because they never have to do it.
b. Dumb people, as you can see in this comment section, are more excited by gimmicks such as juggling 3 balls or giving funny philosophical answers.
You three are my people, apparently. Speaking of lightbeard comms, there is solid, flashing and probably four colors-eight states, which is probably a limit. Might be fun to spec those with this group.