How Robots Learn - Tesla Bot Discussion with Industry Experts James, John & Scott
HTML-код
- Опубликовано: 3 ноя 2022
- Merch www.farzadmesbahi.com/merch
Patreon / farzadmesbahi
Twitter @farzyness
James Douma, Scott Walter, and John (Dr. Know it All) join me in discussing the latest developments in robotics and AI and how these developments will impact Tesla's future.
James' Twitter: / jamesdouma
John's Twitter: / drknowitall16
Scott's Twitter: / goingballistic5
John's RUclips: / drknowitallknowsitall
I worked at Tesla starting in July of 2017 as an Operations Analyst out of the Bethlehem, PA distribution center facility and left as a Program Manager based out of Austin, TX as of September 2021. I spent most of my time in the distribution and supply chain organizations.
Before Tesla, I was a Director of Business Intelligence and Pricing at the largest Pet Food & Supply distributor in the US, Phillips Pet Food & Supplies based out of Easton, PA. My wife and I also owned a small business in Bethlehem, PA between 2016 and 2019.
My thoughts are my own and are not representative of everyone who currently works, or has worked at Tesla.
If you like this content I would greatly appreciate your likes and shares!
www.farzadmesbahi.com
www.linkedin.com/in/farzadmesbahi
Twitter @farzyness
#Tesla #elonmusk #fsd #TeslaNews #TeslaFSDBeta #teslabot #TeslaUpdate #ElonMuskNews #teslanews #EV #electricvehicle #investing #tsla #robotaxis #ai 
Наука
Whoever timestamps this will get a free tshirt from my merch store www.Farzadmesbahi.com/merch
Hey, Farzad! I did my best (see below), but the discussion _was_ kind of all over the place... 😁 I'm in the EU, so I doubt it would be worth sending me a "free" t-shirt, what with shipping and customs and all. IDK, see it as a contribution to the community, a thank you for all the great and freely available content, or bring me some merch whenever you might visit Sweden?! Peace!
one of the best discussions you've ever organized so far. Thank you!
this should become a REGULAR THING! you guys are AWSOME! best teslabot-expert-pannel in the internet!
This show with all experts topics definitely should be a weekly schedule! GO TECHIES!
Man, James is fun to listen to. Gotta find more of him.
I've done it now, I'm done, D-O-N... still a bit uncertain about the accuracy of my labeling, and the granularity of my time-stamping, but use it as you will! =)
00:00 Welcome
00:33 Round of presentations
02:23 Scott goes to (not) see a rocket launch
----
04:35 Actual introduction: The Robots are Coming (Scott)
06:30 The concept of a robot, industrial robots, textbook definitions
09:00 Teaching robots, online and offline programming
12:00 Waypoints, in reality or on a map, vs higher levels of abstraction
15:00 The goal: assemble furniture?!
----
17:06 Robots operating in the dark; proprioception and other modalities of feedback (James)
20:46 Collateral costs for working blindly: the need for precisely defined objects and environments (Scott)
24:30 Utilizing unexpectedly useful feedback, noise becomes data
27:00 Visual feedback and preconceptions; Optimus apparently categorizing classes of objects
29:30 FSD and its trajectory projections, cf Optimus; sensors and cameras in prototypes vs end product
33:21 Neural networks vs heuristics; data, noise, and calibration
35:55 Getting a maximum of data from a minimum of sensors; the richness of camera images
38:43 Modalities other than visual: measured explicitly or implicitly? Deviations become data
42:32 How could Tesla cars perform at, or above, a human level of accuracy, without ultrasonic sensors?
48:08 A wild idea: actively using the car's headlights to gather additional data (Scott)
52:05 Getting the most out of image data, and the very complex problem of making a process simple
1:00:36 Tesla transplants the paradigm to a high margin industry; hunting down the mammoth
---
1:02:22 Back to 'bots! Pros and cons of neural networks; Degrees of freedom vs decision trees
1:09:17 Pruning the tree: What would a human do? World models and anomalies
1:11:52 Approaches to training networks; 1,000 virtual robots learning to walk
1:19:48 From simulation to reality; vision for planning, proprioceptive sensors for precision feedback
1:21:50 Four-wheel robots showing the versatility of neural networks
1:23:53 Few-shot neural networks; GPT-3 and FSD providing starting points for successors and applications
1:26:30 Advantages of having the FSD simulator
1:27:33 Skills vs tasks; deciding what skills are needed for a given task (Scott)
1:30:01 World model, planning, perception, motion control; all have different challenges
1:31:04 AR/VR as a mere crutch for training the neural network; the programmer becomes sensei (Scott)
1:34:21 Mixed reality dexterity training; generalizing basic manipulations to a range of objects
1:37:40 TeslaBot training in virtual environments? Perception subsets and occupancy maps
1:40:42 FSD right now: 100+ different networks trained in different ways
1:42:31 Embodiment; written vs unwritten knowledge, real world interactions, e g walking the dog
1:47:27 Why Optimus simply _can't_ be done quite yet; groking the irl dog/s (James)
1:51:37 More four-wheel robot tricks; finding the best way of training for any given set of objectives
1:53:34 Motion capture and learning by mimicry; motion capture data filtered through a physical model
---
2:00:41 Challenging the floccinaucinihilipilification of TeslaBot; development vs research, converging technologies
2:07:42 Optimus IS going to happen; all core capabilities needed have already been demonstrated
2:10:01 OpenPose; human pose estimation is a solved problem
2:13:03 Corollary: FSD will be able to figure out pedestrians' intent, based on pose and body language!
2:14:20 Distillation of pedestrian prediction and more; from fat transformers to lean control policies
2:17:50 Distilled networks (and AI in general) going Open Source
2:20:22 AGI and robotics: jack or master of all trades? The humanoid form as a good compromise
2:24:18 What current industrial robots can learn/gain from Optimus
2:25:30 Where and when does automation make sense? Optimus as an intermediate step
2:28:50 Human, humanoid, hard automation; Reducing, and ultimately eliminating, human error
---
2:31:15 Watch out for James vs Scott, round two, on Ellie in Space! On SPACE!
2:33:00 Thoughts and conclusions, hopes and dreams
2:38:36 Robotic roofing, cutting corner cases
2:41:42 Robotic surgery, DaVinci suturing, extreme precision orthopedic implants
2:45:38 Finishing words; twitter and merch plug
Thank you! That was very kind of you Sir!
@@aljohnson3717 Oh please, I'm not really a 'sir'... 😁 but thank you for the kind words!
Assuming Scott' ring light is around his camera, and that when he talks he looks right at the camera, the fact that the reflections constantly center on his left eye suggests he is leaning on his left eye to do most of his vision work. He may want to mention that to his optician!
This was a fun discussion. Picturing Optimus walking a dog is hilarious lol but also such a great real-world use case.
Farzad can you get "Anastasi In Tech" on your channel sometime for an interview? She knows her stuff and can hang with this crew in knowledge!
Thanks for considering!
Wow this is out of this world. Y’all make it sound so easy
Great conversation! 👍
Topic for next time:
Military and Law Enforcement Tesla Bots, specifically task built, seem inevitable. We used drones in the Army.. SWAT teams use em.. Many police departments are currently using Teslas and Cybertruck will change the game. Interesting topic for this group. 🤔
I facet gemstones as a hobby. Basically it is high precision sculpture. The goal is to have the surface finish be "scratch free" and have the facet corners meet other facet corners visually "perfectly". Defects must be smaller than 1/4 wave length of light or about 20 microns to be invisible. The process is to grind off large amounts of material with a coarse grit, e.g., 200. Then move to finer and finer grits. Humans can do this by hand! In the final step the polishing grit is 100,000 to 250,000. (1,200 to 3,000 is plenty good for polishing the headlamps of a car.) However at 250,000 you are still removing material and the goal is to step through the grit levels at a rate that will result in the surface becoming "scratch free" at precisely the same moment that the facet corners "meet". Humans do this by reaching "scratch free" just before the corners "meet" and another 5 seconds of polishing with 250,000 grit will get the corners to "meet".
Faceting machines were created to hold the stone, spin the grinding lap and make the process easier. As Scott described, temperature, air pressure, humidity, flex in any of the machine parts is critical and makes complete automation currently out of reach in any practical way. There are automated faceting machines that will produced a finished stone, that might sell for $5, whereas a hand cut stone from the same material by a skilled human, using a manual faceting machine, could sell for several orders of magnitude more.
I guess that it will be much less expensive to have a robot practice with glass before real diamonds. But that would be a great thing for a robot to do. They can have so much smaller hands than humans, and so many other advantages! Maybe for the first year, make "Diamonds" that are 100 scale, about 3/4" diameter, and make them out of stone like marble, that can be sold for $20 or so? Then turn it loose to make them all week long.
Very interesting, thanks
Incredibly interesting discussion. Well done.
To help plants grow in hard-packed dirt you loosen the dirt by sticking a soil grapple into the dirt and angling the grapple 30-40 degrees. An Optimus could do this day and night in all kinds of weather. My daughter would love that Optimus.
PERFECT group and topics. What a joy to listen!!! Nice one Farzhad. Let it not be the last one PLEASEEEEEE!!!! This made my day ❤
This content is nerd gold!
This is an all star cast of some super intelligent guys. Very enjoyable video.
Great idea about the headlight shadows.
Fantastic episode.Thank you Farzad, and everybody there.
Awesome - so good my bot replayed it for me. Thanks
kudos to James. remarkable restraint while dancing with the physically challenged.
What great gems of wisdom were included in this discussion?..... Absolutely brilliant!
I was very excited to see the part that James Douma described at about 1 hour 10 minutes into this video, where the graph was robots learning to walk together. My thought was re-enacting the dance in a movie like White Christmas or another famous one. Even a dance routine from a less famous bar, such as in Seven Ways From Sundown, with 6 ladies dancing around a bar. Then train them to be back up dancers on a rap song or other things!
This is by far my favorite interview/ talk about Tesla Bot on the internet. Scott and James are like rocket fuel! I can’t wait to hear them discus more! -Noah
Much appreciated guys!
very long content, but highly technical and even as a layman I can understand a 100% of it. Thank you for this. This is some deep stuff right there, but hey, testing my investment hypothesis is fun
Walking the dog includes putting a harness on the dog, Connecting the leash, ensuring the leash is direct to the harness, not wrapped around a tree, letterbox, under the dogs legs. Must be able to correct. Of course there's picking up and disposing the poop.
The James Douma show!
This is great, great guys, respect. Farzad is good, he is as smiling focus hawk, got good questions, love this, please do update the progres.
Fantastic! Great learning for us! Thank you all!
I think Google and Tesla joining forces for object identification, creativity and companion AI conversation/speech would make sense. It's rare that Tesla working with another company makes sense, but I could see Google being helpful.
Loved this beyond words! Keep it up, guys!!
My thought was re-enacting the dance in a movie like White Christmas or another famous one. Even a dance routine from a less famous bar, such as in Seven Ways From Sundown, with 6 ladies dancing around a bar. Then train them to be back up dancers on a rap song or other things!
Very informative discussion. Thoroughly enjoyed it.
Thanks!
Thanks
So, James HAS read "On Intelligence" ☺Great book!
Yee this HAS to be a frequent thing!!!
Haven’t finished viewing the video (project due today) , but wanted to say that this show is awesome!
CGI in the movies uses sensors attached to or worn by a human being projected on a 2D screen with 3D objects in their path to simulate obstacles. It seems that CGI sensors should be included in to the X,Y,Z programming of a robot navigating through a 3D universe. Also autopilots in aircraft using a radar altimeter allows an aircraft to navigate from departure then to landing. ATSB allows an aircraft to sense other aircraft in it’s vicinity thereby allowing an aircraft to predict a probable conflict in the airspace allowing the pilot/operator to choose a different course to avoid probable conflict. A robot should be easy with fast processors and cameras to integrate probabilities.
Love this group of guys keep it
These guys create their own problem, discuss between each other and eventually draw possible solutions.. A bunch of cleaver guys.
Bots will be able to hear your heart beats and your breathing frequency and knowbyou feelings and emotions
SCOTT, IS A FUNNY GUY 😊…and JAME’S comments on definitions…is why I DISLIKE DICTIONARY DEFINITIONS 😛
The ring lights in Scott’s eye - Artificial Eye for Hypnosis.
...such positivity...awesome!
Farzad! If you get the chance interview some process engineers on Optimus. Time and time again what kept comming up was manafacturing processes. If you want the absolute best information then there's nothing better than PE's that live it day in and day out. I bet they would have some pretty shocking insights.
16:05 That's a really low bar for GAI. Not even close. GAI might be more like something that can act on the following prompt: "Design and implement an effecient manufacturing process for a device that assembles any Ikea furniture, and develop and train the control software for the device."
Nails and screw will have sensor or stop nubs for bots
Hi everybody! You are AWESOME!!!!
James Douma: "The Optimus would learn a lot that day" ... maybe it was a well-behaved dog ;-)
I knew Scott Walter was a cyborg. That robotic left eye 👁🗨 gives him away.
Loved this!!😊
The problem with walking a 7-15 pound dog should not be a problem. However a 70 pound lab or Dalmation, that is another story! They can pull with 40 pounds in one direction and then quickly pull in the other direction! I hope after learning on a 15 pound dog, they can figure out the 100 pound dogs!
My 65 pound German Shorthair could pull me on a bicycle and did a great job. However a couple of times she made me crash, and I was a bit upset, and the dog was trying to say "I am sorry". Then I bought a SACCO dog cart, that has 4 wheels and did not crash, it had brakes and rear suspension. She could pull me at 15 MPH for about 2 miles.
With the optomos dog walking model. The difference between walking and male dog and female is huge. Also if the dog is small and attacked by a large or aggressive dog, the bot won’t lift the dog up to protect it against attack. It may just come home dragging a scrap of dog on a leash. This is a complex situation just judging the intent of other dogs. Or the common problem of the dog going on one side of the sign post when you are on the other side. Good luck with that. Still throws me a curve sometimes. I can’t imagine the amount of programming needed for a simple job like this
Yeah Sir quite impressive for global boat Tesla family
Any human commands will have to programs with time limits or it will try to find beer and not stop until it completely done
Your toung knows how everything feels just my looking at it
Good discussion. 🙏 Scott’s microphone is picking up hand falls on his desk. I’m sure he can easily fix this if he’s aware of the problem.
My scan tool should also show the VIN.
1:11:43 very lovely moment baby and and neural net comparison🤣😂👍
Scott has a robot eye!
Of all the tasks that a robot might do, I think walking the dog will be one of the last. In fact, I kinda doubt it will ever be able to do that, because dogs instinctively respond to human emotions, and depend on the ability to interpret both vocal intonation and facial expression. I'm not saying it's impossible to mimic those things, but if the "uncanny valley" can make people uncomfortable, imagine how a dog would react.
I feel for you Farzad. You weren’t able to comment much but I would like to hear your viewpoint 👍
Doesn't Tesla calculate the elevation change into the range making a z-axis and holding a specified speed on elevation changes giving it variables on the same axis. Not only that the obstacles should be considered the prime axis. The vehicle no matter what can't crash.
Learning evolves when the fridge is NOT on the 1st floor. Real Men can find the beer 🍺
Right now, we are still in the February of nines.✌😎
When James shrugged off pedestrian intent and said it's actually not a hard problem it shows you how much the average person is underestimating fsd and real world ai.
We need to put Gordon Johnson with this panel.
Sound frequency will be decoded for different jobs by a i
I watched a sandy munro video when asked what Elon should do next and he said use infrared cameras? I was wondering if photon counting would detect infrared noise normally removed by filtering and extract useful data?
Walk the Dog, Bring Me a Beer, and 20 other yoyo tricks you can learn in one day (or less). 😉😉
Where is happening?
Could you tell me more about the neurolink project? I am a recovering TBI patient and sure would love to have an advantage in the world.
Soooo...let's start with Teslabot & Cybertruck on Mars.
Where did I put it, communicate with the vehicle. GPS, local,izing sensors
B2B (Bot 2 Bot)
Nice
Wow this went on forever
Translate: Wow we get a huge amount of high level info for free in a step by step format so we can follow along!
@@lmcclymont yep
Scott Walter = Alastor Moody
1:21:50 Transformers!! 😉 👍
I am a trim carpenter and I know there are times hey I have a piece of crown that I'm putting up and I can hear and feel when I'm about to break the crown just by my taps on the hammer there are times when I stop pull the nail out and readjust to accommodate for what I experienced That's another really complicated scenario for a robot
What is your opinion on implementation of "Fear" as a mechanism for self preservation? in reference to robot operating system. Even the smallest animals ( small insects, or worms) have a sense of danger and react to prevent damage. This behavior will have to be present in robots as well.
Nooooooo!!! That would be the key to creating robots that would destroy humanity. It's the old brain's fear, lust, and domination drives that cause humans to act against the collective's benefit. If we avoid the lizard brain instincts in AI and robots, they will never have the selfish desires that have doomed humanity to murder, rape, and war.
For further reading on the subject I'd suggest Jeff Hawkins' book, "A 1000 Brains"
Can you make a video about how Tesla's AI tools could be re-purposed at Twitter?
- Machine learning
- Auto-labelling
I think one person said that the machines are looking for a Bot on Twitter that is re-twitting, at a speed faster than a speed typist on crack. Then discovering that is not a real person but a Bot.
I have a feeling that the price of Twitter is based on 8 billion real people as customers, and a slight discount for the Bot that is not a human customer that will buy things advertised on Twitter. So the more Bot's that are discovered in the next few weeks will earn a discounted price for Twitter. AI can help discover these Bot that are not real humans.
Why isn't human movement a language model like Tesla is doing with Lane connection at intersections?
How about Twitter search instead of google search
FSD Beta 11.4 has all sorts of ego human interactions. In fact they have a lot of human movement to use. I bet Optimus is walking in the FSD virtual world.
Dream Team 😎
I think gaming industry helped a lot of robots programming considering all motions captures and method they are using doing characters.
AI Will be writing new programs to improve the human conditions!
Optimus will need to understand a dogs weight, triggered responses and bowel movements. Bending down to scoop up a formed stool verses a loose one is not only a visual input, but a olfactory one. My Rottweiler by the way, weighed 120 lbs and was as gentle as a human can be. He also could kill another living being if provoked.
Seems like Scott has a crazy cyborg eye 😃
Hey guys one thing that I realize you're missing is sacrificial action All humans hey right now in the video I'm listening to you about driving a nail but there are times when that nail hits a really hard part of wood called grain and then I have to adjust that would be the sacrificial adjustment because I need to change what I'm doing if not it's a nail's going to go a different direction than I want and I already see it I understand it that's something that we have to teach I hope that makes sense
Your saying that they couldn’t put a super dark filter over the lens?? I can put on a shield and watch my bead run why can’t I film it?
Driving a 3d nail with a 3lb hammer is a very funny hypothetical if tried in the real world.
Robot programmer could simply be the internet. Especially youtube. Like the language models use the internet of text, could we expand that to video? Just make sure it doesn't watch a lot of cartoons I guess
The nerve-stuf=sensors.
IKEA furniture build as a service..?
I like the one that is not so smart.
Which one is that?
2:01:00 SCOTT 😆 😆 🤣 😂 😹
New tools supplies will be invented just for bots.
Bots tools and supplies new TAM
Farzad, the thing you're talking about @53min (that Tesla does much better than others) is called "Integrative Design". One of the hallmarks is using a single element for multiple purposes, such as this idea to use the headlights as an active sensor to detect objects. One of the OG's of this methodology is Amory Lovins, co-founder of the Rocky Mountain Institute. He presented this talk a few weeks ago: _"Radical Energy Efficiency and Profitable Climate Protection through Integrative Design"_
watch?v=V2aJlYRtmxk
ジェームスはRUclipsを日本語で見ていますか？ビックリ！