Always amazed at the number of people doing videos on Tesla that are extremely knowledgeable about what they are reporting on. As in this case your input and understand on the issue of simulations and AI training are much more than speculation or general knowledge experience. Your being blown away with what Tesla is doing and where they are headed means much more than most peoples thoughts on the matter. As always your insightful and knowledgeable discussion is a great aid to the publics understanding of what is going on. The media and the people they bring on are mostly people totally unable to speak knowledgeable on such topics. It's deeply appreciated by so many here that you take the time and effort to produce your videos.
Love your passion and thanks for the bio -- impressive. You are a natural teacher. Can't wait for your new episodes on the topic -- totally captivating stuff. Imagine the tens of thousands of people around the world churning out countless ideas on how to exploit this ground-breaking work. Thanks too for your dedication to this community. We appreciate you!
Something interesting I noticed when I started watching the Tesla AI day stream is that they are also 3D rendering/simulating a full inside the cockpit view, complete with a rendered driver operating the controls. In addition to using the addition view from the cabin camera to help with self driving, I’m sure they are intending to use this to keep track of driver attention and possibly even inputs eventually. Imagine if the turn signal stalk sensor fails but the system still knows to trigger the signal light based on the driver actuating the stalk.
Dr. Know-it-all, I have watched dozens of your videos and appreciate your knowledgable position on a diversity of matters. however, on this video you really seem to “know-it-all”. nice to see you in your glory. thanks for sharing.
The reason there were so many police cars is the whole purpose of using simulated data. To catch the rare edge case, the first one, the left turn was cars going straight when there was a green left arrow, and the second was the whole police force in pursuit. They need to train the AI to handle super rare edge cases.
The point of simulation is it is omnipotent. It knows all. Hence, you can test the labeling software because you know the correct answer and can check answers. Doing that in real world is slow because a human has to tediously double check the answer. That is the beauty of simulation: you can quickly iterate the labeling system and chase the 9's faster, then compare with real world later.
Simulations make it easy to create edge cases that are too rare to be captured in significant numbers in real life (until there are more Teslas). So you can easily create a simulation of a crowd of people exiting a bus that has crashed into a car, or a bunch of cop cars racing to an accident, or a cow lying down in the road.
Thank you. Just wanted to say thank you for producing Dr. Know It All. I have really enjoyed all your shows over the past year. Hope you never lose that passion for learning and the drive to share your learning journey.
I watched the whole AI day and thought 'OK, I get it, well most of it, can't' expect to follow i all eh' but this series of yours has made me realise just how much I missed. Best episode yet? Keep up the good work, and Thank you!
Great point! The physical actions necessary for driving manual autos would be an excellent first project for these robots - and then Tesla already has the FSD software. This could be quite useful if we use MSR power to make hydrocarbons from CO2 and water for powering legacy ICE ground/air transportation, and to accelerate the move to FSD. . Another related task would be product delivery (with or without a FSD vehicle).
@@red-baitingswine8816 They already do the steering, accelerator and brakes and can shift an auto. There are also manual gearboxes that can be shifted. As for CO2 being made into hydrocarbons, I wonder how many EVs could be powered compared to making a tank full of petrol for one ICE car. Do you know how energy intensive converting CO2 would be?
@@SirHackaL0t. No I don't, but I think it should be used for at least those vehicles that require HCs (like the military - biggest polluter on the planet - more even than China) or possibly many existing semi trucks (where battery size is a big issue), if not all ICE vehicles. Simple fast spectrum MSRs burn every kind of nuclear waste - so that their fuel costs (for e.g. manufacturing hydrocarbons) are actually negative! (see "Elysium", "Ed Pheil" on RUclips)
@@SirHackaL0t. "Do you know how energy intensive converting CO2 would be?" I don't have the answer, but the US Navy developed a method to make jet fuel at sea at a cost roughly double the cost of jet fuel on land, but then petro prices stopped fluctuating and the project got dropped. Gasoline would be cheaper than jet fuel, but you can't get it as cheap as sticking one end of a straw in the ground and the other end in your gas tank. The Navy would not have been interested in a process that was too inefficient as the heat signature of the ship would have been excessive for it to shadow a carrier which it was developed for. The process extracts CO2 from sea water, extracts H2 from distilled water, and produces O2 waste gas.
@@DrKnowitallKnows can simplify some of what James Douma says too. Sometimes it just gets abit too technical and hard wrap my head around. @Dr. Know-it-all Knows it all is just so clear at explaining these stuff. Dave should totally join in too!
Another great explainer. I really like that you kept it on point, even though I'm sure you wanted to do a 2 hour deep dive based on your expertise in this area. Loving the series.
As someone not educated in these areas that just finds all this extremely interesting... you've done an amazing job breaking down this part of A.I day! I'll continuing to look forward to more videos great work!
I can really see the advantages of simulations with animal interactions. For example In Australia we have the kangaroo, which has a death wish, whenever it encounters a car. Many car manufacturers have safety sensors that work with other animals, due to predictable natures. But these are defeated by kangaroos, particularly large males, who seem to have a ninja complex... A simulation could also be used for examples of extreme behaviour that can't be normally tested\trained for. You might be able to test drunk\drugged\sleepy driver interactions. Ultimately the best source of training material might be truck drivers or forensic vehicle accident analysts rather than programmers.
@@DrKnowitallKnows Definitely a test of willpower when you have a 7ft Kangaroo in the middle of the road staring you down through a thin glass windshield! Our local access road was called "Kangaroo Valley" & even the school bus had to be careful...
Great video. I was super impressed that you have had such extensive experience with 3D. I have taught basic Cinema 4D for many years (just retired from university), but just super basic stuff for beginning students in Computer Motion Graphics-certainly nothing on your level of experience. Glad you called out Gordon ‘Brain-Dead’ Johnson. I can’t believe the guy is so dense that he apparently doesn’t even understand what he is seeing-so even if he watched your video (or Lex Fridman’s pod cast, or any of James Douma’s interviews) I don’t think he would get it. It’s just too bad that he has the potential to spread his misinformation to thousands of unsuspecting people because mainstream media keeps giving him a platform to spout his false narrative (why does he keep getting interviewed anyway as though he actually has a valid point of view?).
@@DrKnowitallKnows CNBC, Yahoo Finance, Bloomberg, WSJ, consumer reports and an occasional well meaning but very stupid local reporter. 👆 Just reminding you some of the guys you'd be dealing with. And it's not like "watch this detailed video I made" like you do here - it's answering malicious questions, then seeing your answers taken out of context. So. E good experiences too, for sure, but If you're anything like me, you're probably better off working the tech than promoting it to some of these guys.
7:27 I always thought TESLA/SpaceX should release a physics simulation game ("GTA" anyone, Kerbel anyone). The idea being that users will do stupid things that engineers will never think of. In the case of TESLA it could make a really "kewl" corpus of edge cases. In the case of SpaceX it might lead to an unintuitive solution to a problem.
One interesting use of the simulation tech would be for analyzing different road designs for safety and flow efficiency, as well as for looking at specific places with a high reported frequency of accidents to analyse how to avoid getting in one of the bad situations.
Its also very usefull to train AIs in places where Teslas cannot self drive yet or where there isn't a lot of vehicles. For example if a country is making a fit about the self driving, and does not let Tesla check if things work practically, they can use simulations overlayed on their data to pre-train the network on data it could not yet obtain. This is very important for example if Tesla want to do well in countries like france, Germany or japan that severely restrict self driving to protect the interests of their legacy manufacturers
The weird colors from 7 of the 8 camera's is mostly because of the RCCB array used to increase low light sensitivity, instead of the RGGB array normally used. The older (pre AP2.5) camera's even used RCCC. Only the backup camera uses RGGB, and you can see the difference clearly.
Thank you. This is entering into rhe level of detail I want to see. Awesome. 24:40... I was wondering about that. Whether they used a different output function on intensity... humans don't see blue very well, but cameras do... we see yellow well. Camera see yellow better. They get to take advantage of photons I will never see.
That was a fantastic Video. I knew you would have something incredibly digestible. I understood a lot of it, by there was certainly some gray area. Thank you!!!!
Was super excited to as someone who's a big photogrammetry and deep learning guy. Was seriously thinking about applying. Currently stuck in New Zealand though.
Nice interview vid! And very informative to the rest of us. Yes, my understanding from AI day was like yours, simulation let them augment the real world safely!
I noticed the Maya books a long time ago. I started using Maya 2.5 back in grad school after finishing a Bachelors in Architecture, I was pursuing a Master’s in Fine arts at Pratt institute to enhance my architectural studies.
Hey Dr, I think I have a great idea for your next video! A comparison between EXRO Technologies and what Tesla are currently using in their motor and battery tech.
i read about people walking or biking on the highway often enough (luckily its usually the safety lane) last was a week or 2 ago 6 year old girl on her way to grandma in the middle of the night, she only had 100 more kilometers to walk
Regarding the "crappy" cameras. Make sense to capture the images in LOG, but why would Tesla not convert it to Rec 709 in realtime for the screen? Or apply a quick LUT? Would that be too demanding on the hardware?
The incoming traffic keep going in spite of the left turning signal from this side, I don't know that would happen anywhere in the U.S. I saw that in Europe though, I was freaked out.
It looks way better than the really basic and crude simulations Waymo uses. But they don't really use a lot of Machine Learning for image recognition, they seem to rely on very explicitly mapped roads and stick to pre made lines and road rules that the car follows. Rather than doing it all on the fly
Your description flipped the terms simulation and real work just before 14:22. Simulation (I think) is the icing on the 🍰, not real world. 🙂 I believe they are using real world training to do the majority of the training and simulations to fill out edge cases. However, I might be wrong and in theory you could generate so many simulated scenarios with slight variations that it rivals or surpasses the real world data.
I really do enjoy your analysis even though I might not understand all the technical details. But there's something I noticed in all your videos, perhaps you are not aware of it, but I felt I should mention it. You use the word "ANYWAYS' a lot, I think more than necessary. Just something minor I felt I should mention. But overall you do an excellent job.
Hey John ,I got to have that t-shirt,but I don't see it on the merch store .Perhaps it was the lighting but it looks like Black with a coral/pink logo.Maybe it's the red logo but looks pinkish.....
I don't think Gordon Johnson is an idiot, he just intentionally misleads idiots in a pathetic attempt to sabotage Tesla. Mind you, it's Tesla-haters like Gordon Johnson who got me into buying Tesla stock in 2019 though so I'll always be grateful to them!
They need to simulate a 1500 pound big pig running across the road in front of you. I came across the result of this many years ago. The Dodge van that hit it was totaled. It looked like it had hit a slug of concrete.
I think Tesla mentioned they use 3000 FSD computer boards for validation of a beta build before release. So each of these FSD computers thinks it's driving a real car on a real road, but it's actually simulated. This allows them to verify that each new build still passes all the old test cases. In other words, when they fix bugs on a new build, they want to make sure they haven't broken something that used to work. Is this correct?
I only have a basic understanding of neural networks but I had a thought on this subject and a couple questions: When you create simulated images to train a neural network which sets the weights in the network, then you have a mapping between the input images you created and the weights you're trying to find. So can't you theoretically avoid the whole learning process and just create the weights directly? Granted it would be a super hard thing to do but I just wonder if it would be mathematically conceivable. (Neural network learning process is so trial-and-error.) My second question is about the limits of training on a network. Say that one input image is pulling a weight one way and another input image is pulling the same weight a different way. The network should start to "forget" or unlearn if you over-train it. Is there mathematical formula to determine how many nodes/layers you will need for a given problem? Does Tesla know that they have enough and how do they know the existing hardware in the cars will be enough processing power?
In the first simulation, the left turn green arrow was lit, but the oncoming traffic kept coming, that's an inaccurate simulation, since once the green left turn arrow is lit, the oncoming traffic is stopped, allowing you to safely turn left, across traffic.
How come you don’t have any examples of your work on the introduction or the exit of your videos? You really need to do a polished three dimensional bumper introduction for your videos
Tesla needs to make a simulation of 'wrong-way-drivers'. Here in AZ, we have an epidemic of them driving down highways and off ramps in the wrong direction.
A couple of thoughts: 1. With photo realistic rendering Hollywood might be a customer for Dojo, for example a remake of Titanic would be child's play. 2. An example of a potential use for siumulated data is the classic driver's ed scenario of a child playing with and following the ball into the street.
Can FSD work in a situation where in the backdrop is entirely different? For instance in India, we will find cows instead of moose and sunlight is a bit different?
fully respect all FSD work has been done by Tesla. However $10,000.00 price tag is way too far!!!! Not many people really need to understand all these tech details, All we need is a final product that worth $10,000.00 paid for a long time ago.......Any ideas on when it will happen? Not sure if all these experts really bought the FSD for $10.000 yet?
Will simulation allow the creation of a GAN ? One side of the network making up insanely challenging but photorealistic and physically realistic scenarios while the FSD network tries to deal with them, only to encounter even more scenarios in the next round?
Something you touched on briefly is I feel incredibly important: auto labeling. While humans are good at fuzzy logic, if I showed you a scene of people and cars moving around and asked you to label manually not only the objects but their velocity vectors with relative precision, you would have difficulty putting a numbers to those vectors. Is that person walking at 1 ms/s velocity or 1.284745 m/s? The generated scenes with auto labeling knows these values more precisely than any human, so even if the number of objects is low enough that human won’t give up labeling it out of frustration with the tediousness of it, the generated data is invaluable for training in terms of precision. Eventually the system will get better than a human at estimating positions and velocities.
Job well done! Thank you very much. BTW: you really don't want to be a Tesla employee; although they appear to be getting great job satisfaction, they don't appear to be having much fun. Join up with James Douma and the two of you would be really useful to Tesla as advisors/consultants not as problem solvers but instead as problem finders/anticipators.
All the road markings are done like then were painted just yesterday. In Ohio some/a lot of the markings are hardly readable. Shouldn't the simulation have some faded road markings?
Can’t they just get separate video of element and merge, repeat, etc using video editing into a final scene? Seems like it would be dramatically simpler but at the loss of fine grain control but it is camera video.
I'm not versed enough in this stuff to really explain it properly, but none of the images, nor the labeling, makes it into the software on your car. The trained neural net that the car runs only looks at the real-time camera data from your vehicle and decides what to do based on the "experience" baked into the virtual "neurons". So training the network takes a lot of processing power and a lot of data, but while running it, it will automatically make the right decision in a few milliseconds without really having to "think" about it. I hope I didn't confuse you more.. =P
@@RufftaMan so if not images or labels, objects are named and recognized from past "experience" without having a picture of the object? Makes sense, but there must be some base coding somewhere. Guess I need to find a video that explains the basics of neural networks before I fully understand. Thanks.
Some of those passes looked like cryptomatte. Would love to see their pipeline. The albedo and no specular on the cybertruck let me know it wad some RTX magic going on. Sgould have watched on my big screen and not the phone.
2:26 "Gordon the Genius Johnson" Aaahhh ! Almost fell off my chair laughing!
He stole that line from solving the money problem. He gave no credit
@@gylfipleasures2013 Still funny.Probably gonna stick :-)
I was floored by their occlusion strategy and how they used all of their bad weather data to create accurate labeling. Wow.
Always amazed at the number of people doing videos on Tesla that are extremely knowledgeable about what they are reporting on. As in this case your input and understand on the issue of simulations and AI training are much more than speculation or general knowledge experience. Your being blown away with what Tesla is doing and where they are headed means much more than most peoples thoughts on the matter. As always your insightful and knowledgeable discussion is a great aid to the publics understanding of what is going on. The media and the people they bring on are mostly people totally unable to speak knowledgeable on such topics. It's deeply appreciated by so many here that you take the time and effort to produce your videos.
Love your passion and thanks for the bio -- impressive. You are a natural teacher. Can't wait for your new episodes on the topic -- totally captivating stuff. Imagine the tens of thousands of people around the world churning out countless ideas on how to exploit this ground-breaking work. Thanks too for your dedication to this community. We appreciate you!
Something interesting I noticed when I started watching the Tesla AI day stream is that they are also 3D rendering/simulating a full inside the cockpit view, complete with a rendered driver operating the controls. In addition to using the addition view from the cabin camera to help with self driving, I’m sure they are intending to use this to keep track of driver attention and possibly even inputs eventually. Imagine if the turn signal stalk sensor fails but the system still knows to trigger the signal light based on the driver actuating the stalk.
For those with any knowledge in this area, this may be your absolutely best segment yet! Super useful! Congratulations! How informative!
Thank you. For once dr know it all is actually true. Lol
Dr. Know-it-all, I have watched dozens of your videos and appreciate your knowledgable position on a diversity of matters. however, on this video you really seem to “know-it-all”. nice to see you in your glory. thanks for sharing.
The reason there were so many police cars is the whole purpose of using simulated data. To catch the rare edge case, the first one, the left turn was cars going straight when there was a green left arrow, and the second was the whole police force in pursuit. They need to train the AI to handle super rare edge cases.
The point of simulation is it is omnipotent. It knows all. Hence, you can test the labeling software because you know the correct answer and can check answers. Doing that in real world is slow because a human has to tediously double check the answer. That is the beauty of simulation: you can quickly iterate the labeling system and chase the 9's faster, then compare with real world later.
Simulations make it easy to create edge cases that are too rare to be captured in significant numbers in real life (until there are more Teslas). So you can easily create a simulation of a crowd of people exiting a bus that has crashed into a car, or a bunch of cop cars racing to an accident, or a cow lying down in the road.
Thank you. Just wanted to say thank you for producing Dr. Know It All. I have really enjoyed all your shows over the past year. Hope you never lose that passion for learning and the drive to share your learning journey.
Oh, I never made the connection that your son made the intro music. That's cool! And it's genuinely good.
Intro to William F'buckley's "Firing Line".
I watched the whole AI day and thought 'OK, I get it, well most of it, can't' expect to follow i all eh' but this series of yours has made me realise just how much I missed. Best episode yet? Keep up the good work, and Thank you!
Tesla: "You have three old cars without self driving? Get our Tesla Bot, it can drive all of them". Tesla Bot: "Please don't let it be stick shift!"
🤣🤣🤣
Great point! The physical actions necessary for driving manual autos would be an excellent first project for these robots - and then Tesla already has the FSD software. This could be quite useful if we use MSR power to make hydrocarbons from CO2 and water for powering legacy ICE ground/air transportation, and to accelerate the move to FSD.
.
Another related task would be product delivery (with or without a FSD vehicle).
@@red-baitingswine8816 They already do the steering, accelerator and brakes and can shift an auto. There are also manual gearboxes that can be shifted.
As for CO2 being made into hydrocarbons, I wonder how many EVs could be powered compared to making a tank full of petrol for one ICE car. Do you know how energy intensive converting CO2 would be?
@@SirHackaL0t. No I don't, but I think it should be used for at least those vehicles that require HCs (like the military - biggest polluter on the planet - more even than China) or possibly many existing semi trucks (where battery size is a big issue), if not all ICE vehicles. Simple fast spectrum MSRs burn every kind of nuclear waste - so that their fuel costs (for e.g. manufacturing hydrocarbons) are actually negative! (see "Elysium", "Ed Pheil" on RUclips)
@@SirHackaL0t. "Do you know how energy intensive converting CO2 would be?" I don't have the answer, but the US Navy developed a method to make jet fuel at sea at a cost roughly double the cost of jet fuel on land, but then petro prices stopped fluctuating and the project got dropped. Gasoline would be cheaper than jet fuel, but you can't get it as cheap as sticking one end of a straw in the ground and the other end in your gas tank. The Navy would not have been interested in a process that was too inefficient as the heat signature of the ship would have been excessive for it to shadow a carrier which it was developed for. The process extracts CO2 from sea water, extracts H2 from distilled water, and produces O2 waste gas.
You and James Douma should have a “sit down” discussion that we can watch. If you talk over our heads -- so be it 🙂
Agree, you and James would make some stunning content. I do think that the underlining rendering technology is an unreal engine.
I’m going to invite him and see if he’ll take me up on it.
@@DrKnowitallKnows he's been doing a lot of interviews of late I hope he ads you to the list.
@@DrKnowitallKnows can simplify some of what James Douma says too. Sometimes it just gets abit too technical and hard wrap my head around. @Dr. Know-it-all Knows it all is just so clear at explaining these stuff. Dave should totally join in too!
All of those cop cars were probably a nod to them training to be able to recognize emergency vehicles.
Yes, absolutely!
This was a great episode, really appreciate your take.
Another great explainer. I really like that you kept it on point, even though I'm sure you wanted to do a 2 hour deep dive based on your expertise in this area. Loving the series.
As someone not educated in these areas that just finds all this extremely interesting... you've done an amazing job breaking down this part of A.I day! I'll continuing to look forward to more videos great work!
I can really see the advantages of simulations with animal interactions. For example In Australia we have the kangaroo, which has a death wish, whenever it encounters a car. Many car manufacturers have safety sensors that work with other animals, due to predictable natures. But these are defeated by kangaroos, particularly large males, who seem to have a ninja complex...
A simulation could also be used for examples of extreme behaviour that can't be normally tested\trained for. You might be able to test drunk\drugged\sleepy driver interactions. Ultimately the best source of training material might be truck drivers or forensic vehicle accident analysts rather than programmers.
Yikes. You Aussies have all the crazy creatures. Definitely simulate those ninja attacks and learn to run away!
@@DrKnowitallKnows Definitely a test of willpower when you have a 7ft Kangaroo in the middle of the road staring you down through a thin glass windshield! Our local access road was called "Kangaroo Valley" & even the school bus had to be careful...
Congrats DNIA. Thanks, go for it, we need these explanations on labeling, training and the other arcana of the journey to autonomy.
Great video. I was super impressed that you have had such extensive experience with 3D. I have taught basic Cinema 4D for many years (just retired from university), but just super basic stuff for beginning students in Computer Motion Graphics-certainly nothing on your level of experience. Glad you called out Gordon ‘Brain-Dead’ Johnson. I can’t believe the guy is so dense that he apparently doesn’t even understand what he is seeing-so even if he watched your video (or Lex Fridman’s pod cast, or any of James Douma’s interviews) I don’t think he would get it. It’s just too bad that he has the potential to spread his misinformation to thousands of unsuspecting people because mainstream media keeps giving him a platform to spout his false narrative (why does he keep getting interviewed anyway as though he actually has a valid point of view?).
Fellow Georgian here. Loving your insights and that shirt is awesome!!
I know you want to get a job as an engineer but you would be the greatest PR Tesla guy!
He is, as are many other youtubers… they just aren’t getting payed by Tesla.
@@XeonSX agree :-)
That would be my dream job: a communicator for Tesla and their mission. ❤️
@@XeonSX 🤣🤣🤣
@@DrKnowitallKnows CNBC, Yahoo Finance, Bloomberg, WSJ, consumer reports and an occasional well meaning but very stupid local reporter.
👆 Just reminding you some of the guys you'd be dealing with. And it's not like "watch this detailed video I made" like you do here - it's answering malicious questions, then seeing your answers taken out of context.
So. E good experiences too, for sure, but If you're anything like me, you're probably better off working the tech than promoting it to some of these guys.
7:27 I always thought TESLA/SpaceX should release a physics simulation game ("GTA" anyone, Kerbel anyone). The idea being that users will do stupid things that engineers will never think of. In the case of TESLA it could make a really "kewl" corpus of edge cases. In the case of SpaceX it might lead to an unintuitive solution to a problem.
That would be kick ass!
What a great idea!
That's a cool idea
Thank you. Wonderfully clear and enlightening.
One interesting use of the simulation tech would be for analyzing different road designs for safety and flow efficiency, as well as for looking at specific places with a high reported frequency of accidents to analyse how to avoid getting in one of the bad situations.
Its also very usefull to train AIs in places where Teslas cannot self drive yet or where there isn't a lot of vehicles.
For example if a country is making a fit about the self driving, and does not let Tesla check if things work practically, they can use simulations overlayed on their data to pre-train the network on data it could not yet obtain. This is very important for example if Tesla want to do well in countries like france, Germany or japan that severely restrict self driving to protect the interests of their legacy manufacturers
Excellent presentation of complex technical material. Much appreciated.
Excellent! Thank you for the explanation. High quality content as usual!
The weird colors from 7 of the 8 camera's is mostly because of the RCCB array used to increase low light sensitivity, instead of the RGGB array normally used. The older (pre AP2.5) camera's even used RCCC. Only the backup camera uses RGGB, and you can see the difference clearly.
Thank you. This is entering into rhe level of detail I want to see. Awesome.
24:40...
I was wondering about that. Whether they used a different output function on intensity... humans don't see blue very well, but cameras do... we see yellow well. Camera see yellow better. They get to take advantage of photons I will never see.
That was a fantastic Video. I knew you would have something incredibly digestible. I understood a lot of it, by there was certainly some gray area. Thank you!!!!
Was super excited to as someone who's a big photogrammetry and deep learning guy. Was seriously thinking about applying. Currently stuck in New Zealand though.
Waiting for the next episodes... ⏳
Nice interview vid! And very informative to the rest of us. Yes, my understanding from AI day was like yours, simulation let them augment the real world safely!
Great explanations I learned a lot. Thx
I noticed the Maya books a long time ago. I started using Maya 2.5 back in grad school after finishing a Bachelors in Architecture, I was pursuing a Master’s in Fine arts at Pratt institute to enhance my architectural studies.
Thanks for the great video!
Hey Dr, I think I have a great idea for your next video! A comparison between EXRO Technologies and what Tesla are currently using in their motor and battery tech.
The simulation is so good that It must have commercial value for a computer game based around a Tesla car, perhaps to be played in your Tesla!
That would be so cool. Elon needs to make that happen!
One day your Tesla will tell you: "FSD is complete, now I can drive in the real world". "But you are already driving in the real world". "No!"
i read about people walking or biking on the highway often enough (luckily its usually the safety lane) last was a week or 2 ago 6 year old girl on her way to grandma in the middle of the night, she only had 100 more kilometers to walk
Wonderful stuff, much appreciated 🙏
one of your very best videos!
Tesla AI Day was very informative and the tesla bot was fascinating
That merch is sick!
Awesome information Sir, Thank you so much :)
Oh, so THAT was the Easter egg. Hell, I recognized the intro as your CV from your earliest days, so likely did a number of other people....
James Douma and you chatting about Tesla AI
👍🏽👍🏽👍🏽🙏🙏🙏👍🏽👍🏽👍🏽
Regarding the "crappy" cameras. Make sense to capture the images in LOG, but why would Tesla not convert it to Rec 709 in realtime for the screen? Or apply a quick LUT? Would that be too demanding on the hardware?
brilliant episode, thanks
This video consolidated so much of my understanding of what Tesla are doing. Thank you.
You’re welcome. Thanks for watching!
Gordon (My Daddy is a GM Executive) Johnson is just there to keep CNBS's advertiser happy. Nothing more.
The incoming traffic keep going in spite of the left turning signal from this side, I don't know that would happen anywhere in the U.S. I saw that in Europe though, I was freaked out.
It looks way better than the really basic and crude simulations Waymo uses. But they don't really use a lot of Machine Learning for image recognition, they seem to rely on very explicitly mapped roads and stick to pre made lines and road rules that the car follows. Rather than doing it all on the fly
Loved that intro!
A really good use of sims is to use their 100% accurate data as ground truth to train NN's and see how well they do.
Your description flipped the terms simulation and real work just before 14:22. Simulation (I think) is the icing on the 🍰, not real world. 🙂
I believe they are using real world training to do the majority of the training and simulations to fill out edge cases. However, I might be wrong and in theory you could generate so many simulated scenarios with slight variations that it rivals or surpasses the real world data.
Gordon "My middle name is Dunning Kruger" Johnson, oh, know we know who you are talking about
Nice shirt! Nice content!!!
I really do enjoy your analysis even though I might not understand all the technical details.
But there's something I noticed in all your videos, perhaps you are not aware of it, but I felt I should mention it. You use the word "ANYWAYS' a lot, I think more than necessary.
Just something minor I felt I should mention. But overall you do an excellent job.
Great job explaining this and refuting Gordon Johnson’s argument.
Thank you!
Hey John ,I got to have that t-shirt,but I don't see it on the merch store .Perhaps it was the lighting but it looks like Black with a coral/pink logo.Maybe it's the red logo but looks pinkish.....
I don't think Gordon Johnson is an idiot, he just intentionally misleads idiots in a pathetic attempt to sabotage Tesla. Mind you, it's Tesla-haters like Gordon Johnson who got me into buying Tesla stock in 2019 though so I'll always be grateful to them!
Big thumbs up for not putting Gordon Johnson on your wonderful channel .
loved this!!!!!!!!!!!!!!!!!!!!! very informative!!!!!!
Why can’t simulation be used for validation? Can you elaborate on this?
Inverse of garbage in garbage out.
Fantastic talk!
awesome video, thank you.
the reason why several police cars is to give them absolutely priority as they are likely not patrulling but on a mission.
i do miss , wind and movement in tree's and plants , missing birds and sun light
Great job.
Absolutely super. Thanks.
They need to simulate a 1500 pound big pig running across the road in front of you. I came across the result of this many years ago. The Dodge van that hit it was totaled. It looked like it had hit a slug of concrete.
I think Tesla mentioned they use 3000 FSD computer boards for validation of a beta build before release. So each of these FSD computers thinks it's driving a real car on a real road, but it's actually simulated. This allows them to verify that each new build still passes all the old test cases. In other words, when they fix bugs on a new build, they want to make sure they haven't broken something that used to work. Is this correct?
I only have a basic understanding of neural networks but I had a thought on this subject and a couple questions:
When you create simulated images to train a neural network which sets the weights in the network, then you have a mapping between the input images you created and the weights you're trying to find. So can't you theoretically avoid the whole learning process and just create the weights directly? Granted it would be a super hard thing to do but I just wonder if it would be mathematically conceivable. (Neural network learning process is so trial-and-error.)
My second question is about the limits of training on a network. Say that one input image is pulling a weight one way and another input image is pulling the same weight a different way. The network should start to "forget" or unlearn if you over-train it. Is there mathematical formula to determine how many nodes/layers you will need for a given problem? Does Tesla know that they have enough and how do they know the existing hardware in the cars will be enough processing power?
In the first simulation, the left turn green arrow was lit, but the oncoming traffic kept coming, that's an inaccurate simulation, since once the green left turn arrow is lit, the oncoming traffic is stopped, allowing you to safely turn left, across traffic.
How come you don’t have any examples of your work on the introduction or the exit of your videos? You really need to do a polished three dimensional bumper introduction for your videos
They had all those cop cars at 16:56 because they're getting sued for hitting emergency vehicles and they need to train like hell against it ASAP lol.
Thanks.
Why use raytracing and lighting in vector space?
Tesla needs to make a simulation of 'wrong-way-drivers'. Here in AZ, we have an epidemic of them driving down highways and off ramps in the wrong direction.
like the shirt :) great show thanks
Love the new intro!
How do you know we are not now in a really good photo / physics realalistic simulation ?
Good luck with your job application
Congratulations.. thunderfoot. Has acknowledged u too
A couple of thoughts:
1. With photo realistic rendering Hollywood might be a customer for Dojo, for example a remake of Titanic would be child's play.
2. An example of a potential use for siumulated data is the classic driver's ed scenario of a child playing with and following the ball into the street.
Can FSD work in a situation where in the backdrop is entirely different? For instance in India, we will find cows instead of moose and sunlight is a bit different?
fully respect all FSD work has been done by Tesla. However $10,000.00 price tag is way too far!!!!
Not many people really need to understand all these tech details, All we need is a final product that worth $10,000.00 paid for a long time ago.......Any ideas on when it will happen?
Not sure if all these experts really bought the FSD for $10.000 yet?
Will simulation allow the creation of a GAN ? One side of the network making up insanely challenging but photorealistic and physically realistic scenarios while the FSD network tries to deal with them, only to encounter even more scenarios in the next round?
I hope they publish their labelled 3D renderings in an update so we can see realer visualizations while driving!
Something you touched on briefly is I feel incredibly important: auto labeling. While humans are good at fuzzy logic, if I showed you a scene of people and cars moving around and asked you to label manually not only the objects but their velocity vectors with relative precision, you would have difficulty putting a numbers to those vectors. Is that person walking at 1 ms/s velocity or 1.284745 m/s? The generated scenes with auto labeling knows these values more precisely than any human, so even if the number of objects is low enough that human won’t give up labeling it out of frustration with the tediousness of it, the generated data is invaluable for training in terms of precision. Eventually the system will get better than a human at estimating positions and velocities.
Safer for you but don't the sim's get hurt?
Job well done! Thank you very much.
BTW: you really don't want to be a Tesla employee; although they appear to be getting great job satisfaction, they don't appear to be having much fun. Join up with James Douma and the two of you would be really useful to Tesla as advisors/consultants not as problem solvers but instead as problem finders/anticipators.
All the road markings are done like then were painted just yesterday. In Ohio some/a lot of the markings are hardly readable. Shouldn't the simulation have some faded road markings?
Can’t they just get separate video of element and merge, repeat, etc using video editing into a final scene? Seems like it would be dramatically simpler but at the loss of fine grain control but it is camera video.
The more the likes of cnbc give airtime to clowns like Johnson, the more people will turn to RUclips and social media.
Agreed, CNBC is just hastening their demise. Hope the big ICE and petroleum companies are paying them a lot.
How do you move the data to the car? Is it line based programming in some language with a dataset that contains labeled images?
I'm not versed enough in this stuff to really explain it properly, but none of the images, nor the labeling, makes it into the software on your car.
The trained neural net that the car runs only looks at the real-time camera data from your vehicle and decides what to do based on the "experience" baked into the virtual "neurons".
So training the network takes a lot of processing power and a lot of data, but while running it, it will automatically make the right decision in a few milliseconds without really having to "think" about it.
I hope I didn't confuse you more.. =P
@@RufftaMan so if not images or labels, objects are named and recognized from past "experience" without having a picture of the object? Makes sense, but there must be some base coding somewhere. Guess I need to find a video that explains the basics of neural networks before I fully understand. Thanks.
What is your questimate on when fsd will be real
Some of those passes looked like cryptomatte. Would love to see their pipeline. The albedo and no specular on the cybertruck let me know it wad some RTX magic going on. Sgould have watched on my big screen and not the phone.