Thank you all for the support this past month, I really appreciate it! I got loads of exciting projects planned, so make sure to subscribe and turn on the notification bell so you don't miss out on my new videos :))
I have seen, how AI in trackmania, adjust it own track line to make fast lap, maybe try to train, your own ai in trackmania, and then move it to rc ? :)
If you really want to hate fine tuning, you could even start introducing slip angle, as on lower grip surfaces like polished concrete it can be faster to be putting power down earlier into corners and letting the tail of the car kick out a little bit. It would require gyros (which could also help with the oscillations at high speeds) but could also increase corner entry and exit speeds a fair bit
What I love most about these types of videos is that its pretty impressive already, but there's always a way to improve it, and more often than not, its a heavily explored topic
@@lagule just use ML lol. jokes aside, that seems like an "easy" problem to solve with ml, simulate the car, then use the pre-optimized program control of the car with some securities and let correct for the delta between simulation and reality. (by easy I mean it's only a couple months of work)
The class of car might be a problem. This type of RC truck races on carpet or dirt which includes jumps. Onroad (racing without jumps) cars are very light and low. Could easily find a pro driver or hobbyist to race against this but I imagine you'd need a handicap as the equipment on this thing will make it very top heavy by comparison.
@@TheFatAssCat I've seen videos of dirt RC buggy racing so I guess that would be the competition, but yeah this thing is outclassed as a platform already sadly
In the last few clips, the gear mesh sounds awfully tight. Also you can get better tires for it, so it does not slide as much. Final advice, the stock Traxxas servo sucks in keeping position imo, it's the first thing I advice people to change. I'm new to programming, so no idea how that part works (yet). But been racing and working on rc cars for the last 15 years. Love these video's, gets me excited
Get an INJORA servo for a great price point performer. That aside, I suggest the mesh was not getting tighter but rather we’re hearing some tooth wear. He is bashing, after all.
If Steven ripped out the guts of the servo and replaced it with something that could output the steering torque and actual position, it would be possible to feed the controller a "force feedback" that could be used to "feel" the track as well as allow the servo control loop to be tuned dynamically through software.
To be fair to you, people have other senses to rely on when driving. You just have the LIDAR. People can feel when a car is losing grip or how hard they're accelerating or braking or how close they are to something. You've done a really great job with limited sensors on your vehicle.
Exactly, this problem is nowhere near as trivial as it might look, literally years of research have gone into driving simulators only to get the grip formulas right, apparently there are pretty good models for dry and high grip situations but when traction is lost is not yet solved (assetto corsa devs have videos talking about this in depth). A pilot is getting A TON of information from G-Forces applied to their body, so that would be a great sensor to add to the car and formulas.
Dope project! I'll be on the lookout for follow ups. An unwarranted thought: You can probably squeeze out more performance from your controller with something like a model predictive controller with constraints for following the racing line, vehicle dynamics and local obstacle avoidance in your lidar map
You might not even need to run the MPC in real time: Just use the existing optimizer with starting conditions close to the racing line, and linearize the necessary controller output around the optimum. Calculate and store the parameters for every point on the path, then use them to get an optimal linear controller at every point along the track.
@@Christian-mf4jt With slight deviations accumulating, eventually it would get out of sync with the pre computed inputs. There are hybrid approaches, where one MPC generates an upper velocity bound offline and a second MPC tries to reach that limit online.
UWaterloo! Those new buildings have a lot more windows than the previous uh... 'aesthetic'. This is such a cool project. I remember trying to write the firmware for a fully autonomous drag racer one year in-between exams. It didn't go well! Your results are awesome!
For those floors I’d recommend slicks they have a higher contact patch than gridded tyres although slicks aren’t great for anything other than smooth surfaces. Also reducing the sensitivity or speed of steering to reduce instability
As someone who works in the field of autonomous vehicles, I have to say you did a banger of a job with the localization/mapping, and programming the annotations for the RC car to be able to race around the hallways like a pro. I can't wait to see it do more though, like detecting and avoiding obstacles.
Congrats!!! First thing, in my opinion, would have been to change the stock tires. There are some that are so much grippier then those, and would make the car feel much more precise.
Fantastic work! I'm a mechanical engineering student and the software side of this project really blows my mind. Subbed. 👍 This will probably get lost in the comments, but as an RC hobbyist, your motor gear mesh sounds really tight. The truck sounds like its screeching because of the binding in the transmission. There are plenty of videos on RUclips that show how to set the gear mesh between the pinion (motor gear) and spur (driven gear) properly. One trick is to loosen the motor screws, put a strip of paper between the gears, press them together, tighten the motor screws, then remove the paper.
This was so cool to watch! Took me back to my late night mechatronics days, in the lab till 6AM trying to figure out why my motors where going the wrong way and replacing H-Bridges haha!
Steven, I'm wondering if the LIDAR is doing anything during your speed runs, or if it's just used to help build the map? It seems to me you are doing autonomous driving via a sort of multi-stage process where the vehicle doesn't have any capacity to respond to obstacles, changing track conditions, changing internal mechanical (like gears eroding or tire wear.) Don't get me wrong, it *STILL* looks damned hard and we're enjoying the details you're sharing! I think it might be helpful to spell out for the viewers that this is (self)-mapped autonomous rather than real-time autonomous driving. Also, if that LIDAR is a snap-on module and isn't doing anything during racing, maybe pull it off and save your $1500. :) Please let me know if I've misinterpreted your process. It looks super fun either way. Glad you had some friends to help out with the 2am laps.
The car is actually using the LiDAR sensor to localize itself as its racing, which is why it's needed. The code for that is from this repository: github.com/f1tenth/particle_filter You are right that it doesn't have any obstacle avoidance or can adapt to changing track conditions, so this is definitely something that I want to work on in the near future :)) one step at a time is my focus
If the car was only following a preset line and not the lidar data to define it's position along the circuit, any slipping would offset it from the planned line and very quickly it would be on a collision course with an obstacle.
@@stevengongg this would be great to see. Specifically, how narrow of a gap that hasn't previously been mapped can you get the car to drive through at speed! i.e. you add a few barriers around the course that the car needs to drive between in order to complete the loop. You are totally right though - start simple first. Good luck
@@stevengongg at least implement some emergency braking if it's about to go into something or if it's getting out of control. later on you can turn that into collision avoidance based on previous knowledge of the area.
@@stevengongg REALLY COOL PROJECT! I'm not a racer, engineer, or coder, but I thought a cool idea, using what you presented, would be comparing optimal drive lines on existing tracks. you could take an existing track with Ariel photos, sometimes they have track data, and compare your code for the optimal drive line vs center line vs recommended lines from the different race teams. you could hop on that AI learning train and see if an AI can spit out anything that beats your program or existing lines for the track. if you really wanted, you could go down the rabbit hole and start scaling the rc car to comparable race cars that have driven on the track and you can build the track at scale (somewhere idk? space would be an issue)
I think you need an accelerometer to determine the G forces in corners, so you can adjust the steering and acceleration to prevent under/oversteer. Nice results by the way!
So hilarious watching robot vids lmao. I remember building a mapping robot when i studied, we only had ultrasonic sensors, which were garbage in comparison to lidar. But still nice to see your robot crash randomly :), just like ours did
Hello Steven. I just wanted to let you know that I watched your first video about Autonomous RC car, and it was absolutely amazing. Haven't been able to get the idea out of my head, so I just had to make something similar on my own. I am a robotics engeneer student at 4th semester and these are like spot on the things we work with in the projects. Really enjoying the content!
i noticed in your optimization parameters you didn't include the car mass/inertia, that can help reduce the unplanned overshoots (if your PID is optimal anyways)
yeah it feels like there is a bunch of low hanging fruit that could improve the overall performance quite a bit, I understand they might not want to sink insane amounts of time and money on it though
Man, you have a great channel, I like to witness interesting and non-trivial ideas being brought to life! Keep it up like this and pretty soon you'll gonna be big!
I love this video.. I assigned one the interns that we have in my company to modify an RC car (looks pretty similar like yours!) and make it autonomous.. that was two years ago... Project was abandoned due to other commitments... But the idea of modifying RC cars or anything RC and make it smart is the best approach..
Your videos are amazing! Nice to know how you manage to get over all the problems you found and all the solutions you made for this. Do you know the existance of the AWS DeepRacer League? It's more like image-processing more than LiDAR (also used for the obstacle racing) but you can find hours of fun out there
Wow, that is so cool. I built something similar 10 years ago after I took a class from Sabastian Thrun from the Google Self driving car project. I couldn't afford lidar back then, the cheapest I could find was 30,000 dollars so I'm excited to see that there is now a semi-affordable lidar unit. Way to go. Keep going! Here is the video of the one I built back then. Yes, I'm using an Iphone to get a compass heading and a GPS location signal. ruclips.net/video/sPwyYF9dGX0/видео.html
RUclips reccomended me this, and it's awesome! As an amateur sim driver, you can shave some time, if you close the gap in some of the corners, it's getting them a bit too wide, before improving the car itself, also will help knowing the point where the car will loose grip when turning, maybe with a Gyroscope, to apply the correct amount of throttle in any occasion to get the ideal amount of grip. I go with trial and error, if can handle X speed, at X G's, go faster until it drifts. Same with braking, how hard can you brake before locking wheels and start slipping, and if you can accelerate hard enough after, and gear ratios, depending of the track you'll want better acceleration or better top speed. You have to balance it right for the occasion. Then will come all the physical improvements, lowering the car, stiffer springs, grippy tires, and powertrain and weight reduction. As someone suggested, adding another type of sensors could also help, maybe ultrasonic sensors, which aren't as bulky and weighty as a LIDAR and use LIDAR for the first recognition... I don't know programming at all, but I like experimenting and I like electronics and rc cars as well, and most of the things I don't know how to implement them. All the best and good effort!!
This brings memories from mecatronics engineering school, well it was more spidery eldritch horrors trying to learn how to walk using ungodly transform matrixes but it was just as ridiculously fun at 3 AM in the engineering department. We could even swing by the agro-industry department and taste test their latest "original" creations.
I've had an idea like this for years, AI cars racing each other... You're making that dream come true and I'm living vicariously through you!! Keep up the good work sir!!
Mechanical engineering student here (also in Ontario lol), lengthening the wheelbase slightly might improve stability when steering. Less limiting in software, you'll be able to turn more precisely quickly rather than slowly increasing steering to where you want. Might help reduce the oscillating and increase speed.
This could be something like a new version of the battle bots show. You get model enthusiasts, racing people, programmers, fabricators, etc. Sounds like a ton of fun to watch or participate in.
As a teen who's insanely enthusiastic about the RC hobby and an automobile engineering student, this project is something I would absolutely love to do! seems like it requires a lot of coding knowledge and skills tho i'm far from reaching that lvl as of now. Looking forward to the progression of this!
Love that you're doing this with a Traxxas truck. Part of engineering is using the right tools for the job and let me tell you the stock tires aren't right for those floors. You need to look for some foam tires for better grip.
Constraining yourself to tyre grip is not the only approach. By using gyro data and comparing it to steering angle (hint: tap into the servo's potentiometer signal so you know the true current steering angle rather than the requested angle), you can determine the oversteer or understeer angle. Oversteer means the car should countersteer and slow down. Understeer means the car should just slow down. Using the oversteer ar understeer angle to augment the determined direction of travel from the camera/lidar data would allow you to eliminate the oscillation that wheel slip introduces to the control system. You could alternatively use an optical flow sensor from a computer mouse under each axle to for more direct and lower latency data which is less susceptible to noise than an IMU.
The trick to racing is downrange vision. It can only plan lines as far as it can see. It can see better lines if it can see further. It can run them increasingly faster the further it can see and the faster it can interpolate what it sees. Maybe you don't need to see the entire environment so clearly. What if it just focused on a downrange bubble that can allow it to continuously chart where it should go. Just some ideas. This is amazing and I'm blown away and only suggesting this as someone with race experience. For sure a good RC driver will still dust this thing in a race. It wouldn't be close. You have to make it think like they do. They're not looking at the glass trying to see if its glass or not. They're already looking 100ft downrange with tunnel vision. I know that a robot depends on spatial awareness but you should think about ways to focus it. Or maybe make it diminish with distance. 'Push forward' is the term in racing. The next step is compelling it to 'push forward', not see everything and react in the moment. I love this and want to see you take it all the way!
Very interesting! I'm competing in the 2023 season for WRO Future Engineers, and this is essentially what the goal is! Thanks for sharing a funny overview of the concepts.
great work! with the right tires, less weight and a lower CG chassis you could reduce some of the understeer the car has and increase the speed much further. I'd suggest a touring car chassis like the Tamiya TT02 or similar
As someone who did some linefollower racing I really like this project, edit and also I am a little jealous :) For the racing line concept I fully agree that it is complicated because the vehicle model and especially the tire model is very complex and time variable. For you the main variability is off-course the battery level, dirt on tires and surface change. The geometrical line is 2nd order and therefore you have sudden lateral acceleration change on inflection points. This is in reality not possible because the car has load transfer, finite servo time constants, tires with some relaxation lengths etc. Also there is a braking and acceleration part of corner so a clothoid approximation is more appropriate. Keep up the good work.
In most racing games the color of the line for full acceleration is usually green and for the part that requires braking is red yellow usually means to maintain speed or let off throttle
Cheers! I really like what you are doing and I wish you all the best. While I was watching the video I noticed something which reminded me of some car physics. Cars tend to oversteer/understeer if they are RWD/FWD. Also a car's behavior changes when accelerating/breaking since you have a mass shift. This can affect your curve and time.
Really impressed with your work here, Steven! I work on onboard perception at Waymo and previously worked at May Mobility. Some notes from my experiences and studies: * Odometry on this platform seems bad. From what I can tell, the odometry here uses only the motor speed (measured with an encoder?) and some kind of servo position, which is problematic for a few reasons. I wouldn't be too frustrated that it doesn't work well :) * Other commenters have mentioned using model-predictive control (MPC) instead of pure pursuit (PPC). Control theory isn't quite my wheelhouse, but without an inertial measurement unit (IMU) onboard I'm not convinced you'll be able to accurately estimate vehicle dynamics. I suggest trying to measure the "friction ellipse" of your vehicle, then adding acceleration, jerk, and angular velocity limits so that your control commands never exit this ellipse at any tire (minus some safety margin). This will reduce pose divergence between what you plan on doing and what actually ends up happening, and should improve the quality of odometry. Much of the benefit you'd get from MPC, which is to say, being able to optimize the robot's trajectory over a blend of different objective functions (jerk, progress along trajectory, energy spent, and so on), seems to be baked into how you generate your trajectories (racing lines) offline. The incremental benefit from MPC, then, would be doing this online to recover from a mistake or adjust for wheel slip. * Velocity-based lookahead for pure pursuit has merit. I worked on a curvature-based lookahead algorithm as well, which was annoying for slightly different reasons. Take care not to make your implementation too complex -- you could end up losing stability guarantees that you would otherwise have. The particularly ambitious might do analytical control stability analysis to ensure that your controller isn't guaranteed to oscillate at higher velocity. My team once spent quite a while chasing down an oscillation problem that was eventually determined to be because of time delay and a deadzone in the control actuators. * The problem with glass is interesting. Maybe you could design some sort of classification algorithm based on the range and intensity of neighboring lidar returns? * Point-to-point ICP is generally not robust enough to be used for scanmatching in production systems. In 3D we often optimize over point-to-normal distance instead, so the point-to-line algorithm mentioned in the UPenn slides seems like it could be appropriate. I remember reading a paper about point-to-Gaussian matching, where Gaussians were created from little neighborhoods of prior points; could be useful here. * Your lidar pipeline is very, very sensntive to rotational error in the intrinsic transform from body frame to lidar frame, not only for yaw but also for roll and pitch. If you have a way to fine tune this manually, I suggest doing so! You could also try and formulate estimating the intrinsic matrix as an offline optimization problem, but that seems like it would be a lot of work compared to visually aligning pointclouds.
Nice work, keep pushing the limits. Both the car, and yours. Was surprise that your racing on the track counter-clockwise (12:32). Can also test racing clockwise, it would be like having a second track. Would be a good test for your algorithms, once you think you have the car tuned for one track. (might also good for evening out the tread and steering wear ;)
Very cool! I often thought about this approach, but from a different angle. Instead of fully autonomous, I think a better approach would be to teach the car a track. The receiver and transmitter record a track in slow mode (driver inputs). Then parameters are set to for full speed and corner speed. You'd need a gyro to keep it in a straight line, and guide the car around, each lap finessing the course speed etc. So in racing mode, (hands on still) it would auto correct, hold direction and find the best line for each section.
@Steven Gong, great job! But you might want to look at the weight distribution of your vehicle and the gear ratio of your brushless motor in order to achieve higherspeed and consistent turns. Cheers!
Wow, excellent improvements over the last video. Perhaps start in a slower part of the track to prevent the car instantly losing traction and hitting the elevator. Great job with the project. I look forward to the next video.
Steven how are you!… I’m only 40 seconds into the intro and had to LIKE 👍 your video 😊. I love this level of ingenuity and I also enjoy fabricating and building custom rc parts and components that have not yet been done before or just not seen by many. This video hits home base! Okay now back to the video😎 stay awesome 🫵🧠.
It can follow a track, but it doesn't feel like a racing driver yet. It's constantly braking too late because it has less grip than in the simulation, and it's causing it to not hit apexes. You should enable it to go into turns a little stiffer, with more computer confidence, and maybe use the motor to brake when it's detecting that it won't make the apex as intended.
you've probably gotten a few of these comments already, but, if you change the tire to a race tire (less knobby tire= more traction= car goes faster) I recommend Traxxas tires, they have a great compound
Would love to see this using an 1/10 on-road chassis. The best tires for the vinyl floor are ‘rally tread’ as no matter how much you sweep the floor, there is always dust that doesn’t allow full slicks to bite.
This was crazy fun to watch, however, I would suggest applying -3 to -9dB to the onboard car audio if possible. Not sure where the motor sounds are coming from, but boy do they carry energy and drown out your dialog. Compression could help too, but this could probably done during import depending on your workflow 👍
Great work! I would highly suggest checking the gears, they sound very bad near the end of the video, like they are too tight together (this severely lowers performance), and also lack lubrication. Apart from that make the suspension harder and lower it if the chassis allows it. Stickier wheels and finally a different steering servo.
Awesome project! I really enjoyed watching this and it made want to go back to school... lol For the full speed test, try to ramp up the speed instead of starting at full speed. This will avoid tire slippage and traction loss and hence, steering issues.... Once the vehicle is moving, it accelerates to max speed...
Don't mash the gear teeth together as tight as they can go. You need a minimal amount of backlash. After your fix it sounds like the mesh is too tight. On the chassis you have, (TRAXXIS or clone) it's a very fine adjustment to get the best mesh. Sometimes slipping some paper between the gears helps. You're drawing to much current through the ESC if the meshing is too tight, because the motor finds it hard to spin. This can damage your components.
![Coheed and Cambria - Searching for Tomorrow [Official Video]](http://i.ytimg.com/vi/hDsYGLbmadg/mqdefault.jpg)
Thank you all for the support this past month, I really appreciate it! I got loads of exciting projects planned, so make sure to subscribe and turn on the notification bell so you don't miss out on my new videos :))
this is so cool. next up: add a ramp, see how far you can safely jump it inside and land it smoothly. math it out my man, you got it!
So glad you introduced racing lines!!
i am so glad that you are making evolution in racing robots !!! Love your memes in your video keep them in😂😂pls !!!
I have seen, how AI in trackmania, adjust it own track line to make fast lap, maybe try to train, your own ai in trackmania, and then move it to rc ? :)
There are some "driving style" issues with your programming. I'm a driving instructor, so if you want to make it even faster let me know.
If you really want to hate fine tuning, you could even start introducing slip angle, as on lower grip surfaces like polished concrete it can be faster to be putting power down earlier into corners and letting the tail of the car kick out a little bit. It would require gyros (which could also help with the oscillations at high speeds) but could also increase corner entry and exit speeds a fair bit
i was thinking the same thing with powerslides as in some cases it increases speeds quite a bit
this is not just fine tuning this is transforming a mid-tier problem into an insane problem
What I love most about these types of videos is that its pretty impressive already, but there's always a way to improve it, and more often than not, its a heavily explored topic
@@lagule just use ML lol.
jokes aside, that seems like an "easy" problem to solve with ml, simulate the car, then use the pre-optimized program control of the car with some securities and let correct for the delta between simulation and reality. (by easy I mean it's only a couple months of work)
OH YEA SIDEWAYS
i’d love to see this self driving car race against some hobbyists on an actual track
Exactly what I was thinking!!
That would be awesome lol
This is a thing or at least was
The class of car might be a problem. This type of RC truck races on carpet or dirt which includes jumps. Onroad (racing without jumps) cars are very light and low. Could easily find a pro driver or hobbyist to race against this but I imagine you'd need a handicap as the equipment on this thing will make it very top heavy by comparison.
@@TheFatAssCat I've seen videos of dirt RC buggy racing so I guess that would be the competition, but yeah this thing is outclassed as a platform already sadly
In the last few clips, the gear mesh sounds awfully tight. Also you can get better tires for it, so it does not slide as much. Final advice, the stock Traxxas servo sucks in keeping position imo, it's the first thing I advice people to change.
I'm new to programming, so no idea how that part works (yet). But been racing and working on rc cars for the last 15 years. Love these video's, gets me excited
I agree with this, i was hoping it would already be in the comments!
Yea that gear mesh was sounding spicy
Yea you definitely don’t need that much tread on such a smooth surface, braking and acceleration would be much better
Get an INJORA servo for a great price point performer. That aside, I suggest the mesh was not getting tighter but rather we’re hearing some tooth wear. He is bashing, after all.
If Steven ripped out the guts of the servo and replaced it with something that could output the steering torque and actual position, it would be possible to feed the controller a "force feedback" that could be used to "feel" the track as well as allow the servo control loop to be tuned dynamically through software.
To be fair to you, people have other senses to rely on when driving. You just have the LIDAR. People can feel when a car is losing grip or how hard they're accelerating or braking or how close they are to something. You've done a really great job with limited sensors on your vehicle.
Exactly, this problem is nowhere near as trivial as it might look, literally years of research have gone into driving simulators only to get the grip formulas right, apparently there are pretty good models for dry and high grip situations but when traction is lost is not yet solved (assetto corsa devs have videos talking about this in depth).
A pilot is getting A TON of information from G-Forces applied to their body, so that would be a great sensor to add to the car and formulas.
Absolutely epic! This motivates me to start a similar project
Go for it!
The $1600 LIDAR sensor motivates me not starting it 😄
@@Talaxianer could always go the musk method of 2 cameras for depth perception and pray your model doesn't floor it into a overturned box truck
@@StuckOnAFireHydrant everyone knows self driving needs LiDAR except Musk 🤣🤣🤣
@@Talaxianer use a lidar from robot vacuums. They are 150.
Dope project! I'll be on the lookout for follow ups.
An unwarranted thought: You can probably squeeze out more performance from your controller with something like a model predictive controller with constraints for following the racing line, vehicle dynamics and local obstacle avoidance in your lidar map
You might not even need to run the MPC in real time: Just use the existing optimizer with starting conditions close to the racing line, and linearize the necessary controller output around the optimum. Calculate and store the parameters for every point on the path, then use them to get an optimal linear controller at every point along the track.
@@Christian-mf4jt With slight deviations accumulating, eventually it would get out of sync with the pre computed inputs. There are hybrid approaches, where one MPC generates an upper velocity bound offline and a second MPC tries to reach that limit online.
@@VulpeculaJoy The missile knows where it is, thusly… ruclips.net/video/c94bALAgENM/видео.html …so yeah, a map if one can generate one.
As an engineer. I absolutely love this and I am inspired to make my own.
If the government had been able to understand how fascinating this is, they would no doubt have taxed it. Thanks a lot for the ride!
UWaterloo! Those new buildings have a lot more windows than the previous uh... 'aesthetic'. This is such a cool project. I remember trying to write the firmware for a fully autonomous drag racer one year in-between exams. It didn't go well! Your results are awesome!
2:16 I liked the Bionicle reference, when you showed Mata Nui )
I'm making a racing game, your notes are a LIFESAVER my man. Love the videos, I'll be following and not just to scoop more insight.
For those floors I’d recommend slicks they have a higher contact patch than gridded tyres although slicks aren’t great for anything other than smooth surfaces. Also reducing the sensitivity or speed of steering to reduce instability
Also, wiping tires down with simple green adds some tack to the rubber.
This is really awesome!
OMG I LOVE your videos man :D
This is probably one of the coolest projects I've seen showcased on youtube. Editing for the video is fantastic too! Keep it coming!
Never thought I'd see a Slash do that! pretty cool
Maybe it's worth making a "safety square" around the car so that it avoids obstacles in priority over the trajectory of the track
As someone who works in the field of autonomous vehicles, I have to say you did a banger of a job with the localization/mapping, and programming the annotations for the RC car to be able to race around the hallways like a pro. I can't wait to see it do more though, like detecting and avoiding obstacles.
Also! A great addition would be slick tires for more grip on this smooth floor
You should totally make a drone that follows it around like a third person view racing game
RF tag babeeeee!
You don't need a drone, just a 360 camera.
Congrats!!! First thing, in my opinion, would have been to change the stock tires. There are some that are so much grippier then those, and would make the car feel much more precise.
Fantastic work! I'm a mechanical engineering student and the software side of this project really blows my mind. Subbed. 👍
This will probably get lost in the comments, but as an RC hobbyist, your motor gear mesh sounds really tight. The truck sounds like its screeching because of the binding in the transmission. There are plenty of videos on RUclips that show how to set the gear mesh between the pinion (motor gear) and spur (driven gear) properly. One trick is to loosen the motor screws, put a strip of paper between the gears, press them together, tighten the motor screws, then remove the paper.
Man this is such a great content! Im currently doing the same project with a go-kart using Stereo Visual SLAM. Good luck bro!
omggg lucky! i wanna do the same with a gokart, but those things are EXPENSIVE
This was so cool to watch! Took me back to my late night mechatronics days, in the lab till 6AM trying to figure out why my motors where going the wrong way and replacing H-Bridges haha!
Steven, I'm wondering if the LIDAR is doing anything during your speed runs, or if it's just used to help build the map? It seems to me you are doing autonomous driving via a sort of multi-stage process where the vehicle doesn't have any capacity to respond to obstacles, changing track conditions, changing internal mechanical (like gears eroding or tire wear.) Don't get me wrong, it *STILL* looks damned hard and we're enjoying the details you're sharing! I think it might be helpful to spell out for the viewers that this is (self)-mapped autonomous rather than real-time autonomous driving.
Also, if that LIDAR is a snap-on module and isn't doing anything during racing, maybe pull it off and save your $1500. :)
Please let me know if I've misinterpreted your process. It looks super fun either way. Glad you had some friends to help out with the 2am laps.
The car is actually using the LiDAR sensor to localize itself as its racing, which is why it's needed. The code for that is from this repository: github.com/f1tenth/particle_filter
You are right that it doesn't have any obstacle avoidance or can adapt to changing track conditions, so this is definitely something that I want to work on in the near future :)) one step at a time is my focus
If the car was only following a preset line and not the lidar data to define it's position along the circuit, any slipping would offset it from the planned line and very quickly it would be on a collision course with an obstacle.
@@stevengongg this would be great to see. Specifically, how narrow of a gap that hasn't previously been mapped can you get the car to drive through at speed! i.e. you add a few barriers around the course that the car needs to drive between in order to complete the loop. You are totally right though - start simple first. Good luck
@@stevengongg at least implement some emergency braking if it's about to go into something or if it's getting out of control. later on you can turn that into collision avoidance based on previous knowledge of the area.
@@stevengongg REALLY COOL PROJECT! I'm not a racer, engineer, or coder, but I thought a cool idea, using what you presented, would be comparing optimal drive lines on existing tracks.
you could take an existing track with Ariel photos, sometimes they have track data, and compare your code for the optimal drive line vs center line vs recommended lines from the different race teams.
you could hop on that AI learning train and see if an AI can spit out anything that beats your program or existing lines for the track.
if you really wanted, you could go down the rabbit hole and start scaling the rc car to comparable race cars that have driven on the track and you can build the track at scale (somewhere idk? space would be an issue)
Michael Reeves inspiration and Initial D references. 10/10, subscribed
I think you need an accelerometer to determine the G forces in corners, so you can adjust the steering and acceleration to prevent under/oversteer. Nice results by the way!
The sound of the car accelerating and decelerating on the final track is so smooth and satisfying.
Gotta go fast
I'm so stoked to see this work out, but boyo, please set a ceiling on your audio mixing. You just Mario Kart N64'd my eardrums to death.
So hilarious watching robot vids lmao. I remember building a mapping robot when i studied, we only had ultrasonic sensors, which were garbage in comparison to lidar. But still nice to see your robot crash randomly :), just like ours did
Ultrasonic sensors might have mapped the glass sections more effectively
Hello Steven. I just wanted to let you know that I watched your first video about Autonomous RC car, and it was absolutely amazing. Haven't been able to get the idea out of my head, so I just had to make something similar on my own. I am a robotics engeneer student at 4th semester and these are like spot on the things we work with in the projects. Really enjoying the content!
i noticed in your optimization parameters you didn't include the car mass/inertia, that can help reduce the unplanned overshoots (if your PID is optimal anyways)
yeah it feels like there is a bunch of low hanging fruit that could improve the overall performance quite a bit, I understand they might not want to sink insane amounts of time and money on it though
8:38 Curling with autonomous rocks.
Cool project, fun to watch!
Man, you have a great channel, I like to witness interesting and non-trivial ideas being brought to life! Keep it up like this and pretty soon you'll gonna be big!
I love this video.. I assigned one the interns that we have in my company to modify an RC car (looks pretty similar like yours!) and make it autonomous.. that was two years ago... Project was abandoned due to other commitments... But the idea of modifying RC cars or anything RC and make it smart is the best approach..
Your videos are amazing! Nice to know how you manage to get over all the problems you found and all the solutions you made for this. Do you know the existance of the AWS DeepRacer League? It's more like image-processing more than LiDAR (also used for the obstacle racing) but you can find hours of fun out there
i loved this project great job!, and you should prank people saying its running away from maybe its gonna be fun if u do.
Bro u basically copied this entire thing from the MIT 6.141 class on robotics (now titled 6.4200). Please mention that.
true
This might be the best RUclips content I've ever seen. Thank you, so much for making it
Wow, that is so cool. I built something similar 10 years ago after I took a class from Sabastian Thrun from the Google Self driving car project. I couldn't afford lidar back then, the cheapest I could find was 30,000 dollars so I'm excited to see that there is now a semi-affordable lidar unit. Way to go. Keep going! Here is the video of the one I built back then. Yes, I'm using an Iphone to get a compass heading and a GPS location signal. ruclips.net/video/sPwyYF9dGX0/видео.html
RUclips reccomended me this, and it's awesome! As an amateur sim driver, you can shave some time, if you close the gap in some of the corners, it's getting them a bit too wide, before improving the car itself, also will help knowing the point where the car will loose grip when turning, maybe with a Gyroscope, to apply the correct amount of throttle in any occasion to get the ideal amount of grip. I go with trial and error, if can handle X speed, at X G's, go faster until it drifts. Same with braking, how hard can you brake before locking wheels and start slipping, and if you can accelerate hard enough after, and gear ratios, depending of the track you'll want better acceleration or better top speed. You have to balance it right for the occasion. Then will come all the physical improvements, lowering the car, stiffer springs, grippy tires, and powertrain and weight reduction. As someone suggested, adding another type of sensors could also help, maybe ultrasonic sensors, which aren't as bulky and weighty as a LIDAR and use LIDAR for the first recognition... I don't know programming at all, but I like experimenting and I like electronics and rc cars as well, and most of the things I don't know how to implement them. All the best and good effort!!
This is definitely better than tesla
Fabulous job on both the engineering and the video! Thanks for posting it!
This brings memories from mecatronics engineering school, well it was more spidery eldritch horrors trying to learn how to walk using ungodly transform matrixes but it was just as ridiculously fun at 3 AM in the engineering department. We could even swing by the agro-industry department and taste test their latest "original" creations.
I've had an idea like this for years, AI cars racing each other... You're making that dream come true and I'm living vicariously through you!! Keep up the good work sir!!
Amazing project dude! I'm in engineering and am still blown away you made this entire thing! Love your editing style too, very entertaining!
bro i thought you would at least have 500k subs
this production quality is through the roof
Mechanical engineering student here (also in Ontario lol), lengthening the wheelbase slightly might improve stability when steering. Less limiting in software, you'll be able to turn more precisely quickly rather than slowly increasing steering to where you want. Might help reduce the oscillating and increase speed.
I want to see this full size. Imagine rc cars the size of F1. No risk of anyone getting killed. Just battle of the AIs. Awesome video man.
This could be something like a new version of the battle bots show. You get model enthusiasts, racing people, programmers, fabricators, etc. Sounds like a ton of fun to watch or participate in.
As a teen who's insanely enthusiastic about the RC hobby and an automobile engineering student, this project is something I would absolutely love to do! seems like it requires a lot of coding knowledge and skills tho i'm far from reaching that lvl as of now. Looking forward to the progression of this!
Seeing the progression of the build and the progression of his talent in editing makes this series a must watch
Don't beat yourself that much... We all love what you do!!
Love that you're doing this with a Traxxas truck. Part of engineering is using the right tools for the job and let me tell you the stock tires aren't right for those floors. You need to look for some foam tires for better grip.
Constraining yourself to tyre grip is not the only approach. By using gyro data and comparing it to steering angle (hint: tap into the servo's potentiometer signal so you know the true current steering angle rather than the requested angle), you can determine the oversteer or understeer angle. Oversteer means the car should countersteer and slow down. Understeer means the car should just slow down. Using the oversteer ar understeer angle to augment the determined direction of travel from the camera/lidar data would allow you to eliminate the oscillation that wheel slip introduces to the control system. You could alternatively use an optical flow sensor from a computer mouse under each axle to for more direct and lower latency data which is less susceptible to noise than an IMU.
The editing in this video is top tier
the editing really completes the video
Even the victory donuts is on point. Bravo
Totally my kind of project!! Awesome stuff man
As a fan of both F1 and autonomous vehicles, this was a fun watch, nice work Steven!
The Reeves-style transition killed me - keep it up!
The trick to racing is downrange vision. It can only plan lines as far as it can see. It can see better lines if it can see further. It can run them increasingly faster the further it can see and the faster it can interpolate what it sees. Maybe you don't need to see the entire environment so clearly. What if it just focused on a downrange bubble that can allow it to continuously chart where it should go. Just some ideas. This is amazing and I'm blown away and only suggesting this as someone with race experience. For sure a good RC driver will still dust this thing in a race. It wouldn't be close. You have to make it think like they do. They're not looking at the glass trying to see if its glass or not. They're already looking 100ft downrange with tunnel vision. I know that a robot depends on spatial awareness but you should think about ways to focus it. Or maybe make it diminish with distance. 'Push forward' is the term in racing. The next step is compelling it to 'push forward', not see everything and react in the moment. I love this and want to see you take it all the way!
One thing I’d try is better tires with a stickier compound. That would help a lot with the spinning out on 100% launch.
Softer compound slicks would be ideal.
Very interesting! I'm competing in the 2023 season for WRO Future Engineers, and this is essentially what the goal is! Thanks for sharing a funny overview of the concepts.
great work! with the right tires, less weight and a lower CG chassis you could reduce some of the understeer the car has and increase the speed much further. I'd suggest a touring car chassis like the Tamiya TT02 or similar
As someone who did some linefollower racing I really like this project, edit and also I am a little jealous :)
For the racing line concept I fully agree that it is complicated because the vehicle model and especially the tire model is very complex and time variable. For you the main variability is off-course the battery level, dirt on tires and surface change. The geometrical line is 2nd order and therefore you have sudden lateral acceleration change on inflection points. This is in reality not possible because the car has load transfer, finite servo time constants, tires with some relaxation lengths etc. Also there is a braking and acceleration part of corner so a clothoid approximation is more appropriate.
Keep up the good work.
In most racing games the color of the line for full acceleration is usually green and for the part that requires braking is red yellow usually means to maintain speed or let off throttle
Great Video and really cool data representation! Keep it up! Greetings from Switzerland!
Cheers! I really like what you are doing and I wish you all the best.
While I was watching the video I noticed something which reminded me of some car physics.
Cars tend to oversteer/understeer if they are RWD/FWD. Also a car's behavior changes when accelerating/breaking since you have a mass shift.
This can affect your curve and time.
Absolutely insane! You got my subscription!
use tire sauce for grip. You can let it soak and eventually the rubber will absorb it after a number of applications. Cool vid!
Really impressed with your work here, Steven! I work on onboard perception at Waymo and previously worked at May Mobility. Some notes from my experiences and studies:
* Odometry on this platform seems bad. From what I can tell, the odometry here uses only the motor speed (measured with an encoder?) and some kind of servo position, which is problematic for a few reasons. I wouldn't be too frustrated that it doesn't work well :)
* Other commenters have mentioned using model-predictive control (MPC) instead of pure pursuit (PPC). Control theory isn't quite my wheelhouse, but without an inertial measurement unit (IMU) onboard I'm not convinced you'll be able to accurately estimate vehicle dynamics. I suggest trying to measure the "friction ellipse" of your vehicle, then adding acceleration, jerk, and angular velocity limits so that your control commands never exit this ellipse at any tire (minus some safety margin). This will reduce pose divergence between what you plan on doing and what actually ends up happening, and should improve the quality of odometry.
Much of the benefit you'd get from MPC, which is to say, being able to optimize the robot's trajectory over a blend of different objective functions (jerk, progress along trajectory, energy spent, and so on), seems to be baked into how you generate your trajectories (racing lines) offline. The incremental benefit from MPC, then, would be doing this online to recover from a mistake or adjust for wheel slip.
* Velocity-based lookahead for pure pursuit has merit. I worked on a curvature-based lookahead algorithm as well, which was annoying for slightly different reasons. Take care not to make your implementation too complex -- you could end up losing stability guarantees that you would otherwise have. The particularly ambitious might do analytical control stability analysis to ensure that your controller isn't guaranteed to oscillate at higher velocity. My team once spent quite a while chasing down an oscillation problem that was eventually determined to be because of time delay and a deadzone in the control actuators.
* The problem with glass is interesting. Maybe you could design some sort of classification algorithm based on the range and intensity of neighboring lidar returns?
* Point-to-point ICP is generally not robust enough to be used for scanmatching in production systems. In 3D we often optimize over point-to-normal distance instead, so the point-to-line algorithm mentioned in the UPenn slides seems like it could be appropriate. I remember reading a paper about point-to-Gaussian matching, where Gaussians were created from little neighborhoods of prior points; could be useful here.
* Your lidar pipeline is very, very sensntive to rotational error in the intrinsic transform from body frame to lidar frame, not only for yaw but also for roll and pitch. If you have a way to fine tune this manually, I suggest doing so! You could also try and formulate estimating the intrinsic matrix as an offline optimization problem, but that seems like it would be a lot of work compared to visually aligning pointclouds.
thanks so much for all this feedback, it’s super helpful!! I sent you a connection request on LinkedIn, would love to chat with you
Nice work, keep pushing the limits. Both the car, and yours.
Was surprise that your racing on the track counter-clockwise (12:32). Can also test racing clockwise, it would be like having a second track. Would be a good test for your algorithms, once you think you have the car tuned for one track. (might also good for evening out the tread and steering wear ;)
Very cool! I often thought about this approach, but from a different angle. Instead of fully autonomous, I think a better approach would be to teach the car a track. The receiver and transmitter record a track in slow mode (driver inputs). Then parameters are set to for full speed and corner speed. You'd need a gyro to keep it in a straight line, and guide the car around, each lap finessing the course speed etc. So in racing mode, (hands on still) it would auto correct, hold direction and find the best line for each section.
Gyros are great at measuring inertial change. However, it would be pretty difficult for them to be used for localization alone.
@@gstrnerd2 true, but if it's set to counter steering anomalies, it'll stay on the programmed track
@Steven Gong, great job! But you might want to look at the weight distribution of your vehicle and the gear ratio of your brushless motor in order to achieve higherspeed and consistent turns. Cheers!
Really cool project. Very nice that you shared the code. Great job.
Dude! Do more of this! I love it!!!
Thank you for posting this. I work for Amazon AWS and I am comparing what you are doing to the AWS DeepRacer educational device.
Very nice video! With code too! Aspiring engineers would really love your content
impressive work - well done guys! I can't believe how expensive that LIDAR sensor is! ouch!
fantastic work! it reminds me fastest maze solver!
Wow, excellent improvements over the last video. Perhaps start in a slower part of the track to prevent the car instantly losing traction and hitting the elevator. Great job with the project. I look forward to the next video.
By the way, your website is full of great resources. Thank you for sharing!
Steven how are you!… I’m only 40 seconds into the intro and had to LIKE 👍 your video 😊. I love this level of ingenuity and I also enjoy fabricating and building custom rc parts and components that have not yet been done before or just not seen by many. This video hits home base! Okay now back to the video😎 stay awesome 🫵🧠.
this channel is like michael reeves but with less insanity, loving it, keep up the great work
It can follow a track, but it doesn't feel like a racing driver yet. It's constantly braking too late because it has less grip than in the simulation, and it's causing it to not hit apexes. You should enable it to go into turns a little stiffer, with more computer confidence, and maybe use the motor to brake when it's detecting that it won't make the apex as intended.
you've probably gotten a few of these comments already, but, if you change the tire to a race tire (less knobby tire= more traction= car goes faster) I recommend Traxxas tires, they have a great compound
I got into UW mechatronics a bit ago, but im scared of how cutthroat the program is. Seeing you have fun with this has made me want to go!
oh my god this is so good! this was my little dream when i was at school! i can't wait for the next video!
I think you could squeeze out more if you added a body for downforce and better tires for that surface. Cool project!
Would love to see this using an 1/10 on-road chassis. The best tires for the vinyl floor are ‘rally tread’ as no matter how much you sweep the floor, there is always dust that doesn’t allow full slicks to bite.
This was crazy fun to watch, however, I would suggest applying -3 to -9dB to the onboard car audio if possible. Not sure where the motor sounds are coming from, but boy do they carry energy and drown out your dialog.
Compression could help too, but this could probably done during import depending on your workflow 👍
Love the eurobeat in the background
Great work! I would highly suggest checking the gears, they sound very bad near the end of the video, like they are too tight together (this severely lowers performance), and also lack lubrication. Apart from that make the suspension harder and lower it if the chassis allows it. Stickier wheels and finally a different steering servo.
I'm glad I found your video. Loved it! I hope your professors do as well.
Awesome project! I really enjoyed watching this and it made want to go back to school... lol For the full speed test, try to ramp up the speed instead of starting at full speed. This will avoid tire slippage and traction loss and hence, steering issues.... Once the vehicle is moving, it accelerates to max speed...
This type of "run the same course optimally" problem would make for a great use case for iterative learning control.
Don't mash the gear teeth together as tight as they can go. You need a minimal amount of backlash. After your fix it sounds like the mesh is too tight. On the chassis you have, (TRAXXIS or clone) it's a very fine adjustment to get the best mesh. Sometimes slipping some paper between the gears helps.
You're drawing to much current through the ESC if the meshing is too tight, because the motor finds it hard to spin. This can damage your components.
I appreciate the eurobeat and the fpv camera