@@BMSWEB that's a fascinating way to put that. We puny humans rely on that camera facing forward. The all the AI knows is 4 motors equally spaced apart, so it just points and shoots 🤣
@@BMSWEB I wonder if anyone has tried to fly with 360 fpv or placing the fpv camera in the center and pointing it 90 degrees. Maybe it would make this possible in humans.
WOW! I'm 60 years old and only just starting to get into this whole quad scene. I wish i was 20 again! I've had an Atom camera drone for a few months and love flying it. I just bought a Boxer and 3 Sims to practice on before i either buy or build my first freestyle quad. My reflexes will never be near you guys, but i can still have fun with it. If i can get to be 20% as good as you are Thomas, i will be more than happy. That made me smile bigly after that AI run :) We can't even imagine where this tech will take us in 20-30 years time. You guys are living into a fascinating future. Make it count.
I know nothing about AI or Drone-Racing but RUclips thought I should watch this. And Wow! That was f… fast. Keep it up, people like you will make our world a better place.
I'm glad you enjoyed the video! Ha ha drone racing in general is a pretty crazy sport, hard to believe it didn't even exist a few years ago! I'll do my best! -Thomas
Oh man I've been seeing some of the upgrade work they've done since this video and its even more insane. Super excited for a rematch next year, but I suspect in a controlled environment there won't be much a competition for their creation! What a time to be alive hey? :D -Thomas
@@BMSWEB Yes, the bad thing about it is, the military will take advantage of it first and use it... We made many innovative devs, but it was always the military who had in the end the right to say if its for public or not. And drones you can see it in the Ukraine war... are the generation in which we are living... right now. AI at it self is really cool, like a new dimension. I often compare it to LEGO or Fisher-Technik, when you was able to start to build your ideas :-)
It's pretty cool to see it in real life. I'd love to go back through the footage and see if there are any movements that are easy enough to pickup using this method. I don't think there's much but surely there's an advantage for human spec racing there somewhere! -Thomas
@@BMSWEB the camera see 360°, you could get something similar with multiple fpv cameras + VR headset.. The issue is the latency, keeping them < 100ms on digital is a pain (aka: VERY expensive)
Incredible watching the FPV view of the AI drone. It completely has no regard for typical forward motion like us humans need for most of our flights. It'l just fly at whatever angle it wants to get the job done.
I think human brains are capable of it, it's just a matter of training and forcing yourself to "fly blind". Perhaps not to the extent shown here though, as the mo-cap data allows the drone to "see" the entire course at all times.
We've probably got at least one more video on the AI vs Human race that we can do for sure. It's just been so hard finding the time to edit these videos. We've still got the Aussie Nats, WA State Titles and also Thomas' Japan drone Racing League race which will be so much fun to watch. It was a pretty eventful race that one lol
I don't think any AI "Pilot" will be capable to beat Tomas. He's way beyond! Can't wait to watch the race! Nice project by the way and thank you for sharing this. Your BMS Fan.
I am not sure - computers have every single advantage over us and once the AI get's enough flights in it will be able to fly at the edge of the drones aerodynamic capabilities without errors and it will be consistent and will or can keep it at that level of performance for as long as the drone has batteries... However I don't think it matters, we as people should just enjoy what we are, what we can do, etc, have fun pit racing, talking with friends, actually racing and working to improve our skills, etc. The technology is mind boggling, AI is creeping into every single aspect of our lives and one day not too long from now and if we don't blow up the planet before then we will see AI everywhere. Now about job loss to AI, etc... a serious problem, a horrifyingly serious problem really. But I have an idea - it is one that I have had for a year or 2 now - let's ask AI how we can solve that problem. Also, here, a little off frack but related to this video is the greatest idea ever in all humanity - I would say that we should replace 99% of all human politicians with AI. Think about it, an unbiassed, un-corruptible (don't say hackers... yea that idea is concerning, but let's stick to the core for now), what I mean is it would have no friends to give jobs to, no nepotism, no backroom deals, etc... it can work tirelessly for ever, non stop... it will be smarter, make better decisions, make decisions that are the best for us, for the whole world. Solve the energy crisis, solve global warming, solve equity in society, solve everything, cure diseases, and who knows what else... and it will beat us at any game that we challenge it to. I for one welcome our new overlords. Cheers?
Idk if you saw that end footage but damn, no FPV flyer is gonna beat that. Give the AI a few more iterations and a longer simulation time and it's gonna be unbeatable.
@@rizen3467 No I get it, but the computer is capable of such precision, perfection, ability to repeat exactly what is needed lap after lap... But also, who cares - so what if it is 100 times better then us, maybe we would learn some new ways of flying, who knows, but I do this for fun, and I am terrible and that does bum me out sometimes a bit but generally I just enjoy trying, talking to friends, seeing what I could learn, etc. I came from racing motorcycles for run but I can't really do that anymore and I really miss the tracks and racing and everything that went with them. For me, FPV seems like it is tapping into that same need. Cheers
since this one launches from a hover, it will just bring it self to the true starting point. I'm not sure about the race we did the day before, since those were from starting blocks! -Thomas
It doesn't use any yaw, which is what creates that insane perspective. I believe they have another version that keeps the camera oriented more forwards with horizon level! -Thomas
Considering the way it detects the position and orientation of the quad is via cameras mounted around the room, it technically is flying Line-of-Sight....
How does the AI controller compare to a physics-based handcrafted method? With accurate, high-speed absolute position tracking you can control it basically with a physics model and PID tuning. I'd expect this to be much more reliable.
Hi! They've done this before! If I remember correctly they called it a model-based approach, or a roboticist's approach. Basically model the core scientific principles we understand of how the world works, and use that as the basis to guide the drone in 3D space. I believe the short comings vs AI approach was that even in a controlled environment, there are so many uncontrolled variables that the current model-based approach struggles to account for, so they have to limit its speed to account for error. I guess using a neural network, it can find patterns in these inconsistencies and push closer to the limit! I believe they might have some coverage on the website and RUclips channel for the Robotics and perception group about this! -Thomas
Interesting to see how the AI uses the drones control in ways a human would not, Just based on beinng able to see where you are going as a human. It flips and turns Like a human flying it with their eyes closed Just based on muscle memory almost. Pretty trippy. Makes you realize these lil FPV drones are capable of a ton more then even the best pilots have achieved. Stuff a human may never be able to do.
The computer in the background at 3:00 looks so similar to mine that I actually looked over to make sure the computer I'm currently using was still there. 🤦♂
Is there any way to get access to their PID tuning system for the quad? Imagine if that tech could be available for everyone to easy tune their quad based on flights.
Believe it or not that was running stock Betaflight PIDS! The AI just learns to input whatever RC Commands are necessary to make the drone do what it wants in the real world! After we got home, we sent them our PID tune so their motors would run cooler. -Thomas
Is it me or did you use "The vicon drone" at 2:10 again at 2:12 to transition between both scenes? Also, seeing that drone lift off and just full send it made me audibly gasp, that was amazing.
ha ha yes they were from two different recording days. We learn't a lot about voice over setup and made improvements. We could have re-recorded the middle segment but we have so much other footage to push out atm we decided its better to get the information out rather then linger on it and have the info become obsolete! The launches are insanity, I don't think I can move the pitch stick quickly enough let alone manage the throttle to pull off a launch similar to that, but I so wanna learn it because it looks so damn cool! -Thomas
@@BMSWEB Might be able to set up a "launch control" like mode similar to what cars have. Have a separate button set up to turn the motors on full blast for the start, or the reverse. A button that allows you to input full throttle but it not send to the drone until you let go of that button. That way you lose the stick movement input time and a tiny bit of the human latency. No idea how one would code this... or if you would need to code..
I want to be able to map a city with the Mavic 3 or so. Or a 4x4 squared miles grid that I can then plug into a virtual flight and then fly the city in FPV simulator?
This is a very interesting approach to for flight control. I really like the idea of tracking markers. It would be interesting to see what kind of other sensors could be used to gather info about the environment and what could potentially be used for a on-the-fly ai-learning flight system. I worked for Fortem Technologies and we developed a swap-c radar to track drones and shoot them down. I had some ideas that I would have loved to explore, but the path that was rod for the company didn’t really allow for exploration in some of those directions. I love flying fpv and auto racing, I notice that a lot of the fpv pilots don’t take the optional lines, because humans don’t have reaction times necessary, and I’ve always wanted to know what would happen if AI was used. So it’s interesting seeing this happen and extremely satisfying!!
Well keep in mind humans also fly with one camera so the orientations of the quad is limited, where as this thing can fly in virtually any direction. It's a fascinating approach for sure and I'm sure this sort of tech will allow for some amazing breakthroughs that can then flow on to their Vision Based Drone which realistically what's going to be more practical in the real world. Irrespective of everyones different takes on where this tech will go, it's going to be interesting times ahead - Paul
@@BMSWEB That was something I hadn't considered, that it can fly in different directions than just the one facing the camera. As soon as I saw it, I smiled and thought that it was clever and also a very logical step in being more efficient in some of the cases. One of the things I was curious about was how long it takes for them to run the simulation to learn the track. It would be interesting to know the tradeoff of flying slower versus having a high enough resolution to fly the track. When you said something about 32 minutes since designing the track and about 40 minutes since the track was created, how long do they need to run the learning/simulating part of it for the drone to fly it?
When you guys were digitizing the track, you mentioned that you used the markers to define what angle the drone should fly through the gate at. When the AI trains, does it always respect this angle (and i guess inner gate position) or is it free to discover a "faster one" if it can? Do you guys make the gates a bit smaller in the simulation so it leaves some room for error in the real world?
Not having to orient the fpv camera to actually see the course is another advantage the computer has here. The computer can fly in any orientation, since it's looking at the drone from the 3rd person perspective. What sort of techniques does the AI team use here? Some sort of reinforcement learning?
@@BMSWEB thanks for the clarification. Have you guys experimented with ELRS for your own racing? I have been pushing for a decent elrs + vtx combo board...
@@BengalCatChilli We did ages ago and it worked great, however stuck with TBS mainly due to approvals etc Now speaking of approved hardware, when Thomas competed in the Japanese JDL Series (Paul here btw) a couple of months ago, the only approved hardware/software for Japan is Futaba and BetaFPV Nano ELRS! Before heading over to Japan Thomas had heaps of help by the absolutely amazing ELRS guys and learned heaps, but when he got to Japan there seemed to be issues with the Hardware and hence reliability issues with it. Let me be absolutely clear here, the reliability issues are due to the Hardware and version of software that was approved for Japan. I think if you use the right hardware ELRS should work perfectly.
wow, this blew my mind. I now need to at least try to use the TBS module with a pc. The FPV feed looked BRUTAL, but is it really faster being that brutal ? normally smooth is fast. jerky just looks fast (at least in humans?) edit: unless it's only fast for us? as in better for our brains to process faster? passing the gates at uncommon (for us) attitudes might be faster, but the jerkiness idk. edit2: the throttle control it has is AMAZING
You have it basically right, I feel. Smooth is fast for humans in large part because they need to use the camera to see and control. The AI isn't even using the camera at all, so any sense of stabilization, for it is unnecessary. Still quite impressive ofc, I just feel the FPV is a bit misleading because of that. It doesnt have any reason to point it in the right direction even.
it looks to me as if it were always snapping to either full throttle or no throttle. Also, doesn't the sending commands back and forth add significant delay? Humans adapt and train to such delays easily, but i imagine that for best performance, the main flight ai should reside within the drone with externally only some rough course correction data supplied. Additionally same ai should do the collision avoidance i think.
In the previous video we do show their AI Vision Based drone and it does fly extremely well, however it's whole computing system is also on board which makes the drone much heavier. Also I doubt any onboard vision based drone will be able to the way this does because direction / orientation isn't relevant to this Vicon Drone
Amazing stuff! The AI feed is interesting. As drone pilots, we don't/can't fly that way because we need to have visual frames of reference. Is that on the roadmap?
@@BMSWEB Oh wow. I had forgotten about the vision drone. Any chance you can get a hold of the video footage from the vision drone? I would love to see what the AI vision drone sees!
@@Waylandir It's in the previous video, we managed to get their footage from the vision drone. All the processing is done on board with that drone and hence the added weight and significantly less performance than this drone, but it is more realistic in terms of what is currently possible. I think the vision from it is in the part where Thomas explains what the Vision Drone is - Paul
@@BMSWEB I see it now. It starts at 3:55. Really cool stuff. Interested to see the flight improvement. The small clip is impressive but is very reactive. The footage looks like a warm up lap of a human pilot just learning the track. But after learning the track, top fpv pilots learn where to shave time by anticipating and pre-turning. Very interested to see if the vision drone can "learn the track" and then apply that knowledge to anticipate and pre-turn. Thanks Paul! Hope you guys are doing great!
@@Waylandir Thanks buddy, the tech is really interesting, would honestly love to work more closely with them as I feel some of their logic is slightly off. And the flip side is we would also stand to gain from learning from them. The issue with the Vision Based system is it's currently very sensitive to light changes etc. So on some of the days it just didn't perform as expected. Having said that in time, I have no doubt they will resolve a lot of issues and have a more reliable system which is going to be insane. We're doing great, lots of R&D happening now in house which is awesome! Exciting times ahead
Thomas awesome stuff, super cool to see the progress they make. Do you know what’s the next step for them? Is it always about fully controlled environment or they plan to develop something usable everywhere?
I assume they will finalise this experiment, do a scientific paper on in it and that will be it for now, until someone decides to develop it further. It just depends and where and how they see it used. - Paul Here
Well... Its interesting, but its pretty minimally useful. Its not flying FPV and none of the processing is onboard. It becomes really clear that its being treated as an abstract physic object when you see it going through the gates backwards and sideways in ways that a human wouldn't have enough information to make possible. Its kind of like they took a stock game AI for flyers from something like Unreal 4 and glued it together with a massively expensive tracking system. Its just trying to fly a line through space. If you put a pole in its way it would smack right into it without even knowing it was there.
Hi! Refer to the previous video about the Universities work to understand why this specific part of the project exists, and yes your right, in its current form, this system doesn't have any object avoidance! -Thomas
@@BMSWEB Yeah, I mean I know you understand exactly whats going on here, obviously, but my comment was more for other people in the comments section who seem to have literally no idea, despite the explanation, that this is NOT an autonomous vehicle using its onboard camera and some kind of onboard AI to fly it. I'm not sure how they don't understand that none of those things are what is happening, but they don't.
Fascinating. How many simulation iterations are required for a 99%+ lap completion consistency? Also, what rate of iteration are the team using? Using phantom helper/guide gates was a slick move on the team's part!!! Good work Thomas, and Paul too!🥰
Cheers! You'll probably have to ask the university about that! They have an even newer version of the simulation software that I believe no longer requires helper gates! Pretty insane stuff! -Thomas
This is uncharted waters now, for millitary guys are salivating at the prospects of weaponizing these AI drones. Leisure drone have been put to lethal use in Ukraine, now imagine what the addition of AI could do! The future of AI drones is going to be scary af, I shudder to even think about it😯
“We’ve got 19 minutes til we gotta leave for a flight. We’re gonna build a new track and see how it adapts in such short time.” *5 video minutes later* “It’s been 32 minutes since we made the new track...”
Wow! The FPV feed of the AI drone is crazy! The way it goes through the gates what ever way is fastest is nuts! Thanks BMS Fam!
It's really trippy isn't it 😂 orientation has no relevance when it comes to that flight
@@BMSWEB that's a fascinating way to put that. We puny humans rely on that camera facing forward. The all the AI knows is 4 motors equally spaced apart, so it just points and shoots 🤣
No doubt, my brain was hurting! By the time my brain figured it’s orientation it threw it off again!
Amazing Tech !
Keep up the Great work
@@BMSWEB I wonder if anyone has tried to fly with 360 fpv or placing the fpv camera in the center and pointing it 90 degrees. Maybe it would make this possible in humans.
That fpv footage at the end is terrifying. Very unsettling, very impressive.
WOW! I'm 60 years old and only just starting to get into this whole quad scene. I wish i was 20 again! I've had an Atom camera drone for a few months and love flying it.
I just bought a Boxer and 3 Sims to practice on before i either buy or build my first freestyle quad. My reflexes will never be near you guys, but i can still have fun with it.
If i can get to be 20% as good as you are Thomas, i will be more than happy. That made me smile bigly after that AI run :)
We can't even imagine where this tech will take us in 20-30 years time. You guys are living into a fascinating future. Make it count.
Chasing you down the street at night screeching "halt citizen!" Because you missed the trash can and it thinks you littered
Not a good future being built
*Beep Boop* "Pick up that can"
wouldn’t work since they’re too loud and people would complain.
That FPV view at the end is just crazy ridiculous I love it
that shit was going backwards
I know nothing about AI or Drone-Racing but RUclips thought I should watch this. And Wow! That was f… fast. Keep it up, people like you will make our world a better place.
I'm glad you enjoyed the video! Ha ha drone racing in general is a pretty crazy sport, hard to believe it didn't even exist a few years ago! I'll do my best!
-Thomas
I do a lot with AI, but this is freaking cool to see real world action of it! Congrats, that was definitely a success! Awesome job!
Oh man I've been seeing some of the upgrade work they've done since this video and its even more insane. Super excited for a rematch next year, but I suspect in a controlled environment there won't be much a competition for their creation!
What a time to be alive hey? :D
-Thomas
@@BMSWEB Yes, the bad thing about it is, the military will take advantage of it first and use it... We made many innovative devs, but it was always the military who had in the end the right to say if its for public or not. And drones you can see it in the Ukraine war... are the generation in which we are living... right now. AI at it self is really cool, like a new dimension. I often compare it to LEGO or Fisher-Technik, when you was able to start to build your ideas :-)
i love that FPV, it really show how the AI does not care what is "front" and "back" like a human, big advantage
It's pretty cool to see it in real life. I'd love to go back through the footage and see if there are any movements that are easy enough to pickup using this method. I don't think there's much but surely there's an advantage for human spec racing there somewhere!
-Thomas
@@BMSWEB the camera see 360°, you could get something similar with multiple fpv cameras + VR headset.. The issue is the latency, keeping them < 100ms on digital is a pain (aka: VERY expensive)
Commenting just for support! Amazing tech!
Thank You
That gave me chills alone just watching the full send
fantastic
Wtf, this is completly amazing.. Holly shit :D
Amazing! Thanks for sharing this!! Best regards from Brasilia, Brazil.
Incredible watching the FPV view of the AI drone. It completely has no regard for typical forward motion like us humans need for most of our flights. It'l just fly at whatever angle it wants to get the job done.
Some of the sim guys are pretty insane and starting to fly a little like this in some corners, but to see it in real life is mind bending!
-Thomas
I think human brains are capable of it, it's just a matter of training and forcing yourself to "fly blind". Perhaps not to the extent shown here though, as the mo-cap data allows the drone to "see" the entire course at all times.
these are going to be chasing us real soon. thanks a bunch
those things exist and are capable of chasing you for more than 10 years now...
the entire concept of ai race drones blows my mind. phenomenal footage!!!
This is so sick, thanks for the follow up vid!
Thanks buddy, it always surprises me when you comment on our vids! - Paul Here
How cool is that?! Wow!
damn! its very amazing man! great job for teams
I remember being curious about what an FPV might look like. This is great.
nice!!
Amazing! Thank you for sharing this journey!!
Thanks! :D
super cool!
Awesome work guys!
Cheers!
-Thomas
Dat fpv flight at the end looks like mck Line of sight
This is insane
incredible video and production guys! May it gain many millions of views :)
This is beyond interesting to me! Thank you guys!
super amazing lot of work behind
Cool. Very cool. Niiice videorrr guys
So amazing love this pls bring more
We've probably got at least one more video on the AI vs Human race that we can do for sure. It's just been so hard finding the time to edit these videos. We've still got the Aussie Nats, WA State Titles and also Thomas' Japan drone Racing League race which will be so much fun to watch. It was a pretty eventful race that one lol
Holly shit. That was fuckin nuts. I was not even there, and I felt the adrenaline rush.
Re watching the flight footage is still mind blowing for me. I dream I can fly a quad like that 🤩
-Thomas
That was insane.
I don't think any AI "Pilot" will be capable to beat Tomas. He's way beyond! Can't wait to watch the race! Nice project by the way and thank you for sharing this. Your BMS Fan.
I am not sure - computers have every single advantage over us and once the AI get's enough flights in it will be able to fly at the edge of the drones aerodynamic capabilities without errors and it will be consistent and will or can keep it at that level of performance for as long as the drone has batteries... However I don't think it matters, we as people should just enjoy what we are, what we can do, etc, have fun pit racing, talking with friends, actually racing and working to improve our skills, etc.
The technology is mind boggling, AI is creeping into every single aspect of our lives and one day not too long from now and if we don't blow up the planet before then we will see AI everywhere. Now about job loss to AI, etc... a serious problem, a horrifyingly serious problem really. But I have an idea - it is one that I have had for a year or 2 now - let's ask AI how we can solve that problem.
Also, here, a little off frack but related to this video is the greatest idea ever in all humanity - I would say that we should replace 99% of all human politicians with AI. Think about it, an unbiassed, un-corruptible (don't say hackers... yea that idea is concerning, but let's stick to the core for now), what I mean is it would have no friends to give jobs to, no nepotism, no backroom deals, etc... it can work tirelessly for ever, non stop...
it will be smarter, make better decisions, make decisions that are the best for us, for the whole world. Solve the energy crisis, solve global warming, solve equity in society, solve everything, cure diseases, and who knows what else... and it will beat us at any game that we challenge it to. I for one welcome our new overlords. Cheers?
its like he said, maybe it cant match his peak run, but in terms od consistency it can be better than a human pilot
Ai will beat us all soon
Idk if you saw that end footage but damn, no FPV flyer is gonna beat that. Give the AI a few more iterations and a longer simulation time and it's gonna be unbeatable.
@@rizen3467 No I get it, but the computer is capable of such precision, perfection, ability to repeat exactly what is needed lap after lap... But also, who cares - so what if it is 100 times better then us, maybe we would learn some new ways of flying, who knows, but I do this for fun, and I am terrible and that does bum me out sometimes a bit but generally I just enjoy trying, talking to friends, seeing what I could learn, etc. I came from racing motorcycles for run but I can't really do that anymore and I really miss the tracks and racing and everything that went with them. For me, FPV seems like it is tapping into that same need. Cheers
i will follow you because I like what you do and want to learn what you can do
FC From Thailand
This crazy!!!! 🤯 They done a good job wow! Thank you Thomas for sharing this impressive project that the University of Zuirch has!
absolutely insane!
That is so awesome.
Pretty crazy to see that thing full send maiden on that track 🤯💥
We honestly had no idea what to expect and hence why everyone made sure they were well and truly behind that net 😂
Keep those Cold Fusion vibes coming. 👍
Thanks 😊, more like Mini Fusion :P - Paul Here
That's awesome
this is sick
What happens if you move the launching spot few feet backwards? Does it affect everything?
since this one launches from a hover, it will just bring it self to the true starting point. I'm not sure about the race we did the day before, since those were from starting blocks!
-Thomas
@@BMSWEB very cool! I hope one day, I can be part of projects like that
INSANE!! The fpv view looks like its flying it LOS
It doesn't use any yaw, which is what creates that insane perspective. I believe they have another version that keeps the camera oriented more forwards with horizon level!
-Thomas
Considering the way it detects the position and orientation of the quad is via cameras mounted around the room, it technically is flying Line-of-Sight....
Insane stuff but a little bit scary as well lol
How does the AI controller compare to a physics-based handcrafted method? With accurate, high-speed absolute position tracking you can control it basically with a physics model and PID tuning. I'd expect this to be much more reliable.
Hi! They've done this before! If I remember correctly they called it a model-based approach, or a roboticist's approach. Basically model the core scientific principles we understand of how the world works, and use that as the basis to guide the drone in 3D space. I believe the short comings vs AI approach was that even in a controlled environment, there are so many uncontrolled variables that the current model-based approach struggles to account for, so they have to limit its speed to account for error. I guess using a neural network, it can find patterns in these inconsistencies and push closer to the limit!
I believe they might have some coverage on the website and RUclips channel for the Robotics and perception group about this!
-Thomas
INCREDIBLE!!!
Very interesting. Would like to see the same but the turned FPV camera to the moving direction.
Interesting to see how the AI uses the drones control in ways a human would not, Just based on beinng able to see where you are going as a human. It flips and turns Like a human flying it with their eyes closed Just based on muscle memory almost. Pretty trippy. Makes you realize these lil FPV drones are capable of a ton more then even the best pilots have achieved. Stuff a human may never be able to do.
Can I recommend if you send viewers to watch another video before this one, to put the link in the description 😉
Finally arrival
That’s crazy 😮
A nerrr drerrn racing videorrrr. Nice!
Thomas is editing the Japan video right now! I think you'll enjoy that adventure
Awesome
Holy Moly!
Increible!!!
This is badass
The computer in the background at 3:00 looks so similar to mine that I actually looked over to make sure the computer I'm currently using was still there. 🤦♂
Is there any way to get access to their PID tuning system for the quad? Imagine if that tech could be available for everyone to easy tune their quad based on flights.
Believe it or not that was running stock Betaflight PIDS! The AI just learns to input whatever RC Commands are necessary to make the drone do what it wants in the real world! After we got home, we sent them our PID tune so their motors would run cooler.
-Thomas
Is it me or did you use "The vicon drone" at 2:10 again at 2:12 to transition between both scenes?
Also, seeing that drone lift off and just full send it made me audibly gasp, that was amazing.
ha ha yes they were from two different recording days. We learn't a lot about voice over setup and made improvements. We could have re-recorded the middle segment but we have so much other footage to push out atm we decided its better to get the information out rather then linger on it and have the info become obsolete!
The launches are insanity, I don't think I can move the pitch stick quickly enough let alone manage the throttle to pull off a launch similar to that, but I so wanna learn it because it looks so damn cool!
-Thomas
@@BMSWEB Might be able to set up a "launch control" like mode similar to what cars have. Have a separate button set up to turn the motors on full blast for the start, or the reverse. A button that allows you to input full throttle but it not send to the drone until you let go of that button. That way you lose the stick movement input time and a tiny bit of the human latency. No idea how one would code this... or if you would need to code..
Also, external camera tracking is easy. Make it on board then it would be something
The first video covers this :D
-Thomas
"oohh, that's nice, I like it a lot* -skynet
I want to be able to map a city with the Mavic 3 or so. Or a 4x4 squared miles grid that I can then plug into a virtual flight and then fly the city in FPV simulator?
This is a very interesting approach to for flight control. I really like the idea of tracking markers. It would be interesting to see what kind of other sensors could be used to gather info about the environment and what could potentially be used for a on-the-fly ai-learning flight system.
I worked for Fortem Technologies and we developed a swap-c radar to track drones and shoot them down. I had some ideas that I would have loved to explore, but the path that was rod for the company didn’t really allow for exploration in some of those directions.
I love flying fpv and auto racing, I notice that a lot of the fpv pilots don’t take the optional lines, because humans don’t have reaction times necessary, and I’ve always wanted to know what would happen if AI was used. So it’s interesting seeing this happen and extremely satisfying!!
Well keep in mind humans also fly with one camera so the orientations of the quad is limited, where as this thing can fly in virtually any direction. It's a fascinating approach for sure and I'm sure this sort of tech will allow for some amazing breakthroughs that can then flow on to their Vision Based Drone which realistically what's going to be more practical in the real world. Irrespective of everyones different takes on where this tech will go, it's going to be interesting times ahead - Paul
@@BMSWEB That was something I hadn't considered, that it can fly in different directions than just the one facing the camera. As soon as I saw it, I smiled and thought that it was clever and also a very logical step in being more efficient in some of the cases.
One of the things I was curious about was how long it takes for them to run the simulation to learn the track. It would be interesting to know the tradeoff of flying slower versus having a high enough resolution to fly the track. When you said something about 32 minutes since designing the track and about 40 minutes since the track was created, how long do they need to run the learning/simulating part of it for the drone to fly it?
Was the drone flying 3d? I wanna see that next!!!!!!!!!
When you guys were digitizing the track, you mentioned that you used the markers to define what angle the drone should fly through the gate at. When the AI trains, does it always respect this angle (and i guess inner gate position) or is it free to discover a "faster one" if it can? Do you guys make the gates a bit smaller in the simulation so it leaves some room for error in the real world?
It's free to discover it's own path and racing line. Only requirement is it passes through that section/waypoint in the correct direction - Paul
Not having to orient the fpv camera to actually see the course is another advantage the computer has here. The computer can fly in any orientation, since it's looking at the drone from the 3rd person perspective.
What sort of techniques does the AI team use here? Some sort of reinforcement learning?
This was such a good watch! Some amazing results. Someone get these guys some ELRS gear 😅
Tracer is crazy fast though. They’re probably using it to minimize latency.
@@Flumphinator elrs can go 1000hz...tracer only 250.
There is a latency and consistency component to it, but on the refresh rate side their Tracer operates at 1000hz already :)
-Thomas
@@BMSWEB thanks for the clarification. Have you guys experimented with ELRS for your own racing? I have been pushing for a decent elrs + vtx combo board...
@@BengalCatChilli We did ages ago and it worked great, however stuck with TBS mainly due to approvals etc Now speaking of approved hardware, when Thomas competed in the Japanese JDL Series (Paul here btw) a couple of months ago, the only approved hardware/software for Japan is Futaba and BetaFPV Nano ELRS! Before heading over to Japan Thomas had heaps of help by the absolutely amazing ELRS guys and learned heaps, but when he got to Japan there seemed to be issues with the Hardware and hence reliability issues with it. Let me be absolutely clear here, the reliability issues are due to the Hardware and version of software that was approved for Japan. I think if you use the right hardware ELRS should work perfectly.
Does anyone know what langue they would use to write the program for the drone program?
Main advantage is that the AI drone doesn't have to go "forward".
wow, this blew my mind. I now need to at least try to use the TBS module with a pc.
The FPV feed looked BRUTAL, but is it really faster being that brutal ? normally smooth is fast. jerky just looks fast (at least in humans?)
edit: unless it's only fast for us? as in better for our brains to process faster? passing the gates at uncommon (for us) attitudes might be faster, but the jerkiness idk.
edit2: the throttle control it has is AMAZING
You have it basically right, I feel. Smooth is fast for humans in large part because they need to use the camera to see and control. The AI isn't even using the camera at all, so any sense of stabilization, for it is unnecessary.
Still quite impressive ofc, I just feel the FPV is a bit misleading because of that. It doesnt have any reason to point it in the right direction even.
Looking at the lap time, the ai did a lap in about 1/3rd the time of the human pilot.
So yes, faster.
it looks to me as if it were always snapping to either full throttle or no throttle. Also, doesn't the sending commands back and forth add significant delay? Humans adapt and train to such delays easily, but i imagine that for best performance, the main flight ai should reside within the drone with externally only some rough course correction data supplied. Additionally same ai should do the collision avoidance i think.
Just mad!
How long untill a drone can fly this well with just its onboard camera system and AI
In the previous video we do show their AI Vision Based drone and it does fly extremely well, however it's whole computing system is also on board which makes the drone much heavier. Also I doubt any onboard vision based drone will be able to the way this does because direction / orientation isn't relevant to this Vicon Drone
To check if movie is worth of watching check 6:50
Crazy
Next gen military aircraft
Amazing stuff! The AI feed is interesting. As drone pilots, we don't/can't fly that way because we need to have visual frames of reference. Is that on the roadmap?
Hi! The vision drone from the previous video purely flies from onboard vision! It's quite a bit heavier but it's an amazing piece of kit!
-Thomas
@@BMSWEB Oh wow. I had forgotten about the vision drone. Any chance you can get a hold of the video footage from the vision drone? I would love to see what the AI vision drone sees!
@@Waylandir It's in the previous video, we managed to get their footage from the vision drone. All the processing is done on board with that drone and hence the added weight and significantly less performance than this drone, but it is more realistic in terms of what is currently possible. I think the vision from it is in the part where Thomas explains what the Vision Drone is
- Paul
@@BMSWEB I see it now. It starts at 3:55. Really cool stuff. Interested to see the flight improvement. The small clip is impressive but is very reactive. The footage looks like a warm up lap of a human pilot just learning the track. But after learning the track, top fpv pilots learn where to shave time by anticipating and pre-turning. Very interested to see if the vision drone can "learn the track" and then apply that knowledge to anticipate and pre-turn. Thanks Paul! Hope you guys are doing great!
@@Waylandir Thanks buddy, the tech is really interesting, would honestly love to work more closely with them as I feel some of their logic is slightly off. And the flip side is we would also stand to gain from learning from them. The issue with the Vision Based system is it's currently very sensitive to light changes etc. So on some of the days it just didn't perform as expected. Having said that in time, I have no doubt they will resolve a lot of issues and have a more reliable system which is going to be insane.
We're doing great, lots of R&D happening now in house which is awesome! Exciting times ahead
Thomas awesome stuff, super cool to see the progress they make. Do you know what’s the next step for them? Is it always about fully controlled environment or they plan to develop something usable everywhere?
I assume they will finalise this experiment, do a scientific paper on in it and that will be it for now, until someone decides to develop it further. It just depends and where and how they see it used. - Paul Here
Das änglisch dönt chle schwizerdütsch irgendwie 3:25 hehehe
Hi, can we make AI fly Ukrainian FPV drones that target invadors?
imagine chased by a killer drone like this
Well... Its interesting, but its pretty minimally useful. Its not flying FPV and none of the processing is onboard. It becomes really clear that its being treated as an abstract physic object when you see it going through the gates backwards and sideways in ways that a human wouldn't have enough information to make possible.
Its kind of like they took a stock game AI for flyers from something like Unreal 4 and glued it together with a massively expensive tracking system. Its just trying to fly a line through space. If you put a pole in its way it would smack right into it without even knowing it was there.
Hi! Refer to the previous video about the Universities work to understand why this specific part of the project exists, and yes your right, in its current form, this system doesn't have any object avoidance!
-Thomas
@@BMSWEB Yeah, I mean I know you understand exactly whats going on here, obviously, but my comment was more for other people in the comments section who seem to have literally no idea, despite the explanation, that this is NOT an autonomous vehicle using its onboard camera and some kind of onboard AI to fly it.
I'm not sure how they don't understand that none of those things are what is happening, but they don't.
wow alex vannover also involved
So was MarvFPV (Previous Video)
I saw this video 2 month ago - how can this be? maybe it was on another channel
The quad got mad at a little mistake and smashed itself into the ground full speed😂
Fascinating. How many simulation iterations are required for a 99%+ lap completion consistency? Also, what rate of iteration are the team using? Using phantom helper/guide gates was a slick move on the team's part!!! Good work Thomas, and Paul too!🥰
Cheers! You'll probably have to ask the university about that! They have an even newer version of the simulation software that I believe no longer requires helper gates! Pretty insane stuff!
-Thomas
@@BMSWEB thanks buddy! I'll email the team directly if I can. Exciting stuff!
This is uncharted waters now, for millitary guys are salivating at the prospects of weaponizing these AI drones.
Leisure drone have been put to lethal use in Ukraine, now imagine what the addition of AI could do!
The future of AI drones is going to be scary af, I shudder to even think about it😯
Bagaimana dia melakukan itu
There's a lot of nerd testosterone in that room, I only wish I was worthy
wake me up when the AI can control the drone from onboard drone camera alone
That already happened in the previous video lol
I really am concerned about the future...
my god
NEVER piss off a drone pilot…🇦🇺
Now put face ID and strap some C4 onto it
순전히 재미로만 남고 나머진 ai로 대체될 듯
“We’ve got 19 minutes til we gotta leave for a flight. We’re gonna build a new track and see how it adapts in such short time.”
*5 video minutes later*
“It’s been 32 minutes since we made the new track...”
Absolutely no idea where you got 19 mins from 🤣 but thanks 👍
@@BMSWEB 1:10 could’ve said 90 but sounds like 19 to me. Either way the drone is badass. Cool video.
Yet more proof AI will render humans obsolete in just about everything. And yet companies are racing at lightspeed to develop smarter AI.
4:47 - MineCraft ahahhah
The all mighty start of any science project XD
-Thomas
eurver and eurver again