@@NicholasRehm My thoughts exactly. I am a undergrad engineer just moving into the first engineering job in motion control and really love your efficient approach to control. Would be very eager to find out just how much math you do in the process.. :o
I was convinced that the thumbnail was clickbait and the actual contraption would be an abomonation but I'm super impressed with the outcome! Really nice work!!
Optical flow sensor also picks up your angular movements. You need to estimate flow created from tilting (via gyro and lidar) and substract it from from flow sensor reading. While doing so, beware of different latencies.
This was my first thought when I saw what the sensor is doing. Any pitch or roll will give velocity from the optical flow even if the thing isn't really moving anywhere. Not something I would want to try and figure out...
@@theaveragepro1749 It didn't seem like it from the video. Like, I'm sure he's smart enough to know it, but even smart people make dumb mistakes sometimes.
This is an amazing project considering it was put together by a single person, the integration here is fantastic, this dude is a real polymath. It would be a huge software task but I'm sure Nicholas thought about the next level of using the phone camera, sensors, processor themselves to aid in stabilization, this rig literally carries around a camera and processor capable of using ML to stabilize on an object detected target.
Thanks for the kind words. Phone processor integration I guess would make sense, but app development and proper software dev is where my expertise falls off lol
Incredible work and while I would agree that this is not a viable product (a DJI Mini is probably just as good and offers more stability for the relative small cost of having a remote), the learnings are great and now I want to play around with you flight controller even more :)
not a viable product *yet - pretty sure at this point it's a race to the bottom with high integration, just look at all the cheap quad toys you can get for like 30 bucks
Just wait until wireless power transmission gets better. Battery pack in my backpack, and my phone will just always follow me. Won't have to actually hold it. Can't be dropped. With voice control. Would be quite cool.
You're doing something right. You have been in my feeds and my personal suggestions irl for a couple of years without a subscription. Keep being awesome!
Brilliant project to tackle and demonstrate as a flying prototype! You solved so many aspects to making the flying DroneCase a usable product. This is many levels beyond using an extra long extended selfie stick. Now feeling the need to build a DroneCame to experiment and explore, as way more fun! The one challenge I see remaining is adjusting the optical flow for when the DroneCase is tilted significantly from vertical as it try's to maintain position hold (6:00). ie: differentiate between horizontal motion and secondary tilt of DroneCam related to position hold (particularly in a dynamic environment with variable wind). While lidar distance to ground is easy to compensate as tilt angle is known; the optical flow device would see massive motion as DroneCase tilts more than a minimal angle. At 6:24 is an example were optical flow would be swamped (or lost) by large path differences relative to the ground below. If optical flow had 2-axis gimbal, then a 'normal' reference to the ground target could be maintained; at the cost of added complexity (and weight). Alternatively, think it would be easier for DroneCame to maintained a constant radial distance to the subject. If (camera subject of focus) had a remote dongle, then DroneCase could use a bluetooth signal strength to get "range" data. This could help with when adjusting to larger position deviations (associated with larger tilt angles), and optical flow could fine tune position when tilt angles smaller. As a bonus, if the remote dongle had a compass, then when the subject turned, or moved the DroneCase could move in an arc at constant radius to the updated subject orientation. Obstacle detection would then be required. Again, is brilliant how identified core features and avoided the spiral of feature creep. ;)
Well, as a UAV manufacturer I consider your design preaty well made, yet it of course can be improoved upon. The biggest problem you have encountered is generally the problematic nature of PX4Flow. I've been there. As of now VOXL Cam seems to be a much better soultion as it includes stere camera sensors which, at least when used on Ardupilot, seem to be dooing a great job. Also - usually the LIDARs used in integrated solution have very limited range and are prone to feeding the flight controller flase data. Here even a lightweight and low cost sensors like TF-LUNA seem to be doing a stellar job. I also might have ideas about further testing and even improoving your cyclorotor design and would be happy to cooperate with you. If you are interested - please, reply to this comment. All in all - great job.
hm.. VOXL Cam looks like it is really great and would indeed offer an improvement but... have you considered that it is also almost 10x the price at about 1500$? 😅 which is just ridiculous for a small drone-phone case! PX4Flow is
As an aspiring control engineer, I gotta say I absolutely loved this video!! Excellent job :)) you've inspired me to get back to work on my own GNC project! Quick side note, your custom flight controller is actually what got me into my current GNC project, which is basically just an extremely watered down auto pilot for model gliders haha
Oh man! This is epically awesome man. I love drones, I love 3dprinters, don't mind soldering, but don't have the brains to put it all together into anything; this is right down the lines of something I want to make! Thank you for putting this out there with all the code and steps you took. I am going to snag it for one of my next projects, I am super excited to take selfies with this at the next family Christmas get-together! Mind blown
@@MichaelRyannz One of the things making the flight controller work so nicely should be reliable execution times of each command to the rotors. Just imagine your drone crashing because Twitter was refreshed in the background, keeping the CPU busy at just the wrong time. I don’t think your phone allows you to run real-time code, does it? That said, using the camera image itself to measure optical flow and stabilise the position via an app on the phone sounds like a really nice idea, if there was a way to do that.
@@sebidotorg Optical flow boards use dedicated circuits to process the imagery. The phone just won't have the processing power to do it at a decent rate with a low enough latency. The px4flow is just a really old board. If you want to spend the money the herflow is a similar package that integrates the lidar onto the board and weighs 1.2 grams compared to the px4flows 30 grams. Runs at 50hz. Probably save 60-80 grams vs this prototype design. If you really want good performance you get a machine vision camera with a 80hz-400hz global shutter sensor and have that do the flow tracking for ya, but that adds all the weight savings back on.
@@nocare That’s what I thought. While it would be nice to use the actual image you want stabilised as the source for the movement information, I didn‘t think the image processing on a phone would ever reach a latency that‘s low enough for such an application. Which refutes the comment that it would be better to use the phone‘s processing power. Thanks for your input!
@@sebidotorg No problem. Also if you want to have a target you place on the ground to track then you can do what you wanted using the machine vision cameras. Such as the OpenMV camera and others like it. Aruco tags and April tags are examples of fiducial markers. From just 1 tag you can get the position and orientation of the camera in 3d space relative to the tag. Conversely you also have the tags 3d pose as well.
I wish I could subscribe 3 times. I don’t want a selfie drone, but just found your channel. Holy cow, you are so intelligent and creative. Your videos are amazing to watch. Truly master class material.
That is amazing. What gets me is the fact that the scammer had more people watch his video. Then the both of you put together. I think you are build is amazing
2:30 I have the same solder station/power supply as you. I love mine and is a major quality of life upgrade from the ones that plug directly into the wall.
I’ve been building one with your software for the past few months!! I didn’t know you were building one too! I’m using mark robers design but this one is more practical. I’ll post on my channel once complete!
@@NicholasRehm i uploaded a quick video on my channel to show the progress, its still gonna take a while for it to be completed, the design isn't perfect but i really wanted to go with the original peter sripol design.
If anything complicated looks simple, there is a very good chance THOUSANDS of hours of work went it to it. As you say it's not just as simple as "slappin a GPS on it". Respect!
Wouldnt the tilt of the phone rocking around add to the misreading of the velocity sensor? Because the phone dosnt stay in a fixed vertical position it rocks back and fourth, even though the drone may be stationary above the ground as the phone wobbles and rocks it will give it false readings. Mabye some vector math between a gyro sensor and your velocity sensor could correct for the rocking angle of the drone?
PX4Flow does this compensation onboard, but I think mine's gyro went bad and it wasn't doing compensation anymore. I tried to do this my self in the flight controller, but it can only work so well...
Great project! I am not sure how powerful is the flight controller, but you can use LQR or even reinforcement learning based approach to control the motors if you decide to implement something more complex, such as following target
Great, I love it ! One thing you could do would be to use your phone's camera with a self made app to do face tracking and compute your phone position relative to your face but also relative speed, then, the app could send position adjustments requirement to your drone (through USB-C for Androids and Lightning for iPhones), as smartphone have more than enough computing power on their own to do that. Your phone would send position request, drift speed from the target, and your phone already get drift speed from stationary and altitude. One extra step could be to apply targeting from your position holding camera, making it able to not just get the relative speed from the gound, but also, to try and get back to the same initial point and hold it, making it much more stable than only reading horizontal velocity, as you now have a precise point to follow, only downside is that it wouldn't work in a totally uniform ground by lack of easily recognisable details (where speed can still be established) or a too complexe one (where it would looks more like noise than specific single target to hold onto). The only downside of using a specific app, is how the phone won't be able to take 100% of your camera's capabilities as only the native phone app is made for it, so you will have, at best, what other apps (Instagram, Snapchat, etc) can do, meaning, the default camera feature, so, no zoom, macro, portrait, etc. Though, you could try to run your own app in the background and order your phone to take picture with the main phone app through ADB commands.
Awesome idea! Thanks for sharing! You could also use something like a Ram Mount design with 4x sprung arms to make it phone-agnostic (so you don't need to print fixtures for each phone)
As an engineer struggling with ERA/OKID for the past month, the performance you get without system identification by directly tuning the gains is mind blowing! You must understand the pole positions really well to tune gains so well!
Would it be possible to run this off the phone? And all the control of the cam-copter could be done via a phone app(?) It would slim down the amount of components required, possibly by quite a bit. You could possibly access the selfie camera on the phone in place of the image stabilizer module you have? And the phone should have fairly accurate height data as well(?) Possibly removing the need for a lidar sensor.
Did you consider the possibility that the optical flow sensor gets disturbed by sligt rotations of the drone? Maybe that's what's causing the instabillities rather than the low quality fo the sensor.
Could the data weirdness from the flow sensor be due to what it's actually seeing? Typically they work best with a sharp contrast pattern and yours is a long way away from what it's tracking (the ground). Also one thing I just thought of is that with the sensor hanging off the bottom, it has a pendulum effect to it's vision cone, too - Every time the platform attempts to stabilize, the sensor sees a shift due to pendulum.
This turned out awesome! Is there a reason why you didn't use the phone's sensors to help with location stabilization? Other than the difficulty in integration of course, my question is more of "would it be doable and good enough"?
Did you in any way subtract the gyro detected tilt angle (from your control inputs) from the optical flow sensor output? I could see that seriously impacting stability by contaminating your sensor input.
This is a good idea, and if someone has not contacted you about producing it commercially, then they should. At its simplest, it should stay at a level you set when starting it, and it should follow a small transmitter you keep in your pocket. A price under $100 would make this a killer. Good idea.
Great video as always! Alternatively you could try RTK GPS for outside stability control. Something like NEO-M8P would give you 1-2 cm precision in all coordinates.
Or just use the GPS built into the phone. Sure it wouldn't be as accurate or have the same update frequency as a dedicated GPS module, but it's already there. Heck, get IMU data while you're at it.
I would love to see you solve the optical flow quality problem. I am yet to see an implementation of optical flow as a means of RF hardening a drone. If you could improve the optical flow to the point of being a reliable failover for the GPS it would be cool to see an autonomous drone that can't be brought down by the DroneGun!
@@NicholasRehm I think that was the video I found your channel through. I just didn't really see that as rf hardened and capable of long optical flow missions because of the limited test environment it was demonstrated in.
I've watched the videos from the other guys. They were not interesting and imo failed. I am glad that you have done such an amazing job. :) Good work. One thing i have in mid about it is - ur propelers have too much space to the ducts (pipes around them). This looses some performance. The less the space between the blade and the pipe , the more efficient it gets. You can try cutting and sanding the props and make them kinda "bull nose". Then make the space as little as you can. For a bonus point make the upper part of the ducts curved outside like 180 degrees or smth. This makes it even more efficient FOR LIFT. Making those changes could help you a little bit with stability but should help you with battery life too. Oh do not forget to balance your props (even new they are not). Use magnetic balancer as it is most accurate. Hope this gives you some ideas :)
Agreed--ducts were mainly for protecting my hands. The first set were pretty tight on tolerance, but as you saw that led to some problems with resonance and prop strikes
Thanks 4 a lot of interresting videos! 👍😊 Is it possible to use the sensors in the smartphone? I imagine an app that... 1: is an interface with the user 2: sends flight data to the drone - I guess you have all sensors you need in a common snartphone - If they are good/quick enough...
Great video! Can you go into more detail on the coding side of things? I'm curious about the PID control algorithm and I want to do something similar to measure flow rates of liquids for a project.
For the velocity control are you using the raw optical flow reading or are you also compensating for the tilt and roll of the housing. Namely, if the housing tilting back and forth from vertical, then the optical flow sensor will also read an oscillating velocity. When assuming that the altitude stays constant and the ground is horizontal, then one should be able to subtract this induced velocity (assuming the attitude is estimated accurate enough). A dirty solution could also be to filter the optical flow sensor with a low pass filter and/or notch away the typical attitude oscillations.
PX4FLow does angular rate compensation onboard but I think mine's gyro was broken with age. I tried doing this myself manually, but it could only work so well... Lots of "filtering tricks" were tried
This is brilliant, will order the parts immediately! You could also use the phone's sensors + AI to do the high level control and send that to the controller
I wonder how well it would work with a big mat under it with various unique tracking points? Or maybe some small 3d printed shapes you can stake or set onto ground under it?
Awesome idea and build with explanation... Couldn't a portable battery bank go inside the magnetic box this allow quick swapping and if just use a usb cord use any power bank then
This is really cool. Great work! I know that with a DJI Mini SE being just over $200 now, it doesn't seem like this could be a viable product, but with a few minor improvements and the right marketing it could be. If you could get the battery life up to 10-12 minutes, and could maybe give it say 3 height settings and 3 distance settings with a lever to automatically adjust for those 9 positions, and a timer to hold those 8 further positions (maybe with 3 settings as well), I'm confident I could market something like this for $150-$180 easily. It's all about finding the right angle to convince someone why they need something, and having a novel and unique presentation like this helps distract consumers from the fact that there may be other tools to get the job done more effectively.
Does the position controller use the accelerometer too? Your estimated velocity would drift over time but combined with the optical flow it would probably give you better results. Amazing project as always, I hope your channel really takes off!
Fascinating! How does the optical flow sensor distinguish between the movement of the drone along the XZ plane and the panning of the frame caused by the body tilting? Does the sensor have some sort of rotational stabilisation in it?
dang. i was thinking googles project ara's modular phone desing with its hotwappable i^2c power railing would be a great way to fly 4 props with a phone as a flight board but hey this works too! ever consider a monocular slam video?
You beat me too it XD dang it, now i have to add features to be more unique, Great job on the work man! you really did a decent job keep up the good work
Awesome I’ve watched your video twice. I have a suggestion remember when you made the ducts reinforced? After reinforcing the ducts it fully way better. In my experience vibrations can turn into a resident frequency and wreak havoc on your flight controller software. With the improvement you had when you strengthen the ducts, I believe further strengthening or purchasing a already made duct would work better if you could just make one that was stronger I really believe it would help with the flight characteristics .
Dude this thing is amazing! Much better than the contraption I came up with!
Haha but thanks for all the inspiration yours provided!
Self advertise
@МЕМЕ.ехе not really cuz he was mentioned in the video anyways...
@@hadamitMC yes, and he is saying this video's contraption is better, so it's kinda the opposite of self-advertisement.
@@thetheodorusrex9428 still self prmoting
Best channel ever. If you ever do a second channel doing deeper dives on the engineering I would totally watch.
Noted! Thanks a ton
@@NicholasRehm My thoughts exactly. I am a undergrad engineer just moving into the first engineering job in motion control and really love your efficient approach to control. Would be very eager to find out just how much math you do in the process.. :o
I usually fast forward sponsor message, but you made it relevant, straight to point, and includes progress bar that gets me invested in. Nice!
Me and @PCBWay appreciate that!
I was convinced that the thumbnail was clickbait and the actual contraption would be an abomonation but I'm super impressed with the outcome! Really nice work!!
No clickbait on this channel
@@NicholasRehm Yipee! What a breath of fresh air.
Optical flow sensor also picks up your angular movements. You need to estimate flow created from tilting (via gyro and lidar) and substract it from from flow sensor reading. While doing so, beware of different latencies.
This was my first thought when I saw what the sensor is doing. Any pitch or roll will give velocity from the optical flow even if the thing isn't really moving anywhere. Not something I would want to try and figure out...
I'm sure he's aware of that
@@theaveragepro1749 It didn't seem like it from the video. Like, I'm sure he's smart enough to know it, but even smart people make dumb mistakes sometimes.
This is an amazing project considering it was put together by a single person, the integration here is fantastic, this dude is a real polymath. It would be a huge software task but I'm sure Nicholas thought about the next level of using the phone camera, sensors, processor themselves to aid in stabilization, this rig literally carries around a camera and processor capable of using ML to stabilize on an object detected target.
Yes. I thought he was going to make an app on the phone to run it.
Thanks for the kind words. Phone processor integration I guess would make sense, but app development and proper software dev is where my expertise falls off lol
Could it be an issue, that the OS on the phone is not realtime? Control logic tends to struggle with jitter in the timing of sensor readings.
@@MarcoTedaldi that is certainly a concern
@@NicholasRehm I am a proper software dev w/ drone experience. Let's chat?
Love it! And since a selfie only takes a few seconds, 4 minutes flight time is enough for a dozen selfies.
😂😂😂
And who wouldn't want dozens of selfifes?
Worth it for the geek-out factor alone. The fact it does not work “well” is over-awed by the fact that it works “at all”. Kudos!
Crazy. I thought about this about 8 years ago. Im glad someone else has executed this idea.
Incredible work and while I would agree that this is not a viable product (a DJI Mini is probably just as good and offers more stability for the relative small cost of having a remote), the learnings are great and now I want to play around with you flight controller even more :)
Mission accomplished! Thanks for the kind words
not a viable product *yet - pretty sure at this point it's a race to the bottom with high integration, just look at all the cheap quad toys you can get for like 30 bucks
@@graealex yeah, probably
Just wait until wireless power transmission gets better.
Battery pack in my backpack, and my phone will just always follow me. Won't have to actually hold it. Can't be dropped. With voice control.
Would be quite cool.
You're doing something right. You have been in my feeds and my personal suggestions irl for a couple of years without a subscription. Keep being awesome!
Hopefully there's a subscription now :)
Brilliant project to tackle and demonstrate as a flying prototype! You solved so many aspects to making the flying DroneCase a usable product. This is many levels beyond using an extra long extended selfie stick. Now feeling the need to build a DroneCame to experiment and explore, as way more fun!
The one challenge I see remaining is adjusting the optical flow for when the DroneCase is tilted significantly from vertical as it try's to maintain position hold (6:00). ie: differentiate between horizontal motion and secondary tilt of DroneCam related to position hold (particularly in a dynamic environment with variable wind).
While lidar distance to ground is easy to compensate as tilt angle is known; the optical flow device would see massive motion as DroneCase tilts more than a minimal angle. At 6:24 is an example were optical flow would be swamped (or lost) by large path differences relative to the ground below.
If optical flow had 2-axis gimbal, then a 'normal' reference to the ground target could be maintained; at the cost of added complexity (and weight).
Alternatively, think it would be easier for DroneCame to maintained a constant radial distance to the subject. If (camera subject of focus) had a remote dongle, then DroneCase could use a bluetooth signal strength to get "range" data. This could help with when adjusting to larger position deviations (associated with larger tilt angles), and optical flow could fine tune position when tilt angles smaller.
As a bonus, if the remote dongle had a compass, then when the subject turned, or moved the DroneCase could move in an arc at constant radius to the updated subject orientation. Obstacle detection would then be required.
Again, is brilliant how identified core features and avoided the spiral of feature creep. ;)
Well, as a UAV manufacturer I consider your design preaty well made, yet it of course can be improoved upon. The biggest problem you have encountered is generally the problematic nature of PX4Flow. I've been there. As of now VOXL Cam seems to be a much better soultion as it includes stere camera sensors which, at least when used on Ardupilot, seem to be dooing a great job. Also - usually the LIDARs used in integrated solution have very limited range and are prone to feeding the flight controller flase data. Here even a lightweight and low cost sensors like TF-LUNA seem to be doing a stellar job.
I also might have ideas about further testing and even improoving your cyclorotor design and would be happy to cooperate with you. If you are interested - please, reply to this comment.
All in all - great job.
hm.. VOXL Cam looks like it is really great and would indeed offer an improvement but... have you considered that it is also almost 10x the price at about 1500$? 😅 which is just ridiculous for a small drone-phone case! PX4Flow is
maybe make the phone use its GPS for more fine tuned position + another IMU set
also one can potentiality use BT to send data to the FC from the phone
@@DiverseGreen-Anon maybe for you. Some of us care about the price, some of us don't.
As an aspiring control engineer, I gotta say I absolutely loved this video!! Excellent job :)) you've inspired me to get back to work on my own GNC project! Quick side note, your custom flight controller is actually what got me into my current GNC project, which is basically just an extremely watered down auto pilot for model gliders haha
The people on Reddit at r/ControlTheory would probably also be interested in seeing your project.
Awesome!!!
Underrated video - deserves way more views! Well done!
Thanks Jayden!
No greater way to make money than appealing to human narcissism. I bet within five years the selfie drones will be everywhere.
This dude understands jokes
@@NicholasRehm So how do you bring drone tech to the other major thing that sells without getting demonetized by youtube :)
selfie drones already exist, though they don't usually take this particular form
no they wont
Never that deep😂
Hell yeah you still have an oldschool DX7. Such a good looking radio 👍
Best purchase of my life…. Back in 2009 lol
@@NicholasRehm pretty cool that it can still hold up today with all the new advancements in flight stabilization
Oh man! This is epically awesome man. I love drones, I love 3dprinters, don't mind soldering, but don't have the brains to put it all together into anything; this is right down the lines of something I want to make! Thank you for putting this out there with all the code and steps you took. I am going to snag it for one of my next projects, I am super excited to take selfies with this at the next family Christmas get-together! Mind blown
Would be better to use the phone processing power.
@@MichaelRyannz One of the things making the flight controller work so nicely should be reliable execution times of each command to the rotors. Just imagine your drone crashing because Twitter was refreshed in the background, keeping the CPU busy at just the wrong time. I don’t think your phone allows you to run real-time code, does it? That said, using the camera image itself to measure optical flow and stabilise the position via an app on the phone sounds like a really nice idea, if there was a way to do that.
@@sebidotorg Optical flow boards use dedicated circuits to process the imagery. The phone just won't have the processing power to do it at a decent rate with a low enough latency.
The px4flow is just a really old board. If you want to spend the money the herflow is a similar package that integrates the lidar onto the board and weighs 1.2 grams compared to the px4flows 30 grams. Runs at 50hz. Probably save 60-80 grams vs this prototype design.
If you really want good performance you get a machine vision camera with a 80hz-400hz global shutter sensor and have that do the flow tracking for ya, but that adds all the weight savings back on.
@@nocare That’s what I thought. While it would be nice to use the actual image you want stabilised as the source for the movement information, I didn‘t think the image processing on a phone would ever reach a latency that‘s low enough for such an application. Which refutes the comment that it would be better to use the phone‘s processing power. Thanks for your input!
@@sebidotorg No problem.
Also if you want to have a target you place on the ground to track then you can do what you wanted using the machine vision cameras. Such as the OpenMV camera and others like it.
Aruco tags and April tags are examples of fiducial markers.
From just 1 tag you can get the position and orientation of the camera in 3d space relative to the tag.
Conversely you also have the tags 3d pose as well.
I wish I could subscribe 3 times. I don’t want a selfie drone, but just found your channel. Holy cow, you are so intelligent and creative. Your videos are amazing to watch.
Truly master class material.
That's so nice of you, thanks!
And I am off to binge all your videos now. Love finding a new/good channel all about stuff I love. This was a great vid!
Always amazed at your project! That's really cool! 🙌
Absolutely awesome. Great work Nicholas 👍
Thank you!
That is amazing. What gets me is the fact that the scammer had more people watch his video. Then the both of you put together. I think you are build is amazing
This is so cool! I learned so much more about drones!
Nice! Buddy of mine just bought a quad that floats rock solid indoors. I had no idea those things could be so stable!
Neat build!
for being very simple, this works surprisingly very well.
2:30 I have the same solder station/power supply as you. I love mine and is a major quality of life upgrade from the ones that plug directly into the wall.
Agreed, made the investment last month and it’s really improved my workflow!
That was amazing! Cheers from Japan!
I’ve been building one with your software for the past few months!! I didn’t know you were building one too! I’m using mark robers design but this one is more practical. I’ll post on my channel once complete!
Oh that’s awesome! Looking forward to it
@@NicholasRehm i uploaded a quick video on my channel to show the progress, its still gonna take a while for it to be completed, the design isn't perfect but i really wanted to go with the original peter sripol design.
Gosh darn it you TRICKED ME INTO LEARNING !!! IT'S UNFORGIVABLEEEEEEEEEEEE !!!!
😎
Well done young man,Incredible work an amazing project, I wish I Had someone like you on my team 👍👍👍
You're a rockstar at this stuff, very well done and very creative
If anything complicated looks simple, there is a very good chance THOUSANDS of hours of work went it to it. As you say it's not just as simple as "slappin a GPS on it". Respect!
This should go viral.
This is a really cool project and fun video - crazy amount of engineering went into this little project, I am really impressed.
Thanks for the kind words!
Wouldnt the tilt of the phone rocking around add to the misreading of the velocity sensor? Because the phone dosnt stay in a fixed vertical position it rocks back and fourth, even though the drone may be stationary above the ground as the phone wobbles and rocks it will give it false readings. Mabye some vector math between a gyro sensor and your velocity sensor could correct for the rocking angle of the drone?
PX4Flow does this compensation onboard, but I think mine's gyro went bad and it wasn't doing compensation anymore. I tried to do this my self in the flight controller, but it can only work so well...
Great project! I am not sure how powerful is the flight controller, but you can use LQR or even reinforcement learning based approach to control the motors if you decide to implement something more complex, such as following target
absolutly awesome. what a great project!!! You proved that "myth" possible like the real "myth busters". incredible!!!
thanks for commenting on my video:) (this is summit gaming you commented a few months ago)
Great, I love it !
One thing you could do would be to use your phone's camera with a self made app to do face tracking and compute your phone position relative to your face but also relative speed, then, the app could send position adjustments requirement to your drone (through USB-C for Androids and Lightning for iPhones), as smartphone have more than enough computing power on their own to do that.
Your phone would send position request, drift speed from the target, and your phone already get drift speed from stationary and altitude.
One extra step could be to apply targeting from your position holding camera, making it able to not just get the relative speed from the gound, but also, to try and get back to the same initial point and hold it, making it much more stable than only reading horizontal velocity, as you now have a precise point to follow, only downside is that it wouldn't work in a totally uniform ground by lack of easily recognisable details (where speed can still be established) or a too complexe one (where it would looks more like noise than specific single target to hold onto).
The only downside of using a specific app, is how the phone won't be able to take 100% of your camera's capabilities as only the native phone app is made for it, so you will have, at best, what other apps (Instagram, Snapchat, etc) can do, meaning, the default camera feature, so, no zoom, macro, portrait, etc.
Though, you could try to run your own app in the background and order your phone to take picture with the main phone app through ADB commands.
Awesome idea! Thanks for sharing! You could also use something like a Ram Mount design with 4x sprung arms to make it phone-agnostic (so you don't need to print fixtures for each phone)
Awesome! I wish I had the money and talent to do projects like this that I see on RUclips. 👏👌Keep it up!
Thanks!!
Very impressive and frightening at the same time.
As an engineer struggling with ERA/OKID for the past month, the performance you get without system identification by directly tuning the gains is mind blowing! You must understand the pole positions really well to tune gains so well!
Would it be possible to run this off the phone? And all the control of the cam-copter could be done via a phone app(?) It would slim down the amount of components required, possibly by quite a bit.
You could possibly access the selfie camera on the phone in place of the image stabilizer module you have? And the phone should have fairly accurate height data as well(?) Possibly removing the need for a lidar sensor.
Possible maybe, but not by me😅
Now I don't have to get off my couch to get my phone 3 ft away from me. I'll just be like: "Hey siri, fly over here".
Did you consider the possibility that the optical flow sensor gets disturbed by sligt rotations of the drone? Maybe that's what's causing the instabillities rather than the low quality fo the sensor.
my man just embarrassed mark rober in a few months 😂
Incredible!
I'm going to make it following your instruction!
Real neat! It seems like bi-copters are always on the wobbly side, but with prop guards they have a great cool-factor.
Could the data weirdness from the flow sensor be due to what it's actually seeing? Typically they work best with a sharp contrast pattern and yours is a long way away from what it's tracking (the ground). Also one thing I just thought of is that with the sensor hanging off the bottom, it has a pendulum effect to it's vision cone, too - Every time the platform attempts to stabilize, the sensor sees a shift due to pendulum.
PX4Flow has angular rate compensation built in but I think the problem with mine was that that wasn't quite working...maybe bad gyro
This turned out awesome!
Is there a reason why you didn't use the phone's sensors to help with location stabilization? Other than the difficulty in integration of course, my question is more of "would it be doable and good enough"?
Mostly just the integration complexity! But also non-realtime OS is a pain too
and then dronecase decides to join flock of geese going south...
lol
These projects are so impressive, you should make a drone selling company.
good job lad! keep at it!
Did you in any way subtract the gyro detected tilt angle (from your control inputs) from the optical flow sensor output? I could see that seriously impacting stability by contaminating your sensor input.
dude your work is oustanding. (in the field). no really this is impressive
Thanks for the kind words
This is a good idea, and if someone has not contacted you about producing it commercially, then they should. At its simplest, it should stay at a level you set when starting it, and it should follow a small transmitter you keep in your pocket. A price under $100 would make this a killer. Good idea.
You are building Arial robots (if we can call them as such) like how James Bruton builds land robots.. The balancing you achieved is amazing..
Thanks for the kinds words
Genius 😍
Knowing my luck with code implementation, I'd end up with a flyaway phone and no means to call it in! Great work!
Great video as always! Alternatively you could try RTK GPS for outside stability control. Something like NEO-M8P would give you 1-2 cm precision in all coordinates.
This seems to fall into "slapping a GPS on it" territory lol
@@NicholasRehm It totally does, but it's a nice GPS though....
Or just use the GPS built into the phone. Sure it wouldn't be as accurate or have the same update frequency as a dedicated GPS module, but it's already there. Heck, get IMU data while you're at it.
Nice, you always amaze me! Nice upload! Thanks.
Thanks for tricking me into learning something :)
Mission accomplished
"some scammer dude" very well said hahaha
Watching these kind of videos make me think If I would as good in my speciality, btw, this was a really good project.
I would love to see you solve the optical flow quality problem. I am yet to see an implementation of optical flow as a means of RF hardening a drone. If you could improve the optical flow to the point of being a reliable failover for the GPS it would be cool to see an autonomous drone that can't be brought down by the DroneGun!
Check out my video on motion planning. Completely gps/RF free autonomy
@@NicholasRehm I think that was the video I found your channel through.
I just didn't really see that as rf hardened and capable of long optical flow missions because of the limited test environment it was demonstrated in.
I've watched the videos from the other guys. They were not interesting and imo failed.
I am glad that you have done such an amazing job. :) Good work.
One thing i have in mid about it is - ur propelers have too much space to the ducts (pipes around them). This looses some performance. The less the space between the blade and the pipe , the more efficient it gets. You can try cutting and sanding the props and make them kinda "bull nose". Then make the space as little as you can. For a bonus point make the upper part of the ducts curved outside like 180 degrees or smth. This makes it even more efficient FOR LIFT.
Making those changes could help you a little bit with stability but should help you with battery life too. Oh do not forget to balance your props (even new they are not). Use magnetic balancer as it is most accurate. Hope this gives you some ideas :)
Agreed--ducts were mainly for protecting my hands. The first set were pretty tight on tolerance, but as you saw that led to some problems with resonance and prop strikes
I think fusing the IMU and the optical flow measurements with some Kalman filtering could make it much more stable.
That falls into "filtering tricks" category ;)
Nice job making it real. I love the one button simplicity and the point to point soldering. Most of my robots are point to point soldering. LOL
Thanks! Cool projects you got there
@@NicholasRehm Thank you.
Amazing project!! Thank you for making such a nice video.
Thanks 4 a lot of interresting videos! 👍😊 Is it possible to use the sensors in the smartphone? I imagine an app that... 1: is an interface with the user 2: sends flight data to the drone - I guess you have all sensors you need in a common snartphone - If they are good/quick enough...
You are a clever guy Nick, well done mate.
Great video! Can you go into more detail on the coding side of things? I'm curious about the PID control algorithm and I want to do something similar to measure flow rates of liquids for a project.
I definitely want to do a deep dive video on PID control in the future
This video deserves million of views
You're very kind😄
For the velocity control are you using the raw optical flow reading or are you also compensating for the tilt and roll of the housing. Namely, if the housing tilting back and forth from vertical, then the optical flow sensor will also read an oscillating velocity. When assuming that the altitude stays constant and the ground is horizontal, then one should be able to subtract this induced velocity (assuming the attitude is estimated accurate enough). A dirty solution could also be to filter the optical flow sensor with a low pass filter and/or notch away the typical attitude oscillations.
PX4FLow does angular rate compensation onboard but I think mine's gyro was broken with age. I tried doing this myself manually, but it could only work so well... Lots of "filtering tricks" were tried
Bless you. You are a special human being. All good things to you.
This is awesome. Amazing job. But honestly when you took it outside I was rooting for a fly away to happen lol
That woulda been funny
This is a winner.
And you're NOT monetizing any of this? Open source FTW you rule.
Thanks!!
Nick, I'm a first timer..... love the science. I subscribed and will be watching more! Thank you!!! Mark
Welcome aboard Mark!!
This is brilliant, will order the parts immediately! You could also use the phone's sensors + AI to do the high level control and send that to the controller
I wonder how well it would work with a big mat under it with various unique tracking points? Or maybe some small 3d printed shapes you can stake or set onto ground under it?
It liked my textured rug pretty well, and the grass has plenty of features to track
This is freakin' awesome! Great work! Super entertaining and educational!
Thanks!
Awesome idea and build with explanation... Couldn't a portable battery bank go inside the magnetic box this allow quick swapping and if just use a usb cord use any power bank then
This is really good. Imagine after pressing the red button, a countdown to launch is displayed on the phone screen.
That would be really cool
needs toroidal props for less noise, and maybe some sort of marker for a ground reference to focus on and hover above
Super neat! Might also be really cool with something like an Insta360. Sounds like all you need is a bigger battery to get that runtime higher.
Great idea
This is really cool. Great work! I know that with a DJI Mini SE being just over $200 now, it doesn't seem like this could be a viable product, but with a few minor improvements and the right marketing it could be.
If you could get the battery life up to 10-12 minutes, and could maybe give it say 3 height settings and 3 distance settings with a lever to automatically adjust for those 9 positions, and a timer to hold those 8 further positions (maybe with 3 settings as well), I'm confident I could market something like this for $150-$180 easily. It's all about finding the right angle to convince someone why they need something, and having a novel and unique presentation like this helps distract consumers from the fact that there may be other tools to get the job done more effectively.
You have all the capabilities of what snapchat has with their drone but yours is much much better!
Cool video. Hopefully you get more views than the scammers. You deserve it.
Does the position controller use the accelerometer too? Your estimated velocity would drift over time but combined with the optical flow it would probably give you better results. Amazing project as always, I hope your channel really takes off!
Also, I'd be interested to see someone do this with the phone's onboard IMU and perhaps using CV from the selfie camera to remain pointed at the user.
Yea I did a little playing with fusing in accelerometer to improve performance
Fascinating! How does the optical flow sensor distinguish between the movement of the drone along the XZ plane and the panning of the frame caused by the body tilting? Does the sensor have some sort of rotational stabilisation in it?
It has onboard gyro and sonar sensor to resolve induced velocities from rotation
@@NicholasRehm Very cool, even if it doesn't work very well!
dang. i was thinking googles project ara's modular phone desing with its hotwappable i^2c power railing would be a great way to fly 4 props with a phone as a flight board but hey this works too!
ever consider a monocular slam video?
Forget phone drones, let's scale this up just a tittle and make models of sweet bi-copters from Avatar!!!
Always wanted to build one of those!
Nice job! Thanks for teaching me something!
Really cool project! The phone seems to be kind of shakey tho, so I wonder if this gives false movement data to the optical flow sensor.
i just watched 3 videos each recommended another creator this is a good place
You beat me too it XD dang it, now i have to add features to be more unique, Great job on the work man! you really did a decent job keep up the good work
Awesome I’ve watched your video twice. I have a suggestion remember when you made the ducts reinforced?
After reinforcing the ducts it fully way better. In my experience vibrations can turn into a resident frequency and wreak havoc on your flight controller software. With the improvement you had when you strengthen the ducts, I believe further strengthening or purchasing a already made duct would work better if you could just make one that was stronger I really believe it would help with the flight characteristics .
Awesome work, I will suggest to use acceleration sensor for drift control.