OpenBot: Turning Smartphones into Robots
HTML-код
- Опубликовано: 6 авг 2024
- Current robots are either expensive or make significant compromises on sensory richness, computational power, and communication capabilities. We propose to leverage smartphones to equip robots with extensive sensor suites, powerful computational abilities, state-of-the-art communication channels, and access to a thriving software ecosystem. We design a small electric vehicle that costs $50 and serves as a robot body for standard Android smartphones. We develop a software stack that allows smartphones to use this body for mobile operation and demonstrate that the system is sufficiently powerful to support advanced robotics workloads such as person following and real-time autonomous navigation in unstructured environments. Controlled experiments demonstrate that the presented approach is robust across different smartphones and robot bodies.
Paper: arxiv.org/abs/2008.10631
Get started at www.openbot.org 
Наука
This is huge. Instead of everyone throwing out their phones we can reuse them. It is such a smart idea, truly incredible.
Thank you, guys! Love this work!
This project is very interesting. Congratulations!
It's interesting. It makes sense to use mobiles for small robots like the one in video. Congratulations.
Very cool idea! Why reinvent the wheel if modern smartphones are already packed with all the hardware you need?
Great work!
That indicating left and right was cute.
very inspiring project..... Kereeeen 👍👍
Amazing!
Love this. Want to build it.
wow，thanks for sharing
Amazing
Such a cool and interesting project! Very clever to use the smartphone. Did you test the difference in performance when using the smartphones wide-angle or the telephoto lenses?
No we did not, but this would be interesting. With a wide-angle lens navigation should be much easier. But since only recent phones have a wide angle lens, we opted to use the main camera sensor. In the code you can change the camera selection.
Does MCU(Arudino Nano) need to perform calculation? Is it okay if you use other single-chip computers(such as Arudino Uno)?
You can use any other MCU. It just needs the following features: serial communication (communicate with the smartphone), 4 PWM outputs (control the motors), I/O pins for battery monitoring, speed sensors, ultrasonic sensor (optional), indicator LEDs (optional)
Cool.
What cable is used to connect the phone to the Arduino?
Is this mapping and recording home to android phone? Or make robot vacuum cleaner?
I could see ultrasonic sensor @0:11 in the video but it's not mentioned in the components!
Is it required or it can work as in the video with the camera as input only?
As you said, we only use the camera feed as input to a neural network. The ultrasonic sensor is optional and not used in this work. It's mainly for looks of the robot. It can be used for simple obstacle avoidance or collision detection. I will add it to the Github repo. They are very cheap, you can buy them for a few $.
@@matthiasmuller7898 Thank you. Its a great effort will be loving forward to try it once the app is online
Is there a step-by-step guide for those who aren't so tech savvy? Is the custom PCB board in place of the Arduino and other components?
The PCB acts as carrier board for an Arduino Nano and replaces the motor drivers. When using the PCB, you just plug in a Arduino Nano, connect the motors and sensor and you're done.
@@matthiasmuller7898 Will you post a slowed down version of the video which starts at 1:05?
@@matthiasmuller7898 Interesting, this wasn't clear to me... are you saying with PCB we do NOT need the L298N? I bought the "4 wheels plus L298N" from amazon and was trying to figure out how to fit the pcb and l298N in the mount holes on the printed base.
@@smittyplusplus yes. Either L298N or the PCB. The PCB has integrated motor drivers. These are more efficient and the car will go faster as a results. However, if you are just building one robot, using the L298N will be cheaper and simpler.
@@xpxdavex Yes. I'm planning post short videos for the individual steps. But I need to do some editing and re-record some parts.
Fantastic project well done and thanks for making it open source.
When and where can we order the custom PCB?
Coming from the iOS world, would a Samsung GT-i8150 be up to it?
I will add the PCB, BOM and instructions how to order it to the repo tomorrow. As far as I know the Samsung GT-i8150 is 7 or 8 years old. A lot of things have changed in Android since then. My guess would be that it probably will not work or at least not with full functionality. But any phone from the past 2-3 years should work. Feel free to test it; I'd be curious to know myself.
The PCB instructions are now available on Github.
Thanks, I think I’ll try with a more modern phone, but I will definitely give the GT-i8150 a try and let you know
Just buy an android. Is it that hard?
Nice Job! I Like the Idea of using a Smartphone as the brain of the robot. Is it possible to add also the STEP-files for the phone Mount?
Added to the Github repo.
@@matthiasmuller7898 thanks a lot! Looking forward for Updates of your Project. IT IS definately one to look at. Printing in ihr ultimaker is nearly finished
How to be part of the community to get the design
You can find all code and instructions on the Github repo. It is open to all.
Is there a parts list for this project?
Yes, please check the Github repo for BOM and instructions.
Very cool project of 3D printers is already running :-)
So far I always connected Android and Arduino with Bluetooth using the MIT AppInventor app.
What kind of cable do you need? Can you build the cable yourself?
Does the smartphone have to be OTG-capable or is it possible with anybody?
Thanks for the great project, I'm looking forward to more.
The smartphone needs to be OTG capable. I'm not sure about building the cable manually, but I don't see any reason why not. Using an OTG adapter should also work. In principle communication via Bluetooth would also be possible, but we did not implement it. A wired connection is more reliable.
@@matthiasmuller7898 Danke für die Info und Grüsse aus der Schweiz :-)
@@bastelgarage-ch Gerne.
@@matthiasmuller7898 Incorrect, you can also use the audio jack. The Android AntBot we have been selling since 2011 features an audio-serial option. This has allowed us to use budget Android phones starting from 2.x Let me know if you want more information, as our design has also been open source the whole time :)
I am trying to build it, but I couldn't find more instructions and files about how to install and use the "person following" coco dataset as shown on the video, nor the policy files that were shown (to at least start with something), can someone direct me please?
If you install the app, you can download several object detectors and use them. Feel free to join the Slack channel for further discussion and help.
Really great project! a) Please release the APK on the PlayStore. Unexperienced users will need to spend lot of hours to get it finally working b) Is there any shop who offers electronics kit with all required components except 3d printed parts? c) How to control the vehicle. Does OpenBot provide Html webinterface which can be access for control and remote video via browser?
a) We are working on it, but it requires some internal processes which take time. b) You can buy some Arduino robot kits that have most of the components but not all. www.banggood.com/de/Educational-DIY-Smart-Robot-Car-Kit-Gesture-Control-Car-Hand-Gesture-Car-For-Arduino-p-1732917.html We expect that some online shops will offer such kits at some point. c) For now, it does not provide such an interface. Control commands are issued from the phone to the microcontroller directly. There is no web interface to the phone yet. So there are two ways to control the robot. 1) Connect a bluetooth controller to the phone (e.g. PS4, XBox). 2) Run an algorithm on the phone to drive the car autonomously. There is already a neural network preinstalled that allows you to let the car follow a person. This can be turned on/off in the app or via a game controller (R1 button).
makeblock is about the same thing.
Hello, we have been selling these since 2011. I hope that you had no issues finding the open source designs and schematics on our Wiki. Welcome to the conversation :) www.robots-everywhere.com/
I wonder if the battery in the phone could actually power the wheels. That would simplify the hardware.
This is possible, but would make the car very slow and may also not work on many phones.
1) A voltage of only 5V means low rpm for the motors
2) Most phones cannot supply power and if they can only small currents (a better alternative would be a power bank, but it still has issue 1)
3) The phone would get very hot
@@matthiasmuller7898 Ok, thanks. But also bear in mind that the external batteries weigh a lot and can be subtracted from the weight of the vehicle.
@@TheNameOfJesus Yes, but to really reap the full benefits, the car may have to be redesigned. If you would like to explore this directions further, I would be very interested in the results.
@@matthiasmuller7898 True, but some smart phones support "REVERSE inductive charging" through their wireless charging system and that means a totally different limit on wattage. For example the Galaxy S20 has 9W *reverse* wireless charging. That's 80% more than the 5W you mentioned. I do own the Lego robot kit but I don't have a well thought out plan for this idea so I'm not going to use it yet.
@Matthias Müller Hi. I want to make this as my final year project, but the only thing I'm finding difficult for now is the driving policy. If possible can you help me with it when I'm done with the hardware stuff? Thank you
Yes. If you struggle with the instructions, you can open an issue on Github.
Does it work with iPhone
I would like to make with my kids. Is there a step by step video guide?
Not yet. But we are planning to release such a video in the near future.
Thanks my kids are super excited to build this. I am as well, appreciate you open sourcing it!
We sell kits at www.robots-everywhere.com/ and we have been doing so since 2011 :)
In person following, what if there are multiple people on the road? Won't the robot start to follow other people if the algorithm assigns more confidence to other people?
Yes, exactly. We have also observed this behaviour in our experiments. If you want to track a specific person and there will be other persons close by, you will need to integrate a tracking algorithm as well.
This is interesting, is there an already compiled app for Openbot on Android Store?
No, at this point you have to clone the repo and compile the application. However, it is pretty straight forward.
@@matthiasmuller7898 Alright, thank you!
@@matthiasmuller7898 I was able to install Openbot on my Android phone, but it uses front-facing camera instead of the back camera? Is there a way to flip it? Thanks!
@@jimmykan7873 What phone are you using? You can just update the cameraSelection variable.
github.com/intel-isl/OpenBot/issues/15
@@matthiasmuller7898 I am using Moto G5 Plus. will check the setting, Thank you!
I'm gonna build that robot
THIS Will change the world! In 5 years, all college/universities will use this!!
We've been selling kits since 2011 if you are interested.
@@RobotsEverywhereVideos heya need your help in acquiring kits compatible with Openbot
Does the phone just connect to the arduino ?
Yes.
Love this! Is there an IOS version release soon?
Probably not any time soon since it requires a fair amount of work and is not our top priority right now.
Haha, as if...
Apple will gladly sell you an iRobot for $999 in a few years, but then you will have to buy the wheels separately...
Then it will roam around and spy on you all day, phoning home the data to the big iCloud....
They will also claim they invented the mobile-phone robot
Just buy an android, iOS will die soon
And when will the android app be available?
Thank you
Piggybacking off this comment, the Android app is listed as "coming soon" in the Github repo linked from their site: github.com/intel-isl/OpenBot
The code is currently still in our internal open-source review process. Hopefully in 1-2 weeks or by the time you have finished building your OpenBot.
Firmware and Android App are on Github now.
@@matthiasmuller7898 Thank you.
I will take a look at it
it will work with iPhone?
Unfortunately, it only works with Android phones right now. However, the Android Object Detection Demo for iOS would be a good starting point to develop something similar for iOS.
Why is it needed?
Why not?
@@matthiasmuller7898 really)
can we just buy it already made?
Hopefully soon.
PCB 제작비 최소수량이 20개 20만원 가량 되네요.. 공구하면 좋겠는데 ㅠ
I'm not sure if I understand correctly. Could you please comment in English.
This is not a ripple that wants an answer. This article that I am looking for a member to follow this project in the same country as me. I want to share the PCB I ordered.
Yeah but would a Wookie scare it?
Probably not, especially not the Yoda version. ruclips.net/video/PEj8jWapGt4/видео.html
0:49 *only* fifty bucks? Seems expensive for a RC car. Why not start there and save some bucks.
You can also retrofit an RC car with the OpenBot system. Note that the OpenBot design already includes the microcontroller and additional low-level vehicle sensors.
Aqui no Brasil, ela ficaria sem o celular e sem o robô.
Hopefully cheap Chinese knockoffs are available at Best Buy in 12 months.
what android version will it work?
We have tested it on 10 different smartphones from the last two years and it worked on all of them out of the box.
@@matthiasmuller7898 is control via usb or BT ??
@@jamespruett27 The connection between the Arduino and phone is USB. The controls are generated via a controller or AI.