Видео

In this example it was a simulated RGB-D camera.

Yes on ROS. The drone is using open source PX4 Autopilot, with interface to ROS with mavros package. This video shows off the mapping part created by rtabmap_ros package (using libpointmatcher under the hood for scan matching) with an ouster os1 lidar.

@@matlabbe very nice. I wish I didn't wasn't bother that ROS has a big stack of bad design decisions (or ones I don't personally like), it would be more fun to work with. Glad it's working so well for so many, though

We did integrate with a planner called TAPE (see reference in description, I added a video link ruclips.net/video/nROO0BFK4lc/видео.html) that we tested on that drone.

​@@matlabbethanks I'll check it out.

Loop closures are detected visually. Enough discriminative visual features should be detected to close the loop. While scanning, if you come back to a previously scanned area, try looking at it with same point of view than the first traversal to maximize the chance to detect a loop closure. Once a loop closure can be detected, you should see "Loop closure detected" appearing, with background changing to green or yellow like at 0:49.

@@matlabbe I will try this tomorrow again I teach in interior design and this app looks very promising

Can you see texture when you open OBJ in meshlab? If so, maybe an option in meslab DAE exporter should be enabled to export with texture.

This is in simulation (Gazebo)

I thought it’s your house😅 but I guess you made a mesh from real building to gazebo model right?

@@martincerven yeah, it is a 3d scan from a real apartment that I took, imported to Gazebo

In this video it is SLAM, not localization. The db is built as the robot is moving. Yes, three rgb-d cameras (front, left, right) of the robot are used.

It looks like a great lidar for that price!

If you use RTAB-Map iOS app, you can directly export an OBJ file after the assembled mesh has been created. If the database contains a texture mesh (from iOS app for example), in rtabmap-databaseViewer you can do File->Export optimized mesh. If you have a db without mesh generated, you may generate one following instructions from here: github.com/introlab/rtabmap/wiki/Multi-Session-Mapping-with-RTAB-Map-Tango#7-export-the-mesh Afterwards, convert OBJ format to DAE with Meshlab. Once you have the DAE, you can create a model for gazebo. As example, the environment in this video is the "apt" model from this repo: github.com/matlabbe/rtabmap_drone_example/tree/master/models/apt

​@@matlabbe Thanks for your reply!! I will try later

The cameras seem upside down on the robot. The demo is specific to ROS1 (the rtabmap launch can be ported to ROS2 though). I am not sure there is an official ROS2 simulator for that robot (it looks not very convenient to use the controller right now on ROS2 ruclips.net/video/YSedTUxI0wc/видео.html)

The simulated textured environment was captured using RTAB-Map android app on a Google Tango phone. I don't actively support android app anymore, but I would refer you to RTAB-Map iOS instead ruclips.net/video/rVpIcrgD5c0/видео.html

I don't think so, we would need to port them to ROS2.

You can ask on the forum official-rtab-map-forum.206.s1.nabble.com/ or on github github.com/introlab/rtabmap_ros/issues

We can, following that comment to get localization between the robot and iPad/iPhone map.

@@matlabbe Thank you for the reply. I am trying to create a map by using ipad and use it as a map to localize my mobile manipulator(it has RGBD camera and 2D lidar). Could you please let me know if there are any useful references so that I can do this? Thank you.

There is no option to do that, would need to expose that functionality to UI.

@@mathieulabbe4889 thanks for the reply. There is another issue I’m having after I finish the LiDAR scan & try to build the mesh it disappears ? The only way I can bring it back is clicking the button close visualisation at the bottom. Is there an option I am missing in settings ? Such as making the scene size bigger ?

I scanned with Rtabmap for google tango, but it can now also be done woth rtabmap for iOS (should have iphone pro or ipad pro with LiDAR). You can then export in OBJ from the app. To convert to dae, I did it with Meshlab free softwars, open OBJ, then export as dae. Hope it will you getting nice textured environments in Gazebo!

​@@matlabbe thanks for the instructions. i am using samsung s22 ultra (not tango supported but arcore with deptAPI). rtabmap on play store reported tango core outdated and terminated. I guess I have to build from source.. Do you have the build from source instructions for ios/android app?

@@KensonLeung0 I don't provide the android app on play store for other android phones than Google Tango phones because it is too hard to support. I got a version on Huawei P30 Pro or Samsung Note10+ "kinda" working, having both a TOF camera, but since the TOF camera is not officially integrated in ARCore, I am not satisfied with the results and we have to re-code / re-test on every different phones, which is unsustainable (cannot buy every android phones...). It seems there is no TOF camera on S11 Ultra, I think Samsung stopped to put one in their new phones. It is why I recommend iOS now, Apple did a good job integrating the LiDAR camera, same API for all their phones. ARKit is also a lot closer to Google Tango, than ARCore is to Google Tango. If I remember well, I think Apple bought the company behind Google Tango back in the days... so ARKit is kinda Google Tango 2.0.

@@matlabbe i just looked into the android code and i can see the effort you've put on catching the individual android models. I agree that the ios one is much more consistent and sustainable. Thank you i will work on the ios one.

Only stereo images and IMU from the camera

@@matlabbe The same camera had both sensors? Was it your phone or a different device?

@@pablodavidarandarodriguez163 this was with a Mynteye camera

You can enable aruco/april tags detection in settings of the App. They will be used for loop closure detection, but won't be used to transform the map in some world coordinate frame that the tags would be in.

@@mathieulabbe4889 Thank you , but, when I want optimize with this settings the Rtabmap it closes, why ?. I use rtabmap standalone for Windows.

@@mathieulabbe4889 And finally, if I want use only markers for loop closure, is possible ?

@@meinotherside1 Yes, you can disable visual loop closure by setting Preferences->Vocabulary->Maximum words per image... to -1.

@@mathieulabbe4889 thank you for your answers. I will try.

Just tested again on ROS Noetic ubuntu 20.04 and it works as expected. Make sure you are using the right version of mavros/px4 like in the readme of the example.

I recorded it next to where I live.

The final map size is around 1GB (actually at 15:22 you can see 965 MB), but the raw point clouds have been decimated at 20 cm voxel in this example. Handheld LiDAR solutions already exist, though maybe too expensive for gaming. That could be possible to do a low cost version, see for example official-rtab-map-forum.206.s1.nabble.com/Kinect-For-Azure-L515-ICP-lighting-invariant-mapping-td7187.html

@@matlabbe interesting 🤔

get a VLP16.

@@KensonLeung0 Like Scanner sombre in real life.

@@Gfish17 that would be fun. How about getting a wearable glasses with the RTABMAP android app? (To ensure good quality… pick a compatible phone with ARCore and tof depth camera then mirror to the glasses)

RTAB-Map for Google Tango, but now available on iOS (with LiDAR required)

This is a scaled down version (0.6 the size) of default simulated drone from PX4 project (www.arducopter.co.uk/iris-quadcopter-uav.html).

This is RGB-D, a depth camera is required to see obstacles accurately. GPU only used for simulator and visualization. All navigation stuff are done on CPU. To use on less powerful computer, you may have to switch the visual odometry approach used by a lighter one (ideally a VIO approach).

@@matlabbe How do you think using a stereo camera in the place of RGBD effect the output? Great job by the way.

@@jonsundar With stereo, you cannot "see" white/textureless walls, which are often seen in indoor environments. As a workaround, tape a pattern on the walls at the height at which the drone will fly, then it will be able to avoid them!

@@matlabbe Thank you for answering that. I am just in the conception stage of a mobile application for building construction and infrastructure industry. Your tech will certainly be the future in construction applications👍

See github.com/introlab/rtabmap/wiki/Installation#ios

In this video it was an old Bumblebee2 stereo camera (yes the same name than the transformer)

You can check this paper for a comparison with standard datasets: introlab.3it.usherbrooke.ca/mediawiki-introlab/images/7/7a/Labbe18JFR_preprint.pdf