Vision based Navigation for Micro Aerial Vehicles (MAVs)

We design smart computer-vision algorithms that allow small flying robots to fly all by themselves, without any user intervention. Our flying robots only use onboard cameras and inertial sensors to "see" the world and "orientate" themselves. No GPS, no laser, no external positioning systems (e.g., Vicon), no CGI are used in these videos. Our driving motivations are search-and-rescue and remote-inspection scenarios. For example, quadrotors have the great potential to enter and rapidly explore damaged buildings in search of survivors after a catastrophe. Alternatively, they can be used in all remote-inspection operations that fall beyond the line of sight with the operator. For all these operations, it is crucial that they can navigate all by themselves.

To reach a given destination safely and accurately, a micro aerial vehicle needs to be able to avoid obstacles and minimize its state estimation uncertainty at the same time. To achieve this goal, we propose a perception-aware receding horizon approach. In our method, a single forward- looking camera is used for state estimation and mapping. Using the information from the monocular state estimation and mapping system, we generate a library of candidate trajectories and evaluate them in terms of perception quality, collision probability, and distance to the goal. The best trajectory to execute is then selected as the one that maximizes a reward function based on these three metrics. To the best of our knowledge, this is the first work that integrates active vision within a receding horizon navigation framework for a goal reaching task. We demonstrate by simulation and real-world experiments on an actual quadrotor that our active approach leads to improved state estimation accuracy in a goal-reaching task when compared to a purely-reactive navigation system, especially in difficult scenes (e.g., weak texture).

References

References

We present a quadrotor system capable of autonomously landing on a moving platform using only onboard sensing and computing. We rely on state-of-the-art computer vision algorithms, multi-sensor fusion for localization of the robot, detection and motion estimation of the moving platform, and path planning for fully autonomous navigation. Our system does not require any external infrastructure, such as motion capture systems. No prior information about the location of the moving landing target is needed. We validate our system in both synthetic and real-world experiments using low-cost and lightweight consumer hardware. To the best of our knowledge, this is the first demonstration of a fully autonomous quadrotor system capable of landing on a moving target, using only onboard sensing and computing, without relying on any external infrastructure.

Exploring and mapping previously unknown environments while avoiding collisions with obstacles is a fundamental task for autonomous robots. In scenarios where this needs to be done rapidly, multi-rotors are a good choice for the task, as they can cover ground at potentially very high velocities. Flying at high velocities, however, implies the ability to rapidly plan trajectories and to react to new information quickly. In this work, we propose an extension to classical frontier-based exploration that facilitates exploration at high speeds. The extension consists of a reactive mode in which the multi-rotor rapidly selects a goal frontier from its field of view. The goal frontier is selected in a way that minimizes the change in velocity necessary to reach it. While this approach can increase the total path length, it significantly reduces the exploration time, since the multi-rotor can fly at consistently higher speeds.

In the left video, we demonstrate our flying 3D scanner. Our quadrotor flies autonomously over an area of interest using only its onboard sensors: namely, a single, down-looking camera and an inertial measurement unit. All flight-relevant computation is performed onboard, on a smartphone processor. No pilot, GPS, or motion-capture system is required. We stream a subset of the images down to a laptop computer, where we compute a dense 3D model of the environment. Our algorithms are so efficient that we can create these models in real-time. Therefore, our system has high potential for applications in which detailed information about an unknown environment is time critical, such as during search-and-rescue missions. In the right video, we demonstrate several capabilities of our vision-controlled quadrotors, such as autonomous way-point navigation, live, dense 3D reconstruction, and aerial-guided collaborative grasping with a ground robot. Our visual-odometry pipeline (SVO) runs in real-time onboard the quadrotor. The MAV navigates autonomously along predefined way-points. No GPS, nor teleoperation, no motion capture system were used throughout the flight. Our live, dense 3D reconstruction algorithm runs REMODE, our REgularized MOnocular DEnse reconstruction algorithm.

We demonstrate our vision-based micro aerial vehicle on the disaster training site of the Zurich firefighters. Our visual-odometry pipeline (SVO) runs in real-time onboard the quadrotor. The MAV navigates autonomously along predefined way-points. No GPS or teleoperation was used throughout the flight. The altitude in this experiment is 20m above ground.

In this video, our drone autonomously flyes over a person and takes 3D scan of it. Suddenly, the person starts to move, trying to disturb the drone. However, our robust visual odometry and probabilistic dense reconstruction algorithms do not get disturbed by the motion of the subject and manage to get an accurate 3D scan of it and the surrounding scene. The visual odometry, planning, and control algorithms run fully onboard the drone, on a smartphone processor.

In this video, we show a vision-based drone that can autonomously execute a given trajectory and provide a live, dense 3D map of an area. This map is presented to the operator while the quadrotor is mapping, so that there are no unnecessary delays in the mission. Our system does not rely on any external positioning system (e.g., GPS or motion capture systems) as sensing, computation, and control are performed fully onboard on a smartphone processor.

We investigate the following problem: given the image of a scene, what is the trajectory that a robot mounted camera should follow to allow optimal dense 3D reconstruction? The solution we propose is based on maximizing the information gain over a set of candidate trajectories. In order to estimate the information that we expect from a camera pose, we introduce a novel formulation of the measurement uncertainty that accounts for the scene appearance (i.e., texture in the reference view), the scene depth, and the vehicle pose. We successfully demonstrate our approach in the case of real time, monocular reconstruction from a micro aerial vehicle and validate the effectiveness of our solution in both synthetic and real experiments. To the best of our knowledge, this is the first work on active, monocular dense reconstruction, which chooses motion trajectories that minimize perceptual ambiguities inferred by the texture in the scene.

In this paper, we propose a resource-efficient system for real-time 3D terrain reconstruction and landing-spot detection for micro aerial vehicles. The system runs on an on-board smartphone processor and requires only the input of a single downlooking camera and an inertial measurement unit. We generate a two-dimensional elevation map that is probabilistic, of fixed size, and robot-centric, thus, always covering the area immediately underneath the robot. The elevation map is continuously updated at a rate of 1 Hz with depth maps that are triangulated from multiple views using recursive Bayesian estimation. To highlight the usefulness of the proposed mapping framework for autonomous navigation of micro aerial vehicles, we successfully demonstrate fully autonomous landing including landing-spot detection in real-world experiments.

We tackle the problem of globally localizing a camera-equipped micro aerial vehicle flying within urban environments for which a Google Street View image database exists. To avoid the caveats of current image-search algorithms in case of severe viewpoint changes between the query and the database images, we propose to generate virtual views of the scene, which exploit the air-ground geometry of the system. To limit the computational complexity of the algorithm, we rely on a histogram-voting scheme to select the best putative image correspondences. The proposed approach is tested on a 2km image dataset captured with a small quadroctopter flying in the streets of Zurich. experiments.

The scope of the sFly project was to develop a system of multiple vision-controlled micro aerial vehicles that are capable of autonomous navigation in GPS-denied environments, 3D mapping, and radio-beacons localization. The Sfly MAV's are fully autonomous. Flight control is done by computer vision and runs entirely onboard. No laser rangefinder or GPS are used, but just onboard cameras running at 30 Hz. An on-board monocular visual slam computes local camera poses and sparse features. These estimated poses are fused with the IMU measurements in a filter. An additional stereo system captures image data while the MAVs explore the environment. These image data are used to compute a detailed dense 3D model of the environment. An off-board visual SLAM module computes a global map by utilizing loop detection and bundle adjustment optimization. experiments.

sFly Project Website

References

This video shows automatic mapping during a controlled flight during a 200 m long flight on a village ear Zurich. The helicopter flies at more than 15 m. Concurrently to Visual SLAM, a 3D mesh is created out of the 3D points and textured is projected on to them. This 3D mesh mapping run in real-time during flight. Process happened off-board. Images were streamed to a remote 2GHz Dual-Core laptop. The algorithm run at 30 fps.

This video shows our participation at the European Micro Aerial Vehicle competition where we won the 2nd place and we were the only purely vision based autonomous helicopter. Besides being the only one using a single camera as sole sensor, we were also the the only autonomous helicopter among all the other participants! In this video the helicopter takes off from the circular spot, then uses Visual SLAM to build a map of the environment and navigate through via points specified by the user.

Only a single camera is used to control the helicopter. The blob is used to initialize the map and the scale factor at the beginning. Take-off is automatic. Feature points are tracked and used in a visual SLAM framework to build the map of the environment. The helicopter then follows autonomously the via-points created in this map and land automatically onto the second blob, which is used to show the precision of landing. The distance between the start and end position is about 3 meters. With this helicopter, we will partecipate in the EMAV (European Micro Aerial Vehicle) competition in September 2009 in Delft.

References