Student Projects

This project focuses on developing robust reinforcement learning controllers for agile drone navigation using adaptive curricula. Commonly, these controllers are trained with a static, pre-defined curriculum. The goal is to develop a dynamic, adaptive curriculum that evolves online based on the agents' performance to increase the robustness of the controllers.

When drones are operated in industrial environments, they are often flown in close proximity to large structures, such as bridges, buildings or ballast tanks. In those applications, the interactions of the induced flow produced by the drone’s propellers with the surrounding structures are significant and pose challenges to the stability and control of the vehicle. A common methodology to measure the airflow is particle image velocimetry (PIV). Here, smoke and small particles suspended in the surrounding air are tracked to estimate the flow field. In this project, we aim to leverage the high temporal resolution of event cameras to perform smoke-PIV, overcoming the main limitation of frame-based cameras in PIV setups. Applicants should have a strong background in machine learning and programming with Python/C++. Experience in fluid mechanics is beneficial but not a hard requirement.

Recent progress in drone racing enables end-to-end vision-based drone racing, directly from images to control commands without explicit state estimation. In this project, we address the challenge of unforeseen obstacles and changes to the racing environment. The goal is to develop a control policy that can race through a predefined track but is robust to minor track layout changes and gate placement changes. Additionally, the policy should avoid obstacles that are placed on the racetrack, mimicking real-world applications where unforeseen obstacles can be present at any time.

Drone racing is considered a proxy task for many real-world applications, including search and rescue missions. In such an application, doorframes, corridors, and other features of the environment could be used to as “gates” the drone needs to pass through. Relevant information on the layout could be extracted from a floor plan of the environment in which the drone is tasked to operate autonomously. To be able to train such navigation policies, the first step is to simulate the environment. This project aims to develop a simulation of environments that procedurally generate corridors and doors based on an input floor plan. We will compare model-based approaches (placing objects according to some heuristic/rules) with learning-based approaches, which directly generate the model based on the floorplan.

The goal of this project is to develop a shared embedding space for events and frames, enabling the training of a motor policy on simulated frames and deployment on real-world event data.

In this project, the student applies concepts from current advances in image generation to create artificial events from standard frames. Multiple state-of-the-art deep learning methods will be explored in the scope of this project.

Autonomous quadrotors are increasingly used in inspection tasks, where flight time is often limited by battery capacity. his project aims to explore and evaluate state-of-the-art path planning approaches that incorporate energy efficiency into trajectory optimization.

Recent advances in model-free Reinforcement Learning have shown superior performance in different complex tasks, such as the game of chess, quadrupedal locomotion, or even drone racing. Given a reward function, Reinforcement Learning is able to find the optimal policy through trial-and-error in simulation, that can directly be deployed in the real-world. In our lab, we have been able to outrace professional human pilots using model-free Reinforcement Learning trained solely in simulation.

Model-based reinforcement learning (MBRL) methods have greatly improved sample efficiency compared to model-free approaches. Nonetheless, the amount of samples and compute required to train these methods remains too large for real-world training of robot control policies. Ideally, we should be able to leverage expert data (collected by human or artificial agents) to bootstrap MBRL. The exact way to leverage such data is yet unclear and many options are available. For instance, it is possible to only use such data for training high-accuracy dynamics models (world models) that are useful for multiple tasks. Alternatively, expert data can (also) be used for training the policy. Additionally, pretraining MBRL components can itself be very challenging as offline expert data is typically sampled from a very narrow distribution of behaviors, which makes finetuning non-trivial in out-of-distributions areas of the robot’s state-action space. In this thesis, you will look at different ways of incorporating expert data in MBRL and ideally propose new approaches to best do that. You will test these methods in both simulation (simulated drone, wheeled, legged) and in the real world on our quadrotor platform. You will gain insights into MBRL, sim-to-real transfer, robot control.

Vision-based reinforcement learning (RL) is often sample-inefficient and computationally very expensive. One way to bootstrap the learning process is to leverage offline interaction data. However, this approach faces significant challenges, including out-of-distribution (OOD) generalization and neural network plasticity. The goal of this project is to explore methods for transferring offline policies to the online regime in a way that alleviates the OOD problem. By initially training the robot's policies system offline, the project seeks to leverage the knowledge of existing robot interaction data to bootstrap the learning of new policies. The focus is on overcoming domain shift problems and exploring innovative ways to fine-tune the model and policy using online interactions, effectively bridging the gap between offline and online learning. This advancement would enable us to efficiently leverage offline data (e.g. from human or expert agent demonstrations or previous experiments) for training vision-based robotic policies.

Vision-based reinforcement learning (RL) is more sample inefficient and more complex to train compared to state-based RL because the policy is learned directly from raw image pixels rather than from the robot state. In comparison to state-based RL, vision-based policies need to learn some form of visual perception or image understanding from scratch, which makes them way more complex to learn and to generalise. Foundation models trained on vast datasets have shown promising potential in outputting feature representations that are useful for a large variety of downstream tasks. In this project, we investigate the capabilities of such models to provide robust feature representations for learning control policies. We plan to study how different feature representations affect the exploration behavior of RL policies, the resulting sample complexity and the generalisation and robustness to out-of-distribution samples.

Recent research has demonstrated significant success in integrating foundational models with robotic systems. In this project, we aim to investigate how these foundational models can enhance the vision-based navigation of UAVs. The drone will utilize learned semantic relationships from extensive world-scale data to actively explore and navigate through unfamiliar environments. While previous research primarily focused on ground-based robots, our project seeks to explore the potential of integrating foundational models with aerial robots to enhance agility and flexibility.

In this project, we are going to develop a vision-based reinforcement learning policy for drone navigation in dynamic environments. The policy should adapt to two potentially conflicting navigation objectives: maximizing the visibility of a visual object as a perceptual constraint and obstacle avoidance to ensure safe flight.

Explore the use of large vision language models to control a drone.

Use Inverse Reinforcement Learning (IRL) to learn reward functions from previous expert drone demonstrations.

Explore online fine-tuning in the real world of sub-optimal policies.

Reinforcement learning (RL) models devoid of explicit models have showcased remarkable superiority over classical planning and control strategies. This advantage is attributed to their advanced exploration capabilities, enabling them to efficiently discover new optimal trajectories. Leveraging RL, our aim is to create an autonomous racing system capable of swiftly learning optimal racing strategies and navigating tracks more effectively (faster) than traditional methods and human drivers.

Event-based Reinforcement Learning Controller for Drone Racing

Gaussian Splatting meets Reinforcement Learning for Drone Racing

Design and implement efficient event-based networks to achieve low latency inference.

This project in concerned with the development of an agile drone system for autonomous exploration and inspection of ballast tanks using reinforcement learning (RL). Ballast tanks, essential for maintaining stability in marine vessels, pose significant challenges for inspection due to their confined, complex structures and GPS-denied environments. Traditional inspection methods, involving manual entry or remotely operated vehicles, are time-intensive, costly, and hazardous. Leveraging advancements in agile drone technology and RL, this project aims to design and implement a drone capable of navigating and inspecting these environments autonomously. The methodology involves creating a simulation environment replicating ballast tank conditions, training RL models for navigation and obstacle avoidance, and integrating these models into a hardware drone equipped with LIDAR, cameras, IMUs, and onboard processors. The trained system will be tested in controlled environments to evaluate performance in terms of navigation efficiency, area coverage, and robustness against uncertainties. Expected outcomes include a functional drone system that enhances inspection safety, efficiency, and cost-effectiveness, while providing a scalable framework for applying RL-driven drones to inspection in confined-space. The project can leverage our control and RL stacks for drone racing and augment them where necessary to enable the task of ballast tank exploration.

Reinforcement and imitation learning have already enabled great achievements in aerial robotics. However, most successes in this domain (as well as other domains in robotics) mostly rely on simulation as a tool for reducing the cost of training. This approach not only makes the assumption that a simulator can be provided for all tasks but also introduces the challenge of bridging the sim-to-real gap. The latter can be approached using multiple approaches such as state and action abstractions, domain randomization, and better system identification. However, it is not clear whether this gap can fully be fully bridged and whether the approaches we use to overcome the gap might introduce some disadvantages to the resulting system such as reduced performance, lack of robustness or simply overfitting behaviors. In this project, the goal is to learn agile flight policies using only real-world data and no access to a simulator. We will develop methods that can directly leverage real-world interactions for learning robust robot policies that do not overfit to the simulator. This comes with multiple challenges such as ensuring safe exploration and sample efficient training, but we already have some good ideas on approaching them.

In this project, you will investigate the use of event-based cameras for vision-based landing on celestial bodies such as Mars or the Moon.

This project aims to distill large, complex world models into lightweight, efficient versions capable of fast inference, enabling real-time feedback control for mobile robots. Large-scale world models that allow controlled image generation often suffer from high computational demands, limiting their utility on resource-constrained platforms. By leveraging model distillation techniques, knowledge from a pre-trained large model can be transferred to a smaller one through teacher-student learning, optimized loss functions, and methods like pruning, quantization, or neural architecture search (NAS). The distilled model will be deployed on a mobile robot to evaluate its real-world performance in terms of latency, energy efficiency, and task success rates. This project will follow a structured timeline: starting with literature review and dataset preparation, progressing to model distillation and optimization, and culminating in deployment, testing, and analysis. By enabling resource-efficient, real-time inference, this work aims to advance the development of responsive, autonomous mobile robots suitable for practical applications.

In this project we develop spiking neural networks (SNN)-based framework to efficiently compress event camera data, enabling low-latency, power-efficient processing for tasks like classification, object detection, and optical flow prediction. It combines the sparsity and speed of SNNs with the accuracy of ANNs.

This project seeks to leverage the sparse nature of events to accelerate the training of radiance fields.

This project aims to develop a sophisticated Reinforcement Learning (RL) environment to train autonomous drones for efficient disaster response operations. By leveraging insights from drone racing research, the project will focus on creating a highly realistic 3D simulation environment.

This project enhances vision-based drone racing by integrating neural rendering and advanced data augmentation techniques to improve policy generalization and robustness in unseen environments. It focuses on developing methods to strengthen gate detection accuracy and overall perception for autonomous drone navigation in dynamic scenarios.

This project develops a hybrid framework combining the high spatial detail of image-based neural networks with the high temporal resolution of event camera data to achieve accurate, low-latency visual perception. It targets real-time tasks like semantic segmentation and object detection, addressing challenges such as open-vocabulary recognition for dynamic and adaptive applications.

This project leverages spiking neural networks (SNNs) and event cameras to create a real-time system for detecting fast-moving objects with high efficiency and minimal latency.

Master's thesis in collaboration with SynSense: Neuromorphic Intelligence & Application Solutions

Explore novel ideas for low-cost but stable training of Neural Networks.

Perform knowledge distillation from Transformers to more energy-efficient neural network architectures for Event-based Vision.

This project focuses on combining Large Language Models within the area of Event-based Computer Vision.

Study the application of Long Sequence Modeling techniques within Reinforcement Learning (RL) to improve autonomous drone racing capabilities.

Develop a vision-based aerial transportation system with reinforcement / imitation learning.

This project explores a novel approach to graph embeddings using electrical flow computations.

IMU-centric Odometry for Drone Racing and Beyond