Seminar: Machine Learning for Sequential Decision Making

(IN2107, IN4872)

Application

The pre-course meeting with information regarding the course format, possible topics etc. is scheduled for July 10, 2023 10am on zoom (Passcode: 021039). You can find the slides here.

Note that you have to register via the matching system and fill out our application form to apply for a spot!

Schedule

Pre-course meeting: July 10th, 2023 10am
Application deadline (matching system & form): July 19th, 2023
Kick-off meeting: October 18th, 2023
Final presentations: February 14th & February 15th, 2024

Prerequisites

This seminar is intended for Master's students only. You should have attended (and passed) the Machine Learning lecture (IN2064). Having attended Machine Learning for Graphs and Sequential Data (IN2323, formerly Mining Massive Datasets) is a plus.

Description

Real-world problem solving in fields such as healthcare, finance, or robotics often require a series of sequential decisions, with each decision now having an impact on future decisions and outcomes. Naturally, we would like to make the best possible decision at every stage to arrive at the most beneficial outcome. For example, a doctor wants to choose a treatment plan that maximises patient survival. An investor wants to optimise their daily acquisitions to increase long-term profit for themselves or their client. These decisions are often made under uncertainty, meaning the decision-maker must repeatedly assess possible outcomes, probabilities, and costs or benefits to make the best choice at each stage. Fuelled by an unprecedented explosion of data, machine learning is increasingly used to tackle these kinds of problems and support decision-makers throughout the process.

In this seminar, we will explore current research in machine learning methods for sequential decision making. Students will get familiar with the latest research, explore new fields and ideas, and critically analyse recent publications. Options will be provided for students to either delve deep into a single topic or explore a broader range of topics. For the former, students will select a single research paper with available code, analyse its method in depth, and explore its theoretical and practical aspects, e.g., by comparing it to another, potentially missing baseline or making modifications. For the latter, students will select a group of research papers (2-5) covering different approaches to a particular task, analyse, and critically evaluate them. The findings, criticisms, and research ideas will be summarised in a 4-page paper (double column). In both cases, students will prepare 25-minute presentations and present their work during a block seminar at the end of the semester.

Organisers

This seminar will be organised by two external researchers, Patrick Rockenschaub and Tom Haider.

Patrick is a Senior Research Scientist at the Fraunhofer Institute for Cognitive Systems, where he is working on trustworthy AI for medical applications. Patrick's research focuses on deriving robust models from often messy clinical time series, with a particular focus on generalisability of models to previously unseen patient populations. Patrick obtained a PhD at the UCL Institute of Health Informatics and completed a Humboldt postdoctoral fellowship at Charité Universitätsmedizin Berlin.

Tom is a PhD student at the Fraunhofer Institute for Cognitive Systems & TUM. His focus is on Reliability of Reinforcement Learning Systems and anomaly detection in sequential decision making.

Possible topics

Time Series Prediction
Time Series Representation Learning
Reinforcement Learning and its variants (model-based, model-free, off-policy, etc.)
Missing data in Time Series
Out-of-Distribution and Anomaly Detection in Sequential Data
Domain Adaption and Generalisation
Explainability and Interpretability in Sequential Decision Making
Estimation of Causal Effects and Dynamic Treatment Regiments