Our group has three papers accepted at the 2021 International Conference on Machine Learning (ICML):
-
Johannes Klicpera, Marten Lienen, Stephan Günnemann
Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, and More
International Conference on Machine Learning (ICML), 2021
Summary: Entropy-regularized optimal transport (OT) requires a full pairwise cost matrix between all pairs of objects. The resulting quadratic runtime prohibits the use of OT in large-scale machine learning problems. We propose two approximations to the cost matrix with log-linear runtime: first, a sparse approximation based on locality sensitive hashing (LSH) and, second, locally corrected Nyström (LCN), a low rank approximation with LSH-based sparse corrections. Our approximations speed up a state-of-the-art method for unsupervised word embedding alignment 3x and improve the accuracy by 3.1 percentage points. For graph distance regression, we propose the graph transport network (GTN), which combines graph neural networks with LCN. GTN outcompetes previous models by 48% and scales log-linearly in the size of the graphs. -
Anna-Kathrin Kopetzki, Bertrand Charpentier, Daniel Zügner, Sandhya Giri, Stephan Günnemann
Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
International Conference on Machine Learning (ICML), 2021
Summary: Dirichlet-based uncertainty (DBU) models are a recent and promising class of uncertainty-aware models. We present the first large-scale, in-depth study of the robustness of DBU models under adversarial attacks and show that uncertainty estimates of DBU models are not robust w.r.t. three important tasks: (1) indicating correctly and wrongly classified samples; (2) detecting adversarial examples; and (3) distinguishing between in-distribution (ID) and out-of-distribution (OOD) data. Additionally, we explore the first approaches to make DBU models more robust. -
Marin Biloš, Stephan Günnemann
Scalable Normalizing Flows for Permutation Invariant Densities
International Conference on Machine Learning (ICML), 2021
Summary: Modeling sets is an important problem in machine learning since this type of data can be found in many domains. Examples include point clouds, items in a shopping cart, tracking household electricity consumption in a city etc. A promising approach defines a family of symmetric densities with continuous normalizing flows. This allows us to maximize the likelihood directly and sample new realizations with ease. However, calculating the trace of the Jacobian, a crucial step in this method, raises issues that occur both during training and inference, limiting its practicality. We propose an alternative way of defining permutation equivariant transformations that give closed form trace. This leads not only to improvements while training, but also to better final performance. We demonstrate the benefits of our approach on point processes and general set modeling.
Furthermore, we have one paper accepted at the journal track of the 2021 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD) and one paper at the 2021 International Joint Conference on Artificial Intelligence (IJCAI):
- Anna-Kathrin Kopetzki, Stephan Günnemann
Reachable sets of classifiers and regression models: (non-)robustness analysis and robust training
Machine Learning Journal, 2021
Summary: Understanding the behavior of neural networks is an open challenge that requires questions to be addressed on the robustness, explainability and reliability of predictions. We answer these questions by computing reachable sets of neural networks, i.e. sets of outputs resulting from continuous sets of inputs. We provide two efficient approaches that lead to over- and under-approximations of the reachable set and use them to (1) analyze and enhance robustness properties of classifiers and regression models (2) provide techniques to distinguish between reliable and non-reliable predictions for unlabeled inputs and (3) to quantify the influence of each feature on a prediction and compute a feature ranking.
- Oleksandr Shchur, Ali Caner Türkmen, Tim Januschowski, Stephan Günnemann
Neural Temporal Point Processes: A Review
International Joint Conference on Artificial Intelligence (IJCAI), 2021
Summary: In this paper we review neural temporal point processes (TPPs) -- flexible generative models for continuous-time event sequences. We describe the important design choices for neural TPPs, review established and emerging applications for these models, and discuss the main challenges that the research field of neural TPPs currently faces.
Congratulations to all co-authors!