Building Accountable Systems

Project Description

With the rapid deployment of digital systems into all aspects of daily life, the need to understand their unwanted actions grows. As Cyber-physical systems might harm people, and information systems might compromise their assets, they should be accountable. From our experience, it is inherently impractical to specify all legal interactions of these systems, which means that the possibility of illegal interactions cannot be excluded at design time.  Hence, we need to be prepared for failures of the system; and therefore need accountability mechanisms that help us identify the root cause for such a failure, both to  eliminate the underlying (technical) problem and to assign blame.

Research Directions

We are working on approaches for Cyber-Physical Systems (CPS), such as diagnostic systems for Unmanned Aerial Vehicles (UAV), and microservice-based architectures that aid in detecting unwanted events at runtime and attributing them to misbehaving system parts or persons. This research aims to tackle the following topics:

• Building generic frameworks and implementations to operationalize causality (as an enabler to accountability). These frameworks tackle the efficiency and scalability of automated causal reasoning. They also consider reusable, practical domain-specific approaches to modeling and contextualization of unwanted events.
• Building methods to advocate on the origin and the right degree of abstraction of models describing causality and the requirements for the degree of abstraction of logging.
• Building methodologies that guide that considers the above to design accountable systems.

Projects

This research is being conducted as part of the following past and on-going projects:

1. TUM Living Lab Connected Mobility (https://tum-llcm.de/) : a project funded by the Bavarian Ministry of Economic Affairs, Energy and Technology (StMWi) through the Center Digitisation. We conducted a mapping study of the accountability literature and implemented different causality algorithms. Furthermore, we proposed a framework that analyzes flight logs generated by Unmanned Aerial Vehicles (UAV) to find their failure's root causes automatically.
2. Brainloop: We studied the problems related to the security of logging and automated threat modeling within modern cloud systems.   

3. Design Paradigms for Societal-Scale Cyber-Physical Systems funded by the Deutsche Forschungsgemeinschaft (DFG) under grant no. PR1266/3-1, where we investigate two main research questions: 1) How to combine the causal models of multiple agents into a composite model to reason over their interaction? 2) How can we effectively and efficiently infer and reason about causality?

Software

As part of our research in the different projects, we open-sourced a set of reasoning tools to implement our approach.

• HP2SAT 1.0-  a library to check actual causality according to the modified Halpern-Pearl definition of causality 
• HP2Opt- a Java Library that can model and solve binary causality inference questions
  using optimization solving.
• extractr  - a tool to transform attack and fault trees to causal models
• Attack Graph Generator-  automatic generation of attack graphs for micro-services architecture 
• Actual Causality Canvas  an interactive platform for causal modeling and causal checking