Anticipating Human Behavior

Bonn, Germany, 8th September 2023


In contrast to humans that are very good in anticipating the behavior of other objects, animals, or humans, developing methods that anticipate human behavior from video or other sensor data is very challenging and has just recently received an increase of interest. In the past, the features for analyzing, in particular, visual data like images or videos were too weak such that approaches that predict the future were unlikely to succeed.

This burden has been overcome due to recent progress in this field. The anticipation of human behavior, however, is not well defined in the literature and varies depending on the task in terms of granularity and time horizon. In the context of driver assistance systems, the prediction of the trajectory of a pedestrian needs to be within centimeter accuracy but only for a very short time horizon of one second.
For tracking applications or motion planning, the potential destination of a human and trajectories of several seconds or minutes to reach the destination need to be predicted. In order to prioritize several tasks for a service robot during a day, only the rough time and location of an activity is needed. For instance, when the robot anticipates that the owner wants to cook in one hour, the robot will be in the kitchen at the right time.

The purpose of this workshop is to discuss recent approaches that anticipate human behavior from video or other sensor data, to bring together researchers from multiple fields and perspectives, and to discuss major research problems and opportunities and how we should coordinate efforts to advance the field.
The workshop will be located in the lecture hall HS1 of the Hörsaalzentrum Campus Poppelsdorf, University of Bonn, Friedrich-Hirzebruch-Allee 5, 53115 Bonn.

The workshop is organized as part of the DFG funded research unit FOR 2535 - Anticipating Human Behavior and previous workshops were held in Bonn 2019 and Munich 2018.


Registration 24.08.2023 Workshop 08.09.2023


The workshop will be located in the lecture hall HS1 of the Hörsaalzentrum Campus Poppelsdorf, University of Bonn, Friedrich-Hirzebruch-Allee 5, 53115 Bonn (Map). Entrance to the workshop is free, but a registration is required. For a late registration, please contact Michaela Musselmann (musselmann@iai.uni-bonn.de).


Dominik Bach is Hertz Chair for Artificial Intelligence and Neuroscience at the University of Bonn. His research seeks to exploit the power of artificial intelligence in order to understand biological intelligence, how it is enabled in the central nervous system, and how it fails in neuropsychiatric disorder. His goal is to understand how humans gracefully avoid real threat in their natural environment - and why some humans fear and avoid non-threatening situations. Sven Behnke is professor for Autonomous Intelligent Systems at the University of Bonn and director of Computer Science - Intelligent Systems and Robotics. His research interests include cognitive robotics, computer vision, and machine learning. Maren Bennewitz is professor for Computer Science at the University of Bonn and head of the Humanoid Robots Laboratory. The focus of her research lies on robots acting in human environments. Her group develops techniques that allow robots to adapt their behavior to the environment and to the surrounding people thereby exploiting semantic information about objects and information about the activities of users. Dima Damen is a professor for Computer Vision at the University of Bristol and currently an EPSRC Fellow (2020-2025), focusing her research interests in the automatic understanding of object interactions, actions and activities using wearable visual (and depth) sensors. She has contributed to novel research questions including assessing action completion, skill/expertise determination from video sequences, discovering task-relevant objects, dual-domain and dual-time learning as well as multi-modal fusion using vision, audio and language. She is the project lead for EPIC-KITCHENS, the largest dataset in egocentric vision, with accompanying open challenges. Anne Driemel is professor for Theoretical Computer Science at the University of Bonn and HCM Bonn Junior Fellow. Her research interests include discrete and computational geometry, algorithms and data structures, and trajectory and time series analysis. Gianpiero Francesca is AI Technical Manager at Toyota Motor Europe. Juergen Gall is professor and head of the Computer Vision Group at the University of Bonn. He is spokesperson of the DFG research unit FOR 2535 - Anticipating Human Behavior and his research interests include human pose estimation, video analysis, and forecasting. Reinhard Klein is professor for Computer Graphics at the University of Bonn and director of the Institute of Computer Science - Visual Computing. The group covers topics in geometry processing, scientific and geospatial visualization, photo-realistic rendering and physics based animation. Hilde Kuehne is professor and head of the Multimodal Learning Group at the University of Bonn. Tang Siyu is professor and head of the Computer Vision and Learning Group at ETH Zurich. She studies computational models that enable machines to perceive and analyze human activities from visual input. She leverages machine learning and optimization techniques to build statistical models of humans and their behaviors. Her goal is to advance algorithmic foundations of scalable and reliable human digitalization, enabling a broad class of real-world applications.


9:00-9:15 Welcome 9:15-10:15 Talks Social Diffusion: Multiple Human Motion Anticipation Juergen Gall Reconstruction and Synthesis of 3D Humans in 3D Scenes Tang Siyu 10:15-10:45 Coffee 10:45-12:15 Talks Spatial-temporal Grounding and Reasoning in Video Data Hilde Kühne Opportunities in Egocentric Vision Dima Damen Cloth Dynamics Prediction using Physically-based Simulation Reinhard Klein 12:15-13:15 Lunch 13:15-14:45 Talks Critical Intelligence: Towards Agent-based Models of Human Escape Dominik Bach Research and Development on Robotics and Computer Vision at Toyota Motor Europe Gianpiero Francesca Foresighted and Personalized Robot Navigation Maren Bennewitz 14:45-15:15 Coffee 15:15-16:15 Talks Finding Complex Patterns in Trajectory Data via Geometric Set Cover Anne Driemel From Semantic Video Prediction to Anticipative Human-Robot Collaboration Sven Behnke 16:15-16:30 Closing Remarks


Prof. Dr. Juergen Gall

University of Bonn

Institute of Computer Science

Computer Vision Group

Friedrich-Hirzebruch-Allee 8

53115 Bonn, Germany

E-mail: gall@iai.uni-bonn.de

Liability note

Despite careful content control we assume no liability for the content of external links. For the corresponding content of linked pages, their administrators are responsible.

Image source: Poppelsdorf Palace © Dr. Thomas Mauersberg / University of Bonn