Co-adaptive Sensemaking filters
German Research Foundation (DFG): EXC 2025: Matters of Activity. Image Space Material
Machine Learning-driven systems have become one of the most far-reaching environing technologies of our time. They shape our habits by recommending music to us, they shape our social interaction by keeping us up to date on topics relevant within our social circles, they even shape the way we work by providing us with tried and tested answers to specific questions. They have become important tools for making sense of our environment.
However, they are themselves often perceived as part of the environment. As something that we experience but have control over. For instance, when talking about videos or music nowadays, people often say the phrase "this was recommended to me" as if a friend had made the suggestion. This is a loss of agency, that can have unpredictable long-term effects. Especially, in work environments, where we want to create some sort of generalizable/assured/sound knowledge we need to be able to understand and control the systems we work with.
In this project, our vision is to give more agency to users of machine learning systems, to allow them to reap the benefits of these novel systems while at the same time staying in the driver's seat. We view the interaction between three distinct filtering systems.
Firstly, there is the human mind, a naturally evolved filtering system situated in a real-world environment with unique high-level cognitive abilities, such as the ability to think critically. Secondly, there is the Machine Learning model, an artificial filtering system that mimics only certain aspects of human cognition but can be applied with super-human endurance and speed - it can analyze huge amounts of data in a short amount of time. To achieve our envisioned synergy between human and machine filtering systems, we need a third filtering system, the interface, that translates between the other two by providing control mechanisms that are meaningful to the user and can be translated into beneficial changes in the model on the one hand and presenting the state of the Machine Learning model in a way that is intelligible to the user on the other hand. New ML-methods that are not based on statistical assumptions, such as that of normally distributed features, allow us to model how humans naturally form concepts within their situated environment.
Semantic mapping of the comments of a YouTube video. Image credit: Michael Tebbe
- Human-Computer Interaction
- Interactive Machine Learning
- Machine Learning Systems
- Semantic Interaction