Springe direkt zu Inhalt

Investigating AI-assisted Decision-Making Processes using Human-centered HCI Methods

Requirements

  • Required: Successful participation in the course "Human-Computer Interaction"
  • Desirable: Successful participation in the seminar on "Interactive Intelligent Systems" and the lecture on "Wissenschaftliches Arbeiten in der Informatik"

  • Interest in conducting interviews, workshops etc.

  • Knowledge in HCI

  • Useful: Knowledge in data structures, decision trees, cognition (e.g., human biases)

Academic Advisor
Ulrike Schäfer
Discipline
Human-AI Collaboration, decision-making processes
Degree
Master of Science (M. Sc.)

Contents

Research shows that current methods to improve how users interact with AI in decision-making tasks do not lead to expected performance improvement. Thus, instead of focusing on technical available solutions to improve AI-assisted decision making, we want to focus on what humans really need.

Recent research already shows that direct guidance, such as providing information about the AI's capabilities or a tutorial, can have a positive effect on performance. However, these approaches may also lead to a higher cognitive load, as users have to consider more and more information. There is a need for actionable guidance that considers the user and decision task at hand. To do so, decision-making processes need to be explored in more detail, specifically to be able to identify decision points and paths that may lead to good decisions. This then provides the basis for future research to guide users.

The objective for this thesis is to compare creative approaches to identify how users make decisions in a decision-making task with AI assistance. In the end, decision paths consisting of decision points and options users considered are derived.

Procedure

To reach this goal, the following steps can be taken:

  1. Read literature about how to investigate decision-making processes or users' mental models about the AI,
  2. Define a very simple decision-making task using Teachablemachine, for example, to classify animal images (cases defined with my help), 
  3. Decide on at least two creative methods to identify decision paths, such as letting users draw decision trees or letting them write out steps.
  4. Conduct either interviews with users or group discussions to let them make decisions and collect data.
  5. Analyze your data to present the decision trees and reflect on your chosen approaches,
  6. Write your thesis.

This is an explorative thesis, which is recommended for students with an interest in investigating users and analyzing qualitative data in detail. The focus is not on AI development. The task also can be presented using Figma or simple tools, as the focus is on understanding the user.

References

  • Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. _Nature Human Behaviour_, _8_(12), 2293-2303.
  • Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023). Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. _Frontiers in Computer Science_, _5_, 1096257. https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2023.1096257/full (Mental Model Section)
  • Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024). The impact of imperfect XAI on human-AI decision-making. _Proceedings of the ACM on human-computer interaction_, _8_(CSCW1), 1-39. (Simplified decision paths could also be defined by the researcher by showing the AI recommendation later.)
  • Example of technical tools to visualize mental models: https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2021.761882/full