Evaluating Information Landscapes for Interpretation of Research Topic Embedding by Non-technical Stakeholders
- Python coding skills
Machine Learning (ML) systems are part of our everyday life but due to their opaque computations, interpreting their results remains a major challenge. Specifically, there is a need to develop strategies and techniques to support people without formal ML education with the interpretation of such results. A common approach to visualize high dimensional ML results is to create 2D embeddings by the use of dimensionality reduction. In order to make embeddings interpretable for humans there are attempts to use information landscapes that imitate geographical maps (1), (2). However, past research showed that with low level tasks the display of landscape-like features did not have positive effects (3).This thesis is based on the hypothesis that information landscapes do have positive effects when it comes to high level tasks on the level of interpretation of 2D ML embeddings.
The goal of this thesis is to test this hypothesis in a defined use case of a research topic embedding based on natural language processing (NLP). For this the following steps are foreseen:
- Implement a 2D embedding of research topics based on NLP analysis of the project abstracts from http://gepris.dfg.de/ The work and experience from an NLP pipeline developed at HCC can be used.
- Create an information landscape visualizing the embedding in 2D
- Conduct an online crowdsourcing experiment to assess if and how the landscape supports interpretation of the embedded research topics. This will be done by comparing the insights people get from the information landscape vs. an embedding as a simple word cloud.
- Discuss the results and the evaluation method (experiment) in light of the research question above.
Sen, S., Jackson, B., Swoap, A.B., Li, Q., Dippenaar, I., Ngo, M., Pujol, S., Gold, R., Boatman, B., Hecht, B., 2019. Toward Universal Spatialization Through Wikipedia-Based Semantic Enhancement. ACM Trans. Interact. Intell. Syst. 9, 1–29. https://doi.org/10.1145/3213769
Skupin, A., 2004. The world of geography: Visualizing a knowledge domain with cartographic means. Proceedings of the National Academy of Sciences 101, 5274–5278. https://doi.org/10.1073/pnas.0307654100
Tory, M., Sprague, D., Wu, F., So, W.Y., Munzner, T., 2007. Spatialization Design: Comparing Points and Landscapes. IEEE Trans. Visual. Comput. Graphics 13, 1262–1269. https://doi.org/10.1109/TVCG.2007.70596