Result-driven Interactive Visual Support of Parameter Selection for Machine Learning
Machine Learning (ML) systems are used for diverse tasks in various domains but due to their opaque computations, selection of appropriate algorithmic parameters remains a challenge. Specifically, there is a need to develop strategies and techniques to support people without formal ML education with parameter selection to make ML systems accessible for non-technical stakeholders. A common challenge is to choose parameters for dimensionality reduction algorithms such as T-SNE so that they provide embeddings that preserve specific information required for certain tasks (3).
Interactive parameter selection is a technical process that typically includes the choice of technical parameters with interface elements such as sliders or radio buttons. However, such interactive parameter selections may mislead non-technical stakeholders. On the one hand, the numerical values of parameters shown in interface elements may not be meaningful to them. On the other hand, research has shown that interactive elements can suggest non-existent causal links between parameters and outputs (Springer et al. 2017, Kaur et al. 2019). In contrast, we envision visual interfaces for result-driven selection of ML parameters, i.e., interfaces that let humans choose parameters retro-actively based on possible outcome. This would resemble the selection of colors in graphics editors showing the color space without necessarily knowing the exact RGB values. We hypothesize that such interfaces can provide better guidance for ML parameter selection, especially for non-technical stakeholders. In the case of dimensionality reduction, “better” means that humans are more successful in finding a result that contains the information they need to solve their tasks.
The goal of this BSc thesis is to explore how result-driven ML parameter selection can be facilitated in a visual interface, and to evaluate if this supports a non-technical stakeholder with selection of suitable parameters for dimensionality reduction. This thesis will be based on existing work from project IKON that provides an ML topic extraction pipeline.
For this thesis, the following steps are foreseen:
- Implement a 2D embedding of research projects based on the topic extraction pipeline developed at HCC in project IKON
- Develop a prototypical visual interface based on small multiples visualizing the result space of the embedded research projects. For this, similar solutions must be taken into account (e.g. (1), (2)). The design process should build on prior qualitative research in project IKON.
- Conduct a user study to assess the potential of the interface for parameter selection. This can be done as a think-aloud study or similar, comparing the developed interface to a traditional interface with sliders.
- Discuss the results and give recommendations for the design of result-driven ML parameter selection interfaces.
Cutura, R., Holzer, S., Aupetit, M., Sedlmair, M., 2018. VisCoDeR: A Tool for Visually Comparing Dimensionality Reduction Algorithms 6.
Kwon, O.-H., Ma, K.-L., 2019. A Deep Generative Model for Graph Layout. arXiv preprint arXiv:1904.12225.
Sacha, D., Zhang, L., Sedlmair, M., Lee, J.A., Peltonen, J., Weiskopf, D., North, S.C., Keim, D.A., 2017. Visual Interaction with Dimensionality Reduction: A Structured Literature Analysis. IEEE Trans. Visual. Comput. Graphics 23, 241–250. https://doi.org/10.1109/TVCG.2016.2598495