Recently, the topic of algorithm awareness  has received some attention within the disciplines of Human Computer Interaction, Human Factors and Science and Technology Studies. Hamilton et al., in introducing the term, outline that it delineates the extent to which a user is aware of the existence and function of algorithms in a specific context of use. Subsequent researchers have formulated possible methodological characteristics of designing for algorithm awareness [2,3]. One example is Motahhare Eslami's redesign of the Facebook newsfeed . However, studies of algorithm awareness so far are small in scale, much more so than online experiments on platforms such as Amazon MTurk. On the one hand, this is due to the novelty of the topic. On the other, understanding awareness commonly requires qualitative data. These obstacles combine to complicate how algorithm awareness can be evaluated.
In Project IKON, we aim to provide a foundation for expanding studies of algorithm awareness to massive online experiments. Provided with a draft for an experiment design, this BSc will develop and implement a functioning experimental framework for Amazon's Mechanical Turk based on prior research at the HCC research group  for a specific use case .
A possible procedure could consist of:
Comparing Massive Online Experiments on User Experience
Testing the draft against MTurk requirements and capacities
Evaluating how the implementation was successful in generating useful experimental data
Technical implementation will require a working knowledge of Node.js, Vue.js or comparable frameworks.
Hamilton, K. et al. 2014. A Path to Understanding the Effects of Algorithm Awareness. CHI ’14 Extended Abstracts, 631–642.
Motahhare Eslami. 2017. Understanding and Designing Around Users’ Interaction with Hidden Algorithms in Sociotechnical Systems. Comp. of the 2017 ACM CSCW Conference, 57–60.
Kizilcec, R.F. 2016. How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface. Proc. of the 2016 CHI Conference, 2390–2395.
Balaraman, V., Razniewski, S., & Nutt, W. 2018. Recoin: Relative Completeness in Wikidata. http://wikiworkshop.org/2018/papers/wikiworkshop2018_paper_2.pdf
Maximilian Mackeprang, Abderrahmane Khiat, and Claudia Müller-Birn. 2018. Concept Validation During Collaborative Ideation and Its Effect on Ideation Outcome. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ’18), LBW033:1–LBW033:6. https://doi.org/10.1145/3170427.3188485