"Hey Siri, play my favorite song" - an interactive non-verbal workshop experience
This year's edition of Girls' Day took place on April 28, 2022 with a contribution by the HCC Research Group.
The Girls' Day is a nationwide project for career and study orientation for girls. On this annual day of action, schoolgirls learn about professions or fields of study in which the proportion of women is less than 40 percent. Companies, businesses, and universities throughout Germany open their doors to girls on Girls' Day to introduce them to careers and courses of study in IT, crafts, science, and technology.
The research group Human-Centered Computing offered the workshop "Hey Siri, play my favorite song", during which the participants were able to take a look behind the scenes of virtual assistants. After a short introduction to the history and technology behind virtual assistants, we explored the questions of how chatbots work, what user data is important, and what requirements must be met in order to be able to train and program a chatbot. We chose a simplified "wizard-of-oz" approach to interactively allow the participants to take a playful approach to the development of simple dialogs for a recipe chatbot.
During the conception and planning of the workshop, we had to factor in that the participants, girls between the ages of 11 and 13, could not be asked to open their cameras and microphones. Consequently, we had to plan the workshop in such a way that would allow interactive participation while at the same time being limited to dragging elements on the concept board or adding text to the concept board. We asked ourselves: how will this limited set of communication options play out?
During the pandemic, we all got used to conducting and attending meetings, workshops, conferences, and other events online by using video-conference systems. Collectively and iteratively, we developed a new culture of communication in this format. Keeping our microphones closed while others were talking, using a "raise hand" feature or other means of indicating when we wanted to speak. Some people seemed to have carefully selected a suitable background in their home offices and a broad range of home libraries appeared on our screens - others fell back on using virtual or blurred backgrounds.
Being stripped of these established ways of communication during video-conference workshops and meetings, we were curious to see how the participants would communicate within the two groups. One of the central interactive activities required one group to prepare text inputs from the users' perspective while the other one anticipated user inputs and prepared the chatbot's response. The goal, here, was that the participants prepare data input that would be needed in order to "train" the chatbot so that it would be able to engage in simple dialogue.
We were surprised by the spontaneous non-verbal culture of communication that the participants developed right away: by picking up elements on the board, dragging them to the side of the interface mock-up, and moving their mouse over the element, they indicated their intention to enter it into the interface mock-up. When the others of the group agreed, they joined their mouse pointers on the element until one took the initiative and entered it as input on the interface mock-up. This way, a non-verbal discussion, and collective decision-making took place.
Have a look at the video to see how this played out.