New technology today is often presented, perceived and discussed as ‘intelligent’. But what do we mean when we refer to interactive technology in this way? To provide an answer from the perspective of a scientific community, we conducted a data-driven literature analysis of research published at the ACM Conference on Intelligent User Interfaces.
In this project, we did not set out with the intention of coming up with a definition of AI or intelligence. Instead, we followed a bottom-up approach: We extracted the emerging perspective on ‘intelligence’ in a research community working at the intersection of Human-Computer Interaction (HCI) and Artificial Intelligence (AI). Specifically, we analysed 25 years of research papers from the ACM Intelligent User Interfaces (IUI) conferences.
Extracting an emerging perspective on intelligence
We undertook this work in two steps, namely text analysis and manual coding.
Firstly, we used text analysis to extract all sentences with ‘intelligen*’ published at the IUI conference. This resulted in a significant number of 1,111 analysed papers, 504 of which mentioned ‘intelligent’ or a grammatical derivative at least once. In total, we extracted 1,804 sentences in this way.
Secondly, we manually reviewed these sentences looking at three key aspects:
- Entities: What is deemed intelligent?
- Co-descriptors: How (else) is it characterised?
- Actions: What capabilities are attributed to it?
For example, at the first IUI conference in 1993, Woods characterised an intelligent interface as “an autonomous computer agent who mediates between process and practitioner”. This would yield an entity (“agent”), co-descriptor (“autonomous”) and action (“mediate”).
What is intelligent in UIs? What does it do for users?
Based on the corpus described above, here we present and discuss the top ten entities, co-descriptors and actions that people in the IUI research community most frequently refer to when describing something as ‘intelligent’. This is presented in the figure below:
This data reveals three observations that characterise the emerging perspective on intelligence in this community at the intersection of HCI and AI.
Intelligent UIs assist the user. ‘Assisting the user’ is most often seen as the key action that the intelligent entity performs. Further common actions are creating, detecting, adapting, recommending, interacting and understanding. Such varying top actions suggest that these interactive systems and UIs overall address tasks that lend themselves to a mixed-initiative: Both user and AI are actively involved in various roles.
Intelligent UIs adapt to the user and automate tasks. When researchers describe something as intelligent, they also most commonly refer to ‘Adaptation, automation and interaction’ and these aspects remain part of the discourse across all conference years. Hence, they appear as the most coherent core of the emerging understanding of intelligence among experts at this intersection of HCI & AI.
Different UI concepts are considered intelligent in different ways. ‘Interfaces, systems, agents and assistants’ are the most common entities to which people in the IUI research community attribute intelligence. Thus, this community can be seen to have broadly embraced intelligence both for more traditional types of graphical user interfaces and agents, such as chatbots and voice assistants.
There are interesting differences between these concepts, as shown in the figure: Intelligent interfaces and systems tend to be described relatively more as adaptive and interactive, compared to agents and assistants. These, in turn, tend to be described relatively more as autonomous. Assistants are also often labelled as personal. Thus, the IUI experts’ implied assumptions and goals around intelligence vary depending on whether the technology is seen, for example, as a UI, system, tool, or agent.
Trend: Growing diversity of what is ‘intelligent’
The data also revealed a trend over time, shown in the figure below (Read the figure as: At each point, Y distinct codes have appeared until and including year X). This shows the growing diversity in what people in this research community refer to as intelligent and how they describe it. The main actions of intelligent entities are already present during the early years of the conferences.
We discuss two takeaways here: 1) The emerging implicit perspective on intelligence in interactive technology, and 2) how we can engage more explicitly with our assumptions about intelligence in such technology.
Perspective — intelligent technology assists humans
Together, these findings on intelligent user interfaces contribute to the wider ongoing discussion about the role of AI in our technology and usage thereof. In particular, the analysed research implicitly emphasises using AI in interactive technology for assisting users. It sees intelligence manifest itself in adaptation and automation, used interactively.
We can link the perspective emerging here to recent calls for AI to be rendered more controllable, interactive and explainable. In this light, ‘intelligent’ technology might aim both to: 1) use AI to improve current user interfaces; and 2) enable more interactive use of AI through new user interfaces and interactions. In both cases, the goal is to use AI to augment what people can do and not to replace them.
Engaging with diverse perspectives on ‘intelligent’ technology
We found very few attempts at explicitly defining intelligence in the analysed papers. Rather, experts at this intersection of HCI & AI seem mostly to rely on an implicit understanding of intelligence. Our analysis suggests that we can do three things to engage more explicitly with our assumptions and goals around ‘intelligence’ in the technology that we shape and use. Hence, our recommendations are:
- State how you construe interaction — in particular, do you regard the technology and its role for people to be a tool or an agent/assistant?
- State how the user interface relates to key aspects of adaptation and automation, along with how these matter for people that use this technology or are affected by it.
- Use these aspects to build and communicate a pragmatic working definition of what is ‘intelligent’ about the technology under discussion.
These recommendations are intended to serve as a practical communication tool, for example to clarify discussions in interdisciplinary teams or in presentations to multiple stakeholders. The various people involved may have very different views on what is ‘intelligent’ about the technology. Thus, explicating our perspectives in this way should support such teams and conversations. Moreover, while our views on intelligence in technology are likely to change over time, these recommendations build on the emerging perspective from 25 years of research at the intersection of HCI & AI.
A full analysis and report are available in the paper:
Sarah Theres Völkel, Christina Schneegass, Malin Eiband, and Daniel Buschek. 2020. What is ‘intelligent’ in intelligent user interfaces? a meta-analysis from 25 years of IUI. In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI ’20). Association for Computing Machinery, New York, NY, USA, 477–487. DOI: doi.org/10.1145/3377325.3377500
An arXiv version is also available: arxiv.org/abs/2003.03158
The blogs published by the bidt represent the views of the authors; they do not reflect the position of the Institute as a whole.
Daniel Buschek leads a junior research group at the intersection of Human-Computer Interaction and Machine Learning/Artificial Intelligence at the University of Bayreuth, Germany. Previously, he worked at the Media Informatics group at LMU Munich, where he had also completed his doctoral studies, including research stays at the University of Glasgow and Aalto University, Helsinki. In his research, he combines HCI and AI to create novel user interfaces that enable people to use digital technology in more effective, efficient, expressive, explainable and secure ways. In short, he is interested in both ‘AI for better UIs’ and ‘better UIs for AI’.