In the era of HAI, metaphorical design is not only the art of visual expression but also a bridge between user cognition and interactive experience. This article delves into the core concepts of metaphor design, analyzes its application in smart products, and reveals how metaphors can be used to enhance user understanding and improve product usability.
HAI is revolutionizing our relationship with technology and the role it plays in our lives. As designers, we face a key challenge: how do we help people understand agents? More specifically, how do we shape agent interfaces to help people understand how to interact with agents?
In the HCI phase, metaphors are an important way to help people understand how to interact with computers. Xerox designed the computer interface as a familiar office desktop, so that the general public who did not know about computers could quickly get started and use it.
Metaphors are equally important in the HAI stage. Microsoft AI CEO Mustafa in his TED talk “What exactly is AI?” When we think about artificial intelligence, the use of metaphors is particularly important, and it affects how people understand AI. In this article, we will first introduce the concept of metaphor and the basic metaphor in interaction, further analyze the new interaction metaphor in the HAI era, and finally look forward to its future development direction.
01 What is a metaphor?
Metaphor refers to understanding and experiencing something in the present through another thing, and the human mind is largely made up of metaphors, ubiquitous in language and life (Lakoff & Johnson, 2008). For example, “alignment” is a metaphor that migrant workers often say, “alignment” is our experience in space, it is an innate feeling, and we will feel good understanding of using it to express the synchronization of thinking. As you may have discovered, “synchronization” is also a metaphor.
A basic metaphor for interaction
Metaphors are ubiquitous in human-computer interaction, and it helps people understand how to use computers more easily. The desktop we are familiar with is a very successful metaphor, designing the computer interface into a familiar office desktop, with files, folders, trash cans, and ordinary people who are not familiar with computers can quickly get started and use it.
Macintosh desktop from 1984
Desk countertop
There are two basic metaphors for interaction,World metaphorandDialogue metaphor（Hutchins, Hollan, & Norman, 1985）。
World metaphor
In a system based on world metaphors, the interface is a world that the user can manipulate, and changes state based on the user’s actions. The desktop is a metaphor for the world, and the elements on the user click and move the interface are like manipulating real-world objects.
Computer mouse interaction
iOS gesture interaction
Dialogue metaphor
In a system based on conversational metaphors, the interface is a linguistic medium, users have a dialogue with the system through language media. For example, command-line interfaces and various dialogue assistants are products based on dialogue metaphors.
Command line interface
Microsoft Office Assistant (1997)
Since the dawn of computers, interactions have become more natural as technology has evolved, but they still essentially fall under these two basic metaphors.
02 The metaphor of HAI
HAI opens up new possibilities for interaction, where agents can recognize user intentions and make autonomous decisions and actions, allowing users to interact with agents more naturally through language and body. What metaphor should be used in the HAI phase so that users can understand the agent’s abilities and know how to interact with it?
To achieve these three challenges, product managers will only continue to appreciate
Good product managers are very scarce, and product managers who understand users, business, and data are still in demand when they go out of the Internet. On the contrary, if you only do simple communication, inefficient execution, and shallow thinking, I am afraid that you will not be able to go through the torrent of the next 3-5 years.
View details >
Based on the basic metaphors of interaction, HAI has gradually developed the following new types of metaphors:Magic tools, magical items, and companions。 Each of these metaphors has its own unique advantages and design challenges, which will be explained in depth below.
Magic tools
“Any technology that is advanced enough is no different from magic.”
— Source: Arthur C. Clarke
The magic tool metaphor allows users to understand AI as a mysterious, powerful, and controllable force that users need to use magic tools to control it.
Features that use this metaphor are common in design software, such as the Magic Wand tool in Photoshop, which recognizes the content of an image and automatically selects objects in the image. and Apple’s Image Wand in iOS 18, where users use the wand to draw a circle in the blank space of a note, and the AI will generate an image based on the content of the note.
iOS 18 Image Wand
Magic and AI share many similarities.
firstcontrollable。 Users can use “magic” freely using “magic”.
secondUncertainty of the outcome。 Like the randomness of magic, the output of AI is also unpredictable.
thirdDependent intent。 Whether it’s magic or AI, it requires the communication and interpretation of intent.
As users cast their magic, they silently recite spells, while the AI understands user intent by parsing the context. These similarities allow users to have more accurate expectations about the AI’s capabilities and understand the results generated by the AI. For example, when a wand generated graph is not ideal, users don’t blame the AI for being stupid because they know that the results of the magic are uncertain.
Another advantage of magic tools is that they can be brought to the userA sense of control and accomplishment。 While choosing metaphors, designers also define the role of the user. If AI is magic, then the user is the magician who controls it. The metaphor of magical tools can make users feel as powerful as Harry Potter without fear of being replaced by AI.
The “invisibility” of magic eliminates the need for users to understand the mechanics behind AI, but it also bringsLack of transparencyquestion. “Magic” does not explain why a result occurred or what references there are. For simple tasks like image generation, this is fine. However, for more complex tasks, such as writing professional reports and papers, the lack of transparency and explanation can reduce user trust in AI-generated results.
Magical items
Magical items are those that compare AI products to items with superpowers. We often see magical objects in magical film and television works, such as the sorting hat in “Harry Potter”, which can use mind reading to learn what students are thinking and read out the house they want to go to. AI products are also like magic objects, predicting and adapting to people’s needs, such as future self-driving cars that automatically wait at the departure point when they know that users have travel needs.
“Harry Potter” Branch Hat
Autonomous vehicles
The concept of magic objects was coined by American entrepreneur David Rose to describe IoT devices that respond to people’s needs and behaviors in magical ways, such as self-driving cars, smart thermostats.
Rose focuses on devices in the physical world, butMagical items can exist not only in the physical world, but also in the digital world。 Let’s look at an example. In HCI’s desktop metaphor, items are static, responding to the user’s control. Let’s think back to the experience of files on computers and mobile phones: after downloading a file, we need to give it a name, choose the folder where it is stored, or it exists in a default folder where the file is stored until we move it. When we work with a file, we need to open the folder layer by layer and find it. Based on the metaphor of magical objects, we can reconstruct the entire experience outside the framework of the desktop, we can make each file “come alive”, they are automatically archived into the right space, and dynamically organized according to the user’s current scenario and task.
Compared to magic tools that are completely controlled by the user,Magical items can convey the autonomy of AI。 It does not rely on the user’s control and can perceive, decide, and act autonomously. This reduces the burden on users to perform complex operations. But it also reduces the user’s sense of control.It is easy to raise concerns about privacy and security。 For example, many smart speaker users are worried about eavesdropping. In addition, magic item metaphors can easily lead users to have high expectations. Users believe they should perfectly predict and meet needs. If the product doesn’t perform as expected, users will be disappointed.
companion
The companion metaphor refers to the idea of an agent as a digital companion to humans. Most current AI products are designed based on peer metaphors, including ChatGPT and Siri. The earliest digital companions date back to the 60s of the 20th century, when MIT professor Joseph Weizenbaum developed a chatbot called Eliza. Examples of this are also common in popular culture, such as Samantha in the movie “Her” and Jarvis in “Iron Man”.
Samantha, the AI companion from the movie “Her,” calls the protagonist
The peer metaphor is in line with the user’s cognitive habits about technology. The theory of “media equivalence” in communication studies points out that people are accustomed to seeing computers and televisions as individuals like themselves. useHouseholds are accustomed to anthropomorphizing technology, even if they do not have any human qualities, they will be endowed with human characteristics, which is human nature. For the industry, AI is different from all previous human technologies, it is more dynamic, more ambiguous, and more active, and peer metaphors can capture these characteristics, reflect the capabilities and development potential of AI, and bring greater development space to the industry.
Agent interaction based on peer metaphor is close to the interaction between people, and designing agents is designing “people” and designing us humans.How to design the ideal “person” so that users can actively interact with it, bringing positive impact to society, is a key challenge in design. Microsoft has introduced two agents with similar functions, but users interact with them in completely different ways. Xiaoice is an empathetic companion that has become a huge hit since its release, accumulating millions of users every month. Tay is a chat agent launched by Microsoft on Twitter, which was designed to be a calm nineteen-year-old American girl, and users discussed extreme topics with her, taught her to make racist remarks, and eventually Tay was shut down.
The avatar of Microsoft Xiaoice
Tay’s Twitter homepage
As can be seen from the above example,The “persona” of an agent affects users’ expectations, interaction behavior, and evaluation。 “Personality” includes social roles, abilities, personalities and emotions, etc., which are all directions worth exploring and thinking about. For example, in a personal computing device, what should be the role of the agent, whether it is an assistant, housekeeper or other role? The positioning of the assistant is to assist the user in completing specific tasks, and its relationship with the user is dominant-obedient, the user makes commands, and the assistant is responsible for executing them. In addition to performing specific tasks, the steward is also responsible for higher-level management and planning. It has a more equal relationship with users and can provide proactive advice and care. Other characters, such as companions, are positioned not to help users complete tasks, but to provide companionship and emotional support to users. These roles have their own advantages, and there is no absolute answer to how to choose, we need to design it in combination with product positioning and user scenarios.
03 Future prospects
Magic tools, magical objects, and companion metaphors provide different frameworks for understanding agents,The distribution of control between users and agents is a key difference in these metaphors。 In the metaphor of “magic tools”, the control of the task mainly belongs to the user, who explicitly uses the tool to perform the task; In the metaphor of “magic objects”, the control is more biased towards the agent, who has a certain degree of autonomy and actively provides services according to the user’s needs and scenarios; In the “companion” metaphor, the user and agent share control, and both parties make decisions together in the interaction.
Each control distribution model has its advantages. The metaphor of “magic tools” brings a greater sense of control and efficiency; The metaphor of “magic objects” enhances users’ curiosity and desire to explore; The “companion” metaphor helps to establish a long-term interaction between the user and the agent.
The future of HAI may be a hybrid experience, flexibly adapting metaphorical patterns to different scenarios and user needs。 For example, when you need to take control of a task, the system can provide “magic tools”; In daily interactions, the “companion” mode is adopted. We can see this trend in the industry. ChatGPT launched the Canvas function in October last year, when it recognizes that the user wants to write an article in the conversation, the interface will switch from the dialog window to the canvas for document editing, and provide some intelligent tools for document editing, allowing users to make more precise adjustments to the article. This flexible and dynamic switching will make the interaction between users and agents smoother and more natural.
ChatGPT’s canvas feature
For practitioners, metaphors not only help users understand the design methods of agents, but also provide different perspectives on agents. From different metaphors, we can rethink the various experience issues faced by HAIs, such as control, privacy, transparency, and trust, to shape experiences that meet user expectations.
References
- Archetype AI. (2024, Sep) Power of metapor in human computer interaction.
- Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985). Direct manipulation interfaces. Human–computer interaction, 1(4), 311-338.
- Lakoff, G., & Johnson, M. (2008). Metaphors we live by. University of Chicago press.
- Rose, D. (2014, Dec). Enchanted Objects: Design, Human Desire, and the Internet of Things [Video].TEDx.
- Suleyman,M. (2024, April).What is AI Anyway? [Video]. TED.
Source: From online public information, if copyright is involved, please contact to modify
