AI Assistant with AR Character in VR
We solve the problem of complex washer control for people, especially the elderly and children. Using only voice, the user can deal with these functions more easier. Based on Eliza Effect with this AR character, it has a lots of business impact by using GE Appliances.
Team
1 Product Designer,
2 Software Developers (iOS/Android),
3 Embedded Engineers
Company
GE Appliances, a Haier company
Role
Product Designer
(Research, User Testing, 3D Character Design, Motion Design, UX/UI Design for VR/AR)
Duration
6 weeks (Apr - May 2022)
Impact
Achieved 1st prize at in-company (GE Appliances) hackathon
Process
Background
Create the Experience: The Metaverse Era in 2022
With the rise of virtual spaces and VR/AR in 2022, companies are innovating to enhance customer experiences. These efforts add experiential value to physical products, unlocking new and meaningful engagement opportunities.
Business Goal
Improve the user experiences by enhancing usability
We began exploring how to leverage emerging technology to deliver unique GE experiences and create new value for our customers.
Research
What is Current Problem of Home Appliances?
Through two rounds of user surveys, we focused on the "Complex Usability" of the washing machine, which was the most challenging appliance, particularly for the elderly or children who want to operate it on their own.
01
Online Survey
After asking 52 users, we found 128 pain points across 27 home appliances.
02
Data Visualization - Infographic
Organized the pain points on each appliance based on the online survey for collecting additional pain points. The most common pain points were highlighted by increasing the font size, and the most frequently reported problematic appliance was emphasized with a larger size to make the information visually clear and prominent.
03
Follow-up Interview
Through two rounds of user surveys, we focused on the "Complex Usability" of the washing machine, which was the most challenging appliance, particularly for the elderly or children who want to operate it on their own.
Mission
Creating a Distinct and Engaging Virtual Experience
After user survey, the existing chatbot relies on text input, which some users find challenging for typing.
01
Differentiation with other brand
How might we create a strategy that sets GE Appliances apart from other brands in the virtual space?
02
Reaching Future Potential Customers
How might we effectively leverage virtual spaces to engage teenagers, our future potential customers, and build strong brand awareness?
03
Retaining Existing GE Appliances Users
How might we retain existing GE Appliances users by keeping them engaged through innovative virtual experiences?
04
Enhancing Complex Usability
How might we solve the fundamental challenges of complex appliances, such as excessive buttons and underutilized features, to help users make the most of their products?
Problem
Complex Usability and Visibility Challenge
Based on the research, there are two big pain points for the user, especially the elderly and children. First, there are too many buttons and small texts make washer difficult to use. Second, the manual is overwhelming, and the user feels unsure about the features that match their clothes for washing.
Problem Statement
How might we simplify the washer interface so that users can easily access features without being overwhelmed by too many buttons or complex manuals?
Insight
Simplifying User Choices
Users often rely on a few familiar settings, steering clear of other features because of the overwhelming number of options and complicated controls. This highlights the need for a simpler, more intuitive interface that allows users to quickly find and use the best settings for their laundry without the need to analyze or decode various functions. Streamlining the design can help users feel confident in choosing the right options effortlessly.
Solution
Voice Control with AI Assistant
Problem
Users often rely on a few familiar settings, steering clear of other features because of the overwhelming number of options and complicated controls.
➜ Voice Assistant Service
Since they can get right to the menu they want through voice service, it has less Depth and doesn’t require manuals.
AI-driven Character based on Eliza Effect
Problem
Users often rely on a few familiar settings, steering clear of other features because of the overwhelming number of options and complicated controls.
➜ Affection with appliances based on the Eliza Effect for increasing primary users
By creating different entry points, the app now prioritizes displaying only the necessary features for each user type from the beginning.
Design Process
Analyzing & Studying VR/AR
We analyzed the UX of VR devices and summarized the movements that can be utilized, and considered UX flow and hand gestures together when designing the on-screen UI.
Information Architecture for VR
Users often rely on a few familiar settings, steering clear of other features because of the overwhelming number of options and complicated controls. This highlights the need for a simpler, more intuitive interface that allows users to quickly find and use the best settings for their laundry without the need to analyze or decode various functions. Streamlining the design can help users feel confident in choosing the right options effortlessly.
Friendly interaction with AI voice
Making it easier to engage and build emotional connections with home appliances through verbal communication, I designed an AR character and motions to create a more friendly interaction for users in voice services. We also utilized AI to enable speech recognition and natural language processing.
Final Design
AI Voice Assistant with AR Characters
Users often rely on a few familiar settings, steering clear of other features because of the overwhelming number of options and complicated controls. This highlights the need for a simpler, more intuitive interface that allows users to quickly find and use the best settings for their laundry without the need to analyze or decode various functions. Streamlining the design can help users feel confident in choosing the right options effortlessly.
AR Character with Animation
This highlights the need for a simpler, more intuitive interface that allows users to quickly find and use the best settings for their laundry.
VR Application
This highlights the need for a simpler, more intuitive interface that allows users to quickly find and use the best settings for their laundry without the need to analyze or decode various functions. Streamlining the design can help users feel confident in choosing the right options effortlessly.
User Testing
Usability Testing in VR, AR
We connected it to the washer to see if it actually worked. You can tell the character which appliances you want and what you want it to do, and it will do the customization cycles for you. We had positive feedbacks for user and earned first place.
Impact
Achieved 1st Winner at in-company hackathon
Our team won first place (DIAMOND) in a hackathon with this project.
Next Steps
Key Learnings from VR/AR Integration
I have hands-on experience with integrating 3D elements into AR/VR and enhancing user experience in virtual reality. Unfortunately, in real AR, there are too many different objects and situations in the background, so the character doesn't blend into the background very well. I plan to improve it further.
Takeaway
Explored how to design immersive and intuitive virtual 3D spaces, and realized how emerging technologies can offer alternatives to overcome current limitations.
01
Incorporating Diverse User Input Methods
Learned how to seamlessly integrate various user input methods, such as hand gestures, voice commands, and controller interactions, to create an intuitive and inclusive experience in a virtual 3D space.
02
Adapting to Depth and Lighting in 3D Environments
Gained expertise in designing for dynamic depth and lighting changes that occur as users move through virtual spaces, ensuring a consistent and immersive experience.
03
Understanding the Complexity of Voice Processing
Discovered how even seemingly simple voice command features involve many variables in development, which shaped a more thoughtful and realistic design approach.
Next project
Smart Management B2B Solution, GE
