Designing multimodal care experiences
across mirror, watch, and mobile
Designing multimodal
care experiences across
mirror, watch, and mobile
Interaction Design
Interaction Design
Concept Modeling
Concept Modeling
Usability Testing
Usability Testing
What I Faced
Seniors can’t keep up, caregivers can’t keep track
Seniors can’t keep up, caregivers can’t keep track
More seniors live alone, making routines harder to follow and care harder to provide.
More seniors live alone, making routines harder to follow and care harder to provide.
Independence is rising, but so are challenges
Independence is rising, but so are challenges
1 in 4 Seniors Live Alone
1 in 4 Seniors Live Alone
In the U.S., 28% (16.2 million) of seniors live independently, apart from their children.
In the U.S., 28% (16.2 million) of seniors live independently, apart from their children.
Administration for Community Living's 2023 Profile of Older Americans
Administration for Community Living's 2023 Profile of Older Americans
What I Did
End-to-end design
for connected care
End-to-end design
for connected care
Created personas from research synthesis, developed concept models and task flows, designed multimodal interactions across mirror, watch, and mobile, prototyped flows, conducted heuristic evaluation and usability testing with seniors, and built a cross-platform design system to ensure clarity and consistency.
Created personas from research synthesis, developed concept models and task flows, designed multimodal interactions across mirror, watch, and mobile, prototyped flows, conducted heuristic evaluation and usability testing with seniors, and built a cross-platform design system to ensure clarity and consistency.
What it became
One ecosystem, four touchpoints:
mirror, watch, mobile, and web
One ecosystem,
four touchpoints:
mirror, watch,
mobile, and web
I designed NURI, a connected care system using a Smart Mirror, Smart Watch, Mobile App, and Clinic Plugin to guide routines, book taxis, record and summarize consultations, and share updates instantly with caregivers.
I designed NURI, a connected care system using a Smart Mirror, Smart Watch, Mobile App, and Clinic Plugin to guide routines, book taxis, record and summarize consultations, and share updates instantly with caregivers.
Smart Mirror
Voice control, visual prompts, and daily health check-ins help seniors stay on track with minimal effort, making morning and evening tasks intuitive and stress-free.
Voice control, visual prompts, and daily health check-ins help seniors stay on track with minimal effort, making morning and evening tasks intuitive and stress-free.
Smart Watch
Provides on-the-go medication reminders, appointment alerts, and quick taxi booking for hospital visits, ensuring mobility and independence outside the home.
Provides on-the-go medication reminders, appointment alerts, and quick taxi booking for hospital visits, ensuring mobility and independence outside the home.
Mobile App
Keeps caregivers informed with real-time updates on routines, medication intake, and health checks, while automatically delivering consultation summaries and integrating them into routine management through smart notifications.
Keeps caregivers informed with real-time updates on routines, medication intake, and health checks, while automatically delivering consultation summaries and integrating them into routine management through smart notifications.
Web Plugin
Enables doctors to easily record consultations, automatically generate summaries, and securely share them with caregivers, improving clarity and reducing follow-up calls.
Enables doctors to easily record consultations, automatically generate summaries, and securely share them with caregivers, improving clarity and reducing follow-up calls.
Observing care in context
Simple tools, stronger support
Simple tools, stronger support
I conducted contextual inquiries by analyzing videos of caregivers managing the health of senior family members. Through this analysis, I identified key challenges, such as coordinating medical appointments and managing daily health routines. My observations provided valuable insights into the interactions between caregivers and seniors, highlighting opportunities for technology to better support their needs.
I conducted contextual inquiries by analyzing videos of caregivers managing the health of senior family members. Through this analysis, I identified key challenges, such as coordinating medical appointments and managing daily health routines. My observations provided valuable insights into the interactions between caregivers and seniors, highlighting opportunities for technology to better support their needs.
Key Insights
High stress, high stakes
Caregivers need relief from time and emotional burdens.
Simplicity saves
Seniors thrive with intuitive, easy-to-use tech.
Connection matters
Real-time updates for peace of mind.
High stress, high stakes
Caregivers need relief from time and emotional burdens.
Simplicity saves
Seniors thrive with intuitive, easy-to-use tech.
Connection matters
Real-time updates for peace of mind.
Synthesizing insights
From patterns to core ideas
From patterns to core ideas
I synthesized findings from contextual inquiries using an affinity diagram, grouping related insights to uncover recurring patterns and unmet needs. These clusters directly informed and refined potential feature ideas, ensuring they addressed both seniors’ and caregivers’ real-world challenges.
I synthesized findings from contextual inquiries using an affinity diagram, grouping related insights to uncover recurring patterns and unmet needs. These clusters directly informed and refined potential feature ideas, ensuring they addressed both seniors’ and caregivers’ real-world challenges.
Core Ideas
🗓️ Effective Routine Management
🩺 Comprehensive Health Oversight
🗣️ Seamless Voice Interaction
📱 Ease of Use Across Devices
🚕 Optimized Mobility Support
🗓️ Effective Routine Management
🩺 Comprehensive Health Oversight
🗣️ Seamless Voice Interaction
📱 Ease of Use Across Devices
🚕 Optimized Mobility Support
Who I designed for
James, a senior living alone, has a hospital appointment today.
Without reminders, he sometimes forgets the time or what to bring.
James, a senior living alone, has a hospital appointment today.
Without reminders, he sometimes forgets the time or what to bring.
Independent Senior
Struggles with daily health routines
James Miller
Age: 79
Occupation: Retired Engineer
Location: Seattle, WA
“I like being independent, but a little help with my health routines would be nice.”
Life Goal
Manage his health routines
without feeling overwhelmed.
Experience Goal
Stay connected with his daughter in a way that doesn’t disrupt her daily life.
End Goal
Use a simple system
that provides reminders and guidance.
Lisa, living in another city, worries if James will remember the appointment, arrive safely, and understand the doctor’s advice.
Lisa, living in another city, worries if James will remember the appointment, arrive safely, and understand the doctor’s advice.
Distant Caregiver
Needs support for remote health management
Needs support
for remote health management
Lisa Thompson
Age: 46
Occupation: Financial Analyst
Location: Chicago, IL
“I just want to make sure my dad’s taken care of, even when I can’t be there.”
Life Goal
Ensure her father’s health routines and appointments are consistently managed.
Experience Goal
Find an easy-to-use solution that her father can manage independently.
End Goal
Receive real-time updates on her father’s health for peace of mind.
What I aimed to achieve
Ensuring seniors feel confident
and caregivers stay reassured without constant check-ins
Ensuring seniors feel confident
and caregivers stay reassured without constant check-ins
As is
To be
Troublesome
Health Record KeepingReal-time Health Tracking
As is
To be
Instant Access
to Medical Records
Lack of Accurate
Medical Record Sharing
As is
To be
Caregiver-Assisted
Transportation Booking
Difficulty with
Transportation Booking
As is
To be
As is
To be
As is
To be
Troublesome
Health Record Keeping
Real-time Health Tracking
Instant Access
to Medical Records
Caregiver-Assisted
Transportation Booking
Lack of Accurate
Medical Record Sharing
Difficulty with
Transportation Booking
Mapping the ecosystem
Designing a system for seamless senior care
Designing a system for seamless senior care
I created this concept model to design the entire system for streamlined senior health management. By mapping key interactions and integrating voice and AI-driven technologies, I demonstrated how seniors and caregivers can seamlessly manage daily routines, monitor health data, and coordinate medical and transportation needs
I created this concept model to design the entire system for streamlined senior health management. By mapping key interactions and integrating voice and AI-driven technologies, I demonstrated how seniors and caregivers can seamlessly manage daily routines, monitor health data, and coordinate medical and transportation needs
Outlining user paths
A Day with NURI, Hospital Visit
A Day with NURI, Hospital Visit
In the morning, the senior checks their health tasks and logs their mood through the smart mirror.
In the morning, the senior checks their health tasks and logs their mood through the smart mirror.
The app reminds them of a hospital appointment and books a taxi using GPS tracking.
The app reminds them of a hospital appointment and books a taxi using GPS tracking.
When the senior arrives, the system detects their location and sends a real-time notification.
When the senior arrives, the system detects their location and sends a real-time notification.
During the visit, the conversation is recorded and summarized by AI, then shared with the caregiver.
During the visit, the conversation is recorded and summarized by AI, then shared with the caregiver.
Heuristic evaluation
Catching design problems early
Catching design problems early
I conducted a heuristic evaluation to assess the usability and accessibility of the service, with a special focus on senior users. The goal was to identify design issues early and propose improvements that reduce cognitive load and enhance clarity.
I conducted a heuristic evaluation to assess the usability and accessibility of the service, with a special focus on senior users. The goal was to identify design issues early and propose improvements that reduce cognitive load and enhance clarity.
Participants: 11 designers
Participants: 11 designers
Method: Shared heuristic evaluation questions → collected scores & feedback
Method: Shared heuristic evaluation questions → collected scores & feedback
Low-Scoring Category :
Clear Error Messaging & Problem-Solving Support
Low-Scoring Category :
Clear Error Messaging & Problem-Solving Support
Interation
Testing, refining, and shaping a system
seniors can actually use
Testing, refining, and shaping a system
seniors can actually use
Each round of heuristic evaluation and usability testing revealed friction points. I addressed these issues through iterative refinements to the prototypes and design system, which gradually improved clarity, accessibility, and cross-device consistency.
Each round of heuristic evaluation and usability testing revealed friction points. I addressed these issues through iterative refinements to the prototypes and design system, which gradually improved clarity, accessibility, and cross-device consistency.
⛔️
“There is no cancel button on the Smart Mirror while input.”
✅
Added cancel option while input
⛔️
“On the Watch, there’s no way to change the time
once it’s set.”
✅
Enabled time adjustment after a call request
Final Outcomes
🪞 Morning Check-up
🪞 Morning Check-up
Seniors start their day with the Smart Mirror, which guides them through an easy check-in routine.
Face recognition captures their daily condition record.
Voice prompts allow seniors to log mood and symptoms without needing complex navigation.
The data is automatically synced, keeping caregivers updated in real time.
Seniors start their day with the Smart Mirror, which guides them through an easy check-in routine.
Face recognition captures their daily condition record.
Voice prompts allow seniors to log mood and symptoms without needing complex navigation.
The data is automatically synced, keeping caregivers updated in real time.
🚖 Taxi Booking
🚖 Taxi Booking
Seniors prepare for hospital visits with seamless mobility support.
Caregivers book a taxi through the Mobile App.
The Smart Mirror displays booking details, ensuring seniors are aware of their upcoming ride.
The Smart Watch sends real-time arrival alerts, keeping seniors on schedule with minimal effort.
Caregivers track the ride remotely, ensuring safety and peace of mind.
Seniors prepare for hospital visits with seamless mobility support.
Caregivers book a taxi through the Mobile App.
The Smart Mirror displays booking details, ensuring seniors are aware of their upcoming ride.
The Smart Watch sends real-time arrival alerts, keeping seniors on schedule with minimal effort.
Caregivers track the ride remotely, ensuring safety and peace of mind.
🏥 Hospital Visit
🏥 Hospital Visit
Seniors attend hospital appointments with built-in recording support.
When the consultation begins, the Smart Watch notifies the senior that recording has started.
The doctor uses the Web Plugin to securely record and transmit the consultation.
Caregivers instantly receive both the full recording and an AI-generated summary through the Mobile App.
The summary can be directly applied to routine and medication management, ensuring continuity of care.
Seniors attend hospital appointments with built-in recording support.
When the consultation begins, the Smart Watch notifies the senior that recording has started.
The doctor uses the Web Plugin to securely record and transmit the consultation.
Caregivers instantly receive both the full recording and an AI-generated summary through the Mobile App.
The summary can be directly applied to routine and medication management, ensuring continuity of care.
Reflection
I learned that addressing usability issues specific to seniors is crucial when designing a service for this user group. Ensuring the design is accessible, intuitive, and easy to interact with was essential. Additionally, this project taught me the importance of device connectivity, especially when integrating multiple platforms like smart mirrors, mobile phones, watches, and web interfaces. Working with the smart mirror, a relatively new device, was a fun challenge and provided a unique opportunity to explore voice-driven interactions.
I learned that addressing usability issues specific to seniors is crucial when designing a service for this user group. Ensuring the design is accessible, intuitive, and easy to interact with was essential. Additionally, this project taught me the importance of device connectivity, especially when integrating multiple platforms like smart mirrors, mobile phones, watches, and web interfaces. Working with the smart mirror, a relatively new device, was a fun challenge and provided a unique opportunity to explore voice-driven interactions.