Chief of Staff for the Masses: How Meta’s Joshua To Designs Wearables with AI

Beyond The Prompt - How to use AI in your company

In this episode, Joshua To, VP of Product Design at Meta, shares how AI is reshaping how—and where—we interact with technology. He walks us through Meta’s evolving approach to AR and wearables, why notifications are still the killer use case, and how AI is becoming the “brain behind empathy.”

We dig into what it means to build interfaces that understand you, why audio might be the future’s most underrated platform, and how designing for emotion changes everything—from form factor to function. Joshua also reflects on his path from launching a clothing brand to leading design at Google and Meta, and what those worlds taught him about craft, context, and human-centered systems.

This one’s for anyone designing AI into the real world—where every interface choice carries weight, and intelligence starts with listening.

Key takeaways:

• Empathy Is the Real Intelligence — Joshua flips the definition of smart tech. It’s not just about outputs—it’s about understanding you. Context, tone, emotion—that’s what great AI will sense and respond to.
• Design for the Moment, Not the Feed — AR’s killer use case isn’t games—it’s restraint. Joshua shares why the best AI product might just be the one that knows not to ping you. Context-aware computing is the real unlock.
• Audio Is the Interface to Watch — Forget screens. The most powerful interface might be your ears. From wearables to ambient signals, Joshua explains why audio design is the next big frontier for human-centered AI.
• AR Isn’t a Feature—It’s a System of Consideration — Joshua reframes augmented reality as quiet, ambient infrastructure. The real power of AR isn’t spectacle—it’s subtlety. It helps you move through the world with less friction, not more.

LinkedIn: Joshua To | LinkedIn
Website: Home - Joshua To
Meta: Meta Careers

00:00 Intro: Fixing Notifications With AI
00:54 Meet Josh: VP of Product Design at Meta
02:06 From Hoodies to Hardware: Josh's Journey
03:53 The Google Experience: From Ads to Product Management
10:37 The Evolution of Google Glass and AR
19:12 The Role of AI in Josh's Career
22:25 Designing the Future: AR, VR, and Attention Management
32:49 Contextually Aware Suggestions
33:38 Leveraging Generative AI in Design
34:52 AI's Role in Concept Art and Storyboarding
41:24 AI Tools and Model Capabilities
45:54 The Future of AI and Wearables
51:58 Reflections and Takeaways

For more prompts, tips, and AI tools. Check out our website: https://www.beyondtheprompt.ai/ or follow Jeremy or Henrik on Linkedin:

Henrik: https://www.linkedin.com/in/werdelin
Jeremy: https://www.linkedin.com/in/jeremyutley

Show edited by Emma Cecilie Jensen.