The whole dayThu 5 Nov 2026
AI Compass
ArchiveAbout

Meta rolls out Muse Realtime Voice and expressive Avatars across mobile, web and Mac

Driven by a two-step distilled causal Diffusion Transformer streaming 448x768 video at 25 fps with 870 ms latency, Muse avatars beat Runway and HeyGen in live human trials.

Video: MetaMuse Realtime Voice and expressive Realtime Avatar running inside the Muse mobile app.

Meta began rolling out Muse Realtime Voice and Muse Realtime Avatar on 5 November 2026 inside the Muse app across iOS, Android, web and macOS in the United States, following its September Connect 2026 preview. Users tap the phone icon in the top-right corner of the app to hold interruptible spoken conversations with their personal agent across hundreds of selectable voice styles, switching between audio-only mode, a compact floating avatar or full-screen face-to-face mode. Meta confirmed that voice conversations are also coming to AI Glasses and the Muse Charm wearable.

In a companion technical post, Meta detailed how Muse Realtime Voice emits a single stream of vector-quantised (VQ) speech tokens encoding both lexical content and vocal delivery. An audio decoder converts those VQ tokens into speech while Muse Realtime Avatar, an audio-driven, time-causal Diffusion Transformer (DiT), consumes the same stream alongside a reference image and a rolling window of recent video latents to animate photorealistic faces, stylised characters, animals or full-body figures.

System diagram showing User Speech entering Muse Realtime Voice to produce Streaming Speech VQs that drive both an Audio Decoder and a two-step Time-Causal DiT Video Decoder.
Shared speech-token streaming pipeline coupling Muse Realtime Voice with the two-step Time-Causal DiT avatar generator.Image: Meta

Two-step distillation and live human evaluations

Using self-forcing and distribution-matching distillation with four-bit quantisation-aware training and persistent KV caches, Meta compressed its 40-step, three-way classifier-free-guidance teacher (120 neural function evaluations per chunk) into an unguided two-step causal student, a 60 times reduction that streams 448x768 portrait video at 25 frames per second with roughly 870 ms turn-to-response latency. In blind two- to three-minute live conversations, human raters preferred Muse Realtime Avatar 78% to 22% overall against Runway Characters and 88% to 12% against HeyGen LiveAvatar.

Bar chart showing human raters preferred Muse Realtime Avatar 78% to 22% over Runway Characters and 88% to 12% over HeyGen LiveAvatar across live conversations.
Blind live-call human preference results comparing Muse Realtime Avatar against Runway Characters and HeyGen LiveAvatar.Image: Meta

The rest of the day

Read the whole day

Spotted an error? Email aiforsuccess.in+compass@gmail.com or tag @ai_for_success on X