ByteDance Seed Introduces SeedRealtime: a Native Audio-Visual Full-Duplex LLM That Watches, Listens and Speaks in One Model
ByteDances Seed team has introduced SeedRealtime, a native audio-visual full-duplex LLM. The model fuses audio, video and text in a single unified architecture. It interacts in real time over continuous multimodal streams, rather than one turn at a time. Seed positions it as a step toward omni-modal interaction, and claims three breakthroughs: joint audio-visual understanding, proactive interaction, and natural conversational timing. The architectural target is the cascade: chained ASR, VLM and TTS modules that add latency and lose information between stages. SeedRealtime instead runs perception, understanding, decision-making and expression in parallel inside one end-to-end model. Turn-taking moves inside the model as well, replacing the external voice-activity detector most real-time...
ByteDances Seed team has introduced SeedRealtime, a native audio-visual full-duplex LLM. The model fuses audio, video and text in a single unified architecture. It interacts in real time over continuous multimodal streams, rather than one turn at a time. Seed positions it as a step toward omni-modal interaction, and claims three breakthroughs: joint audio-visual understanding, proactive interaction, and natural conversational timing. The architectural target is the cascade: chained ASR, VLM and TTS modules that add latency and lose information between stages. SeedRealtime instead runs perception, understanding, decision-making and expression in parallel inside one end-to-end model. Turn-taking moves inside the model as well, replacing the external voice-activity detector most real-time stacks still depend on. Is it deployable? It is partly deployable. SeedRealtime is live inside the Doubao app, ByteDances consumer assistant. For this specific model, ByteDance has published no technical report, no parameter count, no open weights, and no Volcano Engine or BytePlus endpoint. As a third-party team, you cannot integrate it as of now. What is deployable right now is the idea: a validated reference architecture, and a moved goalpost for anyone shipping real-time voice-plus-camera products. Interactive explainer What is actually new in the demos Seed published seven scenarios. Four are load-bearing. Identity binding across modalities: At a noisy group dinner, the model matches names to faces as people are introduced, then keeps each voice tied to its identity — attributing conflicting travel preferences to the right speaker before proposing a plan. Proactive speech from a held instruction: At the Hebei Museum, a user asks to be reminded when a specific bronze screen stand appears. The camera keeps panning; the model watches and speaks up unprompted when the piece enters frame. The same behavior shows up on a ResNet paper — the model tracks fast page flips, spots the 3.4 Im