--- title: Using AudioContext suspend/resume as a Streaming A/V Sync Gate description: Freeze and release the AudioContext timeline as streamed frames arrive, so frame availability drives playback while audio and video remain synchronized. image: "https://zengliangyi.com/og/blog/en/audiocontext-stream-sync-gate.png" --- **TL;DR:** Buffering N frames before playback merely guesses the network speed. `AudioContext.suspend()` stops audio output and suspends the progression of time in the audio context, so it can act as a synchronization gate: suspend the timeline when the next frame is missing and resume it when that frame arrives. > This article assumes familiarity with the Web Audio API, `requestAnimationFrame`, and frame-by-frame streaming. ## The problem: frames and audio run on separate tracks Consider a streamed lip-sync player. A server sends video frames over SSE or WebSocket while the client receives, decodes, and draws them alongside the corresponding audio. The direct implementation starts audio immediately and uses `audioContext.currentTime` to choose the frame: ```typescript audioSource. start ( 0 ); const startTime = audioContext.currentTime; function renderLoop () { const elapsed = audioContext.currentTime - startTime; const targetFrame = Math. floor (elapsed * fps); if (frames[targetFrame]) { drawFrame (frames[targetFrame]); } requestAnimationFrame (renderLoop); } ``` It works on a healthy connection. When frames arrive more slowly than audio advances: ```text Audio clock: 0s ───────── 2s ───────── 4s ───────── Frames ready: ████████████░░░░░░░░░░░░░░░░░░░░ What users see: [normal] [frozen mouth, audio continues] ``` The A/V drift is immediately visible. ## Why a fixed buffer threshold is not enough A common fix waits for a fixed number of frames: ```typescript const BUFFER_THRESHOLD = 60 ; onFrameDecoded (() => { decodedCount += 1 ; if ( ! started && decodedCount >= BUFFER_THRESHOLD ) { started = true ; audioSource. start ( 0 ); renderLoop (); } }); ``` This postpones the problem: - The threshold depends on network and server speed. Sixty frames may be too few or may only add startup latency. - Any stall after playback begins can still put frames behind audio. - Buffering for the worst case makes healthy connections wait too. The underlying issue is an audio clock that advances without knowing whether the next frame exists. ## The key property: suspend stops time progression MDN defines `AudioContext.suspend()` as suspending the progression of time in the audio context while temporarily halting audio hardware access. It is more than mute: ```typescript const audioContext = new AudioContext (); // After playing for a while: console. log (audioContext.currentTime); // for example, 2.341 await audioContext. suspend (); await sleep ( 3000 ); console. log (audioContext.currentTime); // remains near the suspended position await audioContext. resume (); // The timeline continues from the suspended position. ``` That makes AudioContext useful as a synchronization primitive: ```text Is the next sequential frame ready? yes → resume() → time advances → draw when its timestamp arrives no → suspend() → time freezes → wait for the frame-arrival event ``` ## Initialization: suspend before scheduling audio Audio decoding and frame delivery can run in parallel. Suspend the new context first, then schedule the buffer while it is suspended: ```typescript const AudioContextConstructor = window.AudioContext ?? ( window as typeof window & { webkitAudioContext : typeof AudioContext; } ).webkitAudioContext; const audioContext = new AudioContextConstructor (); await audioContext. suspend (); const buffer = await audioContext. decodeAudioData (audioBytes); const source = audioContext. createBufferSource (); source.buffer = buffer; source. connect (audioContext.destination); source. start ( 0 ); const startTime = audioContext.currentTime; ``` `start(0)` can schedule the source while the context is suspended, but the timeline does not advance. The scheduler calls `resume()` after both audio and the first frame are ready. Browser autoplay policy still applies. The first `resume()` that permits audible output generally needs to follow a user gesture. Production code should not swallow its rejection; show a “play” affordance or propagate the error to the caller. ## Gate functions: track the desired state locally Both `resume()` and `suspend()` are asynchronous. Reading only `audioContext.state` introduces a transition window: a command has been issued, but the property has not updated yet. A synchronously updated flag can represent the desired state: ```typescript let audioContext : AudioContext | null = null ; let audioRunning = false ; function ensureRunning () { if ( ! audioRunning && audioContext) { audioRunning = true ; audioContext. resume (). catch (( error ) => { audioRunning = false ; reportAudioError (error); }); } } function ensureSuspended () { if (audioRunning && audioContext) { audioRunning = false ; audioContext. suspend (). catch (reportAudioError); } } ``` The flag prevents duplicate commands on every animation frame. It does not replace error handling: when autoplay policy rejects `resume()`, reset the flag and update the UI. ## The scheduler: let frames drive the clock Two events can wake the scheduler: - The next sequential frame finishes decoding. - Audio decoding finishes. ```typescript let audioReady = false ; let lastDrawnFrame = - 1 ; function tryAdvance () { if ( ! isPlaying) return ; const nextNeeded = lastDrawnFrame + 1 ; if (streamEnded && nextNeeded >= totalFrames) { ensureSuspended (); isPlaying = false ; onPlayEnd (); return ; } const nextFrame = frames[nextNeeded]; if (nextFrame === null ) { // A frame gap: freeze time and wait for onFrameArrived. ensureSuspended (); return ; } if ( ! audioReady || ! audioContext) { requestAnimationFrame (tryAdvance); return ; } ensureRunning (); const elapsed = audioContext.currentTime - startTime; const frameTime = nextNeeded / fps; if (elapsed >= frameTime) { draw (nextFrame); lastDrawnFrame = nextNeeded; } // Wait if the timestamp is early; otherwise continue with the next frame. requestAnimationFrame (tryAdvance); } ``` The same logic handles both directions: - Server slower than real time: a frame gap suspends audio. - Server faster than real time: the frame exists, but `elapsed < frameTime` prevents early drawing. ## Wake only for the frame that matters ```typescript function onFrameArrived ( index : number , bitmap : ImageBitmap ) { frames[index] = bitmap; if ( ! isPlaying && index === 0 ) { isPlaying = true ; tryAdvance (); return ; } if (isPlaying && index === lastDrawnFrame + 1 ) { tryAdvance (); } } ``` If frames arrive as `0, 1, 3, 2, 4`, frame 3 is stored without waking playback. Frame 2 then matches `lastDrawnFrame + 1` and wakes the scheduler, which proceeds through 2, 3, and 4. The index enforces order without a separate sort. ## Completion and cleanup Stopping playback should cancel scheduling, close the audio context, and release GPU resources owned by each `ImageBitmap`: ```typescript function stop () { isPlaying = false ; if (rafId !== null ) { cancelAnimationFrame (rafId); rafId = null ; } audioContext?. close (). catch (reportAudioError); audioContext = null ; for ( const bitmap of frames) { bitmap?. close (); } } ``` A zero-frame stream needs an explicit completion path. Because normal playback starts when frame zero arrives, `totalFrames === 0` never produces that event. Handle stream completion directly instead of waiting for `tryAdvance()` to wake. Audio duration, `fps`, and `totalFrames` must also describe the same timeline. If audio ends while video frames remain, `currentTime` still advances but users hear nothing. The gate cannot replace input-metadata validation. ## Trade-offs | Benefits | Limitations | | --- | --- | | Strict A/V sync under network jitter | A stall pauses both sound and picture | | No fixed N-frame startup buffer | Autoplay policy and async state need handling | | No AudioWorklet required | Frequent suspension can create perceptible pauses | | Out-of-order frames naturally wait by index | Unnecessary for local or fully buffered media | For local files or stable HLS playback, driving video from the audio clock is simpler. This gate is valuable when frame arrival is inherently unpredictable. ## Source and further reading - [Original TIL issue](https://github.com/ZengLiangYi/til/issues/3) - [Complete TypeScript implementation](https://github.com/ZengLiangYi/til/blob/main/typescript/audiocontext-sync-gate.ts) - [MDN: AudioContext.suspend()](https://developer.mozilla.org/docs/Web/API/AudioContext/suspend) - [MDN: BaseAudioContext.currentTime](https://developer.mozilla.org/docs/Web/API/BaseAudioContext/currentTime) Share: ## Related Posts [## Five AI conversation parsers: retained fields and loss boundaries Build an acceptance matrix from Claude Code, Codex, Cursor, Trae, and Copilot parsers, checking tool flags, thinking, order, and snapshot coverage. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #TypeScript #Node.js](/en/blog/ai-conversation-parser-boundaries) [## AI SDK Provider Abstraction: Capabilities and Protocols Test 7 ChatCrystal providers and AI SDK factories with offline mocks for Responses, Chat Completions, embeddings, and structured output rejection. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #Node.js #TypeScript](/en/blog/ai-sdk-provider-abstraction) [## SourceAdapter internals: identity, content hashes, and reimport Trace ChatCrystal’s registry, importer, and remote payload to verify identity, canonical hashes, metadata shortcuts, and parser-version boundaries. 曾亮懿 October 7, 2026 4 min read #ChatCrystal #TypeScript #Node.js](/en/blog/source-adapter-internals) ## From writing to code Explore ChatCrystal, developer tools and daily learning notes on my GitHub. Questions about the implementation are welcome. [Visit my GitHub](https://github.com/ZengLiangYi) ```json { "@context": "https://schema.org", "@type": "WebSite", "name": "ZengLiangYi Blog", "alternateName": [ "Zeng Liangyi", "ZengLiangYi", "zengliangyi.com" ], "url": "https://zengliangyi.com", "description": "曾亮懿的个人网站。记录 AI Native 探索、开发者工具与全栈产品实践。" } { "@context": "https://schema.org", "@type": "BlogPosting", "headline": "Using AudioContext suspend/resume as a Streaming A/V Sync Gate", "description": "Freeze and release the AudioContext timeline as streamed frames arrive, so frame availability drives playback while audio and video remain synchronized.", "url": "https://zengliangyi.com/en/blog/audiocontext-stream-sync-gate/", "image": "https://zengliangyi.com/images/blog/audiocontext-stream-sync-gate.svg", "datePublished": "2026-03-16T00:00:00.000Z", "dateModified": "2026-03-16T00:00:00.000Z", "author": { "@type": "Person", "name": "曾亮懿", "url": "https://zengliangyi.com/en/about" }, "publisher": { "@type": "Organization", "name": "ZengLiangYi Blog", "logo": { "@type": "ImageObject", "url": "https://zengliangyi.com/favicon-96x96.png" } }, "mainEntityOfPage": { "@type": "WebPage", "@id": "https://zengliangyi.com/en/blog/audiocontext-stream-sync-gate/" } } { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://zengliangyi.com/en" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://zengliangyi.com/en/blog" }, { "@type": "ListItem", "position": 3, "name": "Using AudioContext suspend/resume as a Streaming A/V Sync Gate", "item": "https://zengliangyi.com/en/blog/audiocontext-stream-sync-gate/" } ] } ```