Bridge media, not model logic
AetherRTC is the browser-audio gateway for Voice Agent Runtime. It handles SDP and ICE signaling, terminates WebRTC media, and passes audio both ways over a bidirectional gRPC stream. Inference happens downstream. The gateway owns the codec and the timing.
Convert at the edge
The Pion media engine only registers PCMU/G.711 at 8 kHz, which pushes the browser off Opus so the audio can be decoded in pure Go. Incoming mu-law audio is decoded into PCM chunks. The bridge sends the session ID, start controls, and the source sample rate along with them, so the services downstream know how to read the bytes.
Reply audio takes the reverse path. It is encoded and sent in 20 ms frames, with at most five catch-up frames per tick. The gateway treats the agent as still speaking for 400 ms after audio stops, so a short pause between two synthesized sentences is not mistaken for the end of the reply.
A fix, measured
PR #4 reworked outbound pacing. Before it, the speaking flag dropped after about 300 ms without audio. In a test that replays two sentences with a gap between them, a 350 ms gap reopened the mic mid-reply in 5 of 5 trials. After the fix, a 350 ms gap no longer does. A 450 ms gap still does, so this improved the problem but did not remove it.
The same PR dials the inference service through gRPC's passthrough resolver. On my machine, stream setup went from 56 ms to 20 ms (median of 20). An older note of mine said 11.1 s to 5 ms. That stall came from DNS behaviour on one network and does not reproduce now, so I don't use that number.
Connection state is part of the gateway
Signaling replaces duplicate sessions, cleans up closed connections, and serializes WebSocket writes. These are transport jobs, kept out of the model pipeline.
Current boundaries
The gateway drops caller audio while the agent is speaking. Audio can travel both ways, but the caller cannot interrupt the agent: barge-in does not work.
The setup is for local use: plain HTTP/WebSocket signaling, insecure gRPC credentials, signaling that accepts any origin, and STUN with no TURN relay. A public deployment needs TLS, origin checks, and a relay. Browser sessions work in the demo, but the browser path is not part of the latency benchmark, and a stall after about two exchanges that shows up sometimes in browser testing is still unexplained.